2024-09-14T08:51:53.223 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-14T08:51:53.241 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-14T08:51:53.320 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900516 branch: main description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} email: ceph-qa@ceph.com first_in_suite: false job_id: '7900516' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: main ceph: conf: client: rbd cache: true rbd cache policy: writearound global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zlib bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 0d61375b8ab53be4c233f5b93e463edfdff9316a owner: scheduled_teuthology@teuthology priority: 950 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 5839 sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 sleep_before_teardown: 0 subset: 99/128 suite: rbd suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 0d61375b8ab53be4c233f5b93e463edfdff9316a targets: smithi176.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG/FuUsyZj86EDcDVtrA1orDqBDaZm2QCBWRkVHcNEvORvCxfxG0ycmhBn2Fmh2TlyHPnVX4JDAJ05edUSNNPYc= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGRfq3k86/F4iBQ1BMHZ6PIWHgqGKHcp4eiaUgIYQs2l5tRKOQTUseIftJplwxSNK1vtmsRIos9DyrgwiZo/bn0= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_20:16:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-14T08:51:53.321 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa; will attempt to use it 2024-09-14T08:51:53.321 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa/tasks 2024-09-14T08:51:53.322 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-14T08:51:53.323 INFO:teuthology.task.internal:Checking packages... 2024-09-14T08:51:53.344 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '7f24e589328ae9e639c0009d93340e5b235d0a47' 2024-09-14T08:51:53.345 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-14T08:51:53.345 INFO:teuthology.packaging:ref: None 2024-09-14T08:51:53.345 INFO:teuthology.packaging:tag: None 2024-09-14T08:51:53.345 INFO:teuthology.packaging:branch: main 2024-09-14T08:51:53.345 INFO:teuthology.packaging:sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T08:51:53.345 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=main 2024-09-14T08:51:53.562 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4907-ga2782315-1jammy 2024-09-14T08:51:53.563 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-14T08:51:53.570 INFO:teuthology.task.internal:no buildpackages task found 2024-09-14T08:51:53.570 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-14T08:51:53.609 INFO:teuthology.task.internal:Saving configuration 2024-09-14T08:51:53.623 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-14T08:51:53.642 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-14T08:51:53.667 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi176.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900516', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 08:46:17.537927', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOScgBADhBrYXaK2OQKOKABUnbDfydiUCV0W00f3Bkde'} 2024-09-14T08:51:53.692 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900516', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 08:46:17.536188', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHHwAqWNxdlEVq1D1/SN7NxAaQVtG9ASzi3KYyEfzQS5'} 2024-09-14T08:51:53.692 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-14T08:51:53.699 INFO:teuthology.task.internal:roles: ubuntu@smithi176.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-14T08:51:53.699 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-14T08:51:53.699 INFO:teuthology.run_tasks:Running task console_log... 2024-09-14T08:51:53.790 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc4d9e97e20>, signals=[15]) 2024-09-14T08:51:53.790 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-14T08:51:53.797 INFO:teuthology.task.internal:Opening connections... 2024-09-14T08:51:53.797 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T08:51:53.798 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T08:51:53.881 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-09-14T08:51:53.881 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T08:51:53.963 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-14T08:51:53.969 DEBUG:teuthology.orchestra.run.smithi176:> uname -m 2024-09-14T08:51:53.974 INFO:teuthology.orchestra.run.smithi176.stdout:x86_64 2024-09-14T08:51:53.974 DEBUG:teuthology.orchestra.run.smithi176:> cat /etc/os-release 2024-09-14T08:51:54.022 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-14T08:51:54.022 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="Ubuntu" 2024-09-14T08:51:54.022 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="22.04" 2024-09-14T08:51:54.022 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-14T08:51:54.022 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_CODENAME=jammy 2024-09-14T08:51:54.022 INFO:teuthology.orchestra.run.smithi176.stdout:ID=ubuntu 2024-09-14T08:51:54.022 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE=debian 2024-09-14T08:51:54.022 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-14T08:51:54.023 INFO:teuthology.orchestra.run.smithi176.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-14T08:51:54.023 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-14T08:51:54.023 INFO:teuthology.orchestra.run.smithi176.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-14T08:51:54.023 INFO:teuthology.orchestra.run.smithi176.stdout:UBUNTU_CODENAME=jammy 2024-09-14T08:51:54.023 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2024-09-14T08:51:54.056 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-09-14T08:51:54.061 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-09-14T08:51:54.062 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-09-14T08:51:54.107 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-14T08:51:54.107 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="Ubuntu" 2024-09-14T08:51:54.107 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="22.04" 2024-09-14T08:51:54.107 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-14T08:51:54.107 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_CODENAME=jammy 2024-09-14T08:51:54.107 INFO:teuthology.orchestra.run.smithi183.stdout:ID=ubuntu 2024-09-14T08:51:54.107 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE=debian 2024-09-14T08:51:54.107 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-14T08:51:54.107 INFO:teuthology.orchestra.run.smithi183.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-14T08:51:54.108 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-14T08:51:54.108 INFO:teuthology.orchestra.run.smithi183.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-14T08:51:54.108 INFO:teuthology.orchestra.run.smithi183.stdout:UBUNTU_CODENAME=jammy 2024-09-14T08:51:54.108 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-09-14T08:51:54.140 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-14T08:51:54.148 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-14T08:51:54.184 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-14T08:51:54.184 DEBUG:teuthology.orchestra.run.smithi176:> test '!' -e /home/ubuntu/cephtest 2024-09-14T08:51:54.187 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-09-14T08:51:54.191 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-14T08:51:54.217 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-14T08:51:54.217 DEBUG:teuthology.orchestra.run.smithi176:> test -z $(ls -A /var/lib/ceph) 2024-09-14T08:51:54.231 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-09-14T08:51:54.251 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-14T08:51:54.304 INFO:teuthology.run_tasks:Running task kernel... 2024-09-14T08:51:54.318 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-14T08:51:54.319 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-14T08:51:54.319 DEBUG:teuthology.orchestra.run.smithi176:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T08:51:54.320 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T08:51:54.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T08:51:54.325 DEBUG:teuthology.orchestra.run.smithi176:> uname -r 2024-09-14T08:51:54.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T08:51:54.326 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2024-09-14T08:51:54.378 INFO:teuthology.orchestra.run.smithi176.stdout:5.15.0-56-generic 2024-09-14T08:51:54.379 INFO:teuthology.task.kernel:Running kernel on smithi176: 5.15.0-56-generic 2024-09-14T08:51:54.379 DEBUG:teuthology.orchestra.run.smithi176:> sudo apt-get clean 2024-09-14T08:51:54.380 INFO:teuthology.orchestra.run.smithi183.stdout:5.15.0-56-generic 2024-09-14T08:51:54.380 INFO:teuthology.task.kernel:Running kernel on smithi183: 5.15.0-56-generic 2024-09-14T08:51:54.381 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get clean 2024-09-14T08:51:54.516 DEBUG:teuthology.orchestra.run.smithi176:> sudo apt-get update 2024-09-14T08:51:54.524 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get update 2024-09-14T08:51:54.644 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T08:51:54.660 INFO:teuthology.orchestra.run.smithi176.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-14T08:51:54.662 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T08:51:54.662 INFO:teuthology.orchestra.run.smithi183.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-14T08:51:54.664 INFO:teuthology.orchestra.run.smithi183.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-14T08:51:54.728 INFO:teuthology.orchestra.run.smithi176.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-14T08:51:54.742 INFO:teuthology.orchestra.run.smithi183.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-14T08:51:54.762 INFO:teuthology.orchestra.run.smithi176.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-14T08:51:54.858 INFO:teuthology.orchestra.run.smithi176.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-14T08:51:54.902 INFO:teuthology.orchestra.run.smithi183.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-14T08:51:54.955 INFO:teuthology.orchestra.run.smithi176.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-14T08:51:54.960 INFO:teuthology.orchestra.run.smithi176.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-14T08:51:54.963 INFO:teuthology.orchestra.run.smithi176.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-14T08:51:54.963 INFO:teuthology.orchestra.run.smithi176.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-14T08:51:54.964 INFO:teuthology.orchestra.run.smithi176.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-14T08:51:54.974 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-14T08:51:54.985 INFO:teuthology.orchestra.run.smithi176.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-14T08:51:54.989 INFO:teuthology.orchestra.run.smithi176.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-14T08:51:54.989 INFO:teuthology.orchestra.run.smithi176.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-14T08:51:54.999 INFO:teuthology.orchestra.run.smithi176.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-14T08:51:55.004 INFO:teuthology.orchestra.run.smithi183.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-14T08:51:55.005 INFO:teuthology.orchestra.run.smithi176.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-14T08:51:55.007 INFO:teuthology.orchestra.run.smithi176.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-14T08:51:55.008 INFO:teuthology.orchestra.run.smithi176.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-14T08:51:55.008 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-14T08:51:55.010 INFO:teuthology.orchestra.run.smithi183.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-14T08:51:55.010 INFO:teuthology.orchestra.run.smithi183.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-14T08:51:55.023 INFO:teuthology.orchestra.run.smithi176.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-14T08:51:55.023 INFO:teuthology.orchestra.run.smithi176.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-14T08:51:55.024 INFO:teuthology.orchestra.run.smithi176.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-14T08:51:55.024 INFO:teuthology.orchestra.run.smithi176.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-14T08:51:55.025 INFO:teuthology.orchestra.run.smithi176.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-14T08:51:55.030 INFO:teuthology.orchestra.run.smithi183.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-14T08:51:55.030 INFO:teuthology.orchestra.run.smithi183.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-14T08:51:55.031 INFO:teuthology.orchestra.run.smithi176.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-14T08:51:55.031 INFO:teuthology.orchestra.run.smithi176.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-14T08:51:55.031 INFO:teuthology.orchestra.run.smithi176.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-14T08:51:55.033 INFO:teuthology.orchestra.run.smithi183.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-14T08:51:55.033 INFO:teuthology.orchestra.run.smithi183.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-14T08:51:55.034 INFO:teuthology.orchestra.run.smithi183.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-14T08:51:55.037 INFO:teuthology.orchestra.run.smithi183.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-14T08:51:55.043 INFO:teuthology.orchestra.run.smithi183.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-14T08:51:55.044 INFO:teuthology.orchestra.run.smithi183.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-14T08:51:55.048 INFO:teuthology.orchestra.run.smithi176.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-14T08:51:55.048 INFO:teuthology.orchestra.run.smithi176.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-14T08:51:55.048 INFO:teuthology.orchestra.run.smithi176.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-14T08:51:55.071 INFO:teuthology.orchestra.run.smithi183.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-14T08:51:55.071 INFO:teuthology.orchestra.run.smithi183.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-14T08:51:55.071 INFO:teuthology.orchestra.run.smithi183.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-14T08:51:55.072 INFO:teuthology.orchestra.run.smithi183.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-14T08:51:55.076 INFO:teuthology.orchestra.run.smithi183.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-14T08:51:55.076 INFO:teuthology.orchestra.run.smithi183.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-14T08:51:55.076 INFO:teuthology.orchestra.run.smithi183.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-14T08:51:55.077 INFO:teuthology.orchestra.run.smithi183.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-14T08:51:55.093 INFO:teuthology.orchestra.run.smithi183.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-14T08:51:55.097 INFO:teuthology.orchestra.run.smithi183.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-14T08:51:55.097 INFO:teuthology.orchestra.run.smithi183.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-14T08:51:55.105 INFO:teuthology.orchestra.run.smithi183.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-14T08:51:55.110 INFO:teuthology.orchestra.run.smithi183.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-14T08:51:55.112 INFO:teuthology.orchestra.run.smithi183.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-14T08:51:55.112 INFO:teuthology.orchestra.run.smithi183.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-14T08:51:55.113 INFO:teuthology.orchestra.run.smithi183.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-14T08:51:55.113 INFO:teuthology.orchestra.run.smithi183.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-14T08:51:55.123 INFO:teuthology.orchestra.run.smithi183.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-14T08:51:55.124 INFO:teuthology.orchestra.run.smithi183.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-14T08:51:55.124 INFO:teuthology.orchestra.run.smithi183.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-14T08:51:55.125 INFO:teuthology.orchestra.run.smithi183.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-14T08:51:55.140 INFO:teuthology.orchestra.run.smithi183.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-14T08:51:55.141 INFO:teuthology.orchestra.run.smithi183.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-14T08:51:55.141 INFO:teuthology.orchestra.run.smithi183.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-14T08:51:55.141 INFO:teuthology.orchestra.run.smithi183.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-14T08:51:55.142 INFO:teuthology.orchestra.run.smithi183.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-14T08:51:55.195 INFO:teuthology.orchestra.run.smithi176.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-14T08:51:55.616 INFO:teuthology.orchestra.run.smithi176.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-14T08:51:55.632 INFO:teuthology.orchestra.run.smithi176.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-14T08:51:55.640 INFO:teuthology.orchestra.run.smithi176.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-14T08:51:55.641 INFO:teuthology.orchestra.run.smithi176.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-14T08:51:55.642 INFO:teuthology.orchestra.run.smithi176.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-14T08:51:55.739 INFO:teuthology.orchestra.run.smithi176.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-14T08:51:55.749 INFO:teuthology.orchestra.run.smithi176.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-14T08:51:55.750 INFO:teuthology.orchestra.run.smithi176.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-14T08:51:55.772 INFO:teuthology.orchestra.run.smithi176.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-14T08:51:55.795 INFO:teuthology.orchestra.run.smithi176.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-14T08:51:55.798 INFO:teuthology.orchestra.run.smithi176.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-14T08:51:55.800 INFO:teuthology.orchestra.run.smithi176.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-14T08:51:55.801 INFO:teuthology.orchestra.run.smithi176.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-14T08:51:55.802 INFO:teuthology.orchestra.run.smithi176.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-14T08:51:55.877 INFO:teuthology.orchestra.run.smithi176.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-14T08:52:01.174 INFO:teuthology.orchestra.run.smithi176.stdout:Fetched 16.0 MB in 2s (7,533 kB/s) 2024-09-14T08:52:01.247 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 16.0 MB in 2s (7,459 kB/s) 2024-09-14T08:52:02.543 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2024-09-14T08:52:02.561 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-14T08:52:02.571 INFO:teuthology.orchestra.run.smithi176.stdout:Reading package lists... 2024-09-14T08:52:02.589 DEBUG:teuthology.orchestra.run.smithi176:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-14T08:52:02.614 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2024-09-14T08:52:02.653 INFO:teuthology.orchestra.run.smithi176.stdout:Reading package lists... 2024-09-14T08:52:02.800 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2024-09-14T08:52:02.800 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2024-09-14T08:52:02.862 INFO:teuthology.orchestra.run.smithi176.stdout:Building dependency tree... 2024-09-14T08:52:02.863 INFO:teuthology.orchestra.run.smithi176.stdout:Reading state information... 2024-09-14T08:52:02.969 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T08:52:02.969 INFO:teuthology.orchestra.run.smithi183.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T08:52:02.969 INFO:teuthology.orchestra.run.smithi183.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T08:52:02.970 INFO:teuthology.orchestra.run.smithi183.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T08:52:02.970 INFO:teuthology.orchestra.run.smithi183.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-14T08:52:02.970 INFO:teuthology.orchestra.run.smithi183.stdout: python2.7-minimal 2024-09-14T08:52:02.970 INFO:teuthology.orchestra.run.smithi183.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T08:52:02.971 INFO:teuthology.orchestra.run.smithi183.stdout:The following additional packages will be installed: 2024-09-14T08:52:02.971 INFO:teuthology.orchestra.run.smithi183.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-14T08:52:02.971 INFO:teuthology.orchestra.run.smithi183.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-14T08:52:02.972 INFO:teuthology.orchestra.run.smithi183.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-14T08:52:02.973 INFO:teuthology.orchestra.run.smithi183.stdout:Suggested packages: 2024-09-14T08:52:02.973 INFO:teuthology.orchestra.run.smithi183.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-14T08:52:02.974 INFO:teuthology.orchestra.run.smithi183.stdout:Recommended packages: 2024-09-14T08:52:02.974 INFO:teuthology.orchestra.run.smithi183.stdout: thermald 2024-09-14T08:52:03.032 INFO:teuthology.orchestra.run.smithi183.stdout:The following NEW packages will be installed: 2024-09-14T08:52:03.032 INFO:teuthology.orchestra.run.smithi183.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-14T08:52:03.032 INFO:teuthology.orchestra.run.smithi183.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-14T08:52:03.033 INFO:teuthology.orchestra.run.smithi183.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-14T08:52:03.034 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages will be upgraded: 2024-09-14T08:52:03.035 INFO:teuthology.orchestra.run.smithi183.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-14T08:52:03.050 INFO:teuthology.orchestra.run.smithi176.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T08:52:03.050 INFO:teuthology.orchestra.run.smithi176.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T08:52:03.050 INFO:teuthology.orchestra.run.smithi176.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T08:52:03.051 INFO:teuthology.orchestra.run.smithi176.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T08:52:03.051 INFO:teuthology.orchestra.run.smithi176.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-14T08:52:03.051 INFO:teuthology.orchestra.run.smithi176.stdout: python2.7-minimal 2024-09-14T08:52:03.051 INFO:teuthology.orchestra.run.smithi176.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T08:52:03.053 INFO:teuthology.orchestra.run.smithi176.stdout:The following additional packages will be installed: 2024-09-14T08:52:03.053 INFO:teuthology.orchestra.run.smithi176.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-14T08:52:03.053 INFO:teuthology.orchestra.run.smithi176.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-14T08:52:03.053 INFO:teuthology.orchestra.run.smithi176.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-14T08:52:03.054 INFO:teuthology.orchestra.run.smithi176.stdout:Suggested packages: 2024-09-14T08:52:03.054 INFO:teuthology.orchestra.run.smithi176.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-14T08:52:03.055 INFO:teuthology.orchestra.run.smithi176.stdout:Recommended packages: 2024-09-14T08:52:03.055 INFO:teuthology.orchestra.run.smithi176.stdout: thermald 2024-09-14T08:52:03.108 INFO:teuthology.orchestra.run.smithi176.stdout:The following NEW packages will be installed: 2024-09-14T08:52:03.108 INFO:teuthology.orchestra.run.smithi176.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-14T08:52:03.108 INFO:teuthology.orchestra.run.smithi176.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-14T08:52:03.108 INFO:teuthology.orchestra.run.smithi176.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-14T08:52:03.110 INFO:teuthology.orchestra.run.smithi176.stdout:The following packages will be upgraded: 2024-09-14T08:52:03.110 INFO:teuthology.orchestra.run.smithi176.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-14T08:52:03.164 INFO:teuthology.orchestra.run.smithi176.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-14T08:52:03.164 INFO:teuthology.orchestra.run.smithi176.stdout:Need to get 113 MB of archives. 2024-09-14T08:52:03.164 INFO:teuthology.orchestra.run.smithi176.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-14T08:52:03.164 INFO:teuthology.orchestra.run.smithi176.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-14T08:52:03.215 INFO:teuthology.orchestra.run.smithi183.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-14T08:52:03.215 INFO:teuthology.orchestra.run.smithi183.stdout:Need to get 113 MB of archives. 2024-09-14T08:52:03.215 INFO:teuthology.orchestra.run.smithi183.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-14T08:52:03.215 INFO:teuthology.orchestra.run.smithi183.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-14T08:52:03.567 INFO:teuthology.orchestra.run.smithi176.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-14T08:52:03.657 INFO:teuthology.orchestra.run.smithi176.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-14T08:52:04.461 INFO:teuthology.orchestra.run.smithi176.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-14T08:52:04.461 INFO:teuthology.orchestra.run.smithi176.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-14T08:52:04.461 INFO:teuthology.orchestra.run.smithi176.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-14T08:52:04.589 INFO:teuthology.orchestra.run.smithi176.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-14T08:52:04.609 INFO:teuthology.orchestra.run.smithi176.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-14T08:52:04.711 INFO:teuthology.orchestra.run.smithi183.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-14T08:52:05.004 INFO:teuthology.orchestra.run.smithi176.stdout:Fetched 113 MB in 1s (75.9 MB/s) 2024-09-14T08:52:05.062 INFO:teuthology.orchestra.run.smithi183.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-14T08:52:05.171 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-14T08:52:07.117 INFO:teuthology.orchestra.run.smithi176.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-14T08:52:07.125 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T08:52:07.291 INFO:teuthology.orchestra.run.smithi183.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-14T08:52:07.292 INFO:teuthology.orchestra.run.smithi183.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-14T08:52:07.292 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-14T08:52:07.343 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T08:52:07.642 INFO:teuthology.orchestra.run.smithi183.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-14T08:52:07.723 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-14T08:52:08.120 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 113 MB in 5s (24.2 MB/s) 2024-09-14T08:52:08.306 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-14T08:52:09.335 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-14T08:52:09.356 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T08:52:09.439 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T08:52:09.896 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-14T08:52:09.916 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T08:52:09.954 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T08:52:10.456 INFO:teuthology.orchestra.run.smithi183.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-14T08:52:10.461 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T08:52:10.752 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T08:52:12.911 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-14T08:52:12.933 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T08:52:13.016 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T08:52:13.490 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-14T08:52:13.510 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T08:52:13.548 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T08:52:15.942 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T08:52:16.044 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T08:52:16.393 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T08:52:16.515 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T08:52:16.792 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-14T08:52:16.803 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-14T08:52:16.842 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-14T08:52:19.805 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T08:52:19.932 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T08:52:20.566 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T08:52:20.744 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T08:52:21.030 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-14T08:52:21.053 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-14T08:52:21.097 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-14T08:52:22.019 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-14T08:52:22.052 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T08:52:22.094 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T08:52:25.120 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T08:52:25.245 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T08:52:25.569 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-14T08:52:25.691 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T08:52:25.817 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-14T08:52:25.945 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T08:52:26.629 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-14T08:52:26.666 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T08:52:26.704 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T08:52:27.250 INFO:teuthology.orchestra.run.smithi176.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-14T08:52:27.251 INFO:teuthology.orchestra.run.smithi176.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-14T08:52:27.251 INFO:teuthology.orchestra.run.smithi176.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-14T08:52:27.251 INFO:teuthology.orchestra.run.smithi176.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-14T08:52:27.398 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T08:52:28.848 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T08:52:29.343 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T08:52:29.448 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T08:52:29.782 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-14T08:52:29.912 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T08:52:30.038 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-14T08:52:30.407 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T08:52:30.825 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-14T08:52:30.959 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-14T08:52:31.077 INFO:teuthology.orchestra.run.smithi176.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T08:52:31.127 INFO:teuthology.orchestra.run.smithi176.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-14T08:52:31.127 INFO:teuthology.orchestra.run.smithi176.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-14T08:52:32.510 INFO:teuthology.orchestra.run.smithi183.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-14T08:52:32.511 INFO:teuthology.orchestra.run.smithi183.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-14T08:52:32.511 INFO:teuthology.orchestra.run.smithi183.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-14T08:52:32.511 INFO:teuthology.orchestra.run.smithi183.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-14T08:52:32.669 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T08:52:34.086 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T08:52:35.471 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-14T08:52:35.572 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-14T08:52:35.681 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T08:52:35.746 INFO:teuthology.orchestra.run.smithi183.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-14T08:52:35.746 INFO:teuthology.orchestra.run.smithi183.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-14T08:52:45.366 INFO:teuthology.orchestra.run.smithi176.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-14T08:52:45.366 INFO:teuthology.orchestra.run.smithi176.stdout:Sourcing file `/etc/default/grub' 2024-09-14T08:52:45.381 INFO:teuthology.orchestra.run.smithi176.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-14T08:52:45.400 INFO:teuthology.orchestra.run.smithi176.stdout:Generating grub configuration file ... 2024-09-14T08:52:45.667 INFO:teuthology.orchestra.run.smithi176.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-14T08:52:45.689 INFO:teuthology.orchestra.run.smithi176.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-14T08:52:46.241 INFO:teuthology.orchestra.run.smithi176.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-14T08:52:46.246 INFO:teuthology.orchestra.run.smithi176.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-14T08:52:46.313 INFO:teuthology.orchestra.run.smithi176.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-14T08:52:46.319 INFO:teuthology.orchestra.run.smithi176.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-14T08:52:46.535 INFO:teuthology.orchestra.run.smithi176.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-14T08:52:46.535 INFO:teuthology.orchestra.run.smithi176.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-14T08:52:46.536 INFO:teuthology.orchestra.run.smithi176.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-14T08:52:46.576 INFO:teuthology.orchestra.run.smithi176.stdout:done 2024-09-14T08:52:47.065 DEBUG:teuthology.orchestra.run.smithi176:> dpkg -s linux-image-generic 2024-09-14T08:52:47.085 INFO:teuthology.orchestra.run.smithi176.stdout:Package: linux-image-generic 2024-09-14T08:52:47.086 INFO:teuthology.orchestra.run.smithi176.stdout:Status: install ok installed 2024-09-14T08:52:47.086 INFO:teuthology.orchestra.run.smithi176.stdout:Priority: optional 2024-09-14T08:52:47.086 INFO:teuthology.orchestra.run.smithi176.stdout:Section: kernel 2024-09-14T08:52:47.086 INFO:teuthology.orchestra.run.smithi176.stdout:Installed-Size: 21 2024-09-14T08:52:47.086 INFO:teuthology.orchestra.run.smithi176.stdout:Maintainer: Ubuntu Kernel Team 2024-09-14T08:52:47.086 INFO:teuthology.orchestra.run.smithi176.stdout:Architecture: amd64 2024-09-14T08:52:47.086 INFO:teuthology.orchestra.run.smithi176.stdout:Source: linux-meta 2024-09-14T08:52:47.086 INFO:teuthology.orchestra.run.smithi176.stdout:Version: 5.15.0.121.121 2024-09-14T08:52:47.086 INFO:teuthology.orchestra.run.smithi176.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-14T08:52:47.087 INFO:teuthology.orchestra.run.smithi176.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-14T08:52:47.087 INFO:teuthology.orchestra.run.smithi176.stdout:Recommends: thermald 2024-09-14T08:52:47.087 INFO:teuthology.orchestra.run.smithi176.stdout:Description: Generic Linux kernel image 2024-09-14T08:52:47.087 INFO:teuthology.orchestra.run.smithi176.stdout: This package will always depend on the latest generic kernel image 2024-09-14T08:52:47.087 INFO:teuthology.orchestra.run.smithi176.stdout: available. 2024-09-14T08:52:47.087 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-14T08:52:47.087 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T08:52:47.087 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T08:52:47.087 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-14T08:52:47.087 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi176.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T08:52:47.088 DEBUG:teuthology.orchestra.run.smithi176:> sudo apt-get clean 2024-09-14T08:52:47.250 DEBUG:teuthology.orchestra.run.smithi176:> sudo apt-get update 2024-09-14T08:52:47.510 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T08:52:47.510 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-14T08:52:47.582 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-14T08:52:47.663 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-14T08:52:48.938 INFO:teuthology.orchestra.run.smithi176.stdout:Reading package lists... 2024-09-14T08:52:48.957 DEBUG:teuthology.orchestra.run.smithi176:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-14T08:52:49.010 INFO:teuthology.orchestra.run.smithi176.stdout:Reading package lists... 2024-09-14T08:52:49.209 INFO:teuthology.orchestra.run.smithi176.stdout:Building dependency tree... 2024-09-14T08:52:49.209 INFO:teuthology.orchestra.run.smithi176.stdout:Reading state information... 2024-09-14T08:52:49.380 INFO:teuthology.orchestra.run.smithi176.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-14T08:52:49.380 INFO:teuthology.orchestra.run.smithi176.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T08:52:49.380 INFO:teuthology.orchestra.run.smithi176.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T08:52:49.380 INFO:teuthology.orchestra.run.smithi176.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T08:52:49.380 INFO:teuthology.orchestra.run.smithi176.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T08:52:49.381 INFO:teuthology.orchestra.run.smithi176.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-14T08:52:49.381 INFO:teuthology.orchestra.run.smithi176.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-14T08:52:49.381 INFO:teuthology.orchestra.run.smithi176.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T08:52:49.420 INFO:teuthology.orchestra.run.smithi176.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-14T08:52:49.422 DEBUG:teuthology.orchestra.run.smithi176:> dpkg -s linux-image-generic 2024-09-14T08:52:49.481 INFO:teuthology.orchestra.run.smithi176.stdout:Package: linux-image-generic 2024-09-14T08:52:49.481 INFO:teuthology.orchestra.run.smithi176.stdout:Status: install ok installed 2024-09-14T08:52:49.481 INFO:teuthology.orchestra.run.smithi176.stdout:Priority: optional 2024-09-14T08:52:49.481 INFO:teuthology.orchestra.run.smithi176.stdout:Section: kernel 2024-09-14T08:52:49.481 INFO:teuthology.orchestra.run.smithi176.stdout:Installed-Size: 21 2024-09-14T08:52:49.481 INFO:teuthology.orchestra.run.smithi176.stdout:Maintainer: Ubuntu Kernel Team 2024-09-14T08:52:49.481 INFO:teuthology.orchestra.run.smithi176.stdout:Architecture: amd64 2024-09-14T08:52:49.481 INFO:teuthology.orchestra.run.smithi176.stdout:Source: linux-meta 2024-09-14T08:52:49.481 INFO:teuthology.orchestra.run.smithi176.stdout:Version: 5.15.0.121.121 2024-09-14T08:52:49.481 INFO:teuthology.orchestra.run.smithi176.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-14T08:52:49.482 INFO:teuthology.orchestra.run.smithi176.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-14T08:52:49.482 INFO:teuthology.orchestra.run.smithi176.stdout:Recommends: thermald 2024-09-14T08:52:49.482 INFO:teuthology.orchestra.run.smithi176.stdout:Description: Generic Linux kernel image 2024-09-14T08:52:49.482 INFO:teuthology.orchestra.run.smithi176.stdout: This package will always depend on the latest generic kernel image 2024-09-14T08:52:49.482 INFO:teuthology.orchestra.run.smithi176.stdout: available. 2024-09-14T08:52:49.482 DEBUG:teuthology.orchestra.run.smithi176:> mktemp 2024-09-14T08:52:49.526 INFO:teuthology.orchestra.run.smithi176.stdout:/tmp/tmp.EEpS7ERD4W 2024-09-14T08:52:49.526 DEBUG:teuthology.orchestra.run.smithi176:> sudo cp /boot/grub/grub.cfg /tmp/tmp.EEpS7ERD4W 2024-09-14T08:52:49.549 INFO:teuthology.orchestra.run.smithi183.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-14T08:52:49.549 INFO:teuthology.orchestra.run.smithi183.stdout:Sourcing file `/etc/default/grub' 2024-09-14T08:52:49.560 INFO:teuthology.orchestra.run.smithi183.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-14T08:52:49.580 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 0666 /tmp/tmp.EEpS7ERD4W 2024-09-14T08:52:49.587 INFO:teuthology.orchestra.run.smithi183.stdout:Generating grub configuration file ... 2024-09-14T08:52:49.703 DEBUG:teuthology.orchestra.remote:smithi176:/tmp/tmp.EEpS7ERD4W is 10KB 2024-09-14T08:52:49.716 DEBUG:teuthology.orchestra.run.smithi176:> rm -fr /tmp/tmp.EEpS7ERD4W 2024-09-14T08:52:49.721 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-14T08:52:49.778 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T08:52:49.778 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-14T08:52:49.778 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-14T08:52:49.845 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-14T08:52:49.846 DEBUG:teuthology.orchestra.run.smithi176:> sudo update-grub 2024-09-14T08:52:49.936 INFO:teuthology.orchestra.run.smithi183.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-14T08:52:49.956 INFO:teuthology.orchestra.run.smithi183.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-14T08:52:50.326 INFO:teuthology.orchestra.run.smithi183.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-14T08:52:50.332 INFO:teuthology.orchestra.run.smithi183.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-14T08:52:50.398 INFO:teuthology.orchestra.run.smithi183.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-14T08:52:50.404 INFO:teuthology.orchestra.run.smithi183.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-14T08:52:50.571 INFO:teuthology.orchestra.run.smithi183.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-14T08:52:50.571 INFO:teuthology.orchestra.run.smithi183.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-14T08:52:50.571 INFO:teuthology.orchestra.run.smithi183.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-14T08:52:50.608 INFO:teuthology.orchestra.run.smithi183.stdout:done 2024-09-14T08:52:50.915 INFO:teuthology.orchestra.run.smithi176.stderr:Sourcing file `/etc/default/grub' 2024-09-14T08:52:50.917 INFO:teuthology.orchestra.run.smithi176.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-14T08:52:50.918 INFO:teuthology.orchestra.run.smithi176.stderr:Generating grub configuration file ... 2024-09-14T08:52:51.097 INFO:teuthology.orchestra.run.smithi176.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-14T08:52:51.103 INFO:teuthology.orchestra.run.smithi176.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-14T08:52:51.155 DEBUG:teuthology.orchestra.run.smithi183:> dpkg -s linux-image-generic 2024-09-14T08:52:51.180 INFO:teuthology.orchestra.run.smithi183.stdout:Package: linux-image-generic 2024-09-14T08:52:51.180 INFO:teuthology.orchestra.run.smithi183.stdout:Status: install ok installed 2024-09-14T08:52:51.180 INFO:teuthology.orchestra.run.smithi183.stdout:Priority: optional 2024-09-14T08:52:51.180 INFO:teuthology.orchestra.run.smithi183.stdout:Section: kernel 2024-09-14T08:52:51.181 INFO:teuthology.orchestra.run.smithi183.stdout:Installed-Size: 21 2024-09-14T08:52:51.181 INFO:teuthology.orchestra.run.smithi183.stdout:Maintainer: Ubuntu Kernel Team 2024-09-14T08:52:51.181 INFO:teuthology.orchestra.run.smithi183.stdout:Architecture: amd64 2024-09-14T08:52:51.181 INFO:teuthology.orchestra.run.smithi183.stdout:Source: linux-meta 2024-09-14T08:52:51.181 INFO:teuthology.orchestra.run.smithi183.stdout:Version: 5.15.0.121.121 2024-09-14T08:52:51.181 INFO:teuthology.orchestra.run.smithi183.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-14T08:52:51.181 INFO:teuthology.orchestra.run.smithi183.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-14T08:52:51.181 INFO:teuthology.orchestra.run.smithi183.stdout:Recommends: thermald 2024-09-14T08:52:51.181 INFO:teuthology.orchestra.run.smithi183.stdout:Description: Generic Linux kernel image 2024-09-14T08:52:51.181 INFO:teuthology.orchestra.run.smithi183.stdout: This package will always depend on the latest generic kernel image 2024-09-14T08:52:51.182 INFO:teuthology.orchestra.run.smithi183.stdout: available. 2024-09-14T08:52:51.182 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-14T08:52:51.182 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T08:52:51.182 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T08:52:51.182 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-14T08:52:51.182 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi183.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T08:52:51.182 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get clean 2024-09-14T08:52:51.324 INFO:teuthology.orchestra.run.smithi176.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-14T08:52:51.329 INFO:teuthology.orchestra.run.smithi176.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-14T08:52:51.368 INFO:teuthology.orchestra.run.smithi176.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-14T08:52:51.371 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get update 2024-09-14T08:52:51.373 INFO:teuthology.orchestra.run.smithi176.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-14T08:52:51.506 INFO:teuthology.orchestra.run.smithi176.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-14T08:52:51.506 INFO:teuthology.orchestra.run.smithi176.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-14T08:52:51.506 INFO:teuthology.orchestra.run.smithi176.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-14T08:52:51.508 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T08:52:51.508 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-14T08:52:51.518 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-14T08:52:51.524 INFO:teuthology.orchestra.run.smithi176.stderr:done 2024-09-14T08:52:51.528 DEBUG:teuthology.orchestra.run.smithi176:> sudo shutdown -r now 2024-09-14T08:52:51.535 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-14T08:52:52.931 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2024-09-14T08:52:52.950 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-14T08:52:53.003 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2024-09-14T08:52:53.190 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2024-09-14T08:52:53.191 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2024-09-14T08:52:53.362 INFO:teuthology.orchestra.run.smithi183.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-14T08:52:53.362 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T08:52:53.362 INFO:teuthology.orchestra.run.smithi183.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T08:52:53.362 INFO:teuthology.orchestra.run.smithi183.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T08:52:53.362 INFO:teuthology.orchestra.run.smithi183.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T08:52:53.362 INFO:teuthology.orchestra.run.smithi183.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-14T08:52:53.363 INFO:teuthology.orchestra.run.smithi183.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-14T08:52:53.363 INFO:teuthology.orchestra.run.smithi183.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T08:52:53.401 INFO:teuthology.orchestra.run.smithi183.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-14T08:52:53.404 DEBUG:teuthology.orchestra.run.smithi183:> dpkg -s linux-image-generic 2024-09-14T08:52:53.461 INFO:teuthology.orchestra.run.smithi183.stdout:Package: linux-image-generic 2024-09-14T08:52:53.461 INFO:teuthology.orchestra.run.smithi183.stdout:Status: install ok installed 2024-09-14T08:52:53.461 INFO:teuthology.orchestra.run.smithi183.stdout:Priority: optional 2024-09-14T08:52:53.461 INFO:teuthology.orchestra.run.smithi183.stdout:Section: kernel 2024-09-14T08:52:53.461 INFO:teuthology.orchestra.run.smithi183.stdout:Installed-Size: 21 2024-09-14T08:52:53.462 INFO:teuthology.orchestra.run.smithi183.stdout:Maintainer: Ubuntu Kernel Team 2024-09-14T08:52:53.462 INFO:teuthology.orchestra.run.smithi183.stdout:Architecture: amd64 2024-09-14T08:52:53.462 INFO:teuthology.orchestra.run.smithi183.stdout:Source: linux-meta 2024-09-14T08:52:53.462 INFO:teuthology.orchestra.run.smithi183.stdout:Version: 5.15.0.121.121 2024-09-14T08:52:53.462 INFO:teuthology.orchestra.run.smithi183.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-14T08:52:53.462 INFO:teuthology.orchestra.run.smithi183.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-14T08:52:53.462 INFO:teuthology.orchestra.run.smithi183.stdout:Recommends: thermald 2024-09-14T08:52:53.462 INFO:teuthology.orchestra.run.smithi183.stdout:Description: Generic Linux kernel image 2024-09-14T08:52:53.462 INFO:teuthology.orchestra.run.smithi183.stdout: This package will always depend on the latest generic kernel image 2024-09-14T08:52:53.462 INFO:teuthology.orchestra.run.smithi183.stdout: available. 2024-09-14T08:52:53.462 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2024-09-14T08:52:53.506 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.SLFnH9VuLs 2024-09-14T08:52:53.506 DEBUG:teuthology.orchestra.run.smithi183:> sudo cp /boot/grub/grub.cfg /tmp/tmp.SLFnH9VuLs 2024-09-14T08:52:53.562 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.SLFnH9VuLs 2024-09-14T08:52:53.677 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.SLFnH9VuLs is 10KB 2024-09-14T08:52:53.689 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.SLFnH9VuLs 2024-09-14T08:52:53.694 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-14T08:52:53.749 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T08:52:53.749 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-14T08:52:53.750 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-14T08:52:53.816 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-14T08:52:53.816 DEBUG:teuthology.orchestra.run.smithi183:> sudo update-grub 2024-09-14T08:52:55.062 INFO:teuthology.orchestra.run.smithi183.stderr:Sourcing file `/etc/default/grub' 2024-09-14T08:52:55.063 INFO:teuthology.orchestra.run.smithi183.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-14T08:52:55.065 INFO:teuthology.orchestra.run.smithi183.stderr:Generating grub configuration file ... 2024-09-14T08:52:55.303 INFO:teuthology.orchestra.run.smithi183.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-14T08:52:55.309 INFO:teuthology.orchestra.run.smithi183.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-14T08:52:55.623 INFO:teuthology.orchestra.run.smithi183.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-14T08:52:55.629 INFO:teuthology.orchestra.run.smithi183.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-14T08:52:55.666 INFO:teuthology.orchestra.run.smithi183.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-14T08:52:55.671 INFO:teuthology.orchestra.run.smithi183.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-14T08:52:55.801 INFO:teuthology.orchestra.run.smithi183.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-14T08:52:55.801 INFO:teuthology.orchestra.run.smithi183.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-14T08:52:55.801 INFO:teuthology.orchestra.run.smithi183.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-14T08:52:55.812 INFO:teuthology.orchestra.run.smithi183.stderr:done 2024-09-14T08:52:55.814 DEBUG:teuthology.orchestra.run.smithi183:> sudo shutdown -r now 2024-09-14T08:53:21.589 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-14T08:53:21.590 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-14T08:53:21.591 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T08:53:25.821 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-14T08:53:25.821 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-14T08:53:25.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T08:53:40.029 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.176 2024-09-14T08:53:44.381 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.183 2024-09-14T08:53:49.033 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-14T08:53:49.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T08:53:52.097 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.176 2024-09-14T08:53:53.383 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-14T08:53:53.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T08:53:56.445 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.183 2024-09-14T08:54:04.105 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-14T08:54:04.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T08:54:08.449 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-14T08:54:08.449 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T08:54:22.525 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.176 2024-09-14T08:54:26.877 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.183 2024-09-14T08:54:37.537 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-14T08:54:37.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T08:54:38.059 DEBUG:teuthology.orchestra.run.smithi176:> true 2024-09-14T08:54:38.999 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-14T08:54:38.999 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-121-generic"... 2024-09-14T08:54:38.999 DEBUG:teuthology.orchestra.run.smithi176:> uname -r 2024-09-14T08:54:39.046 INFO:teuthology.orchestra.run.smithi176.stdout:5.15.0-121-generic 2024-09-14T08:54:39.047 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-14T08:54:39.047 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-14T08:54:39.047 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-14T08:54:40.048 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-14T08:54:40.048 DEBUG:teuthology.orchestra.run.smithi176:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-14T08:54:40.506 INFO:teuthology.orchestra.run.smithi176.stdout:ttyS1 2024-09-14T08:54:40.541 DEBUG:teuthology.parallel:result is None 2024-09-14T08:54:41.878 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-14T08:54:41.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T08:54:42.308 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-09-14T08:54:43.125 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-14T08:54:43.125 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-121-generic"... 2024-09-14T08:54:43.126 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2024-09-14T08:54:43.172 INFO:teuthology.orchestra.run.smithi183.stdout:5.15.0-121-generic 2024-09-14T08:54:43.172 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-14T08:54:43.172 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-14T08:54:43.172 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-14T08:54:44.173 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-14T08:54:44.174 DEBUG:teuthology.orchestra.run.smithi183:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-14T08:54:44.248 INFO:teuthology.orchestra.run.smithi183.stdout:ttyS1 2024-09-14T08:54:44.281 DEBUG:teuthology.parallel:result is None 2024-09-14T08:54:44.281 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-14T08:54:44.288 INFO:teuthology.task.internal:Creating test directory... 2024-09-14T08:54:44.288 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-14T08:54:44.290 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-14T08:54:44.296 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-14T08:54:44.329 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-14T08:54:44.379 INFO:teuthology.task.internal:Creating archive directory... 2024-09-14T08:54:44.380 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-14T08:54:44.382 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-14T08:54:44.483 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-14T08:54:44.490 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-14T08:54:44.490 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-14T08:54:44.493 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-14T08:54:44.508 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T08:54:44.513 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T08:54:44.516 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T08:54:44.521 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T08:54:44.523 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-14T08:54:44.529 INFO:teuthology.task.internal:Configuring sudo... 2024-09-14T08:54:44.530 DEBUG:teuthology.orchestra.run.smithi176:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-14T08:54:44.564 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-14T08:54:44.597 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-14T08:54:44.606 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-14T08:54:44.607 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-14T08:54:44.640 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-14T08:54:44.645 DEBUG:teuthology.orchestra.run.smithi176:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T08:54:44.691 DEBUG:teuthology.orchestra.run.smithi176:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T08:54:44.739 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T08:54:44.739 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-14T08:54:44.803 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T08:54:44.809 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T08:54:44.856 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T08:54:44.856 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-14T08:54:44.923 DEBUG:teuthology.orchestra.run.smithi176:> sudo service rsyslog restart 2024-09-14T08:54:44.926 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-09-14T08:54:45.003 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-14T08:54:45.011 INFO:teuthology.task.internal:Starting timer... 2024-09-14T08:54:45.011 INFO:teuthology.run_tasks:Running task pcp... 2024-09-14T08:54:45.060 INFO:teuthology.run_tasks:Running task selinux... 2024-09-14T08:54:45.107 DEBUG:teuthology.task.selinux:Excluding smithi176: OS 'ubuntu' does not support SELinux 2024-09-14T08:54:45.108 DEBUG:teuthology.task.selinux:Excluding smithi183: OS 'ubuntu' does not support SELinux 2024-09-14T08:54:45.108 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-14T08:54:45.108 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-14T08:54:45.108 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-14T08:54:45.108 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-14T08:54:45.157 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-14T08:54:45.168 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-14T08:54:45.169 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi176.front.sepia.ceph.com,smithi183.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-14T09:02:13.016 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi176.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com')] 2024-09-14T09:02:13.018 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-14T09:02:13.020 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T09:02:13.102 DEBUG:teuthology.orchestra.run.smithi176:> true 2024-09-14T09:02:13.181 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-09-14T09:02:13.181 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-14T09:02:13.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T09:02:13.259 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-09-14T09:02:13.336 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-14T09:02:13.337 INFO:teuthology.run_tasks:Running task clock... 2024-09-14T09:02:13.347 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-14T09:02:13.347 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-14T09:02:13.347 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T09:02:13.350 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-14T09:02:13.350 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T09:02:13.377 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-14T09:02:13.377 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: Command line: ntpd -gq 2024-09-14T09:02:13.377 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: ---------------------------------------------------- 2024-09-14T09:02:13.378 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: ntp-4 is maintained by Network Time Foundation, 2024-09-14T09:02:13.378 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-14T09:02:13.378 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: corporation. Support and training for ntp-4 are 2024-09-14T09:02:13.378 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: available at https://www.nwtime.org/support 2024-09-14T09:02:13.378 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: ---------------------------------------------------- 2024-09-14T09:02:13.378 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: proto: precision = 0.061 usec (-24) 2024-09-14T09:02:13.378 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: basedate set to 2022-02-04 2024-09-14T09:02:13.379 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: gps base set to 2022-02-06 (week 2196) 2024-09-14T09:02:13.379 INFO:teuthology.orchestra.run.smithi176.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-14T09:02:13.379 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-14T09:02:13.379 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: restrict ::: KOD does nothing without LIMITED. 2024-09-14T09:02:13.380 INFO:teuthology.orchestra.run.smithi176.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-14T09:02:13.381 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: Listen and drop on 0 v6wildcard [::]:123 2024-09-14T09:02:13.381 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-14T09:02:13.381 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: Listen normally on 2 lo 127.0.0.1:123 2024-09-14T09:02:13.381 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: Listen normally on 3 enp3s0f1 172.21.15.176:123 2024-09-14T09:02:13.381 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: Listen normally on 4 lo [::1]:123 2024-09-14T09:02:13.381 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6f61%5]:123 2024-09-14T09:02:13.381 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:13 ntpd[18457]: Listening on routing socket on fd #22 for interface updates 2024-09-14T09:02:13.409 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-14T09:02:13.409 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: Command line: ntpd -gq 2024-09-14T09:02:13.409 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: ---------------------------------------------------- 2024-09-14T09:02:13.409 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: ntp-4 is maintained by Network Time Foundation, 2024-09-14T09:02:13.410 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-14T09:02:13.410 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: corporation. Support and training for ntp-4 are 2024-09-14T09:02:13.410 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: available at https://www.nwtime.org/support 2024-09-14T09:02:13.410 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: ---------------------------------------------------- 2024-09-14T09:02:13.410 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: proto: precision = 0.060 usec (-24) 2024-09-14T09:02:13.410 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: basedate set to 2022-02-04 2024-09-14T09:02:13.410 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: gps base set to 2022-02-06 (week 2196) 2024-09-14T09:02:13.411 INFO:teuthology.orchestra.run.smithi183.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-14T09:02:13.411 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-14T09:02:13.411 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: restrict ::: KOD does nothing without LIMITED. 2024-09-14T09:02:13.411 INFO:teuthology.orchestra.run.smithi183.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-14T09:02:13.412 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: Listen and drop on 0 v6wildcard [::]:123 2024-09-14T09:02:13.412 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-14T09:02:13.412 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: Listen normally on 2 lo 127.0.0.1:123 2024-09-14T09:02:13.413 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: Listen normally on 3 enp3s0f1 172.21.15.183:123 2024-09-14T09:02:13.413 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: Listen normally on 4 lo [::1]:123 2024-09-14T09:02:13.413 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8085%5]:123 2024-09-14T09:02:13.413 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:13 ntpd[18495]: Listening on routing socket on fd #22 for interface updates 2024-09-14T09:02:14.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:14 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:14.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:14 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:14.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:14 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:14.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:14 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:14.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:14 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T09:02:14.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:14 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T09:02:14.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:14 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:14.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:14 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:14.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:14 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:14.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:14 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:14.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:14 ntpd[18495]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T09:02:14.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:14 ntpd[18495]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T09:02:15.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:15 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:15.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:15 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:15.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:15 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:15.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:15 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:15.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:15 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T09:02:15.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:15 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T09:02:15.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:15 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:15.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:15 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:15.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:15 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:15.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:15 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:15.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:15 ntpd[18495]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T09:02:15.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:15 ntpd[18495]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T09:02:16.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:16 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:16.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:16 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:16.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:16 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:16.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:16 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:16.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:16 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T09:02:16.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:16 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T09:02:16.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:16 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:16.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:16 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:16.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:16 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:16.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:16 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:16.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:16 ntpd[18495]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T09:02:16.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:16 ntpd[18495]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T09:02:17.379 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:17 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:17.379 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:17 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:17.379 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:17 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:17.379 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:17 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:17.379 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:17 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:17.379 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:17 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:17.379 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:17 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:17.379 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:17 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:17.379 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:17 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T09:02:17.379 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:17 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T09:02:17.379 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:17 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:17.379 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:17 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:17.380 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:17 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T09:02:17.380 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:17 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T09:02:17.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:17 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:17.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:17 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:17.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:17 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:17.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:17 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:17.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:17 ntpd[18495]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T09:02:17.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:17 ntpd[18495]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T09:02:18.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:18 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:18.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:18 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:18.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:18 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:18.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:18 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:18.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:18 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T09:02:18.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:18 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T09:02:18.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:18 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:18.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:18 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:18.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:18 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:18.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:18 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:18.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:18 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:18.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:18 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:18.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:18 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:18.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:18 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:18.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:18 ntpd[18495]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T09:02:18.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:18 ntpd[18495]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T09:02:18.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:18 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:18.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:18 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:18.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:18 ntpd[18495]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T09:02:18.412 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:18 ntpd[18495]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T09:02:19.379 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:19 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:19.379 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:19 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:19.379 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:19 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:19.380 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:19 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:19.380 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:19 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:19.380 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:19 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:19.380 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:19 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T09:02:19.380 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:19 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T09:02:19.380 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:19 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:19.381 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:19 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:19.381 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:19 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T09:02:19.381 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:19 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T09:02:19.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:19 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:19.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:19 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:19.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:19 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:19.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:19 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:19.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:19 ntpd[18495]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T09:02:19.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:19 ntpd[18495]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T09:02:20.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:20 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:20.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:20 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:20.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:20 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:20.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:20 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:20.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:20 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T09:02:20.378 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:20 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T09:02:20.379 INFO:teuthology.orchestra.run.smithi176.stdout:14 Sep 09:02:20 ntpd[18457]: ntpd: time slew +0.001582 s 2024-09-14T09:02:20.379 INFO:teuthology.orchestra.run.smithi176.stdout:ntpd: time slew +0.001582s 2024-09-14T09:02:20.379 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:20 ntpd[18457]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-14T09:02:20.379 INFO:teuthology.orchestra.run.smithi176.stderr:14 Sep 09:02:20 ntpd[18457]: can't open /var/log/ntpstats/loopstats.20240914: Permission denied 2024-09-14T09:02:20.410 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:20 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:20.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:20 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:20.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:20 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:20.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:20 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:20.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:20 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:20.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:20 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:20.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:20 ntpd[18495]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T09:02:20.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:20 ntpd[18495]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T09:02:20.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:20 ntpd[18495]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T09:02:20.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:20 ntpd[18495]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T09:02:20.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:20 ntpd[18495]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T09:02:20.411 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:20 ntpd[18495]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T09:02:20.412 INFO:teuthology.orchestra.run.smithi183.stdout:14 Sep 09:02:20 ntpd[18495]: ntpd: time slew -0.003442 s 2024-09-14T09:02:20.412 INFO:teuthology.orchestra.run.smithi183.stdout:ntpd: time slew -0.003442s 2024-09-14T09:02:20.412 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:20 ntpd[18495]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-14T09:02:20.412 INFO:teuthology.orchestra.run.smithi183.stderr:14 Sep 09:02:20 ntpd[18495]: can't open /var/log/ntpstats/loopstats.20240914: Permission denied 2024-09-14T09:02:20.429 INFO:teuthology.orchestra.run.smithi176.stdout: remote refid st t when poll reach delay offset jitter 2024-09-14T09:02:20.429 INFO:teuthology.orchestra.run.smithi176.stdout:============================================================================== 2024-09-14T09:02:20.429 INFO:teuthology.orchestra.run.smithi176.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T09:02:20.429 INFO:teuthology.orchestra.run.smithi176.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T09:02:20.429 INFO:teuthology.orchestra.run.smithi176.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T09:02:20.429 INFO:teuthology.orchestra.run.smithi176.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T09:02:20.463 INFO:teuthology.orchestra.run.smithi183.stdout: remote refid st t when poll reach delay offset jitter 2024-09-14T09:02:20.463 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================== 2024-09-14T09:02:20.463 INFO:teuthology.orchestra.run.smithi183.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T09:02:20.463 INFO:teuthology.orchestra.run.smithi183.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T09:02:20.463 INFO:teuthology.orchestra.run.smithi183.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T09:02:20.463 INFO:teuthology.orchestra.run.smithi183.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T09:02:20.464 INFO:teuthology.run_tasks:Running task install... 2024-09-14T09:02:20.472 DEBUG:teuthology.task.install:project ceph 2024-09-14T09:02:20.472 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}} 2024-09-14T09:02:20.473 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'} 2024-09-14T09:02:20.473 INFO:teuthology.task.install:Using flavor: default 2024-09-14T09:02:20.478 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-14T09:02:20.478 INFO:teuthology.task.install:extra packages: [] 2024-09-14T09:02:20.479 DEBUG:teuthology.orchestra.run.smithi176:> sudo apt-key list | grep Ceph 2024-09-14T09:02:20.479 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-key list | grep Ceph 2024-09-14T09:02:20.537 INFO:teuthology.orchestra.run.smithi176.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-14T09:02:20.564 INFO:teuthology.orchestra.run.smithi176.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-14T09:02:20.564 INFO:teuthology.orchestra.run.smithi176.stdout:uid [ unknown] Ceph.com (release key) 2024-09-14T09:02:20.565 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-14T09:02:20.566 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T09:02:20.572 INFO:teuthology.orchestra.run.smithi183.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-14T09:02:20.604 INFO:teuthology.orchestra.run.smithi183.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-14T09:02:20.604 INFO:teuthology.orchestra.run.smithi183.stdout:uid [ unknown] Ceph.com (release key) 2024-09-14T09:02:20.604 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-14T09:02:20.604 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T09:02:20.706 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default/ 2024-09-14T09:02:20.706 INFO:teuthology.task.install.deb:Package version is 19.3.0-4775-g7f24e589-1jammy 2024-09-14T09:02:20.743 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default/ 2024-09-14T09:02:20.743 INFO:teuthology.task.install.deb:Package version is 19.3.0-4775-g7f24e589-1jammy 2024-09-14T09:02:20.815 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:02:20.816 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-14T09:02:20.829 DEBUG:teuthology.orchestra.run.smithi176:> sudo apt-get update 2024-09-14T09:02:20.851 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T09:02:20.851 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-14T09:02:20.864 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get update 2024-09-14T09:02:21.000 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T09:02:21.009 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-14T09:02:21.026 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-14T09:02:21.028 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T09:02:21.028 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-14T09:02:21.039 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-14T09:02:21.053 INFO:teuthology.orchestra.run.smithi176.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy InRelease 2024-09-14T09:02:21.056 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-14T09:02:21.074 INFO:teuthology.orchestra.run.smithi176.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-14T09:02:21.082 INFO:teuthology.orchestra.run.smithi183.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy InRelease 2024-09-14T09:02:21.096 INFO:teuthology.orchestra.run.smithi176.stdout:Ign:6 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-14T09:02:21.104 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-14T09:02:21.117 INFO:teuthology.orchestra.run.smithi176.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-14T09:02:21.123 INFO:teuthology.orchestra.run.smithi176.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-14T09:02:21.126 INFO:teuthology.orchestra.run.smithi183.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-14T09:02:21.144 INFO:teuthology.orchestra.run.smithi176.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,420 B] 2024-09-14T09:02:21.148 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-14T09:02:21.174 INFO:teuthology.orchestra.run.smithi183.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,420 B] 2024-09-14T09:02:21.350 INFO:teuthology.orchestra.run.smithi176.stdout:Fetched 29.3 kB in 0s (85.6 kB/s) 2024-09-14T09:02:21.368 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 29.3 kB in 0s (89.2 kB/s) 2024-09-14T09:02:22.320 INFO:teuthology.orchestra.run.smithi176.stdout:Reading package lists... 2024-09-14T09:02:22.337 DEBUG:teuthology.orchestra.run.smithi176:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4775-g7f24e589-1jammy cephadm=19.3.0-4775-g7f24e589-1jammy ceph-mds=19.3.0-4775-g7f24e589-1jammy ceph-mgr=19.3.0-4775-g7f24e589-1jammy ceph-common=19.3.0-4775-g7f24e589-1jammy ceph-fuse=19.3.0-4775-g7f24e589-1jammy ceph-test=19.3.0-4775-g7f24e589-1jammy ceph-volume=19.3.0-4775-g7f24e589-1jammy radosgw=19.3.0-4775-g7f24e589-1jammy python3-rados=19.3.0-4775-g7f24e589-1jammy python3-rgw=19.3.0-4775-g7f24e589-1jammy python3-cephfs=19.3.0-4775-g7f24e589-1jammy python3-rbd=19.3.0-4775-g7f24e589-1jammy libcephfs2=19.3.0-4775-g7f24e589-1jammy libcephfs-dev=19.3.0-4775-g7f24e589-1jammy librados2=19.3.0-4775-g7f24e589-1jammy librbd1=19.3.0-4775-g7f24e589-1jammy rbd-fuse=19.3.0-4775-g7f24e589-1jammy 2024-09-14T09:02:22.398 INFO:teuthology.orchestra.run.smithi176.stdout:Reading package lists... 2024-09-14T09:02:22.422 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2024-09-14T09:02:22.439 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4775-g7f24e589-1jammy cephadm=19.3.0-4775-g7f24e589-1jammy ceph-mds=19.3.0-4775-g7f24e589-1jammy ceph-mgr=19.3.0-4775-g7f24e589-1jammy ceph-common=19.3.0-4775-g7f24e589-1jammy ceph-fuse=19.3.0-4775-g7f24e589-1jammy ceph-test=19.3.0-4775-g7f24e589-1jammy ceph-volume=19.3.0-4775-g7f24e589-1jammy radosgw=19.3.0-4775-g7f24e589-1jammy python3-rados=19.3.0-4775-g7f24e589-1jammy python3-rgw=19.3.0-4775-g7f24e589-1jammy python3-cephfs=19.3.0-4775-g7f24e589-1jammy python3-rbd=19.3.0-4775-g7f24e589-1jammy libcephfs2=19.3.0-4775-g7f24e589-1jammy libcephfs-dev=19.3.0-4775-g7f24e589-1jammy librados2=19.3.0-4775-g7f24e589-1jammy librbd1=19.3.0-4775-g7f24e589-1jammy rbd-fuse=19.3.0-4775-g7f24e589-1jammy 2024-09-14T09:02:22.501 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2024-09-14T09:02:22.607 INFO:teuthology.orchestra.run.smithi176.stdout:Building dependency tree... 2024-09-14T09:02:22.608 INFO:teuthology.orchestra.run.smithi176.stdout:Reading state information... 2024-09-14T09:02:22.712 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2024-09-14T09:02:22.712 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2024-09-14T09:02:22.805 INFO:teuthology.orchestra.run.smithi176.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T09:02:22.805 INFO:teuthology.orchestra.run.smithi176.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T09:02:22.805 INFO:teuthology.orchestra.run.smithi176.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T09:02:22.805 INFO:teuthology.orchestra.run.smithi176.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T09:02:22.806 INFO:teuthology.orchestra.run.smithi176.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-14T09:02:22.806 INFO:teuthology.orchestra.run.smithi176.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-14T09:02:22.806 INFO:teuthology.orchestra.run.smithi176.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T09:02:22.807 INFO:teuthology.orchestra.run.smithi176.stdout:The following additional packages will be installed: 2024-09-14T09:02:22.807 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-14T09:02:22.807 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-14T09:02:22.807 INFO:teuthology.orchestra.run.smithi176.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-14T09:02:22.807 INFO:teuthology.orchestra.run.smithi176.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-14T09:02:22.807 INFO:teuthology.orchestra.run.smithi176.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-14T09:02:22.808 INFO:teuthology.orchestra.run.smithi176.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-14T09:02:22.808 INFO:teuthology.orchestra.run.smithi176.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-14T09:02:22.808 INFO:teuthology.orchestra.run.smithi176.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-14T09:02:22.809 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-14T09:02:22.809 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-14T09:02:22.809 INFO:teuthology.orchestra.run.smithi176.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-14T09:02:22.809 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-14T09:02:22.809 INFO:teuthology.orchestra.run.smithi176.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-14T09:02:22.809 INFO:teuthology.orchestra.run.smithi176.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-14T09:02:22.809 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-14T09:02:22.809 INFO:teuthology.orchestra.run.smithi176.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-14T09:02:22.809 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-14T09:02:22.809 INFO:teuthology.orchestra.run.smithi176.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-14T09:02:22.809 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-14T09:02:22.809 INFO:teuthology.orchestra.run.smithi176.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-14T09:02:22.809 INFO:teuthology.orchestra.run.smithi176.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-14T09:02:22.809 INFO:teuthology.orchestra.run.smithi176.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-14T09:02:22.809 INFO:teuthology.orchestra.run.smithi176.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-14T09:02:22.810 INFO:teuthology.orchestra.run.smithi176.stdout: xmlstarlet zip 2024-09-14T09:02:22.810 INFO:teuthology.orchestra.run.smithi176.stdout:Suggested packages: 2024-09-14T09:02:22.810 INFO:teuthology.orchestra.run.smithi176.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-14T09:02:22.810 INFO:teuthology.orchestra.run.smithi176.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-14T09:02:22.810 INFO:teuthology.orchestra.run.smithi176.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-14T09:02:22.810 INFO:teuthology.orchestra.run.smithi176.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-14T09:02:22.810 INFO:teuthology.orchestra.run.smithi176.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-14T09:02:22.810 INFO:teuthology.orchestra.run.smithi176.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-14T09:02:22.810 INFO:teuthology.orchestra.run.smithi176.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-14T09:02:22.811 INFO:teuthology.orchestra.run.smithi176.stdout:Recommended packages: 2024-09-14T09:02:22.811 INFO:teuthology.orchestra.run.smithi176.stdout: btrfs-tools 2024-09-14T09:02:22.909 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T09:02:22.909 INFO:teuthology.orchestra.run.smithi183.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T09:02:22.909 INFO:teuthology.orchestra.run.smithi183.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T09:02:22.910 INFO:teuthology.orchestra.run.smithi183.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T09:02:22.910 INFO:teuthology.orchestra.run.smithi183.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-14T09:02:22.910 INFO:teuthology.orchestra.run.smithi183.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-14T09:02:22.910 INFO:teuthology.orchestra.run.smithi183.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T09:02:22.911 INFO:teuthology.orchestra.run.smithi183.stdout:The following additional packages will be installed: 2024-09-14T09:02:22.911 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-14T09:02:22.911 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-14T09:02:22.911 INFO:teuthology.orchestra.run.smithi183.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-14T09:02:22.911 INFO:teuthology.orchestra.run.smithi183.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-14T09:02:22.912 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-14T09:02:22.912 INFO:teuthology.orchestra.run.smithi183.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-14T09:02:22.912 INFO:teuthology.orchestra.run.smithi183.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-14T09:02:22.912 INFO:teuthology.orchestra.run.smithi183.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-14T09:02:22.912 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-14T09:02:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-14T09:02:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-14T09:02:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-14T09:02:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-14T09:02:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-14T09:02:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-14T09:02:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-14T09:02:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-14T09:02:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-14T09:02:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-14T09:02:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-14T09:02:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-14T09:02:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-14T09:02:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-14T09:02:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet zip 2024-09-14T09:02:22.914 INFO:teuthology.orchestra.run.smithi183.stdout:Suggested packages: 2024-09-14T09:02:22.914 INFO:teuthology.orchestra.run.smithi183.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-14T09:02:22.914 INFO:teuthology.orchestra.run.smithi183.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-14T09:02:22.914 INFO:teuthology.orchestra.run.smithi183.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-14T09:02:22.914 INFO:teuthology.orchestra.run.smithi183.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-14T09:02:22.915 INFO:teuthology.orchestra.run.smithi183.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-14T09:02:22.915 INFO:teuthology.orchestra.run.smithi183.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-14T09:02:22.915 INFO:teuthology.orchestra.run.smithi183.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-14T09:02:22.915 INFO:teuthology.orchestra.run.smithi183.stdout:Recommended packages: 2024-09-14T09:02:22.915 INFO:teuthology.orchestra.run.smithi183.stdout: btrfs-tools 2024-09-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi176.stdout:The following NEW packages will be installed: 2024-09-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi176.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi176.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-14T09:02:22.916 INFO:teuthology.orchestra.run.smithi176.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi176.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi176.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi176.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-14T09:02:22.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-14T09:02:22.918 INFO:teuthology.orchestra.run.smithi176.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-14T09:02:22.918 INFO:teuthology.orchestra.run.smithi176.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-14T09:02:22.918 INFO:teuthology.orchestra.run.smithi176.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-14T09:02:22.918 INFO:teuthology.orchestra.run.smithi176.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-14T09:02:22.918 INFO:teuthology.orchestra.run.smithi176.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-14T09:02:22.918 INFO:teuthology.orchestra.run.smithi176.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-14T09:02:23.016 INFO:teuthology.orchestra.run.smithi183.stdout:The following NEW packages will be installed: 2024-09-14T09:02:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-14T09:02:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-14T09:02:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-14T09:02:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-14T09:02:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-14T09:02:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-14T09:02:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-14T09:02:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-14T09:02:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-14T09:02:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-14T09:02:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-14T09:02:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-14T09:02:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-14T09:02:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-14T09:02:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-14T09:02:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-14T09:02:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-14T09:02:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-14T09:02:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-14T09:02:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-14T09:02:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-14T09:02:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-14T09:02:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-14T09:02:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-14T09:02:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-14T09:02:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-14T09:02:23.077 INFO:teuthology.orchestra.run.smithi183.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-14T09:02:23.077 INFO:teuthology.orchestra.run.smithi183.stdout:Need to get 251 MB of archives. 2024-09-14T09:02:23.078 INFO:teuthology.orchestra.run.smithi183.stdout:After this operation, 934 MB of additional disk space will be used. 2024-09-14T09:02:23.078 INFO:teuthology.orchestra.run.smithi183.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-14T09:02:23.102 INFO:teuthology.orchestra.run.smithi176.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-14T09:02:23.102 INFO:teuthology.orchestra.run.smithi176.stdout:Need to get 251 MB of archives. 2024-09-14T09:02:23.102 INFO:teuthology.orchestra.run.smithi176.stdout:After this operation, 934 MB of additional disk space will be used. 2024-09-14T09:02:23.102 INFO:teuthology.orchestra.run.smithi176.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-14T09:02:23.165 INFO:teuthology.orchestra.run.smithi183.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-14T09:02:23.170 INFO:teuthology.orchestra.run.smithi183.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-14T09:02:23.185 INFO:teuthology.orchestra.run.smithi183.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-14T09:02:23.244 INFO:teuthology.orchestra.run.smithi183.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-14T09:02:23.249 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-14T09:02:23.261 INFO:teuthology.orchestra.run.smithi183.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-14T09:02:23.266 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-14T09:02:23.267 INFO:teuthology.orchestra.run.smithi183.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-14T09:02:23.267 INFO:teuthology.orchestra.run.smithi183.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-14T09:02:23.268 INFO:teuthology.orchestra.run.smithi183.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-14T09:02:23.277 INFO:teuthology.orchestra.run.smithi183.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-14T09:02:23.277 INFO:teuthology.orchestra.run.smithi176.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4775-g7f24e589-1jammy [3,631 kB] 2024-09-14T09:02:23.282 INFO:teuthology.orchestra.run.smithi183.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-14T09:02:23.282 INFO:teuthology.orchestra.run.smithi183.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-14T09:02:23.283 INFO:teuthology.orchestra.run.smithi183.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-14T09:02:23.283 INFO:teuthology.orchestra.run.smithi183.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-14T09:02:23.300 INFO:teuthology.orchestra.run.smithi183.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-14T09:02:23.302 INFO:teuthology.orchestra.run.smithi183.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-14T09:02:23.304 INFO:teuthology.orchestra.run.smithi183.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-14T09:02:23.304 INFO:teuthology.orchestra.run.smithi183.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-14T09:02:23.305 INFO:teuthology.orchestra.run.smithi183.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-14T09:02:23.306 INFO:teuthology.orchestra.run.smithi183.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-14T09:02:23.306 INFO:teuthology.orchestra.run.smithi183.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-14T09:02:23.317 INFO:teuthology.orchestra.run.smithi183.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-14T09:02:23.317 INFO:teuthology.orchestra.run.smithi183.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-14T09:02:23.318 INFO:teuthology.orchestra.run.smithi183.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-14T09:02:23.334 INFO:teuthology.orchestra.run.smithi183.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-14T09:02:23.335 INFO:teuthology.orchestra.run.smithi183.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-14T09:02:23.337 INFO:teuthology.orchestra.run.smithi183.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-14T09:02:23.337 INFO:teuthology.orchestra.run.smithi183.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-14T09:02:23.338 INFO:teuthology.orchestra.run.smithi183.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-14T09:02:23.338 INFO:teuthology.orchestra.run.smithi183.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-14T09:02:23.339 INFO:teuthology.orchestra.run.smithi183.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-14T09:02:23.351 INFO:teuthology.orchestra.run.smithi183.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-14T09:02:23.351 INFO:teuthology.orchestra.run.smithi183.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-14T09:02:23.353 INFO:teuthology.orchestra.run.smithi183.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-14T09:02:23.368 INFO:teuthology.orchestra.run.smithi183.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-14T09:02:23.368 INFO:teuthology.orchestra.run.smithi183.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-14T09:02:23.374 INFO:teuthology.orchestra.run.smithi183.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-14T09:02:23.374 INFO:teuthology.orchestra.run.smithi183.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-14T09:02:23.375 INFO:teuthology.orchestra.run.smithi183.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-14T09:02:23.375 INFO:teuthology.orchestra.run.smithi183.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-14T09:02:23.376 INFO:teuthology.orchestra.run.smithi183.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-14T09:02:23.385 INFO:teuthology.orchestra.run.smithi183.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-14T09:02:23.386 INFO:teuthology.orchestra.run.smithi183.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-14T09:02:23.387 INFO:teuthology.orchestra.run.smithi183.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-14T09:02:23.394 INFO:teuthology.orchestra.run.smithi183.stdout:Get:47 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4775-g7f24e589-1jammy [3,631 kB] 2024-09-14T09:02:23.402 INFO:teuthology.orchestra.run.smithi183.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-14T09:02:23.404 INFO:teuthology.orchestra.run.smithi183.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-14T09:02:23.405 INFO:teuthology.orchestra.run.smithi183.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-14T09:02:23.405 INFO:teuthology.orchestra.run.smithi183.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-14T09:02:23.408 INFO:teuthology.orchestra.run.smithi183.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-14T09:02:23.460 INFO:teuthology.orchestra.run.smithi183.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-14T09:02:23.461 INFO:teuthology.orchestra.run.smithi183.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-14T09:02:23.462 INFO:teuthology.orchestra.run.smithi183.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-14T09:02:23.462 INFO:teuthology.orchestra.run.smithi183.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-14T09:02:23.463 INFO:teuthology.orchestra.run.smithi183.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-14T09:02:23.480 INFO:teuthology.orchestra.run.smithi183.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-14T09:02:23.482 INFO:teuthology.orchestra.run.smithi183.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-14T09:02:23.482 INFO:teuthology.orchestra.run.smithi183.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-14T09:02:23.496 INFO:teuthology.orchestra.run.smithi183.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-14T09:02:23.496 INFO:teuthology.orchestra.run.smithi183.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-14T09:02:23.504 INFO:teuthology.orchestra.run.smithi183.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-14T09:02:23.505 INFO:teuthology.orchestra.run.smithi183.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-14T09:02:23.505 INFO:teuthology.orchestra.run.smithi183.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-14T09:02:23.506 INFO:teuthology.orchestra.run.smithi183.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-14T09:02:23.506 INFO:teuthology.orchestra.run.smithi183.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-14T09:02:23.506 INFO:teuthology.orchestra.run.smithi183.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-14T09:02:23.509 INFO:teuthology.orchestra.run.smithi183.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-14T09:02:23.521 INFO:teuthology.orchestra.run.smithi183.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-14T09:02:23.522 INFO:teuthology.orchestra.run.smithi183.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-14T09:02:23.522 INFO:teuthology.orchestra.run.smithi183.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-14T09:02:23.524 INFO:teuthology.orchestra.run.smithi176.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-14T09:02:23.536 INFO:teuthology.orchestra.run.smithi176.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-14T09:02:23.538 INFO:teuthology.orchestra.run.smithi183.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-14T09:02:23.540 INFO:teuthology.orchestra.run.smithi183.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-14T09:02:23.541 INFO:teuthology.orchestra.run.smithi183.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-14T09:02:23.541 INFO:teuthology.orchestra.run.smithi183.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-14T09:02:23.545 INFO:teuthology.orchestra.run.smithi183.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-14T09:02:23.546 INFO:teuthology.orchestra.run.smithi183.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-14T09:02:23.549 INFO:teuthology.orchestra.run.smithi183.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-14T09:02:23.555 INFO:teuthology.orchestra.run.smithi183.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-14T09:02:23.556 INFO:teuthology.orchestra.run.smithi183.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-14T09:02:23.556 INFO:teuthology.orchestra.run.smithi183.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-14T09:02:23.572 INFO:teuthology.orchestra.run.smithi183.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-14T09:02:23.572 INFO:teuthology.orchestra.run.smithi183.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-14T09:02:23.578 INFO:teuthology.orchestra.run.smithi183.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-14T09:02:23.578 INFO:teuthology.orchestra.run.smithi183.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-14T09:02:23.579 INFO:teuthology.orchestra.run.smithi183.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-14T09:02:23.580 INFO:teuthology.orchestra.run.smithi183.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-14T09:02:23.581 INFO:teuthology.orchestra.run.smithi183.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-14T09:02:23.617 INFO:teuthology.orchestra.run.smithi176.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-14T09:02:23.847 INFO:teuthology.orchestra.run.smithi176.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-14T09:02:23.857 INFO:teuthology.orchestra.run.smithi176.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-14T09:02:23.887 INFO:teuthology.orchestra.run.smithi176.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-14T09:02:23.896 INFO:teuthology.orchestra.run.smithi176.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-14T09:02:23.898 INFO:teuthology.orchestra.run.smithi176.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-14T09:02:23.899 INFO:teuthology.orchestra.run.smithi176.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-14T09:02:23.901 INFO:teuthology.orchestra.run.smithi176.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-14T09:02:23.919 INFO:teuthology.orchestra.run.smithi176.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-14T09:02:23.924 INFO:teuthology.orchestra.run.smithi176.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-14T09:02:23.929 INFO:teuthology.orchestra.run.smithi176.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-14T09:02:24.010 INFO:teuthology.orchestra.run.smithi176.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-14T09:02:24.010 INFO:teuthology.orchestra.run.smithi176.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-14T09:02:24.011 INFO:teuthology.orchestra.run.smithi176.stdout:Get:18 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4775-g7f24e589-1jammy [3,330 kB] 2024-09-14T09:02:24.015 INFO:teuthology.orchestra.run.smithi176.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-14T09:02:24.019 INFO:teuthology.orchestra.run.smithi176.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-14T09:02:24.022 INFO:teuthology.orchestra.run.smithi176.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-14T09:02:24.023 INFO:teuthology.orchestra.run.smithi176.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-14T09:02:24.024 INFO:teuthology.orchestra.run.smithi176.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-14T09:02:24.026 INFO:teuthology.orchestra.run.smithi176.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-14T09:02:24.026 INFO:teuthology.orchestra.run.smithi176.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-14T09:02:24.092 INFO:teuthology.orchestra.run.smithi176.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-14T09:02:24.110 INFO:teuthology.orchestra.run.smithi176.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-14T09:02:24.110 INFO:teuthology.orchestra.run.smithi176.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-14T09:02:24.111 INFO:teuthology.orchestra.run.smithi176.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-14T09:02:24.112 INFO:teuthology.orchestra.run.smithi176.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-14T09:02:24.117 INFO:teuthology.orchestra.run.smithi176.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-14T09:02:24.174 INFO:teuthology.orchestra.run.smithi176.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-14T09:02:24.175 INFO:teuthology.orchestra.run.smithi176.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-14T09:02:24.176 INFO:teuthology.orchestra.run.smithi176.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-14T09:02:24.178 INFO:teuthology.orchestra.run.smithi176.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-14T09:02:24.256 INFO:teuthology.orchestra.run.smithi176.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-14T09:02:24.256 INFO:teuthology.orchestra.run.smithi176.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-14T09:02:24.259 INFO:teuthology.orchestra.run.smithi176.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-14T09:02:24.261 INFO:teuthology.orchestra.run.smithi176.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-14T09:02:24.262 INFO:teuthology.orchestra.run.smithi176.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-14T09:02:24.337 INFO:teuthology.orchestra.run.smithi176.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-14T09:02:24.338 INFO:teuthology.orchestra.run.smithi176.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-14T09:02:24.339 INFO:teuthology.orchestra.run.smithi176.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-14T09:02:24.340 INFO:teuthology.orchestra.run.smithi176.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-14T09:02:24.342 INFO:teuthology.orchestra.run.smithi176.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-14T09:02:24.399 INFO:teuthology.orchestra.run.smithi176.stdout:Get:46 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4775-g7f24e589-1jammy [722 kB] 2024-09-14T09:02:24.418 INFO:teuthology.orchestra.run.smithi176.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-14T09:02:24.420 INFO:teuthology.orchestra.run.smithi176.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-14T09:02:24.423 INFO:teuthology.orchestra.run.smithi176.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-14T09:02:24.425 INFO:teuthology.orchestra.run.smithi176.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-14T09:02:24.429 INFO:teuthology.orchestra.run.smithi176.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-14T09:02:24.500 INFO:teuthology.orchestra.run.smithi176.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-14T09:02:24.501 INFO:teuthology.orchestra.run.smithi176.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-14T09:02:24.506 INFO:teuthology.orchestra.run.smithi176.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-14T09:02:24.527 INFO:teuthology.orchestra.run.smithi176.stdout:Get:55 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4775-g7f24e589-1jammy [356 kB] 2024-09-14T09:02:24.535 INFO:teuthology.orchestra.run.smithi176.stdout:Get:56 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4775-g7f24e589-1jammy [32.8 kB] 2024-09-14T09:02:24.536 INFO:teuthology.orchestra.run.smithi176.stdout:Get:57 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4775-g7f24e589-1jammy [180 kB] 2024-09-14T09:02:24.540 INFO:teuthology.orchestra.run.smithi176.stdout:Get:58 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4775-g7f24e589-1jammy [71.2 kB] 2024-09-14T09:02:24.542 INFO:teuthology.orchestra.run.smithi176.stdout:Get:59 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4775-g7f24e589-1jammy [338 kB] 2024-09-14T09:02:24.556 INFO:teuthology.orchestra.run.smithi176.stdout:Get:60 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4775-g7f24e589-1jammy [7,984 kB] 2024-09-14T09:02:24.669 INFO:teuthology.orchestra.run.smithi183.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4775-g7f24e589-1jammy [3,330 kB] 2024-09-14T09:02:24.671 INFO:teuthology.orchestra.run.smithi176.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-14T09:02:24.673 INFO:teuthology.orchestra.run.smithi176.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-14T09:02:24.676 INFO:teuthology.orchestra.run.smithi176.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-14T09:02:24.677 INFO:teuthology.orchestra.run.smithi176.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-14T09:02:24.679 INFO:teuthology.orchestra.run.smithi176.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-14T09:02:24.714 INFO:teuthology.orchestra.run.smithi176.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-14T09:02:24.747 INFO:teuthology.orchestra.run.smithi176.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-14T09:02:24.747 INFO:teuthology.orchestra.run.smithi176.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-14T09:02:24.777 INFO:teuthology.orchestra.run.smithi176.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-14T09:02:24.777 INFO:teuthology.orchestra.run.smithi176.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-14T09:02:24.838 INFO:teuthology.orchestra.run.smithi176.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-14T09:02:24.839 INFO:teuthology.orchestra.run.smithi176.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-14T09:02:24.840 INFO:teuthology.orchestra.run.smithi176.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-14T09:02:24.841 INFO:teuthology.orchestra.run.smithi176.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-14T09:02:24.842 INFO:teuthology.orchestra.run.smithi176.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-14T09:02:24.843 INFO:teuthology.orchestra.run.smithi176.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-14T09:02:24.850 INFO:teuthology.orchestra.run.smithi176.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-14T09:02:24.920 INFO:teuthology.orchestra.run.smithi176.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-14T09:02:24.923 INFO:teuthology.orchestra.run.smithi176.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-14T09:02:24.925 INFO:teuthology.orchestra.run.smithi176.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-14T09:02:25.001 INFO:teuthology.orchestra.run.smithi176.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-14T09:02:25.008 INFO:teuthology.orchestra.run.smithi176.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-14T09:02:25.010 INFO:teuthology.orchestra.run.smithi176.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-14T09:02:25.012 INFO:teuthology.orchestra.run.smithi176.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-14T09:02:25.023 INFO:teuthology.orchestra.run.smithi176.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-14T09:02:25.024 INFO:teuthology.orchestra.run.smithi176.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-14T09:02:25.032 INFO:teuthology.orchestra.run.smithi176.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-14T09:02:25.082 INFO:teuthology.orchestra.run.smithi176.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-14T09:02:25.084 INFO:teuthology.orchestra.run.smithi176.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-14T09:02:25.085 INFO:teuthology.orchestra.run.smithi176.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-14T09:02:25.139 INFO:teuthology.orchestra.run.smithi183.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4775-g7f24e589-1jammy [722 kB] 2024-09-14T09:02:25.165 INFO:teuthology.orchestra.run.smithi176.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-14T09:02:25.166 INFO:teuthology.orchestra.run.smithi176.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-14T09:02:25.185 INFO:teuthology.orchestra.run.smithi176.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-14T09:02:25.186 INFO:teuthology.orchestra.run.smithi176.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-14T09:02:25.187 INFO:teuthology.orchestra.run.smithi176.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-14T09:02:25.192 INFO:teuthology.orchestra.run.smithi176.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-14T09:02:25.194 INFO:teuthology.orchestra.run.smithi176.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-14T09:02:25.246 INFO:teuthology.orchestra.run.smithi183.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4775-g7f24e589-1jammy [356 kB] 2024-09-14T09:02:25.254 INFO:teuthology.orchestra.run.smithi183.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4775-g7f24e589-1jammy [32.8 kB] 2024-09-14T09:02:25.303 INFO:teuthology.orchestra.run.smithi183.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4775-g7f24e589-1jammy [180 kB] 2024-09-14T09:02:25.316 INFO:teuthology.orchestra.run.smithi183.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4775-g7f24e589-1jammy [71.2 kB] 2024-09-14T09:02:25.318 INFO:teuthology.orchestra.run.smithi183.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4775-g7f24e589-1jammy [338 kB] 2024-09-14T09:02:25.364 INFO:teuthology.orchestra.run.smithi176.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4775-g7f24e589-1jammy [112 kB] 2024-09-14T09:02:25.366 INFO:teuthology.orchestra.run.smithi176.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4775-g7f24e589-1jammy [473 kB] 2024-09-14T09:02:25.375 INFO:teuthology.orchestra.run.smithi183.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4775-g7f24e589-1jammy [7,984 kB] 2024-09-14T09:02:25.431 INFO:teuthology.orchestra.run.smithi176.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4775-g7f24e589-1jammy [28.3 MB] 2024-09-14T09:02:26.186 INFO:teuthology.orchestra.run.smithi183.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4775-g7f24e589-1jammy [112 kB] 2024-09-14T09:02:26.189 INFO:teuthology.orchestra.run.smithi183.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4775-g7f24e589-1jammy [473 kB] 2024-09-14T09:02:26.203 INFO:teuthology.orchestra.run.smithi183.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4775-g7f24e589-1jammy [28.3 MB] 2024-09-14T09:02:28.497 INFO:teuthology.orchestra.run.smithi183.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4775-g7f24e589-1jammy [5,206 kB] 2024-09-14T09:02:28.769 INFO:teuthology.orchestra.run.smithi183.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4775-g7f24e589-1jammy [251 kB] 2024-09-14T09:02:28.781 INFO:teuthology.orchestra.run.smithi183.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4775-g7f24e589-1jammy [124 kB] 2024-09-14T09:02:28.784 INFO:teuthology.orchestra.run.smithi183.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4775-g7f24e589-1jammy [1,561 kB] 2024-09-14T09:02:28.868 INFO:teuthology.orchestra.run.smithi183.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4775-g7f24e589-1jammy [6,311 kB] 2024-09-14T09:02:28.958 INFO:teuthology.orchestra.run.smithi176.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4775-g7f24e589-1jammy [5,206 kB] 2024-09-14T09:02:29.222 INFO:teuthology.orchestra.run.smithi183.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4775-g7f24e589-1jammy [23.1 MB] 2024-09-14T09:02:29.492 INFO:teuthology.orchestra.run.smithi176.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4775-g7f24e589-1jammy [251 kB] 2024-09-14T09:02:29.553 INFO:teuthology.orchestra.run.smithi176.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4775-g7f24e589-1jammy [124 kB] 2024-09-14T09:02:29.555 INFO:teuthology.orchestra.run.smithi176.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4775-g7f24e589-1jammy [1,561 kB] 2024-09-14T09:02:29.756 INFO:teuthology.orchestra.run.smithi176.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4775-g7f24e589-1jammy [6,311 kB] 2024-09-14T09:02:30.319 INFO:teuthology.orchestra.run.smithi176.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4775-g7f24e589-1jammy [23.1 MB] 2024-09-14T09:02:30.671 INFO:teuthology.orchestra.run.smithi183.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4775-g7f24e589-1jammy [14.1 kB] 2024-09-14T09:02:30.672 INFO:teuthology.orchestra.run.smithi183.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4775-g7f24e589-1jammy [880 kB] 2024-09-14T09:02:30.689 INFO:teuthology.orchestra.run.smithi183.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4775-g7f24e589-1jammy [2,452 kB] 2024-09-14T09:02:30.772 INFO:teuthology.orchestra.run.smithi183.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4775-g7f24e589-1jammy [812 kB] 2024-09-14T09:02:30.819 INFO:teuthology.orchestra.run.smithi183.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4775-g7f24e589-1jammy [160 kB] 2024-09-14T09:02:30.822 INFO:teuthology.orchestra.run.smithi183.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4775-g7f24e589-1jammy [61.4 MB] 2024-09-14T09:02:32.211 INFO:teuthology.orchestra.run.smithi176.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4775-g7f24e589-1jammy [14.1 kB] 2024-09-14T09:02:32.211 INFO:teuthology.orchestra.run.smithi176.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4775-g7f24e589-1jammy [880 kB] 2024-09-14T09:02:32.228 INFO:teuthology.orchestra.run.smithi176.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4775-g7f24e589-1jammy [2,452 kB] 2024-09-14T09:02:32.325 INFO:teuthology.orchestra.run.smithi176.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4775-g7f24e589-1jammy [812 kB] 2024-09-14T09:02:32.356 INFO:teuthology.orchestra.run.smithi176.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4775-g7f24e589-1jammy [160 kB] 2024-09-14T09:02:32.360 INFO:teuthology.orchestra.run.smithi176.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4775-g7f24e589-1jammy [61.4 MB] 2024-09-14T09:02:33.085 INFO:teuthology.orchestra.run.smithi183.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4775-g7f24e589-1jammy [8,625 kB] 2024-09-14T09:02:33.221 INFO:teuthology.orchestra.run.smithi183.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4775-g7f24e589-1jammy [14.2 kB] 2024-09-14T09:02:33.221 INFO:teuthology.orchestra.run.smithi183.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4775-g7f24e589-1jammy [55.8 MB] 2024-09-14T09:02:34.897 INFO:teuthology.orchestra.run.smithi183.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4775-g7f24e589-1jammy [130 kB] 2024-09-14T09:02:34.899 INFO:teuthology.orchestra.run.smithi183.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4775-g7f24e589-1jammy [39.0 kB] 2024-09-14T09:02:34.901 INFO:teuthology.orchestra.run.smithi183.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4775-g7f24e589-1jammy [15.9 MB] 2024-09-14T09:02:35.270 INFO:teuthology.orchestra.run.smithi183.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4775-g7f24e589-1jammy [91.3 kB] 2024-09-14T09:02:35.334 INFO:teuthology.orchestra.run.smithi176.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4775-g7f24e589-1jammy [8,625 kB] 2024-09-14T09:02:35.527 INFO:teuthology.orchestra.run.smithi176.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4775-g7f24e589-1jammy [14.2 kB] 2024-09-14T09:02:35.527 INFO:teuthology.orchestra.run.smithi176.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4775-g7f24e589-1jammy [55.8 MB] 2024-09-14T09:02:35.849 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 251 MB in 12s (20.5 MB/s) 2024-09-14T09:02:36.014 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-14T09:02:36.115 INFO:teuthology.orchestra.run.smithi183.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-14T09:02:36.121 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-14T09:02:36.204 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-14T09:02:36.499 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-14T09:02:36.526 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-14T09:02:36.566 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-14T09:02:36.869 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-14T09:02:36.895 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-14T09:02:36.935 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-14T09:02:37.364 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-14T09:02:37.390 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T09:02:37.464 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T09:02:37.570 INFO:teuthology.orchestra.run.smithi176.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4775-g7f24e589-1jammy [130 kB] 2024-09-14T09:02:37.573 INFO:teuthology.orchestra.run.smithi176.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4775-g7f24e589-1jammy [39.0 kB] 2024-09-14T09:02:37.574 INFO:teuthology.orchestra.run.smithi176.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4775-g7f24e589-1jammy [15.9 MB] 2024-09-14T09:02:37.842 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-14T09:02:37.868 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T09:02:37.908 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T09:02:38.039 INFO:teuthology.orchestra.run.smithi176.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4775-g7f24e589-1jammy [91.3 kB] 2024-09-14T09:02:38.237 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-14T09:02:38.263 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T09:02:38.303 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T09:02:38.611 INFO:teuthology.orchestra.run.smithi176.stdout:Fetched 251 MB in 15s (16.6 MB/s) 2024-09-14T09:02:38.664 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-14T09:02:38.690 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-14T09:02:38.731 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-14T09:02:38.758 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-14T09:02:38.858 INFO:teuthology.orchestra.run.smithi176.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-14T09:02:38.863 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-14T09:02:38.948 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-14T09:02:39.059 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package librados2. 2024-09-14T09:02:39.085 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../007-librados2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:39.126 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librados2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:39.293 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-14T09:02:39.320 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-14T09:02:39.360 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-14T09:02:39.595 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libnbd0. 2024-09-14T09:02:39.621 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-14T09:02:39.654 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-14T09:02:39.661 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-14T09:02:39.681 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-14T09:02:39.713 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-14T09:02:39.922 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package librbd1. 2024-09-14T09:02:39.949 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../009-librbd1_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:39.989 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librbd1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:40.149 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-14T09:02:40.176 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T09:02:40.224 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T09:02:40.434 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libcephfs2. 2024-09-14T09:02:40.460 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:40.500 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libcephfs2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:40.627 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-14T09:02:40.654 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T09:02:40.702 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T09:02:40.778 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rados. 2024-09-14T09:02:40.804 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:40.844 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rados (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:41.039 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-14T09:02:41.065 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T09:02:41.097 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T09:02:41.138 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-14T09:02:41.165 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T09:02:41.205 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-ceph-argparse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:41.457 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cephfs. 2024-09-14T09:02:41.472 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:41.641 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cephfs (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:41.648 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-14T09:02:41.675 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-14T09:02:41.815 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-14T09:02:42.136 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package librados2. 2024-09-14T09:02:42.162 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../007-librados2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:42.211 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking librados2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:42.394 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-14T09:02:42.420 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T09:02:42.460 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:42.655 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libnbd0. 2024-09-14T09:02:42.682 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-14T09:02:42.722 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-14T09:02:42.796 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-14T09:02:42.823 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-14T09:02:42.863 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-14T09:02:43.025 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package librbd1. 2024-09-14T09:02:43.051 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../009-librbd1_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:43.091 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking librbd1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:43.158 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-prettytable. 2024-09-14T09:02:43.184 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-14T09:02:43.224 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-14T09:02:43.452 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rbd. 2024-09-14T09:02:43.478 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:43.518 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:43.544 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libcephfs2. 2024-09-14T09:02:43.570 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:43.619 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libcephfs2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:43.854 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-14T09:02:43.881 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-14T09:02:43.913 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-rados. 2024-09-14T09:02:43.921 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-14T09:02:43.939 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:43.980 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-rados (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:44.207 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-14T09:02:44.222 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-14T09:02:44.265 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-14T09:02:44.291 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-14T09:02:44.318 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T09:02:44.358 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-ceph-argparse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:44.551 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-14T09:02:44.566 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-14T09:02:44.601 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-14T09:02:44.610 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-cephfs. 2024-09-14T09:02:44.636 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:44.676 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-cephfs (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:44.896 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package lua5.1. 2024-09-14T09:02:44.922 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-14T09:02:44.962 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-14T09:02:44.963 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-14T09:02:44.989 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T09:02:45.021 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:45.299 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-14T09:02:45.307 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package lua-any. 2024-09-14T09:02:45.326 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-14T09:02:45.333 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-14T09:02:45.357 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-14T09:02:45.374 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-14T09:02:45.618 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package zip. 2024-09-14T09:02:45.633 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-14T09:02:45.676 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking zip (3.0-12build2) ... 2024-09-14T09:02:45.677 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-prettytable. 2024-09-14T09:02:45.703 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-14T09:02:45.743 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-14T09:02:45.954 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package unzip. 2024-09-14T09:02:45.979 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-rbd. 2024-09-14T09:02:45.980 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-14T09:02:45.994 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:46.012 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-14T09:02:46.038 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-rbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:46.390 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-14T09:02:46.416 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package luarocks. 2024-09-14T09:02:46.417 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-14T09:02:46.443 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-14T09:02:46.482 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-14T09:02:46.483 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-14T09:02:46.851 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-14T09:02:46.877 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-14T09:02:46.909 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-14T09:02:47.014 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package librgw2. 2024-09-14T09:02:47.041 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../026-librgw2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:47.124 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librgw2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:47.187 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-14T09:02:47.203 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-14T09:02:47.246 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-14T09:02:47.565 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package lua5.1. 2024-09-14T09:02:47.592 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-14T09:02:47.632 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-14T09:02:47.734 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rgw. 2024-09-14T09:02:47.749 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:47.793 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:47.943 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package lua-any. 2024-09-14T09:02:47.966 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-14T09:02:48.001 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-14T09:02:48.079 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-14T09:02:48.105 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-14T09:02:48.137 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-14T09:02:48.237 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package zip. 2024-09-14T09:02:48.252 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-14T09:02:48.296 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking zip (3.0-12build2) ... 2024-09-14T09:02:48.440 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libradosstriper1. 2024-09-14T09:02:48.466 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:48.515 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libradosstriper1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:48.607 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package unzip. 2024-09-14T09:02:48.634 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-14T09:02:48.673 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-14T09:02:48.968 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-common. 2024-09-14T09:02:48.994 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:49.034 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:49.077 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package luarocks. 2024-09-14T09:02:49.103 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-14T09:02:49.144 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-14T09:02:49.545 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package librgw2. 2024-09-14T09:02:49.569 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../026-librgw2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:49.611 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking librgw2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:50.164 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-rgw. 2024-09-14T09:02:50.179 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:50.222 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-rgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:50.473 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-base. 2024-09-14T09:02:50.499 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:50.533 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-14T09:02:50.541 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-base (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:50.559 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-14T09:02:50.599 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-14T09:02:50.878 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libradosstriper1. 2024-09-14T09:02:50.904 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:50.936 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libradosstriper1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:51.119 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-14T09:02:51.145 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-14T09:02:51.185 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-14T09:02:51.364 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-common. 2024-09-14T09:02:51.391 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:51.430 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:51.513 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cheroot. 2024-09-14T09:02:51.539 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-14T09:02:51.579 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-14T09:02:51.899 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-14T09:02:51.926 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-14T09:02:51.966 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-14T09:02:52.210 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-14T09:02:52.237 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-14T09:02:52.268 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-14T09:02:52.571 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-14T09:02:52.597 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-14T09:02:52.638 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-14T09:02:52.919 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-base. 2024-09-14T09:02:52.945 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:02:52.966 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-tempora. 2024-09-14T09:02:52.992 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-14T09:02:52.995 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-base (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:02:53.032 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-14T09:02:53.310 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-portend. 2024-09-14T09:02:53.337 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-14T09:02:53.368 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-14T09:02:53.548 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-14T09:02:53.575 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-14T09:02:53.615 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-14T09:02:53.638 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-14T09:02:53.664 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-14T09:02:53.704 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-14T09:02:53.859 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-cheroot. 2024-09-14T09:02:53.874 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-14T09:02:53.909 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-14T09:02:54.016 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-14T09:02:54.042 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-14T09:02:54.082 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-14T09:02:54.220 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-14T09:02:54.247 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-14T09:02:54.286 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-14T09:02:54.369 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-natsort. 2024-09-14T09:02:54.395 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-14T09:02:54.427 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-14T09:02:54.606 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-14T09:02:54.633 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-14T09:02:54.673 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-14T09:02:54.721 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-logutils. 2024-09-14T09:02:54.748 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-14T09:02:54.788 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-14T09:02:54.992 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-14T09:02:55.019 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-14T09:02:55.058 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-14T09:02:55.074 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-14T09:02:55.101 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-14T09:02:55.141 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-14T09:02:55.311 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-tempora. 2024-09-14T09:02:55.336 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-14T09:02:55.369 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-14T09:02:55.426 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-mako. 2024-09-14T09:02:55.442 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-14T09:02:55.477 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-14T09:02:55.681 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-portend. 2024-09-14T09:02:55.707 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-14T09:02:55.747 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-14T09:02:55.780 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-14T09:02:55.806 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-14T09:02:55.838 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-14T09:02:56.050 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-14T09:02:56.077 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-14T09:02:56.116 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-14T09:02:56.117 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-14T09:02:56.144 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-14T09:02:56.184 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-14T09:02:57.137 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-14T09:02:57.146 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-webob. 2024-09-14T09:02:57.163 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-14T09:02:57.173 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-14T09:02:57.199 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-14T09:02:57.212 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-14T09:02:57.736 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-natsort. 2024-09-14T09:02:57.763 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-14T09:02:57.765 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-waitress. 2024-09-14T09:02:57.792 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-14T09:02:57.806 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-14T09:02:57.835 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-14T09:02:58.109 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-logutils. 2024-09-14T09:02:58.126 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-tempita. 2024-09-14T09:02:58.136 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-14T09:02:58.153 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-14T09:02:58.176 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-14T09:02:58.193 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-14T09:02:58.420 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-14T09:02:58.436 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-14T09:02:58.470 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-14T09:02:58.495 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-paste. 2024-09-14T09:02:58.522 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-14T09:02:58.562 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-14T09:02:58.773 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-mako. 2024-09-14T09:02:58.800 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-14T09:02:58.840 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-14T09:02:58.873 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-14T09:02:58.889 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-14T09:02:58.931 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-14T09:02:59.168 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-14T09:02:59.194 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-14T09:02:59.234 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-14T09:02:59.234 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-14T09:02:59.261 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-14T09:02:59.301 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-14T09:02:59.537 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-14T09:02:59.564 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-14T09:02:59.570 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-webtest. 2024-09-14T09:02:59.597 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-14T09:02:59.603 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-14T09:02:59.628 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-14T09:02:59.881 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-webob. 2024-09-14T09:02:59.890 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pecan. 2024-09-14T09:02:59.905 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-14T09:02:59.916 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-14T09:02:59.939 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-14T09:02:59.956 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-14T09:03:00.209 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-waitress. 2024-09-14T09:03:00.236 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-14T09:03:00.279 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-14T09:03:00.286 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-certifi. 2024-09-14T09:03:00.313 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-14T09:03:00.352 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-14T09:03:00.603 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-tempita. 2024-09-14T09:03:00.630 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-14T09:03:00.670 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-14T09:03:00.697 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-idna. 2024-09-14T09:03:00.723 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-14T09:03:00.763 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-14T09:03:00.973 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-paste. 2024-09-14T09:03:00.998 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-14T09:03:01.039 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-14T09:03:01.058 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-urllib3. 2024-09-14T09:03:01.084 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-14T09:03:01.124 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-14T09:03:01.343 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-14T09:03:01.358 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-14T09:03:01.401 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-14T09:03:01.453 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-requests. 2024-09-14T09:03:01.479 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-14T09:03:01.519 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-14T09:03:01.703 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-14T09:03:01.730 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-14T09:03:01.770 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-14T09:03:01.797 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-14T09:03:01.823 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-14T09:03:01.855 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-14T09:03:02.073 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-webtest. 2024-09-14T09:03:02.100 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-14T09:03:02.139 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-14T09:03:02.141 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-14T09:03:02.157 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T09:03:02.199 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:02.417 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pecan. 2024-09-14T09:03:02.444 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-14T09:03:02.475 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-14T09:03:02.580 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-14T09:03:02.606 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:02.646 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:02.797 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-certifi. 2024-09-14T09:03:02.823 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-14T09:03:02.855 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-14T09:03:02.974 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-14T09:03:03.000 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-14T09:03:03.040 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-14T09:03:03.166 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-idna. 2024-09-14T09:03:03.193 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-14T09:03:03.233 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-14T09:03:03.318 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-openssl. 2024-09-14T09:03:03.345 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-14T09:03:03.376 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-14T09:03:03.560 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-urllib3. 2024-09-14T09:03:03.587 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-14T09:03:03.627 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-14T09:03:03.680 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr. 2024-09-14T09:03:03.706 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:03.746 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:03.947 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-requests. 2024-09-14T09:03:03.974 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-14T09:03:04.013 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-14T09:03:04.149 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mon. 2024-09-14T09:03:04.175 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:04.215 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mon (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:04.325 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-14T09:03:04.351 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-14T09:03:04.391 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-14T09:03:04.685 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-14T09:03:04.701 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T09:03:04.743 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:04.827 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-osd. 2024-09-14T09:03:04.854 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:04.894 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-osd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:05.099 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-14T09:03:05.126 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:05.165 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:05.484 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-14T09:03:05.511 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-14T09:03:05.551 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-14T09:03:05.853 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-openssl. 2024-09-14T09:03:05.881 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-14T09:03:05.888 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph. 2024-09-14T09:03:05.904 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../067-ceph_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:05.920 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-14T09:03:05.947 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:06.248 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mgr. 2024-09-14T09:03:06.266 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-fuse. 2024-09-14T09:03:06.275 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:06.292 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:06.306 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mgr (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:06.332 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:06.668 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mon. 2024-09-14T09:03:06.694 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:06.710 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mds. 2024-09-14T09:03:06.734 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mon (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:06.737 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:06.780 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mds (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:07.138 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package cephadm. 2024-09-14T09:03:07.153 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../070-cephadm_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:07.187 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:07.338 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-osd. 2024-09-14T09:03:07.364 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:07.404 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-osd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:07.465 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-14T09:03:07.492 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-14T09:03:07.531 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-14T09:03:07.851 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-14T09:03:07.867 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-14T09:03:08.068 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-14T09:03:08.491 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph. 2024-09-14T09:03:08.518 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../067-ceph_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:08.557 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:08.843 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-fuse. 2024-09-14T09:03:08.870 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:08.909 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:09.106 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-babel. 2024-09-14T09:03:09.133 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-14T09:03:09.172 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-14T09:03:09.246 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mds. 2024-09-14T09:03:09.272 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:09.312 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mds (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:09.492 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jinja2. 2024-09-14T09:03:09.519 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-14T09:03:09.558 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-14T09:03:09.698 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package cephadm. 2024-09-14T09:03:09.725 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../070-cephadm_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:09.765 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:09.903 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-14T09:03:09.930 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T09:03:09.970 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:10.076 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-14T09:03:10.102 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-14T09:03:10.142 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-14T09:03:10.290 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-14T09:03:10.317 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-14T09:03:10.356 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-14T09:03:10.436 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-14T09:03:10.463 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-14T09:03:10.503 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-14T09:03:10.659 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-routes. 2024-09-14T09:03:10.686 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-14T09:03:10.734 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-14T09:03:11.070 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-14T09:03:11.097 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T09:03:11.137 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:11.483 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-babel. 2024-09-14T09:03:11.498 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-14T09:03:11.532 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-14T09:03:11.861 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-jinja2. 2024-09-14T09:03:11.888 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-14T09:03:11.918 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-14T09:03:12.213 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-14T09:03:12.240 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T09:03:12.271 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:12.583 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-14T09:03:12.610 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-14T09:03:12.641 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-14T09:03:12.919 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-routes. 2024-09-14T09:03:12.946 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-14T09:03:12.985 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-14T09:03:13.322 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-14T09:03:13.335 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T09:03:13.371 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:13.866 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-14T09:03:13.896 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-14T09:03:13.932 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-14T09:03:14.436 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-joblib. 2024-09-14T09:03:14.464 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-14T09:03:14.503 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-14T09:03:14.881 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-14T09:03:14.909 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-14T09:03:14.947 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-14T09:03:15.250 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-sklearn. 2024-09-14T09:03:15.278 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-14T09:03:15.411 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-14T09:03:16.076 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-14T09:03:16.093 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-14T09:03:16.128 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-14T09:03:16.134 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-14T09:03:16.157 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T09:03:16.195 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:16.621 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-joblib. 2024-09-14T09:03:16.648 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-14T09:03:16.687 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-14T09:03:17.024 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-14T09:03:17.052 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-14T09:03:17.082 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-14T09:03:17.139 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cachetools. 2024-09-14T09:03:17.167 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-14T09:03:17.205 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-14T09:03:17.368 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-sklearn. 2024-09-14T09:03:17.396 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-14T09:03:17.483 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-14T09:03:17.511 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-14T09:03:17.537 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-14T09:03:17.541 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-14T09:03:17.812 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-14T09:03:17.828 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-14T09:03:17.870 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-14T09:03:18.156 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rsa. 2024-09-14T09:03:18.184 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-14T09:03:18.222 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-14T09:03:18.363 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-14T09:03:18.391 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T09:03:18.429 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:18.777 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-google-auth. 2024-09-14T09:03:18.804 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-14T09:03:18.843 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-14T09:03:19.146 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-14T09:03:19.174 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-14T09:03:19.212 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-14T09:03:19.499 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-websocket. 2024-09-14T09:03:19.526 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-14T09:03:19.549 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-cachetools. 2024-09-14T09:03:19.565 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-14T09:03:19.576 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-14T09:03:19.615 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-14T09:03:19.885 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-14T09:03:19.885 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-14T09:03:19.901 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-14T09:03:19.913 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-14T09:03:19.935 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-14T09:03:19.943 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-14T09:03:20.238 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-14T09:03:20.266 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-14T09:03:20.304 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-14T09:03:20.582 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-14T09:03:20.598 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T09:03:20.607 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-rsa. 2024-09-14T09:03:20.635 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-14T09:03:20.640 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:20.674 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-14T09:03:20.976 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-14T09:03:20.994 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-google-auth. 2024-09-14T09:03:21.004 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-14T09:03:21.022 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-14T09:03:21.043 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-14T09:03:21.061 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-14T09:03:21.364 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-14T09:03:21.379 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-14T09:03:21.391 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-14T09:03:21.407 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-14T09:03:21.430 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-14T09:03:21.445 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-14T09:03:21.648 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package jq. 2024-09-14T09:03:21.664 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-14T09:03:21.706 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-14T09:03:21.733 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-websocket. 2024-09-14T09:03:21.761 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-14T09:03:21.791 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-14T09:03:21.959 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package socat. 2024-09-14T09:03:21.987 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-14T09:03:22.025 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-14T09:03:22.078 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-14T09:03:22.106 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-14T09:03:22.144 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-14T09:03:22.337 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package xmlstarlet. 2024-09-14T09:03:22.353 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-14T09:03:22.395 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-14T09:03:22.733 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-14T09:03:22.750 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T09:03:22.766 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-test. 2024-09-14T09:03:22.791 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:22.794 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:22.832 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-test (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:23.077 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-14T09:03:23.104 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-14T09:03:23.152 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-14T09:03:23.489 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-14T09:03:23.516 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-14T09:03:23.555 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-14T09:03:23.808 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package jq. 2024-09-14T09:03:23.835 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-14T09:03:23.874 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-14T09:03:24.101 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package socat. 2024-09-14T09:03:24.118 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-14T09:03:24.160 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-14T09:03:24.438 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package xmlstarlet. 2024-09-14T09:03:24.454 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-14T09:03:24.487 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-14T09:03:24.842 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-test. 2024-09-14T09:03:24.858 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:24.900 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-test (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:25.262 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-volume. 2024-09-14T09:03:25.290 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T09:03:25.328 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-volume (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:25.623 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-14T09:03:25.640 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:25.681 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libcephfs-dev (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:26.010 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-14T09:03:26.038 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-14T09:03:26.076 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-14T09:03:26.414 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-14T09:03:26.442 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-14T09:03:26.480 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-14T09:03:26.851 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package nvme-cli. 2024-09-14T09:03:26.878 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-14T09:03:26.917 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-14T09:03:27.321 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package ceph-volume. 2024-09-14T09:03:27.349 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T09:03:27.352 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package pkg-config. 2024-09-14T09:03:27.379 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking ceph-volume (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:27.380 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-14T09:03:27.422 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-14T09:03:27.666 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-14T09:03:27.682 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:27.683 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-14T09:03:27.699 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-14T09:03:27.716 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking libcephfs-dev (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:27.741 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-14T09:03:28.010 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-14T09:03:28.038 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-14T09:03:28.077 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-14T09:03:28.170 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-14T09:03:28.198 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-14T09:03:28.236 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-14T09:03:28.414 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-14T09:03:28.442 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-14T09:03:28.481 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-14T09:03:28.539 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pastescript. 2024-09-14T09:03:28.567 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-14T09:03:28.605 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-14T09:03:28.877 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package nvme-cli. 2024-09-14T09:03:28.904 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-14T09:03:28.925 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pluggy. 2024-09-14T09:03:28.943 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-14T09:03:28.953 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-14T09:03:28.991 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-14T09:03:29.269 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-psutil. 2024-09-14T09:03:29.285 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-14T09:03:29.319 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-14T09:03:29.323 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package pkg-config. 2024-09-14T09:03:29.351 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-14T09:03:29.381 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-14T09:03:29.628 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-14T09:03:29.639 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-py. 2024-09-14T09:03:29.655 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-14T09:03:29.656 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-14T09:03:29.689 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-14T09:03:29.694 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-14T09:03:29.975 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pygments. 2024-09-14T09:03:29.991 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-14T09:03:30.025 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-14T09:03:30.097 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-14T09:03:30.126 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-14T09:03:30.163 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-14T09:03:30.458 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pastescript. 2024-09-14T09:03:30.462 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-14T09:03:30.486 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-14T09:03:30.491 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-14T09:03:30.524 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-14T09:03:30.529 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-14T09:03:30.815 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-toml. 2024-09-14T09:03:30.819 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pluggy. 2024-09-14T09:03:30.836 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-14T09:03:30.843 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-14T09:03:30.869 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-14T09:03:30.881 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-14T09:03:31.172 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-psutil. 2024-09-14T09:03:31.200 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-14T09:03:31.209 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pytest. 2024-09-14T09:03:31.237 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-14T09:03:31.238 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-14T09:03:31.275 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-14T09:03:31.566 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-py. 2024-09-14T09:03:31.594 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-14T09:03:31.621 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-simplejson. 2024-09-14T09:03:31.632 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-14T09:03:31.649 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-14T09:03:31.687 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-14T09:03:31.936 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pygments. 2024-09-14T09:03:31.956 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-14T09:03:31.964 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-14T09:03:31.972 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-14T09:03:31.994 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-14T09:03:32.014 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-14T09:03:32.440 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-14T09:03:32.468 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-14T09:03:32.506 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-14T09:03:32.744 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package radosgw. 2024-09-14T09:03:32.766 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-toml. 2024-09-14T09:03:32.771 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../117-radosgw_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:32.783 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-14T09:03:32.810 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking radosgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:32.825 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-14T09:03:33.128 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-pytest. 2024-09-14T09:03:33.157 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-14T09:03:33.194 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-14T09:03:33.531 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package python3-simplejson. 2024-09-14T09:03:33.560 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-14T09:03:33.597 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-14T09:03:33.696 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package rbd-fuse. 2024-09-14T09:03:33.712 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:33.746 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking rbd-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:33.875 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-14T09:03:33.903 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-14T09:03:33.941 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-14T09:03:34.054 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-14T09:03:34.476 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-14T09:03:34.602 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-14T09:03:34.863 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-14T09:03:34.879 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package radosgw. 2024-09-14T09:03:34.908 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../117-radosgw_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:34.945 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking radosgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:35.214 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-14T09:03:35.645 INFO:teuthology.orchestra.run.smithi183.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-14T09:03:35.720 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:35.840 INFO:teuthology.orchestra.run.smithi176.stdout:Selecting previously unselected package rbd-fuse. 2024-09-14T09:03:35.856 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T09:03:35.898 INFO:teuthology.orchestra.run.smithi176.stdout:Unpacking rbd-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:36.196 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-14T09:03:36.219 INFO:teuthology.orchestra.run.smithi183.stdout:Adding system user cephadm....done 2024-09-14T09:03:36.368 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-14T09:03:36.453 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-14T09:03:36.562 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-14T09:03:36.636 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-14T09:03:36.804 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-14T09:03:36.887 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-14T09:03:37.013 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-14T09:03:37.149 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-14T09:03:37.264 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-14T09:03:37.515 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-14T09:03:37.562 INFO:teuthology.orchestra.run.smithi176.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-14T09:03:37.616 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-14T09:03:37.632 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:37.928 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-14T09:03:38.122 INFO:teuthology.orchestra.run.smithi176.stdout:Adding system user cephadm....done 2024-09-14T09:03:38.269 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-14T09:03:38.297 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-14T09:03:38.545 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-14T09:03:38.564 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-14T09:03:38.796 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-14T09:03:38.840 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-14T09:03:38.922 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-14T09:03:39.092 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-14T09:03:39.181 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-14T09:03:39.343 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-14T09:03:39.433 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-14T09:03:39.567 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-14T09:03:39.602 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-14T09:03:39.853 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-ceph-argparse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:39.902 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-14T09:03:40.121 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-14T09:03:40.247 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-14T09:03:40.288 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-14T09:03:40.373 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-14T09:03:40.498 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-14T09:03:40.539 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-14T09:03:40.607 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-14T09:03:40.807 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-14T09:03:40.842 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-14T09:03:41.083 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-14T09:03:41.256 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-14T09:03:41.289 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-14T09:03:41.351 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-14T09:03:41.362 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-14T09:03:41.487 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-14T09:03:41.610 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-14T09:03:41.847 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-14T09:03:41.869 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-ceph-argparse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:41.978 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-14T09:03:42.146 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-14T09:03:42.240 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-14T09:03:42.254 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-14T09:03:42.381 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-14T09:03:42.491 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-14T09:03:42.506 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-14T09:03:42.632 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-14T09:03:42.793 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up zip (3.0-12build2) ... 2024-09-14T09:03:42.908 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-14T09:03:42.918 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-14T09:03:43.287 INFO:teuthology.orchestra.run.smithi176.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-14T09:03:43.315 INFO:teuthology.orchestra.run.smithi176.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-14T09:03:43.378 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-14T09:03:43.504 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-14T09:03:43.654 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-14T09:03:43.872 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-14T09:03:43.913 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-14T09:03:44.000 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-14T09:03:44.030 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-14T09:03:44.131 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-14T09:03:44.257 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-14T09:03:44.424 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-14T09:03:44.516 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-14T09:03:44.800 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-14T09:03:44.826 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up zip (3.0-12build2) ... 2024-09-14T09:03:44.952 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-14T09:03:45.051 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-14T09:03:45.436 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-14T09:03:45.621 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-14T09:03:45.729 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-14T09:03:45.863 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-14T09:03:45.989 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-14T09:03:46.072 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-14T09:03:46.098 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-14T09:03:46.323 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-14T09:03:46.383 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-14T09:03:46.432 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:46.733 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-14T09:03:46.759 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-14T09:03:47.010 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-14T09:03:47.059 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-14T09:03:47.378 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-14T09:03:47.654 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-14T09:03:47.989 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-14T09:03:48.206 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-14T09:03:48.240 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-14T09:03:48.349 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:48.425 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T09:03:48.551 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-14T09:03:48.684 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-14T09:03:48.810 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-14T09:03:48.944 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-14T09:03:49.010 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-14T09:03:49.070 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-14T09:03:49.463 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-14T09:03:49.847 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-14T09:03:50.098 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T09:03:50.199 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-14T09:03:50.509 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-14T09:03:50.565 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-14T09:03:50.760 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-14T09:03:50.875 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T09:03:51.010 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-14T09:03:51.036 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-14T09:03:51.285 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-14T09:03:51.287 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-14T09:03:51.411 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-14T09:03:51.537 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-14T09:03:51.538 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-14T09:03:51.797 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-14T09:03:51.830 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-14T09:03:51.898 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-14T09:03:52.073 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-14T09:03:52.149 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-14T09:03:52.258 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-14T09:03:52.324 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T09:03:52.458 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-14T09:03:52.542 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-14T09:03:52.760 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-14T09:03:52.844 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-14T09:03:53.002 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-14T09:03:53.103 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-14T09:03:53.229 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-14T09:03:53.262 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-14T09:03:53.355 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-14T09:03:53.427 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-14T09:03:53.505 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-14T09:03:53.648 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-14T09:03:53.755 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-14T09:03:53.765 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-14T09:03:54.023 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-14T09:03:54.150 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-14T09:03:54.166 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-14T09:03:54.451 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-14T09:03:54.484 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-14T09:03:54.561 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-14T09:03:54.609 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-14T09:03:54.812 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T09:03:54.911 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-14T09:03:54.938 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-14T09:03:55.203 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-14T09:03:55.255 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-14T09:03:55.471 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-14T09:03:55.548 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-14T09:03:55.597 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-14T09:03:55.714 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-14T09:03:55.787 INFO:teuthology.orchestra.run.smithi176.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-14T09:03:55.807 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-14T09:03:55.933 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-14T09:03:56.024 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-14T09:03:56.159 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-14T09:03:56.201 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-14T09:03:56.562 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-14T09:03:56.568 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-14T09:03:56.830 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-14T09:03:56.836 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-14T09:03:56.979 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-14T09:03:57.190 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-14T09:03:57.230 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T09:03:57.356 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-14T09:03:57.474 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-14T09:03:57.665 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-14T09:03:57.793 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-14T09:03:57.911 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librados2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:57.957 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-14T09:03:58.037 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:58.205 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-14T09:03:58.234 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-14T09:03:58.360 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-14T09:03:58.541 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libcephfs2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:58.628 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-14T09:03:58.667 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libradosstriper1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:58.776 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-14T09:03:58.972 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-14T09:03:59.036 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librbd1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:59.162 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-modules-core (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:59.231 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-14T09:03:59.288 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:03:59.501 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-14T09:03:59.501 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-14T09:03:59.609 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-14T09:03:59.927 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-14T09:03:59.955 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libcephfs-dev (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:00.073 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rados (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:00.199 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librgw2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:00.287 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-14T09:04:00.325 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-14T09:04:00.413 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up librados2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:00.539 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:00.665 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-14T09:04:01.001 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libcephfs2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:01.110 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libradosstriper1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:01.220 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-14T09:04:01.471 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up librbd1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:01.537 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:01.597 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mgr-modules-core (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:01.655 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up rbd-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:01.723 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:01.755 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:01.856 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cephfs (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:01.932 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-14T09:04:01.932 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-14T09:04:01.990 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:02.400 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up libcephfs-dev (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:02.508 INFO:teuthology.orchestra.run.smithi183.stdout:Adding group ceph....done 2024-09-14T09:04:02.542 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-rados (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:02.668 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up librgw2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:02.811 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-14T09:04:02.860 INFO:teuthology.orchestra.run.smithi183.stdout:Adding system user ceph....done 2024-09-14T09:04:02.977 INFO:teuthology.orchestra.run.smithi183.stdout:Setting system user ceph properties....done 2024-09-14T09:04:02.994 INFO:teuthology.orchestra.run.smithi183.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-14T09:04:03.132 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-14T09:04:03.494 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-14T09:04:03.941 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-test (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:04.056 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-rbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:04.067 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up radosgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:04.173 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up rbd-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:04.274 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-rgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:04.384 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up python3-cephfs (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:04.510 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:04.534 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-14T09:04:04.534 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-14T09:04:05.027 INFO:teuthology.orchestra.run.smithi176.stdout:Adding group ceph....done 2024-09-14T09:04:05.064 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-base (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:05.413 INFO:teuthology.orchestra.run.smithi176.stdout:Adding system user ceph....done 2024-09-14T09:04:05.554 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-14T09:04:05.580 INFO:teuthology.orchestra.run.smithi176.stdout:Setting system user ceph properties....done 2024-09-14T09:04:05.598 INFO:teuthology.orchestra.run.smithi176.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-14T09:04:05.729 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-14T09:04:06.014 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-14T09:04:06.020 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mds (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:06.247 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-14T09:04:06.247 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-14T09:04:06.715 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-test (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:06.730 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:06.828 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up radosgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:06.957 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-14T09:04:06.957 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-14T09:04:07.310 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-14T09:04:07.310 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-14T09:04:07.440 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-osd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:07.712 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-14T09:04:07.712 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-14T09:04:07.747 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-base (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:08.089 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-14T09:04:08.199 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:08.300 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:08.435 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mon (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:08.532 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mds (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:08.647 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-14T09:04:08.647 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-14T09:04:08.762 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-14T09:04:08.762 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-14T09:04:09.186 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:09.225 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mgr (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:09.338 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:09.441 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-14T09:04:09.441 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-14T09:04:09.455 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-dashboard (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:09.606 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-volume (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:09.885 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-osd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:09.969 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-14T09:04:10.133 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-14T09:04:10.142 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-14T09:04:10.142 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-14T09:04:10.434 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-14T09:04:10.611 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:10.737 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:10.880 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mon (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:11.093 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-14T09:04:11.093 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-14T09:04:11.547 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-14T09:04:11.755 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mgr-cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:11.886 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:12.012 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-mgr-dashboard (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:12.155 INFO:teuthology.orchestra.run.smithi176.stdout:Setting up ceph-volume (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T09:04:12.212 INFO:teuthology.orchestra.run.smithi183.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-14T09:04:12.217 DEBUG:teuthology.parallel:result is None 2024-09-14T09:04:12.491 INFO:teuthology.orchestra.run.smithi176.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-14T09:04:12.600 INFO:teuthology.orchestra.run.smithi176.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-14T09:04:12.744 INFO:teuthology.orchestra.run.smithi176.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-14T09:04:13.794 INFO:teuthology.orchestra.run.smithi176.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-14T09:04:14.434 INFO:teuthology.orchestra.run.smithi176.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-14T09:04:14.439 DEBUG:teuthology.parallel:result is None 2024-09-14T09:04:14.440 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T09:04:14.586 DEBUG:teuthology.orchestra.run.smithi176:> dpkg-query -W -f '${Version}' ceph 2024-09-14T09:04:14.605 INFO:teuthology.orchestra.run.smithi176.stdout:19.3.0-4775-g7f24e589-1jammy 2024-09-14T09:04:14.606 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4775-g7f24e589-1jammy 2024-09-14T09:04:14.606 INFO:teuthology.task.install:The correct ceph version 19.3.0-4775-g7f24e589-1jammy is installed. 2024-09-14T09:04:14.608 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T09:04:14.747 DEBUG:teuthology.orchestra.run.smithi183:> dpkg-query -W -f '${Version}' ceph 2024-09-14T09:04:14.767 INFO:teuthology.orchestra.run.smithi183.stdout:19.3.0-4775-g7f24e589-1jammy 2024-09-14T09:04:14.767 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4775-g7f24e589-1jammy 2024-09-14T09:04:14.767 INFO:teuthology.task.install:The correct ceph version 19.3.0-4775-g7f24e589-1jammy is installed. 2024-09-14T09:04:14.769 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-14T09:04:14.770 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:14.770 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-14T09:04:14.784 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T09:04:14.784 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-14T09:04:14.823 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-14T09:04:14.823 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:14.823 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/daemon-helper 2024-09-14T09:04:14.838 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-14T09:04:14.893 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T09:04:14.893 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2024-09-14T09:04:14.907 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-14T09:04:14.961 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-14T09:04:14.961 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:14.961 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-14T09:04:14.973 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-14T09:04:15.029 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T09:04:15.029 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-14T09:04:15.041 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-14T09:04:15.093 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-14T09:04:15.093 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:15.093 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/stdin-killer 2024-09-14T09:04:15.105 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-14T09:04:15.158 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T09:04:15.158 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2024-09-14T09:04:15.171 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-14T09:04:15.228 INFO:teuthology.run_tasks:Running task ceph... 2024-09-14T09:04:15.314 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-14T09:04:15.314 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 777 /var/log/ceph 2024-09-14T09:04:15.317 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /var/log/ceph 2024-09-14T09:04:15.330 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-14T09:04:15.330 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-14T09:04:15.372 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-14T09:04:15.385 INFO:tasks.ceph:Creating extra log directories... 2024-09-14T09:04:15.385 DEBUG:teuthology.orchestra.run.smithi176:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-14T09:04:15.427 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-14T09:04:15.448 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-14T09:04:15.448 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-14T09:04:15.449 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900516', 'branch': 'main', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7900516', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '0d61375b8ab53be4c233f5b93e463edfdff9316a'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 950, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 5839, 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47', 'sleep_before_teardown': 0, 'subset': '99/128', 'suite': 'rbd', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '0d61375b8ab53be4c233f5b93e463edfdff9316a', 'targets': {'smithi176.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG/FuUsyZj86EDcDVtrA1orDqBDaZm2QCBWRkVHcNEvORvCxfxG0ycmhBn2Fmh2TlyHPnVX4JDAJ05edUSNNPYc=', 'smithi183.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGRfq3k86/F4iBQ1BMHZ6PIWHgqGKHcp4eiaUgIYQs2l5tRKOQTUseIftJplwxSNK1vtmsRIos9DyrgwiZo/bn0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_20:16:17', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-14T09:04:15.449 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-14T09:04:15.484 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-14T09:04:15.491 DEBUG:teuthology.orchestra.run.smithi176:> sudo install -d -m0777 -- /var/run/ceph 2024-09-14T09:04:15.531 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m0777 -- /var/run/ceph 2024-09-14T09:04:15.546 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:15.547 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/scratch_devs of=/dev/stdout 2024-09-14T09:04:15.590 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-14T09:04:15.591 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_1 2024-09-14T09:04:15.639 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-14T09:04:15.639 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T09:04:15.639 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 888 Links: 1 2024-09-14T09:04:15.639 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T09:04:15.639 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-09-14 08:58:42.952016262 +0000 2024-09-14T09:04:15.639 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-09-14 08:58:42.684018656 +0000 2024-09-14T09:04:15.639 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-09-14 08:58:42.684018656 +0000 2024-09-14T09:04:15.639 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: - 2024-09-14T09:04:15.639 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-14T09:04:15.693 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-09-14T09:04:15.693 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-09-14T09:04:15.693 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000340916 s, 1.5 MB/s 2024-09-14T09:04:15.694 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-14T09:04:15.744 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_2 2024-09-14T09:04:15.790 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-14T09:04:15.791 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T09:04:15.791 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 901 Links: 1 2024-09-14T09:04:15.791 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T09:04:15.791 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-09-14 08:58:43.420012082 +0000 2024-09-14T09:04:15.791 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-09-14 08:58:43.180014226 +0000 2024-09-14T09:04:15.791 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-09-14 08:58:43.180014226 +0000 2024-09-14T09:04:15.791 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: - 2024-09-14T09:04:15.791 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-14T09:04:15.844 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-09-14T09:04:15.845 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-09-14T09:04:15.845 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.00041502 s, 1.2 MB/s 2024-09-14T09:04:15.846 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-14T09:04:15.897 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_3 2024-09-14T09:04:15.943 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-14T09:04:15.944 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T09:04:15.944 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 922 Links: 1 2024-09-14T09:04:15.944 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T09:04:15.944 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-09-14 08:58:43.916007651 +0000 2024-09-14T09:04:15.944 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-09-14 08:58:43.648010045 +0000 2024-09-14T09:04:15.944 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-09-14 08:58:43.648010045 +0000 2024-09-14T09:04:15.945 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: - 2024-09-14T09:04:15.945 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-14T09:04:15.997 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-09-14T09:04:15.997 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-09-14T09:04:15.997 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000327288 s, 1.6 MB/s 2024-09-14T09:04:15.998 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-14T09:04:16.048 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_4 2024-09-14T09:04:16.094 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-14T09:04:16.095 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T09:04:16.095 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 939 Links: 1 2024-09-14T09:04:16.095 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T09:04:16.095 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-09-14 08:58:44.404003291 +0000 2024-09-14T09:04:16.095 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-09-14 08:58:44.124005793 +0000 2024-09-14T09:04:16.095 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-09-14 08:58:44.124005793 +0000 2024-09-14T09:04:16.095 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: - 2024-09-14T09:04:16.095 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-14T09:04:16.148 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-09-14T09:04:16.149 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-09-14T09:04:16.149 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000292202 s, 1.8 MB/s 2024-09-14T09:04:16.150 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-14T09:04:16.199 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T09:04:16.200 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T09:04:16.200 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-09-14T09:04:16.206 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-14T09:04:16.206 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-09-14T09:04:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-14T09:04:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T09:04:16.257 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-09-14T09:04:16.257 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T09:04:16.257 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-14 08:58:45.100673982 +0000 2024-09-14T09:04:16.257 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-14 08:58:44.848676220 +0000 2024-09-14T09:04:16.257 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-14 08:58:44.848676220 +0000 2024-09-14T09:04:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2024-09-14T09:04:16.257 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-14T09:04:16.309 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-14T09:04:16.309 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-14T09:04:16.309 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000390642 s, 1.3 MB/s 2024-09-14T09:04:16.311 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-14T09:04:16.359 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-09-14T09:04:16.410 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-14T09:04:16.410 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T09:04:16.410 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 899 Links: 1 2024-09-14T09:04:16.410 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T09:04:16.410 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-14 08:58:45.600669542 +0000 2024-09-14T09:04:16.411 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-14 08:58:45.332671923 +0000 2024-09-14T09:04:16.411 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-14 08:58:45.332671923 +0000 2024-09-14T09:04:16.411 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2024-09-14T09:04:16.411 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-14T09:04:16.464 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-14T09:04:16.464 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-14T09:04:16.465 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000366808 s, 1.4 MB/s 2024-09-14T09:04:16.466 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-14T09:04:16.515 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-09-14T09:04:16.562 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-14T09:04:16.563 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T09:04:16.563 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 915 Links: 1 2024-09-14T09:04:16.563 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T09:04:16.563 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-14 08:58:46.092665175 +0000 2024-09-14T09:04:16.563 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-14 08:58:45.840667413 +0000 2024-09-14T09:04:16.563 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-14 08:58:45.840667413 +0000 2024-09-14T09:04:16.563 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2024-09-14T09:04:16.563 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-14T09:04:16.616 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-14T09:04:16.616 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-14T09:04:16.617 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000389246 s, 1.3 MB/s 2024-09-14T09:04:16.618 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-14T09:04:16.671 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-09-14T09:04:16.718 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-14T09:04:16.719 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T09:04:16.720 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 939 Links: 1 2024-09-14T09:04:16.720 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T09:04:16.720 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-14 08:58:46.592660736 +0000 2024-09-14T09:04:16.720 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-14 08:58:46.328663080 +0000 2024-09-14T09:04:16.720 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-14 08:58:46.328663080 +0000 2024-09-14T09:04:16.720 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2024-09-14T09:04:16.720 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-14T09:04:16.772 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-14T09:04:16.773 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-14T09:04:16.773 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000361848 s, 1.4 MB/s 2024-09-14T09:04:16.774 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-14T09:04:16.828 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T09:04:16.828 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi176.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi183.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-14T09:04:16.828 INFO:tasks.ceph:Generating config... 2024-09-14T09:04:16.830 INFO:tasks.ceph:[client] rbd cache = True 2024-09-14T09:04:16.830 INFO:tasks.ceph:[client] rbd cache policy = writearound 2024-09-14T09:04:16.830 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-14T09:04:16.830 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-14T09:04:16.830 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-14T09:04:16.830 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-14T09:04:16.830 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-14T09:04:16.830 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-14T09:04:16.830 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-14T09:04:16.830 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-14T09:04:16.830 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-14T09:04:16.830 INFO:tasks.ceph:[osd] bluestore compression algorithm = zlib 2024-09-14T09:04:16.830 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-14T09:04:16.831 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-14T09:04:16.831 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-14T09:04:16.831 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-14T09:04:16.831 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-14T09:04:16.831 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-14T09:04:16.831 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-14T09:04:16.831 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-14T09:04:16.831 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-14T09:04:16.831 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-14T09:04:16.831 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-14T09:04:16.831 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-14T09:04:16.831 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-14T09:04:16.831 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-14T09:04:16.831 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-14T09:04:16.832 INFO:tasks.ceph:Setting up mon.a... 2024-09-14T09:04:16.832 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-14T09:04:16.867 INFO:teuthology.orchestra.run.smithi176.stdout:creating /etc/ceph/ceph.keyring 2024-09-14T09:04:16.878 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-14T09:04:16.957 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T09:04:17.010 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.176'), ('mon.c', '[v2:172.21.15.176:3301,v1:172.21.15.176:6790]'), ('mon.b', '172.21.15.183')] 2024-09-14T09:04:17.010 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.176,[v2:172.21.15.176:3301,v1:172.21.15.176:6790],172.21.15.183', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache policy': 'writearound'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-14T09:04:17.011 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:17.011 DEBUG:teuthology.orchestra.run.smithi176:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-14T09:04:17.059 DEBUG:teuthology.orchestra.run.smithi176:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.176 --addv c '[v2:172.21.15.176:3301,v1:172.21.15.176:6790]' --add b 172.21.15.183 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-14T09:04:17.144 INFO:teuthology.orchestra.run.smithi176.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-14T09:04:17.144 INFO:teuthology.orchestra.run.smithi176.stdout:monmaptool: generated fsid c635667d-994e-454f-bcea-51b45637b2a2 2024-09-14T09:04:17.144 INFO:teuthology.orchestra.run.smithi176.stdout:setting min_mon_release = quincy 2024-09-14T09:04:17.144 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 0 2024-09-14T09:04:17.144 INFO:teuthology.orchestra.run.smithi176.stdout:fsid c635667d-994e-454f-bcea-51b45637b2a2 2024-09-14T09:04:17.145 INFO:teuthology.orchestra.run.smithi176.stdout:last_changed 2024-09-14T09:04:17.146078+0000 2024-09-14T09:04:17.145 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-09-14T09:04:17.146078+0000 2024-09-14T09:04:17.145 INFO:teuthology.orchestra.run.smithi176.stdout:min_mon_release 17 (quincy) 2024-09-14T09:04:17.145 INFO:teuthology.orchestra.run.smithi176.stdout:election_strategy: 1 2024-09-14T09:04:17.145 INFO:teuthology.orchestra.run.smithi176.stdout:0: [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon.a 2024-09-14T09:04:17.145 INFO:teuthology.orchestra.run.smithi176.stdout:1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-09-14T09:04:17.145 INFO:teuthology.orchestra.run.smithi176.stdout:2: [v2:172.21.15.176:3301/0,v1:172.21.15.176:6790/0] mon.c 2024-09-14T09:04:17.145 INFO:teuthology.orchestra.run.smithi176.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-14T09:04:17.146 DEBUG:teuthology.orchestra.run.smithi176:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-14T09:04:17.194 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c635667d-994e-454f-bcea-51b45637b2a2... 2024-09-14T09:04:17.195 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-14T09:04:17.240 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-14T09:04:17.268 INFO:teuthology.orchestra.run.smithi176.stdout:[global] 2024-09-14T09:04:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: chdir = "" 2024-09-14T09:04:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-14T09:04:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: auth supported = cephx 2024-09-14T09:04:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: filestore xattr use omap = true 2024-09-14T09:04:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: mon clock drift allowed = 1.000 2024-09-14T09:04:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.268 INFO:teuthology.orchestra.run.smithi176.stdout: osd crush chooseleaf type = 0 2024-09-14T09:04:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: auth debug = true 2024-09-14T09:04:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: ms die on old message = true 2024-09-14T09:04:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: ms die on bug = true 2024-09-14T09:04:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-14T09:04:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: mon pg warn max object skew = 0 2024-09-14T09:04:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: # disable pg_autoscaler by default for new pools 2024-09-14T09:04:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-14T09:04:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: osd pool default size = 2 2024-09-14T09:04:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.269 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd allow primary affinity = true 2024-09-14T09:04:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd allow pg remap = true 2024-09-14T09:04:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: mon warn on legacy crush tunables = false 2024-09-14T09:04:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: mon warn on crush straw calc version zero = false 2024-09-14T09:04:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: mon warn on no sortbitwise = false 2024-09-14T09:04:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: mon warn on osd down out interval zero = false 2024-09-14T09:04:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: mon warn on too few osds = false 2024-09-14T09:04:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-14T09:04:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-14T09:04:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: mon_allow_pool_size_one = true 2024-09-14T09:04:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-14T09:04:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: osd default data pool replay window = 5 2024-09-14T09:04:17.270 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: mon allow pool delete = true 2024-09-14T09:04:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: mon cluster log file level = debug 2024-09-14T09:04:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: debug asserts on shutdown = true 2024-09-14T09:04:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: mon health detail to clog = false 2024-09-14T09:04:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: mon host = "172.21.15.176,[v2:172.21.15.176:3301,v1:172.21.15.176:6790],172.21.15.183" 2024-09-14T09:04:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: mon client directed command retry = 5 2024-09-14T09:04:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: mon warn on pool no app = False 2024-09-14T09:04:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: ms inject socket failures = 5000 2024-09-14T09:04:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: fsid = c635667d-994e-454f-bcea-51b45637b2a2 2024-09-14T09:04:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.271 INFO:teuthology.orchestra.run.smithi176.stdout:[osd] 2024-09-14T09:04:17.271 INFO:teuthology.orchestra.run.smithi176.stdout: osd journal size = 100 2024-09-14T09:04:17.273 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.273 INFO:teuthology.orchestra.run.smithi176.stdout: osd scrub load threshold = 5.0 2024-09-14T09:04:17.273 INFO:teuthology.orchestra.run.smithi176.stdout: osd scrub max interval = 600 2024-09-14T09:04:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: osd mclock profile = high_recovery_ops 2024-09-14T09:04:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: osd recover clone overlap = true 2024-09-14T09:04:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: osd recovery max chunk = 1048576 2024-09-14T09:04:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: osd debug shutdown = true 2024-09-14T09:04:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: osd debug op order = true 2024-09-14T09:04:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: osd debug verify stray on activate = true 2024-09-14T09:04:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: osd debug trim objects = true 2024-09-14T09:04:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: osd open classes on start = true 2024-09-14T09:04:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: osd debug pg log writeout = true 2024-09-14T09:04:17.274 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: osd deep scrub update digest min age = 30 2024-09-14T09:04:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: osd map max advance = 10 2024-09-14T09:04:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: journal zero on create = true 2024-09-14T09:04:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: filestore ondisk finisher threads = 3 2024-09-14T09:04:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: filestore apply finisher threads = 3 2024-09-14T09:04:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: bdev debug aio = true 2024-09-14T09:04:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: osd debug misdirected ops = true 2024-09-14T09:04:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: bluestore block size = 96636764160 2024-09-14T09:04:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: bluestore compression algorithm = zlib 2024-09-14T09:04:17.275 INFO:teuthology.orchestra.run.smithi176.stdout: bluestore compression mode = aggressive 2024-09-14T09:04:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: bluestore fsck on mount = True 2024-09-14T09:04:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: debug bluefs = 1/20 2024-09-14T09:04:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: debug bluestore = 1/20 2024-09-14T09:04:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: debug ms = 1 2024-09-14T09:04:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: debug osd = 20 2024-09-14T09:04:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: debug rocksdb = 4/10 2024-09-14T09:04:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd backfillfull_ratio = 0.85 2024-09-14T09:04:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd full ratio = 0.9 2024-09-14T09:04:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd nearfull ratio = 0.8 2024-09-14T09:04:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: osd blocked scrub grace period = 3600 2024-09-14T09:04:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: osd failsafe full ratio = 0.95 2024-09-14T09:04:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: osd max markdown count = 1000 2024-09-14T09:04:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: osd objectstore = bluestore 2024-09-14T09:04:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: osd shutdown pgref assert = True 2024-09-14T09:04:17.276 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.277 INFO:teuthology.orchestra.run.smithi176.stdout:[mgr] 2024-09-14T09:04:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: debug ms = 1 2024-09-14T09:04:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: debug mgr = 20 2024-09-14T09:04:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: debug mon = 20 2024-09-14T09:04:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: debug auth = 20 2024-09-14T09:04:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: mon reweight min pgs per osd = 4 2024-09-14T09:04:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: mon reweight min bytes per osd = 10 2024-09-14T09:04:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: mgr/telemetry/nag = false 2024-09-14T09:04:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.277 INFO:teuthology.orchestra.run.smithi176.stdout:[mon] 2024-09-14T09:04:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: debug ms = 1 2024-09-14T09:04:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: debug mon = 20 2024-09-14T09:04:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: debug paxos = 20 2024-09-14T09:04:17.277 INFO:teuthology.orchestra.run.smithi176.stdout: debug auth = 20 2024-09-14T09:04:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: mon data avail warn = 5 2024-09-14T09:04:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: mon mgr mkfs grace = 240 2024-09-14T09:04:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: mon reweight min pgs per osd = 4 2024-09-14T09:04:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd reporter subtree level = osd 2024-09-14T09:04:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: mon osd prime pg temp = true 2024-09-14T09:04:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: mon reweight min bytes per osd = 10 2024-09-14T09:04:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-14T09:04:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: auth mon ticket ttl = 660 # 11m 2024-09-14T09:04:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: auth service ticket ttl = 240 # 4m 2024-09-14T09:04:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: # don't complain about insecure global_id in the test suite 2024-09-14T09:04:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-14T09:04:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-14T09:04:17.278 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: # 1m isn't quite enough 2024-09-14T09:04:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: mon_down_mkfs_grace = 2m 2024-09-14T09:04:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: mon_warn_on_filestore_osds = false 2024-09-14T09:04:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:17.279 INFO:teuthology.orchestra.run.smithi176.stdout:[client] 2024-09-14T09:04:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: rgw cache enabled = true 2024-09-14T09:04:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: rgw enable ops log = true 2024-09-14T09:04:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: rgw enable usage log = true 2024-09-14T09:04:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-14T09:04:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-14T09:04:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: rbd cache = True 2024-09-14T09:04:17.279 INFO:teuthology.orchestra.run.smithi176.stdout: rbd cache policy = writearound 2024-09-14T09:04:17.279 INFO:teuthology.orchestra.run.smithi176.stdout:[mon.a] 2024-09-14T09:04:17.279 INFO:teuthology.orchestra.run.smithi176.stdout:[mon.c] 2024-09-14T09:04:17.280 INFO:teuthology.orchestra.run.smithi176.stdout:[mon.b] 2024-09-14T09:04:17.281 INFO:teuthology.orchestra.run.smithi183.stdout:[global] 2024-09-14T09:04:17.281 INFO:teuthology.orchestra.run.smithi183.stdout: chdir = "" 2024-09-14T09:04:17.281 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-14T09:04:17.281 INFO:teuthology.orchestra.run.smithi183.stdout: auth supported = cephx 2024-09-14T09:04:17.282 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.282 INFO:teuthology.orchestra.run.smithi183.stdout: filestore xattr use omap = true 2024-09-14T09:04:17.282 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.282 INFO:teuthology.orchestra.run.smithi183.stdout: mon clock drift allowed = 1.000 2024-09-14T09:04:17.282 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.282 INFO:teuthology.orchestra.run.smithi183.stdout: osd crush chooseleaf type = 0 2024-09-14T09:04:17.282 INFO:teuthology.orchestra.run.smithi183.stdout: auth debug = true 2024-09-14T09:04:17.282 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.282 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on old message = true 2024-09-14T09:04:17.282 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on bug = true 2024-09-14T09:04:17.282 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.282 INFO:teuthology.orchestra.run.smithi183.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-14T09:04:17.282 INFO:teuthology.orchestra.run.smithi183.stdout: mon pg warn max object skew = 0 2024-09-14T09:04:17.282 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.283 INFO:teuthology.orchestra.run.smithi183.stdout: # disable pg_autoscaler by default for new pools 2024-09-14T09:04:17.283 INFO:teuthology.orchestra.run.smithi183.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-14T09:04:17.283 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.283 INFO:teuthology.orchestra.run.smithi183.stdout: osd pool default size = 2 2024-09-14T09:04:17.283 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.283 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd allow primary affinity = true 2024-09-14T09:04:17.283 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd allow pg remap = true 2024-09-14T09:04:17.283 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on legacy crush tunables = false 2024-09-14T09:04:17.283 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on crush straw calc version zero = false 2024-09-14T09:04:17.283 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on no sortbitwise = false 2024-09-14T09:04:17.283 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on osd down out interval zero = false 2024-09-14T09:04:17.283 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on too few osds = false 2024-09-14T09:04:17.283 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-14T09:04:17.284 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-14T09:04:17.284 INFO:teuthology.orchestra.run.smithi183.stdout: mon_allow_pool_size_one = true 2024-09-14T09:04:17.284 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.284 INFO:teuthology.orchestra.run.smithi183.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-14T09:04:17.284 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.284 INFO:teuthology.orchestra.run.smithi183.stdout: osd default data pool replay window = 5 2024-09-14T09:04:17.284 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.284 INFO:teuthology.orchestra.run.smithi183.stdout: mon allow pool delete = true 2024-09-14T09:04:17.284 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.284 INFO:teuthology.orchestra.run.smithi183.stdout: mon cluster log file level = debug 2024-09-14T09:04:17.284 INFO:teuthology.orchestra.run.smithi183.stdout: debug asserts on shutdown = true 2024-09-14T09:04:17.284 INFO:teuthology.orchestra.run.smithi183.stdout: mon health detail to clog = false 2024-09-14T09:04:17.284 INFO:teuthology.orchestra.run.smithi183.stdout: mon host = "172.21.15.176,[v2:172.21.15.176:3301,v1:172.21.15.176:6790],172.21.15.183" 2024-09-14T09:04:17.285 INFO:teuthology.orchestra.run.smithi183.stdout: mon client directed command retry = 5 2024-09-14T09:04:17.285 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on pool no app = False 2024-09-14T09:04:17.285 INFO:teuthology.orchestra.run.smithi183.stdout: ms inject socket failures = 5000 2024-09-14T09:04:17.285 INFO:teuthology.orchestra.run.smithi183.stdout: fsid = c635667d-994e-454f-bcea-51b45637b2a2 2024-09-14T09:04:17.285 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.285 INFO:teuthology.orchestra.run.smithi183.stdout:[osd] 2024-09-14T09:04:17.285 INFO:teuthology.orchestra.run.smithi183.stdout: osd journal size = 100 2024-09-14T09:04:17.285 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.285 INFO:teuthology.orchestra.run.smithi183.stdout: osd scrub load threshold = 5.0 2024-09-14T09:04:17.285 INFO:teuthology.orchestra.run.smithi183.stdout: osd scrub max interval = 600 2024-09-14T09:04:17.285 INFO:teuthology.orchestra.run.smithi183.stdout: osd mclock profile = high_recovery_ops 2024-09-14T09:04:17.285 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.285 INFO:teuthology.orchestra.run.smithi183.stdout: osd recover clone overlap = true 2024-09-14T09:04:17.285 INFO:teuthology.orchestra.run.smithi183.stdout: osd recovery max chunk = 1048576 2024-09-14T09:04:17.286 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.286 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug shutdown = true 2024-09-14T09:04:17.286 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug op order = true 2024-09-14T09:04:17.286 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug verify stray on activate = true 2024-09-14T09:04:17.286 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug trim objects = true 2024-09-14T09:04:17.286 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.286 INFO:teuthology.orchestra.run.smithi183.stdout: osd open classes on start = true 2024-09-14T09:04:17.286 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug pg log writeout = true 2024-09-14T09:04:17.286 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.286 INFO:teuthology.orchestra.run.smithi183.stdout: osd deep scrub update digest min age = 30 2024-09-14T09:04:17.286 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.286 INFO:teuthology.orchestra.run.smithi183.stdout: osd map max advance = 10 2024-09-14T09:04:17.286 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.286 INFO:teuthology.orchestra.run.smithi183.stdout: journal zero on create = true 2024-09-14T09:04:17.287 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.287 INFO:teuthology.orchestra.run.smithi183.stdout: filestore ondisk finisher threads = 3 2024-09-14T09:04:17.287 INFO:teuthology.orchestra.run.smithi183.stdout: filestore apply finisher threads = 3 2024-09-14T09:04:17.287 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.287 INFO:teuthology.orchestra.run.smithi183.stdout: bdev debug aio = true 2024-09-14T09:04:17.287 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug misdirected ops = true 2024-09-14T09:04:17.287 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore block size = 96636764160 2024-09-14T09:04:17.287 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore compression algorithm = zlib 2024-09-14T09:04:17.287 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore compression mode = aggressive 2024-09-14T09:04:17.287 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore fsck on mount = True 2024-09-14T09:04:17.287 INFO:teuthology.orchestra.run.smithi183.stdout: debug bluefs = 1/20 2024-09-14T09:04:17.287 INFO:teuthology.orchestra.run.smithi183.stdout: debug bluestore = 1/20 2024-09-14T09:04:17.287 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2024-09-14T09:04:17.287 INFO:teuthology.orchestra.run.smithi183.stdout: debug osd = 20 2024-09-14T09:04:17.288 INFO:teuthology.orchestra.run.smithi183.stdout: debug rocksdb = 4/10 2024-09-14T09:04:17.288 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd backfillfull_ratio = 0.85 2024-09-14T09:04:17.288 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd full ratio = 0.9 2024-09-14T09:04:17.288 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd nearfull ratio = 0.8 2024-09-14T09:04:17.288 INFO:teuthology.orchestra.run.smithi183.stdout: osd blocked scrub grace period = 3600 2024-09-14T09:04:17.288 INFO:teuthology.orchestra.run.smithi183.stdout: osd failsafe full ratio = 0.95 2024-09-14T09:04:17.288 INFO:teuthology.orchestra.run.smithi183.stdout: osd max markdown count = 1000 2024-09-14T09:04:17.288 INFO:teuthology.orchestra.run.smithi183.stdout: osd objectstore = bluestore 2024-09-14T09:04:17.288 INFO:teuthology.orchestra.run.smithi183.stdout: osd shutdown pgref assert = True 2024-09-14T09:04:17.288 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.288 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr] 2024-09-14T09:04:17.288 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2024-09-14T09:04:17.288 INFO:teuthology.orchestra.run.smithi183.stdout: debug mgr = 20 2024-09-14T09:04:17.289 INFO:teuthology.orchestra.run.smithi183.stdout: debug mon = 20 2024-09-14T09:04:17.289 INFO:teuthology.orchestra.run.smithi183.stdout: debug auth = 20 2024-09-14T09:04:17.289 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min pgs per osd = 4 2024-09-14T09:04:17.289 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min bytes per osd = 10 2024-09-14T09:04:17.289 INFO:teuthology.orchestra.run.smithi183.stdout: mgr/telemetry/nag = false 2024-09-14T09:04:17.289 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.289 INFO:teuthology.orchestra.run.smithi183.stdout:[mon] 2024-09-14T09:04:17.289 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2024-09-14T09:04:17.289 INFO:teuthology.orchestra.run.smithi183.stdout: debug mon = 20 2024-09-14T09:04:17.289 INFO:teuthology.orchestra.run.smithi183.stdout: debug paxos = 20 2024-09-14T09:04:17.289 INFO:teuthology.orchestra.run.smithi183.stdout: debug auth = 20 2024-09-14T09:04:17.289 INFO:teuthology.orchestra.run.smithi183.stdout: mon data avail warn = 5 2024-09-14T09:04:17.289 INFO:teuthology.orchestra.run.smithi183.stdout: mon mgr mkfs grace = 240 2024-09-14T09:04:17.289 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min pgs per osd = 4 2024-09-14T09:04:17.290 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd reporter subtree level = osd 2024-09-14T09:04:17.290 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd prime pg temp = true 2024-09-14T09:04:17.290 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min bytes per osd = 10 2024-09-14T09:04:17.290 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.290 INFO:teuthology.orchestra.run.smithi183.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-14T09:04:17.290 INFO:teuthology.orchestra.run.smithi183.stdout: auth mon ticket ttl = 660 # 11m 2024-09-14T09:04:17.290 INFO:teuthology.orchestra.run.smithi183.stdout: auth service ticket ttl = 240 # 4m 2024-09-14T09:04:17.290 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.290 INFO:teuthology.orchestra.run.smithi183.stdout: # don't complain about insecure global_id in the test suite 2024-09-14T09:04:17.290 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-14T09:04:17.290 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-14T09:04:17.290 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.290 INFO:teuthology.orchestra.run.smithi183.stdout: # 1m isn't quite enough 2024-09-14T09:04:17.290 INFO:teuthology.orchestra.run.smithi183.stdout: mon_down_mkfs_grace = 2m 2024-09-14T09:04:17.291 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.291 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_filestore_osds = false 2024-09-14T09:04:17.291 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-14T09:04:17.291 INFO:teuthology.orchestra.run.smithi183.stdout:[client] 2024-09-14T09:04:17.291 INFO:teuthology.orchestra.run.smithi183.stdout: rgw cache enabled = true 2024-09-14T09:04:17.291 INFO:teuthology.orchestra.run.smithi183.stdout: rgw enable ops log = true 2024-09-14T09:04:17.291 INFO:teuthology.orchestra.run.smithi183.stdout: rgw enable usage log = true 2024-09-14T09:04:17.291 INFO:teuthology.orchestra.run.smithi183.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-14T09:04:17.291 INFO:teuthology.orchestra.run.smithi183.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-14T09:04:17.291 INFO:teuthology.orchestra.run.smithi183.stdout: rbd cache = True 2024-09-14T09:04:17.291 INFO:teuthology.orchestra.run.smithi183.stdout: rbd cache policy = writearound 2024-09-14T09:04:17.291 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.a] 2024-09-14T09:04:17.291 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.c] 2024-09-14T09:04:17.291 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.b] 2024-09-14T09:04:17.292 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-14T09:04:17.292 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-14T09:04:17.370 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-14T09:04:17.370 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:17.370 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-14T09:04:17.375 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:17.375 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-14T09:04:17.421 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T09:04:17.422 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:17.422 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-14T09:04:17.422 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T09:04:17.489 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:17.489 DEBUG:teuthology.orchestra.run.smithi176:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-14T09:04:17.538 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi183.front.sepia.ceph.com 2024-09-14T09:04:17.538 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T09:04:17.538 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-14T09:04:17.539 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T09:04:17.560 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T09:04:17.560 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-14T09:04:17.607 INFO:tasks.ceph:Setting up mon nodes... 2024-09-14T09:04:17.607 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-14T09:04:17.607 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-14T09:04:17.650 INFO:teuthology.orchestra.run.smithi176.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-14T09:04:17.653 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-14T09:04:17.696 INFO:teuthology.orchestra.run.smithi183.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-14T09:04:17.708 INFO:tasks.ceph:Setting up mds nodes... 2024-09-14T09:04:17.708 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-14T09:04:17.708 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-14T09:04:17.742 INFO:teuthology.orchestra.run.smithi176.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-14T09:04:17.753 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-14T09:04:17.791 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-14T09:04:17.802 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-14T09:04:17.802 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi176.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi183.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-14T09:04:17.803 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-14T09:04:17.815 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T09:04:17.815 INFO:tasks.ceph:role: osd.0 2024-09-14T09:04:17.815 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T09:04:17.815 DEBUG:teuthology.orchestra.run.smithi176:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-14T09:04:17.881 INFO:teuthology.orchestra.run.smithi176.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T09:04:17.882 INFO:teuthology.orchestra.run.smithi176.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T09:04:17.882 INFO:teuthology.orchestra.run.smithi176.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T09:04:17.882 INFO:teuthology.orchestra.run.smithi176.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T09:04:17.882 INFO:teuthology.orchestra.run.smithi176.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T09:04:17.882 INFO:teuthology.orchestra.run.smithi176.stdout: = sunit=0 swidth=0 blks 2024-09-14T09:04:17.882 INFO:teuthology.orchestra.run.smithi176.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T09:04:17.882 INFO:teuthology.orchestra.run.smithi176.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T09:04:17.882 INFO:teuthology.orchestra.run.smithi176.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T09:04:17.882 INFO:teuthology.orchestra.run.smithi176.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T09:04:18.356 INFO:teuthology.orchestra.run.smithi176.stdout:Discarding blocks...Done. 2024-09-14T09:04:18.358 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi176.front.sepia.ceph.com -o noatime 2024-09-14T09:04:18.358 DEBUG:teuthology.orchestra.run.smithi176:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-14T09:04:18.488 DEBUG:teuthology.orchestra.run.smithi176:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-14T09:04:18.499 INFO:teuthology.orchestra.run.smithi176.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T09:04:18.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T09:04:18.500 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-14T09:04:18.555 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T09:04:18.555 INFO:tasks.ceph:role: osd.1 2024-09-14T09:04:18.555 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T09:04:18.555 DEBUG:teuthology.orchestra.run.smithi176:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-14T09:04:18.617 INFO:teuthology.orchestra.run.smithi176.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T09:04:18.618 INFO:teuthology.orchestra.run.smithi176.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T09:04:18.618 INFO:teuthology.orchestra.run.smithi176.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T09:04:18.618 INFO:teuthology.orchestra.run.smithi176.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T09:04:18.618 INFO:teuthology.orchestra.run.smithi176.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T09:04:18.618 INFO:teuthology.orchestra.run.smithi176.stdout: = sunit=0 swidth=0 blks 2024-09-14T09:04:18.618 INFO:teuthology.orchestra.run.smithi176.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T09:04:18.618 INFO:teuthology.orchestra.run.smithi176.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T09:04:18.618 INFO:teuthology.orchestra.run.smithi176.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T09:04:18.618 INFO:teuthology.orchestra.run.smithi176.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T09:04:19.044 INFO:teuthology.orchestra.run.smithi176.stdout:Discarding blocks...Done. 2024-09-14T09:04:19.046 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi176.front.sepia.ceph.com -o noatime 2024-09-14T09:04:19.046 DEBUG:teuthology.orchestra.run.smithi176:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-14T09:04:19.112 DEBUG:teuthology.orchestra.run.smithi176:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-14T09:04:19.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T09:04:19.163 INFO:teuthology.orchestra.run.smithi176.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T09:04:19.163 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-14T09:04:19.216 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T09:04:19.217 INFO:tasks.ceph:role: osd.2 2024-09-14T09:04:19.217 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T09:04:19.217 DEBUG:teuthology.orchestra.run.smithi176:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-14T09:04:19.277 INFO:teuthology.orchestra.run.smithi176.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T09:04:19.277 INFO:teuthology.orchestra.run.smithi176.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T09:04:19.277 INFO:teuthology.orchestra.run.smithi176.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T09:04:19.277 INFO:teuthology.orchestra.run.smithi176.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T09:04:19.277 INFO:teuthology.orchestra.run.smithi176.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T09:04:19.277 INFO:teuthology.orchestra.run.smithi176.stdout: = sunit=0 swidth=0 blks 2024-09-14T09:04:19.277 INFO:teuthology.orchestra.run.smithi176.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T09:04:19.278 INFO:teuthology.orchestra.run.smithi176.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T09:04:19.278 INFO:teuthology.orchestra.run.smithi176.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T09:04:19.278 INFO:teuthology.orchestra.run.smithi176.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T09:04:19.743 INFO:teuthology.orchestra.run.smithi176.stdout:Discarding blocks...Done. 2024-09-14T09:04:19.745 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi176.front.sepia.ceph.com -o noatime 2024-09-14T09:04:19.745 DEBUG:teuthology.orchestra.run.smithi176:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-14T09:04:19.835 DEBUG:teuthology.orchestra.run.smithi176:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-14T09:04:19.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T09:04:19.845 INFO:teuthology.orchestra.run.smithi176.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T09:04:19.845 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-14T09:04:19.898 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T09:04:19.898 INFO:tasks.ceph:role: osd.3 2024-09-14T09:04:19.899 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T09:04:19.899 DEBUG:teuthology.orchestra.run.smithi176:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-14T09:04:19.962 INFO:teuthology.orchestra.run.smithi176.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T09:04:19.962 INFO:teuthology.orchestra.run.smithi176.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T09:04:19.963 INFO:teuthology.orchestra.run.smithi176.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T09:04:19.963 INFO:teuthology.orchestra.run.smithi176.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T09:04:19.963 INFO:teuthology.orchestra.run.smithi176.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T09:04:19.963 INFO:teuthology.orchestra.run.smithi176.stdout: = sunit=0 swidth=0 blks 2024-09-14T09:04:19.963 INFO:teuthology.orchestra.run.smithi176.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T09:04:19.963 INFO:teuthology.orchestra.run.smithi176.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T09:04:19.963 INFO:teuthology.orchestra.run.smithi176.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T09:04:19.963 INFO:teuthology.orchestra.run.smithi176.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T09:04:20.422 INFO:teuthology.orchestra.run.smithi176.stdout:Discarding blocks...Done. 2024-09-14T09:04:20.424 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi176.front.sepia.ceph.com -o noatime 2024-09-14T09:04:20.424 DEBUG:teuthology.orchestra.run.smithi176:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-14T09:04:20.574 DEBUG:teuthology.orchestra.run.smithi176:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-14T09:04:20.585 INFO:teuthology.orchestra.run.smithi176.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T09:04:20.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T09:04:20.586 DEBUG:teuthology.orchestra.run.smithi176:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T09:04:20.696 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T09:04:20.693+0000 7f944240b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-14T09:04:20.700 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T09:04:20.697+0000 7f944240b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-14T09:04:20.700 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T09:04:20.697+0000 7f944240b7c0 -1 bdev(0x55eb899c3800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-14T09:04:20.700 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T09:04:20.697+0000 7f944240b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-14T09:04:23.870 DEBUG:teuthology.orchestra.run.smithi176:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-14T09:04:23.885 DEBUG:teuthology.orchestra.run.smithi176:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T09:04:23.965 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T09:04:23.961+0000 7f1b2da017c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-14T09:04:23.965 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T09:04:23.961+0000 7f1b2da017c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-14T09:04:23.966 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T09:04:23.961+0000 7f1b2da017c0 -1 bdev(0x5617938bd800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-14T09:04:23.966 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T09:04:23.961+0000 7f1b2da017c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-14T09:04:27.123 DEBUG:teuthology.orchestra.run.smithi176:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-14T09:04:27.181 DEBUG:teuthology.orchestra.run.smithi176:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T09:04:27.261 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T09:04:27.256+0000 7f1c5f9707c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-14T09:04:27.262 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T09:04:27.260+0000 7f1c5f9707c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-14T09:04:27.263 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T09:04:27.260+0000 7f1c5f9707c0 -1 bdev(0x562647be3800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-14T09:04:27.263 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T09:04:27.260+0000 7f1c5f9707c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-14T09:04:30.411 DEBUG:teuthology.orchestra.run.smithi176:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-14T09:04:30.467 DEBUG:teuthology.orchestra.run.smithi176:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T09:04:30.549 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T09:04:30.544+0000 7fb340c337c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-14T09:04:30.549 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T09:04:30.544+0000 7fb340c337c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-14T09:04:30.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T09:04:30.544+0000 7fb340c337c0 -1 bdev(0x565487285800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-14T09:04:30.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T09:04:30.544+0000 7fb340c337c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-14T09:04:33.685 DEBUG:teuthology.orchestra.run.smithi176:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-14T09:04:33.739 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-14T09:04:33.754 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T09:04:33.754 INFO:tasks.ceph:role: osd.4 2024-09-14T09:04:33.754 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi183.front.sepia.ceph.com 2024-09-14T09:04:33.754 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-14T09:04:33.818 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T09:04:33.818 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T09:04:33.818 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T09:04:33.818 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T09:04:33.819 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T09:04:33.819 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2024-09-14T09:04:33.819 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T09:04:33.819 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T09:04:33.819 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T09:04:33.819 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T09:04:34.304 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2024-09-14T09:04:34.305 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2024-09-14T09:04:34.306 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-14T09:04:34.437 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-14T09:04:34.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T09:04:34.449 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T09:04:34.449 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-14T09:04:34.506 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T09:04:34.506 INFO:tasks.ceph:role: osd.5 2024-09-14T09:04:34.506 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi183.front.sepia.ceph.com 2024-09-14T09:04:34.506 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-14T09:04:34.570 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T09:04:34.570 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T09:04:34.570 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T09:04:34.570 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T09:04:34.570 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T09:04:34.570 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2024-09-14T09:04:34.570 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T09:04:34.570 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T09:04:34.570 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T09:04:34.570 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T09:04:34.995 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2024-09-14T09:04:34.997 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2024-09-14T09:04:34.997 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-14T09:04:35.063 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-14T09:04:35.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T09:04:35.119 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T09:04:35.119 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-14T09:04:35.172 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T09:04:35.172 INFO:tasks.ceph:role: osd.6 2024-09-14T09:04:35.172 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi183.front.sepia.ceph.com 2024-09-14T09:04:35.172 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-14T09:04:35.234 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T09:04:35.234 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T09:04:35.234 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T09:04:35.234 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T09:04:35.234 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T09:04:35.234 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2024-09-14T09:04:35.234 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T09:04:35.234 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T09:04:35.234 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T09:04:35.234 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T09:04:35.709 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2024-09-14T09:04:35.711 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2024-09-14T09:04:35.711 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-14T09:04:35.798 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-14T09:04:35.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T09:04:35.850 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T09:04:35.850 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-14T09:04:35.905 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T09:04:35.905 INFO:tasks.ceph:role: osd.7 2024-09-14T09:04:35.905 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi183.front.sepia.ceph.com 2024-09-14T09:04:35.905 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-14T09:04:35.972 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T09:04:35.972 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T09:04:35.972 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T09:04:35.972 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T09:04:35.972 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T09:04:35.972 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2024-09-14T09:04:35.972 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T09:04:35.972 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T09:04:35.972 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T09:04:35.972 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T09:04:36.454 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2024-09-14T09:04:36.456 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2024-09-14T09:04:36.456 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-14T09:04:36.583 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-14T09:04:36.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T09:04:36.595 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T09:04:36.595 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T09:04:36.714 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-14T09:04:36.712+0000 7fd4797027c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-14T09:04:36.716 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-14T09:04:36.712+0000 7fd4797027c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-14T09:04:36.716 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-14T09:04:36.712+0000 7fd4797027c0 -1 bdev(0x558c9032d800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-14T09:04:36.716 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-14T09:04:36.712+0000 7fd4797027c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-14T09:04:39.851 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-14T09:04:39.864 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T09:04:39.948 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-14T09:04:39.944+0000 7f68b56127c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-14T09:04:39.949 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-14T09:04:39.944+0000 7f68b56127c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-14T09:04:39.949 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-14T09:04:39.944+0000 7f68b56127c0 -1 bdev(0x5563d6837800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-14T09:04:39.949 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-14T09:04:39.944+0000 7f68b56127c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-14T09:04:43.101 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-14T09:04:43.157 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T09:04:43.241 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-14T09:04:43.240+0000 7f4e1ae757c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-14T09:04:43.242 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-14T09:04:43.240+0000 7f4e1ae757c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-14T09:04:43.243 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-14T09:04:43.240+0000 7f4e1ae757c0 -1 bdev(0x55f436867800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-14T09:04:43.243 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-14T09:04:43.240+0000 7f4e1ae757c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-14T09:04:46.385 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-14T09:04:46.443 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T09:04:46.529 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-14T09:04:46.528+0000 7fe139d917c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-14T09:04:46.530 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-14T09:04:46.528+0000 7fe139d917c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-14T09:04:46.530 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-14T09:04:46.528+0000 7fe139d917c0 -1 bdev(0x5648b28cb800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-14T09:04:46.530 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-14T09:04:46.528+0000 7fe139d917c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-14T09:04:49.660 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-14T09:04:49.713 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-14T09:04:49.714 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:49.714 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-14T09:04:49.727 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:49.727 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-14T09:04:49.782 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:49.782 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-14T09:04:49.837 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:49.837 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-14T09:04:49.893 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:49.893 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-14T09:04:49.945 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T09:04:49.946 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-14T09:04:49.959 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T09:04:49.959 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-14T09:04:50.013 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T09:04:50.014 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-14T09:04:50.069 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T09:04:50.069 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-14T09:04:50.124 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T09:04:50.124 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-14T09:04:50.178 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:50.178 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-14T09:04:50.184 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T09:04:50.184 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-14T09:04:50.226 INFO:tasks.ceph:Adding keys to all mons... 2024-09-14T09:04:50.227 DEBUG:teuthology.orchestra.run.smithi176:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-14T09:04:50.229 DEBUG:teuthology.orchestra.run.smithi183:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-14T09:04:50.272 INFO:teuthology.orchestra.run.smithi176.stdout:[mgr.y] 2024-09-14T09:04:50.272 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCRUeVmao/jJhAAuVxSZVNEQokrz7QOl1UQPg== 2024-09-14T09:04:50.272 INFO:teuthology.orchestra.run.smithi176.stdout:[osd.0] 2024-09-14T09:04:50.272 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCUUeVmDyakKRAA5LCPfPznqYoKp2jIHblMeA== 2024-09-14T09:04:50.272 INFO:teuthology.orchestra.run.smithi176.stdout:[osd.1] 2024-09-14T09:04:50.272 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCXUeVmyPmgORAAJ30CVDl3Z/lNFb8neqbWlQ== 2024-09-14T09:04:50.272 INFO:teuthology.orchestra.run.smithi176.stdout:[osd.2] 2024-09-14T09:04:50.273 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCbUeVmol+5DxAAotjX9W2KeXMYVKEWCurKJg== 2024-09-14T09:04:50.273 INFO:teuthology.orchestra.run.smithi176.stdout:[osd.3] 2024-09-14T09:04:50.273 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCeUeVmJlHVIBAA/cR6JLRTDSOJ26oMjwUw1g== 2024-09-14T09:04:50.273 INFO:teuthology.orchestra.run.smithi176.stdout:[mgr.x] 2024-09-14T09:04:50.273 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCRUeVmYMKcKRAAZ/D93b5dUN+jsRK7HA42lA== 2024-09-14T09:04:50.273 INFO:teuthology.orchestra.run.smithi176.stdout:[osd.4] 2024-09-14T09:04:50.273 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCkUeVmqMezKhAAHRuz7Y/AuPt5VJcpJB38tw== 2024-09-14T09:04:50.273 INFO:teuthology.orchestra.run.smithi176.stdout:[osd.5] 2024-09-14T09:04:50.273 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCnUeVmq/utOBAAuN5E8VkETopOWKgsR6LekQ== 2024-09-14T09:04:50.273 INFO:teuthology.orchestra.run.smithi176.stdout:[osd.6] 2024-09-14T09:04:50.273 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCrUeVmXQuSDhAAu7Msr6qnCoqIbmPHMRj6rg== 2024-09-14T09:04:50.273 INFO:teuthology.orchestra.run.smithi176.stdout:[osd.7] 2024-09-14T09:04:50.274 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCuUeVmsS+6HxAAeiJScSdcYfWcEasAxTkHFw== 2024-09-14T09:04:50.274 INFO:teuthology.orchestra.run.smithi176.stdout:[client.0] 2024-09-14T09:04:50.274 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCRUeVmWitfLBAAfAMbDPUXIQcCuIRspNkWmQ== 2024-09-14T09:04:50.274 INFO:teuthology.orchestra.run.smithi176.stdout:[client.1] 2024-09-14T09:04:50.274 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQCRUeVmRmJRLxAAw6uCArNzHD4qkEwbEkEkoQ== 2024-09-14T09:04:50.280 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr.y] 2024-09-14T09:04:50.280 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCRUeVmao/jJhAAuVxSZVNEQokrz7QOl1UQPg== 2024-09-14T09:04:50.281 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.0] 2024-09-14T09:04:50.281 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCUUeVmDyakKRAA5LCPfPznqYoKp2jIHblMeA== 2024-09-14T09:04:50.281 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.1] 2024-09-14T09:04:50.281 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCXUeVmyPmgORAAJ30CVDl3Z/lNFb8neqbWlQ== 2024-09-14T09:04:50.281 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.2] 2024-09-14T09:04:50.281 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCbUeVmol+5DxAAotjX9W2KeXMYVKEWCurKJg== 2024-09-14T09:04:50.281 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.3] 2024-09-14T09:04:50.281 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCeUeVmJlHVIBAA/cR6JLRTDSOJ26oMjwUw1g== 2024-09-14T09:04:50.281 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr.x] 2024-09-14T09:04:50.281 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCRUeVmYMKcKRAAZ/D93b5dUN+jsRK7HA42lA== 2024-09-14T09:04:50.281 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.4] 2024-09-14T09:04:50.281 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCkUeVmqMezKhAAHRuz7Y/AuPt5VJcpJB38tw== 2024-09-14T09:04:50.281 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.5] 2024-09-14T09:04:50.282 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCnUeVmq/utOBAAuN5E8VkETopOWKgsR6LekQ== 2024-09-14T09:04:50.282 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.6] 2024-09-14T09:04:50.282 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCrUeVmXQuSDhAAu7Msr6qnCoqIbmPHMRj6rg== 2024-09-14T09:04:50.283 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.7] 2024-09-14T09:04:50.283 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCuUeVmsS+6HxAAeiJScSdcYfWcEasAxTkHFw== 2024-09-14T09:04:50.283 INFO:teuthology.orchestra.run.smithi183.stdout:[client.0] 2024-09-14T09:04:50.283 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCRUeVmWitfLBAAfAMbDPUXIQcCuIRspNkWmQ== 2024-09-14T09:04:50.283 INFO:teuthology.orchestra.run.smithi183.stdout:[client.1] 2024-09-14T09:04:50.283 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCRUeVmRmJRLxAAw6uCArNzHD4qkEwbEkEkoQ== 2024-09-14T09:04:50.284 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T09:04:50.319 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T09:04:50.370 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T09:04:50.372 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T09:04:50.457 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T09:04:50.459 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T09:04:50.537 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T09:04:50.543 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T09:04:50.624 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T09:04:50.627 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T09:04:50.708 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T09:04:50.711 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T09:04:50.794 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T09:04:50.797 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T09:04:50.839 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T09:04:50.883 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T09:04:50.927 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T09:04:50.967 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T09:04:51.013 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T09:04:51.055 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T09:04:51.102 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T09:04:51.139 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T09:04:51.186 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T09:04:51.189 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T09:04:51.272 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-14T09:04:51.273 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-14T09:04:51.284 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T09:04:51.376 DEBUG:teuthology.orchestra.run.smithi176:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-14T09:04:51.386 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-14T09:04:51.440 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T09:04:51.530 DEBUG:teuthology.orchestra.run.smithi176:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-14T09:04:51.541 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-14T09:04:51.553 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T09:04:51.648 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-14T09:04:51.659 DEBUG:teuthology.orchestra.run.smithi176:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-14T09:04:51.662 DEBUG:teuthology.orchestra.run.smithi183:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-14T09:04:51.709 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-14T09:04:51.709 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-14T09:04:51.709 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-14T09:04:51.711 INFO:tasks.ceph.mon.a:Started 2024-09-14T09:04:51.711 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-14T09:04:51.712 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-14T09:04:51.714 INFO:tasks.ceph.mon.c:Started 2024-09-14T09:04:51.714 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-14T09:04:51.714 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-14T09:04:51.755 INFO:tasks.ceph.mon.b:Started 2024-09-14T09:04:51.755 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-14T09:04:51.755 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-14T09:04:51.755 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-14T09:04:51.758 INFO:tasks.ceph.mgr.y:Started 2024-09-14T09:04:51.758 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-14T09:04:51.758 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-14T09:04:51.761 INFO:tasks.ceph.mgr.x:Started 2024-09-14T09:04:51.761 DEBUG:tasks.ceph:set 0 configs 2024-09-14T09:04:51.761 INFO:tasks.ceph:Setting crush tunables to default 2024-09-14T09:04:51.761 DEBUG:teuthology.orchestra.run.smithi176:> sudo ceph --cluster ceph osd crush tunables default 2024-09-14T09:04:57.196 INFO:teuthology.orchestra.run.smithi176.stderr:adjusted tunables profile to default 2024-09-14T09:04:57.214 INFO:tasks.ceph:check_enable_crimson: False 2024-09-14T09:04:57.214 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-14T09:04:57.215 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:57.215 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-14T09:04:57.225 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:57.225 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-14T09:04:57.275 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:57.276 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-14T09:04:57.330 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-09-14T09:04:57.330 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-14T09:04:57.385 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T09:04:57.386 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-14T09:04:57.399 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T09:04:57.399 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-14T09:04:57.452 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T09:04:57.452 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-14T09:04:57.504 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-14T09:04:57.504 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-14T09:04:57.556 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 4ceedb4f-6398-467f-9562-ccf27200860b 0 2024-09-14T09:04:57.814 INFO:teuthology.orchestra.run.smithi183.stdout:0 2024-09-14T09:04:57.830 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 604e2f62-7c13-46be-a97f-029b28c44a89 1 2024-09-14T09:04:58.039 INFO:teuthology.orchestra.run.smithi183.stdout:1 2024-09-14T09:04:58.056 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 1f65589e-8b0a-4033-94c0-4ab66531db2b 2 2024-09-14T09:04:58.268 INFO:teuthology.orchestra.run.smithi183.stdout:2 2024-09-14T09:04:58.286 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 209d4f65-79f9-4a7d-9d23-832a4209507f 3 2024-09-14T09:04:58.482 INFO:teuthology.orchestra.run.smithi183.stdout:3 2024-09-14T09:04:58.499 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 8f9a8c9b-282e-4523-9e62-5824f4aba401 4 2024-09-14T09:04:58.698 INFO:teuthology.orchestra.run.smithi183.stdout:4 2024-09-14T09:04:58.715 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80 5 2024-09-14T09:04:58.913 INFO:teuthology.orchestra.run.smithi183.stdout:5 2024-09-14T09:04:58.929 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 59d9259d-b9dd-4326-8ab9-9d599a2bf57b 6 2024-09-14T09:04:59.125 INFO:teuthology.orchestra.run.smithi183.stdout:6 2024-09-14T09:04:59.141 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new e2f27d42-786b-449b-a2fb-9dd28f4e9788 7 2024-09-14T09:04:59.347 INFO:teuthology.orchestra.run.smithi183.stdout:7 2024-09-14T09:04:59.363 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-14T09:04:59.364 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-14T09:04:59.366 INFO:tasks.ceph.osd.0:Started 2024-09-14T09:04:59.366 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-14T09:04:59.366 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-14T09:04:59.369 INFO:tasks.ceph.osd.1:Started 2024-09-14T09:04:59.369 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-14T09:04:59.369 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-14T09:04:59.371 INFO:tasks.ceph.osd.2:Started 2024-09-14T09:04:59.371 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-14T09:04:59.371 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-14T09:04:59.373 INFO:tasks.ceph.osd.3:Started 2024-09-14T09:04:59.374 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-14T09:04:59.374 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-14T09:04:59.376 INFO:tasks.ceph.osd.4:Started 2024-09-14T09:04:59.376 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-14T09:04:59.376 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-14T09:04:59.378 INFO:tasks.ceph.osd.5:Started 2024-09-14T09:04:59.378 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-14T09:04:59.378 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-14T09:04:59.380 INFO:tasks.ceph.osd.6:Started 2024-09-14T09:04:59.380 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-14T09:04:59.380 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-14T09:04:59.383 INFO:tasks.ceph.osd.7:Started 2024-09-14T09:04:59.383 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T09:04:59.579 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:04:59.579 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":10,"fsid":"c635667d-994e-454f-bcea-51b45637b2a2","created":"2024-09-14T09:04:57.077293+0000","modified":"2024-09-14T09:04:59.344709+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T09:04:59.344709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4ceedb4f-6398-467f-9562-ccf27200860b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"604e2f62-7c13-46be-a97f-029b28c44a89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1f65589e-8b0a-4033-94c0-4ab66531db2b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"209d4f65-79f9-4a7d-9d23-832a4209507f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8f9a8c9b-282e-4523-9e62-5824f4aba401","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"59d9259d-b9dd-4326-8ab9-9d599a2bf57b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e2f27d42-786b-449b-a2fb-9dd28f4e9788","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T09:04:59.595 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-14T09:04:59.596 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-14T09:05:00.115 INFO:tasks.ceph.mgr.x.smithi183.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-14T09:05:00.115 INFO:tasks.ceph.mgr.x.smithi183.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-14T09:05:00.115 INFO:tasks.ceph.mgr.x.smithi183.stderr: from numpy import show_config as show_numpy_config 2024-09-14T09:05:00.119 INFO:tasks.ceph.mgr.y.smithi176.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-14T09:05:00.119 INFO:tasks.ceph.mgr.y.smithi176.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-14T09:05:00.119 INFO:tasks.ceph.mgr.y.smithi176.stderr: from numpy import show_config as show_numpy_config 2024-09-14T09:05:00.402 DEBUG:teuthology.orchestra.run.smithi176:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T09:05:00.582 INFO:teuthology.misc.health.smithi176.stdout: 2024-09-14T09:05:00.583 INFO:teuthology.misc.health.smithi176.stdout:{"epoch":10,"fsid":"c635667d-994e-454f-bcea-51b45637b2a2","created":"2024-09-14T09:04:57.077293+0000","modified":"2024-09-14T09:04:59.344709+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T09:04:59.344709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4ceedb4f-6398-467f-9562-ccf27200860b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"604e2f62-7c13-46be-a97f-029b28c44a89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1f65589e-8b0a-4033-94c0-4ab66531db2b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"209d4f65-79f9-4a7d-9d23-832a4209507f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8f9a8c9b-282e-4523-9e62-5824f4aba401","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"59d9259d-b9dd-4326-8ab9-9d599a2bf57b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e2f27d42-786b-449b-a2fb-9dd28f4e9788","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T09:05:00.599 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-14T09:05:00.787 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:00.784+0000 7fb5cfa127c0 -1 Falling back to public interface 2024-09-14T09:05:00.800 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:00.796+0000 7f73d26797c0 -1 Falling back to public interface 2024-09-14T09:05:00.803 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:00.800+0000 7fce62caf7c0 -1 Falling back to public interface 2024-09-14T09:05:00.835 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:00.832+0000 7f7eb14807c0 -1 Falling back to public interface 2024-09-14T09:05:01.042 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:01.040+0000 7f7aa1b197c0 -1 Falling back to public interface 2024-09-14T09:05:01.135 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:01.132+0000 7f609788b7c0 -1 Falling back to public interface 2024-09-14T09:05:01.243 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:01.240+0000 7fecee0bf7c0 -1 Falling back to public interface 2024-09-14T09:05:01.582 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:01.580+0000 7fdc39fb67c0 -1 Falling back to public interface 2024-09-14T09:05:02.967 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-09-14T09:05:02.964+0000 7f321da70640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-14T09:05:02.972 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-09-14T09:05:02.968+0000 7f321da70640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-14T09:05:04.792 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:04.788+0000 7f73d26797c0 -1 osd.1 0 log_to_monitors true 2024-09-14T09:05:04.829 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:04.828+0000 7fb5cfa127c0 -1 osd.7 0 log_to_monitors true 2024-09-14T09:05:04.895 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:04.892+0000 7f7eb14807c0 -1 osd.3 0 log_to_monitors true 2024-09-14T09:05:05.027 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:05.024+0000 7f7aa1b197c0 -1 osd.4 0 log_to_monitors true 2024-09-14T09:05:05.177 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:05.172+0000 7fce62caf7c0 -1 osd.2 0 log_to_monitors true 2024-09-14T09:05:05.382 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:05.380+0000 7f609788b7c0 -1 osd.0 0 log_to_monitors true 2024-09-14T09:05:05.601 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:05.596+0000 7fdc39fb67c0 -1 osd.5 0 log_to_monitors true 2024-09-14T09:05:05.785 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:05.784+0000 7fecee0bf7c0 -1 osd.6 0 log_to_monitors true 2024-09-14T09:05:07.404 DEBUG:teuthology.orchestra.run.smithi176:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T09:05:07.894 INFO:teuthology.misc.health.smithi176.stdout: 2024-09-14T09:05:07.894 INFO:teuthology.misc.health.smithi176.stdout:{"epoch":13,"fsid":"c635667d-994e-454f-bcea-51b45637b2a2","created":"2024-09-14T09:04:57.077293+0000","modified":"2024-09-14T09:05:07.116749+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T09:04:59.344709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4ceedb4f-6398-467f-9562-ccf27200860b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"604e2f62-7c13-46be-a97f-029b28c44a89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1f65589e-8b0a-4033-94c0-4ab66531db2b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"209d4f65-79f9-4a7d-9d23-832a4209507f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8f9a8c9b-282e-4523-9e62-5824f4aba401","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"59d9259d-b9dd-4326-8ab9-9d599a2bf57b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e2f27d42-786b-449b-a2fb-9dd28f4e9788","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T09:05:07.917 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-14T09:05:09.182 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:09.180+0000 7f7a9dac2640 -1 osd.4 0 waiting for initial osdmap 2024-09-14T09:05:09.210 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:09.208+0000 7f7a958f4640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T09:05:09.234 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:09.232+0000 7fb5cb9bb640 -1 osd.7 0 waiting for initial osdmap 2024-09-14T09:05:09.269 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:09.264+0000 7f7ead429640 -1 osd.3 0 waiting for initial osdmap 2024-09-14T09:05:09.280 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:09.276+0000 7fb5c37ed640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T09:05:09.295 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:09.292+0000 7f73ce622640 -1 osd.1 0 waiting for initial osdmap 2024-09-14T09:05:09.302 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:09.300+0000 7f7ea525b640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T09:05:09.318 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:09.316+0000 7f73c6454640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T09:05:10.196 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:10.192+0000 7fce5ec58640 -1 osd.2 0 waiting for initial osdmap 2024-09-14T09:05:10.198 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:10.196+0000 7fecea068640 -1 osd.6 0 waiting for initial osdmap 2024-09-14T09:05:10.229 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:10.228+0000 7fece1e9a640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T09:05:10.234 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:10.232+0000 7fce56a8a640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T09:05:10.245 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:10.244+0000 7fdc36771640 -1 osd.5 0 waiting for initial osdmap 2024-09-14T09:05:10.281 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:10.280+0000 7fdc2dd91640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T09:05:10.288 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:10.284+0000 7f6093834640 -1 osd.0 0 waiting for initial osdmap 2024-09-14T09:05:10.311 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:10.308+0000 7f608b666640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T09:05:14.724 DEBUG:teuthology.orchestra.run.smithi176:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T09:05:15.032 INFO:teuthology.misc.health.smithi176.stdout: 2024-09-14T09:05:15.033 INFO:teuthology.misc.health.smithi176.stdout:{"epoch":18,"fsid":"c635667d-994e-454f-bcea-51b45637b2a2","created":"2024-09-14T09:04:57.077293+0000","modified":"2024-09-14T09:05:14.233630+0000","last_up_change":"2024-09-14T09:05:11.210835+0000","last_in_change":"2024-09-14T09:04:59.344709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T09:05:11.997673+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ceedb4f-6398-467f-9562-ccf27200860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6825","nonce":3384710819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6827","nonce":3384710819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6831","nonce":3384710819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6829","nonce":3384710819}]},"public_addr":"172.21.15.176:6825/3384710819","cluster_addr":"172.21.15.176:6827/3384710819","heartbeat_back_addr":"172.21.15.176:6831/3384710819","heartbeat_front_addr":"172.21.15.176:6829/3384710819","state":["exists","up"]},{"osd":1,"uuid":"604e2f62-7c13-46be-a97f-029b28c44a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6801","nonce":612012679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6803","nonce":612012679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6807","nonce":612012679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6805","nonce":612012679}]},"public_addr":"172.21.15.176:6801/612012679","cluster_addr":"172.21.15.176:6803/612012679","heartbeat_back_addr":"172.21.15.176:6807/612012679","heartbeat_front_addr":"172.21.15.176:6805/612012679","state":["exists","up"]},{"osd":2,"uuid":"1f65589e-8b0a-4033-94c0-4ab66531db2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6809","nonce":4167379279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6811","nonce":4167379279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6815","nonce":4167379279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6813","nonce":4167379279}]},"public_addr":"172.21.15.176:6809/4167379279","cluster_addr":"172.21.15.176:6811/4167379279","heartbeat_back_addr":"172.21.15.176:6815/4167379279","heartbeat_front_addr":"172.21.15.176:6813/4167379279","state":["exists","up"]},{"osd":3,"uuid":"209d4f65-79f9-4a7d-9d23-832a4209507f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6817","nonce":1658209015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6819","nonce":1658209015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6823","nonce":1658209015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6821","nonce":1658209015}]},"public_addr":"172.21.15.176:6817/1658209015","cluster_addr":"172.21.15.176:6819/1658209015","heartbeat_back_addr":"172.21.15.176:6823/1658209015","heartbeat_front_addr":"172.21.15.176:6821/1658209015","state":["exists","up"]},{"osd":4,"uuid":"8f9a8c9b-282e-4523-9e62-5824f4aba401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":1576054389},{"type":"v1","addr":"172.21.15.183:6809","nonce":1576054389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":1576054389},{"type":"v1","addr":"172.21.15.183:6811","nonce":1576054389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":1576054389},{"type":"v1","addr":"172.21.15.183:6815","nonce":1576054389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":1576054389},{"type":"v1","addr":"172.21.15.183:6813","nonce":1576054389}]},"public_addr":"172.21.15.183:6809/1576054389","cluster_addr":"172.21.15.183:6811/1576054389","heartbeat_back_addr":"172.21.15.183:6815/1576054389","heartbeat_front_addr":"172.21.15.183:6813/1576054389","state":["exists","up"]},{"osd":5,"uuid":"1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6825","nonce":1711292386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6827","nonce":1711292386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6831","nonce":1711292386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6829","nonce":1711292386}]},"public_addr":"172.21.15.183:6825/1711292386","cluster_addr":"172.21.15.183:6827/1711292386","heartbeat_back_addr":"172.21.15.183:6831/1711292386","heartbeat_front_addr":"172.21.15.183:6829/1711292386","state":["exists","up"]},{"osd":6,"uuid":"59d9259d-b9dd-4326-8ab9-9d599a2bf57b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6817","nonce":2721245607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6819","nonce":2721245607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6823","nonce":2721245607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6821","nonce":2721245607}]},"public_addr":"172.21.15.183:6817/2721245607","cluster_addr":"172.21.15.183:6819/2721245607","heartbeat_back_addr":"172.21.15.183:6823/2721245607","heartbeat_front_addr":"172.21.15.183:6821/2721245607","state":["exists","up"]},{"osd":7,"uuid":"e2f27d42-786b-449b-a2fb-9dd28f4e9788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6801","nonce":2122823465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6803","nonce":2122823465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6807","nonce":2122823465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6805","nonce":2122823465}]},"public_addr":"172.21.15.183:6801/2122823465","cluster_addr":"172.21.15.183:6803/2122823465","heartbeat_back_addr":"172.21.15.183:6807/2122823465","heartbeat_front_addr":"172.21.15.183:6805/2122823465","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.411789+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:05.801873+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.206725+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:05.882180+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.013864+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.652975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.741359+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:05.873675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T09:05:15.050 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-14T09:05:15.050 INFO:tasks.ceph:Creating RBD pool 2024-09-14T09:05:15.051 DEBUG:teuthology.orchestra.run.smithi176:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-14T09:05:16.269 INFO:teuthology.orchestra.run.smithi176.stderr:pool 'rbd' created 2024-09-14T09:05:16.290 DEBUG:teuthology.orchestra.run.smithi176:> rbd --cluster ceph pool init rbd 2024-09-14T09:05:19.324 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-14T09:05:19.325 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-14T09:05:19.325 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-14T09:05:19.641 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:05:19.659 INFO:teuthology.orchestra.run.smithi176.stdout:[{"version":9,"timestamp":"2024-09-14T09:05:10.291588+0000","name":"","changes":[{"name":"osd.0/osd_mclock_max_capacity_iops_ssd","new_value":"5437.615459"}]}] 2024-09-14T09:05:19.660 INFO:tasks.ceph_manager:config epoch is 9 2024-09-14T09:05:19.660 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-14T09:05:19.660 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-14T09:05:19.660 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-14T09:05:20.009 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:05:20.027 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":5,"flags":0,"active_gid":4114,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":916020142},{"type":"v1","addr":"172.21.15.183:6833","nonce":916020142}]},"active_addr":"172.21.15.183:6833/916020142","active_change":"2024-09-14T09:05:01.946571+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4100,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":2424748919}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":3989307032}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":3581577740}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":138132957}]}]} 2024-09-14T09:05:20.032 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-14T09:05:20.032 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-14T09:05:20.032 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T09:05:20.317 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:05:20.317 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":22,"fsid":"c635667d-994e-454f-bcea-51b45637b2a2","created":"2024-09-14T09:04:57.077293+0000","modified":"2024-09-14T09:05:19.287473+0000","last_up_change":"2024-09-14T09:05:11.210835+0000","last_in_change":"2024-09-14T09:04:59.344709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T09:05:11.997673+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T09:05:15.352551+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ceedb4f-6398-467f-9562-ccf27200860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6825","nonce":3384710819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6827","nonce":3384710819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6831","nonce":3384710819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6829","nonce":3384710819}]},"public_addr":"172.21.15.176:6825/3384710819","cluster_addr":"172.21.15.176:6827/3384710819","heartbeat_back_addr":"172.21.15.176:6831/3384710819","heartbeat_front_addr":"172.21.15.176:6829/3384710819","state":["exists","up"]},{"osd":1,"uuid":"604e2f62-7c13-46be-a97f-029b28c44a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6801","nonce":612012679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6803","nonce":612012679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6807","nonce":612012679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6805","nonce":612012679}]},"public_addr":"172.21.15.176:6801/612012679","cluster_addr":"172.21.15.176:6803/612012679","heartbeat_back_addr":"172.21.15.176:6807/612012679","heartbeat_front_addr":"172.21.15.176:6805/612012679","state":["exists","up"]},{"osd":2,"uuid":"1f65589e-8b0a-4033-94c0-4ab66531db2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6809","nonce":4167379279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6811","nonce":4167379279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6815","nonce":4167379279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6813","nonce":4167379279}]},"public_addr":"172.21.15.176:6809/4167379279","cluster_addr":"172.21.15.176:6811/4167379279","heartbeat_back_addr":"172.21.15.176:6815/4167379279","heartbeat_front_addr":"172.21.15.176:6813/4167379279","state":["exists","up"]},{"osd":3,"uuid":"209d4f65-79f9-4a7d-9d23-832a4209507f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6817","nonce":1658209015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6819","nonce":1658209015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6823","nonce":1658209015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6821","nonce":1658209015}]},"public_addr":"172.21.15.176:6817/1658209015","cluster_addr":"172.21.15.176:6819/1658209015","heartbeat_back_addr":"172.21.15.176:6823/1658209015","heartbeat_front_addr":"172.21.15.176:6821/1658209015","state":["exists","up"]},{"osd":4,"uuid":"8f9a8c9b-282e-4523-9e62-5824f4aba401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":1576054389},{"type":"v1","addr":"172.21.15.183:6809","nonce":1576054389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":1576054389},{"type":"v1","addr":"172.21.15.183:6811","nonce":1576054389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":1576054389},{"type":"v1","addr":"172.21.15.183:6815","nonce":1576054389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":1576054389},{"type":"v1","addr":"172.21.15.183:6813","nonce":1576054389}]},"public_addr":"172.21.15.183:6809/1576054389","cluster_addr":"172.21.15.183:6811/1576054389","heartbeat_back_addr":"172.21.15.183:6815/1576054389","heartbeat_front_addr":"172.21.15.183:6813/1576054389","state":["exists","up"]},{"osd":5,"uuid":"1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6825","nonce":1711292386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6827","nonce":1711292386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6831","nonce":1711292386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6829","nonce":1711292386}]},"public_addr":"172.21.15.183:6825/1711292386","cluster_addr":"172.21.15.183:6827/1711292386","heartbeat_back_addr":"172.21.15.183:6831/1711292386","heartbeat_front_addr":"172.21.15.183:6829/1711292386","state":["exists","up"]},{"osd":6,"uuid":"59d9259d-b9dd-4326-8ab9-9d599a2bf57b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6817","nonce":2721245607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6819","nonce":2721245607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6823","nonce":2721245607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6821","nonce":2721245607}]},"public_addr":"172.21.15.183:6817/2721245607","cluster_addr":"172.21.15.183:6819/2721245607","heartbeat_back_addr":"172.21.15.183:6823/2721245607","heartbeat_front_addr":"172.21.15.183:6821/2721245607","state":["exists","up"]},{"osd":7,"uuid":"e2f27d42-786b-449b-a2fb-9dd28f4e9788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6801","nonce":2122823465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6803","nonce":2122823465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6807","nonce":2122823465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6805","nonce":2122823465}]},"public_addr":"172.21.15.183:6801/2122823465","cluster_addr":"172.21.15.183:6803/2122823465","heartbeat_back_addr":"172.21.15.183:6807/2122823465","heartbeat_front_addr":"172.21.15.183:6805/2122823465","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.411789+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:05.801873+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.206725+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:05.882180+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.013864+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.652975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.741359+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:05.873675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T09:05:20.331 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-14T09:05:20.331 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T09:05:20.628 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:05:20.628 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":22,"fsid":"c635667d-994e-454f-bcea-51b45637b2a2","created":"2024-09-14T09:04:57.077293+0000","modified":"2024-09-14T09:05:19.287473+0000","last_up_change":"2024-09-14T09:05:11.210835+0000","last_in_change":"2024-09-14T09:04:59.344709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T09:05:11.997673+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T09:05:15.352551+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ceedb4f-6398-467f-9562-ccf27200860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6825","nonce":3384710819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6827","nonce":3384710819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6831","nonce":3384710819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6829","nonce":3384710819}]},"public_addr":"172.21.15.176:6825/3384710819","cluster_addr":"172.21.15.176:6827/3384710819","heartbeat_back_addr":"172.21.15.176:6831/3384710819","heartbeat_front_addr":"172.21.15.176:6829/3384710819","state":["exists","up"]},{"osd":1,"uuid":"604e2f62-7c13-46be-a97f-029b28c44a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6801","nonce":612012679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6803","nonce":612012679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6807","nonce":612012679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6805","nonce":612012679}]},"public_addr":"172.21.15.176:6801/612012679","cluster_addr":"172.21.15.176:6803/612012679","heartbeat_back_addr":"172.21.15.176:6807/612012679","heartbeat_front_addr":"172.21.15.176:6805/612012679","state":["exists","up"]},{"osd":2,"uuid":"1f65589e-8b0a-4033-94c0-4ab66531db2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6809","nonce":4167379279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6811","nonce":4167379279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6815","nonce":4167379279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6813","nonce":4167379279}]},"public_addr":"172.21.15.176:6809/4167379279","cluster_addr":"172.21.15.176:6811/4167379279","heartbeat_back_addr":"172.21.15.176:6815/4167379279","heartbeat_front_addr":"172.21.15.176:6813/4167379279","state":["exists","up"]},{"osd":3,"uuid":"209d4f65-79f9-4a7d-9d23-832a4209507f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6817","nonce":1658209015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6819","nonce":1658209015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6823","nonce":1658209015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6821","nonce":1658209015}]},"public_addr":"172.21.15.176:6817/1658209015","cluster_addr":"172.21.15.176:6819/1658209015","heartbeat_back_addr":"172.21.15.176:6823/1658209015","heartbeat_front_addr":"172.21.15.176:6821/1658209015","state":["exists","up"]},{"osd":4,"uuid":"8f9a8c9b-282e-4523-9e62-5824f4aba401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":1576054389},{"type":"v1","addr":"172.21.15.183:6809","nonce":1576054389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":1576054389},{"type":"v1","addr":"172.21.15.183:6811","nonce":1576054389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":1576054389},{"type":"v1","addr":"172.21.15.183:6815","nonce":1576054389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":1576054389},{"type":"v1","addr":"172.21.15.183:6813","nonce":1576054389}]},"public_addr":"172.21.15.183:6809/1576054389","cluster_addr":"172.21.15.183:6811/1576054389","heartbeat_back_addr":"172.21.15.183:6815/1576054389","heartbeat_front_addr":"172.21.15.183:6813/1576054389","state":["exists","up"]},{"osd":5,"uuid":"1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6825","nonce":1711292386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6827","nonce":1711292386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6831","nonce":1711292386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6829","nonce":1711292386}]},"public_addr":"172.21.15.183:6825/1711292386","cluster_addr":"172.21.15.183:6827/1711292386","heartbeat_back_addr":"172.21.15.183:6831/1711292386","heartbeat_front_addr":"172.21.15.183:6829/1711292386","state":["exists","up"]},{"osd":6,"uuid":"59d9259d-b9dd-4326-8ab9-9d599a2bf57b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6817","nonce":2721245607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6819","nonce":2721245607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6823","nonce":2721245607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6821","nonce":2721245607}]},"public_addr":"172.21.15.183:6817/2721245607","cluster_addr":"172.21.15.183:6819/2721245607","heartbeat_back_addr":"172.21.15.183:6823/2721245607","heartbeat_front_addr":"172.21.15.183:6821/2721245607","state":["exists","up"]},{"osd":7,"uuid":"e2f27d42-786b-449b-a2fb-9dd28f4e9788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6801","nonce":2122823465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6803","nonce":2122823465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6807","nonce":2122823465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6805","nonce":2122823465}]},"public_addr":"172.21.15.183:6801/2122823465","cluster_addr":"172.21.15.183:6803/2122823465","heartbeat_back_addr":"172.21.15.183:6807/2122823465","heartbeat_front_addr":"172.21.15.183:6805/2122823465","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.411789+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:05.801873+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.206725+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:05.882180+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.013864+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.652975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.741359+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:05.873675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T09:05:20.646 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-14T09:05:20.647 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-14T09:05:20.647 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-14T09:05:20.648 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-14T09:05:20.648 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-14T09:05:20.648 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-14T09:05:20.649 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-14T09:05:20.649 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-14T09:05:20.908 INFO:teuthology.orchestra.run.smithi176.stdout:60129542149 2024-09-14T09:05:20.908 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T09:05:20.909 INFO:teuthology.orchestra.run.smithi176.stdout:64424509448 2024-09-14T09:05:20.910 INFO:teuthology.orchestra.run.smithi176.stdout:64424509445 2024-09-14T09:05:20.910 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T09:05:20.911 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T09:05:20.911 INFO:teuthology.orchestra.run.smithi176.stdout:60129542148 2024-09-14T09:05:20.912 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T09:05:20.919 INFO:teuthology.orchestra.run.smithi176.stdout:64424509445 2024-09-14T09:05:20.919 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T09:05:20.921 INFO:teuthology.orchestra.run.smithi176.stdout:60129542149 2024-09-14T09:05:20.921 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T09:05:20.930 INFO:teuthology.orchestra.run.smithi176.stdout:60129542150 2024-09-14T09:05:20.931 INFO:teuthology.orchestra.run.smithi176.stdout:64424509446 2024-09-14T09:05:20.931 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T09:05:20.931 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T09:05:21.423 INFO:teuthology.orchestra.run.smithi176.stdout:60129542148 2024-09-14T09:05:21.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.4 2024-09-14T09:05:21.447 INFO:teuthology.orchestra.run.smithi176.stdout:64424509444 2024-09-14T09:05:21.459 INFO:teuthology.orchestra.run.smithi176.stdout:64424509447 2024-09-14T09:05:21.460 INFO:teuthology.orchestra.run.smithi176.stdout:64424509444 2024-09-14T09:05:21.470 INFO:teuthology.orchestra.run.smithi176.stdout:60129542148 2024-09-14T09:05:21.472 INFO:teuthology.orchestra.run.smithi176.stdout:60129542147 2024-09-14T09:05:21.476 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.6 2024-09-14T09:05:21.494 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.3 2024-09-14T09:05:21.495 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.5 2024-09-14T09:05:21.495 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.0 2024-09-14T09:05:21.505 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.7 2024-09-14T09:05:21.505 INFO:teuthology.orchestra.run.smithi176.stdout:64424509445 2024-09-14T09:05:21.516 INFO:teuthology.orchestra.run.smithi176.stdout:60129542149 2024-09-14T09:05:21.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.2 2024-09-14T09:05:21.533 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.1 2024-09-14T09:05:22.446 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T09:05:22.476 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T09:05:22.495 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T09:05:22.496 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T09:05:22.496 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T09:05:22.506 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T09:05:22.524 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T09:05:22.534 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T09:05:22.977 INFO:teuthology.orchestra.run.smithi176.stdout:60129542150 2024-09-14T09:05:23.000 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.4 2024-09-14T09:05:23.000 DEBUG:teuthology.parallel:result is None 2024-09-14T09:05:23.029 INFO:teuthology.orchestra.run.smithi176.stdout:60129542150 2024-09-14T09:05:23.039 INFO:teuthology.orchestra.run.smithi176.stdout:64424509445 2024-09-14T09:05:23.049 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.3 2024-09-14T09:05:23.049 DEBUG:teuthology.parallel:result is None 2024-09-14T09:05:23.066 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.6 2024-09-14T09:05:23.066 DEBUG:teuthology.parallel:result is None 2024-09-14T09:05:23.069 INFO:teuthology.orchestra.run.smithi176.stdout:64424509446 2024-09-14T09:05:23.081 INFO:teuthology.orchestra.run.smithi176.stdout:60129542149 2024-09-14T09:05:23.083 INFO:teuthology.orchestra.run.smithi176.stdout:64424509445 2024-09-14T09:05:23.087 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.2 2024-09-14T09:05:23.087 DEBUG:teuthology.parallel:result is None 2024-09-14T09:05:23.096 INFO:teuthology.orchestra.run.smithi176.stdout:64424509448 2024-09-14T09:05:23.101 INFO:teuthology.orchestra.run.smithi176.stdout:60129542151 2024-09-14T09:05:23.107 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.7 2024-09-14T09:05:23.107 DEBUG:teuthology.parallel:result is None 2024-09-14T09:05:23.112 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.5 2024-09-14T09:05:23.112 DEBUG:teuthology.parallel:result is None 2024-09-14T09:05:23.119 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.0 2024-09-14T09:05:23.120 DEBUG:teuthology.parallel:result is None 2024-09-14T09:05:23.124 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.1 2024-09-14T09:05:23.124 DEBUG:teuthology.parallel:result is None 2024-09-14T09:05:23.124 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T09:05:23.124 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:05:23.440 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:05:23.441 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:05:23.457 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-14T09:05:21.961290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218112,"kb_used_data":1848,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756608,"statfs":{"total":773094113280,"available":772870766592,"internally_reserved":0,"allocated":1892352,"data_stored":1784998,"data_compressed":12424,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.697839"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.793177+0000","last_change":"2024-09-14T09:05:19.793674+0000","last_active":"2024-09-14T09:05:19.793177+0000","last_peered":"2024-09-14T09:05:19.793177+0000","last_clean":"2024-09-14T09:05:19.793177+0000","last_became_active":"2024-09-14T09:05:17.284977+0000","last_became_peered":"2024-09-14T09:05:17.284977+0000","last_unstale":"2024-09-14T09:05:19.793177+0000","last_undegraded":"2024-09-14T09:05:19.793177+0000","last_fullsized":"2024-09-14T09:05:19.793177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:30:38.832027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.314029+0000","last_change":"2024-09-14T09:05:19.314029+0000","last_active":"2024-09-14T09:05:19.314029+0000","last_peered":"2024-09-14T09:05:19.314029+0000","last_clean":"2024-09-14T09:05:19.314029+0000","last_became_active":"2024-09-14T09:05:17.282890+0000","last_became_peered":"2024-09-14T09:05:17.282890+0000","last_unstale":"2024-09-14T09:05:19.314029+0000","last_undegraded":"2024-09-14T09:05:19.314029+0000","last_fullsized":"2024-09-14T09:05:19.314029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:25.304105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.794165+0000","last_change":"2024-09-14T09:05:19.794165+0000","last_active":"2024-09-14T09:05:19.794165+0000","last_peered":"2024-09-14T09:05:19.794165+0000","last_clean":"2024-09-14T09:05:19.794165+0000","last_became_active":"2024-09-14T09:05:17.295864+0000","last_became_peered":"2024-09-14T09:05:17.295864+0000","last_unstale":"2024-09-14T09:05:19.794165+0000","last_undegraded":"2024-09-14T09:05:19.794165+0000","last_fullsized":"2024-09-14T09:05:19.794165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:53.901865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011399719999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.314129+0000","last_change":"2024-09-14T09:05:19.314129+0000","last_active":"2024-09-14T09:05:19.314129+0000","last_peered":"2024-09-14T09:05:19.314129+0000","last_clean":"2024-09-14T09:05:19.314129+0000","last_became_active":"2024-09-14T09:05:17.295196+0000","last_became_peered":"2024-09-14T09:05:17.295196+0000","last_unstale":"2024-09-14T09:05:19.314129+0000","last_undegraded":"2024-09-14T09:05:19.314129+0000","last_fullsized":"2024-09-14T09:05:19.314129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:35.269115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011613800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.312262+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:05:19.312262+0000","last_peered":"2024-09-14T09:05:19.312262+0000","last_clean":"2024-09-14T09:05:19.312262+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:05:19.312262+0000","last_undegraded":"2024-09-14T09:05:19.312262+0000","last_fullsized":"2024-09-14T09:05:19.312262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.302701+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:05:19.302701+0000","last_peered":"2024-09-14T09:05:19.302701+0000","last_clean":"2024-09-14T09:05:19.302701+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:05:19.302701+0000","last_undegraded":"2024-09-14T09:05:19.302701+0000","last_fullsized":"2024-09-14T09:05:19.302701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.793999+0000","last_change":"2024-09-14T09:05:19.793999+0000","last_active":"2024-09-14T09:05:19.793999+0000","last_peered":"2024-09-14T09:05:19.793999+0000","last_clean":"2024-09-14T09:05:19.793999+0000","last_became_active":"2024-09-14T09:05:17.284122+0000","last_became_peered":"2024-09-14T09:05:17.284122+0000","last_unstale":"2024-09-14T09:05:19.793999+0000","last_undegraded":"2024-09-14T09:05:19.793999+0000","last_fullsized":"2024-09-14T09:05:19.793999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:12:01.133215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010234269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.310371+0000","last_change":"2024-09-14T09:05:19.310814+0000","last_active":"2024-09-14T09:05:19.310371+0000","last_peered":"2024-09-14T09:05:19.310371+0000","last_clean":"2024-09-14T09:05:19.310371+0000","last_became_active":"2024-09-14T09:05:17.276973+0000","last_became_peered":"2024-09-14T09:05:17.276973+0000","last_unstale":"2024-09-14T09:05:19.310371+0000","last_undegraded":"2024-09-14T09:05:19.310371+0000","last_fullsized":"2024-09-14T09:05:19.310371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:33:35.187184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086884099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.793037+0000","last_change":"2024-09-14T09:05:13.239808+0000","last_active":"2024-09-14T09:05:19.793037+0000","last_peered":"2024-09-14T09:05:19.793037+0000","last_clean":"2024-09-14T09:05:19.793037+0000","last_became_active":"2024-09-14T09:05:13.239064+0000","last_became_peered":"2024-09-14T09:05:13.239064+0000","last_unstale":"2024-09-14T09:05:19.793037+0000","last_undegraded":"2024-09-14T09:05:19.793037+0000","last_fullsized":"2024-09-14T09:05:19.793037+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:40:11.559161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12424,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":75547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":75528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":665896,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":278528,"data_stored":665896,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":75547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:05:23.458 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:05:23.736 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:05:23.736 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:05:23.752 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-14T09:05:21.961290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218112,"kb_used_data":1848,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756608,"statfs":{"total":773094113280,"available":772870766592,"internally_reserved":0,"allocated":1892352,"data_stored":1784998,"data_compressed":12424,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.697839"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.793177+0000","last_change":"2024-09-14T09:05:19.793674+0000","last_active":"2024-09-14T09:05:19.793177+0000","last_peered":"2024-09-14T09:05:19.793177+0000","last_clean":"2024-09-14T09:05:19.793177+0000","last_became_active":"2024-09-14T09:05:17.284977+0000","last_became_peered":"2024-09-14T09:05:17.284977+0000","last_unstale":"2024-09-14T09:05:19.793177+0000","last_undegraded":"2024-09-14T09:05:19.793177+0000","last_fullsized":"2024-09-14T09:05:19.793177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:30:38.832027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.314029+0000","last_change":"2024-09-14T09:05:19.314029+0000","last_active":"2024-09-14T09:05:19.314029+0000","last_peered":"2024-09-14T09:05:19.314029+0000","last_clean":"2024-09-14T09:05:19.314029+0000","last_became_active":"2024-09-14T09:05:17.282890+0000","last_became_peered":"2024-09-14T09:05:17.282890+0000","last_unstale":"2024-09-14T09:05:19.314029+0000","last_undegraded":"2024-09-14T09:05:19.314029+0000","last_fullsized":"2024-09-14T09:05:19.314029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:25.304105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.794165+0000","last_change":"2024-09-14T09:05:19.794165+0000","last_active":"2024-09-14T09:05:19.794165+0000","last_peered":"2024-09-14T09:05:19.794165+0000","last_clean":"2024-09-14T09:05:19.794165+0000","last_became_active":"2024-09-14T09:05:17.295864+0000","last_became_peered":"2024-09-14T09:05:17.295864+0000","last_unstale":"2024-09-14T09:05:19.794165+0000","last_undegraded":"2024-09-14T09:05:19.794165+0000","last_fullsized":"2024-09-14T09:05:19.794165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:53.901865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011399719999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.314129+0000","last_change":"2024-09-14T09:05:19.314129+0000","last_active":"2024-09-14T09:05:19.314129+0000","last_peered":"2024-09-14T09:05:19.314129+0000","last_clean":"2024-09-14T09:05:19.314129+0000","last_became_active":"2024-09-14T09:05:17.295196+0000","last_became_peered":"2024-09-14T09:05:17.295196+0000","last_unstale":"2024-09-14T09:05:19.314129+0000","last_undegraded":"2024-09-14T09:05:19.314129+0000","last_fullsized":"2024-09-14T09:05:19.314129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:35.269115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011613800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.312262+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:05:19.312262+0000","last_peered":"2024-09-14T09:05:19.312262+0000","last_clean":"2024-09-14T09:05:19.312262+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:05:19.312262+0000","last_undegraded":"2024-09-14T09:05:19.312262+0000","last_fullsized":"2024-09-14T09:05:19.312262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.302701+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:05:19.302701+0000","last_peered":"2024-09-14T09:05:19.302701+0000","last_clean":"2024-09-14T09:05:19.302701+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:05:19.302701+0000","last_undegraded":"2024-09-14T09:05:19.302701+0000","last_fullsized":"2024-09-14T09:05:19.302701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.793999+0000","last_change":"2024-09-14T09:05:19.793999+0000","last_active":"2024-09-14T09:05:19.793999+0000","last_peered":"2024-09-14T09:05:19.793999+0000","last_clean":"2024-09-14T09:05:19.793999+0000","last_became_active":"2024-09-14T09:05:17.284122+0000","last_became_peered":"2024-09-14T09:05:17.284122+0000","last_unstale":"2024-09-14T09:05:19.793999+0000","last_undegraded":"2024-09-14T09:05:19.793999+0000","last_fullsized":"2024-09-14T09:05:19.793999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:12:01.133215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010234269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.310371+0000","last_change":"2024-09-14T09:05:19.310814+0000","last_active":"2024-09-14T09:05:19.310371+0000","last_peered":"2024-09-14T09:05:19.310371+0000","last_clean":"2024-09-14T09:05:19.310371+0000","last_became_active":"2024-09-14T09:05:17.276973+0000","last_became_peered":"2024-09-14T09:05:17.276973+0000","last_unstale":"2024-09-14T09:05:19.310371+0000","last_undegraded":"2024-09-14T09:05:19.310371+0000","last_fullsized":"2024-09-14T09:05:19.310371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:33:35.187184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086884099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.793037+0000","last_change":"2024-09-14T09:05:13.239808+0000","last_active":"2024-09-14T09:05:19.793037+0000","last_peered":"2024-09-14T09:05:19.793037+0000","last_clean":"2024-09-14T09:05:19.793037+0000","last_became_active":"2024-09-14T09:05:13.239064+0000","last_became_peered":"2024-09-14T09:05:13.239064+0000","last_unstale":"2024-09-14T09:05:19.793037+0000","last_undegraded":"2024-09-14T09:05:19.793037+0000","last_fullsized":"2024-09-14T09:05:19.793037+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:40:11.559161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12424,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":75547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":75528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":665896,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":278528,"data_stored":665896,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":75547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:05:23.753 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T09:05:23.753 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-14T09:05:23.753 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-14T09:05:23.753 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-14T09:05:24.066 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:05:24.066 INFO:teuthology.orchestra.run.smithi176.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-14T09:05:24.084 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-14T09:05:24.084 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-14T09:05:24.094 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-14T09:05:24.094 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '0d61375b8ab53be4c233f5b93e463edfdff9316a'}} 2024-09-14T09:05:24.095 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-14T09:05:24.095 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-14T09:05:24.409 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-14T09:05:24.409 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T09:05:24.409 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:05:24.684 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:05:24.684 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:05:24.701 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-14T09:05:23.962041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218112,"kb_used_data":1848,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756608,"statfs":{"total":773094113280,"available":772870766592,"internally_reserved":0,"allocated":1892352,"data_stored":1784998,"data_compressed":12424,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.696068"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.793177+0000","last_change":"2024-09-14T09:05:19.793674+0000","last_active":"2024-09-14T09:05:19.793177+0000","last_peered":"2024-09-14T09:05:19.793177+0000","last_clean":"2024-09-14T09:05:19.793177+0000","last_became_active":"2024-09-14T09:05:17.284977+0000","last_became_peered":"2024-09-14T09:05:17.284977+0000","last_unstale":"2024-09-14T09:05:19.793177+0000","last_undegraded":"2024-09-14T09:05:19.793177+0000","last_fullsized":"2024-09-14T09:05:19.793177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:30:38.832027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.314029+0000","last_change":"2024-09-14T09:05:19.314029+0000","last_active":"2024-09-14T09:05:19.314029+0000","last_peered":"2024-09-14T09:05:19.314029+0000","last_clean":"2024-09-14T09:05:19.314029+0000","last_became_active":"2024-09-14T09:05:17.282890+0000","last_became_peered":"2024-09-14T09:05:17.282890+0000","last_unstale":"2024-09-14T09:05:19.314029+0000","last_undegraded":"2024-09-14T09:05:19.314029+0000","last_fullsized":"2024-09-14T09:05:19.314029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:25.304105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.794165+0000","last_change":"2024-09-14T09:05:19.794165+0000","last_active":"2024-09-14T09:05:19.794165+0000","last_peered":"2024-09-14T09:05:19.794165+0000","last_clean":"2024-09-14T09:05:19.794165+0000","last_became_active":"2024-09-14T09:05:17.295864+0000","last_became_peered":"2024-09-14T09:05:17.295864+0000","last_unstale":"2024-09-14T09:05:19.794165+0000","last_undegraded":"2024-09-14T09:05:19.794165+0000","last_fullsized":"2024-09-14T09:05:19.794165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:53.901865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011399719999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.314129+0000","last_change":"2024-09-14T09:05:19.314129+0000","last_active":"2024-09-14T09:05:19.314129+0000","last_peered":"2024-09-14T09:05:19.314129+0000","last_clean":"2024-09-14T09:05:19.314129+0000","last_became_active":"2024-09-14T09:05:17.295196+0000","last_became_peered":"2024-09-14T09:05:17.295196+0000","last_unstale":"2024-09-14T09:05:19.314129+0000","last_undegraded":"2024-09-14T09:05:19.314129+0000","last_fullsized":"2024-09-14T09:05:19.314129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:35.269115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011613800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.312262+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:05:19.312262+0000","last_peered":"2024-09-14T09:05:19.312262+0000","last_clean":"2024-09-14T09:05:19.312262+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:05:19.312262+0000","last_undegraded":"2024-09-14T09:05:19.312262+0000","last_fullsized":"2024-09-14T09:05:19.312262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.302701+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:05:19.302701+0000","last_peered":"2024-09-14T09:05:19.302701+0000","last_clean":"2024-09-14T09:05:19.302701+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:05:19.302701+0000","last_undegraded":"2024-09-14T09:05:19.302701+0000","last_fullsized":"2024-09-14T09:05:19.302701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.793999+0000","last_change":"2024-09-14T09:05:19.793999+0000","last_active":"2024-09-14T09:05:19.793999+0000","last_peered":"2024-09-14T09:05:19.793999+0000","last_clean":"2024-09-14T09:05:19.793999+0000","last_became_active":"2024-09-14T09:05:17.284122+0000","last_became_peered":"2024-09-14T09:05:17.284122+0000","last_unstale":"2024-09-14T09:05:19.793999+0000","last_undegraded":"2024-09-14T09:05:19.793999+0000","last_fullsized":"2024-09-14T09:05:19.793999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:12:01.133215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010234269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.310371+0000","last_change":"2024-09-14T09:05:19.310814+0000","last_active":"2024-09-14T09:05:19.310371+0000","last_peered":"2024-09-14T09:05:19.310371+0000","last_clean":"2024-09-14T09:05:19.310371+0000","last_became_active":"2024-09-14T09:05:17.276973+0000","last_became_peered":"2024-09-14T09:05:17.276973+0000","last_unstale":"2024-09-14T09:05:19.310371+0000","last_undegraded":"2024-09-14T09:05:19.310371+0000","last_fullsized":"2024-09-14T09:05:19.310371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:33:35.187184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086884099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.793037+0000","last_change":"2024-09-14T09:05:13.239808+0000","last_active":"2024-09-14T09:05:19.793037+0000","last_peered":"2024-09-14T09:05:19.793037+0000","last_clean":"2024-09-14T09:05:19.793037+0000","last_became_active":"2024-09-14T09:05:13.239064+0000","last_became_peered":"2024-09-14T09:05:13.239064+0000","last_unstale":"2024-09-14T09:05:19.793037+0000","last_undegraded":"2024-09-14T09:05:19.793037+0000","last_fullsized":"2024-09-14T09:05:19.793037+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:40:11.559161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12424,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":75547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":75528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":665896,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":278528,"data_stored":665896,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":75547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:05:24.702 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:05:24.987 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:05:24.988 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:05:25.005 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-14T09:05:23.962041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218112,"kb_used_data":1848,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756608,"statfs":{"total":773094113280,"available":772870766592,"internally_reserved":0,"allocated":1892352,"data_stored":1784998,"data_compressed":12424,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.696068"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.793177+0000","last_change":"2024-09-14T09:05:19.793674+0000","last_active":"2024-09-14T09:05:19.793177+0000","last_peered":"2024-09-14T09:05:19.793177+0000","last_clean":"2024-09-14T09:05:19.793177+0000","last_became_active":"2024-09-14T09:05:17.284977+0000","last_became_peered":"2024-09-14T09:05:17.284977+0000","last_unstale":"2024-09-14T09:05:19.793177+0000","last_undegraded":"2024-09-14T09:05:19.793177+0000","last_fullsized":"2024-09-14T09:05:19.793177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:30:38.832027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.314029+0000","last_change":"2024-09-14T09:05:19.314029+0000","last_active":"2024-09-14T09:05:19.314029+0000","last_peered":"2024-09-14T09:05:19.314029+0000","last_clean":"2024-09-14T09:05:19.314029+0000","last_became_active":"2024-09-14T09:05:17.282890+0000","last_became_peered":"2024-09-14T09:05:17.282890+0000","last_unstale":"2024-09-14T09:05:19.314029+0000","last_undegraded":"2024-09-14T09:05:19.314029+0000","last_fullsized":"2024-09-14T09:05:19.314029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:25.304105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.794165+0000","last_change":"2024-09-14T09:05:19.794165+0000","last_active":"2024-09-14T09:05:19.794165+0000","last_peered":"2024-09-14T09:05:19.794165+0000","last_clean":"2024-09-14T09:05:19.794165+0000","last_became_active":"2024-09-14T09:05:17.295864+0000","last_became_peered":"2024-09-14T09:05:17.295864+0000","last_unstale":"2024-09-14T09:05:19.794165+0000","last_undegraded":"2024-09-14T09:05:19.794165+0000","last_fullsized":"2024-09-14T09:05:19.794165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:53.901865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011399719999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.314129+0000","last_change":"2024-09-14T09:05:19.314129+0000","last_active":"2024-09-14T09:05:19.314129+0000","last_peered":"2024-09-14T09:05:19.314129+0000","last_clean":"2024-09-14T09:05:19.314129+0000","last_became_active":"2024-09-14T09:05:17.295196+0000","last_became_peered":"2024-09-14T09:05:17.295196+0000","last_unstale":"2024-09-14T09:05:19.314129+0000","last_undegraded":"2024-09-14T09:05:19.314129+0000","last_fullsized":"2024-09-14T09:05:19.314129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:35.269115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011613800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.312262+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:05:19.312262+0000","last_peered":"2024-09-14T09:05:19.312262+0000","last_clean":"2024-09-14T09:05:19.312262+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:05:19.312262+0000","last_undegraded":"2024-09-14T09:05:19.312262+0000","last_fullsized":"2024-09-14T09:05:19.312262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.302701+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:05:19.302701+0000","last_peered":"2024-09-14T09:05:19.302701+0000","last_clean":"2024-09-14T09:05:19.302701+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:05:19.302701+0000","last_undegraded":"2024-09-14T09:05:19.302701+0000","last_fullsized":"2024-09-14T09:05:19.302701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.793999+0000","last_change":"2024-09-14T09:05:19.793999+0000","last_active":"2024-09-14T09:05:19.793999+0000","last_peered":"2024-09-14T09:05:19.793999+0000","last_clean":"2024-09-14T09:05:19.793999+0000","last_became_active":"2024-09-14T09:05:17.284122+0000","last_became_peered":"2024-09-14T09:05:17.284122+0000","last_unstale":"2024-09-14T09:05:19.793999+0000","last_undegraded":"2024-09-14T09:05:19.793999+0000","last_fullsized":"2024-09-14T09:05:19.793999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:12:01.133215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010234269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.310371+0000","last_change":"2024-09-14T09:05:19.310814+0000","last_active":"2024-09-14T09:05:19.310371+0000","last_peered":"2024-09-14T09:05:19.310371+0000","last_clean":"2024-09-14T09:05:19.310371+0000","last_became_active":"2024-09-14T09:05:17.276973+0000","last_became_peered":"2024-09-14T09:05:17.276973+0000","last_unstale":"2024-09-14T09:05:19.310371+0000","last_undegraded":"2024-09-14T09:05:19.310371+0000","last_fullsized":"2024-09-14T09:05:19.310371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:33:35.187184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086884099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T09:05:19.793037+0000","last_change":"2024-09-14T09:05:13.239808+0000","last_active":"2024-09-14T09:05:19.793037+0000","last_peered":"2024-09-14T09:05:19.793037+0000","last_clean":"2024-09-14T09:05:19.793037+0000","last_became_active":"2024-09-14T09:05:13.239064+0000","last_became_peered":"2024-09-14T09:05:13.239064+0000","last_unstale":"2024-09-14T09:05:19.793037+0000","last_undegraded":"2024-09-14T09:05:19.793037+0000","last_fullsized":"2024-09-14T09:05:19.793037+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:40:11.559161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12424,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":75547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":75528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":665896,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":278528,"data_stored":665896,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":75547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:05:25.006 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T09:05:25.006 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-14T09:05:25.286 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 22 2024-09-14T09:05:25.286 INFO:teuthology.orchestra.run.smithi176.stdout:fsid c635667d-994e-454f-bcea-51b45637b2a2 2024-09-14T09:05:25.286 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-09-14T09:04:57.077293+0000 2024-09-14T09:05:25.286 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-09-14T09:05:19.287473+0000 2024-09-14T09:05:25.286 INFO:teuthology.orchestra.run.smithi176.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-14T09:05:25.286 INFO:teuthology.orchestra.run.smithi176.stdout:crush_version 5 2024-09-14T09:05:25.286 INFO:teuthology.orchestra.run.smithi176.stdout:full_ratio 0.95 2024-09-14T09:05:25.286 INFO:teuthology.orchestra.run.smithi176.stdout:backfillfull_ratio 0.9 2024-09-14T09:05:25.286 INFO:teuthology.orchestra.run.smithi176.stdout:nearfull_ratio 0.85 2024-09-14T09:05:25.286 INFO:teuthology.orchestra.run.smithi176.stdout:require_min_compat_client luminous 2024-09-14T09:05:25.286 INFO:teuthology.orchestra.run.smithi176.stdout:min_compat_client jewel 2024-09-14T09:05:25.287 INFO:teuthology.orchestra.run.smithi176.stdout:require_osd_release squid 2024-09-14T09:05:25.287 INFO:teuthology.orchestra.run.smithi176.stdout:stretch_mode_enabled false 2024-09-14T09:05:25.287 INFO:teuthology.orchestra.run.smithi176.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-14T09:05:25.287 INFO:teuthology.orchestra.run.smithi176.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 22 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-14T09:05:25.287 INFO:teuthology.orchestra.run.smithi176.stdout: removed_snaps_queue [2~1] 2024-09-14T09:05:25.287 INFO:teuthology.orchestra.run.smithi176.stdout:max_osd 8 2024-09-14T09:05:25.287 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6824/3384710819,v1:172.21.15.176:6825/3384710819] [v2:172.21.15.176:6826/3384710819,v1:172.21.15.176:6827/3384710819] exists,up 4ceedb4f-6398-467f-9562-ccf27200860b 2024-09-14T09:05:25.287 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6800/612012679,v1:172.21.15.176:6801/612012679] [v2:172.21.15.176:6802/612012679,v1:172.21.15.176:6803/612012679] exists,up 604e2f62-7c13-46be-a97f-029b28c44a89 2024-09-14T09:05:25.288 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6808/4167379279,v1:172.21.15.176:6809/4167379279] [v2:172.21.15.176:6810/4167379279,v1:172.21.15.176:6811/4167379279] exists,up 1f65589e-8b0a-4033-94c0-4ab66531db2b 2024-09-14T09:05:25.288 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6816/1658209015,v1:172.21.15.176:6817/1658209015] [v2:172.21.15.176:6818/1658209015,v1:172.21.15.176:6819/1658209015] exists,up 209d4f65-79f9-4a7d-9d23-832a4209507f 2024-09-14T09:05:25.288 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/1576054389,v1:172.21.15.183:6809/1576054389] [v2:172.21.15.183:6810/1576054389,v1:172.21.15.183:6811/1576054389] exists,up 8f9a8c9b-282e-4523-9e62-5824f4aba401 2024-09-14T09:05:25.288 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/1711292386,v1:172.21.15.183:6825/1711292386] [v2:172.21.15.183:6826/1711292386,v1:172.21.15.183:6827/1711292386] exists,up 1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80 2024-09-14T09:05:25.288 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/2721245607,v1:172.21.15.183:6817/2721245607] [v2:172.21.15.183:6818/2721245607,v1:172.21.15.183:6819/2721245607] exists,up 59d9259d-b9dd-4326-8ab9-9d599a2bf57b 2024-09-14T09:05:25.288 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/2122823465,v1:172.21.15.183:6801/2122823465] [v2:172.21.15.183:6802/2122823465,v1:172.21.15.183:6803/2122823465] exists,up e2f27d42-786b-449b-a2fb-9dd28f4e9788 2024-09-14T09:05:25.303 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6824/3384710819,v1:172.21.15.176:6825/3384710819] [v2:172.21.15.176:6826/3384710819,v1:172.21.15.176:6827/3384710819] exists,up 4ceedb4f-6398-467f-9562-ccf27200860b', 'osd.1 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6800/612012679,v1:172.21.15.176:6801/612012679] [v2:172.21.15.176:6802/612012679,v1:172.21.15.176:6803/612012679] exists,up 604e2f62-7c13-46be-a97f-029b28c44a89', 'osd.2 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6808/4167379279,v1:172.21.15.176:6809/4167379279] [v2:172.21.15.176:6810/4167379279,v1:172.21.15.176:6811/4167379279] exists,up 1f65589e-8b0a-4033-94c0-4ab66531db2b', 'osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6816/1658209015,v1:172.21.15.176:6817/1658209015] [v2:172.21.15.176:6818/1658209015,v1:172.21.15.176:6819/1658209015] exists,up 209d4f65-79f9-4a7d-9d23-832a4209507f', 'osd.4 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/1576054389,v1:172.21.15.183:6809/1576054389] [v2:172.21.15.183:6810/1576054389,v1:172.21.15.183:6811/1576054389] exists,up 8f9a8c9b-282e-4523-9e62-5824f4aba401', 'osd.5 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/1711292386,v1:172.21.15.183:6825/1711292386] [v2:172.21.15.183:6826/1711292386,v1:172.21.15.183:6827/1711292386] exists,up 1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80', 'osd.6 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/2721245607,v1:172.21.15.183:6817/2721245607] [v2:172.21.15.183:6818/2721245607,v1:172.21.15.183:6819/2721245607] exists,up 59d9259d-b9dd-4326-8ab9-9d599a2bf57b', 'osd.7 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/2122823465,v1:172.21.15.183:6801/2122823465] [v2:172.21.15.183:6802/2122823465,v1:172.21.15.183:6803/2122823465] exists,up e2f27d42-786b-449b-a2fb-9dd28f4e9788'] 2024-09-14T09:05:25.304 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-14T09:05:25.407 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:25.407 INFO:teuthology.orchestra.run.smithi176.stdout: "name": "mon.a", 2024-09-14T09:05:25.407 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster": "ceph", 2024-09-14T09:05:25.407 INFO:teuthology.orchestra.run.smithi176.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-14T09:05:25.407 INFO:teuthology.orchestra.run.smithi176.stdout: "admin_socket_mode": "", 2024-09-14T09:05:25.407 INFO:teuthology.orchestra.run.smithi176.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-14T09:05:25.407 INFO:teuthology.orchestra.run.smithi176.stdout: "auth_client_required": "cephx, none", 2024-09-14T09:05:25.407 INFO:teuthology.orchestra.run.smithi176.stdout: "auth_cluster_required": "cephx", 2024-09-14T09:05:25.408 INFO:teuthology.orchestra.run.smithi176.stdout: "auth_debug": "true", 2024-09-14T09:05:25.408 INFO:teuthology.orchestra.run.smithi176.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-14T09:05:25.408 INFO:teuthology.orchestra.run.smithi176.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-14T09:05:25.408 INFO:teuthology.orchestra.run.smithi176.stdout: "auth_service_required": "cephx", 2024-09-14T09:05:25.408 INFO:teuthology.orchestra.run.smithi176.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-14T09:05:25.408 INFO:teuthology.orchestra.run.smithi176.stdout: "auth_supported": "cephx", 2024-09-14T09:05:25.408 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_aio": "true", 2024-09-14T09:05:25.408 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-14T09:05:25.408 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_aio_poll_ms": "250", 2024-09-14T09:05:25.408 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_aio_reap_max": "16", 2024-09-14T09:05:25.408 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_async_discard_threads": "0", 2024-09-14T09:05:25.409 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_block_size": "4096", 2024-09-14T09:05:25.409 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_debug_aio": "false", 2024-09-14T09:05:25.409 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-14T09:05:25.409 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-14T09:05:25.409 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_debug_inflight_ios": "false", 2024-09-14T09:05:25.409 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_enable_discard": "false", 2024-09-14T09:05:25.409 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_flock_retry": "3", 2024-09-14T09:05:25.409 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-14T09:05:25.409 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_inject_crash": "0", 2024-09-14T09:05:25.409 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-14T09:05:25.409 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_ioring": "false", 2024-09-14T09:05:25.409 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_ioring_hipri": "false", 2024-09-14T09:05:25.409 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-14T09:05:25.410 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-14T09:05:25.410 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-14T09:05:25.410 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-14T09:05:25.410 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-14T09:05:25.410 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-14T09:05:25.410 INFO:teuthology.orchestra.run.smithi176.stdout: "bdev_type": "", 2024-09-14T09:05:25.410 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_alloc_size": "1048576", 2024-09-14T09:05:25.410 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_allocator": "hybrid", 2024-09-14T09:05:25.410 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_buffered_io": "true", 2024-09-14T09:05:25.410 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_check_for_zeros": "false", 2024-09-14T09:05:25.410 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-14T09:05:25.410 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-14T09:05:25.410 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_compact_log_sync": "false", 2024-09-14T09:05:25.410 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-14T09:05:25.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-14T09:05:25.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-14T09:05:25.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-14T09:05:25.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_max_log_runway": "4194304", 2024-09-14T09:05:25.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_max_prefetch": "1048576", 2024-09-14T09:05:25.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_min_flush_size": "524288", 2024-09-14T09:05:25.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_min_log_runway": "1048576", 2024-09-14T09:05:25.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_replay_recovery": "false", 2024-09-14T09:05:25.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-14T09:05:25.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-14T09:05:25.411 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_sync_write": "false", 2024-09-14T09:05:25.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-14T09:05:25.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-14T09:05:25.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-14T09:05:25.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_allocation_from_file": "true", 2024-09-14T09:05:25.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_allocator": "hybrid", 2024-09-14T09:05:25.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_async_db_compaction": "true", 2024-09-14T09:05:25.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-14T09:05:25.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-14T09:05:25.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-14T09:05:25.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-14T09:05:25.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_bdev_label_multi": "true", 2024-09-14T09:05:25.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-14T09:05:25.412 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-14T09:05:25.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-14T09:05:25.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-14T09:05:25.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-14T09:05:25.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_block_create": "true", 2024-09-14T09:05:25.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_block_db_create": "false", 2024-09-14T09:05:25.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_block_db_path": "", 2024-09-14T09:05:25.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_block_db_size": "0", 2024-09-14T09:05:25.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_block_path": "", 2024-09-14T09:05:25.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_block_preallocate_file": "false", 2024-09-14T09:05:25.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_block_size": "107374182400", 2024-09-14T09:05:25.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_block_wal_create": "false", 2024-09-14T09:05:25.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_block_wal_path": "", 2024-09-14T09:05:25.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_block_wal_size": "100663296", 2024-09-14T09:05:25.413 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_bluefs": "true", 2024-09-14T09:05:25.414 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-14T09:05:25.414 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-14T09:05:25.414 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-14T09:05:25.414 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-14T09:05:25.414 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-14T09:05:25.414 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T09:05:25.414 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T09:05:25.414 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-14T09:05:25.414 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T09:05:25.414 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_autotune": "true", 2024-09-14T09:05:25.414 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-14T09:05:25.414 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-14T09:05:25.414 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-14T09:05:25.415 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-14T09:05:25.415 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_size": "0", 2024-09-14T09:05:25.415 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-14T09:05:25.415 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-14T09:05:25.415 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-14T09:05:25.415 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-14T09:05:25.415 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_type": "2q", 2024-09-14T09:05:25.415 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-14T09:05:25.415 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_clone_cow": "true", 2024-09-14T09:05:25.415 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-14T09:05:25.415 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-14T09:05:25.415 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-14T09:05:25.415 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-14T09:05:25.415 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-14T09:05:25.416 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-14T09:05:25.416 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-14T09:05:25.416 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_compression_mode": "none", 2024-09-14T09:05:25.416 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-14T09:05:25.416 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_csum_type": "crc32c", 2024-09-14T09:05:25.416 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-14T09:05:25.416 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-14T09:05:25.416 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_debug_freelist": "false", 2024-09-14T09:05:25.416 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-14T09:05:25.416 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-14T09:05:25.416 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-14T09:05:25.416 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-14T09:05:25.416 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-14T09:05:25.417 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-14T09:05:25.417 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-14T09:05:25.417 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-14T09:05:25.417 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-14T09:05:25.417 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-14T09:05:25.417 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-14T09:05:25.417 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-14T09:05:25.417 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_debug_small_allocations": "0", 2024-09-14T09:05:25.417 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-14T09:05:25.417 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_default_buffered_read": "true", 2024-09-14T09:05:25.417 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_default_buffered_write": "false", 2024-09-14T09:05:25.417 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-14T09:05:25.417 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-14T09:05:25.418 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-14T09:05:25.418 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-14T09:05:25.418 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-14T09:05:25.418 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-14T09:05:25.418 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-14T09:05:25.418 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-14T09:05:25.418 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-14T09:05:25.418 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fail_eio": "false", 2024-09-14T09:05:25.418 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-14T09:05:25.418 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-14T09:05:25.418 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-14T09:05:25.418 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-14T09:05:25.418 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-14T09:05:25.419 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-14T09:05:25.419 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_on_mount": "false", 2024-09-14T09:05:25.419 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-14T09:05:25.419 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_on_umount": "false", 2024-09-14T09:05:25.419 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-14T09:05:25.419 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-14T09:05:25.419 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-14T09:05:25.419 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-14T09:05:25.419 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-14T09:05:25.419 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-14T09:05:25.420 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-14T09:05:25.420 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-14T09:05:25.420 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_ignore_data_csum": "false", 2024-09-14T09:05:25.420 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-14T09:05:25.420 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-14T09:05:25.420 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-14T09:05:25.420 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-14T09:05:25.420 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_log_op_age": "5.000000", 2024-09-14T09:05:25.420 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_max_alloc_size": "0", 2024-09-14T09:05:25.420 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_max_blob_size": "0", 2024-09-14T09:05:25.420 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-14T09:05:25.420 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-14T09:05:25.420 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-14T09:05:25.420 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_max_deferred_txc": "32", 2024-09-14T09:05:25.421 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_min_alloc_size": "0", 2024-09-14T09:05:25.421 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-14T09:05:25.421 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-14T09:05:25.421 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_nid_prealloc": "1024", 2024-09-14T09:05:25.421 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-14T09:05:25.421 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-14T09:05:25.421 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-14T09:05:25.421 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-14T09:05:25.421 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_retry_disk_reads": "3", 2024-09-14T09:05:25.421 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_rocksdb_cf": "true", 2024-09-14T09:05:25.421 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-14T09:05:25.421 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-14T09:05:25.421 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-14T09:05:25.422 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-14T09:05:25.422 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-14T09:05:25.422 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-14T09:05:25.422 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-14T09:05:25.422 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-14T09:05:25.422 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_spdk_mem": "512", 2024-09-14T09:05:25.422 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-14T09:05:25.422 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-14T09:05:25.422 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-14T09:05:25.422 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-14T09:05:25.422 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-14T09:05:25.422 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-14T09:05:25.422 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-14T09:05:25.422 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_tracing": "false", 2024-09-14T09:05:25.423 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-14T09:05:25.423 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-14T09:05:25.423 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-14T09:05:25.423 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-14T09:05:25.423 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-14T09:05:25.423 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-14T09:05:25.423 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-14T09:05:25.423 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-14T09:05:25.423 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-14T09:05:25.423 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_write_v2": "false", 2024-09-14T09:05:25.423 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_write_v2_random": "false", 2024-09-14T09:05:25.423 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_zero_block_detection": "false", 2024-09-14T09:05:25.423 INFO:teuthology.orchestra.run.smithi176.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-14T09:05:25.423 INFO:teuthology.orchestra.run.smithi176.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-14T09:05:25.424 INFO:teuthology.orchestra.run.smithi176.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-14T09:05:25.424 INFO:teuthology.orchestra.run.smithi176.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-14T09:05:25.424 INFO:teuthology.orchestra.run.smithi176.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-14T09:05:25.424 INFO:teuthology.orchestra.run.smithi176.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-14T09:05:25.424 INFO:teuthology.orchestra.run.smithi176.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-14T09:05:25.424 INFO:teuthology.orchestra.run.smithi176.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-14T09:05:25.424 INFO:teuthology.orchestra.run.smithi176.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-14T09:05:25.424 INFO:teuthology.orchestra.run.smithi176.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-14T09:05:25.424 INFO:teuthology.orchestra.run.smithi176.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-14T09:05:25.424 INFO:teuthology.orchestra.run.smithi176.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-14T09:05:25.424 INFO:teuthology.orchestra.run.smithi176.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-14T09:05:25.425 INFO:teuthology.orchestra.run.smithi176.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-14T09:05:25.425 INFO:teuthology.orchestra.run.smithi176.stdout: "cephx_cluster_require_signatures": "false", 2024-09-14T09:05:25.425 INFO:teuthology.orchestra.run.smithi176.stdout: "cephx_cluster_require_version": "2", 2024-09-14T09:05:25.425 INFO:teuthology.orchestra.run.smithi176.stdout: "cephx_require_signatures": "false", 2024-09-14T09:05:25.425 INFO:teuthology.orchestra.run.smithi176.stdout: "cephx_require_version": "2", 2024-09-14T09:05:25.425 INFO:teuthology.orchestra.run.smithi176.stdout: "cephx_service_require_signatures": "false", 2024-09-14T09:05:25.425 INFO:teuthology.orchestra.run.smithi176.stdout: "cephx_service_require_version": "2", 2024-09-14T09:05:25.425 INFO:teuthology.orchestra.run.smithi176.stdout: "cephx_sign_messages": "true", 2024-09-14T09:05:25.425 INFO:teuthology.orchestra.run.smithi176.stdout: "chdir": "", 2024-09-14T09:05:25.425 INFO:teuthology.orchestra.run.smithi176.stdout: "client_acl_type": "", 2024-09-14T09:05:25.425 INFO:teuthology.orchestra.run.smithi176.stdout: "client_asio_thread_count": "2", 2024-09-14T09:05:25.425 INFO:teuthology.orchestra.run.smithi176.stdout: "client_cache_mid": "0.750000", 2024-09-14T09:05:25.425 INFO:teuthology.orchestra.run.smithi176.stdout: "client_cache_size": "16384", 2024-09-14T09:05:25.425 INFO:teuthology.orchestra.run.smithi176.stdout: "client_caps_release_delay": "5", 2024-09-14T09:05:25.426 INFO:teuthology.orchestra.run.smithi176.stdout: "client_check_pool_perm": "true", 2024-09-14T09:05:25.426 INFO:teuthology.orchestra.run.smithi176.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-14T09:05:25.426 INFO:teuthology.orchestra.run.smithi176.stdout: "client_debug_force_sync_read": "false", 2024-09-14T09:05:25.426 INFO:teuthology.orchestra.run.smithi176.stdout: "client_debug_getattr_caps": "false", 2024-09-14T09:05:25.426 INFO:teuthology.orchestra.run.smithi176.stdout: "client_debug_inject_features": "", 2024-09-14T09:05:25.426 INFO:teuthology.orchestra.run.smithi176.stdout: "client_debug_inject_tick_delay": "0", 2024-09-14T09:05:25.426 INFO:teuthology.orchestra.run.smithi176.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-14T09:05:25.426 INFO:teuthology.orchestra.run.smithi176.stdout: "client_die_on_failed_remount": "false", 2024-09-14T09:05:25.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_dirsize_rbytes": "true", 2024-09-14T09:05:25.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_force_lazyio": "false", 2024-09-14T09:05:25.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_fs": "", 2024-09-14T09:05:25.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-14T09:05:25.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_inject_release_failure": "false", 2024-09-14T09:05:25.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_max_inline_size": "4096", 2024-09-14T09:05:25.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-14T09:05:25.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_mds_namespace": "", 2024-09-14T09:05:25.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_metadata": "", 2024-09-14T09:05:25.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_mount_gid": "-1", 2024-09-14T09:05:25.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_mount_timeout": "300", 2024-09-14T09:05:25.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_mount_uid": "-1", 2024-09-14T09:05:25.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_mountpoint": "/", 2024-09-14T09:05:25.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_notify_timeout": "10", 2024-09-14T09:05:25.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_oc": "true", 2024-09-14T09:05:25.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_oc_max_dirty": "104857600", 2024-09-14T09:05:25.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-14T09:05:25.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_oc_max_objects": "1000", 2024-09-14T09:05:25.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_oc_size": "209715200", 2024-09-14T09:05:25.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_oc_target_dirty": "8388608", 2024-09-14T09:05:25.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_permissions": "true", 2024-09-14T09:05:25.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_quota": "true", 2024-09-14T09:05:25.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_quota_df": "true", 2024-09-14T09:05:25.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_readahead_max_bytes": "0", 2024-09-14T09:05:25.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_readahead_max_periods": "4", 2024-09-14T09:05:25.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_readahead_min": "131072", 2024-09-14T09:05:25.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_reconnect_stale": "false", 2024-09-14T09:05:25.429 INFO:teuthology.orchestra.run.smithi176.stdout: "client_shutdown_timeout": "30", 2024-09-14T09:05:25.429 INFO:teuthology.orchestra.run.smithi176.stdout: "client_snapdir": ".snap", 2024-09-14T09:05:25.429 INFO:teuthology.orchestra.run.smithi176.stdout: "client_tick_interval": "1", 2024-09-14T09:05:25.429 INFO:teuthology.orchestra.run.smithi176.stdout: "client_trace": "", 2024-09-14T09:05:25.429 INFO:teuthology.orchestra.run.smithi176.stdout: "client_try_dentry_invalidate": "false", 2024-09-14T09:05:25.429 INFO:teuthology.orchestra.run.smithi176.stdout: "client_use_faked_inos": "false", 2024-09-14T09:05:25.429 INFO:teuthology.orchestra.run.smithi176.stdout: "client_use_random_mds": "false", 2024-09-14T09:05:25.429 INFO:teuthology.orchestra.run.smithi176.stdout: "clog_to_graylog": "false", 2024-09-14T09:05:25.429 INFO:teuthology.orchestra.run.smithi176.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-14T09:05:25.429 INFO:teuthology.orchestra.run.smithi176.stdout: "clog_to_graylog_port": "12201", 2024-09-14T09:05:25.429 INFO:teuthology.orchestra.run.smithi176.stdout: "clog_to_monitors": "default=true", 2024-09-14T09:05:25.429 INFO:teuthology.orchestra.run.smithi176.stdout: "clog_to_syslog": "false", 2024-09-14T09:05:25.429 INFO:teuthology.orchestra.run.smithi176.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-14T09:05:25.429 INFO:teuthology.orchestra.run.smithi176.stdout: "clog_to_syslog_level": "info", 2024-09-14T09:05:25.430 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "-", 2024-09-14T09:05:25.430 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_network": "", 2024-09-14T09:05:25.430 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_network_interface": "", 2024-09-14T09:05:25.430 INFO:teuthology.orchestra.run.smithi176.stdout: "compressor_zlib_isal": "false", 2024-09-14T09:05:25.430 INFO:teuthology.orchestra.run.smithi176.stdout: "compressor_zlib_level": "5", 2024-09-14T09:05:25.430 INFO:teuthology.orchestra.run.smithi176.stdout: "compressor_zlib_winsize": "-15", 2024-09-14T09:05:25.430 INFO:teuthology.orchestra.run.smithi176.stdout: "compressor_zstd_level": "1", 2024-09-14T09:05:25.430 INFO:teuthology.orchestra.run.smithi176.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-14T09:05:25.430 INFO:teuthology.orchestra.run.smithi176.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-14T09:05:25.430 INFO:teuthology.orchestra.run.smithi176.stdout: "crimson_alien_op_num_threads": "6", 2024-09-14T09:05:25.430 INFO:teuthology.orchestra.run.smithi176.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-14T09:05:25.430 INFO:teuthology.orchestra.run.smithi176.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-14T09:05:25.430 INFO:teuthology.orchestra.run.smithi176.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-14T09:05:25.430 INFO:teuthology.orchestra.run.smithi176.stdout: "crimson_osd_stat_interval": "0", 2024-09-14T09:05:25.430 INFO:teuthology.orchestra.run.smithi176.stdout: "crimson_seastar_cpu_cores": "", 2024-09-14T09:05:25.431 INFO:teuthology.orchestra.run.smithi176.stdout: "crimson_seastar_num_threads": "0", 2024-09-14T09:05:25.431 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_location": "", 2024-09-14T09:05:25.431 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_location_hook": "", 2024-09-14T09:05:25.431 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_location_hook_timeout": "10", 2024-09-14T09:05:25.431 INFO:teuthology.orchestra.run.smithi176.stdout: "daemonize": "false", 2024-09-14T09:05:25.431 INFO:teuthology.orchestra.run.smithi176.stdout: "daos_pool": "tank", 2024-09-14T09:05:25.431 INFO:teuthology.orchestra.run.smithi176.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-14T09:05:25.431 INFO:teuthology.orchestra.run.smithi176.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-14T09:05:25.431 INFO:teuthology.orchestra.run.smithi176.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-14T09:05:25.431 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_alienstore": "0/5", 2024-09-14T09:05:25.431 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_allow_any_pool_priority": "false", 2024-09-14T09:05:25.431 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_asok": "1/5", 2024-09-14T09:05:25.431 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_asok_assert_abort": "false", 2024-09-14T09:05:25.431 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_asserts_on_shutdown": "true", 2024-09-14T09:05:25.432 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_auth": "20/20", 2024-09-14T09:05:25.432 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_bdev": "1/3", 2024-09-14T09:05:25.432 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_bluefs": "1/5", 2024-09-14T09:05:25.432 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_bluestore": "1/5", 2024-09-14T09:05:25.432 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_buffer": "0/1", 2024-09-14T09:05:25.432 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_ceph_dedup": "0/5", 2024-09-14T09:05:25.432 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_ceph_exporter": "1/5", 2024-09-14T09:05:25.432 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_cephfs_mirror": "0/5", 2024-09-14T09:05:25.432 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_cephsqlite": "0/5", 2024-09-14T09:05:25.432 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_client": "0/5", 2024-09-14T09:05:25.432 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_compressor": "1/5", 2024-09-14T09:05:25.432 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_context": "0/1", 2024-09-14T09:05:25.432 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_crimson_interrupt": "0/5", 2024-09-14T09:05:25.433 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_crush": "1/1", 2024-09-14T09:05:25.433 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_crypto": "1/5", 2024-09-14T09:05:25.433 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_cyanstore": "0/5", 2024-09-14T09:05:25.433 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_deliberately_leak_memory": "false", 2024-09-14T09:05:25.433 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_disable_randomized_ping": "false", 2024-09-14T09:05:25.433 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_dpdk": "1/5", 2024-09-14T09:05:25.433 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_eventtrace": "1/5", 2024-09-14T09:05:25.433 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_filer": "0/1", 2024-09-14T09:05:25.433 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_filestore": "1/3", 2024-09-14T09:05:25.433 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_finisher": "1/1", 2024-09-14T09:05:25.433 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_fuse": "1/5", 2024-09-14T09:05:25.433 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_heartbeat_testing_span": "0", 2024-09-14T09:05:25.433 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_heartbeatmap": "1/5", 2024-09-14T09:05:25.433 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-14T09:05:25.433 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_javaclient": "1/5", 2024-09-14T09:05:25.434 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_journal": "1/3", 2024-09-14T09:05:25.434 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_journaler": "0/5", 2024-09-14T09:05:25.434 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_kstore": "1/5", 2024-09-14T09:05:25.434 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_lockdep": "0/1", 2024-09-14T09:05:25.434 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_mclock": "1/5", 2024-09-14T09:05:25.434 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_mds": "1/5", 2024-09-14T09:05:25.434 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_mds_balancer": "1/5", 2024-09-14T09:05:25.434 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_mds_locker": "1/5", 2024-09-14T09:05:25.434 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_mds_log": "1/5", 2024-09-14T09:05:25.434 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_mds_log_expire": "1/5", 2024-09-14T09:05:25.435 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_mds_migrator": "1/5", 2024-09-14T09:05:25.435 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_mds_quiesce": "3/5", 2024-09-14T09:05:25.435 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_memstore": "1/5", 2024-09-14T09:05:25.435 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_mgr": "2/5", 2024-09-14T09:05:25.435 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_mgrc": "1/5", 2024-09-14T09:05:25.435 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_mon": "20/20", 2024-09-14T09:05:25.435 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_monc": "0/10", 2024-09-14T09:05:25.435 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_ms": "1/1", 2024-09-14T09:05:25.435 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_none": "0/5", 2024-09-14T09:05:25.435 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_objclass": "0/5", 2024-09-14T09:05:25.435 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_objectcacher": "0/5", 2024-09-14T09:05:25.435 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_objecter": "0/1", 2024-09-14T09:05:25.435 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_optracker": "0/5", 2024-09-14T09:05:25.435 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_osd": "1/5", 2024-09-14T09:05:25.436 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_paxos": "20/20", 2024-09-14T09:05:25.436 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_perfcounter": "1/5", 2024-09-14T09:05:25.436 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_prioritycache": "1/5", 2024-09-14T09:05:25.436 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_rados": "0/5", 2024-09-14T09:05:25.436 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_rbd": "0/5", 2024-09-14T09:05:25.436 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_rbd_mirror": "0/5", 2024-09-14T09:05:25.436 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_rbd_pwl": "0/5", 2024-09-14T09:05:25.436 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_rbd_replay": "0/5", 2024-09-14T09:05:25.436 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_refs": "0/0", 2024-09-14T09:05:25.436 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_reserver": "1/1", 2024-09-14T09:05:25.436 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_rgw": "1/5", 2024-09-14T09:05:25.436 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_rgw_access": "1/5", 2024-09-14T09:05:25.436 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_rgw_datacache": "1/5", 2024-09-14T09:05:25.437 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_rgw_dbstore": "1/5", 2024-09-14T09:05:25.437 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_rgw_flight": "1/5", 2024-09-14T09:05:25.437 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-14T09:05:25.437 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_rgw_notification": "1/5", 2024-09-14T09:05:25.437 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_rgw_sync": "1/5", 2024-09-14T09:05:25.437 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_rocksdb": "4/5", 2024-09-14T09:05:25.437 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_seastore": "0/5", 2024-09-14T09:05:25.437 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_seastore_backref": "0/5", 2024-09-14T09:05:25.437 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_seastore_cache": "0/5", 2024-09-14T09:05:25.437 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_seastore_cleaner": "0/5", 2024-09-14T09:05:25.437 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_seastore_device": "0/5", 2024-09-14T09:05:25.437 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_seastore_epm": "0/5", 2024-09-14T09:05:25.437 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-14T09:05:25.437 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_seastore_journal": "0/5", 2024-09-14T09:05:25.438 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_seastore_lba": "0/5", 2024-09-14T09:05:25.438 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_seastore_odata": "0/5", 2024-09-14T09:05:25.438 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_seastore_omap": "0/5", 2024-09-14T09:05:25.438 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_seastore_onode": "0/5", 2024-09-14T09:05:25.438 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_seastore_t": "0/5", 2024-09-14T09:05:25.438 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_seastore_tm": "0/5", 2024-09-14T09:05:25.438 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_striper": "0/1", 2024-09-14T09:05:25.438 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_test": "0/5", 2024-09-14T09:05:25.438 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_throttle": "1/1", 2024-09-14T09:05:25.438 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_timer": "0/1", 2024-09-14T09:05:25.438 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_tp": "0/5", 2024-09-14T09:05:25.438 INFO:teuthology.orchestra.run.smithi176.stdout: "debug_trace": "1/5", 2024-09-14T09:05:25.438 INFO:teuthology.orchestra.run.smithi176.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-14T09:05:25.438 INFO:teuthology.orchestra.run.smithi176.stdout: "device_failure_prediction_mode": "none", 2024-09-14T09:05:25.438 INFO:teuthology.orchestra.run.smithi176.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-14T09:05:25.439 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-14T09:05:25.439 INFO:teuthology.orchestra.run.smithi176.stdout: "err_to_graylog": "false", 2024-09-14T09:05:25.439 INFO:teuthology.orchestra.run.smithi176.stdout: "err_to_journald": "false", 2024-09-14T09:05:25.439 INFO:teuthology.orchestra.run.smithi176.stdout: "err_to_stderr": "true", 2024-09-14T09:05:25.439 INFO:teuthology.orchestra.run.smithi176.stdout: "err_to_syslog": "false", 2024-09-14T09:05:25.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event_tracing": "false", 2024-09-14T09:05:25.439 INFO:teuthology.orchestra.run.smithi176.stdout: "exporter_addr": "0.0.0.0", 2024-09-14T09:05:25.439 INFO:teuthology.orchestra.run.smithi176.stdout: "exporter_cert_file": "", 2024-09-14T09:05:25.439 INFO:teuthology.orchestra.run.smithi176.stdout: "exporter_http_port": "9926", 2024-09-14T09:05:25.439 INFO:teuthology.orchestra.run.smithi176.stdout: "exporter_key_file": "", 2024-09-14T09:05:25.439 INFO:teuthology.orchestra.run.smithi176.stdout: "exporter_prio_limit": "5", 2024-09-14T09:05:25.439 INFO:teuthology.orchestra.run.smithi176.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-14T09:05:25.439 INFO:teuthology.orchestra.run.smithi176.stdout: "exporter_sort_metrics": "true", 2024-09-14T09:05:25.439 INFO:teuthology.orchestra.run.smithi176.stdout: "exporter_stats_period": "5", 2024-09-14T09:05:25.439 INFO:teuthology.orchestra.run.smithi176.stdout: "fake_statfs_for_testing": "0", 2024-09-14T09:05:25.440 INFO:teuthology.orchestra.run.smithi176.stdout: "fatal_signal_handlers": "true", 2024-09-14T09:05:25.440 INFO:teuthology.orchestra.run.smithi176.stdout: "filer_max_purge_ops": "10", 2024-09-14T09:05:25.440 INFO:teuthology.orchestra.run.smithi176.stdout: "filer_max_truncate_ops": "128", 2024-09-14T09:05:25.440 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_apply_finisher_threads": "1", 2024-09-14T09:05:25.440 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_blackhole": "false", 2024-09-14T09:05:25.440 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_btrfs_clone_range": "true", 2024-09-14T09:05:25.440 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_btrfs_snap": "true", 2024-09-14T09:05:25.440 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_caller_concurrency": "10", 2024-09-14T09:05:25.440 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_collect_device_partition_information": "true", 2024-09-14T09:05:25.440 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_commit_timeout": "600.000000", 2024-09-14T09:05:25.440 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_debug_inject_read_err": "false", 2024-09-14T09:05:25.440 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_debug_omap_check": "false", 2024-09-14T09:05:25.441 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_debug_verify_split": "false", 2024-09-14T09:05:25.441 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_dump_file": "", 2024-09-14T09:05:25.441 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-14T09:05:25.441 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-14T09:05:25.441 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_fadvise": "true", 2024-09-14T09:05:25.441 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_fail_eio": "true", 2024-09-14T09:05:25.441 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_fd_cache_shards": "16", 2024-09-14T09:05:25.441 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_fd_cache_size": "128", 2024-09-14T09:05:25.441 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_fiemap": "false", 2024-09-14T09:05:25.441 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_fiemap_threshold": "4096", 2024-09-14T09:05:25.441 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-14T09:05:25.442 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-14T09:05:25.442 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_inject_stall": "0", 2024-09-14T09:05:25.442 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_journal_parallel": "false", 2024-09-14T09:05:25.442 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_journal_trailing": "false", 2024-09-14T09:05:25.442 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_journal_writeahead": "false", 2024-09-14T09:05:25.442 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_kill_at": "0", 2024-09-14T09:05:25.442 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-14T09:05:25.442 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-14T09:05:25.442 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-14T09:05:25.442 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-14T09:05:25.442 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-14T09:05:25.442 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_max_inline_xattrs": "0", 2024-09-14T09:05:25.442 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-14T09:05:25.442 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-14T09:05:25.443 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-14T09:05:25.443 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-14T09:05:25.443 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_max_xattr_value_size": "0", 2024-09-14T09:05:25.443 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-14T09:05:25.443 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-14T09:05:25.443 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-14T09:05:25.443 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_merge_threshold": "-10", 2024-09-14T09:05:25.443 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-14T09:05:25.443 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_odsync_write": "false", 2024-09-14T09:05:25.443 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_omap_backend": "rocksdb", 2024-09-14T09:05:25.443 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_omap_backend_path": "", 2024-09-14T09:05:25.443 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-14T09:05:25.443 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-14T09:05:25.444 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-14T09:05:25.444 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_op_thread_timeout": "60", 2024-09-14T09:05:25.444 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_op_threads": "2", 2024-09-14T09:05:25.444 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_punch_hole": "false", 2024-09-14T09:05:25.444 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-14T09:05:25.444 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-14T09:05:25.444 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-14T09:05:25.444 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-14T09:05:25.444 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-14T09:05:25.444 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-14T09:05:25.444 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-14T09:05:25.444 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-14T09:05:25.444 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-14T09:05:25.444 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_queue_max_ops": "50", 2024-09-14T09:05:25.444 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-14T09:05:25.445 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_seek_data_hole": "false", 2024-09-14T09:05:25.445 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_sloppy_crc": "false", 2024-09-14T09:05:25.445 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-14T09:05:25.445 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_splice": "false", 2024-09-14T09:05:25.445 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_split_multiple": "2", 2024-09-14T09:05:25.445 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_split_rand_factor": "20", 2024-09-14T09:05:25.445 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_update_to": "1000", 2024-09-14T09:05:25.445 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-14T09:05:25.445 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-14T09:05:25.445 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-14T09:05:25.445 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-14T09:05:25.445 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-14T09:05:25.445 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-14T09:05:25.445 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_wbthrottle_enable": "true", 2024-09-14T09:05:25.445 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-14T09:05:25.446 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-14T09:05:25.446 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-14T09:05:25.446 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-14T09:05:25.446 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-14T09:05:25.446 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-14T09:05:25.446 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_xfs_extsize": "false", 2024-09-14T09:05:25.446 INFO:teuthology.orchestra.run.smithi176.stdout: "filestore_zfs_snap": "false", 2024-09-14T09:05:25.446 INFO:teuthology.orchestra.run.smithi176.stdout: "fio_dir": "/tmp/fio", 2024-09-14T09:05:25.446 INFO:teuthology.orchestra.run.smithi176.stdout: "fsid": "c635667d-994e-454f-bcea-51b45637b2a2", 2024-09-14T09:05:25.446 INFO:teuthology.orchestra.run.smithi176.stdout: "fuse_allow_other": "true", 2024-09-14T09:05:25.446 INFO:teuthology.orchestra.run.smithi176.stdout: "fuse_atomic_o_trunc": "true", 2024-09-14T09:05:25.446 INFO:teuthology.orchestra.run.smithi176.stdout: "fuse_big_writes": "true", 2024-09-14T09:05:25.446 INFO:teuthology.orchestra.run.smithi176.stdout: "fuse_debug": "false", 2024-09-14T09:05:25.446 INFO:teuthology.orchestra.run.smithi176.stdout: "fuse_default_permissions": "false", 2024-09-14T09:05:25.446 INFO:teuthology.orchestra.run.smithi176.stdout: "fuse_disable_pagecache": "false", 2024-09-14T09:05:25.447 INFO:teuthology.orchestra.run.smithi176.stdout: "fuse_max_write": "0", 2024-09-14T09:05:25.447 INFO:teuthology.orchestra.run.smithi176.stdout: "fuse_multithreaded": "true", 2024-09-14T09:05:25.447 INFO:teuthology.orchestra.run.smithi176.stdout: "fuse_require_active_mds": "true", 2024-09-14T09:05:25.447 INFO:teuthology.orchestra.run.smithi176.stdout: "fuse_set_user_groups": "true", 2024-09-14T09:05:25.447 INFO:teuthology.orchestra.run.smithi176.stdout: "fuse_splice_move": "true", 2024-09-14T09:05:25.447 INFO:teuthology.orchestra.run.smithi176.stdout: "fuse_splice_read": "true", 2024-09-14T09:05:25.447 INFO:teuthology.orchestra.run.smithi176.stdout: "fuse_splice_write": "true", 2024-09-14T09:05:25.447 INFO:teuthology.orchestra.run.smithi176.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-14T09:05:25.447 INFO:teuthology.orchestra.run.smithi176.stdout: "fuse_use_invalidate_cb": "true", 2024-09-14T09:05:25.447 INFO:teuthology.orchestra.run.smithi176.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-14T09:05:25.447 INFO:teuthology.orchestra.run.smithi176.stdout: "gss_target_name": "ceph", 2024-09-14T09:05:25.447 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_file": "", 2024-09-14T09:05:25.447 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_inject_failure": "0", 2024-09-14T09:05:25.447 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_interval": "5", 2024-09-14T09:05:25.447 INFO:teuthology.orchestra.run.smithi176.stdout: "host": "smithi176", 2024-09-14T09:05:25.448 INFO:teuthology.orchestra.run.smithi176.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-14T09:05:25.448 INFO:teuthology.orchestra.run.smithi176.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-14T09:05:25.448 INFO:teuthology.orchestra.run.smithi176.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-14T09:05:25.448 INFO:teuthology.orchestra.run.smithi176.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-14T09:05:25.448 INFO:teuthology.orchestra.run.smithi176.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-14T09:05:25.448 INFO:teuthology.orchestra.run.smithi176.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-14T09:05:25.448 INFO:teuthology.orchestra.run.smithi176.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-14T09:05:25.448 INFO:teuthology.orchestra.run.smithi176.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-14T09:05:25.448 INFO:teuthology.orchestra.run.smithi176.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-14T09:05:25.448 INFO:teuthology.orchestra.run.smithi176.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-14T09:05:25.448 INFO:teuthology.orchestra.run.smithi176.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-14T09:05:25.448 INFO:teuthology.orchestra.run.smithi176.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-14T09:05:25.448 INFO:teuthology.orchestra.run.smithi176.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-14T09:05:25.448 INFO:teuthology.orchestra.run.smithi176.stdout: "inject_early_sigterm": "false", 2024-09-14T09:05:25.449 INFO:teuthology.orchestra.run.smithi176.stdout: "jaeger_agent_port": "6799", 2024-09-14T09:05:25.449 INFO:teuthology.orchestra.run.smithi176.stdout: "jaeger_tracing_enable": "false", 2024-09-14T09:05:25.449 INFO:teuthology.orchestra.run.smithi176.stdout: "journal_aio": "true", 2024-09-14T09:05:25.449 INFO:teuthology.orchestra.run.smithi176.stdout: "journal_align_min_size": "65536", 2024-09-14T09:05:25.449 INFO:teuthology.orchestra.run.smithi176.stdout: "journal_block_align": "true", 2024-09-14T09:05:25.449 INFO:teuthology.orchestra.run.smithi176.stdout: "journal_block_size": "4096", 2024-09-14T09:05:25.449 INFO:teuthology.orchestra.run.smithi176.stdout: "journal_dio": "true", 2024-09-14T09:05:25.449 INFO:teuthology.orchestra.run.smithi176.stdout: "journal_discard": "false", 2024-09-14T09:05:25.449 INFO:teuthology.orchestra.run.smithi176.stdout: "journal_force_aio": "false", 2024-09-14T09:05:25.449 INFO:teuthology.orchestra.run.smithi176.stdout: "journal_ignore_corruption": "false", 2024-09-14T09:05:25.449 INFO:teuthology.orchestra.run.smithi176.stdout: "journal_max_write_bytes": "10485760", 2024-09-14T09:05:25.449 INFO:teuthology.orchestra.run.smithi176.stdout: "journal_max_write_entries": "100", 2024-09-14T09:05:25.449 INFO:teuthology.orchestra.run.smithi176.stdout: "journal_replay_from": "0", 2024-09-14T09:05:25.449 INFO:teuthology.orchestra.run.smithi176.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-14T09:05:25.449 INFO:teuthology.orchestra.run.smithi176.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-14T09:05:25.450 INFO:teuthology.orchestra.run.smithi176.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-14T09:05:25.450 INFO:teuthology.orchestra.run.smithi176.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-14T09:05:25.450 INFO:teuthology.orchestra.run.smithi176.stdout: "journal_write_header_frequency": "0", 2024-09-14T09:05:25.450 INFO:teuthology.orchestra.run.smithi176.stdout: "journal_zero_on_create": "false", 2024-09-14T09:05:25.450 INFO:teuthology.orchestra.run.smithi176.stdout: "journaler_prefetch_periods": "10", 2024-09-14T09:05:25.450 INFO:teuthology.orchestra.run.smithi176.stdout: "journaler_prezero_periods": "5", 2024-09-14T09:05:25.450 INFO:teuthology.orchestra.run.smithi176.stdout: "journaler_write_head_interval": "15", 2024-09-14T09:05:25.450 INFO:teuthology.orchestra.run.smithi176.stdout: "key": "", 2024-09-14T09:05:25.450 INFO:teuthology.orchestra.run.smithi176.stdout: "keyfile": "", 2024-09-14T09:05:25.450 INFO:teuthology.orchestra.run.smithi176.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-14T09:05:25.450 INFO:teuthology.orchestra.run.smithi176.stdout: "kstore_backend": "rocksdb", 2024-09-14T09:05:25.450 INFO:teuthology.orchestra.run.smithi176.stdout: "kstore_default_stripe_size": "65536", 2024-09-14T09:05:25.450 INFO:teuthology.orchestra.run.smithi176.stdout: "kstore_fsck_on_mount": "false", 2024-09-14T09:05:25.450 INFO:teuthology.orchestra.run.smithi176.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-14T09:05:25.451 INFO:teuthology.orchestra.run.smithi176.stdout: "kstore_max_bytes": "67108864", 2024-09-14T09:05:25.451 INFO:teuthology.orchestra.run.smithi176.stdout: "kstore_max_ops": "512", 2024-09-14T09:05:25.451 INFO:teuthology.orchestra.run.smithi176.stdout: "kstore_nid_prealloc": "1024", 2024-09-14T09:05:25.451 INFO:teuthology.orchestra.run.smithi176.stdout: "kstore_onode_map_size": "1024", 2024-09-14T09:05:25.451 INFO:teuthology.orchestra.run.smithi176.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-14T09:05:25.451 INFO:teuthology.orchestra.run.smithi176.stdout: "kstore_sync_submit_transaction": "false", 2024-09-14T09:05:25.451 INFO:teuthology.orchestra.run.smithi176.stdout: "kstore_sync_transaction": "false", 2024-09-14T09:05:25.451 INFO:teuthology.orchestra.run.smithi176.stdout: "librados_thread_count": "2", 2024-09-14T09:05:25.451 INFO:teuthology.orchestra.run.smithi176.stdout: "lockdep": "false", 2024-09-14T09:05:25.451 INFO:teuthology.orchestra.run.smithi176.stdout: "lockdep_force_backtrace": "false", 2024-09-14T09:05:25.452 INFO:teuthology.orchestra.run.smithi176.stdout: "log_coarse_timestamps": "true", 2024-09-14T09:05:25.452 INFO:teuthology.orchestra.run.smithi176.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-14T09:05:25.452 INFO:teuthology.orchestra.run.smithi176.stdout: "log_flush_on_exit": "false", 2024-09-14T09:05:25.452 INFO:teuthology.orchestra.run.smithi176.stdout: "log_graylog_host": "127.0.0.1", 2024-09-14T09:05:25.452 INFO:teuthology.orchestra.run.smithi176.stdout: "log_graylog_port": "12201", 2024-09-14T09:05:25.452 INFO:teuthology.orchestra.run.smithi176.stdout: "log_max_new": "1000", 2024-09-14T09:05:25.452 INFO:teuthology.orchestra.run.smithi176.stdout: "log_max_recent": "10000", 2024-09-14T09:05:25.452 INFO:teuthology.orchestra.run.smithi176.stdout: "log_stderr_prefix": "", 2024-09-14T09:05:25.452 INFO:teuthology.orchestra.run.smithi176.stdout: "log_stop_at_utilization": "0.970000", 2024-09-14T09:05:25.452 INFO:teuthology.orchestra.run.smithi176.stdout: "log_to_file": "true", 2024-09-14T09:05:25.452 INFO:teuthology.orchestra.run.smithi176.stdout: "log_to_graylog": "false", 2024-09-14T09:05:25.452 INFO:teuthology.orchestra.run.smithi176.stdout: "log_to_journald": "false", 2024-09-14T09:05:25.452 INFO:teuthology.orchestra.run.smithi176.stdout: "log_to_stderr": "false", 2024-09-14T09:05:25.452 INFO:teuthology.orchestra.run.smithi176.stdout: "log_to_syslog": "false", 2024-09-14T09:05:25.453 INFO:teuthology.orchestra.run.smithi176.stdout: "max_rotating_auth_attempts": "10", 2024-09-14T09:05:25.453 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-14T09:05:25.453 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_action_on_write_error": "1", 2024-09-14T09:05:25.453 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_alternate_name_max": "8192", 2024-09-14T09:05:25.453 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_asio_thread_count": "2", 2024-09-14T09:05:25.453 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_export_pin": "true", 2024-09-14T09:05:25.453 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_fragment_dirs": "true", 2024-09-14T09:05:25.453 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-14T09:05:25.453 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_fragment_interval": "5", 2024-09-14T09:05:25.453 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-14T09:05:25.453 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-14T09:05:25.453 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_interval": "10", 2024-09-14T09:05:25.453 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_max": "-1", 2024-09-14T09:05:25.454 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_max_until": "-1", 2024-09-14T09:05:25.454 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_merge_size": "50", 2024-09-14T09:05:25.454 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_midchunk": "0.300000", 2024-09-14T09:05:25.454 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-14T09:05:25.454 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_min_start": "0.200000", 2024-09-14T09:05:25.454 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_minchunk": "0.001000", 2024-09-14T09:05:25.454 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_mode": "0", 2024-09-14T09:05:25.454 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_need_max": "1.200000", 2024-09-14T09:05:25.454 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_need_min": "0.800000", 2024-09-14T09:05:25.454 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_overload_epochs": "2", 2024-09-14T09:05:25.454 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-14T09:05:25.454 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-14T09:05:25.454 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_split_bits": "3", 2024-09-14T09:05:25.455 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-14T09:05:25.455 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_split_size": "10000", 2024-09-14T09:05:25.455 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-14T09:05:25.455 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_target_decay": "10.000000", 2024-09-14T09:05:25.455 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-14T09:05:25.455 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_beacon_grace": "15.000000", 2024-09-14T09:05:25.455 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_beacon_interval": "4.000000", 2024-09-14T09:05:25.455 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-14T09:05:25.455 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-14T09:05:25.455 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache_mid": "0.700000", 2024-09-14T09:05:25.455 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-14T09:05:25.455 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache_quiesce_delay": "0", 2024-09-14T09:05:25.456 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-14T09:05:25.456 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-14T09:05:25.456 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-14T09:05:25.456 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache_release_free_interval": "10", 2024-09-14T09:05:25.456 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache_reservation": "0.050000", 2024-09-14T09:05:25.456 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-14T09:05:25.456 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache_trim_interval": "1", 2024-09-14T09:05:25.456 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache_trim_threshold": "262144", 2024-09-14T09:05:25.456 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-14T09:05:25.456 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-14T09:05:25.456 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-14T09:05:25.456 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_client_prealloc_inos": "1000", 2024-09-14T09:05:25.456 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-14T09:05:25.457 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_connect_bootstrapping": "false", 2024-09-14T09:05:25.457 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_damage_table_max_entries": "10000", 2024-09-14T09:05:25.457 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-14T09:05:25.457 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_debug_auth_pins": "false", 2024-09-14T09:05:25.457 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_debug_frag": "false", 2024-09-14T09:05:25.457 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_debug_scatterstat": "false", 2024-09-14T09:05:25.457 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_debug_subtrees": "false", 2024-09-14T09:05:25.457 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_decay_halflife": "5.000000", 2024-09-14T09:05:25.457 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_default_dir_hash": "2", 2024-09-14T09:05:25.457 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_defer_session_stale": "true", 2024-09-14T09:05:25.457 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_deny_all_reconnect": "false", 2024-09-14T09:05:25.457 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_dir_keys_per_op": "16384", 2024-09-14T09:05:25.457 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_dir_max_commit_size": "10", 2024-09-14T09:05:25.457 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_dir_max_entries": "0", 2024-09-14T09:05:25.458 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_dir_prefetch": "true", 2024-09-14T09:05:25.458 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-14T09:05:25.458 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-14T09:05:25.458 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_dump_cache_on_map": "false", 2024-09-14T09:05:25.458 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-14T09:05:25.458 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-14T09:05:25.458 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_early_reply": "true", 2024-09-14T09:05:25.458 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_enable_op_tracker": "true", 2024-09-14T09:05:25.458 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_enforce_unique_name": "true", 2024-09-14T09:05:25.458 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-14T09:05:25.458 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-14T09:05:25.458 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_export_ephemeral_random": "true", 2024-09-14T09:05:25.458 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-14T09:05:25.458 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-14T09:05:25.458 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-14T09:05:25.459 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-14T09:05:25.459 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-14T09:05:25.459 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-14T09:05:25.459 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-14T09:05:25.459 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-14T09:05:25.459 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_health_summarize_threshold": "10", 2024-09-14T09:05:25.459 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-14T09:05:25.459 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-14T09:05:25.459 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_inject_health_dummy": "false", 2024-09-14T09:05:25.459 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-14T09:05:25.459 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_inject_migrator_session_race": "false", 2024-09-14T09:05:25.459 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-14T09:05:25.459 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-14T09:05:25.459 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-14T09:05:25.459 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_join_fs": "", 2024-09-14T09:05:25.460 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_journal_format": "1", 2024-09-14T09:05:25.460 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-14T09:05:25.460 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_kill_create_at": "0", 2024-09-14T09:05:25.460 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_kill_dirfrag_at": "0", 2024-09-14T09:05:25.460 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_kill_export_at": "0", 2024-09-14T09:05:25.460 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_kill_import_at": "0", 2024-09-14T09:05:25.460 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_kill_journal_at": "0", 2024-09-14T09:05:25.460 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_kill_journal_expire_at": "0", 2024-09-14T09:05:25.460 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_kill_journal_replay_at": "0", 2024-09-14T09:05:25.460 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_kill_link_at": "0", 2024-09-14T09:05:25.460 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_kill_mdstable_at": "0", 2024-09-14T09:05:25.460 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_kill_openc_at": "0", 2024-09-14T09:05:25.460 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_kill_rename_at": "0", 2024-09-14T09:05:25.460 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_kill_shutdown_at": "0", 2024-09-14T09:05:25.461 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log_event_large_threshold": "524288", 2024-09-14T09:05:25.461 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log_events_per_segment": "1024", 2024-09-14T09:05:25.461 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-14T09:05:25.461 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log_max_events": "-1", 2024-09-14T09:05:25.461 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log_max_segments": "128", 2024-09-14T09:05:25.461 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log_pause": "false", 2024-09-14T09:05:25.461 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log_segment_size": "0", 2024-09-14T09:05:25.461 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-14T09:05:25.461 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-14T09:05:25.461 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-14T09:05:25.461 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log_trim_threshold": "128", 2024-09-14T09:05:25.461 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-14T09:05:25.461 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log_warn_factor": "2.000000", 2024-09-14T09:05:25.461 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_max_caps_per_client": "1048576", 2024-09-14T09:05:25.461 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_max_completed_flushes": "100000", 2024-09-14T09:05:25.462 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_max_completed_requests": "100000", 2024-09-14T09:05:25.462 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_max_export_size": "20971520", 2024-09-14T09:05:25.462 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_max_file_recover": "32", 2024-09-14T09:05:25.462 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_max_purge_files": "64", 2024-09-14T09:05:25.462 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_max_purge_ops": "8192", 2024-09-14T09:05:25.462 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-14T09:05:25.462 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-14T09:05:25.462 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_max_snaps_per_dir": "100", 2024-09-14T09:05:25.462 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_metrics_update_interval": "2", 2024-09-14T09:05:25.462 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_min_caps_per_client": "100", 2024-09-14T09:05:25.462 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_min_caps_working_set": "10000", 2024-09-14T09:05:25.462 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-14T09:05:25.462 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_numa_node": "-1", 2024-09-14T09:05:25.462 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-14T09:05:25.462 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_op_complaint_time": "30.000000", 2024-09-14T09:05:25.462 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_op_history_duration": "600", 2024-09-14T09:05:25.463 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_op_history_size": "20", 2024-09-14T09:05:25.463 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_op_history_slow_op_size": "20", 2024-09-14T09:05:25.463 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-14T09:05:25.463 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_op_log_threshold": "5", 2024-09-14T09:05:25.463 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_ping_grace": "15", 2024-09-14T09:05:25.463 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_ping_interval": "5", 2024-09-14T09:05:25.463 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-14T09:05:25.463 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-14T09:05:25.463 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_recall_max_caps": "30000", 2024-09-14T09:05:25.463 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-14T09:05:25.463 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-14T09:05:25.463 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-14T09:05:25.463 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_recall_warning_threshold": "262144", 2024-09-14T09:05:25.463 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-14T09:05:25.463 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_replay_interval": "1.000000", 2024-09-14T09:05:25.464 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-14T09:05:25.464 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-14T09:05:25.464 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_root_ino_gid": "0", 2024-09-14T09:05:25.464 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_root_ino_uid": "0", 2024-09-14T09:05:25.464 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-14T09:05:25.464 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_scrub_stats_review_period": "1", 2024-09-14T09:05:25.464 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-14T09:05:25.464 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-14T09:05:25.464 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-14T09:05:25.464 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-14T09:05:25.464 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-14T09:05:25.464 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-14T09:05:25.464 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-14T09:05:25.464 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-14T09:05:25.464 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-14T09:05:25.465 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-14T09:05:25.465 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-14T09:05:25.465 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_shutdown_check": "0", 2024-09-14T09:05:25.465 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_skip_ino": "0", 2024-09-14T09:05:25.465 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-14T09:05:25.465 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_snap_max_uid": "4294967294", 2024-09-14T09:05:25.465 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_snap_min_uid": "0", 2024-09-14T09:05:25.465 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_snap_rstat": "false", 2024-09-14T09:05:25.465 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_standby_replay_damaged": "false", 2024-09-14T09:05:25.465 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_symlink_recovery": "true", 2024-09-14T09:05:25.465 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-14T09:05:25.465 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_thrash_exports": "0", 2024-09-14T09:05:25.465 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_thrash_fragments": "0", 2024-09-14T09:05:25.465 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_tick_interval": "5.000000", 2024-09-14T09:05:25.466 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_valgrind_exit": "false", 2024-09-14T09:05:25.466 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_verify_backtrace": "1", 2024-09-14T09:05:25.466 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_verify_scatter": "false", 2024-09-14T09:05:25.466 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-14T09:05:25.466 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_wipe_sessions": "false", 2024-09-14T09:05:25.466 INFO:teuthology.orchestra.run.smithi176.stdout: "mempool_debug": "false", 2024-09-14T09:05:25.466 INFO:teuthology.orchestra.run.smithi176.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-14T09:05:25.466 INFO:teuthology.orchestra.run.smithi176.stdout: "memstore_device_bytes": "1073741824", 2024-09-14T09:05:25.466 INFO:teuthology.orchestra.run.smithi176.stdout: "memstore_page_set": "false", 2024-09-14T09:05:25.466 INFO:teuthology.orchestra.run.smithi176.stdout: "memstore_page_size": "65536", 2024-09-14T09:05:25.466 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_client_bytes": "134217728", 2024-09-14T09:05:25.466 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_client_messages": "512", 2024-09-14T09:05:25.466 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-14T09:05:25.466 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-14T09:05:25.466 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-14T09:05:25.466 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-14T09:05:25.467 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_disabled_modules": "", 2024-09-14T09:05:25.467 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_enable_op_tracker": "true", 2024-09-14T09:05:25.467 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-14T09:05:25.467 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_max_pg_creating": "1024", 2024-09-14T09:05:25.467 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_max_pg_num_change": "128", 2024-09-14T09:05:25.467 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_mds_bytes": "134217728", 2024-09-14T09:05:25.467 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_mds_messages": "128", 2024-09-14T09:05:25.467 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-14T09:05:25.467 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_mon_bytes": "134217728", 2024-09-14T09:05:25.467 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_mon_messages": "128", 2024-09-14T09:05:25.467 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-14T09:05:25.467 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-14T09:05:25.467 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_op_history_duration": "600", 2024-09-14T09:05:25.467 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_op_history_size": "20", 2024-09-14T09:05:25.467 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-14T09:05:25.468 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-14T09:05:25.468 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_op_log_threshold": "5", 2024-09-14T09:05:25.468 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_osd_bytes": "536870912", 2024-09-14T09:05:25.468 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_osd_messages": "8192", 2024-09-14T09:05:25.468 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_pool": "true", 2024-09-14T09:05:25.468 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-14T09:05:25.468 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_standby_modules": "true", 2024-09-14T09:05:25.468 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_stats_period": "5", 2024-09-14T09:05:25.468 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_stats_threshold": "5", 2024-09-14T09:05:25.468 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_tick_period": "2", 2024-09-14T09:05:25.468 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-14T09:05:25.468 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-14T09:05:25.468 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_allow_pool_delete": "true", 2024-09-14T09:05:25.469 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_allow_pool_size_one": "true", 2024-09-14T09:05:25.469 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_auth_validate_all_caps": "true", 2024-09-14T09:05:25.469 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-14T09:05:25.469 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-14T09:05:25.469 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_client_bytes": "104857600", 2024-09-14T09:05:25.469 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_client_directed_command_retry": "5", 2024-09-14T09:05:25.469 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-14T09:05:25.469 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-14T09:05:25.469 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-14T09:05:25.469 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-14T09:05:25.469 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_client_hunt_parallel": "3", 2024-09-14T09:05:25.469 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_client_log_interval": "1.000000", 2024-09-14T09:05:25.469 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-14T09:05:25.469 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_client_ping_interval": "10.000000", 2024-09-14T09:05:25.470 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-14T09:05:25.470 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_client_target_rank": "-1", 2024-09-14T09:05:25.470 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-14T09:05:25.470 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-14T09:05:25.470 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-14T09:05:25.470 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_cluster_log_level": "debug", 2024-09-14T09:05:25.470 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_cluster_log_to_file": "true", 2024-09-14T09:05:25.470 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-14T09:05:25.470 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-14T09:05:25.470 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-14T09:05:25.470 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_cluster_log_to_journald": "false", 2024-09-14T09:05:25.470 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-14T09:05:25.470 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-14T09:05:25.470 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-14T09:05:25.470 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_compact_on_bootstrap": "false", 2024-09-14T09:05:25.470 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_compact_on_start": "false", 2024-09-14T09:05:25.471 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_compact_on_trim": "true", 2024-09-14T09:05:25.471 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-14T09:05:25.471 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-14T09:05:25.471 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-14T09:05:25.471 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_cpu_threads": "4", 2024-09-14T09:05:25.471 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_crush_min_required_version": "hammer", 2024-09-14T09:05:25.471 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_daemon_bytes": "419430400", 2024-09-14T09:05:25.471 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-14T09:05:25.471 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_data_avail_crit": "5", 2024-09-14T09:05:25.471 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_data_avail_warn": "5", 2024-09-14T09:05:25.471 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_data_size_warn": "16106127360", 2024-09-14T09:05:25.471 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-14T09:05:25.471 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-14T09:05:25.472 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_debug_dump_json": "false", 2024-09-14T09:05:25.472 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-14T09:05:25.472 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_debug_dump_transactions": "false", 2024-09-14T09:05:25.472 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_debug_extra_checks": "false", 2024-09-14T09:05:25.472 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-14T09:05:25.472 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-14T09:05:25.472 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_debug_no_require_reef": "false", 2024-09-14T09:05:25.472 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_debug_no_require_squid": "false", 2024-09-14T09:05:25.472 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-14T09:05:25.472 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-14T09:05:25.472 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-14T09:05:25.472 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_down_mkfs_grace": "120", 2024-09-14T09:05:25.472 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_down_uptime_grace": "60", 2024-09-14T09:05:25.472 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_election_default_strategy": "1", 2024-09-14T09:05:25.473 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_election_timeout": "5.000000", 2024-09-14T09:05:25.473 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-14T09:05:25.473 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_elector_ping_divisor": "2", 2024-09-14T09:05:25.473 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-14T09:05:25.473 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_enable_op_tracker": "true", 2024-09-14T09:05:25.473 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_fake_pool_delete": "false", 2024-09-14T09:05:25.473 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_force_quorum_join": "false", 2024-09-14T09:05:25.473 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-14T09:05:25.473 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_globalid_prealloc": "10000", 2024-09-14T09:05:25.473 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_health_detail_to_clog": "false", 2024-09-14T09:05:25.473 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_health_log_update_period": "5", 2024-09-14T09:05:25.473 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_health_max_detail": "50", 2024-09-14T09:05:25.473 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_health_to_clog": "true", 2024-09-14T09:05:25.473 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_health_to_clog_interval": "600", 2024-09-14T09:05:25.474 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-14T09:05:25.474 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_host": "172.21.15.176,[v2:172.21.15.176:3301,v1:172.21.15.176:6790],172.21.15.183", 2024-09-14T09:05:25.474 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_host_override": "", 2024-09-14T09:05:25.474 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_initial_members": "", 2024-09-14T09:05:25.474 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-14T09:05:25.474 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-14T09:05:25.474 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-14T09:05:25.474 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-14T09:05:25.474 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-14T09:05:25.474 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_lease": "5.000000", 2024-09-14T09:05:25.474 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-14T09:05:25.474 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-14T09:05:25.474 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_log_full_interval": "50", 2024-09-14T09:05:25.475 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_log_max": "10000", 2024-09-14T09:05:25.475 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_log_max_summary": "50", 2024-09-14T09:05:25.475 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-14T09:05:25.475 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_max_log_epochs": "500", 2024-09-14T09:05:25.475 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-14T09:05:25.475 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-14T09:05:25.475 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-14T09:05:25.475 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_max_osd": "10000", 2024-09-14T09:05:25.475 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_max_pg_per_osd": "10000", 2024-09-14T09:05:25.475 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_max_pool_pg_num": "65536", 2024-09-14T09:05:25.475 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-14T09:05:25.475 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-14T09:05:25.475 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_mds_force_trim_to": "0", 2024-09-14T09:05:25.475 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_mds_skip_sanity": "false", 2024-09-14T09:05:25.475 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_memory_autotune": "true", 2024-09-14T09:05:25.476 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_memory_target": "2147483648", 2024-09-14T09:05:25.476 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_mgr_beacon_grace": "30", 2024-09-14T09:05:25.476 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-14T09:05:25.476 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_mgr_digest_period": "5", 2024-09-14T09:05:25.476 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_mgr_inactive_grace": "60", 2024-09-14T09:05:25.476 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-14T09:05:25.476 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-14T09:05:25.476 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_min_osdmap_epochs": "500", 2024-09-14T09:05:25.476 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-14T09:05:25.476 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-14T09:05:25.476 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_op_complaint_time": "30", 2024-09-14T09:05:25.476 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_op_history_duration": "600", 2024-09-14T09:05:25.477 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_op_history_size": "20", 2024-09-14T09:05:25.477 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_op_history_slow_op_size": "20", 2024-09-14T09:05:25.477 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-14T09:05:25.477 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_op_log_threshold": "5", 2024-09-14T09:05:25.477 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-14T09:05:25.477 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-14T09:05:25.477 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-14T09:05:25.477 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_auto_mark_in": "false", 2024-09-14T09:05:25.477 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-14T09:05:25.477 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-14T09:05:25.477 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-14T09:05:25.477 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_cache_size": "500", 2024-09-14T09:05:25.478 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-14T09:05:25.478 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-14T09:05:25.478 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-14T09:05:25.478 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_down_out_interval": "600", 2024-09-14T09:05:25.478 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-14T09:05:25.478 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-14T09:05:25.478 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_force_trim_to": "0", 2024-09-14T09:05:25.478 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-14T09:05:25.478 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-14T09:05:25.478 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-14T09:05:25.478 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-14T09:05:25.478 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-14T09:05:25.478 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-14T09:05:25.478 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-14T09:05:25.478 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_min_down_reporters": "2", 2024-09-14T09:05:25.479 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-14T09:05:25.479 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-14T09:05:25.479 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-14T09:05:25.479 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-14T09:05:25.479 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-14T09:05:25.479 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-14T09:05:25.479 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_report_timeout": "900", 2024-09-14T09:05:25.479 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-14T09:05:25.479 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-14T09:05:25.479 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-14T09:05:25.479 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-14T09:05:25.479 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-14T09:05:25.479 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-14T09:05:25.479 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-14T09:05:25.480 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-14T09:05:25.480 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-14T09:05:25.480 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_pg_stuck_threshold": "60", 2024-09-14T09:05:25.480 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-14T09:05:25.480 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-14T09:05:25.480 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-14T09:05:25.480 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-14T09:05:25.480 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-14T09:05:25.480 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-14T09:05:25.480 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_probe_timeout": "2.000000", 2024-09-14T09:05:25.480 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_reweight_max_change": "0.050000", 2024-09-14T09:05:25.480 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_reweight_max_osds": "4", 2024-09-14T09:05:25.480 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-14T09:05:25.480 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-14T09:05:25.481 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-14T09:05:25.481 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-14T09:05:25.481 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-14T09:05:25.481 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_scrub_interval": "86400", 2024-09-14T09:05:25.481 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_scrub_max_keys": "100", 2024-09-14T09:05:25.481 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_scrub_timeout": "300", 2024-09-14T09:05:25.481 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_session_timeout": "300", 2024-09-14T09:05:25.481 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_smart_report_timeout": "5", 2024-09-14T09:05:25.481 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_stat_smooth_intervals": "6", 2024-09-14T09:05:25.481 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-14T09:05:25.481 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_stretch_pool_min_size": "2", 2024-09-14T09:05:25.481 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_stretch_pool_size": "4", 2024-09-14T09:05:25.481 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-14T09:05:25.481 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-14T09:05:25.481 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_sync_debug": "false", 2024-09-14T09:05:25.482 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-14T09:05:25.482 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-14T09:05:25.482 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_sync_provider_kill_at": "0", 2024-09-14T09:05:25.482 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_sync_requester_kill_at": "0", 2024-09-14T09:05:25.482 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_sync_timeout": "60.000000", 2024-09-14T09:05:25.482 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_target_pg_per_osd": "100", 2024-09-14T09:05:25.482 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_tick_interval": "5", 2024-09-14T09:05:25.482 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_timecheck_interval": "300.000000", 2024-09-14T09:05:25.482 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-14T09:05:25.482 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_use_min_delay_socket": "false", 2024-09-14T09:05:25.482 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_older_version_delay": "604800", 2024-09-14T09:05:25.482 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-14T09:05:25.482 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-14T09:05:25.483 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-14T09:05:25.483 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-14T09:05:25.483 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-14T09:05:25.483 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-14T09:05:25.483 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-14T09:05:25.483 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_on_misplaced": "false", 2024-09-14T09:05:25.483 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-14T09:05:25.483 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_on_older_version": "true", 2024-09-14T09:05:25.483 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-14T09:05:25.483 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-14T09:05:25.483 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-14T09:05:25.483 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-14T09:05:25.483 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-14T09:05:25.483 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-14T09:05:25.484 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-14T09:05:25.484 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-14T09:05:25.484 INFO:teuthology.orchestra.run.smithi176.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-14T09:05:25.484 INFO:teuthology.orchestra.run.smithi176.stdout: "monmap": "", 2024-09-14T09:05:25.484 INFO:teuthology.orchestra.run.smithi176.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-14T09:05:25.484 INFO:teuthology.orchestra.run.smithi176.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-14T09:05:25.484 INFO:teuthology.orchestra.run.smithi176.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-14T09:05:25.484 INFO:teuthology.orchestra.run.smithi176.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-14T09:05:25.484 INFO:teuthology.orchestra.run.smithi176.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-14T09:05:25.484 INFO:teuthology.orchestra.run.smithi176.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-14T09:05:25.484 INFO:teuthology.orchestra.run.smithi176.stdout: "motr_tracing_enabled": "false", 2024-09-14T09:05:25.484 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_async_op_threads": "3", 2024-09-14T09:05:25.484 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-14T09:05:25.484 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_async_rdma_cm": "false", 2024-09-14T09:05:25.485 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_async_rdma_device_name": "", 2024-09-14T09:05:25.485 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_async_rdma_dscp": "96", 2024-09-14T09:05:25.485 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-14T09:05:25.485 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-14T09:05:25.485 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_async_rdma_local_gid": "", 2024-09-14T09:05:25.485 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-14T09:05:25.485 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_async_rdma_port_num": "1", 2024-09-14T09:05:25.485 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-14T09:05:25.485 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-14T09:05:25.485 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-14T09:05:25.485 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-14T09:05:25.485 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_async_rdma_sl": "3", 2024-09-14T09:05:25.485 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_async_rdma_support_srq": "true", 2024-09-14T09:05:25.485 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_async_rdma_type": "ib", 2024-09-14T09:05:25.485 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_async_reap_threshold": "5", 2024-09-14T09:05:25.486 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_bind_before_connect": "false", 2024-09-14T09:05:25.486 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_bind_ipv4": "true", 2024-09-14T09:05:25.486 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_bind_ipv6": "false", 2024-09-14T09:05:25.486 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_bind_msgr1": "true", 2024-09-14T09:05:25.486 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_bind_msgr2": "true", 2024-09-14T09:05:25.486 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_bind_port_max": "7568", 2024-09-14T09:05:25.486 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_bind_port_min": "6800", 2024-09-14T09:05:25.487 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-14T09:05:25.487 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_bind_retry_count": "3", 2024-09-14T09:05:25.487 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_bind_retry_delay": "5", 2024-09-14T09:05:25.487 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_blackhole_client": "false", 2024-09-14T09:05:25.487 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_blackhole_mds": "false", 2024-09-14T09:05:25.487 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_blackhole_mgr": "false", 2024-09-14T09:05:25.487 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_blackhole_mon": "false", 2024-09-14T09:05:25.487 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_blackhole_osd": "false", 2024-09-14T09:05:25.487 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_client_mode": "crc secure", 2024-09-14T09:05:25.487 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-14T09:05:25.487 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_cluster_mode": "crc secure", 2024-09-14T09:05:25.487 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_cluster_type": "", 2024-09-14T09:05:25.487 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_compress_secure": "false", 2024-09-14T09:05:25.487 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_connection_idle_timeout": "900", 2024-09-14T09:05:25.487 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_connection_ready_timeout": "10", 2024-09-14T09:05:25.488 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_crc_data": "true", 2024-09-14T09:05:25.488 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_crc_header": "true", 2024-09-14T09:05:25.488 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_die_on_bad_msg": "false", 2024-09-14T09:05:25.488 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_die_on_bug": "true", 2024-09-14T09:05:25.488 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_die_on_old_message": "true", 2024-09-14T09:05:25.488 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_die_on_skipped_message": "false", 2024-09-14T09:05:25.488 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-14T09:05:25.489 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-14T09:05:25.489 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_dpdk_coremask": "0xF", 2024-09-14T09:05:25.489 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-14T09:05:25.489 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-14T09:05:25.489 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_dpdk_enable_tso": "true", 2024-09-14T09:05:25.489 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-14T09:05:25.489 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-14T09:05:25.489 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_dpdk_hugepages": "", 2024-09-14T09:05:25.489 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-14T09:05:25.489 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-14T09:05:25.489 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_dpdk_lro": "true", 2024-09-14T09:05:25.489 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_dpdk_memory_channel": "4", 2024-09-14T09:05:25.489 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-14T09:05:25.489 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_dpdk_pmd": "", 2024-09-14T09:05:25.490 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_dpdk_port_id": "0", 2024-09-14T09:05:25.490 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-14T09:05:25.490 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-14T09:05:25.490 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_dump_on_send": "false", 2024-09-14T09:05:25.490 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_initial_backoff": "0.200000", 2024-09-14T09:05:25.490 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_inject_delay_max": "1.000000", 2024-09-14T09:05:25.490 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-14T09:05:25.490 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_inject_delay_type": "", 2024-09-14T09:05:25.490 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-14T09:05:25.490 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_inject_network_congestion": "0", 2024-09-14T09:05:25.490 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_inject_socket_failures": "5000", 2024-09-14T09:05:25.490 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_learn_addr_from_peer": "true", 2024-09-14T09:05:25.490 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_max_accept_failures": "4", 2024-09-14T09:05:25.490 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_max_backoff": "15.000000", 2024-09-14T09:05:25.490 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_mon_client_mode": "secure crc", 2024-09-14T09:05:25.491 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-14T09:05:25.491 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_mon_service_mode": "secure crc", 2024-09-14T09:05:25.491 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_osd_compress_min_size": "1024", 2024-09-14T09:05:25.491 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_osd_compress_mode": "none", 2024-09-14T09:05:25.491 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-14T09:05:25.491 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-14T09:05:25.491 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_pq_min_cost": "65536", 2024-09-14T09:05:25.491 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_public_type": "", 2024-09-14T09:05:25.491 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_service_mode": "crc secure", 2024-09-14T09:05:25.491 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_tcp_listen_backlog": "512", 2024-09-14T09:05:25.491 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_tcp_nodelay": "true", 2024-09-14T09:05:25.491 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-14T09:05:25.491 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_tcp_rcvbuf": "0", 2024-09-14T09:05:25.491 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-14T09:05:25.491 INFO:teuthology.orchestra.run.smithi176.stdout: "ms_type": "async+posix", 2024-09-14T09:05:25.492 INFO:teuthology.orchestra.run.smithi176.stdout: "no_config_file": "false", 2024-09-14T09:05:25.492 INFO:teuthology.orchestra.run.smithi176.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-14T09:05:25.492 INFO:teuthology.orchestra.run.smithi176.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-14T09:05:25.492 INFO:teuthology.orchestra.run.smithi176.stdout: "objecter_completion_locks_per_session": "32", 2024-09-14T09:05:25.492 INFO:teuthology.orchestra.run.smithi176.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-14T09:05:25.492 INFO:teuthology.orchestra.run.smithi176.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-14T09:05:25.492 INFO:teuthology.orchestra.run.smithi176.stdout: "objecter_inflight_ops": "1024", 2024-09-14T09:05:25.492 INFO:teuthology.orchestra.run.smithi176.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-14T09:05:25.492 INFO:teuthology.orchestra.run.smithi176.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-14T09:05:25.492 INFO:teuthology.orchestra.run.smithi176.stdout: "objecter_tick_interval": "5.000000", 2024-09-14T09:05:25.492 INFO:teuthology.orchestra.run.smithi176.stdout: "objecter_timeout": "10.000000", 2024-09-14T09:05:25.492 INFO:teuthology.orchestra.run.smithi176.stdout: "objectstore_blackhole": "false", 2024-09-14T09:05:25.492 INFO:teuthology.orchestra.run.smithi176.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-14T09:05:25.492 INFO:teuthology.orchestra.run.smithi176.stdout: "openssl_engine_opts": "", 2024-09-14T09:05:25.492 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_agent_delay_time": "5.000000", 2024-09-14T09:05:25.493 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_agent_hist_halflife": "1000", 2024-09-14T09:05:25.493 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_agent_max_low_ops": "2", 2024-09-14T09:05:25.493 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_agent_max_ops": "4", 2024-09-14T09:05:25.493 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-14T09:05:25.493 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-14T09:05:25.493 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_agent_slop": "0.020000", 2024-09-14T09:05:25.493 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-14T09:05:25.493 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-14T09:05:25.493 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_asio_thread_count": "2", 2024-09-14T09:05:25.493 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_async_recovery_min_cost": "100", 2024-09-14T09:05:25.493 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-14T09:05:25.494 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-14T09:05:25.494 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_backfill_scan_max": "512", 2024-09-14T09:05:25.494 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_backfill_scan_min": "64", 2024-09-14T09:05:25.494 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_backoff_on_degraded": "false", 2024-09-14T09:05:25.494 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_backoff_on_peering": "false", 2024-09-14T09:05:25.494 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_backoff_on_unfound": "true", 2024-09-14T09:05:25.494 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_beacon_report_interval": "300", 2024-09-14T09:05:25.494 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_bench_duration": "30", 2024-09-14T09:05:25.494 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-14T09:05:25.494 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_bench_max_block_size": "67108864", 2024-09-14T09:05:25.494 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-14T09:05:25.494 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_blkin_trace_all": "false", 2024-09-14T09:05:25.494 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-14T09:05:25.494 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-14T09:05:25.494 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-14T09:05:25.495 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-14T09:05:25.495 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_check_for_log_corruption": "false", 2024-09-14T09:05:25.495 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-14T09:05:25.495 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-14T09:05:25.495 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-14T09:05:25.495 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-14T09:05:25.495 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_class_update_on_start": "true", 2024-09-14T09:05:25.495 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_client_message_cap": "256", 2024-09-14T09:05:25.495 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_client_message_size_cap": "524288000", 2024-09-14T09:05:25.495 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_client_op_priority": "63", 2024-09-14T09:05:25.495 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_client_watch_timeout": "30", 2024-09-14T09:05:25.495 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_command_max_records": "256", 2024-09-14T09:05:25.495 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_compact_on_start": "false", 2024-09-14T09:05:25.495 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-14T09:05:25.495 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-14T09:05:25.495 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-14T09:05:25.496 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_crush_update_on_start": "true", 2024-09-14T09:05:25.496 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_crush_update_weight_set": "true", 2024-09-14T09:05:25.496 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-14T09:05:25.496 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-14T09:05:25.496 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-14T09:05:25.496 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-14T09:05:25.496 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-14T09:05:25.496 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_feed_pullee": "-1", 2024-09-14T09:05:25.496 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-14T09:05:25.496 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-14T09:05:25.496 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-14T09:05:25.496 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_misdirected_ops": "false", 2024-09-14T09:05:25.496 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_no_acting_change": "false", 2024-09-14T09:05:25.496 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_no_purge_strays": "false", 2024-09-14T09:05:25.496 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_op_order": "false", 2024-09-14T09:05:25.497 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-14T09:05:25.497 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-14T09:05:25.497 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-14T09:05:25.497 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-14T09:05:25.497 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_shutdown": "false", 2024-09-14T09:05:25.497 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-14T09:05:25.497 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-14T09:05:25.497 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_trim_objects": "false", 2024-09-14T09:05:25.497 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-14T09:05:25.497 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-14T09:05:25.497 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_verify_snaps": "false", 2024-09-14T09:05:25.497 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-14T09:05:25.497 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-14T09:05:25.497 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_deep_scrub_keys": "1024", 2024-09-14T09:05:25.497 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-14T09:05:25.498 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-14T09:05:25.498 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-14T09:05:25.498 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_deep_scrub_stride": "524288", 2024-09-14T09:05:25.498 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-14T09:05:25.498 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-14T09:05:25.498 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_default_notify_timeout": "30", 2024-09-14T09:05:25.498 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_delete_sleep": "0.000000", 2024-09-14T09:05:25.498 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-14T09:05:25.498 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-14T09:05:25.498 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-14T09:05:25.498 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_discard_disconnected_ops": "true", 2024-09-14T09:05:25.498 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_ec_partial_reads": "true", 2024-09-14T09:05:25.498 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_enable_op_tracker": "true", 2024-09-14T09:05:25.498 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-14T09:05:25.498 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-14T09:05:25.499 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-14T09:05:25.499 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-14T09:05:25.499 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_fast_info": "true", 2024-09-14T09:05:25.499 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_fast_shutdown": "true", 2024-09-14T09:05:25.499 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-14T09:05:25.499 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-14T09:05:25.499 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-14T09:05:25.499 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-14T09:05:25.499 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-14T09:05:25.499 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_function_tracing": "false", 2024-09-14T09:05:25.499 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_heartbeat_grace": "20", 2024-09-14T09:05:25.499 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_heartbeat_interval": "6", 2024-09-14T09:05:25.499 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-14T09:05:25.499 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_heartbeat_min_peers": "10", 2024-09-14T09:05:25.500 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_heartbeat_min_size": "2000", 2024-09-14T09:05:25.500 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_heartbeat_stale": "600", 2024-09-14T09:05:25.500 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-14T09:05:25.500 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_hit_set_max_size": "100000", 2024-09-14T09:05:25.500 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_hit_set_min_size": "1000", 2024-09-14T09:05:25.500 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-14T09:05:25.500 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-14T09:05:25.500 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-14T09:05:25.500 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-14T09:05:25.500 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-14T09:05:25.500 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-14T09:05:25.500 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_journal_size": "5120", 2024-09-14T09:05:25.500 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_kill_backfill_at": "0", 2024-09-14T09:05:25.500 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-14T09:05:25.500 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_map_cache_size": "50", 2024-09-14T09:05:25.501 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_map_dedup": "true", 2024-09-14T09:05:25.501 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_map_message_max": "40", 2024-09-14T09:05:25.501 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-14T09:05:25.501 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_map_share_max_epochs": "40", 2024-09-14T09:05:25.501 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_attr_name_len": "100", 2024-09-14T09:05:25.501 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_attr_size": "0", 2024-09-14T09:05:25.501 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_backfills": "1", 2024-09-14T09:05:25.501 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_markdown_count": "5", 2024-09-14T09:05:25.501 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_markdown_period": "600", 2024-09-14T09:05:25.501 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_object_name_len": "2048", 2024-09-14T09:05:25.501 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_object_namespace_len": "256", 2024-09-14T09:05:25.501 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_object_size": "134217728", 2024-09-14T09:05:25.501 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-14T09:05:25.501 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-14T09:05:25.501 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_pg_blocked_by": "16", 2024-09-14T09:05:25.502 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_pg_log_entries": "10000", 2024-09-14T09:05:25.502 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-14T09:05:25.502 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_pgls": "1024", 2024-09-14T09:05:25.502 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_push_cost": "8388608", 2024-09-14T09:05:25.502 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_push_objects": "10", 2024-09-14T09:05:25.502 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_scrubs": "3", 2024-09-14T09:05:25.502 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-14T09:05:25.502 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_trimming_pgs": "2", 2024-09-14T09:05:25.502 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_write_op_reply_len": "64", 2024-09-14T09:05:25.502 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_max_write_size": "90", 2024-09-14T09:05:25.502 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-14T09:05:25.502 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-14T09:05:25.502 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-14T09:05:25.502 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-14T09:05:25.502 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-14T09:05:25.503 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-14T09:05:25.503 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-14T09:05:25.503 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-14T09:05:25.503 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_profile": "balanced", 2024-09-14T09:05:25.503 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-14T09:05:25.503 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-14T09:05:25.503 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-14T09:05:25.503 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-14T09:05:25.503 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-14T09:05:25.503 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-14T09:05:25.503 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-14T09:05:25.503 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-14T09:05:25.503 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-14T09:05:25.503 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-14T09:05:25.503 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-14T09:05:25.503 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_memory_base": "805306368", 2024-09-14T09:05:25.504 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_memory_cache_min": "134217728", 2024-09-14T09:05:25.504 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-14T09:05:25.504 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-14T09:05:25.504 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_memory_target": "4294967296", 2024-09-14T09:05:25.504 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_memory_target_autotune": "false", 2024-09-14T09:05:25.504 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-14T09:05:25.504 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_min_pg_log_entries": "250", 2024-09-14T09:05:25.504 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_min_recovery_priority": "0", 2024-09-14T09:05:25.504 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-14T09:05:25.504 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-14T09:05:25.504 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mon_report_interval": "5", 2024-09-14T09:05:25.504 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-14T09:05:25.504 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-14T09:05:25.504 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_num_cache_shards": "32", 2024-09-14T09:05:25.505 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_num_op_tracker_shard": "32", 2024-09-14T09:05:25.505 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_numa_auto_affinity": "true", 2024-09-14T09:05:25.505 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_numa_node": "-1", 2024-09-14T09:05:25.505 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_numa_prefer_iface": "true", 2024-09-14T09:05:25.505 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-14T09:05:25.505 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_objecter_finishers": "1", 2024-09-14T09:05:25.505 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_objectstore": "bluestore", 2024-09-14T09:05:25.505 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_objectstore_fuse": "false", 2024-09-14T09:05:25.505 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-14T09:05:25.505 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_objectstore_tracing": "false", 2024-09-14T09:05:25.505 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_op_complaint_time": "30.000000", 2024-09-14T09:05:25.505 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_op_history_duration": "600", 2024-09-14T09:05:25.505 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_op_history_size": "20", 2024-09-14T09:05:25.505 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_op_history_slow_op_size": "20", 2024-09-14T09:05:25.505 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-14T09:05:25.505 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_op_log_threshold": "5", 2024-09-14T09:05:25.506 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_op_num_shards": "0", 2024-09-14T09:05:25.506 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_op_num_shards_hdd": "5", 2024-09-14T09:05:25.506 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_op_num_shards_ssd": "8", 2024-09-14T09:05:25.506 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-14T09:05:25.506 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-14T09:05:25.506 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-14T09:05:25.506 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-14T09:05:25.506 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_op_pq_min_cost": "65536", 2024-09-14T09:05:25.506 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-14T09:05:25.506 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_op_queue_cut_off": "high", 2024-09-14T09:05:25.506 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-14T09:05:25.506 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_op_thread_timeout": "15", 2024-09-14T09:05:25.506 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_open_classes_on_start": "true", 2024-09-14T09:05:25.506 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_os_flags": "0", 2024-09-14T09:05:25.506 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_peering_op_priority": "255", 2024-09-14T09:05:25.507 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pg_delete_cost": "1048576", 2024-09-14T09:05:25.507 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pg_delete_priority": "5", 2024-09-14T09:05:25.507 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-14T09:05:25.507 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-14T09:05:25.507 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-14T09:05:25.507 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pg_log_trim_max": "10000", 2024-09-14T09:05:25.507 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pg_log_trim_min": "100", 2024-09-14T09:05:25.507 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-14T09:05:25.507 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-14T09:05:25.507 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-14T09:05:25.507 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-14T09:05:25.507 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-14T09:05:25.507 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-14T09:05:25.507 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-14T09:05:25.507 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-14T09:05:25.508 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-14T09:05:25.508 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-14T09:05:25.508 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_crimson": "false", 2024-09-14T09:05:25.508 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-14T09:05:25.508 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-14T09:05:25.508 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-14T09:05:25.508 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-14T09:05:25.508 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-14T09:05:25.508 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-14T09:05:25.508 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-14T09:05:25.508 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-14T09:05:25.509 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_flags": "0", 2024-09-14T09:05:25.509 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-14T09:05:25.509 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_min_size": "0", 2024-09-14T09:05:25.509 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-14T09:05:25.509 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_pg_num": "32", 2024-09-14T09:05:25.509 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_pgp_num": "0", 2024-09-14T09:05:25.509 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-14T09:05:25.509 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_read_ratio": "70", 2024-09-14T09:05:25.509 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_size": "2", 2024-09-14T09:05:25.509 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_default_type": "replicated", 2024-09-14T09:05:25.509 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-14T09:05:25.510 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-14T09:05:25.510 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_push_per_object_cost": "1000", 2024-09-14T09:05:25.510 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-14T09:05:25.510 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_recover_clone_overlap": "true", 2024-09-14T09:05:25.510 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-14T09:05:25.510 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_recovery_cost": "20971520", 2024-09-14T09:05:25.510 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-14T09:05:25.510 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_recovery_max_active": "0", 2024-09-14T09:05:25.510 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-14T09:05:25.510 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-14T09:05:25.510 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-14T09:05:25.510 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-14T09:05:25.510 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_recovery_max_single_start": "1", 2024-09-14T09:05:25.511 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_recovery_op_priority": "3", 2024-09-14T09:05:25.511 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-14T09:05:25.511 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_recovery_priority": "5", 2024-09-14T09:05:25.511 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-14T09:05:25.511 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_recovery_sleep": "0.000000", 2024-09-14T09:05:25.511 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-14T09:05:25.511 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-14T09:05:25.511 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-14T09:05:25.511 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_repair_during_recovery": "false", 2024-09-14T09:05:25.511 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_requested_scrub_priority": "120", 2024-09-14T09:05:25.511 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-14T09:05:25.511 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-14T09:05:25.511 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_auto_repair": "false", 2024-09-14T09:05:25.511 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-14T09:05:25.511 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-14T09:05:25.511 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_begin_hour": "0", 2024-09-14T09:05:25.512 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_begin_week_day": "0", 2024-09-14T09:05:25.512 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_chunk_max": "25", 2024-09-14T09:05:25.512 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_chunk_min": "5", 2024-09-14T09:05:25.512 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_cost": "52428800", 2024-09-14T09:05:25.512 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-14T09:05:25.512 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_during_recovery": "false", 2024-09-14T09:05:25.512 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_end_hour": "0", 2024-09-14T09:05:25.512 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_end_week_day": "0", 2024-09-14T09:05:25.512 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_event_cost": "4096", 2024-09-14T09:05:25.512 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-14T09:05:25.513 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-14T09:05:25.513 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_invalid_stats": "true", 2024-09-14T09:05:25.513 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-14T09:05:25.513 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-14T09:05:25.513 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_max_preemptions": "5", 2024-09-14T09:05:25.513 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-14T09:05:25.513 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_priority": "5", 2024-09-14T09:05:25.513 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_scrub_sleep": "0.000000", 2024-09-14T09:05:25.513 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-14T09:05:25.513 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-14T09:05:25.513 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-14T09:05:25.513 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-14T09:05:25.513 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_skip_data_digest": "false", 2024-09-14T09:05:25.513 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_smart_report_timeout": "5", 2024-09-14T09:05:25.513 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_snap_trim_cost": "1048576", 2024-09-14T09:05:25.514 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_snap_trim_priority": "5", 2024-09-14T09:05:25.514 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-14T09:05:25.514 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-14T09:05:25.514 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-14T09:05:25.514 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-14T09:05:25.514 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-14T09:05:25.514 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-14T09:05:25.514 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-14T09:05:25.514 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_target_transaction_size": "30", 2024-09-14T09:05:25.514 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-14T09:05:25.514 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-14T09:05:25.514 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-14T09:05:25.514 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-14T09:05:25.514 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-14T09:05:25.514 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-14T09:05:25.515 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-14T09:05:25.515 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-14T09:05:25.515 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-14T09:05:25.515 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-14T09:05:25.515 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_tracing": "false", 2024-09-14T09:05:25.515 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_use_stale_snap": "false", 2024-09-14T09:05:25.515 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-14T09:05:25.515 INFO:teuthology.orchestra.run.smithi176.stdout: "osdc_blkin_trace_all": "false", 2024-09-14T09:05:25.515 INFO:teuthology.orchestra.run.smithi176.stdout: "paxos_kill_at": "0", 2024-09-14T09:05:25.515 INFO:teuthology.orchestra.run.smithi176.stdout: "paxos_max_join_drift": "10", 2024-09-14T09:05:25.515 INFO:teuthology.orchestra.run.smithi176.stdout: "paxos_min": "500", 2024-09-14T09:05:25.515 INFO:teuthology.orchestra.run.smithi176.stdout: "paxos_min_wait": "0.050000", 2024-09-14T09:05:25.515 INFO:teuthology.orchestra.run.smithi176.stdout: "paxos_propose_interval": "1.000000", 2024-09-14T09:05:25.516 INFO:teuthology.orchestra.run.smithi176.stdout: "paxos_service_trim_max": "500", 2024-09-14T09:05:25.516 INFO:teuthology.orchestra.run.smithi176.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-14T09:05:25.516 INFO:teuthology.orchestra.run.smithi176.stdout: "paxos_service_trim_min": "250", 2024-09-14T09:05:25.516 INFO:teuthology.orchestra.run.smithi176.stdout: "paxos_stash_full_interval": "25", 2024-09-14T09:05:25.516 INFO:teuthology.orchestra.run.smithi176.stdout: "paxos_trim_max": "500", 2024-09-14T09:05:25.516 INFO:teuthology.orchestra.run.smithi176.stdout: "paxos_trim_min": "250", 2024-09-14T09:05:25.516 INFO:teuthology.orchestra.run.smithi176.stdout: "perf": "true", 2024-09-14T09:05:25.516 INFO:teuthology.orchestra.run.smithi176.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-14T09:05:25.516 INFO:teuthology.orchestra.run.smithi176.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-14T09:05:25.516 INFO:teuthology.orchestra.run.smithi176.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-14T09:05:25.516 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "-", 2024-09-14T09:05:25.516 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrv": "", 2024-09-14T09:05:25.516 INFO:teuthology.orchestra.run.smithi176.stdout: "public_bind_addr": "-", 2024-09-14T09:05:25.516 INFO:teuthology.orchestra.run.smithi176.stdout: "public_network": "", 2024-09-14T09:05:25.516 INFO:teuthology.orchestra.run.smithi176.stdout: "public_network_interface": "", 2024-09-14T09:05:25.517 INFO:teuthology.orchestra.run.smithi176.stdout: "qat_compressor_busy_polling": "false", 2024-09-14T09:05:25.517 INFO:teuthology.orchestra.run.smithi176.stdout: "qat_compressor_enabled": "false", 2024-09-14T09:05:25.517 INFO:teuthology.orchestra.run.smithi176.stdout: "qat_compressor_session_max_number": "256", 2024-09-14T09:05:25.517 INFO:teuthology.orchestra.run.smithi176.stdout: "rados_mon_op_timeout": "0", 2024-09-14T09:05:25.517 INFO:teuthology.orchestra.run.smithi176.stdout: "rados_osd_op_timeout": "0", 2024-09-14T09:05:25.517 INFO:teuthology.orchestra.run.smithi176.stdout: "rados_tracing": "false", 2024-09-14T09:05:25.517 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_atime_update_interval": "60", 2024-09-14T09:05:25.517 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-14T09:05:25.517 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_balance_parent_reads": "false", 2024-09-14T09:05:25.517 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_balance_snap_reads": "false", 2024-09-14T09:05:25.517 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_blkin_trace_all": "false", 2024-09-14T09:05:25.517 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-14T09:05:25.517 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-14T09:05:25.517 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_cache": "true", 2024-09-14T09:05:25.517 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-14T09:05:25.518 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-14T09:05:25.518 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-14T09:05:25.518 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-14T09:05:25.518 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_cache_policy": "writearound", 2024-09-14T09:05:25.518 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_cache_size": "33554432", 2024-09-14T09:05:25.518 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-14T09:05:25.518 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-14T09:05:25.518 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_clone_copy_on_read": "false", 2024-09-14T09:05:25.518 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_compression_hint": "none", 2024-09-14T09:05:25.518 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_concurrent_management_ops": "10", 2024-09-14T09:05:25.518 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-14T09:05:25.518 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_default_clone_format": "auto", 2024-09-14T09:05:25.518 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_default_data_pool": "", 2024-09-14T09:05:25.518 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_default_features": "61", 2024-09-14T09:05:25.518 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_default_format": "2", 2024-09-14T09:05:25.519 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_default_map_options": "", 2024-09-14T09:05:25.519 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_default_order": "22", 2024-09-14T09:05:25.519 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_default_pool": "rbd", 2024-09-14T09:05:25.519 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-14T09:05:25.519 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_default_stripe_count": "0", 2024-09-14T09:05:25.519 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_default_stripe_unit": "0", 2024-09-14T09:05:25.519 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-14T09:05:25.519 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-14T09:05:25.519 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-14T09:05:25.519 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_enable_alloc_hint": "true", 2024-09-14T09:05:25.519 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-14T09:05:25.519 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_io_scheduler": "simple", 2024-09-14T09:05:25.519 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-14T09:05:25.519 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-14T09:05:25.519 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-14T09:05:25.519 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-14T09:05:25.520 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-14T09:05:25.520 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-14T09:05:25.520 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-14T09:05:25.520 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-14T09:05:25.520 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-14T09:05:25.520 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_journal_order": "24", 2024-09-14T09:05:25.520 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_journal_pool": "", 2024-09-14T09:05:25.520 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_journal_splay_width": "4", 2024-09-14T09:05:25.520 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_localize_parent_reads": "false", 2024-09-14T09:05:25.520 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_localize_snap_reads": "false", 2024-09-14T09:05:25.520 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-14T09:05:25.520 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-14T09:05:25.520 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-14T09:05:25.520 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-14T09:05:25.521 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-14T09:05:25.521 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-14T09:05:25.521 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-14T09:05:25.521 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-14T09:05:25.521 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-14T09:05:25.521 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-14T09:05:25.521 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-14T09:05:25.521 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-14T09:05:25.521 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-14T09:05:25.521 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-14T09:05:25.521 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-14T09:05:25.521 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-14T09:05:25.521 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-14T09:05:25.521 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-14T09:05:25.521 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-14T09:05:25.522 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-14T09:05:25.522 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-14T09:05:25.522 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-14T09:05:25.522 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-14T09:05:25.522 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-14T09:05:25.522 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-14T09:05:25.522 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-14T09:05:25.522 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-14T09:05:25.522 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-14T09:05:25.522 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-14T09:05:25.522 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-14T09:05:25.522 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-14T09:05:25.522 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_mtime_update_interval": "60", 2024-09-14T09:05:25.522 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_non_blocking_aio": "true", 2024-09-14T09:05:25.522 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_op_thread_timeout": "60", 2024-09-14T09:05:25.523 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_op_threads": "1", 2024-09-14T09:05:25.523 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_parent_cache_enabled": "false", 2024-09-14T09:05:25.523 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-14T09:05:25.523 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-14T09:05:25.523 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-14T09:05:25.523 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_plugins": "", 2024-09-14T09:05:25.523 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_bps_burst": "0", 2024-09-14T09:05:25.523 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-14T09:05:25.523 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_bps_limit": "0", 2024-09-14T09:05:25.523 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_exclude_ops": "0", 2024-09-14T09:05:25.523 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_iops_burst": "0", 2024-09-14T09:05:25.523 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-14T09:05:25.523 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_iops_limit": "0", 2024-09-14T09:05:25.523 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-14T09:05:25.524 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-14T09:05:25.524 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-14T09:05:25.524 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-14T09:05:25.524 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-14T09:05:25.524 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-14T09:05:25.524 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-14T09:05:25.524 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-14T09:05:25.524 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-14T09:05:25.524 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-14T09:05:25.524 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-14T09:05:25.524 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-14T09:05:25.524 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-14T09:05:25.524 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-14T09:05:25.525 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_read_from_replica_policy": "default", 2024-09-14T09:05:25.525 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-14T09:05:25.525 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-14T09:05:25.525 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-14T09:05:25.525 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-14T09:05:25.525 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_skip_partial_discard": "true", 2024-09-14T09:05:25.525 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-14T09:05:25.525 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_tracing": "false", 2024-09-14T09:05:25.525 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_validate_names": "true", 2024-09-14T09:05:25.525 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd_validate_pool": "true", 2024-09-14T09:05:25.525 INFO:teuthology.orchestra.run.smithi176.stdout: "restapi_base_url": "", 2024-09-14T09:05:25.525 INFO:teuthology.orchestra.run.smithi176.stdout: "restapi_log_level": "", 2024-09-14T09:05:25.525 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-14T09:05:25.525 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-14T09:05:25.526 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_acl_grants_max_num": "100", 2024-09-14T09:05:25.526 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_admin_entry": "admin", 2024-09-14T09:05:25.526 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-14T09:05:25.526 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_asio_assert_yielding": "false", 2024-09-14T09:05:25.526 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_backend_store": "rados", 2024-09-14T09:05:25.526 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_barbican_url": "", 2024-09-14T09:05:25.526 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_beast_enable_async": "true", 2024-09-14T09:05:25.526 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_bucket_counters_cache": "false", 2024-09-14T09:05:25.526 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-14T09:05:25.526 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-14T09:05:25.526 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-14T09:05:25.526 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-14T09:05:25.526 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-14T09:05:25.526 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-14T09:05:25.527 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-14T09:05:25.527 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-14T09:05:25.527 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_cache_enabled": "true", 2024-09-14T09:05:25.527 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_cache_expiry_interval": "900", 2024-09-14T09:05:25.527 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_cache_lru_size": "25000", 2024-09-14T09:05:25.527 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_config_store": "rados", 2024-09-14T09:05:25.527 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_content_length_compat": "false", 2024-09-14T09:05:25.527 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_copy_obj_progress": "true", 2024-09-14T09:05:25.527 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-14T09:05:25.527 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_cors_rules_max_num": "100", 2024-09-14T09:05:25.527 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_cross_domain_policy": "", 2024-09-14T09:05:25.527 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-14T09:05:25.528 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_kmip_addr": "", 2024-09-14T09:05:25.528 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-14T09:05:25.528 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-14T09:05:25.528 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-14T09:05:25.528 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-14T09:05:25.528 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_kmip_password": "", 2024-09-14T09:05:25.528 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-14T09:05:25.528 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_kmip_username": "", 2024-09-14T09:05:25.528 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_require_ssl": "true", 2024-09-14T09:05:25.528 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-14T09:05:25.528 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-14T09:05:25.528 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-14T09:05:25.528 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-14T09:05:25.528 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-14T09:05:25.529 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-14T09:05:25.529 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-14T09:05:25.529 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-14T09:05:25.529 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-14T09:05:25.529 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-14T09:05:25.529 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-14T09:05:25.529 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-14T09:05:25.529 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-14T09:05:25.529 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-14T09:05:25.529 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-14T09:05:25.529 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_vault_addr": "", 2024-09-14T09:05:25.529 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_vault_auth": "token", 2024-09-14T09:05:25.529 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_vault_namespace": "", 2024-09-14T09:05:25.530 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_vault_prefix": "", 2024-09-14T09:05:25.530 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-14T09:05:25.530 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-14T09:05:25.530 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-14T09:05:25.530 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-14T09:05:25.530 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_vault_token_file": "", 2024-09-14T09:05:25.530 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-14T09:05:25.530 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_curl_buffersize": "524288", 2024-09-14T09:05:25.530 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-14T09:05:25.530 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_curl_low_speed_time": "300", 2024-09-14T09:05:25.530 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-14T09:05:25.530 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-14T09:05:25.530 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-14T09:05:25.530 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-14T09:05:25.531 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-14T09:05:25.531 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-14T09:05:25.531 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-14T09:05:25.531 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-14T09:05:25.531 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-14T09:05:25.531 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-14T09:05:25.531 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-14T09:05:25.531 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-14T09:05:25.531 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-14T09:05:25.531 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-14T09:05:25.531 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-14T09:05:25.531 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-14T09:05:25.531 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-14T09:05:25.531 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-14T09:05:25.532 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-14T09:05:25.532 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_data_log_changes_size": "1000", 2024-09-14T09:05:25.532 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_data_log_num_shards": "128", 2024-09-14T09:05:25.532 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-14T09:05:25.532 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_data_log_window": "30", 2024-09-14T09:05:25.532 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-14T09:05:25.532 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-14T09:05:25.532 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-14T09:05:25.532 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-14T09:05:25.532 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-14T09:05:25.532 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-14T09:05:25.532 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-14T09:05:25.532 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-14T09:05:25.533 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-14T09:05:25.533 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-14T09:05:25.533 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-14T09:05:25.533 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-14T09:05:25.533 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-14T09:05:25.533 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_disable_s3select": "false", 2024-09-14T09:05:25.533 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-14T09:05:25.533 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-14T09:05:25.533 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-14T09:05:25.533 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-14T09:05:25.533 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-14T09:05:25.533 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-14T09:05:25.533 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-14T09:05:25.533 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-14T09:05:25.533 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-14T09:05:25.534 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-14T09:05:25.534 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-14T09:05:25.534 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-14T09:05:25.534 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_dns_name": "", 2024-09-14T09:05:25.534 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_dns_s3website_name": "", 2024-09-14T09:05:25.534 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_dynamic_resharding": "true", 2024-09-14T09:05:25.534 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-14T09:05:25.534 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-14T09:05:25.534 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-14T09:05:25.534 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_enable_gc_threads": "true", 2024-09-14T09:05:25.534 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_enable_lc_threads": "true", 2024-09-14T09:05:25.534 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_enable_mdsearch": "true", 2024-09-14T09:05:25.534 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_enable_ops_log": "false", 2024-09-14T09:05:25.534 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_enable_quota_threads": "true", 2024-09-14T09:05:25.535 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_enable_static_website": "false", 2024-09-14T09:05:25.535 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_enable_usage_log": "false", 2024-09-14T09:05:25.535 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_enforce_swift_acls": "true", 2024-09-14T09:05:25.535 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_exit_timeout_secs": "120", 2024-09-14T09:05:25.535 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_expose_bucket": "false", 2024-09-14T09:05:25.535 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_extended_http_attrs": "", 2024-09-14T09:05:25.535 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_filter": "none", 2024-09-14T09:05:25.535 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-14T09:05:25.535 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_frontends": "beast port=7480", 2024-09-14T09:05:25.535 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-14T09:05:25.535 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_gc_max_deferred": "50", 2024-09-14T09:05:25.535 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-14T09:05:25.535 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_gc_max_objs": "32", 2024-09-14T09:05:25.536 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-14T09:05:25.536 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-14T09:05:25.536 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-14T09:05:25.536 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-14T09:05:25.536 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_gc_processor_period": "3600", 2024-09-14T09:05:25.536 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-14T09:05:25.536 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-14T09:05:25.536 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-14T09:05:25.536 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-14T09:05:25.536 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_init_timeout": "300", 2024-09-14T09:05:25.536 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-14T09:05:25.536 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-14T09:05:25.536 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_kafka_connection_idle": "300", 2024-09-14T09:05:25.536 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-14T09:05:25.537 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-14T09:05:25.537 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-14T09:05:25.537 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-14T09:05:25.537 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-14T09:05:25.537 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_admin_domain": "", 2024-09-14T09:05:25.537 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_admin_password": "", 2024-09-14T09:05:25.537 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_admin_password_path": "", 2024-09-14T09:05:25.537 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_admin_project": "", 2024-09-14T09:05:25.537 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_admin_tenant": "", 2024-09-14T09:05:25.537 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_admin_token": "", 2024-09-14T09:05:25.537 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_admin_token_path": "", 2024-09-14T09:05:25.537 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_admin_user": "", 2024-09-14T09:05:25.537 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_barbican_domain": "", 2024-09-14T09:05:25.537 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_barbican_password": "", 2024-09-14T09:05:25.538 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_barbican_project": "", 2024-09-14T09:05:25.538 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-14T09:05:25.538 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_barbican_user": "", 2024-09-14T09:05:25.538 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-14T09:05:25.538 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-14T09:05:25.538 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-14T09:05:25.538 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-14T09:05:25.538 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-14T09:05:25.538 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_url": "", 2024-09-14T09:05:25.538 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-14T09:05:25.538 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_lc_debug_interval": "-1", 2024-09-14T09:05:25.538 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_lc_lock_max_time": "90", 2024-09-14T09:05:25.538 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_lc_max_objs": "32", 2024-09-14T09:05:25.539 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_lc_max_rules": "1000", 2024-09-14T09:05:25.539 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_lc_max_worker": "3", 2024-09-14T09:05:25.539 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-14T09:05:25.539 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_lc_thread_delay": "0", 2024-09-14T09:05:25.539 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-14T09:05:25.539 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_ldap_dnattr": "uid", 2024-09-14T09:05:25.539 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-14T09:05:25.539 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_ldap_searchfilter": "", 2024-09-14T09:05:25.539 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-14T09:05:25.539 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_ldap_uri": "", 2024-09-14T09:05:25.539 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-14T09:05:25.539 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-14T09:05:25.540 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-14T09:05:25.540 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-14T09:05:25.540 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_log_http_headers": "", 2024-09-14T09:05:25.540 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-14T09:05:25.540 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-14T09:05:25.540 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_log_object_name_utc": "false", 2024-09-14T09:05:25.540 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-14T09:05:25.540 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-14T09:05:25.540 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_max_attr_name_len": "0", 2024-09-14T09:05:25.540 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_max_attr_size": "0", 2024-09-14T09:05:25.540 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-14T09:05:25.540 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_max_chunk_size": "4194304", 2024-09-14T09:05:25.541 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-14T09:05:25.541 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-14T09:05:25.541 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-14T09:05:25.541 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_max_listing_results": "1000", 2024-09-14T09:05:25.541 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_max_notify_retries": "10", 2024-09-14T09:05:25.541 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-14T09:05:25.541 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_max_put_param_size": "1048576", 2024-09-14T09:05:25.541 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_max_put_size": "5368709120", 2024-09-14T09:05:25.541 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_max_slo_entries": "1000", 2024-09-14T09:05:25.541 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_md_log_max_shards": "64", 2024-09-14T09:05:25.541 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-14T09:05:25.542 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-14T09:05:25.542 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-14T09:05:25.542 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-14T09:05:25.542 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_mp_lock_max_time": "600", 2024-09-14T09:05:25.542 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-14T09:05:25.542 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-14T09:05:25.542 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-14T09:05:25.542 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-14T09:05:25.542 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-14T09:05:25.542 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-14T09:05:25.542 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-14T09:05:25.542 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-14T09:05:25.542 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_nfs_max_gc": "300", 2024-09-14T09:05:25.542 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-14T09:05:25.543 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-14T09:05:25.543 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-14T09:05:25.543 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-14T09:05:25.543 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-14T09:05:25.543 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-14T09:05:25.543 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-14T09:05:25.543 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_num_async_rados_threads": "32", 2024-09-14T09:05:25.543 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_num_control_oids": "8", 2024-09-14T09:05:25.543 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_numa_node": "-1", 2024-09-14T09:05:25.543 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-14T09:05:25.543 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-14T09:05:25.543 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_objexp_chunk_size": "100", 2024-09-14T09:05:25.543 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_objexp_gc_interval": "600", 2024-09-14T09:05:25.543 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-14T09:05:25.544 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-14T09:05:25.544 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-14T09:05:25.544 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_op_thread_timeout": "600", 2024-09-14T09:05:25.544 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_op_tracing": "false", 2024-09-14T09:05:25.544 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_opa_token": "", 2024-09-14T09:05:25.544 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_opa_url": "", 2024-09-14T09:05:25.544 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_opa_verify_ssl": "true", 2024-09-14T09:05:25.544 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-14T09:05:25.544 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-14T09:05:25.544 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_ops_log_rados": "false", 2024-09-14T09:05:25.544 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_ops_log_socket_path": "", 2024-09-14T09:05:25.544 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-14T09:05:25.544 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-14T09:05:25.544 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-14T09:05:25.545 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_period_push_interval": "2.000000", 2024-09-14T09:05:25.545 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-14T09:05:25.545 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-14T09:05:25.545 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-14T09:05:25.545 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-14T09:05:25.545 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_posix_cache_lanes": "3", 2024-09-14T09:05:25.545 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-14T09:05:25.545 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-14T09:05:25.545 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_posix_cache_partitions": "3", 2024-09-14T09:05:25.545 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-14T09:05:25.545 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_print_continue": "true", 2024-09-14T09:05:25.545 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-14T09:05:25.545 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-14T09:05:25.545 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-14T09:05:25.545 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-14T09:05:25.546 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-14T09:05:25.546 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_rados_tracing": "false", 2024-09-14T09:05:25.546 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_realm": "", 2024-09-14T09:05:25.546 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_realm_id": "", 2024-09-14T09:05:25.546 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-14T09:05:25.546 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_region": "", 2024-09-14T09:05:25.546 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-14T09:05:25.546 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-14T09:05:25.546 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-14T09:05:25.546 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-14T09:05:25.546 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_request_uri": "", 2024-09-14T09:05:25.546 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_reshard_batch_size": "64", 2024-09-14T09:05:25.546 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-14T09:05:25.546 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-14T09:05:25.546 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_reshard_max_aio": "128", 2024-09-14T09:05:25.546 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_reshard_num_logs": "16", 2024-09-14T09:05:25.547 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-14T09:05:25.547 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-14T09:05:25.547 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_reshard_thread_interval": "600", 2024-09-14T09:05:25.547 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-14T09:05:25.547 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_resolve_cname": "false", 2024-09-14T09:05:25.547 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-14T09:05:25.547 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_run_sync_thread": "true", 2024-09-14T09:05:25.547 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-14T09:05:25.547 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-14T09:05:25.547 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-14T09:05:25.547 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-14T09:05:25.547 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-14T09:05:25.547 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-14T09:05:25.547 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-14T09:05:25.547 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-14T09:05:25.547 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-14T09:05:25.548 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_scheduler_type": "throttler", 2024-09-14T09:05:25.548 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_script_uri": "", 2024-09-14T09:05:25.548 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_service_provider_name": "", 2024-09-14T09:05:25.548 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-14T09:05:25.548 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_sts_client_id": "", 2024-09-14T09:05:25.548 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_sts_client_secret": "", 2024-09-14T09:05:25.548 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_sts_entry": "sts", 2024-09-14T09:05:25.548 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_sts_key": "sts", 2024-09-14T09:05:25.548 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-14T09:05:25.548 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_sts_min_session_duration": "900", 2024-09-14T09:05:25.548 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_sts_token_introspection_url": "", 2024-09-14T09:05:25.548 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_swift_account_in_url": "false", 2024-09-14T09:05:25.548 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_swift_auth_entry": "auth", 2024-09-14T09:05:25.548 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_swift_auth_url": "", 2024-09-14T09:05:25.548 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_swift_custom_header": "", 2024-09-14T09:05:25.549 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-14T09:05:25.549 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_swift_need_stats": "true", 2024-09-14T09:05:25.549 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_swift_tenant_name": "", 2024-09-14T09:05:25.549 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_swift_token_expiration": "86400", 2024-09-14T09:05:25.549 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_swift_url": "", 2024-09-14T09:05:25.549 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_swift_url_prefix": "swift", 2024-09-14T09:05:25.549 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-14T09:05:25.549 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-14T09:05:25.549 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-14T09:05:25.549 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_sync_lease_period": "120", 2024-09-14T09:05:25.549 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-14T09:05:25.549 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-14T09:05:25.549 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-14T09:05:25.549 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-14T09:05:25.550 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-14T09:05:25.550 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-14T09:05:25.550 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-14T09:05:25.550 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-14T09:05:25.550 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-14T09:05:25.550 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_thread_pool_size": "512", 2024-09-14T09:05:25.550 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-14T09:05:25.550 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-14T09:05:25.550 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-14T09:05:25.550 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-14T09:05:25.550 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_torrent_comment": "", 2024-09-14T09:05:25.550 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_torrent_createby": "", 2024-09-14T09:05:25.550 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_torrent_encoding": "", 2024-09-14T09:05:25.550 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_torrent_flag": "false", 2024-09-14T09:05:25.550 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-14T09:05:25.551 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_torrent_origin": "", 2024-09-14T09:05:25.551 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-14T09:05:25.551 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_torrent_tracker": "", 2024-09-14T09:05:25.551 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_trust_forwarded_https": "false", 2024-09-14T09:05:25.551 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-14T09:05:25.551 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-14T09:05:25.551 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_usage_max_shards": "32", 2024-09-14T09:05:25.551 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_usage_max_user_shards": "1", 2024-09-14T09:05:25.551 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_use_opa_authz": "false", 2024-09-14T09:05:25.551 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_user_counters_cache": "false", 2024-09-14T09:05:25.551 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-14T09:05:25.551 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-14T09:05:25.551 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-14T09:05:25.551 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_user_max_buckets": "1000", 2024-09-14T09:05:25.552 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_user_policies_max_num": "100", 2024-09-14T09:05:25.552 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-14T09:05:25.552 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-14T09:05:25.552 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-14T09:05:25.552 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-14T09:05:25.552 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_user_unique_email": "true", 2024-09-14T09:05:25.552 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_verify_ssl": "true", 2024-09-14T09:05:25.552 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-14T09:05:25.552 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_zone": "", 2024-09-14T09:05:25.552 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_zone_id": "", 2024-09-14T09:05:25.552 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-14T09:05:25.553 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_zonegroup": "", 2024-09-14T09:05:25.553 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_zonegroup_id": "", 2024-09-14T09:05:25.553 INFO:teuthology.orchestra.run.smithi176.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-14T09:05:25.553 INFO:teuthology.orchestra.run.smithi176.stdout: "rgwlc_auto_session_clear": "true", 2024-09-14T09:05:25.553 INFO:teuthology.orchestra.run.smithi176.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-14T09:05:25.553 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_block_size": "4096", 2024-09-14T09:05:25.553 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-14T09:05:25.553 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-14T09:05:25.553 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-14T09:05:25.553 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-14T09:05:25.553 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-14T09:05:25.553 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_cache_size": "536870912", 2024-09-14T09:05:25.553 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-14T09:05:25.553 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-14T09:05:25.554 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-14T09:05:25.554 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-14T09:05:25.554 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-14T09:05:25.554 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-14T09:05:25.554 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-14T09:05:25.554 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-14T09:05:25.554 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_index_type": "binary_search", 2024-09-14T09:05:25.554 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-14T09:05:25.554 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-14T09:05:25.554 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_partition_filters": "false", 2024-09-14T09:05:25.554 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_perf": "false", 2024-09-14T09:05:25.554 INFO:teuthology.orchestra.run.smithi176.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-14T09:05:25.554 INFO:teuthology.orchestra.run.smithi176.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-14T09:05:25.554 INFO:teuthology.orchestra.run.smithi176.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-14T09:05:25.555 INFO:teuthology.orchestra.run.smithi176.stdout: "run_dir": "/var/run/ceph", 2024-09-14T09:05:25.555 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_block_create": "true", 2024-09-14T09:05:25.555 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_cache_lru_size": "67108864", 2024-09-14T09:05:25.555 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-14T09:05:25.555 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-14T09:05:25.555 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_default_max_object_size": "16777216", 2024-09-14T09:05:25.555 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-14T09:05:25.555 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_device_size": "53687091200", 2024-09-14T09:05:25.555 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-14T09:05:25.555 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_full_integrity_check": "false", 2024-09-14T09:05:25.555 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_journal_batch_capacity": "16", 2024-09-14T09:05:25.556 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-14T09:05:25.556 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-14T09:05:25.556 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-14T09:05:25.556 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_main_device_type": "SSD", 2024-09-14T09:05:25.556 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-14T09:05:25.556 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-14T09:05:25.556 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-14T09:05:25.556 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-14T09:05:25.556 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-14T09:05:25.556 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-14T09:05:25.556 INFO:teuthology.orchestra.run.smithi176.stdout: "seastore_segment_size": "67108864", 2024-09-14T09:05:25.556 INFO:teuthology.orchestra.run.smithi176.stdout: "set_keepcaps": "false", 2024-09-14T09:05:25.556 INFO:teuthology.orchestra.run.smithi176.stdout: "setgroup": "", 2024-09-14T09:05:25.556 INFO:teuthology.orchestra.run.smithi176.stdout: "setuser": "", 2024-09-14T09:05:25.556 INFO:teuthology.orchestra.run.smithi176.stdout: "setuser_match_path": "", 2024-09-14T09:05:25.557 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-14T09:05:25.557 INFO:teuthology.orchestra.run.smithi176.stdout: "thp": "false", 2024-09-14T09:05:25.557 INFO:teuthology.orchestra.run.smithi176.stdout: "threadpool_default_timeout": "60", 2024-09-14T09:05:25.557 INFO:teuthology.orchestra.run.smithi176.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-14T09:05:25.557 INFO:teuthology.orchestra.run.smithi176.stdout: "throttler_perf_counter": "true", 2024-09-14T09:05:25.557 INFO:teuthology.orchestra.run.smithi176.stdout: "tmp_dir": "/tmp", 2024-09-14T09:05:25.557 INFO:teuthology.orchestra.run.smithi176.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-14T09:05:25.557 INFO:teuthology.orchestra.run.smithi176.stdout: "uadk_compressor_enabled": "false", 2024-09-14T09:05:25.557 INFO:teuthology.orchestra.run.smithi176.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-14T09:05:25.557 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:25.559 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-14T09:05:25.692 INFO:teuthology.orchestra.run.smithi176.stdout:mon.a: {} 2024-09-14T09:05:25.692 INFO:teuthology.orchestra.run.smithi176.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T09:05:25.705 INFO:teuthology.orchestra.run.smithi176.stdout:mon.b: {} 2024-09-14T09:05:25.705 INFO:teuthology.orchestra.run.smithi176.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T09:05:25.715 INFO:teuthology.orchestra.run.smithi176.stdout:mon.c: {} 2024-09-14T09:05:25.715 INFO:teuthology.orchestra.run.smithi176.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T09:05:25.731 DEBUG:teuthology.orchestra.run.smithi183:> type ceph-objectstore-tool 2024-09-14T09:05:25.737 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T09:05:25.737 DEBUG:teuthology.orchestra.run.smithi176:> type ceph-objectstore-tool 2024-09-14T09:05:25.741 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T09:05:25.742 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-14T09:05:25.751 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-14T09:05:25.751 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-14T09:05:25.751 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:05:25.751 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:05:25.752 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-14T09:05:25.752 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-14T09:05:25.752 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-14T09:05:25.752 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-14T09:05:25.753 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:05:25.753 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-14T09:05:25.753 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:05:25.753 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-14T09:05:25.754 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T09:05:25.754 DEBUG:teuthology.orchestra.run.smithi176:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-14T09:05:25.791 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:25.788+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:25.814 INFO:teuthology.orchestra.run.smithi176.stdout:mapped writes DISABLED 2024-09-14T09:05:25.814 INFO:teuthology.orchestra.run.smithi176.stdout:mapped reads DISABLED 2024-09-14T09:05:25.814 INFO:teuthology.orchestra.run.smithi176.stdout:Seed set to 4305 2024-09-14T09:05:25.858 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:25.856+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:25.905 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:25.905 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:25.905 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:25.905 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:25.921 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:05:25.930 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:05:25.930 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:05:25.947 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:05:25.947 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:05:25.948 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-14T09:05:25.965 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:05:25.965 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:05:25.965 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:25.960+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:25.981 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:05:25.981 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:05:26.002 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:05:26.002 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:05:26.023 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:05:26.023 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:05:26.043 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:26.043 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:26.044 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:05:26.044 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:05:26.055 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:26.056+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:26.062 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:05:26.070 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:05:26.070 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:05:26.119 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-14T09:05:26.156 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:26.156+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:26.181 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:26.181 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:26.194 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:05:26.286 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:26.284+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:26.308 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-14T09:05:26.330 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is set 2024-09-14T09:05:26.359 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:26.359 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:26.360 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:26.356+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:26.373 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:05:26.459 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:26.460+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:26.480 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-14T09:05:26.522 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:26.522 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:26.536 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:05:26.578 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:26.576+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:26.624 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-14T09:05:26.661 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:26.664+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:26.683 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:26.683 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:26.696 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:05:26.763 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:26.764+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:26.823 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-14T09:05:26.865 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:26.865 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:26.865 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:26.860+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:26.876 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:05:26.966 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:26.968+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:26.969 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-14T09:05:27.030 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:27.031 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:27.044 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:05:27.067 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:27.068+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:27.153 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:27.153 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:27.166 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:05:27.167 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:27.168+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:27.268 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:27.272+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:27.327 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:27.327 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:27.341 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:05:27.386 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:27.384+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:27.452 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:27.452 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:27.465 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:05:27.470 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:27.472+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:27.571 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:27.572+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:27.628 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:27.628 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:27.641 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:05:27.672 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:27.668+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:27.771 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:27.771 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:27.772 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:27.772+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:27.785 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:05:27.873 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:27.876+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:27.949 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:27.949 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:27.963 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:05:27.975 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:27.972+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:28.076 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:28.076+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:28.092 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:05:28.124 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:28.124 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:28.137 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:05:28.177 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:28.172+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:28.246 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:05:28.246 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:05:28.262 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:05:28.262 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:05:28.278 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:28.280+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:28.278 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:05:28.278 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:05:28.294 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:05:28.295 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:05:28.301 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:28.301 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:28.315 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:05:28.316 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:05:28.316 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:05:28.333 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:05:28.334 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:05:28.352 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:05:28.352 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:05:28.358 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T09:05:28.370 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:05:28.370 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:05:28.401 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:28.404+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:28.476 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:05:28.476 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:05:28.476 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:05:28.477 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:05:28.477 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:05:28.479 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:28.480+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:28.491 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:05:28.580 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:28.576+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:28.657 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:05:28.657 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:05:28.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:05:28.658 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:05:28.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.658 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4214.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T09:05:28.658 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:17.279713+0000", 2024-09-14T09:05:28.658 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 11.379140504, 2024-09-14T09:05:28.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042262769999999996, 2024-09-14T09:05:28.658 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:28.658 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:28.658 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:28.658 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:28.658 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4214", 2024-09-14T09:05:28.658 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3243929894", 2024-09-14T09:05:28.658 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-09-14T09:05:28.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.659 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:28.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:28.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.279713+0000", 2024-09-14T09:05:28.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:28.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:28.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.279713+0000", 2024-09-14T09:05:28.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:28.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:28.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.279724+0000", 2024-09-14T09:05:28.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.111e-05 2024-09-14T09:05:28.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:28.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.279737+0000", 2024-09-14T09:05:28.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2955999999999999e-05 2024-09-14T09:05:28.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:28.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.279767+0000", 2024-09-14T09:05:28.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0633999999999998e-05 2024-09-14T09:05:28.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:28.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.279792+0000", 2024-09-14T09:05:28.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5109e-05 2024-09-14T09:05:28.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:28.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.279947+0000", 2024-09-14T09:05:28.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015501600000000001 2024-09-14T09:05:28.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:28.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.280305+0000", 2024-09-14T09:05:28.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035740400000000002 2024-09-14T09:05:28.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-09-14T09:05:28.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.280585+0000", 2024-09-14T09:05:28.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000280105 2024-09-14T09:05:28.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:28.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.282869+0000", 2024-09-14T09:05:28.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00228452 2024-09-14T09:05:28.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:28.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.283675+0000", 2024-09-14T09:05:28.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00080521499999999997 2024-09-14T09:05:28.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:28.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.283830+0000", 2024-09-14T09:05:28.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015576199999999999 2024-09-14T09:05:28.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:28.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.283939+0000", 2024-09-14T09:05:28.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000108446 2024-09-14T09:05:28.664 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:28.664 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:28.664 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:28.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.664 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4214.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-14T09:05:28.664 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:17.283360+0000", 2024-09-14T09:05:28.664 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 11.375493005999999, 2024-09-14T09:05:28.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00059344599999999997, 2024-09-14T09:05:28.665 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:28.665 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:28.665 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:05:28.665 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:28.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:28.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.283360+0000", 2024-09-14T09:05:28.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:28.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:28.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.283360+0000", 2024-09-14T09:05:28.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:28.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:28.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.283362+0000", 2024-09-14T09:05:28.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9e-06 2024-09-14T09:05:28.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:28.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.283376+0000", 2024-09-14T09:05:28.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.358e-05 2024-09-14T09:05:28.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:28.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.283414+0000", 2024-09-14T09:05:28.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8296000000000001e-05 2024-09-14T09:05:28.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:28.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.283438+0000", 2024-09-14T09:05:28.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3626e-05 2024-09-14T09:05:28.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:28.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.283631+0000", 2024-09-14T09:05:28.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000193866 2024-09-14T09:05:28.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:28.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.283655+0000", 2024-09-14T09:05:28.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3597000000000001e-05 2024-09-14T09:05:28.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:28.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.283954+0000", 2024-09-14T09:05:28.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029858100000000002 2024-09-14T09:05:28.668 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:28.668 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:28.668 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:28.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.668 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4214.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-14T09:05:28.668 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:17.289017+0000", 2024-09-14T09:05:28.668 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 11.369836613, 2024-09-14T09:05:28.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040572399999999999, 2024-09-14T09:05:28.669 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:28.669 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:28.669 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:05:28.669 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:28.669 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4214", 2024-09-14T09:05:28.669 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3243929894", 2024-09-14T09:05:28.669 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 2 2024-09-14T09:05:28.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.669 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:28.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:28.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.289017+0000", 2024-09-14T09:05:28.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:28.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.289017+0000", 2024-09-14T09:05:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.289019+0000", 2024-09-14T09:05:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4729999999999999e-06 2024-09-14T09:05:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.289032+0000", 2024-09-14T09:05:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2726e-05 2024-09-14T09:05:28.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.289062+0000", 2024-09-14T09:05:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0726999999999999e-05 2024-09-14T09:05:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.289079+0000", 2024-09-14T09:05:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6702999999999999e-05 2024-09-14T09:05:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.289201+0000", 2024-09-14T09:05:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012223900000000001 2024-09-14T09:05:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.289422+0000", 2024-09-14T09:05:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022085599999999999 2024-09-14T09:05:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.672 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4214.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-14T09:05:28.673 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:17.289871+0000", 2024-09-14T09:05:28.673 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 11.36898212, 2024-09-14T09:05:28.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036247800000000001, 2024-09-14T09:05:28.673 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:28.673 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:28.673 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:05:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4214", 2024-09-14T09:05:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3243929894", 2024-09-14T09:05:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 3 2024-09-14T09:05:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.289871+0000", 2024-09-14T09:05:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.289871+0000", 2024-09-14T09:05:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.289873+0000", 2024-09-14T09:05:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3070000000000001e-06 2024-09-14T09:05:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.289885+0000", 2024-09-14T09:05:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1617000000000001e-05 2024-09-14T09:05:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:28.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.289916+0000", 2024-09-14T09:05:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1099999999999997e-05 2024-09-14T09:05:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.289938+0000", 2024-09-14T09:05:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2402999999999999e-05 2024-09-14T09:05:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.290060+0000", 2024-09-14T09:05:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121552 2024-09-14T09:05:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:17.290233+0000", 2024-09-14T09:05:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000173499 2024-09-14T09:05:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4214.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-14T09:05:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:18.294868+0000", 2024-09-14T09:05:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.363985040999999, 2024-09-14T09:05:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.009136992, 2024-09-14T09:05:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:28.677 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4214", 2024-09-14T09:05:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3243929894", 2024-09-14T09:05:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 5 2024-09-14T09:05:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.294868+0000", 2024-09-14T09:05:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.294868+0000", 2024-09-14T09:05:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:28.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.294871+0000", 2024-09-14T09:05:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.247e-06 2024-09-14T09:05:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.294885+0000", 2024-09-14T09:05:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3497e-05 2024-09-14T09:05:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.294919+0000", 2024-09-14T09:05:28.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4425999999999999e-05 2024-09-14T09:05:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.294945+0000", 2024-09-14T09:05:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5723000000000001e-05 2024-09-14T09:05:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.295092+0000", 2024-09-14T09:05:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014744199999999999 2024-09-14T09:05:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op must wait for map", 2024-09-14T09:05:28.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.295106+0000", 2024-09-14T09:05:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3413e-05 2024-09-14T09:05:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.300333+0000", 2024-09-14T09:05:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052275070000000002 2024-09-14T09:05:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.300707+0000", 2024-09-14T09:05:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037337699999999999 2024-09-14T09:05:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:05:28.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.301070+0000", 2024-09-14T09:05:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000363631 2024-09-14T09:05:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.302733+0000", 2024-09-14T09:05:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016627110000000001 2024-09-14T09:05:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.303744+0000", 2024-09-14T09:05:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010107829999999999 2024-09-14T09:05:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:28.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.303888+0000", 2024-09-14T09:05:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000144633 2024-09-14T09:05:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.304005+0000", 2024-09-14T09:05:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011660200000000001 2024-09-14T09:05:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4214.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-14T09:05:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:18.303522+0000", 2024-09-14T09:05:28.683 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.355331635000001, 2024-09-14T09:05:28.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050191899999999998, 2024-09-14T09:05:28.684 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:28.684 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:28.684 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:05:28.684 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:28.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:28.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.303522+0000", 2024-09-14T09:05:28.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:28.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:28.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.303522+0000", 2024-09-14T09:05:28.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:28.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:28.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.303523+0000", 2024-09-14T09:05:28.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.827e-06 2024-09-14T09:05:28.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:28.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.303533+0000", 2024-09-14T09:05:28.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2299999999999997e-06 2024-09-14T09:05:28.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:28.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.303561+0000", 2024-09-14T09:05:28.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8612999999999999e-05 2024-09-14T09:05:28.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:28.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.303575+0000", 2024-09-14T09:05:28.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4109e-05 2024-09-14T09:05:28.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:28.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.303705+0000", 2024-09-14T09:05:28.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013015600000000001 2024-09-14T09:05:28.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:28.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.303728+0000", 2024-09-14T09:05:28.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2212999999999999e-05 2024-09-14T09:05:28.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:28.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:18.304023+0000", 2024-09-14T09:05:28.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029577100000000001 2024-09-14T09:05:28.687 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:28.687 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:28.687 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:28.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.688 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4214.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-14T09:05:28.688 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:19.306486+0000", 2024-09-14T09:05:28.688 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.3523668850000004, 2024-09-14T09:05:28.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0070416070000000001, 2024-09-14T09:05:28.688 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:28.688 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:28.689 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:28.689 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:28.689 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4214", 2024-09-14T09:05:28.689 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3243929894", 2024-09-14T09:05:28.689 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 7 2024-09-14T09:05:28.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.689 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:28.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:28.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.306486+0000", 2024-09-14T09:05:28.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:28.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:28.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.306486+0000", 2024-09-14T09:05:28.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:28.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:28.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.306489+0000", 2024-09-14T09:05:28.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.027e-06 2024-09-14T09:05:28.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:28.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.306504+0000", 2024-09-14T09:05:28.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4419e-05 2024-09-14T09:05:28.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:28.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.306537+0000", 2024-09-14T09:05:28.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3077000000000002e-05 2024-09-14T09:05:28.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:28.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.306555+0000", 2024-09-14T09:05:28.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7836e-05 2024-09-14T09:05:28.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:28.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.306726+0000", 2024-09-14T09:05:28.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000171869 2024-09-14T09:05:28.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op must wait for map", 2024-09-14T09:05:28.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.306746+0000", 2024-09-14T09:05:28.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9677000000000002e-05 2024-09-14T09:05:28.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:28.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.310563+0000", 2024-09-14T09:05:28.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038171699999999999 2024-09-14T09:05:28.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:28.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.310726+0000", 2024-09-14T09:05:28.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016288199999999999 2024-09-14T09:05:28.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:05:28.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.310931+0000", 2024-09-14T09:05:28.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020511100000000001 2024-09-14T09:05:28.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:28.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.312719+0000", 2024-09-14T09:05:28.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001787748 2024-09-14T09:05:28.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:28.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.313376+0000", 2024-09-14T09:05:28.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00065648900000000001 2024-09-14T09:05:28.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:28.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.313461+0000", 2024-09-14T09:05:28.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5385999999999999e-05 2024-09-14T09:05:28.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:28.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.313528+0000", 2024-09-14T09:05:28.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6915999999999994e-05 2024-09-14T09:05:28.696 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:28.696 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:28.696 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:28.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.696 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4214.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-14T09:05:28.696 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:19.313170+0000", 2024-09-14T09:05:28.696 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.3456829520000007, 2024-09-14T09:05:28.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037024099999999999, 2024-09-14T09:05:28.696 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:28.696 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:28.696 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:05:28.697 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:28.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:28.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.313170+0000", 2024-09-14T09:05:28.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:28.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:28.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.313170+0000", 2024-09-14T09:05:28.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:28.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:28.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.313172+0000", 2024-09-14T09:05:28.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2469999999999998e-06 2024-09-14T09:05:28.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:28.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.313182+0000", 2024-09-14T09:05:28.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8490000000000001e-06 2024-09-14T09:05:28.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:28.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.313214+0000", 2024-09-14T09:05:28.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1420000000000001e-05 2024-09-14T09:05:28.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:28.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.313228+0000", 2024-09-14T09:05:28.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.433e-05 2024-09-14T09:05:28.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:28.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.313353+0000", 2024-09-14T09:05:28.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000125036 2024-09-14T09:05:28.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:28.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.313367+0000", 2024-09-14T09:05:28.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3643e-05 2024-09-14T09:05:28.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:28.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:28.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:28.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:19.313540+0000", 2024-09-14T09:05:28.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000173716 2024-09-14T09:05:28.701 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:28.701 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:28.701 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:28.701 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:28.701 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:28.701 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:05:28.703 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:05:28.703 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:28.704+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:28.804 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:28.800+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:28.823 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is set 2024-09-14T09:05:28.832 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:05:28.832 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:05:28.832 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:05:28.833 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:05:28.851 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:05:28.904 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:28.908+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:28.973 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:05:28.973 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:05:28.973 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:05:28.973 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:05:28.973 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:05:28.987 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:05:29.029 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:29.032+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:29.106 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:29.104+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:29.154 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:05:29.154 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:05:29.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:05:29.154 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-09-14T09:05:29.154 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:05:29.168 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:05:29.212 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:29.216+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:29.287 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:05:29.288 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:05:29.288 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:05:29.288 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:05:29.301 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:05:29.308 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:29.304+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:29.409 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:29.404+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:29.468 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:05:29.468 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:05:29.468 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:05:29.468 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:05:29.468 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:05:29.482 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:05:29.509 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:29.504+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:29.603 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:05:29.603 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:05:29.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:05:29.603 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:05:29.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.604 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.604 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.275457+0000", 2024-09-14T09:05:29.604 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.326611631999999, 2024-09-14T09:05:29.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013137757, 2024-09-14T09:05:29.604 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.604 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.604 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.604 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.604 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.604 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.604 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 24 2024-09-14T09:05:29.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.604 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275457+0000", 2024-09-14T09:05:29.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275457+0000", 2024-09-14T09:05:29.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275458+0000", 2024-09-14T09:05:29.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0240000000000001e-06 2024-09-14T09:05:29.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275626+0000", 2024-09-14T09:05:29.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000168443 2024-09-14T09:05:29.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275645+0000", 2024-09-14T09:05:29.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8987999999999998e-05 2024-09-14T09:05:29.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275655+0000", 2024-09-14T09:05:29.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0183999999999999e-05 2024-09-14T09:05:29.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.281987+0000", 2024-09-14T09:05:29.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063320709999999999 2024-09-14T09:05:29.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.282158+0000", 2024-09-14T09:05:29.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017053599999999999 2024-09-14T09:05:29.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.282381+0000", 2024-09-14T09:05:29.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022338800000000001 2024-09-14T09:05:29.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.285360+0000", 2024-09-14T09:05:29.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029784960000000002 2024-09-14T09:05:29.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.288429+0000", 2024-09-14T09:05:29.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030686239999999998 2024-09-14T09:05:29.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.288522+0000", 2024-09-14T09:05:29.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3820000000000006e-05 2024-09-14T09:05:29.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.288595+0000", 2024-09-14T09:05:29.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2182999999999997e-05 2024-09-14T09:05:29.610 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.610 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.610 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.610 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.610 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.275677+0000", 2024-09-14T09:05:29.610 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.326391400999999, 2024-09-14T09:05:29.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013095005, 2024-09-14T09:05:29.610 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.610 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.610 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.610 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.610 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.611 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.611 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 25 2024-09-14T09:05:29.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.611 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275677+0000", 2024-09-14T09:05:29.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275677+0000", 2024-09-14T09:05:29.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275678+0000", 2024-09-14T09:05:29.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0440000000000001e-06 2024-09-14T09:05:29.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275687+0000", 2024-09-14T09:05:29.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6659999999999997e-06 2024-09-14T09:05:29.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275701+0000", 2024-09-14T09:05:29.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4591e-05 2024-09-14T09:05:29.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275709+0000", 2024-09-14T09:05:29.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0439999999999996e-06 2024-09-14T09:05:29.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.283868+0000", 2024-09-14T09:05:29.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0081590250000000003 2024-09-14T09:05:29.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.283953+0000", 2024-09-14T09:05:29.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5044999999999996e-05 2024-09-14T09:05:29.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.284083+0000", 2024-09-14T09:05:29.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012993299999999999 2024-09-14T09:05:29.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.285813+0000", 2024-09-14T09:05:29.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001729424 2024-09-14T09:05:29.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.288608+0000", 2024-09-14T09:05:29.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027955100000000002 2024-09-14T09:05:29.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.288716+0000", 2024-09-14T09:05:29.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000107613 2024-09-14T09:05:29.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.288772+0000", 2024-09-14T09:05:29.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6110000000000003e-05 2024-09-14T09:05:29.615 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.615 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.615 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.616 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.616 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.378740+0000", 2024-09-14T09:05:29.616 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.223328119000001, 2024-09-14T09:05:29.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.012652679999999999, 2024-09-14T09:05:29.616 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.616 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.616 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.617 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.617 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.617 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.617 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 44 2024-09-14T09:05:29.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.617 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378740+0000", 2024-09-14T09:05:29.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378740+0000", 2024-09-14T09:05:29.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378741+0000", 2024-09-14T09:05:29.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2699999999999998e-07 2024-09-14T09:05:29.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378847+0000", 2024-09-14T09:05:29.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000105466 2024-09-14T09:05:29.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378859+0000", 2024-09-14T09:05:29.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.235e-05 2024-09-14T09:05:29.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378869+0000", 2024-09-14T09:05:29.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0213e-05 2024-09-14T09:05:29.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.386294+0000", 2024-09-14T09:05:29.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0074248300000000003 2024-09-14T09:05:29.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.386537+0000", 2024-09-14T09:05:29.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024260899999999999 2024-09-14T09:05:29.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.386943+0000", 2024-09-14T09:05:29.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040585399999999998 2024-09-14T09:05:29.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.389595+0000", 2024-09-14T09:05:29.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026525609999999999 2024-09-14T09:05:29.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.391137+0000", 2024-09-14T09:05:29.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015414890000000001 2024-09-14T09:05:29.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.391274+0000", 2024-09-14T09:05:29.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013721499999999999 2024-09-14T09:05:29.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.391393+0000", 2024-09-14T09:05:29.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119166 2024-09-14T09:05:29.622 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.622 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.622 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.622 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.622 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.378892+0000", 2024-09-14T09:05:29.622 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.223176252999998, 2024-09-14T09:05:29.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016746298, 2024-09-14T09:05:29.622 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.622 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.623 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.623 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.623 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.623 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.623 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 45 2024-09-14T09:05:29.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.623 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378892+0000", 2024-09-14T09:05:29.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378892+0000", 2024-09-14T09:05:29.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378893+0000", 2024-09-14T09:05:29.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.08e-06 2024-09-14T09:05:29.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378901+0000", 2024-09-14T09:05:29.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7770000000000002e-06 2024-09-14T09:05:29.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378914+0000", 2024-09-14T09:05:29.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.271e-05 2024-09-14T09:05:29.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378927+0000", 2024-09-14T09:05:29.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3166000000000001e-05 2024-09-14T09:05:29.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.389712+0000", 2024-09-14T09:05:29.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.01078542 2024-09-14T09:05:29.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.389953+0000", 2024-09-14T09:05:29.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024113100000000001 2024-09-14T09:05:29.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.390323+0000", 2024-09-14T09:05:29.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036942099999999999 2024-09-14T09:05:29.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395157+0000", 2024-09-14T09:05:29.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048345690000000004 2024-09-14T09:05:29.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395371+0000", 2024-09-14T09:05:29.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000213842 2024-09-14T09:05:29.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395520+0000", 2024-09-14T09:05:29.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014902599999999999 2024-09-14T09:05:29.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395638+0000", 2024-09-14T09:05:29.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118156 2024-09-14T09:05:29.628 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.628 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.628 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.628 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.628 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.378952+0000", 2024-09-14T09:05:29.628 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.223116306000001, 2024-09-14T09:05:29.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016969825000000001, 2024-09-14T09:05:29.629 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.629 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.629 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.629 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.629 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.629 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.629 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 46 2024-09-14T09:05:29.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.629 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378952+0000", 2024-09-14T09:05:29.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378952+0000", 2024-09-14T09:05:29.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378953+0000", 2024-09-14T09:05:29.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3200000000000001e-06 2024-09-14T09:05:29.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378964+0000", 2024-09-14T09:05:29.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0183000000000001e-05 2024-09-14T09:05:29.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378991+0000", 2024-09-14T09:05:29.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7158999999999999e-05 2024-09-14T09:05:29.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379006+0000", 2024-09-14T09:05:29.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.49e-05 2024-09-14T09:05:29.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.391517+0000", 2024-09-14T09:05:29.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.012511520999999999 2024-09-14T09:05:29.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.391677+0000", 2024-09-14T09:05:29.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000159856 2024-09-14T09:05:29.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.392054+0000", 2024-09-14T09:05:29.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037711299999999999 2024-09-14T09:05:29.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395333+0000", 2024-09-14T09:05:29.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0032786310000000002 2024-09-14T09:05:29.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395666+0000", 2024-09-14T09:05:29.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033285399999999999 2024-09-14T09:05:29.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395811+0000", 2024-09-14T09:05:29.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014544300000000001 2024-09-14T09:05:29.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395922+0000", 2024-09-14T09:05:29.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000110845 2024-09-14T09:05:29.633 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.634 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.634 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.634 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.634 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379034+0000", 2024-09-14T09:05:29.634 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.223034600999998, 2024-09-14T09:05:29.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018420946000000001, 2024-09-14T09:05:29.634 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.634 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.634 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.634 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.635 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.635 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.635 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 47 2024-09-14T09:05:29.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.635 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379034+0000", 2024-09-14T09:05:29.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379034+0000", 2024-09-14T09:05:29.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379035+0000", 2024-09-14T09:05:29.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4839999999999999e-06 2024-09-14T09:05:29.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379160+0000", 2024-09-14T09:05:29.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012504200000000001 2024-09-14T09:05:29.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379177+0000", 2024-09-14T09:05:29.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.698e-05 2024-09-14T09:05:29.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379186+0000", 2024-09-14T09:05:29.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0000000000000002e-06 2024-09-14T09:05:29.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.392838+0000", 2024-09-14T09:05:29.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013651949 2024-09-14T09:05:29.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.393080+0000", 2024-09-14T09:05:29.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024200900000000001 2024-09-14T09:05:29.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.393455+0000", 2024-09-14T09:05:29.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037501 2024-09-14T09:05:29.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.397149+0000", 2024-09-14T09:05:29.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0036935409999999998 2024-09-14T09:05:29.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.397187+0000", 2024-09-14T09:05:29.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8445999999999997e-05 2024-09-14T09:05:29.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.397323+0000", 2024-09-14T09:05:29.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135599 2024-09-14T09:05:29.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.397455+0000", 2024-09-14T09:05:29.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000131886 2024-09-14T09:05:29.640 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.640 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.640 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.640 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.640 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379206+0000", 2024-09-14T09:05:29.640 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.222862495000001, 2024-09-14T09:05:29.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.022450290000000001, 2024-09-14T09:05:29.640 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.640 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.640 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.641 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.641 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.641 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.641 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 48 2024-09-14T09:05:29.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.641 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379206+0000", 2024-09-14T09:05:29.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379206+0000", 2024-09-14T09:05:29.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379207+0000", 2024-09-14T09:05:29.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.167e-06 2024-09-14T09:05:29.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379216+0000", 2024-09-14T09:05:29.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1630000000000002e-06 2024-09-14T09:05:29.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379226+0000", 2024-09-14T09:05:29.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9029999999999999e-06 2024-09-14T09:05:29.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379233+0000", 2024-09-14T09:05:29.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.17e-06 2024-09-14T09:05:29.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395986+0000", 2024-09-14T09:05:29.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016752195000000001 2024-09-14T09:05:29.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.396143+0000", 2024-09-14T09:05:29.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000157878 2024-09-14T09:05:29.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.396346+0000", 2024-09-14T09:05:29.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020270499999999999 2024-09-14T09:05:29.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401328+0000", 2024-09-14T09:05:29.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0049818379999999997 2024-09-14T09:05:29.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401367+0000", 2024-09-14T09:05:29.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8926e-05 2024-09-14T09:05:29.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401541+0000", 2024-09-14T09:05:29.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017380600000000001 2024-09-14T09:05:29.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401656+0000", 2024-09-14T09:05:29.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011553900000000001 2024-09-14T09:05:29.646 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.646 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.646 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.646 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.646 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379246+0000", 2024-09-14T09:05:29.646 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.222822711999999, 2024-09-14T09:05:29.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.023032166, 2024-09-14T09:05:29.646 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.646 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.646 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.646 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.647 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.647 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.647 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49 2024-09-14T09:05:29.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.647 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379246+0000", 2024-09-14T09:05:29.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379246+0000", 2024-09-14T09:05:29.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379247+0000", 2024-09-14T09:05:29.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7300000000000005e-07 2024-09-14T09:05:29.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379253+0000", 2024-09-14T09:05:29.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.28e-06 2024-09-14T09:05:29.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379267+0000", 2024-09-14T09:05:29.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4087000000000001e-05 2024-09-14T09:05:29.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379274+0000", 2024-09-14T09:05:29.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8929999999999999e-06 2024-09-14T09:05:29.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.397561+0000", 2024-09-14T09:05:29.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018286903 2024-09-14T09:05:29.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.397798+0000", 2024-09-14T09:05:29.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000236989 2024-09-14T09:05:29.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.398157+0000", 2024-09-14T09:05:29.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035888700000000001 2024-09-14T09:05:29.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401799+0000", 2024-09-14T09:05:29.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0036428139999999999 2024-09-14T09:05:29.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402029+0000", 2024-09-14T09:05:29.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022967800000000001 2024-09-14T09:05:29.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402165+0000", 2024-09-14T09:05:29.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013627600000000001 2024-09-14T09:05:29.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402278+0000", 2024-09-14T09:05:29.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011248599999999999 2024-09-14T09:05:29.651 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.652 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.652 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.652 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.652 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379312+0000", 2024-09-14T09:05:29.652 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.222755935999999, 2024-09-14T09:05:29.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.024727301, 2024-09-14T09:05:29.652 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.652 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.652 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.652 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.652 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.652 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.652 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 50 2024-09-14T09:05:29.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.653 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379312+0000", 2024-09-14T09:05:29.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379312+0000", 2024-09-14T09:05:29.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379314+0000", 2024-09-14T09:05:29.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.17e-06 2024-09-14T09:05:29.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379408+0000", 2024-09-14T09:05:29.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4562999999999996e-05 2024-09-14T09:05:29.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379429+0000", 2024-09-14T09:05:29.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1189e-05 2024-09-14T09:05:29.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379448+0000", 2024-09-14T09:05:29.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8734000000000001e-05 2024-09-14T09:05:29.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.399051+0000", 2024-09-14T09:05:29.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.019602441000000002 2024-09-14T09:05:29.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.399218+0000", 2024-09-14T09:05:29.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000167095 2024-09-14T09:05:29.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.399438+0000", 2024-09-14T09:05:29.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022046500000000001 2024-09-14T09:05:29.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401991+0000", 2024-09-14T09:05:29.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025529250000000002 2024-09-14T09:05:29.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.403780+0000", 2024-09-14T09:05:29.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017893939999999999 2024-09-14T09:05:29.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.403915+0000", 2024-09-14T09:05:29.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013478200000000001 2024-09-14T09:05:29.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.404040+0000", 2024-09-14T09:05:29.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012454299999999999 2024-09-14T09:05:29.657 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.657 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.657 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.657 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.658 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379481+0000", 2024-09-14T09:05:29.658 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.222587004000001, 2024-09-14T09:05:29.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.026050659, 2024-09-14T09:05:29.658 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.658 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.658 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.658 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.658 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.658 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.658 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 51 2024-09-14T09:05:29.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.658 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379481+0000", 2024-09-14T09:05:29.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379481+0000", 2024-09-14T09:05:29.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379483+0000", 2024-09-14T09:05:29.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3799999999999999e-06 2024-09-14T09:05:29.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379493+0000", 2024-09-14T09:05:29.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0164e-05 2024-09-14T09:05:29.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379535+0000", 2024-09-14T09:05:29.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1672999999999999e-05 2024-09-14T09:05:29.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379552+0000", 2024-09-14T09:05:29.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7615999999999999e-05 2024-09-14T09:05:29.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402365+0000", 2024-09-14T09:05:29.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.022812857999999998 2024-09-14T09:05:29.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402601+0000", 2024-09-14T09:05:29.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000235648 2024-09-14T09:05:29.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402966+0000", 2024-09-14T09:05:29.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036486100000000002 2024-09-14T09:05:29.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405223+0000", 2024-09-14T09:05:29.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002257144 2024-09-14T09:05:29.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405271+0000", 2024-09-14T09:05:29.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7769999999999998e-05 2024-09-14T09:05:29.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405410+0000", 2024-09-14T09:05:29.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013921499999999999 2024-09-14T09:05:29.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405532+0000", 2024-09-14T09:05:29.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012233 2024-09-14T09:05:29.664 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.664 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.664 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.664 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.664 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379582+0000", 2024-09-14T09:05:29.664 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.222486295, 2024-09-14T09:05:29.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.026468416000000002, 2024-09-14T09:05:29.664 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.664 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.664 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.664 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.665 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.665 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.665 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 52 2024-09-14T09:05:29.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.665 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379582+0000", 2024-09-14T09:05:29.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379582+0000", 2024-09-14T09:05:29.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379584+0000", 2024-09-14T09:05:29.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6139999999999999e-06 2024-09-14T09:05:29.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379596+0000", 2024-09-14T09:05:29.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1956e-05 2024-09-14T09:05:29.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379614+0000", 2024-09-14T09:05:29.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8099999999999999e-05 2024-09-14T09:05:29.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379638+0000", 2024-09-14T09:05:29.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4136999999999999e-05 2024-09-14T09:05:29.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.404134+0000", 2024-09-14T09:05:29.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.024496418999999998 2024-09-14T09:05:29.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.404400+0000", 2024-09-14T09:05:29.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026532500000000001 2024-09-14T09:05:29.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.404616+0000", 2024-09-14T09:05:29.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021586500000000001 2024-09-14T09:05:29.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405703+0000", 2024-09-14T09:05:29.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010879220000000001 2024-09-14T09:05:29.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405887+0000", 2024-09-14T09:05:29.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018396499999999999 2024-09-14T09:05:29.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405985+0000", 2024-09-14T09:05:29.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8033000000000001e-05 2024-09-14T09:05:29.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.406051+0000", 2024-09-14T09:05:29.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5079999999999994e-05 2024-09-14T09:05:29.670 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.670 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.670 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.670 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:56 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.670 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.407006+0000", 2024-09-14T09:05:29.670 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.195062885999999, 2024-09-14T09:05:29.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011314423000000001, 2024-09-14T09:05:29.670 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.670 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.670 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.670 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.671 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.671 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.671 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-09-14T09:05:29.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.671 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.407006+0000", 2024-09-14T09:05:29.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.407006+0000", 2024-09-14T09:05:29.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.407007+0000", 2024-09-14T09:05:29.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.13e-06 2024-09-14T09:05:29.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.407079+0000", 2024-09-14T09:05:29.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2189999999999998e-05 2024-09-14T09:05:29.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.407091+0000", 2024-09-14T09:05:29.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.224e-05 2024-09-14T09:05:29.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.407099+0000", 2024-09-14T09:05:29.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0900000000000005e-06 2024-09-14T09:05:29.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.414622+0000", 2024-09-14T09:05:29.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075226959999999997 2024-09-14T09:05:29.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.414854+0000", 2024-09-14T09:05:29.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00023222499999999999 2024-09-14T09:05:29.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.415301+0000", 2024-09-14T09:05:29.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00044665299999999998 2024-09-14T09:05:29.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.417204+0000", 2024-09-14T09:05:29.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00190299 2024-09-14T09:05:29.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.418087+0000", 2024-09-14T09:05:29.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00088314999999999997 2024-09-14T09:05:29.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.418214+0000", 2024-09-14T09:05:29.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012739000000000001 2024-09-14T09:05:29.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.418320+0000", 2024-09-14T09:05:29.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010566899999999999 2024-09-14T09:05:29.675 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.675 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.675 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.676 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.676 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.646870+0000", 2024-09-14T09:05:29.676 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.955198360000001, 2024-09-14T09:05:29.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013689005000000001, 2024-09-14T09:05:29.676 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.676 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.676 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.676 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.676 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.676 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.676 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 69 2024-09-14T09:05:29.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.676 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.646870+0000", 2024-09-14T09:05:29.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.646870+0000", 2024-09-14T09:05:29.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.646871+0000", 2024-09-14T09:05:29.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4899999999999995e-07 2024-09-14T09:05:29.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.646957+0000", 2024-09-14T09:05:29.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6310999999999994e-05 2024-09-14T09:05:29.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.646971+0000", 2024-09-14T09:05:29.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3895e-05 2024-09-14T09:05:29.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.646980+0000", 2024-09-14T09:05:29.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7960000000000001e-06 2024-09-14T09:05:29.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.652859+0000", 2024-09-14T09:05:29.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058793040000000001 2024-09-14T09:05:29.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.653028+0000", 2024-09-14T09:05:29.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016869300000000001 2024-09-14T09:05:29.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.653216+0000", 2024-09-14T09:05:29.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018876599999999999 2024-09-14T09:05:29.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.656708+0000", 2024-09-14T09:05:29.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003491105 2024-09-14T09:05:29.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660421+0000", 2024-09-14T09:05:29.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003713413 2024-09-14T09:05:29.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660496+0000", 2024-09-14T09:05:29.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4847999999999999e-05 2024-09-14T09:05:29.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660559+0000", 2024-09-14T09:05:29.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3225000000000005e-05 2024-09-14T09:05:29.681 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.681 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.681 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.681 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.682 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.646998+0000", 2024-09-14T09:05:29.682 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.955070248, 2024-09-14T09:05:29.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013692605, 2024-09-14T09:05:29.682 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.682 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.682 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.682 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.682 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.682 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.682 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-09-14T09:05:29.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.682 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.646998+0000", 2024-09-14T09:05:29.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.646998+0000", 2024-09-14T09:05:29.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.646999+0000", 2024-09-14T09:05:29.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2200000000000002e-07 2024-09-14T09:05:29.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647006+0000", 2024-09-14T09:05:29.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9999999999999999e-06 2024-09-14T09:05:29.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647016+0000", 2024-09-14T09:05:29.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5650000000000007e-06 2024-09-14T09:05:29.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647023+0000", 2024-09-14T09:05:29.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7719999999999994e-06 2024-09-14T09:05:29.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.656766+0000", 2024-09-14T09:05:29.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0097425310000000005 2024-09-14T09:05:29.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.656889+0000", 2024-09-14T09:05:29.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012292499999999999 2024-09-14T09:05:29.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.657085+0000", 2024-09-14T09:05:29.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000196078 2024-09-14T09:05:29.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660398+0000", 2024-09-14T09:05:29.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033131020000000001 2024-09-14T09:05:29.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660570+0000", 2024-09-14T09:05:29.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017209199999999999 2024-09-14T09:05:29.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660633+0000", 2024-09-14T09:05:29.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2461999999999999e-05 2024-09-14T09:05:29.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660691+0000", 2024-09-14T09:05:29.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8155999999999997e-05 2024-09-14T09:05:29.687 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.687 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.687 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.687 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.687 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.647044+0000", 2024-09-14T09:05:29.687 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.955024448, 2024-09-14T09:05:29.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015792967000000002, 2024-09-14T09:05:29.687 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.687 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.687 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.687 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.688 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.688 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.688 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 71 2024-09-14T09:05:29.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.688 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647044+0000", 2024-09-14T09:05:29.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647044+0000", 2024-09-14T09:05:29.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647045+0000", 2024-09-14T09:05:29.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2600000000000002e-07 2024-09-14T09:05:29.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647051+0000", 2024-09-14T09:05:29.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3910000000000003e-06 2024-09-14T09:05:29.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647063+0000", 2024-09-14T09:05:29.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1684e-05 2024-09-14T09:05:29.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647070+0000", 2024-09-14T09:05:29.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0169999999999999e-06 2024-09-14T09:05:29.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.657564+0000", 2024-09-14T09:05:29.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010494204 2024-09-14T09:05:29.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.657630+0000", 2024-09-14T09:05:29.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5674999999999997e-05 2024-09-14T09:05:29.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.657729+0000", 2024-09-14T09:05:29.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9036999999999994e-05 2024-09-14T09:05:29.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660706+0000", 2024-09-14T09:05:29.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029772959999999999 2024-09-14T09:05:29.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.662689+0000", 2024-09-14T09:05:29.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00198321 2024-09-14T09:05:29.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.662773+0000", 2024-09-14T09:05:29.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3763000000000004e-05 2024-09-14T09:05:29.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.662837+0000", 2024-09-14T09:05:29.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3963999999999999e-05 2024-09-14T09:05:29.692 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.692 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.693 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.693 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.693 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.647112+0000", 2024-09-14T09:05:29.693 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.954956184, 2024-09-14T09:05:29.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016113565, 2024-09-14T09:05:29.693 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.693 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.693 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.693 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.693 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.693 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.693 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 72 2024-09-14T09:05:29.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.694 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647112+0000", 2024-09-14T09:05:29.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647112+0000", 2024-09-14T09:05:29.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647113+0000", 2024-09-14T09:05:29.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0300000000000001e-06 2024-09-14T09:05:29.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647176+0000", 2024-09-14T09:05:29.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3138000000000004e-05 2024-09-14T09:05:29.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647199+0000", 2024-09-14T09:05:29.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.213e-05 2024-09-14T09:05:29.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647210+0000", 2024-09-14T09:05:29.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1854e-05 2024-09-14T09:05:29.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.658101+0000", 2024-09-14T09:05:29.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010891059999999999 2024-09-14T09:05:29.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.658225+0000", 2024-09-14T09:05:29.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012339100000000001 2024-09-14T09:05:29.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.658465+0000", 2024-09-14T09:05:29.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000240399 2024-09-14T09:05:29.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.662626+0000", 2024-09-14T09:05:29.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041610019999999996 2024-09-14T09:05:29.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.663020+0000", 2024-09-14T09:05:29.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000393736 2024-09-14T09:05:29.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.663111+0000", 2024-09-14T09:05:29.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1218999999999995e-05 2024-09-14T09:05:29.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.663226+0000", 2024-09-14T09:05:29.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011460599999999999 2024-09-14T09:05:29.698 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.698 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.698 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.698 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.698 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.647229+0000", 2024-09-14T09:05:29.698 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.954839562, 2024-09-14T09:05:29.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017463752999999999, 2024-09-14T09:05:29.699 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.699 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.699 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.699 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.699 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.699 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.699 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 73 2024-09-14T09:05:29.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.699 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647229+0000", 2024-09-14T09:05:29.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647229+0000", 2024-09-14T09:05:29.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647230+0000", 2024-09-14T09:05:29.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-06 2024-09-14T09:05:29.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647238+0000", 2024-09-14T09:05:29.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2819999999999996e-06 2024-09-14T09:05:29.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647251+0000", 2024-09-14T09:05:29.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3064999999999999e-05 2024-09-14T09:05:29.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647259+0000", 2024-09-14T09:05:29.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2069999999999998e-06 2024-09-14T09:05:29.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.659745+0000", 2024-09-14T09:05:29.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.01248609 2024-09-14T09:05:29.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.659821+0000", 2024-09-14T09:05:29.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6493000000000001e-05 2024-09-14T09:05:29.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.659919+0000", 2024-09-14T09:05:29.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.781e-05 2024-09-14T09:05:29.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.662855+0000", 2024-09-14T09:05:29.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029360900000000001 2024-09-14T09:05:29.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.664495+0000", 2024-09-14T09:05:29.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016393289999999999 2024-09-14T09:05:29.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.664592+0000", 2024-09-14T09:05:29.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7672999999999998e-05 2024-09-14T09:05:29.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.664693+0000", 2024-09-14T09:05:29.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100423 2024-09-14T09:05:29.705 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.705 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.705 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.706 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.706 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.647274+0000", 2024-09-14T09:05:29.706 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.954794140000001, 2024-09-14T09:05:29.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018286337999999999, 2024-09-14T09:05:29.706 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.706 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.706 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.706 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.706 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.706 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.706 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 74 2024-09-14T09:05:29.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.706 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647274+0000", 2024-09-14T09:05:29.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647274+0000", 2024-09-14T09:05:29.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647275+0000", 2024-09-14T09:05:29.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8299999999999996e-07 2024-09-14T09:05:29.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647285+0000", 2024-09-14T09:05:29.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.533e-06 2024-09-14T09:05:29.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647295+0000", 2024-09-14T09:05:29.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0216e-05 2024-09-14T09:05:29.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647303+0000", 2024-09-14T09:05:29.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5820000000000002e-06 2024-09-14T09:05:29.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660744+0000", 2024-09-14T09:05:29.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013440934999999999 2024-09-14T09:05:29.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660828+0000", 2024-09-14T09:05:29.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3949999999999994e-05 2024-09-14T09:05:29.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660954+0000", 2024-09-14T09:05:29.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012590600000000001 2024-09-14T09:05:29.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.662869+0000", 2024-09-14T09:05:29.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001914415 2024-09-14T09:05:29.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.665383+0000", 2024-09-14T09:05:29.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025140560000000002 2024-09-14T09:05:29.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.665478+0000", 2024-09-14T09:05:29.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5642000000000003e-05 2024-09-14T09:05:29.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.665561+0000", 2024-09-14T09:05:29.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2319999999999998e-05 2024-09-14T09:05:29.712 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.712 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.712 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.712 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.712 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.647322+0000", 2024-09-14T09:05:29.712 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.954746266000001, 2024-09-14T09:05:29.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021121719000000001, 2024-09-14T09:05:29.712 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.712 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.712 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.713 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.713 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.713 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.713 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 75 2024-09-14T09:05:29.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.713 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647322+0000", 2024-09-14T09:05:29.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647322+0000", 2024-09-14T09:05:29.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647323+0000", 2024-09-14T09:05:29.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.04e-06 2024-09-14T09:05:29.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647407+0000", 2024-09-14T09:05:29.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3652999999999993e-05 2024-09-14T09:05:29.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647416+0000", 2024-09-14T09:05:29.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2310000000000002e-06 2024-09-14T09:05:29.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647424+0000", 2024-09-14T09:05:29.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3240000000000003e-06 2024-09-14T09:05:29.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.661338+0000", 2024-09-14T09:05:29.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013913114000000001 2024-09-14T09:05:29.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.661461+0000", 2024-09-14T09:05:29.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012374200000000001 2024-09-14T09:05:29.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.661653+0000", 2024-09-14T09:05:29.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000191872 2024-09-14T09:05:29.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.664371+0000", 2024-09-14T09:05:29.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027177600000000001 2024-09-14T09:05:29.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668357+0000", 2024-09-14T09:05:29.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039863229999999999 2024-09-14T09:05:29.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668403+0000", 2024-09-14T09:05:29.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6276000000000002e-05 2024-09-14T09:05:29.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668444+0000", 2024-09-14T09:05:29.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0383999999999998e-05 2024-09-14T09:05:29.717 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.717 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.717 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.717 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:29.718 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.647440+0000", 2024-09-14T09:05:29.718 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.954627949000001, 2024-09-14T09:05:29.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021273368000000001, 2024-09-14T09:05:29.718 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:29.718 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:29.718 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:29.718 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:29.718 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:29.718 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:29.718 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 76 2024-09-14T09:05:29.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.718 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:29.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:29.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647440+0000", 2024-09-14T09:05:29.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:29.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647440+0000", 2024-09-14T09:05:29.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:29.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:29.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647441+0000", 2024-09-14T09:05:29.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1500000000000003e-07 2024-09-14T09:05:29.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:29.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647448+0000", 2024-09-14T09:05:29.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4420000000000003e-06 2024-09-14T09:05:29.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:29.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647457+0000", 2024-09-14T09:05:29.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:05:29.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:29.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647465+0000", 2024-09-14T09:05:29.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2250000000000003e-06 2024-09-14T09:05:29.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:29.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.663329+0000", 2024-09-14T09:05:29.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015863931000000001 2024-09-14T09:05:29.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:29.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.663474+0000", 2024-09-14T09:05:29.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014509200000000001 2024-09-14T09:05:29.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:29.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.663695+0000", 2024-09-14T09:05:29.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022155499999999999 2024-09-14T09:05:29.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:29.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.664823+0000", 2024-09-14T09:05:29.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001128046 2024-09-14T09:05:29.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:29.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668550+0000", 2024-09-14T09:05:29.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003726493 2024-09-14T09:05:29.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:29.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668638+0000", 2024-09-14T09:05:29.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8282999999999995e-05 2024-09-14T09:05:29.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:29.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:29.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:29.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668714+0000", 2024-09-14T09:05:29.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5856e-05 2024-09-14T09:05:29.723 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.723 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.723 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.723 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:29.723 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:29.723 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:05:29.725 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:05:29.725 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:29.728+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:29.826 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:29.820+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:29.843 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:29.843 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:29.843 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:29.843 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:29.856 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:05:29.927 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:29.928+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:30.022 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:30.023 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:30.023 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:30.023 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:30.023 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:30.027 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:30.024+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:30.036 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:05:30.128 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:30.128+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:30.194 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:30.194 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:30.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:30.194 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:30.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.195 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-14T09:05:30.195 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.274102+0000", 2024-09-14T09:05:30.195 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.918683088000002, 2024-09-14T09:05:30.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003545763, 2024-09-14T09:05:30.195 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.195 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.195 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.195 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274102+0000", 2024-09-14T09:05:30.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274102+0000", 2024-09-14T09:05:30.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274103+0000", 2024-09-14T09:05:30.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4670000000000001e-06 2024-09-14T09:05:30.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274328+0000", 2024-09-14T09:05:30.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022502500000000001 2024-09-14T09:05:30.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274354+0000", 2024-09-14T09:05:30.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5993e-05 2024-09-14T09:05:30.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274372+0000", 2024-09-14T09:05:30.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8117e-05 2024-09-14T09:05:30.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274455+0000", 2024-09-14T09:05:30.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2785000000000001e-05 2024-09-14T09:05:30.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274486+0000", 2024-09-14T09:05:30.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0864000000000001e-05 2024-09-14T09:05:30.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277538+0000", 2024-09-14T09:05:30.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0030521860000000001 2024-09-14T09:05:30.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277647+0000", 2024-09-14T09:05:30.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109326 2024-09-14T09:05:30.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.199 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.199 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-14T09:05:30.200 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.275860+0000", 2024-09-14T09:05:30.200 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.916924913999999, 2024-09-14T09:05:30.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003513562, 2024-09-14T09:05:30.200 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.200 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.200 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.200 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.275860+0000", 2024-09-14T09:05:30.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.275860+0000", 2024-09-14T09:05:30.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.275861+0000", 2024-09-14T09:05:30.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.513e-06 2024-09-14T09:05:30.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276039+0000", 2024-09-14T09:05:30.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017758199999999999 2024-09-14T09:05:30.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276060+0000", 2024-09-14T09:05:30.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0956000000000001e-05 2024-09-14T09:05:30.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276076+0000", 2024-09-14T09:05:30.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6137000000000002e-05 2024-09-14T09:05:30.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276190+0000", 2024-09-14T09:05:30.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011359200000000001 2024-09-14T09:05:30.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276226+0000", 2024-09-14T09:05:30.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6300000000000001e-05 2024-09-14T09:05:30.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.279274+0000", 2024-09-14T09:05:30.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00304813 2024-09-14T09:05:30.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.279373+0000", 2024-09-14T09:05:30.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9351999999999993e-05 2024-09-14T09:05:30.204 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.204 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.204 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.205 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:22 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'12, mlcod=17'8)", 2024-09-14T09:05:30.205 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.277665+0000", 2024-09-14T09:05:30.205 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.915119368999999, 2024-09-14T09:05:30.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0031553430000000001, 2024-09-14T09:05:30.205 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.205 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.205 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.205 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277665+0000", 2024-09-14T09:05:30.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277665+0000", 2024-09-14T09:05:30.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277667+0000", 2024-09-14T09:05:30.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.207e-06 2024-09-14T09:05:30.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277847+0000", 2024-09-14T09:05:30.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018088199999999999 2024-09-14T09:05:30.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277868+0000", 2024-09-14T09:05:30.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0582999999999999e-05 2024-09-14T09:05:30.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277883+0000", 2024-09-14T09:05:30.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4934e-05 2024-09-14T09:05:30.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277978+0000", 2024-09-14T09:05:30.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5031999999999994e-05 2024-09-14T09:05:30.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.278030+0000", 2024-09-14T09:05:30.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1929999999999999e-05 2024-09-14T09:05:30.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.280706+0000", 2024-09-14T09:05:30.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002676042 2024-09-14T09:05:30.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.280821+0000", 2024-09-14T09:05:30.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114733 2024-09-14T09:05:30.210 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.210 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.210 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.210 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:23 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'13, mlcod=17'8)", 2024-09-14T09:05:30.211 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.279040+0000", 2024-09-14T09:05:30.211 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.913744952000002, 2024-09-14T09:05:30.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002363371, 2024-09-14T09:05:30.211 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.211 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.211 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.211 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.279040+0000", 2024-09-14T09:05:30.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.279040+0000", 2024-09-14T09:05:30.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.279041+0000", 2024-09-14T09:05:30.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4130000000000001e-06 2024-09-14T09:05:30.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.279233+0000", 2024-09-14T09:05:30.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019182800000000001 2024-09-14T09:05:30.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.279253+0000", 2024-09-14T09:05:30.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.003e-05 2024-09-14T09:05:30.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.279267+0000", 2024-09-14T09:05:30.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4443e-05 2024-09-14T09:05:30.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.279426+0000", 2024-09-14T09:05:30.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000158209 2024-09-14T09:05:30.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.279458+0000", 2024-09-14T09:05:30.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2682999999999999e-05 2024-09-14T09:05:30.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.281344+0000", 2024-09-14T09:05:30.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018852960000000001 2024-09-14T09:05:30.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.281403+0000", 2024-09-14T09:05:30.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9469000000000003e-05 2024-09-14T09:05:30.214 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.214 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.215 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.215 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:24 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'14, mlcod=17'10)", 2024-09-14T09:05:30.215 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.281383+0000", 2024-09-14T09:05:30.215 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.911401684000001, 2024-09-14T09:05:30.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0024238850000000002, 2024-09-14T09:05:30.215 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.215 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.215 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.215 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.281383+0000", 2024-09-14T09:05:30.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.281383+0000", 2024-09-14T09:05:30.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.281384+0000", 2024-09-14T09:05:30.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3230000000000001e-06 2024-09-14T09:05:30.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.281485+0000", 2024-09-14T09:05:30.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100466 2024-09-14T09:05:30.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.281505+0000", 2024-09-14T09:05:30.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0627000000000001e-05 2024-09-14T09:05:30.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.281522+0000", 2024-09-14T09:05:30.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6313000000000001e-05 2024-09-14T09:05:30.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.281605+0000", 2024-09-14T09:05:30.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3269000000000005e-05 2024-09-14T09:05:30.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.281635+0000", 2024-09-14T09:05:30.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9640000000000001e-05 2024-09-14T09:05:30.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.283691+0000", 2024-09-14T09:05:30.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0020568510000000002 2024-09-14T09:05:30.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.283807+0000", 2024-09-14T09:05:30.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115396 2024-09-14T09:05:30.219 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.219 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.219 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.220 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:34 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'20, mlcod=17'18)", 2024-09-14T09:05:30.220 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.369468+0000", 2024-09-14T09:05:30.220 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.823316640000002, 2024-09-14T09:05:30.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001973565, 2024-09-14T09:05:30.220 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.220 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.220 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.220 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.369468+0000", 2024-09-14T09:05:30.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.369468+0000", 2024-09-14T09:05:30.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.369475+0000", 2024-09-14T09:05:30.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9129999999999997e-06 2024-09-14T09:05:30.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.369502+0000", 2024-09-14T09:05:30.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6633000000000001e-05 2024-09-14T09:05:30.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.369542+0000", 2024-09-14T09:05:30.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0806000000000002e-05 2024-09-14T09:05:30.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.369570+0000", 2024-09-14T09:05:30.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7123000000000001e-05 2024-09-14T09:05:30.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.369796+0000", 2024-09-14T09:05:30.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000226462 2024-09-14T09:05:30.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.369865+0000", 2024-09-14T09:05:30.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8730000000000001e-05 2024-09-14T09:05:30.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.371388+0000", 2024-09-14T09:05:30.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0015236690000000001 2024-09-14T09:05:30.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.371442+0000", 2024-09-14T09:05:30.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3229000000000001e-05 2024-09-14T09:05:30.224 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.224 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.224 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.224 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'28)", 2024-09-14T09:05:30.224 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.397128+0000", 2024-09-14T09:05:30.224 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.795656375, 2024-09-14T09:05:30.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0022071930000000001, 2024-09-14T09:05:30.224 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.224 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.224 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.224 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.397128+0000", 2024-09-14T09:05:30.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.397128+0000", 2024-09-14T09:05:30.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.397129+0000", 2024-09-14T09:05:30.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T09:05:30.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.397133+0000", 2024-09-14T09:05:30.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0489999999999997e-06 2024-09-14T09:05:30.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.397145+0000", 2024-09-14T09:05:30.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1503e-05 2024-09-14T09:05:30.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.397153+0000", 2024-09-14T09:05:30.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7870000000000007e-06 2024-09-14T09:05:30.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.397197+0000", 2024-09-14T09:05:30.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3868000000000001e-05 2024-09-14T09:05:30.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.397212+0000", 2024-09-14T09:05:30.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5187e-05 2024-09-14T09:05:30.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.399288+0000", 2024-09-14T09:05:30.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0020752639999999998 2024-09-14T09:05:30.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.399336+0000", 2024-09-14T09:05:30.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7868000000000003e-05 2024-09-14T09:05:30.228 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.228 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.228 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.229 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'42)", 2024-09-14T09:05:30.229 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.651370+0000", 2024-09-14T09:05:30.229 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.541414753000002, 2024-09-14T09:05:30.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002005855, 2024-09-14T09:05:30.229 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.229 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.229 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.230 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.651370+0000", 2024-09-14T09:05:30.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.651370+0000", 2024-09-14T09:05:30.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.651371+0000", 2024-09-14T09:05:30.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0130000000000001e-06 2024-09-14T09:05:30.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.651380+0000", 2024-09-14T09:05:30.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1800000000000002e-06 2024-09-14T09:05:30.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.651402+0000", 2024-09-14T09:05:30.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1914000000000001e-05 2024-09-14T09:05:30.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.651419+0000", 2024-09-14T09:05:30.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.662e-05 2024-09-14T09:05:30.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.651474+0000", 2024-09-14T09:05:30.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5757999999999997e-05 2024-09-14T09:05:30.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.651491+0000", 2024-09-14T09:05:30.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6518999999999999e-05 2024-09-14T09:05:30.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.653342+0000", 2024-09-14T09:05:30.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001850745 2024-09-14T09:05:30.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.653376+0000", 2024-09-14T09:05:30.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4106000000000002e-05 2024-09-14T09:05:30.233 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.233 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.233 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.233 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:71 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'49, mlcod=17'45)", 2024-09-14T09:05:30.234 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.656686+0000", 2024-09-14T09:05:30.234 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.536098946999999, 2024-09-14T09:05:30.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0029447330000000002, 2024-09-14T09:05:30.234 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.234 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.234 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.234 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.656686+0000", 2024-09-14T09:05:30.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.656686+0000", 2024-09-14T09:05:30.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.656688+0000", 2024-09-14T09:05:30.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9e-06 2024-09-14T09:05:30.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.656702+0000", 2024-09-14T09:05:30.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4073000000000001e-05 2024-09-14T09:05:30.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.656730+0000", 2024-09-14T09:05:30.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8543000000000001e-05 2024-09-14T09:05:30.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.656743+0000", 2024-09-14T09:05:30.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.275e-05 2024-09-14T09:05:30.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.656916+0000", 2024-09-14T09:05:30.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017337799999999999 2024-09-14T09:05:30.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.656952+0000", 2024-09-14T09:05:30.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5689999999999999e-05 2024-09-14T09:05:30.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659554+0000", 2024-09-14T09:05:30.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002601557 2024-09-14T09:05:30.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659630+0000", 2024-09-14T09:05:30.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6841999999999994e-05 2024-09-14T09:05:30.238 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.238 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.238 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.238 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:72 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'50, mlcod=17'45)", 2024-09-14T09:05:30.238 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.657508+0000", 2024-09-14T09:05:30.238 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.535276601, 2024-09-14T09:05:30.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0033585239999999999, 2024-09-14T09:05:30.238 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.238 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.239 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.239 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657508+0000", 2024-09-14T09:05:30.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657508+0000", 2024-09-14T09:05:30.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657510+0000", 2024-09-14T09:05:30.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8899999999999999e-06 2024-09-14T09:05:30.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657625+0000", 2024-09-14T09:05:30.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115216 2024-09-14T09:05:30.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657649+0000", 2024-09-14T09:05:30.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4043e-05 2024-09-14T09:05:30.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657669+0000", 2024-09-14T09:05:30.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9579999999999999e-05 2024-09-14T09:05:30.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657790+0000", 2024-09-14T09:05:30.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012084200000000001 2024-09-14T09:05:30.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657828+0000", 2024-09-14T09:05:30.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8019999999999999e-05 2024-09-14T09:05:30.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660805+0000", 2024-09-14T09:05:30.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002976874 2024-09-14T09:05:30.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660867+0000", 2024-09-14T09:05:30.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2058999999999997e-05 2024-09-14T09:05:30.243 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.243 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.243 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.243 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:73 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'51, mlcod=17'45)", 2024-09-14T09:05:30.243 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.658866+0000", 2024-09-14T09:05:30.243 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.533919164, 2024-09-14T09:05:30.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035579930000000002, 2024-09-14T09:05:30.243 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.243 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.244 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.244 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658866+0000", 2024-09-14T09:05:30.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658866+0000", 2024-09-14T09:05:30.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658867+0000", 2024-09-14T09:05:30.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.886e-06 2024-09-14T09:05:30.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658880+0000", 2024-09-14T09:05:30.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2204e-05 2024-09-14T09:05:30.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658908+0000", 2024-09-14T09:05:30.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8555999999999999e-05 2024-09-14T09:05:30.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658921+0000", 2024-09-14T09:05:30.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2566e-05 2024-09-14T09:05:30.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659682+0000", 2024-09-14T09:05:30.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000760865 2024-09-14T09:05:30.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659710+0000", 2024-09-14T09:05:30.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8612e-05 2024-09-14T09:05:30.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662349+0000", 2024-09-14T09:05:30.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0026388370000000002 2024-09-14T09:05:30.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662424+0000", 2024-09-14T09:05:30.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4467000000000004e-05 2024-09-14T09:05:30.247 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.247 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.247 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.247 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:74 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'52, mlcod=17'47)", 2024-09-14T09:05:30.248 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.659878+0000", 2024-09-14T09:05:30.248 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.532906596, 2024-09-14T09:05:30.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0040591689999999996, 2024-09-14T09:05:30.248 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.248 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.248 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.248 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659878+0000", 2024-09-14T09:05:30.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659878+0000", 2024-09-14T09:05:30.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659892+0000", 2024-09-14T09:05:30.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3487e-05 2024-09-14T09:05:30.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659902+0000", 2024-09-14T09:05:30.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.062e-05 2024-09-14T09:05:30.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659928+0000", 2024-09-14T09:05:30.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6060000000000001e-05 2024-09-14T09:05:30.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659941+0000", 2024-09-14T09:05:30.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2649999999999999e-05 2024-09-14T09:05:30.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660304+0000", 2024-09-14T09:05:30.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036354699999999998 2024-09-14T09:05:30.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660349+0000", 2024-09-14T09:05:30.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4938999999999998e-05 2024-09-14T09:05:30.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.663847+0000", 2024-09-14T09:05:30.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0034973339999999999 2024-09-14T09:05:30.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.663937+0000", 2024-09-14T09:05:30.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0532000000000007e-05 2024-09-14T09:05:30.252 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.252 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.252 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.252 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'47)", 2024-09-14T09:05:30.252 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.660695+0000", 2024-09-14T09:05:30.252 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.532089445, 2024-09-14T09:05:30.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0062206520000000001, 2024-09-14T09:05:30.253 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.253 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.253 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.253 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660695+0000", 2024-09-14T09:05:30.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660695+0000", 2024-09-14T09:05:30.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660697+0000", 2024-09-14T09:05:30.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.897e-06 2024-09-14T09:05:30.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660816+0000", 2024-09-14T09:05:30.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118442 2024-09-14T09:05:30.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660839+0000", 2024-09-14T09:05:30.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3790000000000001e-05 2024-09-14T09:05:30.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660852+0000", 2024-09-14T09:05:30.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-05 2024-09-14T09:05:30.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660967+0000", 2024-09-14T09:05:30.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115276 2024-09-14T09:05:30.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.661016+0000", 2024-09-14T09:05:30.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8785999999999997e-05 2024-09-14T09:05:30.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.666825+0000", 2024-09-14T09:05:30.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0058087549999999996 2024-09-14T09:05:30.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.666916+0000", 2024-09-14T09:05:30.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1325999999999999e-05 2024-09-14T09:05:30.257 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.257 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.257 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.257 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'49)", 2024-09-14T09:05:30.257 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.662683+0000", 2024-09-14T09:05:30.257 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.530102133, 2024-09-14T09:05:30.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0042817899999999997, 2024-09-14T09:05:30.257 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.257 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.257 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.257 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662683+0000", 2024-09-14T09:05:30.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662683+0000", 2024-09-14T09:05:30.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662684+0000", 2024-09-14T09:05:30.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6559999999999999e-06 2024-09-14T09:05:30.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662696+0000", 2024-09-14T09:05:30.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1454000000000001e-05 2024-09-14T09:05:30.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662724+0000", 2024-09-14T09:05:30.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8535999999999999e-05 2024-09-14T09:05:30.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662752+0000", 2024-09-14T09:05:30.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7503000000000001e-05 2024-09-14T09:05:30.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662867+0000", 2024-09-14T09:05:30.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011500199999999999 2024-09-14T09:05:30.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662899+0000", 2024-09-14T09:05:30.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2122999999999997e-05 2024-09-14T09:05:30.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.666919+0000", 2024-09-14T09:05:30.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004020633 2024-09-14T09:05:30.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.666964+0000", 2024-09-14T09:05:30.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4882999999999997e-05 2024-09-14T09:05:30.262 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.262 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.262 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.262 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:77 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'55, mlcod=17'53)", 2024-09-14T09:05:30.262 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.668436+0000", 2024-09-14T09:05:30.262 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.524348371999999, 2024-09-14T09:05:30.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035848659999999999, 2024-09-14T09:05:30.262 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.262 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.263 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.263 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668436+0000", 2024-09-14T09:05:30.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668436+0000", 2024-09-14T09:05:30.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668438+0000", 2024-09-14T09:05:30.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.81e-06 2024-09-14T09:05:30.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668551+0000", 2024-09-14T09:05:30.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112726 2024-09-14T09:05:30.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668580+0000", 2024-09-14T09:05:30.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8702999999999999e-05 2024-09-14T09:05:30.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668592+0000", 2024-09-14T09:05:30.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.276e-05 2024-09-14T09:05:30.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668767+0000", 2024-09-14T09:05:30.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017485600000000001 2024-09-14T09:05:30.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668808+0000", 2024-09-14T09:05:30.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1189000000000002e-05 2024-09-14T09:05:30.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.671940+0000", 2024-09-14T09:05:30.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0031317329999999998 2024-09-14T09:05:30.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672021+0000", 2024-09-14T09:05:30.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1088999999999996e-05 2024-09-14T09:05:30.267 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.267 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.267 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.268 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:78 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'56, mlcod=17'53)", 2024-09-14T09:05:30.268 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.670356+0000", 2024-09-14T09:05:30.268 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.522428587, 2024-09-14T09:05:30.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0036555289999999998, 2024-09-14T09:05:30.268 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.268 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.268 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.268 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670356+0000", 2024-09-14T09:05:30.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670356+0000", 2024-09-14T09:05:30.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670358+0000", 2024-09-14T09:05:30.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.79e-06 2024-09-14T09:05:30.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670469+0000", 2024-09-14T09:05:30.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111273 2024-09-14T09:05:30.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670495+0000", 2024-09-14T09:05:30.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5516000000000001e-05 2024-09-14T09:05:30.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670508+0000", 2024-09-14T09:05:30.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2986999999999999e-05 2024-09-14T09:05:30.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670679+0000", 2024-09-14T09:05:30.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017094199999999999 2024-09-14T09:05:30.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670717+0000", 2024-09-14T09:05:30.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8322999999999999e-05 2024-09-14T09:05:30.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.673942+0000", 2024-09-14T09:05:30.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032255489999999999 2024-09-14T09:05:30.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.674012+0000", 2024-09-14T09:05:30.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9148999999999998e-05 2024-09-14T09:05:30.273 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.273 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.273 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.273 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:79 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'57, mlcod=17'53)", 2024-09-14T09:05:30.273 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.672051+0000", 2024-09-14T09:05:30.273 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.520733885999999, 2024-09-14T09:05:30.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0033465410000000002, 2024-09-14T09:05:30.273 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.273 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.274 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.274 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672051+0000", 2024-09-14T09:05:30.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672051+0000", 2024-09-14T09:05:30.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672053+0000", 2024-09-14T09:05:30.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0159999999999998e-06 2024-09-14T09:05:30.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672128+0000", 2024-09-14T09:05:30.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4909999999999999e-05 2024-09-14T09:05:30.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672162+0000", 2024-09-14T09:05:30.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4725999999999999e-05 2024-09-14T09:05:30.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672181+0000", 2024-09-14T09:05:30.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8357000000000001e-05 2024-09-14T09:05:30.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672311+0000", 2024-09-14T09:05:30.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012992200000000001 2024-09-14T09:05:30.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672342+0000", 2024-09-14T09:05:30.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1473000000000002e-05 2024-09-14T09:05:30.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.675298+0000", 2024-09-14T09:05:30.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0029560910000000001 2024-09-14T09:05:30.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.675397+0000", 2024-09-14T09:05:30.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9046e-05 2024-09-14T09:05:30.277 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.277 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.277 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.278 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:80 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'58, mlcod=17'53)", 2024-09-14T09:05:30.278 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.673619+0000", 2024-09-14T09:05:30.278 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.519165701999999, 2024-09-14T09:05:30.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0028893719999999999, 2024-09-14T09:05:30.278 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.278 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.278 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.278 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.673619+0000", 2024-09-14T09:05:30.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.673619+0000", 2024-09-14T09:05:30.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.673621+0000", 2024-09-14T09:05:30.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5999999999999999e-06 2024-09-14T09:05:30.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.673679+0000", 2024-09-14T09:05:30.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8866999999999997e-05 2024-09-14T09:05:30.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.673712+0000", 2024-09-14T09:05:30.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2215999999999998e-05 2024-09-14T09:05:30.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.673755+0000", 2024-09-14T09:05:30.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3556000000000001e-05 2024-09-14T09:05:30.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.674068+0000", 2024-09-14T09:05:30.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031252199999999997 2024-09-14T09:05:30.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.674099+0000", 2024-09-14T09:05:30.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.154e-05 2024-09-14T09:05:30.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.676419+0000", 2024-09-14T09:05:30.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0023200880000000001 2024-09-14T09:05:30.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.676508+0000", 2024-09-14T09:05:30.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8982999999999998e-05 2024-09-14T09:05:30.282 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.282 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.283 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.283 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-14T09:05:30.283 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:14.056046+0000", 2024-09-14T09:05:30.283 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.136738656999999, 2024-09-14T09:05:30.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002064715, 2024-09-14T09:05:30.283 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.283 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.283 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.283 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:14.056046+0000", 2024-09-14T09:05:30.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:14.056046+0000", 2024-09-14T09:05:30.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:14.056047+0000", 2024-09-14T09:05:30.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5700000000000001e-07 2024-09-14T09:05:30.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:14.056054+0000", 2024-09-14T09:05:30.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.72e-06 2024-09-14T09:05:30.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:14.056073+0000", 2024-09-14T09:05:30.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9148999999999999e-05 2024-09-14T09:05:30.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:14.056087+0000", 2024-09-14T09:05:30.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4327e-05 2024-09-14T09:05:30.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:14.056167+0000", 2024-09-14T09:05:30.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0307000000000002e-05 2024-09-14T09:05:30.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:14.056184+0000", 2024-09-14T09:05:30.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6444999999999999e-05 2024-09-14T09:05:30.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:14.058072+0000", 2024-09-14T09:05:30.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001888425 2024-09-14T09:05:30.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:14.058111+0000", 2024-09-14T09:05:30.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8485000000000002e-05 2024-09-14T09:05:30.291 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.291 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.292 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.292 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T09:05:30.292 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:28.835461+0000", 2024-09-14T09:05:30.293 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.357323507, 2024-09-14T09:05:30.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050182409999999997, 2024-09-14T09:05:30.293 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:30.293 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:30.293 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:30.293 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:30.294 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:30.294 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:30.294 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 11 2024-09-14T09:05:30.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.294 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:30.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:30.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835461+0000", 2024-09-14T09:05:30.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:30.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835461+0000", 2024-09-14T09:05:30.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:30.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:30.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835463+0000", 2024-09-14T09:05:30.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9570000000000001e-06 2024-09-14T09:05:30.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:30.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835474+0000", 2024-09-14T09:05:30.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0949e-05 2024-09-14T09:05:30.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:30.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835495+0000", 2024-09-14T09:05:30.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1325e-05 2024-09-14T09:05:30.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:30.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835506+0000", 2024-09-14T09:05:30.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0854e-05 2024-09-14T09:05:30.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835579+0000", 2024-09-14T09:05:30.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2743000000000005e-05 2024-09-14T09:05:30.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T09:05:30.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835585+0000", 2024-09-14T09:05:30.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9519999999999999e-06 2024-09-14T09:05:30.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:30.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.838428+0000", 2024-09-14T09:05:30.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0028425830000000001 2024-09-14T09:05:30.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:30.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.838557+0000", 2024-09-14T09:05:30.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012985000000000001 2024-09-14T09:05:30.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:05:30.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.838743+0000", 2024-09-14T09:05:30.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018533199999999999 2024-09-14T09:05:30.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:30.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.839964+0000", 2024-09-14T09:05:30.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001221387 2024-09-14T09:05:30.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:30.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.840333+0000", 2024-09-14T09:05:30.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036902899999999999 2024-09-14T09:05:30.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:30.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.840410+0000", 2024-09-14T09:05:30.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6860999999999995e-05 2024-09-14T09:05:30.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:30.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:30.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:30.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.840479+0000", 2024-09-14T09:05:30.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9418999999999994e-05 2024-09-14T09:05:30.306 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.306 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.306 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.306 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:30.306 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:30.306 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:30.308 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:05:30.308 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:30.308+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:30.387 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:05:30.424 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:30.424 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:30.424 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:30.424 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:30.425 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:30.420+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:30.437 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:05:30.509 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:30.512+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:30.533 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:05:30.533 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:05:30.552 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:05:30.552 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:05:30.570 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:05:30.570 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:05:30.588 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:05:30.588 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:05:30.599 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:30.599 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:30.609 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:05:30.610 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:05:30.610 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:30.612+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:30.614 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:05:30.631 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:05:30.631 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:05:30.653 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:05:30.653 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:05:30.674 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:05:30.674 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:05:30.730 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:30.730 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:30.730 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:30.728+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:30.743 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:05:30.775 INFO:teuthology.orchestra.run.smithi176.stdout:1 writesame 0x3a00ac7 thru 0x3a0938f data_size 0x13(0x88c9 bytes) 2024-09-14T09:05:30.776 INFO:teuthology.orchestra.run.smithi176.stdout:2 read 0x24d9fb0 thru 0x24da7d2 (0x823 bytes) 2024-09-14T09:05:30.776 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:30.801 INFO:teuthology.orchestra.run.smithi176.stdout:4 write 0xd5a6832 thru 0xd5ae992 (0x8161 bytes) 2024-09-14T09:05:30.811 INFO:teuthology.orchestra.run.smithi176.stdout:6 read 0xacecaf0 thru 0xacf7b59 (0xb06a bytes) 2024-09-14T09:05:30.811 INFO:teuthology.orchestra.run.smithi176.stdout:8 write 0x99066dd thru 0x9908f73 (0x2897 bytes) 2024-09-14T09:05:30.811 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:30.812+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:30.818 INFO:teuthology.orchestra.run.smithi176.stdout:9 punch from 0x284532 to 0x289cd8, (0x57a6 bytes) 2024-09-14T09:05:30.818 INFO:teuthology.orchestra.run.smithi176.stdout:11 punch from 0x9084e91 to 0x9088ada, (0x3c49 bytes) 2024-09-14T09:05:30.818 INFO:teuthology.orchestra.run.smithi176.stdout:12 write 0x4b7e9ea thru 0x4b839a4 (0x4fbb bytes) 2024-09-14T09:05:30.828 INFO:teuthology.orchestra.run.smithi176.stdout:14 punch from 0x4d64c17 to 0x4d68ff7, (0x43e0 bytes) 2024-09-14T09:05:30.843 INFO:teuthology.orchestra.run.smithi176.stdout:17 write 0xe34a4bd thru 0xe3515c4 (0x7108 bytes) 2024-09-14T09:05:30.848 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T09:05:30.850 INFO:teuthology.orchestra.run.smithi176.stdout:truncating to largest ever: 0xdeac69b 2024-09-14T09:05:30.850 INFO:teuthology.orchestra.run.smithi176.stdout:18 trunc from 0xe3515c5 to 0xdeac69b 2024-09-14T09:05:30.874 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:30.874 INFO:teuthology.orchestra.run.smithi176.stdout:21 read 0xdba8119 thru 0xdbae19b (0x6083 bytes) 2024-09-14T09:05:30.874 INFO:teuthology.orchestra.run.smithi176.stdout:22 writesame 0x8d55d37 thru 0x8d5d2c6 data_size 0x98(0x7590 bytes) 2024-09-14T09:05:30.882 INFO:teuthology.orchestra.run.smithi176.stdout:23 punch from 0x5e1119a to 0x5e17706, (0x656c bytes) 2024-09-14T09:05:30.882 INFO:teuthology.orchestra.run.smithi176.stdout:25 read 0x6a38e85 thru 0x6a447d4 (0xb950 bytes) 2024-09-14T09:05:30.882 INFO:teuthology.orchestra.run.smithi176.stdout:27 read 0xd828d8f thru 0xd82dadf (0x4d51 bytes) 2024-09-14T09:05:30.882 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:30.882 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:30.882 INFO:teuthology.orchestra.run.smithi176.stdout:32 write 0x3deb1fc thru 0x3ded178 (0x1f7d bytes) 2024-09-14T09:05:30.889 INFO:teuthology.orchestra.run.smithi176.stdout:33 punch from 0xea58ef to 0xea80dd, (0x27ee bytes) 2024-09-14T09:05:30.889 INFO:teuthology.orchestra.run.smithi176.stdout:34 read 0x17a4af9 thru 0x17b0e67 (0xc36f bytes) 2024-09-14T09:05:30.889 INFO:teuthology.orchestra.run.smithi176.stdout:35 write 0xac9302d thru 0xac9adb5 (0x7d89 bytes) 2024-09-14T09:05:30.896 INFO:teuthology.orchestra.run.smithi176.stdout:36 read 0x95ce10d thru 0x95cee27 (0xd1b bytes) 2024-09-14T09:05:30.897 INFO:teuthology.orchestra.run.smithi176.stdout:37 writesame 0x5e74523 thru 0x5e79a62 data_size 0x58(0x5540 bytes) 2024-09-14T09:05:30.903 INFO:teuthology.orchestra.run.smithi176.stdout:38 read 0x10c9ee2 thru 0x10ca6ba (0x7d9 bytes) 2024-09-14T09:05:30.903 INFO:teuthology.orchestra.run.smithi176.stdout:40 write 0x1f5004d thru 0x1f53728 (0x36dc bytes) 2024-09-14T09:05:30.906 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:30.906 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:30.911 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:30.912 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:30.912+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:30.921 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:05:30.925 INFO:teuthology.orchestra.run.smithi176.stdout:43 write 0xe4f6d40 thru 0xe5014f6 (0xa7b7 bytes) 2024-09-14T09:05:30.933 INFO:teuthology.orchestra.run.smithi176.stdout:44 write 0x5a66cba thru 0x5a72c1a (0xbf61 bytes) 2024-09-14T09:05:30.945 INFO:teuthology.orchestra.run.smithi176.stdout:45 read 0x6f169db thru 0x6f19871 (0x2e97 bytes) 2024-09-14T09:05:30.946 INFO:teuthology.orchestra.run.smithi176.stdout:46 read 0x58d17d8 thru 0x58d7efb (0x6724 bytes) 2024-09-14T09:05:30.946 INFO:teuthology.orchestra.run.smithi176.stdout:48 punch from 0xb2f88 to 0xb9f07, (0x6f7f bytes) 2024-09-14T09:05:30.946 INFO:teuthology.orchestra.run.smithi176.stdout:49 read 0xb19dfec thru 0xb1ad8b2 (0xf8c7 bytes) 2024-09-14T09:05:30.947 INFO:teuthology.orchestra.run.smithi176.stdout:50 read 0xc09afc7 thru 0xc0a78cb (0xc905 bytes) 2024-09-14T09:05:30.947 INFO:teuthology.orchestra.run.smithi176.stdout:52 read 0xbf1f9bc thru 0xbf273ed (0x7a32 bytes) 2024-09-14T09:05:30.952 INFO:teuthology.orchestra.run.smithi176.stdout:53 trunc from 0xe5014f7 to 0xa5da5f3 2024-09-14T09:05:30.976 INFO:teuthology.orchestra.run.smithi176.stdout:54 read 0x527910c thru 0x527f0d4 (0x5fc9 bytes) 2024-09-14T09:05:30.976 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:30.976 INFO:teuthology.orchestra.run.smithi176.stdout:57 read 0x4d9ba90 thru 0x4da197a (0x5eeb bytes) 2024-09-14T09:05:30.984 INFO:teuthology.orchestra.run.smithi176.stdout:59 trunc from 0xa5da5f3 to 0x398fb4f 2024-09-14T09:05:31.013 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:31.008+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:31.031 INFO:teuthology.orchestra.run.smithi176.stdout:62 writesame 0xb14868e thru 0xb15687b data_size 0x79(0xe1ee bytes) 2024-09-14T09:05:31.031 INFO:teuthology.orchestra.run.smithi176.stdout:63 punch from 0x1cdd8cb to 0x1cdf835, (0x1f6a bytes) 2024-09-14T09:05:31.038 INFO:teuthology.orchestra.run.smithi176.stdout:64 trunc from 0xb15687c to 0x55bb240 2024-09-14T09:05:31.053 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:31.053 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:31.069 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:05:31.069 INFO:teuthology.orchestra.run.smithi176.stdout:68 write 0x9292029 thru 0x92987b7 (0x678f bytes) 2024-09-14T09:05:31.077 INFO:teuthology.orchestra.run.smithi176.stdout:69 punch from 0x2e3a825 to 0x2e3e544, (0x3d1f bytes) 2024-09-14T09:05:31.077 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:31.091 INFO:teuthology.orchestra.run.smithi176.stdout:72 writesame 0x9c94b06 thru 0x9c9a686 data_size 0x5(0x5b81 bytes) 2024-09-14T09:05:31.091 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:31.091 INFO:teuthology.orchestra.run.smithi176.stdout:74 read 0x2f44e67 thru 0x2f4e466 (0x9600 bytes) 2024-09-14T09:05:31.108 INFO:teuthology.orchestra.run.smithi176.stdout:75 write 0xe2b42de thru 0xe2b4c67 (0x98a bytes) 2024-09-14T09:05:31.114 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:31.112+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:31.116 INFO:teuthology.orchestra.run.smithi176.stdout:76 trunc from 0xe2b4c68 to 0xcc207f8 2024-09-14T09:05:31.139 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:31.139 INFO:teuthology.orchestra.run.smithi176.stdout:79 writesame 0xa8829e0 thru 0xa88804d data_size 0x2e(0x566e bytes) 2024-09-14T09:05:31.146 INFO:teuthology.orchestra.run.smithi176.stdout:truncating to largest ever: 0xdeb6d72 2024-09-14T09:05:31.148 INFO:teuthology.orchestra.run.smithi176.stdout:80 trunc from 0xcc207f8 to 0xdeb6d72 2024-09-14T09:05:31.161 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:31.161 INFO:teuthology.orchestra.run.smithi176.stdout:84 write 0x2300e2 thru 0x23ec88 (0xeba7 bytes) 2024-09-14T09:05:31.179 INFO:teuthology.orchestra.run.smithi176.stdout:85 trunc from 0xdeb6d72 to 0x9be424b 2024-09-14T09:05:31.194 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:31.194 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:31.208 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:05:31.215 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:31.216+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:31.222 INFO:teuthology.orchestra.run.smithi176.stdout:86 write 0xcad9662 thru 0xcae046a (0x6e09 bytes) 2024-09-14T09:05:31.242 INFO:teuthology.orchestra.run.smithi176.stdout:87 trunc from 0xcae046b to 0x47774b8 2024-09-14T09:05:31.266 INFO:teuthology.orchestra.run.smithi176.stdout:88 punch from 0x2ff577 to 0x309a27, (0xa4b0 bytes) 2024-09-14T09:05:31.266 INFO:teuthology.orchestra.run.smithi176.stdout:89 writesame 0xd4972e thru 0xd55fc4 data_size 0x1(0xc897 bytes) 2024-09-14T09:05:31.317 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:31.320+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:31.339 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:31.340 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:31.353 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:05:31.418 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:31.420+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:31.458 INFO:teuthology.orchestra.run.smithi176.stdout:90 writesame 0xb217fb5 thru 0xb21c42d data_size 0x1(0x4479 bytes) 2024-09-14T09:05:31.459 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:31.459 INFO:teuthology.orchestra.run.smithi176.stdout:truncating to largest ever: 0xe0ec37d 2024-09-14T09:05:31.464 INFO:teuthology.orchestra.run.smithi176.stdout:92 trunc from 0xb21c42e to 0xe0ec37d 2024-09-14T09:05:31.478 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:31.478 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:31.478 INFO:teuthology.orchestra.run.smithi176.stdout:95 write 0xcf2d936 thru 0xcf3d27c (0xf947 bytes) 2024-09-14T09:05:31.494 INFO:teuthology.orchestra.run.smithi176.stdout:98 write 0x57265be thru 0x5729c09 (0x364c bytes) 2024-09-14T09:05:31.502 INFO:teuthology.orchestra.run.smithi176.stdout:99 writesame 0x20917dc thru 0x2093fa7 data_size 0x12(0x27cc bytes) 2024-09-14T09:05:31.507 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:31.507 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:31.508 INFO:teuthology.orchestra.run.smithi176.stdout:100 read 0x17a284a thru 0x17a94b1 (0x6c68 bytes) 2024-09-14T09:05:31.509 INFO:teuthology.orchestra.run.smithi176.stdout:101 punch from 0x7d1a8b7 to 0x7d1c63c, (0x1d85 bytes) 2024-09-14T09:05:31.509 INFO:teuthology.orchestra.run.smithi176.stdout:103 read 0x377d35d thru 0x377d5ff (0x2a3 bytes) 2024-09-14T09:05:31.509 INFO:teuthology.orchestra.run.smithi176.stdout:104 read 0x309d9e6 thru 0x309ff3e (0x2559 bytes) 2024-09-14T09:05:31.509 INFO:teuthology.orchestra.run.smithi176.stdout:105 punch from 0x4906b70 to 0x490857f, (0x1a0f bytes) 2024-09-14T09:05:31.509 INFO:teuthology.orchestra.run.smithi176.stdout:106 punch from 0x5c6ddf1 to 0x5c74357, (0x6566 bytes) 2024-09-14T09:05:31.509 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:31.509 INFO:teuthology.orchestra.run.smithi176.stdout:108 write 0x61ad149 thru 0x61b054f (0x3407 bytes) 2024-09-14T09:05:31.515 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:31.515 INFO:teuthology.orchestra.run.smithi176.stdout:110 write 0x16adcd1 thru 0x16bdad3 (0xfe03 bytes) 2024-09-14T09:05:31.518 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:31.520+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:31.521 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:05:31.540 INFO:teuthology.orchestra.run.smithi176.stdout:111 writesame 0xe4a8206 thru 0xe4b7abb data_size 0x5(0xf8b6 bytes) 2024-09-14T09:05:31.540 INFO:teuthology.orchestra.run.smithi176.stdout:112 read 0x7ca84de thru 0x7cab134 (0x2c57 bytes) 2024-09-14T09:05:31.540 INFO:teuthology.orchestra.run.smithi176.stdout:113 write 0xaebfec4 thru 0xaec6173 (0x62b0 bytes) 2024-09-14T09:05:31.547 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:31.547 INFO:teuthology.orchestra.run.smithi176.stdout:116 write 0xb5b54ac thru 0xb5b7d18 (0x286d bytes) 2024-09-14T09:05:31.551 INFO:teuthology.orchestra.run.smithi176.stdout:118 write 0x5183e43 thru 0x5191bae (0xdd6c bytes) 2024-09-14T09:05:31.558 INFO:teuthology.orchestra.run.smithi176.stdout:120 write 0xca24a19 thru 0xca2f648 (0xac30 bytes) 2024-09-14T09:05:31.581 INFO:teuthology.orchestra.run.smithi176.stdout:121 writesame 0xe9c4af9 thru 0xe9d1471 data_size 0x1(0xc979 bytes) 2024-09-14T09:05:31.581 INFO:teuthology.orchestra.run.smithi176.stdout:122 punch from 0x3e19c72 to 0x3e24786, (0xab14 bytes) 2024-09-14T09:05:31.581 INFO:teuthology.orchestra.run.smithi176.stdout:125 punch from 0x8167aa2 to 0x8171ddf, (0xa33d bytes) 2024-09-14T09:05:31.582 INFO:teuthology.orchestra.run.smithi176.stdout:126 read 0x1654510 thru 0x1655c55 (0x1746 bytes) 2024-09-14T09:05:31.582 INFO:teuthology.orchestra.run.smithi176.stdout:127 punch from 0x65eeba to 0x667e42, (0x8f88 bytes) 2024-09-14T09:05:31.582 INFO:teuthology.orchestra.run.smithi176.stdout:128 read 0x60a3471 thru 0x60a4a9e (0x162e bytes) 2024-09-14T09:05:31.583 INFO:teuthology.orchestra.run.smithi176.stdout:129 writesame 0x9f5db01 thru 0x9f6b58a data_size 0xb(0xda8a bytes) 2024-09-14T09:05:31.598 INFO:teuthology.orchestra.run.smithi176.stdout:130 punch from 0x904ca67 to 0x9055021, (0x85ba bytes) 2024-09-14T09:05:31.599 INFO:teuthology.orchestra.run.smithi176.stdout:131 punch from 0x232e1d3 to 0x2331309, (0x3136 bytes) 2024-09-14T09:05:31.599 INFO:teuthology.orchestra.run.smithi176.stdout:132 write 0x15b432c thru 0x15bab14 (0x67e9 bytes) 2024-09-14T09:05:31.604 INFO:teuthology.orchestra.run.smithi176.stdout:134 read 0x714b5f6 thru 0x7154146 (0x8b51 bytes) 2024-09-14T09:05:31.604 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:31.604 INFO:teuthology.orchestra.run.smithi176.stdout:136 write 0x54cf251 thru 0x54d091d (0x16cd bytes) 2024-09-14T09:05:31.607 INFO:teuthology.orchestra.run.smithi176.stdout:137 read 0x99d987f thru 0x99ddcf9 (0x447b bytes) 2024-09-14T09:05:31.607 INFO:teuthology.orchestra.run.smithi176.stdout:138 write 0xc433110 thru 0xc441047 (0xdf38 bytes) 2024-09-14T09:05:31.616 INFO:teuthology.orchestra.run.smithi176.stdout:140 read 0x254986d thru 0x255886d (0xf001 bytes) 2024-09-14T09:05:31.617 INFO:teuthology.orchestra.run.smithi176.stdout:141 read 0x1dea4a thru 0x1e1d82 (0x3339 bytes) 2024-09-14T09:05:31.618 INFO:teuthology.orchestra.run.smithi176.stdout:143 read 0x2f52b6f thru 0x2f5f3e6 (0xc878 bytes) 2024-09-14T09:05:31.618 INFO:teuthology.orchestra.run.smithi176.stdout:144 writesame 0x2f839c4 thru 0x2f8bf69 data_size 0x1(0x85a6 bytes) 2024-09-14T09:05:31.619 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:31.620+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:31.640 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:31.640 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:31.653 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:05:31.720 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:31.720+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:31.741 INFO:teuthology.orchestra.run.smithi176.stdout:145 read 0xc33fae2 thru 0xc34d16e (0xd68d bytes) 2024-09-14T09:05:31.741 INFO:teuthology.orchestra.run.smithi176.stdout:146 punch from 0x24c0e8 to 0x254bf7, (0x8b0f bytes) 2024-09-14T09:05:31.741 INFO:teuthology.orchestra.run.smithi176.stdout:147 read 0x3446edb thru 0x344de8a (0x6fb0 bytes) 2024-09-14T09:05:31.756 INFO:teuthology.orchestra.run.smithi176.stdout:151 trunc from 0xe9d1472 to 0x78c42fe 2024-09-14T09:05:31.782 INFO:teuthology.orchestra.run.smithi176.stdout:152 write 0x725c7f3 thru 0x7262721 (0x5f2f bytes) 2024-09-14T09:05:31.789 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:31.790 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:31.791 INFO:teuthology.orchestra.run.smithi176.stdout:154 write 0x6a4cc92 thru 0x6a5ba79 (0xede8 bytes) 2024-09-14T09:05:31.804 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:05:31.805 INFO:teuthology.orchestra.run.smithi176.stdout:155 write 0x71ef47c thru 0x71f4563 (0x50e8 bytes) 2024-09-14T09:05:31.809 INFO:teuthology.orchestra.run.smithi176.stdout:156 write 0x7768558 thru 0x776b4a9 (0x2f52 bytes) 2024-09-14T09:05:31.820 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:31.816+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:31.833 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is unset 2024-09-14T09:05:31.833 INFO:teuthology.orchestra.run.smithi176.stdout:157 write 0xb757f96 thru 0xb76332b (0xb396 bytes) 2024-09-14T09:05:31.845 INFO:teuthology.orchestra.run.smithi176.stdout:158 punch from 0x628c7ff to 0x62980ff, (0xb900 bytes) 2024-09-14T09:05:31.845 INFO:teuthology.orchestra.run.smithi176.stdout:159 punch from 0x390a544 to 0x39162f0, (0xbdac bytes) 2024-09-14T09:05:31.859 INFO:teuthology.orchestra.run.smithi176.stdout:161 trunc from 0xb76332c to 0x4808486 2024-09-14T09:05:31.877 INFO:teuthology.orchestra.run.smithi176.stdout:162 read 0x1548bbc thru 0x1557aad (0xeef2 bytes) 2024-09-14T09:05:31.878 INFO:teuthology.orchestra.run.smithi176.stdout:163 read 0x6ca5d6 thru 0x6cf8ed (0x5318 bytes) 2024-09-14T09:05:31.898 INFO:teuthology.orchestra.run.smithi176.stdout:164 write 0xb2ee654 thru 0xb2f0413 (0x1dc0 bytes) 2024-09-14T09:05:31.913 INFO:teuthology.orchestra.run.smithi176.stdout:165 writesame 0x6ba4a94 thru 0x6bac902 data_size 0x1(0x7e6f bytes) 2024-09-14T09:05:31.921 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:31.916+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:31.971 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:31.971 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:31.985 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:05:32.021 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:32.016+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:32.035 INFO:teuthology.orchestra.run.smithi176.stdout:166 read 0xa17cd8c thru 0xa184233 (0x74a8 bytes) 2024-09-14T09:05:32.035 INFO:teuthology.orchestra.run.smithi176.stdout:167 write 0x4a68a53 thru 0x4a6934e (0x8fc bytes) 2024-09-14T09:05:32.040 INFO:teuthology.orchestra.run.smithi176.stdout:168 writesame 0x1ad70ec thru 0x1ae5c3b data_size 0x30(0xeb50 bytes) 2024-09-14T09:05:32.047 INFO:teuthology.orchestra.run.smithi176.stdout:169 writesame 0x9bd2b78 thru 0x9be1b0a data_size 0x1(0xef93 bytes) 2024-09-14T09:05:32.112 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:32.112 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:32.122 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:05:32.122 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:05:32.122 INFO:tasks.thrashosds.thrasher:Reweighting osd 0 to 0.7646213501939743 2024-09-14T09:05:32.122 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 0.7646213501939743 2024-09-14T09:05:32.123 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:32.124+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:32.126 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:05:32.222 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:32.224+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:32.235 INFO:teuthology.orchestra.run.smithi176.stdout:truncating to largest ever: 0xe87abb9 2024-09-14T09:05:32.239 INFO:teuthology.orchestra.run.smithi176.stdout:171 trunc from 0xb2f0414 to 0xe87abb9 2024-09-14T09:05:32.248 INFO:teuthology.orchestra.run.smithi176.stdout:172 write 0x87cd126 thru 0x87d176f (0x464a bytes) 2024-09-14T09:05:32.253 INFO:teuthology.orchestra.run.smithi176.stdout:173 read 0x8ae9682 thru 0x8af7ae3 (0xe462 bytes) 2024-09-14T09:05:32.253 INFO:teuthology.orchestra.run.smithi176.stdout:174 writesame 0xe3ce3fb thru 0xe3d22ff data_size 0x11(0x3f05 bytes) 2024-09-14T09:05:32.257 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:32.257 INFO:teuthology.orchestra.run.smithi176.stdout:179 write 0xd6d204d thru 0xd6dc25b (0xa20f bytes) 2024-09-14T09:05:32.263 INFO:teuthology.orchestra.run.smithi176.stdout:180 write 0x6e7ea02 thru 0x6e81e4d (0x344c bytes) 2024-09-14T09:05:32.268 INFO:teuthology.orchestra.run.smithi176.stdout:183 writesame 0x6e8481e thru 0x6e91b22 data_size 0xb(0xd305 bytes) 2024-09-14T09:05:32.295 INFO:teuthology.orchestra.run.smithi176.stdout:184 trunc from 0xe87abb9 to 0x49a372f 2024-09-14T09:05:32.297 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:32.297 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:32.314 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:05:32.323 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:32.320+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:32.334 INFO:teuthology.orchestra.run.smithi176.stdout:186 write 0x8425ca0 thru 0x8432b4e (0xceaf bytes) 2024-09-14T09:05:32.341 INFO:teuthology.orchestra.run.smithi176.stdout:187 writesame 0x90e5be thru 0x9110d0 data_size 0x1(0x2b13 bytes) 2024-09-14T09:05:32.353 INFO:teuthology.orchestra.run.smithi176.stdout:188 punch from 0x1f3b91b to 0x1f4ad89, (0xf46e bytes) 2024-09-14T09:05:32.354 INFO:teuthology.orchestra.run.smithi176.stdout:189 write 0x530b6c7 thru 0x5311656 (0x5f90 bytes) 2024-09-14T09:05:32.369 INFO:teuthology.orchestra.run.smithi176.stdout:190 trunc from 0x8432b4f to 0x35b41ab 2024-09-14T09:05:32.397 INFO:teuthology.orchestra.run.smithi176.stdout:191 writesame 0x5978e91 thru 0x597b6cd data_size 0x1(0x283d bytes) 2024-09-14T09:05:32.397 INFO:teuthology.orchestra.run.smithi176.stdout:193 punch from 0x2e19828 to 0x2e25ade, (0xc2b6 bytes) 2024-09-14T09:05:32.402 INFO:teuthology.orchestra.run.smithi176.stdout:194 trunc from 0x597b6ce to 0x814ba13 2024-09-14T09:05:32.424 INFO:teuthology.orchestra.run.smithi176.stdout:195 writesame 0xac83835 thru 0xac889d6 data_size 0x56(0x51a2 bytes) 2024-09-14T09:05:32.424 INFO:teuthology.orchestra.run.smithi176.stdout:197 read 0x8fad31f thru 0x8fb1dba (0x4a9c bytes) 2024-09-14T09:05:32.424 INFO:teuthology.orchestra.run.smithi176.stdout:200 punch from 0x29d8729 to 0x29db2d1, (0x2ba8 bytes) 2024-09-14T09:05:32.424 INFO:teuthology.orchestra.run.smithi176.stdout:201 read 0x8cfad79 thru 0x8cfd404 (0x268c bytes) 2024-09-14T09:05:32.424 INFO:teuthology.orchestra.run.smithi176.stdout:203 writesame 0x52f2ff1 thru 0x52ff3ea data_size 0x91(0xc3fa bytes) 2024-09-14T09:05:32.425 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:32.424+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:32.430 INFO:teuthology.orchestra.run.smithi176.stdout:204 punch from 0x79bf0a to 0x7aa45c, (0xe552 bytes) 2024-09-14T09:05:32.430 INFO:teuthology.orchestra.run.smithi176.stdout:205 read 0x7c3b2b3 thru 0x7c4688d (0xb5db bytes) 2024-09-14T09:05:32.430 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:32.431 INFO:teuthology.orchestra.run.smithi176.stdout:207 write 0x7a3bd71 thru 0x7a3e62a (0x28ba bytes) 2024-09-14T09:05:32.439 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:32.439 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:32.457 INFO:teuthology.orchestra.run.smithi176.stdout:212 writesame 0xecd12a8 thru 0xecd4a85 data_size 0x1(0x37de bytes) 2024-09-14T09:05:32.457 INFO:teuthology.orchestra.run.smithi176.stdout:213 write 0x2406eda thru 0x240adbf (0x3ee6 bytes) 2024-09-14T09:05:32.476 INFO:teuthology.orchestra.run.smithi176.stdout:214 write 0x1e063c1 thru 0x1e0ff0f (0x9b4f bytes) 2024-09-14T09:05:32.485 INFO:teuthology.orchestra.run.smithi176.stdout:216 read 0x45af69d thru 0x45b60f3 (0x6a57 bytes) 2024-09-14T09:05:32.485 INFO:teuthology.orchestra.run.smithi176.stdout:219 punch from 0x172836f to 0x17314e4, (0x9175 bytes) 2024-09-14T09:05:32.486 INFO:teuthology.orchestra.run.smithi176.stdout:220 read 0xb6b443f thru 0xb6ba6eb (0x62ad bytes) 2024-09-14T09:05:32.486 INFO:teuthology.orchestra.run.smithi176.stdout:221 write 0xd270d64 thru 0xd273b5f (0x2dfc bytes) 2024-09-14T09:05:32.488 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:32.489 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:32.493 INFO:teuthology.orchestra.run.smithi176.stdout:225 clone 1 order 25 su 262144 sc 15 2024-09-14T09:05:32.504 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:05:32.525 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:32.520+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:32.626 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:32.624+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:32.663 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:32.663 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:32.678 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:05:32.693 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:05:32.738 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:32.736+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:32.800 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:32.800 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:32.815 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:05:32.840 INFO:teuthology.orchestra.run.smithi176.stderr:reweighted osd.0 to 0.764621 (c3be) 2024-09-14T09:05:32.857 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:05:32.858 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:05:32.864 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:32.868+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:32.876 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:05:32.877 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:05:32.894 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:05:32.895 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:05:32.913 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:05:32.913 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:05:32.928 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:32.924+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:32.934 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:05:32.934 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:05:32.946 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:32.946 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:32.953 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:05:32.953 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:05:32.961 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:05:32.974 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:05:32.974 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:05:32.991 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:05:32.992 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:05:33.053 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:33.056+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:33.073 INFO:teuthology.orchestra.run.smithi176.stdout:227 write 0x7fd0e3 thru 0x80c83c (0xf75a bytes) 2024-09-14T09:05:33.130 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:33.128+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:33.138 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:05:33.139 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:05:33.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:05:33.139 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-09-14T09:05:33.139 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:05:33.153 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:05:33.241 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:33.244+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:33.323 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:05:33.323 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:05:33.323 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:05:33.324 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:05:33.332 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:33.332+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:33.337 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:05:33.433 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:33.428+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:33.505 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:05:33.506 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:05:33.506 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:05:33.506 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:05:33.506 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:05:33.520 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:05:33.534 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:33.532+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:33.635 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:33.632+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:33.644 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:05:33.644 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:05:33.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:05:33.644 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:05:33.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.644 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.644 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.275457+0000", 2024-09-14T09:05:33.644 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.367933880999999, 2024-09-14T09:05:33.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013137757, 2024-09-14T09:05:33.644 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.645 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.645 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.645 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.645 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.645 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.645 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 24 2024-09-14T09:05:33.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.645 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275457+0000", 2024-09-14T09:05:33.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275457+0000", 2024-09-14T09:05:33.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275458+0000", 2024-09-14T09:05:33.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0240000000000001e-06 2024-09-14T09:05:33.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275626+0000", 2024-09-14T09:05:33.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000168443 2024-09-14T09:05:33.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275645+0000", 2024-09-14T09:05:33.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8987999999999998e-05 2024-09-14T09:05:33.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275655+0000", 2024-09-14T09:05:33.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0183999999999999e-05 2024-09-14T09:05:33.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.281987+0000", 2024-09-14T09:05:33.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063320709999999999 2024-09-14T09:05:33.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.282158+0000", 2024-09-14T09:05:33.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017053599999999999 2024-09-14T09:05:33.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.282381+0000", 2024-09-14T09:05:33.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022338800000000001 2024-09-14T09:05:33.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.285360+0000", 2024-09-14T09:05:33.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029784960000000002 2024-09-14T09:05:33.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.288429+0000", 2024-09-14T09:05:33.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030686239999999998 2024-09-14T09:05:33.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.288522+0000", 2024-09-14T09:05:33.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3820000000000006e-05 2024-09-14T09:05:33.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.288595+0000", 2024-09-14T09:05:33.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2182999999999997e-05 2024-09-14T09:05:33.650 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.650 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.650 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.650 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.650 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.275677+0000", 2024-09-14T09:05:33.650 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.367713649999999, 2024-09-14T09:05:33.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013095005, 2024-09-14T09:05:33.650 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.650 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.650 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.650 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.650 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.651 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.651 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 25 2024-09-14T09:05:33.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.651 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275677+0000", 2024-09-14T09:05:33.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275677+0000", 2024-09-14T09:05:33.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275678+0000", 2024-09-14T09:05:33.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0440000000000001e-06 2024-09-14T09:05:33.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275687+0000", 2024-09-14T09:05:33.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6659999999999997e-06 2024-09-14T09:05:33.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275701+0000", 2024-09-14T09:05:33.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4591e-05 2024-09-14T09:05:33.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.275709+0000", 2024-09-14T09:05:33.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0439999999999996e-06 2024-09-14T09:05:33.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.283868+0000", 2024-09-14T09:05:33.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0081590250000000003 2024-09-14T09:05:33.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.283953+0000", 2024-09-14T09:05:33.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5044999999999996e-05 2024-09-14T09:05:33.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.284083+0000", 2024-09-14T09:05:33.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012993299999999999 2024-09-14T09:05:33.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.285813+0000", 2024-09-14T09:05:33.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001729424 2024-09-14T09:05:33.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.288608+0000", 2024-09-14T09:05:33.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027955100000000002 2024-09-14T09:05:33.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.288716+0000", 2024-09-14T09:05:33.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000107613 2024-09-14T09:05:33.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.288772+0000", 2024-09-14T09:05:33.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6110000000000003e-05 2024-09-14T09:05:33.655 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.655 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.655 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.655 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.655 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.378740+0000", 2024-09-14T09:05:33.656 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.264650368000002, 2024-09-14T09:05:33.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.012652679999999999, 2024-09-14T09:05:33.656 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.656 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.656 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.656 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.656 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.656 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.656 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 44 2024-09-14T09:05:33.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.656 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378740+0000", 2024-09-14T09:05:33.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378740+0000", 2024-09-14T09:05:33.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378741+0000", 2024-09-14T09:05:33.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2699999999999998e-07 2024-09-14T09:05:33.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378847+0000", 2024-09-14T09:05:33.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000105466 2024-09-14T09:05:33.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378859+0000", 2024-09-14T09:05:33.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.235e-05 2024-09-14T09:05:33.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378869+0000", 2024-09-14T09:05:33.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0213e-05 2024-09-14T09:05:33.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.386294+0000", 2024-09-14T09:05:33.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0074248300000000003 2024-09-14T09:05:33.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.386537+0000", 2024-09-14T09:05:33.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024260899999999999 2024-09-14T09:05:33.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.386943+0000", 2024-09-14T09:05:33.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040585399999999998 2024-09-14T09:05:33.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.389595+0000", 2024-09-14T09:05:33.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026525609999999999 2024-09-14T09:05:33.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.391137+0000", 2024-09-14T09:05:33.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015414890000000001 2024-09-14T09:05:33.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.391274+0000", 2024-09-14T09:05:33.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013721499999999999 2024-09-14T09:05:33.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.391393+0000", 2024-09-14T09:05:33.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119166 2024-09-14T09:05:33.662 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.662 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.662 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.663 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.663 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.378892+0000", 2024-09-14T09:05:33.663 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.264498501999999, 2024-09-14T09:05:33.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016746298, 2024-09-14T09:05:33.663 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.663 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.663 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.663 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.663 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.664 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.664 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 45 2024-09-14T09:05:33.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.664 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378892+0000", 2024-09-14T09:05:33.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378892+0000", 2024-09-14T09:05:33.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378893+0000", 2024-09-14T09:05:33.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.08e-06 2024-09-14T09:05:33.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378901+0000", 2024-09-14T09:05:33.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7770000000000002e-06 2024-09-14T09:05:33.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378914+0000", 2024-09-14T09:05:33.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.271e-05 2024-09-14T09:05:33.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378927+0000", 2024-09-14T09:05:33.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3166000000000001e-05 2024-09-14T09:05:33.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.389712+0000", 2024-09-14T09:05:33.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.01078542 2024-09-14T09:05:33.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.389953+0000", 2024-09-14T09:05:33.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024113100000000001 2024-09-14T09:05:33.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.390323+0000", 2024-09-14T09:05:33.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036942099999999999 2024-09-14T09:05:33.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395157+0000", 2024-09-14T09:05:33.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048345690000000004 2024-09-14T09:05:33.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395371+0000", 2024-09-14T09:05:33.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000213842 2024-09-14T09:05:33.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395520+0000", 2024-09-14T09:05:33.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014902599999999999 2024-09-14T09:05:33.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395638+0000", 2024-09-14T09:05:33.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118156 2024-09-14T09:05:33.669 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.669 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.669 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.670 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.670 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.378952+0000", 2024-09-14T09:05:33.670 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.264438555000002, 2024-09-14T09:05:33.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016969825000000001, 2024-09-14T09:05:33.670 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.670 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.670 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.670 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.670 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.670 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.671 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 46 2024-09-14T09:05:33.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.671 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378952+0000", 2024-09-14T09:05:33.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378952+0000", 2024-09-14T09:05:33.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378953+0000", 2024-09-14T09:05:33.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3200000000000001e-06 2024-09-14T09:05:33.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378964+0000", 2024-09-14T09:05:33.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0183000000000001e-05 2024-09-14T09:05:33.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.378991+0000", 2024-09-14T09:05:33.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7158999999999999e-05 2024-09-14T09:05:33.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379006+0000", 2024-09-14T09:05:33.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.49e-05 2024-09-14T09:05:33.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.391517+0000", 2024-09-14T09:05:33.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.012511520999999999 2024-09-14T09:05:33.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.391677+0000", 2024-09-14T09:05:33.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000159856 2024-09-14T09:05:33.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.392054+0000", 2024-09-14T09:05:33.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037711299999999999 2024-09-14T09:05:33.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395333+0000", 2024-09-14T09:05:33.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0032786310000000002 2024-09-14T09:05:33.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395666+0000", 2024-09-14T09:05:33.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033285399999999999 2024-09-14T09:05:33.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395811+0000", 2024-09-14T09:05:33.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014544300000000001 2024-09-14T09:05:33.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395922+0000", 2024-09-14T09:05:33.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000110845 2024-09-14T09:05:33.676 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.677 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.677 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.677 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.677 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379034+0000", 2024-09-14T09:05:33.677 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.264356849999999, 2024-09-14T09:05:33.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018420946000000001, 2024-09-14T09:05:33.677 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.677 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.677 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.678 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.678 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.678 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.678 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 47 2024-09-14T09:05:33.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.678 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379034+0000", 2024-09-14T09:05:33.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379034+0000", 2024-09-14T09:05:33.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379035+0000", 2024-09-14T09:05:33.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4839999999999999e-06 2024-09-14T09:05:33.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379160+0000", 2024-09-14T09:05:33.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012504200000000001 2024-09-14T09:05:33.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379177+0000", 2024-09-14T09:05:33.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.698e-05 2024-09-14T09:05:33.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379186+0000", 2024-09-14T09:05:33.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0000000000000002e-06 2024-09-14T09:05:33.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.392838+0000", 2024-09-14T09:05:33.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013651949 2024-09-14T09:05:33.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.393080+0000", 2024-09-14T09:05:33.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024200900000000001 2024-09-14T09:05:33.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.393455+0000", 2024-09-14T09:05:33.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037501 2024-09-14T09:05:33.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.397149+0000", 2024-09-14T09:05:33.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0036935409999999998 2024-09-14T09:05:33.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.397187+0000", 2024-09-14T09:05:33.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8445999999999997e-05 2024-09-14T09:05:33.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.397323+0000", 2024-09-14T09:05:33.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135599 2024-09-14T09:05:33.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.397455+0000", 2024-09-14T09:05:33.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000131886 2024-09-14T09:05:33.684 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.684 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.684 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.684 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.684 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379206+0000", 2024-09-14T09:05:33.684 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.264184744000001, 2024-09-14T09:05:33.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.022450290000000001, 2024-09-14T09:05:33.684 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.684 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.685 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.685 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.685 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.685 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.685 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 48 2024-09-14T09:05:33.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.685 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379206+0000", 2024-09-14T09:05:33.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379206+0000", 2024-09-14T09:05:33.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379207+0000", 2024-09-14T09:05:33.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.167e-06 2024-09-14T09:05:33.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379216+0000", 2024-09-14T09:05:33.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1630000000000002e-06 2024-09-14T09:05:33.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379226+0000", 2024-09-14T09:05:33.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9029999999999999e-06 2024-09-14T09:05:33.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379233+0000", 2024-09-14T09:05:33.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.17e-06 2024-09-14T09:05:33.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395986+0000", 2024-09-14T09:05:33.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016752195000000001 2024-09-14T09:05:33.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.396143+0000", 2024-09-14T09:05:33.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000157878 2024-09-14T09:05:33.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.396346+0000", 2024-09-14T09:05:33.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020270499999999999 2024-09-14T09:05:33.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401328+0000", 2024-09-14T09:05:33.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0049818379999999997 2024-09-14T09:05:33.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401367+0000", 2024-09-14T09:05:33.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8926e-05 2024-09-14T09:05:33.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401541+0000", 2024-09-14T09:05:33.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017380600000000001 2024-09-14T09:05:33.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401656+0000", 2024-09-14T09:05:33.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011553900000000001 2024-09-14T09:05:33.691 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.691 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.691 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.691 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.691 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379246+0000", 2024-09-14T09:05:33.691 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.264144961, 2024-09-14T09:05:33.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.023032166, 2024-09-14T09:05:33.691 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.691 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.691 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.691 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.692 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.692 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.692 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49 2024-09-14T09:05:33.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.692 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379246+0000", 2024-09-14T09:05:33.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379246+0000", 2024-09-14T09:05:33.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379247+0000", 2024-09-14T09:05:33.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7300000000000005e-07 2024-09-14T09:05:33.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379253+0000", 2024-09-14T09:05:33.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.28e-06 2024-09-14T09:05:33.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379267+0000", 2024-09-14T09:05:33.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4087000000000001e-05 2024-09-14T09:05:33.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379274+0000", 2024-09-14T09:05:33.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8929999999999999e-06 2024-09-14T09:05:33.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.397561+0000", 2024-09-14T09:05:33.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018286903 2024-09-14T09:05:33.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.397798+0000", 2024-09-14T09:05:33.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000236989 2024-09-14T09:05:33.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.398157+0000", 2024-09-14T09:05:33.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035888700000000001 2024-09-14T09:05:33.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401799+0000", 2024-09-14T09:05:33.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0036428139999999999 2024-09-14T09:05:33.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402029+0000", 2024-09-14T09:05:33.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022967800000000001 2024-09-14T09:05:33.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402165+0000", 2024-09-14T09:05:33.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013627600000000001 2024-09-14T09:05:33.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402278+0000", 2024-09-14T09:05:33.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011248599999999999 2024-09-14T09:05:33.697 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.697 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.698 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.698 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.698 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379312+0000", 2024-09-14T09:05:33.698 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.264078184999999, 2024-09-14T09:05:33.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.024727301, 2024-09-14T09:05:33.698 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.698 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.698 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.698 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.698 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.698 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.699 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 50 2024-09-14T09:05:33.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.699 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379312+0000", 2024-09-14T09:05:33.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379312+0000", 2024-09-14T09:05:33.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379314+0000", 2024-09-14T09:05:33.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.17e-06 2024-09-14T09:05:33.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379408+0000", 2024-09-14T09:05:33.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4562999999999996e-05 2024-09-14T09:05:33.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379429+0000", 2024-09-14T09:05:33.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1189e-05 2024-09-14T09:05:33.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379448+0000", 2024-09-14T09:05:33.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8734000000000001e-05 2024-09-14T09:05:33.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.399051+0000", 2024-09-14T09:05:33.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.019602441000000002 2024-09-14T09:05:33.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.399218+0000", 2024-09-14T09:05:33.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000167095 2024-09-14T09:05:33.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.399438+0000", 2024-09-14T09:05:33.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022046500000000001 2024-09-14T09:05:33.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401991+0000", 2024-09-14T09:05:33.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025529250000000002 2024-09-14T09:05:33.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.403780+0000", 2024-09-14T09:05:33.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017893939999999999 2024-09-14T09:05:33.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.403915+0000", 2024-09-14T09:05:33.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013478200000000001 2024-09-14T09:05:33.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.404040+0000", 2024-09-14T09:05:33.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012454299999999999 2024-09-14T09:05:33.704 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.704 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.704 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.705 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.705 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379481+0000", 2024-09-14T09:05:33.705 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.263909253000001, 2024-09-14T09:05:33.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.026050659, 2024-09-14T09:05:33.705 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.705 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.705 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.705 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.705 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.705 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.705 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 51 2024-09-14T09:05:33.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.706 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379481+0000", 2024-09-14T09:05:33.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379481+0000", 2024-09-14T09:05:33.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379483+0000", 2024-09-14T09:05:33.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3799999999999999e-06 2024-09-14T09:05:33.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379493+0000", 2024-09-14T09:05:33.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0164e-05 2024-09-14T09:05:33.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379535+0000", 2024-09-14T09:05:33.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1672999999999999e-05 2024-09-14T09:05:33.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379552+0000", 2024-09-14T09:05:33.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7615999999999999e-05 2024-09-14T09:05:33.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402365+0000", 2024-09-14T09:05:33.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.022812857999999998 2024-09-14T09:05:33.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402601+0000", 2024-09-14T09:05:33.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000235648 2024-09-14T09:05:33.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402966+0000", 2024-09-14T09:05:33.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036486100000000002 2024-09-14T09:05:33.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405223+0000", 2024-09-14T09:05:33.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002257144 2024-09-14T09:05:33.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405271+0000", 2024-09-14T09:05:33.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7769999999999998e-05 2024-09-14T09:05:33.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405410+0000", 2024-09-14T09:05:33.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013921499999999999 2024-09-14T09:05:33.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405532+0000", 2024-09-14T09:05:33.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012233 2024-09-14T09:05:33.711 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.711 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.711 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.711 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.711 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379582+0000", 2024-09-14T09:05:33.711 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.263808544, 2024-09-14T09:05:33.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.026468416000000002, 2024-09-14T09:05:33.712 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.712 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.712 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.712 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.712 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.712 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.712 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 52 2024-09-14T09:05:33.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.712 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379582+0000", 2024-09-14T09:05:33.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379582+0000", 2024-09-14T09:05:33.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379584+0000", 2024-09-14T09:05:33.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6139999999999999e-06 2024-09-14T09:05:33.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379596+0000", 2024-09-14T09:05:33.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1956e-05 2024-09-14T09:05:33.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379614+0000", 2024-09-14T09:05:33.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8099999999999999e-05 2024-09-14T09:05:33.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379638+0000", 2024-09-14T09:05:33.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4136999999999999e-05 2024-09-14T09:05:33.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.404134+0000", 2024-09-14T09:05:33.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.024496418999999998 2024-09-14T09:05:33.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.404400+0000", 2024-09-14T09:05:33.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026532500000000001 2024-09-14T09:05:33.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.404616+0000", 2024-09-14T09:05:33.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021586500000000001 2024-09-14T09:05:33.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405703+0000", 2024-09-14T09:05:33.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010879220000000001 2024-09-14T09:05:33.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405887+0000", 2024-09-14T09:05:33.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018396499999999999 2024-09-14T09:05:33.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405985+0000", 2024-09-14T09:05:33.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8033000000000001e-05 2024-09-14T09:05:33.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.406051+0000", 2024-09-14T09:05:33.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5079999999999994e-05 2024-09-14T09:05:33.718 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.718 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.718 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.718 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:56 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.718 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.407006+0000", 2024-09-14T09:05:33.718 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.236385134999999, 2024-09-14T09:05:33.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011314423000000001, 2024-09-14T09:05:33.719 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.719 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.719 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.719 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-09-14T09:05:33.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.719 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.407006+0000", 2024-09-14T09:05:33.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.407006+0000", 2024-09-14T09:05:33.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.407007+0000", 2024-09-14T09:05:33.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.13e-06 2024-09-14T09:05:33.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.407079+0000", 2024-09-14T09:05:33.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2189999999999998e-05 2024-09-14T09:05:33.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.407091+0000", 2024-09-14T09:05:33.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.224e-05 2024-09-14T09:05:33.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.407099+0000", 2024-09-14T09:05:33.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0900000000000005e-06 2024-09-14T09:05:33.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.414622+0000", 2024-09-14T09:05:33.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075226959999999997 2024-09-14T09:05:33.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.414854+0000", 2024-09-14T09:05:33.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00023222499999999999 2024-09-14T09:05:33.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.415301+0000", 2024-09-14T09:05:33.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00044665299999999998 2024-09-14T09:05:33.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.417204+0000", 2024-09-14T09:05:33.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00190299 2024-09-14T09:05:33.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.418087+0000", 2024-09-14T09:05:33.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00088314999999999997 2024-09-14T09:05:33.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.418214+0000", 2024-09-14T09:05:33.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012739000000000001 2024-09-14T09:05:33.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.418320+0000", 2024-09-14T09:05:33.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010566899999999999 2024-09-14T09:05:33.725 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.725 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.725 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.726 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.726 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.646870+0000", 2024-09-14T09:05:33.726 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.996520609000001, 2024-09-14T09:05:33.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013689005000000001, 2024-09-14T09:05:33.726 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.726 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.726 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.726 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.726 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.726 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.726 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 69 2024-09-14T09:05:33.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.727 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.646870+0000", 2024-09-14T09:05:33.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.646870+0000", 2024-09-14T09:05:33.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.646871+0000", 2024-09-14T09:05:33.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4899999999999995e-07 2024-09-14T09:05:33.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.646957+0000", 2024-09-14T09:05:33.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6310999999999994e-05 2024-09-14T09:05:33.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.729 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.646971+0000", 2024-09-14T09:05:33.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3895e-05 2024-09-14T09:05:33.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.729 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.729 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.646980+0000", 2024-09-14T09:05:33.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7960000000000001e-06 2024-09-14T09:05:33.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.729 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.729 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.652859+0000", 2024-09-14T09:05:33.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058793040000000001 2024-09-14T09:05:33.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.653028+0000", 2024-09-14T09:05:33.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016869300000000001 2024-09-14T09:05:33.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.653216+0000", 2024-09-14T09:05:33.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018876599999999999 2024-09-14T09:05:33.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.656708+0000", 2024-09-14T09:05:33.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003491105 2024-09-14T09:05:33.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660421+0000", 2024-09-14T09:05:33.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003713413 2024-09-14T09:05:33.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660496+0000", 2024-09-14T09:05:33.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4847999999999999e-05 2024-09-14T09:05:33.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660559+0000", 2024-09-14T09:05:33.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3225000000000005e-05 2024-09-14T09:05:33.732 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.732 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.732 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.732 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.732 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.646998+0000", 2024-09-14T09:05:33.732 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.996392496999999, 2024-09-14T09:05:33.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013692605, 2024-09-14T09:05:33.732 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.733 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.733 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.733 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.733 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.733 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.733 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-09-14T09:05:33.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.733 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.646998+0000", 2024-09-14T09:05:33.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.646998+0000", 2024-09-14T09:05:33.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.646999+0000", 2024-09-14T09:05:33.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2200000000000002e-07 2024-09-14T09:05:33.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647006+0000", 2024-09-14T09:05:33.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9999999999999999e-06 2024-09-14T09:05:33.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647016+0000", 2024-09-14T09:05:33.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5650000000000007e-06 2024-09-14T09:05:33.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647023+0000", 2024-09-14T09:05:33.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7719999999999994e-06 2024-09-14T09:05:33.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.656766+0000", 2024-09-14T09:05:33.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0097425310000000005 2024-09-14T09:05:33.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.656889+0000", 2024-09-14T09:05:33.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012292499999999999 2024-09-14T09:05:33.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.657085+0000", 2024-09-14T09:05:33.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000196078 2024-09-14T09:05:33.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660398+0000", 2024-09-14T09:05:33.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033131020000000001 2024-09-14T09:05:33.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660570+0000", 2024-09-14T09:05:33.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017209199999999999 2024-09-14T09:05:33.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660633+0000", 2024-09-14T09:05:33.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2461999999999999e-05 2024-09-14T09:05:33.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660691+0000", 2024-09-14T09:05:33.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8155999999999997e-05 2024-09-14T09:05:33.739 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.739 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.739 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.739 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.739 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.647044+0000", 2024-09-14T09:05:33.739 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.996346697, 2024-09-14T09:05:33.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015792967000000002, 2024-09-14T09:05:33.740 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.740 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.740 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.740 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.740 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.740 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.740 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 71 2024-09-14T09:05:33.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.740 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647044+0000", 2024-09-14T09:05:33.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647044+0000", 2024-09-14T09:05:33.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647045+0000", 2024-09-14T09:05:33.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2600000000000002e-07 2024-09-14T09:05:33.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647051+0000", 2024-09-14T09:05:33.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3910000000000003e-06 2024-09-14T09:05:33.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647063+0000", 2024-09-14T09:05:33.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1684e-05 2024-09-14T09:05:33.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647070+0000", 2024-09-14T09:05:33.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0169999999999999e-06 2024-09-14T09:05:33.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.657564+0000", 2024-09-14T09:05:33.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010494204 2024-09-14T09:05:33.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.657630+0000", 2024-09-14T09:05:33.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5674999999999997e-05 2024-09-14T09:05:33.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.657729+0000", 2024-09-14T09:05:33.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9036999999999994e-05 2024-09-14T09:05:33.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660706+0000", 2024-09-14T09:05:33.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029772959999999999 2024-09-14T09:05:33.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.662689+0000", 2024-09-14T09:05:33.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00198321 2024-09-14T09:05:33.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.662773+0000", 2024-09-14T09:05:33.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3763000000000004e-05 2024-09-14T09:05:33.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.662837+0000", 2024-09-14T09:05:33.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3963999999999999e-05 2024-09-14T09:05:33.746 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.746 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.746 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.746 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.747 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.647112+0000", 2024-09-14T09:05:33.747 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.996278433000001, 2024-09-14T09:05:33.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016113565, 2024-09-14T09:05:33.747 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.747 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.747 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.747 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.747 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.747 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.747 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 72 2024-09-14T09:05:33.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.747 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647112+0000", 2024-09-14T09:05:33.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647112+0000", 2024-09-14T09:05:33.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647113+0000", 2024-09-14T09:05:33.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0300000000000001e-06 2024-09-14T09:05:33.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647176+0000", 2024-09-14T09:05:33.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3138000000000004e-05 2024-09-14T09:05:33.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647199+0000", 2024-09-14T09:05:33.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.213e-05 2024-09-14T09:05:33.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647210+0000", 2024-09-14T09:05:33.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1854e-05 2024-09-14T09:05:33.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.658101+0000", 2024-09-14T09:05:33.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010891059999999999 2024-09-14T09:05:33.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.658225+0000", 2024-09-14T09:05:33.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012339100000000001 2024-09-14T09:05:33.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.658465+0000", 2024-09-14T09:05:33.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000240399 2024-09-14T09:05:33.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.662626+0000", 2024-09-14T09:05:33.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041610019999999996 2024-09-14T09:05:33.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.663020+0000", 2024-09-14T09:05:33.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000393736 2024-09-14T09:05:33.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.663111+0000", 2024-09-14T09:05:33.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1218999999999995e-05 2024-09-14T09:05:33.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.663226+0000", 2024-09-14T09:05:33.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011460599999999999 2024-09-14T09:05:33.753 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.753 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.753 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.753 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.753 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.647229+0000", 2024-09-14T09:05:33.753 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.996161811, 2024-09-14T09:05:33.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017463752999999999, 2024-09-14T09:05:33.754 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.754 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.754 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.754 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.754 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.754 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.754 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 73 2024-09-14T09:05:33.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.754 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647229+0000", 2024-09-14T09:05:33.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647229+0000", 2024-09-14T09:05:33.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647230+0000", 2024-09-14T09:05:33.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-06 2024-09-14T09:05:33.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647238+0000", 2024-09-14T09:05:33.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2819999999999996e-06 2024-09-14T09:05:33.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647251+0000", 2024-09-14T09:05:33.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3064999999999999e-05 2024-09-14T09:05:33.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647259+0000", 2024-09-14T09:05:33.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2069999999999998e-06 2024-09-14T09:05:33.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.659745+0000", 2024-09-14T09:05:33.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.01248609 2024-09-14T09:05:33.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.659821+0000", 2024-09-14T09:05:33.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6493000000000001e-05 2024-09-14T09:05:33.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.659919+0000", 2024-09-14T09:05:33.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.781e-05 2024-09-14T09:05:33.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.662855+0000", 2024-09-14T09:05:33.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029360900000000001 2024-09-14T09:05:33.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.664495+0000", 2024-09-14T09:05:33.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016393289999999999 2024-09-14T09:05:33.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.664592+0000", 2024-09-14T09:05:33.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7672999999999998e-05 2024-09-14T09:05:33.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.664693+0000", 2024-09-14T09:05:33.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100423 2024-09-14T09:05:33.760 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.760 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.760 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.760 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.760 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.647274+0000", 2024-09-14T09:05:33.760 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.996116389000001, 2024-09-14T09:05:33.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018286337999999999, 2024-09-14T09:05:33.761 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.761 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.761 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.761 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.761 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.761 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.761 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 74 2024-09-14T09:05:33.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.761 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647274+0000", 2024-09-14T09:05:33.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647274+0000", 2024-09-14T09:05:33.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647275+0000", 2024-09-14T09:05:33.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8299999999999996e-07 2024-09-14T09:05:33.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647285+0000", 2024-09-14T09:05:33.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.533e-06 2024-09-14T09:05:33.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647295+0000", 2024-09-14T09:05:33.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0216e-05 2024-09-14T09:05:33.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647303+0000", 2024-09-14T09:05:33.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5820000000000002e-06 2024-09-14T09:05:33.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660744+0000", 2024-09-14T09:05:33.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013440934999999999 2024-09-14T09:05:33.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660828+0000", 2024-09-14T09:05:33.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3949999999999994e-05 2024-09-14T09:05:33.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.660954+0000", 2024-09-14T09:05:33.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012590600000000001 2024-09-14T09:05:33.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.662869+0000", 2024-09-14T09:05:33.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001914415 2024-09-14T09:05:33.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.665383+0000", 2024-09-14T09:05:33.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025140560000000002 2024-09-14T09:05:33.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.665478+0000", 2024-09-14T09:05:33.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5642000000000003e-05 2024-09-14T09:05:33.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.665561+0000", 2024-09-14T09:05:33.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2319999999999998e-05 2024-09-14T09:05:33.767 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.767 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.767 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.767 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.767 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.647322+0000", 2024-09-14T09:05:33.767 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.996068515000001, 2024-09-14T09:05:33.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021121719000000001, 2024-09-14T09:05:33.767 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.767 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.767 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.768 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.768 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.768 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.768 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 75 2024-09-14T09:05:33.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.768 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647322+0000", 2024-09-14T09:05:33.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647322+0000", 2024-09-14T09:05:33.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647323+0000", 2024-09-14T09:05:33.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.04e-06 2024-09-14T09:05:33.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647407+0000", 2024-09-14T09:05:33.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3652999999999993e-05 2024-09-14T09:05:33.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647416+0000", 2024-09-14T09:05:33.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2310000000000002e-06 2024-09-14T09:05:33.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647424+0000", 2024-09-14T09:05:33.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3240000000000003e-06 2024-09-14T09:05:33.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.661338+0000", 2024-09-14T09:05:33.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013913114000000001 2024-09-14T09:05:33.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.661461+0000", 2024-09-14T09:05:33.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012374200000000001 2024-09-14T09:05:33.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.661653+0000", 2024-09-14T09:05:33.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000191872 2024-09-14T09:05:33.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.664371+0000", 2024-09-14T09:05:33.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027177600000000001 2024-09-14T09:05:33.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668357+0000", 2024-09-14T09:05:33.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039863229999999999 2024-09-14T09:05:33.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668403+0000", 2024-09-14T09:05:33.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6276000000000002e-05 2024-09-14T09:05:33.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668444+0000", 2024-09-14T09:05:33.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0383999999999998e-05 2024-09-14T09:05:33.773 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.773 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.773 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.774 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:33.774 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.647440+0000", 2024-09-14T09:05:33.774 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.995950197999999, 2024-09-14T09:05:33.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021273368000000001, 2024-09-14T09:05:33.774 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:33.774 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:33.774 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:33.774 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:33.774 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:33.774 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:33.774 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 76 2024-09-14T09:05:33.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.775 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:33.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:33.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647440+0000", 2024-09-14T09:05:33.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:33.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647440+0000", 2024-09-14T09:05:33.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:33.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:33.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647441+0000", 2024-09-14T09:05:33.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1500000000000003e-07 2024-09-14T09:05:33.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:33.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647448+0000", 2024-09-14T09:05:33.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4420000000000003e-06 2024-09-14T09:05:33.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:33.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647457+0000", 2024-09-14T09:05:33.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:05:33.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:33.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647465+0000", 2024-09-14T09:05:33.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2250000000000003e-06 2024-09-14T09:05:33.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:33.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.663329+0000", 2024-09-14T09:05:33.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015863931000000001 2024-09-14T09:05:33.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:33.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.663474+0000", 2024-09-14T09:05:33.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014509200000000001 2024-09-14T09:05:33.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:33.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.663695+0000", 2024-09-14T09:05:33.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022155499999999999 2024-09-14T09:05:33.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:33.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.664823+0000", 2024-09-14T09:05:33.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001128046 2024-09-14T09:05:33.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:33.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668550+0000", 2024-09-14T09:05:33.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003726493 2024-09-14T09:05:33.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:33.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668638+0000", 2024-09-14T09:05:33.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8282999999999995e-05 2024-09-14T09:05:33.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:33.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:33.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:33.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668714+0000", 2024-09-14T09:05:33.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5856e-05 2024-09-14T09:05:33.780 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.780 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.780 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.780 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:33.780 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:33.780 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:05:33.782 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:33.784+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:33.782 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:05:33.864 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T09:05:33.882 INFO:teuthology.orchestra.run.smithi176.stdout:228 trunc from 0xecd4a86 to 0xbf73d53 2024-09-14T09:05:33.882 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:33.884+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:33.935 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:33.935 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:33.935 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:33.935 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:33.948 INFO:teuthology.orchestra.run.smithi176.stdout:230 trunc from 0xbf73d53 to 0x5d4b45e 2024-09-14T09:05:33.957 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:05:33.964 INFO:teuthology.orchestra.run.smithi176.stdout:231 read 0x163c2f9 thru 0x163dd29 (0x1a31 bytes) 2024-09-14T09:05:33.978 INFO:teuthology.orchestra.run.smithi176.stdout:232 write 0x9ad6f1e thru 0x9adde65 (0x6f48 bytes) 2024-09-14T09:05:33.983 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:33.984+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:33.999 INFO:teuthology.orchestra.run.smithi176.stdout:233 write 0xe8437b9 thru 0xe848ff4 (0x583c bytes) 2024-09-14T09:05:34.001 INFO:teuthology.orchestra.run.smithi176.stdout:234 read 0xde07fa1 thru 0xde16629 (0xe689 bytes) 2024-09-14T09:05:34.002 INFO:teuthology.orchestra.run.smithi176.stdout:235 writesame 0xa69e6f3 thru 0xa69f521 data_size 0x1(0xe2f bytes) 2024-09-14T09:05:34.005 INFO:teuthology.orchestra.run.smithi176.stdout:236 read 0x43b5e46 thru 0x43b9d4d (0x3f08 bytes) 2024-09-14T09:05:34.005 INFO:teuthology.orchestra.run.smithi176.stdout:237 read 0xcb944e5 thru 0xcb97fb6 (0x3ad2 bytes) 2024-09-14T09:05:34.006 INFO:teuthology.orchestra.run.smithi176.stdout:238 read 0x2db210e thru 0x2db2ed4 (0xdc7 bytes) 2024-09-14T09:05:34.006 INFO:teuthology.orchestra.run.smithi176.stdout:241 clone 2 order 20 su 32768 sc 5 2024-09-14T09:05:34.084 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:34.088+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:34.088 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:34.088 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:34.088 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:34.088 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:34.088 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:34.103 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:05:34.190 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:34.188+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:34.263 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:34.264 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:34.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:34.264 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:34.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.264 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-14T09:05:34.264 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.274102+0000", 2024-09-14T09:05:34.264 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.987681282, 2024-09-14T09:05:34.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003545763, 2024-09-14T09:05:34.265 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.265 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.265 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.265 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274102+0000", 2024-09-14T09:05:34.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274102+0000", 2024-09-14T09:05:34.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274103+0000", 2024-09-14T09:05:34.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4670000000000001e-06 2024-09-14T09:05:34.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274328+0000", 2024-09-14T09:05:34.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022502500000000001 2024-09-14T09:05:34.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274354+0000", 2024-09-14T09:05:34.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5993e-05 2024-09-14T09:05:34.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274372+0000", 2024-09-14T09:05:34.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8117e-05 2024-09-14T09:05:34.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274455+0000", 2024-09-14T09:05:34.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2785000000000001e-05 2024-09-14T09:05:34.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274486+0000", 2024-09-14T09:05:34.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0864000000000001e-05 2024-09-14T09:05:34.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277538+0000", 2024-09-14T09:05:34.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0030521860000000001 2024-09-14T09:05:34.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277647+0000", 2024-09-14T09:05:34.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109326 2024-09-14T09:05:34.269 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.269 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.269 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.269 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-14T09:05:34.269 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.275860+0000", 2024-09-14T09:05:34.269 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.985923108000001, 2024-09-14T09:05:34.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003513562, 2024-09-14T09:05:34.269 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.269 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.269 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.270 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.275860+0000", 2024-09-14T09:05:34.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.275860+0000", 2024-09-14T09:05:34.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.275861+0000", 2024-09-14T09:05:34.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.513e-06 2024-09-14T09:05:34.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276039+0000", 2024-09-14T09:05:34.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017758199999999999 2024-09-14T09:05:34.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276060+0000", 2024-09-14T09:05:34.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0956000000000001e-05 2024-09-14T09:05:34.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276076+0000", 2024-09-14T09:05:34.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6137000000000002e-05 2024-09-14T09:05:34.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276190+0000", 2024-09-14T09:05:34.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011359200000000001 2024-09-14T09:05:34.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276226+0000", 2024-09-14T09:05:34.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6300000000000001e-05 2024-09-14T09:05:34.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.279274+0000", 2024-09-14T09:05:34.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00304813 2024-09-14T09:05:34.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.279373+0000", 2024-09-14T09:05:34.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9351999999999993e-05 2024-09-14T09:05:34.273 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.273 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.273 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.274 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:22 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'12, mlcod=17'8)", 2024-09-14T09:05:34.274 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.277665+0000", 2024-09-14T09:05:34.274 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.984117563000002, 2024-09-14T09:05:34.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0031553430000000001, 2024-09-14T09:05:34.274 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.274 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.274 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.274 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277665+0000", 2024-09-14T09:05:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277665+0000", 2024-09-14T09:05:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277667+0000", 2024-09-14T09:05:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.207e-06 2024-09-14T09:05:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277847+0000", 2024-09-14T09:05:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018088199999999999 2024-09-14T09:05:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277868+0000", 2024-09-14T09:05:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0582999999999999e-05 2024-09-14T09:05:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277883+0000", 2024-09-14T09:05:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4934e-05 2024-09-14T09:05:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277978+0000", 2024-09-14T09:05:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5031999999999994e-05 2024-09-14T09:05:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.278030+0000", 2024-09-14T09:05:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1929999999999999e-05 2024-09-14T09:05:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.280706+0000", 2024-09-14T09:05:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002676042 2024-09-14T09:05:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.280821+0000", 2024-09-14T09:05:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114733 2024-09-14T09:05:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:71 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'49, mlcod=17'45)", 2024-09-14T09:05:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.656686+0000", 2024-09-14T09:05:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.605097141000002, 2024-09-14T09:05:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0029447330000000002, 2024-09-14T09:05:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.656686+0000", 2024-09-14T09:05:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.656686+0000", 2024-09-14T09:05:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.656688+0000", 2024-09-14T09:05:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9e-06 2024-09-14T09:05:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.656702+0000", 2024-09-14T09:05:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4073000000000001e-05 2024-09-14T09:05:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.656730+0000", 2024-09-14T09:05:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8543000000000001e-05 2024-09-14T09:05:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.656743+0000", 2024-09-14T09:05:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.275e-05 2024-09-14T09:05:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.656916+0000", 2024-09-14T09:05:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017337799999999999 2024-09-14T09:05:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.656952+0000", 2024-09-14T09:05:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5689999999999999e-05 2024-09-14T09:05:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659554+0000", 2024-09-14T09:05:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002601557 2024-09-14T09:05:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659630+0000", 2024-09-14T09:05:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6841999999999994e-05 2024-09-14T09:05:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:72 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'50, mlcod=17'45)", 2024-09-14T09:05:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.657508+0000", 2024-09-14T09:05:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.604274794999998, 2024-09-14T09:05:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0033585239999999999, 2024-09-14T09:05:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657508+0000", 2024-09-14T09:05:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657508+0000", 2024-09-14T09:05:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657510+0000", 2024-09-14T09:05:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8899999999999999e-06 2024-09-14T09:05:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657625+0000", 2024-09-14T09:05:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115216 2024-09-14T09:05:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657649+0000", 2024-09-14T09:05:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4043e-05 2024-09-14T09:05:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657669+0000", 2024-09-14T09:05:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9579999999999999e-05 2024-09-14T09:05:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657790+0000", 2024-09-14T09:05:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012084200000000001 2024-09-14T09:05:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657828+0000", 2024-09-14T09:05:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8019999999999999e-05 2024-09-14T09:05:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660805+0000", 2024-09-14T09:05:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002976874 2024-09-14T09:05:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660867+0000", 2024-09-14T09:05:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2058999999999997e-05 2024-09-14T09:05:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:73 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'51, mlcod=17'45)", 2024-09-14T09:05:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.658866+0000", 2024-09-14T09:05:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.602917357999999, 2024-09-14T09:05:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035579930000000002, 2024-09-14T09:05:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658866+0000", 2024-09-14T09:05:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658866+0000", 2024-09-14T09:05:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658867+0000", 2024-09-14T09:05:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.886e-06 2024-09-14T09:05:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658880+0000", 2024-09-14T09:05:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2204e-05 2024-09-14T09:05:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658908+0000", 2024-09-14T09:05:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8555999999999999e-05 2024-09-14T09:05:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658921+0000", 2024-09-14T09:05:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2566e-05 2024-09-14T09:05:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659682+0000", 2024-09-14T09:05:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000760865 2024-09-14T09:05:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659710+0000", 2024-09-14T09:05:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8612e-05 2024-09-14T09:05:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662349+0000", 2024-09-14T09:05:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0026388370000000002 2024-09-14T09:05:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662424+0000", 2024-09-14T09:05:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4467000000000004e-05 2024-09-14T09:05:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:74 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'52, mlcod=17'47)", 2024-09-14T09:05:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.659878+0000", 2024-09-14T09:05:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.601904789999999, 2024-09-14T09:05:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0040591689999999996, 2024-09-14T09:05:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659878+0000", 2024-09-14T09:05:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659878+0000", 2024-09-14T09:05:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659892+0000", 2024-09-14T09:05:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3487e-05 2024-09-14T09:05:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659902+0000", 2024-09-14T09:05:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.062e-05 2024-09-14T09:05:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659928+0000", 2024-09-14T09:05:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6060000000000001e-05 2024-09-14T09:05:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659941+0000", 2024-09-14T09:05:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2649999999999999e-05 2024-09-14T09:05:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660304+0000", 2024-09-14T09:05:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036354699999999998 2024-09-14T09:05:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660349+0000", 2024-09-14T09:05:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4938999999999998e-05 2024-09-14T09:05:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.663847+0000", 2024-09-14T09:05:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0034973339999999999 2024-09-14T09:05:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.663937+0000", 2024-09-14T09:05:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0532000000000007e-05 2024-09-14T09:05:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'47)", 2024-09-14T09:05:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.660695+0000", 2024-09-14T09:05:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.601087638999999, 2024-09-14T09:05:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0062206520000000001, 2024-09-14T09:05:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660695+0000", 2024-09-14T09:05:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660695+0000", 2024-09-14T09:05:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660697+0000", 2024-09-14T09:05:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.897e-06 2024-09-14T09:05:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660816+0000", 2024-09-14T09:05:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118442 2024-09-14T09:05:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660839+0000", 2024-09-14T09:05:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3790000000000001e-05 2024-09-14T09:05:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660852+0000", 2024-09-14T09:05:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-05 2024-09-14T09:05:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660967+0000", 2024-09-14T09:05:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115276 2024-09-14T09:05:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.661016+0000", 2024-09-14T09:05:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8785999999999997e-05 2024-09-14T09:05:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.666825+0000", 2024-09-14T09:05:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0058087549999999996 2024-09-14T09:05:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.666916+0000", 2024-09-14T09:05:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1325999999999999e-05 2024-09-14T09:05:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'49)", 2024-09-14T09:05:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.662683+0000", 2024-09-14T09:05:34.303 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.599100326999999, 2024-09-14T09:05:34.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0042817899999999997, 2024-09-14T09:05:34.303 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.303 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.303 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.303 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662683+0000", 2024-09-14T09:05:34.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662683+0000", 2024-09-14T09:05:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662684+0000", 2024-09-14T09:05:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6559999999999999e-06 2024-09-14T09:05:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662696+0000", 2024-09-14T09:05:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1454000000000001e-05 2024-09-14T09:05:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662724+0000", 2024-09-14T09:05:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8535999999999999e-05 2024-09-14T09:05:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662752+0000", 2024-09-14T09:05:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7503000000000001e-05 2024-09-14T09:05:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662867+0000", 2024-09-14T09:05:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011500199999999999 2024-09-14T09:05:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662899+0000", 2024-09-14T09:05:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2122999999999997e-05 2024-09-14T09:05:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.666919+0000", 2024-09-14T09:05:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004020633 2024-09-14T09:05:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.666964+0000", 2024-09-14T09:05:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4882999999999997e-05 2024-09-14T09:05:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:77 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'55, mlcod=17'53)", 2024-09-14T09:05:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.668436+0000", 2024-09-14T09:05:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.593346566000001, 2024-09-14T09:05:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035848659999999999, 2024-09-14T09:05:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668436+0000", 2024-09-14T09:05:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668436+0000", 2024-09-14T09:05:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668438+0000", 2024-09-14T09:05:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.81e-06 2024-09-14T09:05:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668551+0000", 2024-09-14T09:05:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112726 2024-09-14T09:05:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668580+0000", 2024-09-14T09:05:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8702999999999999e-05 2024-09-14T09:05:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668592+0000", 2024-09-14T09:05:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.276e-05 2024-09-14T09:05:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668767+0000", 2024-09-14T09:05:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017485600000000001 2024-09-14T09:05:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668808+0000", 2024-09-14T09:05:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1189000000000002e-05 2024-09-14T09:05:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.671940+0000", 2024-09-14T09:05:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0031317329999999998 2024-09-14T09:05:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672021+0000", 2024-09-14T09:05:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1088999999999996e-05 2024-09-14T09:05:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:78 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'56, mlcod=17'53)", 2024-09-14T09:05:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.670356+0000", 2024-09-14T09:05:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.591426780999999, 2024-09-14T09:05:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0036555289999999998, 2024-09-14T09:05:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670356+0000", 2024-09-14T09:05:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670356+0000", 2024-09-14T09:05:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670358+0000", 2024-09-14T09:05:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.79e-06 2024-09-14T09:05:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670469+0000", 2024-09-14T09:05:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111273 2024-09-14T09:05:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670495+0000", 2024-09-14T09:05:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5516000000000001e-05 2024-09-14T09:05:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670508+0000", 2024-09-14T09:05:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2986999999999999e-05 2024-09-14T09:05:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670679+0000", 2024-09-14T09:05:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017094199999999999 2024-09-14T09:05:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670717+0000", 2024-09-14T09:05:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8322999999999999e-05 2024-09-14T09:05:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.673942+0000", 2024-09-14T09:05:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032255489999999999 2024-09-14T09:05:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.674012+0000", 2024-09-14T09:05:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9148999999999998e-05 2024-09-14T09:05:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:79 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'57, mlcod=17'53)", 2024-09-14T09:05:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.672051+0000", 2024-09-14T09:05:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.589732080000001, 2024-09-14T09:05:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0033465410000000002, 2024-09-14T09:05:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672051+0000", 2024-09-14T09:05:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672051+0000", 2024-09-14T09:05:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672053+0000", 2024-09-14T09:05:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0159999999999998e-06 2024-09-14T09:05:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672128+0000", 2024-09-14T09:05:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4909999999999999e-05 2024-09-14T09:05:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672162+0000", 2024-09-14T09:05:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4725999999999999e-05 2024-09-14T09:05:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672181+0000", 2024-09-14T09:05:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8357000000000001e-05 2024-09-14T09:05:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672311+0000", 2024-09-14T09:05:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012992200000000001 2024-09-14T09:05:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672342+0000", 2024-09-14T09:05:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1473000000000002e-05 2024-09-14T09:05:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.675298+0000", 2024-09-14T09:05:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0029560910000000001 2024-09-14T09:05:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.675397+0000", 2024-09-14T09:05:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9046e-05 2024-09-14T09:05:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:80 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'58, mlcod=17'53)", 2024-09-14T09:05:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.673619+0000", 2024-09-14T09:05:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.588163896000001, 2024-09-14T09:05:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0028893719999999999, 2024-09-14T09:05:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.673619+0000", 2024-09-14T09:05:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.673619+0000", 2024-09-14T09:05:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.673621+0000", 2024-09-14T09:05:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5999999999999999e-06 2024-09-14T09:05:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.673679+0000", 2024-09-14T09:05:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8866999999999997e-05 2024-09-14T09:05:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.673712+0000", 2024-09-14T09:05:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2215999999999998e-05 2024-09-14T09:05:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.673755+0000", 2024-09-14T09:05:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3556000000000001e-05 2024-09-14T09:05:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.674068+0000", 2024-09-14T09:05:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031252199999999997 2024-09-14T09:05:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.674099+0000", 2024-09-14T09:05:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.154e-05 2024-09-14T09:05:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.676419+0000", 2024-09-14T09:05:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0023200880000000001 2024-09-14T09:05:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.676508+0000", 2024-09-14T09:05:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8982999999999998e-05 2024-09-14T09:05:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T09:05:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:28.835461+0000", 2024-09-14T09:05:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.426321701, 2024-09-14T09:05:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050182409999999997, 2024-09-14T09:05:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 11 2024-09-14T09:05:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835461+0000", 2024-09-14T09:05:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835461+0000", 2024-09-14T09:05:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835463+0000", 2024-09-14T09:05:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9570000000000001e-06 2024-09-14T09:05:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835474+0000", 2024-09-14T09:05:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0949e-05 2024-09-14T09:05:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835495+0000", 2024-09-14T09:05:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1325e-05 2024-09-14T09:05:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835506+0000", 2024-09-14T09:05:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0854e-05 2024-09-14T09:05:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835579+0000", 2024-09-14T09:05:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2743000000000005e-05 2024-09-14T09:05:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T09:05:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835585+0000", 2024-09-14T09:05:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9519999999999999e-06 2024-09-14T09:05:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.838428+0000", 2024-09-14T09:05:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0028425830000000001 2024-09-14T09:05:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.838557+0000", 2024-09-14T09:05:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012985000000000001 2024-09-14T09:05:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:05:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.838743+0000", 2024-09-14T09:05:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018533199999999999 2024-09-14T09:05:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.839964+0000", 2024-09-14T09:05:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001221387 2024-09-14T09:05:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.840333+0000", 2024-09-14T09:05:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036902899999999999 2024-09-14T09:05:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.840410+0000", 2024-09-14T09:05:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6860999999999995e-05 2024-09-14T09:05:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.840479+0000", 2024-09-14T09:05:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9418999999999994e-05 2024-09-14T09:05:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:657 3.6 3:65b8693c:::rbd_data.115e1e95bf99.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 524288~51261] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.871474+0000", 2024-09-14T09:05:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.390309199, 2024-09-14T09:05:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0082168940000000006, 2024-09-14T09:05:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 657 2024-09-14T09:05:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871474+0000", 2024-09-14T09:05:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871474+0000", 2024-09-14T09:05:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871476+0000", 2024-09-14T09:05:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4109999999999999e-06 2024-09-14T09:05:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871588+0000", 2024-09-14T09:05:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111891 2024-09-14T09:05:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871609+0000", 2024-09-14T09:05:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0856000000000002e-05 2024-09-14T09:05:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871622+0000", 2024-09-14T09:05:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3175e-05 2024-09-14T09:05:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871755+0000", 2024-09-14T09:05:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013305599999999999 2024-09-14T09:05:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872022+0000", 2024-09-14T09:05:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026700600000000003 2024-09-14T09:05:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:05:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872297+0000", 2024-09-14T09:05:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027538299999999998 2024-09-14T09:05:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.879290+0000", 2024-09-14T09:05:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069921779999999999 2024-09-14T09:05:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.879580+0000", 2024-09-14T09:05:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029007000000000002 2024-09-14T09:05:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.879619+0000", 2024-09-14T09:05:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9311999999999999e-05 2024-09-14T09:05:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.879691+0000", 2024-09-14T09:05:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1556000000000004e-05 2024-09-14T09:05:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:673 3.6 3:65b8693c:::rbd_data.115e1e95bf99.0000000000000002:head [stat,zero 13369344~3230342] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.885224+0000", 2024-09-14T09:05:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.37655877599999998, 2024-09-14T09:05:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0046878689999999999, 2024-09-14T09:05:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 673 2024-09-14T09:05:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885224+0000", 2024-09-14T09:05:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885224+0000", 2024-09-14T09:05:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885225+0000", 2024-09-14T09:05:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2600000000000002e-07 2024-09-14T09:05:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885229+0000", 2024-09-14T09:05:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5220000000000001e-06 2024-09-14T09:05:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885238+0000", 2024-09-14T09:05:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5019999999999992e-06 2024-09-14T09:05:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885245+0000", 2024-09-14T09:05:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7909999999999999e-06 2024-09-14T09:05:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885481+0000", 2024-09-14T09:05:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023645 2024-09-14T09:05:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.886357+0000", 2024-09-14T09:05:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00087546399999999995 2024-09-14T09:05:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.889912+0000", 2024-09-14T09:05:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035554139999999998 2024-09-14T09:05:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:844 3.6 3:63058ec8:::rbd_data.115e1e95bf99.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13369344~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.920660+0000", 2024-09-14T09:05:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.34112333900000003, 2024-09-14T09:05:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0024727289999999999, 2024-09-14T09:05:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 844 2024-09-14T09:05:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.920660+0000", 2024-09-14T09:05:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.920660+0000", 2024-09-14T09:05:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.920662+0000", 2024-09-14T09:05:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T09:05:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.920691+0000", 2024-09-14T09:05:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8909000000000001e-05 2024-09-14T09:05:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.920715+0000", 2024-09-14T09:05:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3952999999999999e-05 2024-09-14T09:05:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.920726+0000", 2024-09-14T09:05:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0433e-05 2024-09-14T09:05:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.920805+0000", 2024-09-14T09:05:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9417000000000002e-05 2024-09-14T09:05:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.920990+0000", 2024-09-14T09:05:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018461199999999999 2024-09-14T09:05:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:05:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.921233+0000", 2024-09-14T09:05:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024331600000000001 2024-09-14T09:05:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.922551+0000", 2024-09-14T09:05:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0013177460000000001 2024-09-14T09:05:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.922997+0000", 2024-09-14T09:05:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00044675100000000001 2024-09-14T09:05:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.923068+0000", 2024-09-14T09:05:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0699999999999997e-05 2024-09-14T09:05:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.923132+0000", 2024-09-14T09:05:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4197999999999998e-05 2024-09-14T09:05:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:864 3.6 3:65b8693c:::rbd_data.115e1e95bf99.0000000000000002:head [stat,zero 6553600~6815744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.950678+0000", 2024-09-14T09:05:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.311104667, 2024-09-14T09:05:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0025187429999999999, 2024-09-14T09:05:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 864 2024-09-14T09:05:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.950678+0000", 2024-09-14T09:05:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.950678+0000", 2024-09-14T09:05:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.950681+0000", 2024-09-14T09:05:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1870000000000001e-06 2024-09-14T09:05:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.950694+0000", 2024-09-14T09:05:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2257e-05 2024-09-14T09:05:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.950719+0000", 2024-09-14T09:05:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5556000000000001e-05 2024-09-14T09:05:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.950737+0000", 2024-09-14T09:05:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7774e-05 2024-09-14T09:05:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.950838+0000", 2024-09-14T09:05:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010064999999999999 2024-09-14T09:05:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.950945+0000", 2024-09-14T09:05:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010707799999999999 2024-09-14T09:05:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.953197+0000", 2024-09-14T09:05:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0022522409999999999 2024-09-14T09:05:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:868 3.6 3:63058ec8:::rbd_data.115e1e95bf99.0000000000000006:head [stat,zero 6553600~6815744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.950854+0000", 2024-09-14T09:05:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.31092904700000001, 2024-09-14T09:05:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002481624, 2024-09-14T09:05:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 868 2024-09-14T09:05:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.950854+0000", 2024-09-14T09:05:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.950854+0000", 2024-09-14T09:05:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.950855+0000", 2024-09-14T09:05:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.288e-06 2024-09-14T09:05:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.950862+0000", 2024-09-14T09:05:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8410000000000002e-06 2024-09-14T09:05:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.950875+0000", 2024-09-14T09:05:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3151e-05 2024-09-14T09:05:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.950883+0000", 2024-09-14T09:05:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2209999999999997e-06 2024-09-14T09:05:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951310+0000", 2024-09-14T09:05:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00042649799999999998 2024-09-14T09:05:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951393+0000", 2024-09-14T09:05:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3584999999999993e-05 2024-09-14T09:05:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.953335+0000", 2024-09-14T09:05:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0019420399999999999 2024-09-14T09:05:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:908 3.6 3:63058ec8:::rbd_data.115e1e95bf99.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:34.004614+0000", 2024-09-14T09:05:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.25716885099999998, 2024-09-14T09:05:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0024271729999999999, 2024-09-14T09:05:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 908 2024-09-14T09:05:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.004614+0000", 2024-09-14T09:05:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.004614+0000", 2024-09-14T09:05:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.004616+0000", 2024-09-14T09:05:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.402e-06 2024-09-14T09:05:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.004628+0000", 2024-09-14T09:05:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1523e-05 2024-09-14T09:05:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.004651+0000", 2024-09-14T09:05:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3339000000000001e-05 2024-09-14T09:05:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.004662+0000", 2024-09-14T09:05:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0694e-05 2024-09-14T09:05:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.004740+0000", 2024-09-14T09:05:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7799999999999994e-05 2024-09-14T09:05:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.004811+0000", 2024-09-14T09:05:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0761999999999998e-05 2024-09-14T09:05:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:05:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.005184+0000", 2024-09-14T09:05:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000373867 2024-09-14T09:05:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.006787+0000", 2024-09-14T09:05:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0016028080000000001 2024-09-14T09:05:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.006971+0000", 2024-09-14T09:05:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018422999999999999 2024-09-14T09:05:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.007009+0000", 2024-09-14T09:05:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7314000000000002e-05 2024-09-14T09:05:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.007041+0000", 2024-09-14T09:05:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2434000000000002e-05 2024-09-14T09:05:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.363 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:34.364 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:05:34.364 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:34.364+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:34.465 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:34.468+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:34.532 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:34.533 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:34.533 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:34.533 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:34.546 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:05:34.566 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:34.567+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:34.664 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:34.664 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:34.664 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:34.665 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:34.665 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:34.667 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:34.667+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:34.678 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:05:34.768 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:34.764+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:34.796 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:34.796 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:34.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:34.796 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:34.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.797 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4214.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T09:05:34.797 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:18.300171+0000", 2024-09-14T09:05:34.797 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.496478496000002, 2024-09-14T09:05:34.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018859, 2024-09-14T09:05:34.797 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.797 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.797 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.797 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.300171+0000", 2024-09-14T09:05:34.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.300171+0000", 2024-09-14T09:05:34.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.300171+0000", 2024-09-14T09:05:34.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.06e-07 2024-09-14T09:05:34.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.300180+0000", 2024-09-14T09:05:34.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0240000000000003e-06 2024-09-14T09:05:34.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.300225+0000", 2024-09-14T09:05:34.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.456e-05 2024-09-14T09:05:34.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.300251+0000", 2024-09-14T09:05:34.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6679e-05 2024-09-14T09:05:34.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.300445+0000", 2024-09-14T09:05:34.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019341200000000001 2024-09-14T09:05:34.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.300518+0000", 2024-09-14T09:05:34.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2723000000000002e-05 2024-09-14T09:05:34.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.301977+0000", 2024-09-14T09:05:34.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0014591649999999999 2024-09-14T09:05:34.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.302056+0000", 2024-09-14T09:05:34.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9730999999999999e-05 2024-09-14T09:05:34.801 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.801 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.801 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.801 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4214.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-14T09:05:34.801 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:19.309932+0000", 2024-09-14T09:05:34.801 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.486716705999999, 2024-09-14T09:05:34.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0017493020000000001, 2024-09-14T09:05:34.801 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.801 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.801 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.802 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:19.309932+0000", 2024-09-14T09:05:34.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:19.309932+0000", 2024-09-14T09:05:34.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:19.309933+0000", 2024-09-14T09:05:34.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5999999999999991e-07 2024-09-14T09:05:34.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:19.309947+0000", 2024-09-14T09:05:34.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3416e-05 2024-09-14T09:05:34.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:19.309981+0000", 2024-09-14T09:05:34.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4693000000000001e-05 2024-09-14T09:05:34.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:19.310008+0000", 2024-09-14T09:05:34.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7112999999999999e-05 2024-09-14T09:05:34.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:19.310165+0000", 2024-09-14T09:05:34.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015642600000000001 2024-09-14T09:05:34.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:19.310204+0000", 2024-09-14T09:05:34.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9082999999999998e-05 2024-09-14T09:05:34.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:19.311604+0000", 2024-09-14T09:05:34.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001399742 2024-09-14T09:05:34.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:19.311682+0000", 2024-09-14T09:05:34.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7868999999999996e-05 2024-09-14T09:05:34.805 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.805 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.805 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.806 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:9 3.1a e26/23 3:5cea7035:::rbd_info:head v 26'2, mlcod=0'0)", 2024-09-14T09:05:34.806 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:28.826459+0000", 2024-09-14T09:05:34.806 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.9701896879999996, 2024-09-14T09:05:34.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0011434629999999999, 2024-09-14T09:05:34.806 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.806 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.806 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.806 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.826459+0000", 2024-09-14T09:05:34.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.826459+0000", 2024-09-14T09:05:34.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.826460+0000", 2024-09-14T09:05:34.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5400000000000001e-07 2024-09-14T09:05:34.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.826472+0000", 2024-09-14T09:05:34.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2089999999999999e-05 2024-09-14T09:05:34.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.826492+0000", 2024-09-14T09:05:34.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0326000000000002e-05 2024-09-14T09:05:34.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.826511+0000", 2024-09-14T09:05:34.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8943000000000001e-05 2024-09-14T09:05:34.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.826601+0000", 2024-09-14T09:05:34.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9251000000000003e-05 2024-09-14T09:05:34.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.826629+0000", 2024-09-14T09:05:34.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8045999999999999e-05 2024-09-14T09:05:34.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.827567+0000", 2024-09-14T09:05:34.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000938604 2024-09-14T09:05:34.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.827603+0000", 2024-09-14T09:05:34.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5549000000000001e-05 2024-09-14T09:05:34.810 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.810 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.810 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.810 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "MOSDPGPush(3.18 29/28 [PushOp(3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:head, version: 26'1, data_included: [595932~10188], data_size: 10188, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:head@26'1, size: 606120, copy_subset: [(0, 606120)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 606120, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T09:05:34.810 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.852529+0000", 2024-09-14T09:05:34.810 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.94412015699999996, 2024-09-14T09:05:34.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00084772000000000005, 2024-09-14T09:05:34.810 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.810 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.811 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:34.811 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.852529+0000", 2024-09-14T09:05:34.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.852529+0000", 2024-09-14T09:05:34.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.852531+0000", 2024-09-14T09:05:34.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.725e-06 2024-09-14T09:05:34.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.852595+0000", 2024-09-14T09:05:34.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4116999999999996e-05 2024-09-14T09:05:34.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.852637+0000", 2024-09-14T09:05:34.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2268000000000003e-05 2024-09-14T09:05:34.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.852664+0000", 2024-09-14T09:05:34.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.711e-05 2024-09-14T09:05:34.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.852822+0000", 2024-09-14T09:05:34.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000157923 2024-09-14T09:05:34.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.852837+0000", 2024-09-14T09:05:34.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.49e-05 2024-09-14T09:05:34.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.853377+0000", 2024-09-14T09:05:34.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00053967700000000002 2024-09-14T09:05:34.814 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.814 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.814 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.814 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "MOSDPGRecoveryDelete(3.18 e29,28 3:1fe08b02:::rbd_data.115e724e1d99.000000000000001c:head,27'6)", 2024-09-14T09:05:34.814 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.854599+0000", 2024-09-14T09:05:34.814 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.94205050300000004, 2024-09-14T09:05:34.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022691700000000001, 2024-09-14T09:05:34.815 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.815 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.815 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:34.815 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.854599+0000", 2024-09-14T09:05:34.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.854599+0000", 2024-09-14T09:05:34.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.854599+0000", 2024-09-14T09:05:34.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4399999999999998e-07 2024-09-14T09:05:34.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.854606+0000", 2024-09-14T09:05:34.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2550000000000003e-06 2024-09-14T09:05:34.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.854620+0000", 2024-09-14T09:05:34.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4019000000000001e-05 2024-09-14T09:05:34.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.854628+0000", 2024-09-14T09:05:34.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9680000000000006e-06 2024-09-14T09:05:34.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.854713+0000", 2024-09-14T09:05:34.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4908999999999996e-05 2024-09-14T09:05:34.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.854726+0000", 2024-09-14T09:05:34.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.341e-05 2024-09-14T09:05:34.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.854825+0000", 2024-09-14T09:05:34.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9412000000000003e-05 2024-09-14T09:05:34.818 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.818 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.818 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.818 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "MOSDPGRecoveryDelete(3.18 e29,28 3:1d316c6e:::rbd_data.115e724e1d99.0000000000000021:head,27'12)", 2024-09-14T09:05:34.818 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.856335+0000", 2024-09-14T09:05:34.819 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.940313968, 2024-09-14T09:05:34.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023024100000000001, 2024-09-14T09:05:34.819 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.819 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.819 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:34.819 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.856335+0000", 2024-09-14T09:05:34.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.856335+0000", 2024-09-14T09:05:34.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.856336+0000", 2024-09-14T09:05:34.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7100000000000001e-07 2024-09-14T09:05:34.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.856342+0000", 2024-09-14T09:05:34.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0270000000000001e-06 2024-09-14T09:05:34.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.856356+0000", 2024-09-14T09:05:34.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4032999999999999e-05 2024-09-14T09:05:34.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.856364+0000", 2024-09-14T09:05:34.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7300000000000005e-06 2024-09-14T09:05:34.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.856438+0000", 2024-09-14T09:05:34.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4244000000000003e-05 2024-09-14T09:05:34.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.856453+0000", 2024-09-14T09:05:34.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5370999999999999e-05 2024-09-14T09:05:34.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.856565+0000", 2024-09-14T09:05:34.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011206500000000001 2024-09-14T09:05:34.822 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.822 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.822 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.823 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "MOSDPGPush(3.18 29/28 [PushOp(3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:head, version: 27'13, data_included: [3092465~50170], data_size: 50170, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:head@27'13, size: 3142635, copy_subset: [(0, 3142635)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3142635, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T09:05:34.823 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.859721+0000", 2024-09-14T09:05:34.823 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.93692777199999999, 2024-09-14T09:05:34.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00093989599999999996, 2024-09-14T09:05:34.823 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.823 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.823 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:34.823 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.859721+0000", 2024-09-14T09:05:34.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.859721+0000", 2024-09-14T09:05:34.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.859722+0000", 2024-09-14T09:05:34.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.063e-06 2024-09-14T09:05:34.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.859764+0000", 2024-09-14T09:05:34.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1782000000000001e-05 2024-09-14T09:05:34.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.859791+0000", 2024-09-14T09:05:34.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6599000000000001e-05 2024-09-14T09:05:34.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.859810+0000", 2024-09-14T09:05:34.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9153000000000001e-05 2024-09-14T09:05:34.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.859926+0000", 2024-09-14T09:05:34.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115775 2024-09-14T09:05:34.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.859939+0000", 2024-09-14T09:05:34.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3551e-05 2024-09-14T09:05:34.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.860661+0000", 2024-09-14T09:05:34.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00072197299999999999 2024-09-14T09:05:34.826 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.826 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.827 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.827 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "MOSDPGPush(3.18 29/28 [PushOp(3:188a8141:::rbd_data.115e724e1d99.000000000000001e:head, version: 27'14, data_included: [2342257~10426], data_size: 10426, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:188a8141:::rbd_data.115e724e1d99.000000000000001e:head@27'14, size: 2352683, copy_subset: [(0, 2352683)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2352683, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T09:05:34.827 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.863142+0000", 2024-09-14T09:05:34.827 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.93350716700000003, 2024-09-14T09:05:34.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000954524, 2024-09-14T09:05:34.827 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.827 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.827 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:34.827 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.863142+0000", 2024-09-14T09:05:34.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.863142+0000", 2024-09-14T09:05:34.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.863143+0000", 2024-09-14T09:05:34.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0999999999999997e-07 2024-09-14T09:05:34.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.863182+0000", 2024-09-14T09:05:34.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8933000000000001e-05 2024-09-14T09:05:34.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.863207+0000", 2024-09-14T09:05:34.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4830000000000001e-05 2024-09-14T09:05:34.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.863222+0000", 2024-09-14T09:05:34.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5753999999999999e-05 2024-09-14T09:05:34.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.863321+0000", 2024-09-14T09:05:34.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8895999999999996e-05 2024-09-14T09:05:34.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.863331+0000", 2024-09-14T09:05:34.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0135999999999999e-05 2024-09-14T09:05:34.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.864096+0000", 2024-09-14T09:05:34.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00076506499999999999 2024-09-14T09:05:34.831 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.831 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.831 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.831 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:657 3.6 e29/23 3:65b8693c:::rbd_data.115e1e95bf99.0000000000000002:head v 29'2, mlcod=0'0)", 2024-09-14T09:05:34.831 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.872567+0000", 2024-09-14T09:05:34.831 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.92408206100000001, 2024-09-14T09:05:34.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0056247060000000002, 2024-09-14T09:05:34.831 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.831 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.832 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.832 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872567+0000", 2024-09-14T09:05:34.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872567+0000", 2024-09-14T09:05:34.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872568+0000", 2024-09-14T09:05:34.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T09:05:34.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872772+0000", 2024-09-14T09:05:34.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020363799999999999 2024-09-14T09:05:34.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872799+0000", 2024-09-14T09:05:34.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6917e-05 2024-09-14T09:05:34.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872815+0000", 2024-09-14T09:05:34.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5812999999999999e-05 2024-09-14T09:05:34.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872928+0000", 2024-09-14T09:05:34.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011381499999999999 2024-09-14T09:05:34.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872957+0000", 2024-09-14T09:05:34.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9118000000000001e-05 2024-09-14T09:05:34.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.878076+0000", 2024-09-14T09:05:34.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0051180690000000003 2024-09-14T09:05:34.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.878192+0000", 2024-09-14T09:05:34.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116123 2024-09-14T09:05:34.835 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.835 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.835 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.836 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "pg_update_log_missing(3.6 epoch 29/23 rep_tid 23 entries 29'3 (0'0) error 3:65b8693c:::rbd_data.115e1e95bf99.0000000000000002:head by client.4446.0:673 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'1)", 2024-09-14T09:05:34.836 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.888034+0000", 2024-09-14T09:05:34.836 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.90861463499999995, 2024-09-14T09:05:34.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00096039900000000002, 2024-09-14T09:05:34.836 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.836 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.836 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:34.836 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.888034+0000", 2024-09-14T09:05:34.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.888034+0000", 2024-09-14T09:05:34.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.888036+0000", 2024-09-14T09:05:34.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.232e-06 2024-09-14T09:05:34.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.888049+0000", 2024-09-14T09:05:34.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3399e-05 2024-09-14T09:05:34.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.888076+0000", 2024-09-14T09:05:34.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6970000000000001e-05 2024-09-14T09:05:34.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.888100+0000", 2024-09-14T09:05:34.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4138000000000002e-05 2024-09-14T09:05:34.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.888269+0000", 2024-09-14T09:05:34.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000168531 2024-09-14T09:05:34.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.888995+0000", 2024-09-14T09:05:34.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00072612900000000005 2024-09-14T09:05:34.839 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.839 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.839 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.839 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:844 3.6 e29/23 3:63058ec8:::rbd_data.115e1e95bf99.0000000000000006:head v 29'4, mlcod=29'2)", 2024-09-14T09:05:34.839 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.921358+0000", 2024-09-14T09:05:34.839 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.87529117199999995, 2024-09-14T09:05:34.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0014616710000000001, 2024-09-14T09:05:34.839 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.840 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.840 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.840 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.921358+0000", 2024-09-14T09:05:34.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.921358+0000", 2024-09-14T09:05:34.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.921359+0000", 2024-09-14T09:05:34.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3850000000000001e-06 2024-09-14T09:05:34.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.921393+0000", 2024-09-14T09:05:34.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4245000000000002e-05 2024-09-14T09:05:34.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.921422+0000", 2024-09-14T09:05:34.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8062999999999999e-05 2024-09-14T09:05:34.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.921434+0000", 2024-09-14T09:05:34.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2884000000000001e-05 2024-09-14T09:05:34.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.921526+0000", 2024-09-14T09:05:34.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1370000000000001e-05 2024-09-14T09:05:34.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.921556+0000", 2024-09-14T09:05:34.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9833999999999999e-05 2024-09-14T09:05:34.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.922762+0000", 2024-09-14T09:05:34.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001206015 2024-09-14T09:05:34.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.922820+0000", 2024-09-14T09:05:34.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7874999999999997e-05 2024-09-14T09:05:34.843 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.843 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.844 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.844 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:848 3.1a e29/23 3:5ade8e73:::rbd_data.115e1e95bf99.000000000000000a:head v 29'3, mlcod=25'1)", 2024-09-14T09:05:34.844 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.923872+0000", 2024-09-14T09:05:34.844 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.872776942, 2024-09-14T09:05:34.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00146058, 2024-09-14T09:05:34.844 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.844 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.844 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.844 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.923872+0000", 2024-09-14T09:05:34.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.923872+0000", 2024-09-14T09:05:34.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.923872+0000", 2024-09-14T09:05:34.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.41e-07 2024-09-14T09:05:34.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.924099+0000", 2024-09-14T09:05:34.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000226216 2024-09-14T09:05:34.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.924115+0000", 2024-09-14T09:05:34.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6589999999999999e-05 2024-09-14T09:05:34.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.924125+0000", 2024-09-14T09:05:34.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5310000000000007e-06 2024-09-14T09:05:34.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.924197+0000", 2024-09-14T09:05:34.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2513000000000002e-05 2024-09-14T09:05:34.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.924220+0000", 2024-09-14T09:05:34.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2755000000000002e-05 2024-09-14T09:05:34.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.925276+0000", 2024-09-14T09:05:34.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001056256 2024-09-14T09:05:34.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.925333+0000", 2024-09-14T09:05:34.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6477999999999997e-05 2024-09-14T09:05:34.848 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.848 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.848 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.848 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "pg_update_log_missing(3.6 epoch 29/23 rep_tid 25 entries 29'5 (0'0) error 3:65b8693c:::rbd_data.115e1e95bf99.0000000000000002:head by client.4446.0:864 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 29'3)", 2024-09-14T09:05:34.848 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.951223+0000", 2024-09-14T09:05:34.848 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.84542564799999997, 2024-09-14T09:05:34.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001657197, 2024-09-14T09:05:34.848 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.849 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.849 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:34.849 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951223+0000", 2024-09-14T09:05:34.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951223+0000", 2024-09-14T09:05:34.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951225+0000", 2024-09-14T09:05:34.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7489999999999999e-06 2024-09-14T09:05:34.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951241+0000", 2024-09-14T09:05:34.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6308e-05 2024-09-14T09:05:34.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951281+0000", 2024-09-14T09:05:34.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9524000000000003e-05 2024-09-14T09:05:34.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951321+0000", 2024-09-14T09:05:34.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0510999999999999e-05 2024-09-14T09:05:34.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951458+0000", 2024-09-14T09:05:34.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013608399999999999 2024-09-14T09:05:34.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.952881+0000", 2024-09-14T09:05:34.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001423021 2024-09-14T09:05:34.852 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.852 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.852 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.852 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "pg_update_log_missing(3.6 epoch 29/23 rep_tid 26 entries 29'6 (0'0) error 3:63058ec8:::rbd_data.115e1e95bf99.0000000000000006:head by client.4446.0:868 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 29'3)", 2024-09-14T09:05:34.852 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.951691+0000", 2024-09-14T09:05:34.852 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.84495785099999998, 2024-09-14T09:05:34.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001234391, 2024-09-14T09:05:34.852 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.852 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.852 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:34.852 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951691+0000", 2024-09-14T09:05:34.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951691+0000", 2024-09-14T09:05:34.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951692+0000", 2024-09-14T09:05:34.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0279999999999999e-06 2024-09-14T09:05:34.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951702+0000", 2024-09-14T09:05:34.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0125e-05 2024-09-14T09:05:34.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951731+0000", 2024-09-14T09:05:34.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8804999999999999e-05 2024-09-14T09:05:34.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951748+0000", 2024-09-14T09:05:34.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6957e-05 2024-09-14T09:05:34.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951838+0000", 2024-09-14T09:05:34.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9723999999999997e-05 2024-09-14T09:05:34.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.952926+0000", 2024-09-14T09:05:34.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001087752 2024-09-14T09:05:34.855 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.855 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.856 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.856 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "pg_update_log_missing(3.1a epoch 29/23 rep_tid 121 entries 29'4 (0'0) error 3:5ade8e73:::rbd_data.115e1e95bf99.000000000000000a:head by client.4446.0:872 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 26'2)", 2024-09-14T09:05:34.856 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.951926+0000", 2024-09-14T09:05:34.856 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.84472316000000003, 2024-09-14T09:05:34.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00098779800000000002, 2024-09-14T09:05:34.856 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.856 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.856 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:34.856 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951926+0000", 2024-09-14T09:05:34.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951926+0000", 2024-09-14T09:05:34.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951927+0000", 2024-09-14T09:05:34.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4099999999999997e-07 2024-09-14T09:05:34.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951937+0000", 2024-09-14T09:05:34.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0540000000000001e-05 2024-09-14T09:05:34.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951959+0000", 2024-09-14T09:05:34.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2106000000000001e-05 2024-09-14T09:05:34.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.951977+0000", 2024-09-14T09:05:34.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7710000000000002e-05 2024-09-14T09:05:34.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.952089+0000", 2024-09-14T09:05:34.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112135 2024-09-14T09:05:34.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.952914+0000", 2024-09-14T09:05:34.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00082436599999999999 2024-09-14T09:05:34.859 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.859 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.859 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.860 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:908 3.6 e29/23 3:63058ec8:::rbd_data.115e1e95bf99.0000000000000006:head v 29'7, mlcod=29'5)", 2024-09-14T09:05:34.860 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:34.005837+0000", 2024-09-14T09:05:34.860 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.79081154499999995, 2024-09-14T09:05:34.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001008955, 2024-09-14T09:05:34.860 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:34.860 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:34.860 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:34.860 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:34.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:34.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.005837+0000", 2024-09-14T09:05:34.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:34.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.005837+0000", 2024-09-14T09:05:34.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:34.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:34.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.005839+0000", 2024-09-14T09:05:34.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9e-06 2024-09-14T09:05:34.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:34.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.005856+0000", 2024-09-14T09:05:34.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6574000000000001e-05 2024-09-14T09:05:34.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:34.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.005876+0000", 2024-09-14T09:05:34.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0115e-05 2024-09-14T09:05:34.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:34.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.005886+0000", 2024-09-14T09:05:34.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9820000000000003e-06 2024-09-14T09:05:34.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:34.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.005962+0000", 2024-09-14T09:05:34.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6095999999999998e-05 2024-09-14T09:05:34.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:34.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.005988+0000", 2024-09-14T09:05:34.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5707999999999998e-05 2024-09-14T09:05:34.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:34.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.006813+0000", 2024-09-14T09:05:34.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000825331 2024-09-14T09:05:34.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:34.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:34.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:34.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.006846+0000", 2024-09-14T09:05:34.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3249e-05 2024-09-14T09:05:34.864 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.864 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.864 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.864 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:34.864 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:34.864 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:34.865 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:05:34.865 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is unset 2024-09-14T09:05:34.880 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:34.876+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:34.974 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:34.972+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:35.001 INFO:teuthology.orchestra.run.smithi176.stdout:leaving image image_client.0-clone1 intact 2024-09-14T09:05:35.012 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:05:35.059 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:35.059 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:35.059 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:35.060 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:35.070 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:35.068+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:35.077 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:05:35.170 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:35.168+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:35.175 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:05:35.175 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:05:35.192 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:05:35.193 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:05:35.211 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:05:35.211 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:05:35.228 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:05:35.229 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:05:35.250 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:05:35.250 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:05:35.251 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-14T09:05:35.253 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:35.253 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:35.270 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:05:35.271 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:05:35.271 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:05:35.272 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:35.268+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:35.292 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:05:35.293 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:05:35.314 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:05:35.314 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:05:35.382 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:35.380+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:35.394 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:35.394 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:35.410 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:05:35.502 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:35.500+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:35.574 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:35.574 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:35.574 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:35.575+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:35.588 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:05:35.651 INFO:teuthology.orchestra.run.smithi176.stdout:242 write 0xa6d201f thru 0xa6d2ff0 (0xfd2 bytes) 2024-09-14T09:05:35.665 INFO:teuthology.orchestra.run.smithi176.stdout:243 clone 3 order 25 su 2097152 sc 14 2024-09-14T09:05:35.675 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:35.672+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:35.750 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:35.750 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:35.764 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:05:35.776 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:35.775+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:35.877 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:35.879+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:35.930 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:35.930 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:35.945 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:05:35.989 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:35.991+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:36.000 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone2 from 0xe848ff5 (overlap 0xe848ff5) to 0xd10e7f6 2024-09-14T09:05:36.079 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:36.079+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:36.101 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:36.102 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:36.116 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:05:36.204 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:36.207+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:36.281 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:36.281 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:36.282 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:36.283+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:36.297 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:05:36.321 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-14T09:05:36.381 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:36.376+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:36.457 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:36.457 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:36.471 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:05:36.517 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:36.519+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:36.582 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:36.580+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:36.594 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:36.594 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:36.608 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:05:36.683 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:36.680+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:36.750 INFO:teuthology.orchestra.run.smithi176.stdout:247 write 0xeb8510a thru 0xeb8a55a (0x5451 bytes) 2024-09-14T09:05:36.784 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:36.783+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:36.788 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:36.788 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:36.797 INFO:teuthology.orchestra.run.smithi176.stdout:248 read 0x54c1602 thru 0x54c5a32 (0x4431 bytes) 2024-09-14T09:05:36.804 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:05:36.820 INFO:teuthology.orchestra.run.smithi176.stdout:249 trunc from 0xeb8a55b to 0x232b664 2024-09-14T09:05:36.881 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T09:05:36.893 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:36.895+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:36.985 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:36.980+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:37.038 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:37.038 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:37.058 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:05:37.086 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:37.080+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:37.145 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:37.145 INFO:teuthology.orchestra.run.smithi176.stdout:252 read 0x21f769f thru 0x21fc82e (0x5190 bytes) 2024-09-14T09:05:37.146 INFO:teuthology.orchestra.run.smithi176.stdout:253 read 0x2ce933 thru 0x2d9166 (0xa834 bytes) 2024-09-14T09:05:37.170 INFO:teuthology.orchestra.run.smithi176.stdout:254 write 0xd756c76 thru 0xd75f9bb (0x8d46 bytes) 2024-09-14T09:05:37.171 INFO:teuthology.orchestra.run.smithi176.stdout:255 read 0xac4ece7 thru 0xac53b24 (0x4e3e bytes) 2024-09-14T09:05:37.172 INFO:teuthology.orchestra.run.smithi176.stdout:257 read 0xaebc80b thru 0xaec4f51 (0x8747 bytes) 2024-09-14T09:05:37.172 INFO:teuthology.orchestra.run.smithi176.stdout:259 writesame 0x7225182 thru 0x72297e4 data_size 0x1(0x4663 bytes) 2024-09-14T09:05:37.187 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:37.187+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:37.199 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:37.199 INFO:teuthology.orchestra.run.smithi176.stdout:262 writesame 0x6588af5 thru 0x6588b99 data_size 0x1(0xa5 bytes) 2024-09-14T09:05:37.201 INFO:teuthology.orchestra.run.smithi176.stdout:264 writesame 0x14817df thru 0x148602e data_size 0x59(0x4850 bytes) 2024-09-14T09:05:37.205 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:37.205 INFO:teuthology.orchestra.run.smithi176.stdout:267 write 0x90847f7 thru 0x908d335 (0x8b3f bytes) 2024-09-14T09:05:37.210 INFO:teuthology.orchestra.run.smithi176.stdout:268 read 0xb774823 thru 0xb77f6af (0xae8d bytes) 2024-09-14T09:05:37.210 INFO:teuthology.orchestra.run.smithi176.stdout:269 writesame 0x2b74be2 thru 0x2b7805a data_size 0x7(0x3479 bytes) 2024-09-14T09:05:37.220 INFO:teuthology.orchestra.run.smithi176.stdout:270 punch from 0x4e4d334 to 0x4e51cb4, (0x4980 bytes) 2024-09-14T09:05:37.220 INFO:teuthology.orchestra.run.smithi176.stdout:271 write 0x592f37b thru 0x5932cc1 (0x3947 bytes) 2024-09-14T09:05:37.225 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:37.225 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:37.231 INFO:teuthology.orchestra.run.smithi176.stdout:272 trunc from 0xd75f9bc to 0x967a0fb 2024-09-14T09:05:37.239 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:05:37.255 INFO:teuthology.orchestra.run.smithi176.stdout:273 writesame 0x96844ae thru 0x9684ee1 data_size 0x2(0xa34 bytes) 2024-09-14T09:05:37.256 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:37.257 INFO:teuthology.orchestra.run.smithi176.stdout:276 write 0x6f6392b thru 0x6f71217 (0xd8ed bytes) 2024-09-14T09:05:37.272 INFO:teuthology.orchestra.run.smithi176.stdout:278 write 0xaf33dcd thru 0xaf33f24 (0x158 bytes) 2024-09-14T09:05:37.274 INFO:teuthology.orchestra.run.smithi176.stdout:279 write 0x7aa0c22 thru 0x7aa59f9 (0x4dd8 bytes) 2024-09-14T09:05:37.278 INFO:teuthology.orchestra.run.smithi176.stdout:280 writesame 0x619cc0e thru 0x61a5da1 data_size 0x2c(0x9194 bytes) 2024-09-14T09:05:37.282 INFO:teuthology.orchestra.run.smithi176.stdout:283 write 0x5e95eaf thru 0x5e98781 (0x28d3 bytes) 2024-09-14T09:05:37.285 INFO:teuthology.orchestra.run.smithi176.stdout:284 trunc from 0xaf33f25 to 0xb0c5978 2024-09-14T09:05:37.287 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:37.284+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:37.292 INFO:teuthology.orchestra.run.smithi176.stdout:287 writesame 0x3611987 thru 0x3615f31 data_size 0xf(0x45ab bytes) 2024-09-14T09:05:37.295 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:37.305 INFO:teuthology.orchestra.run.smithi176.stdout:290 write 0xd9bf749 thru 0xd9c16ed (0x1fa5 bytes) 2024-09-14T09:05:37.307 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:37.308 INFO:teuthology.orchestra.run.smithi176.stdout:292 write 0x77b1279 thru 0x77ba488 (0x9210 bytes) 2024-09-14T09:05:37.311 INFO:teuthology.orchestra.run.smithi176.stdout:293 read 0x3ff5350 thru 0x3ffac64 (0x5915 bytes) 2024-09-14T09:05:37.311 INFO:teuthology.orchestra.run.smithi176.stdout:294 write 0x84d3b70 thru 0x84d9622 (0x5ab3 bytes) 2024-09-14T09:05:37.316 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:37.316 INFO:teuthology.orchestra.run.smithi176.stdout:296 writesame 0x8759953 thru 0x875f711 data_size 0x1(0x5dbf bytes) 2024-09-14T09:05:37.337 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:05:37.372 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:37.372 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:37.386 INFO:teuthology.orchestra.run.smithi176.stdout:299 write 0xe6df321 thru 0xe6ed5aa (0xe28a bytes) 2024-09-14T09:05:37.388 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:37.391+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:37.389 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:37.396 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:37.396 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:37.406 INFO:teuthology.orchestra.run.smithi176.stdout:303 trunc from 0xe6ed5ab to 0x4b7d298 2024-09-14T09:05:37.411 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:05:37.420 INFO:teuthology.orchestra.run.smithi176.stdout:305 read 0x3c881f7 thru 0x3c8cfb5 (0x4dbf bytes) 2024-09-14T09:05:37.420 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:37.439 INFO:teuthology.orchestra.run.smithi176.stdout:309 writesame 0xba0921f thru 0xba167b1 data_size 0x87(0xd593 bytes) 2024-09-14T09:05:37.441 INFO:teuthology.orchestra.run.smithi176.stdout:310 punch from 0xa5e5939 to 0xa5f0df1, (0xb4b8 bytes) 2024-09-14T09:05:37.441 INFO:teuthology.orchestra.run.smithi176.stdout:311 write 0x9eeea08 thru 0x9efb1e1 (0xc7da bytes) 2024-09-14T09:05:37.449 INFO:teuthology.orchestra.run.smithi176.stdout:312 write 0x988a342 thru 0x988c5e2 (0x22a1 bytes) 2024-09-14T09:05:37.452 INFO:teuthology.orchestra.run.smithi176.stdout:313 writesame 0x89d147c thru 0x89d77df data_size 0x2(0x6364 bytes) 2024-09-14T09:05:37.460 INFO:teuthology.orchestra.run.smithi176.stdout:315 write 0x9db4569 thru 0x9db9976 (0x540e bytes) 2024-09-14T09:05:37.464 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:37.464 INFO:teuthology.orchestra.run.smithi176.stdout:317 writesame 0x4cda822 thru 0x4ce55be data_size 0x5(0xad9d bytes) 2024-09-14T09:05:37.496 INFO:teuthology.orchestra.run.smithi176.stdout:319 write 0x6d4b1d1 thru 0x6d4f83f (0x466f bytes) 2024-09-14T09:05:37.498 INFO:teuthology.orchestra.run.smithi176.stdout:320 read 0x6e6be56 thru 0x6e70c09 (0x4db4 bytes) 2024-09-14T09:05:37.499 INFO:teuthology.orchestra.run.smithi176.stdout:323 punch from 0x8747af9 to 0x874dc07, (0x610e bytes) 2024-09-14T09:05:37.501 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:37.503+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:37.501 INFO:teuthology.orchestra.run.smithi176.stdout:324 trunc from 0xba167b2 to 0x9d3de96 2024-09-14T09:05:37.509 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:05:37.509 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:05:37.514 INFO:teuthology.orchestra.run.smithi176.stdout:325 read 0x5eb4e4c thru 0x5ec0e65 (0xc01a bytes) 2024-09-14T09:05:37.514 INFO:teuthology.orchestra.run.smithi176.stdout:326 read 0x64b2dab thru 0x64b8b16 (0x5d6c bytes) 2024-09-14T09:05:37.515 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:37.515 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:37.515 INFO:teuthology.orchestra.run.smithi176.stdout:331 read 0x726b019 thru 0x7270196 (0x517e bytes) 2024-09-14T09:05:37.515 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:37.515 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:37.516 INFO:teuthology.orchestra.run.smithi176.stdout:335 writesame 0x23e4362 thru 0x23eff2b data_size 0x2b(0xbbca bytes) 2024-09-14T09:05:37.518 INFO:teuthology.orchestra.run.smithi176.stdout:336 write 0x86a63d1 thru 0x86acb10 (0x6740 bytes) 2024-09-14T09:05:37.522 INFO:teuthology.orchestra.run.smithi176.stdout:truncating to largest ever: 0xe97bb3c 2024-09-14T09:05:37.526 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:05:37.526 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:05:37.528 INFO:teuthology.orchestra.run.smithi176.stdout:337 trunc from 0x9d3de96 to 0xe97bb3c 2024-09-14T09:05:37.535 INFO:teuthology.orchestra.run.smithi176.stdout:338 read 0xccb178e thru 0xccb603b (0x48ae bytes) 2024-09-14T09:05:37.544 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:05:37.544 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:05:37.548 INFO:teuthology.orchestra.run.smithi176.stdout:339 trunc from 0xe97bb3c to 0x4d21573 2024-09-14T09:05:37.567 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:05:37.568 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:05:37.572 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:37.572 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:37.578 INFO:teuthology.orchestra.run.smithi176.stdout:340 write 0x8bd3535 thru 0x8bd51b4 (0x1c80 bytes) 2024-09-14T09:05:37.582 INFO:teuthology.orchestra.run.smithi176.stdout:341 trunc from 0x8bd51b5 to 0x94d9316 2024-09-14T09:05:37.587 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:05:37.589 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:05:37.589 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:05:37.590 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:37.588+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:37.594 INFO:teuthology.orchestra.run.smithi176.stdout:342 punch from 0x29d7d5b to 0x29d9786, (0x1a2b bytes) 2024-09-14T09:05:37.594 INFO:teuthology.orchestra.run.smithi176.stdout:344 writesame 0x50dd3da thru 0x50e3dcc data_size 0x1(0x69f3 bytes) 2024-09-14T09:05:37.609 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:05:37.610 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:05:37.627 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:05:37.627 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:05:37.636 INFO:teuthology.orchestra.run.smithi176.stdout:346 punch from 0x8f9f41f to 0x8fa61d1, (0x6db2 bytes) 2024-09-14T09:05:37.637 INFO:teuthology.orchestra.run.smithi176.stdout:348 write 0x360c2d7 thru 0x361312c (0x6e56 bytes) 2024-09-14T09:05:37.642 INFO:teuthology.orchestra.run.smithi176.stdout:349 write 0x897d7a5 thru 0x89817da (0x4036 bytes) 2024-09-14T09:05:37.645 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:05:37.645 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:05:37.661 INFO:teuthology.orchestra.run.smithi176.stdout:351 trunc from 0x94d9316 to 0x1523ea3 2024-09-14T09:05:37.679 INFO:teuthology.orchestra.run.smithi176.stdout:352 read 0x6961e8 thru 0x69f37a (0x9193 bytes) 2024-09-14T09:05:37.680 INFO:teuthology.orchestra.run.smithi176.stdout:354 read 0x625590 thru 0x62768d (0x20fe bytes) 2024-09-14T09:05:37.691 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:37.691+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:37.707 INFO:teuthology.orchestra.run.smithi176.stdout:355 write 0xedae367 thru 0xedbd3d9 (0xf073 bytes) 2024-09-14T09:05:37.710 INFO:teuthology.orchestra.run.smithi176.stdout:358 punch from 0xd641a9e to 0xd641b0a, (0x6c bytes) 2024-09-14T09:05:37.710 INFO:teuthology.orchestra.run.smithi176.stdout:359 punch from 0x43e9c57 to 0x43f9764, (0xfb0d bytes) 2024-09-14T09:05:37.710 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:37.710 INFO:teuthology.orchestra.run.smithi176.stdout:361 writesame 0x39dc6d2 thru 0x39e2c6a data_size 0x1(0x6599 bytes) 2024-09-14T09:05:37.755 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:05:37.756 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:05:37.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:37.756 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:10026 3.f 3:f6f4e415:::rbd_data.115eadec874.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:37.756 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.713901+0000", 2024-09-14T09:05:37.756 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.044733987000000003, 2024-09-14T09:05:37.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.044786393000000001, 2024-09-14T09:05:37.756 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:37.756 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:37.756 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for sub ops", 2024-09-14T09:05:37.756 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:37.756 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:37.756 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:37.757 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 10026 2024-09-14T09:05:37.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:37.757 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:37.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:37.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:37.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713901+0000", 2024-09-14T09:05:37.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:37.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:37.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:37.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:37.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713901+0000", 2024-09-14T09:05:37.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:37.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:37.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:37.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:37.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713903+0000", 2024-09-14T09:05:37.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4750000000000001e-06 2024-09-14T09:05:37.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:37.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:37.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:37.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713908+0000", 2024-09-14T09:05:37.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.242e-06 2024-09-14T09:05:37.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:37.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:37.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:37.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713920+0000", 2024-09-14T09:05:37.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2456000000000001e-05 2024-09-14T09:05:37.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:37.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:37.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:37.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713927+0000", 2024-09-14T09:05:37.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0849999999999999e-06 2024-09-14T09:05:37.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:37.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:37.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:37.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713973+0000", 2024-09-14T09:05:37.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5179000000000002e-05 2024-09-14T09:05:37.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:37.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:37.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:37.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.714023+0000", 2024-09-14T09:05:37.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0469000000000001e-05 2024-09-14T09:05:37.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:37.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:37.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:37.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.715791+0000", 2024-09-14T09:05:37.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017676580000000001 2024-09-14T09:05:37.760 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:37.760 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:37.760 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:37.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:37.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:37.760 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:10026 3.f e32/23)", 2024-09-14T09:05:37.760 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.722702+0000", 2024-09-14T09:05:37.760 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.035933121999999998, 2024-09-14T09:05:37.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.036094663999999999, 2024-09-14T09:05:37.760 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:37.760 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:37.760 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:37.760 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:37.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:37.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:37.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.722702+0000", 2024-09-14T09:05:37.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:37.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:37.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:37.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:37.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.722702+0000", 2024-09-14T09:05:37.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:37.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:37.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:37.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:37.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.722703+0000", 2024-09-14T09:05:37.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.046e-06 2024-09-14T09:05:37.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:37.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:37.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:37.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.722708+0000", 2024-09-14T09:05:37.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2429999999999996e-06 2024-09-14T09:05:37.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:37.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:37.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:37.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.722725+0000", 2024-09-14T09:05:37.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6748e-05 2024-09-14T09:05:37.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:37.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:37.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:37.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.722734+0000", 2024-09-14T09:05:37.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3589999999999997e-06 2024-09-14T09:05:37.763 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:37.763 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:37.763 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:37.763 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:37.763 INFO:teuthology.orchestra.run.smithi183.stdout: ], 2024-09-14T09:05:37.763 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 2 2024-09-14T09:05:37.763 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:05:37.764 INFO:teuthology.orchestra.run.smithi176.stdout:363 read 0x58808cb thru 0x588848c (0x7bc2 bytes) 2024-09-14T09:05:37.764 INFO:teuthology.orchestra.run.smithi176.stdout:365 punch from 0xab36f64 to 0xab379d4, (0xa70 bytes) 2024-09-14T09:05:37.764 INFO:teuthology.orchestra.run.smithi176.stdout:366 writesame 0x524c8e9 thru 0x5258a98 data_size 0x18(0xc1b0 bytes) 2024-09-14T09:05:37.764 INFO:teuthology.orchestra.run.smithi176.stdout:368 read 0x1eabae thru 0x1f7458 (0xc8ab bytes) 2024-09-14T09:05:37.764 INFO:teuthology.orchestra.run.smithi176.stdout:369 write 0xcfd97a0 thru 0xcfe678c (0xcfed bytes) 2024-09-14T09:05:37.770 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:05:37.773 INFO:teuthology.orchestra.run.smithi176.stdout:370 writesame 0xb877507 thru 0xb8805c5 data_size 0x1(0x90bf bytes) 2024-09-14T09:05:37.813 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:37.815+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:37.863 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:05:37.863 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:05:37.864 INFO:tasks.thrashosds.thrasher:Reweighting osd 4 to 0.15818148871551563 2024-09-14T09:05:37.864 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 0.15818148871551563 2024-09-14T09:05:37.884 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is set 2024-09-14T09:05:37.893 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:37.888+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:37.900 INFO:teuthology.orchestra.run.smithi176.stdout:373 write 0x77998a9 thru 0x779f86e (0x5fc6 bytes) 2024-09-14T09:05:37.905 INFO:teuthology.orchestra.run.smithi176.stdout:374 read 0xc4dfe68 thru 0xc4e7959 (0x7af2 bytes) 2024-09-14T09:05:37.905 INFO:teuthology.orchestra.run.smithi176.stdout:375 punch from 0xd3d6a60 to 0xd3e0bef, (0xa18f bytes) 2024-09-14T09:05:37.905 INFO:teuthology.orchestra.run.smithi176.stdout:376 write 0x78aed29 thru 0x78b49ee (0x5cc6 bytes) 2024-09-14T09:05:37.908 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:37.908 INFO:teuthology.orchestra.run.smithi176.stdout:378 write 0xb5cd17c thru 0xb5d0169 (0x2fee bytes) 2024-09-14T09:05:37.912 INFO:teuthology.orchestra.run.smithi176.stdout:379 read 0x767534c thru 0x76763f7 (0x10ac bytes) 2024-09-14T09:05:37.912 INFO:teuthology.orchestra.run.smithi176.stdout:380 write 0x5757c51 thru 0x57669a9 (0xed59 bytes) 2024-09-14T09:05:37.934 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:05:37.934 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:05:37.934 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:05:37.934 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:05:37.934 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:05:37.946 INFO:teuthology.orchestra.run.smithi176.stdout:381 trunc from 0xedbd3da to 0x838a43 2024-09-14T09:05:37.947 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:05:37.979 INFO:teuthology.orchestra.run.smithi176.stdout:382 read 0x57847e thru 0x581591 (0x9114 bytes) 2024-09-14T09:05:37.994 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:37.995+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:37.995 INFO:teuthology.orchestra.run.smithi176.stdout:384 write 0x4e98919 thru 0x4ea505e (0xc746 bytes) 2024-09-14T09:05:38.013 INFO:teuthology.orchestra.run.smithi176.stdout:385 trunc from 0x4ea505f to 0xce6a76e 2024-09-14T09:05:38.020 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:38.020 INFO:teuthology.orchestra.run.smithi176.stdout:388 read 0x264e36 thru 0x270aba (0xbc85 bytes) 2024-09-14T09:05:38.021 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:38.031 INFO:teuthology.orchestra.run.smithi176.stdout:392 trunc from 0xce6a76e to 0x6779a69 2024-09-14T09:05:38.041 INFO:teuthology.orchestra.run.smithi176.stdout:393 read 0x223e883 thru 0x2240d6d (0x24eb bytes) 2024-09-14T09:05:38.046 INFO:teuthology.orchestra.run.smithi176.stdout:397 trunc from 0x6779a69 to 0x9bf85e6 2024-09-14T09:05:38.055 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:38.055 INFO:teuthology.orchestra.run.smithi176.stdout:399 write 0x7238135 thru 0x7243e99 (0xbd65 bytes) 2024-09-14T09:05:38.062 INFO:teuthology.orchestra.run.smithi176.stdout:400 punch from 0x57c4b3f to 0x57ce5d9, (0x9a9a bytes) 2024-09-14T09:05:38.062 INFO:teuthology.orchestra.run.smithi176.stdout:401 writesame 0x3947e5b thru 0x3954dc3 data_size 0xb(0xcf69 bytes) 2024-09-14T09:05:38.075 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:05:38.076 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:05:38.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:05:38.076 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:05:38.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.076 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:38.076 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379206+0000", 2024-09-14T09:05:38.076 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.697999900999999, 2024-09-14T09:05:38.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.022450290000000001, 2024-09-14T09:05:38.076 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.076 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.076 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.076 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:38.076 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:38.077 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:38.077 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 48 2024-09-14T09:05:38.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.077 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379206+0000", 2024-09-14T09:05:38.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379206+0000", 2024-09-14T09:05:38.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379207+0000", 2024-09-14T09:05:38.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.167e-06 2024-09-14T09:05:38.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379216+0000", 2024-09-14T09:05:38.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1630000000000002e-06 2024-09-14T09:05:38.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379226+0000", 2024-09-14T09:05:38.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9029999999999999e-06 2024-09-14T09:05:38.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.079 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.079 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379233+0000", 2024-09-14T09:05:38.079 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.17e-06 2024-09-14T09:05:38.079 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.079 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.079 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.079 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395986+0000", 2024-09-14T09:05:38.079 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016752195000000001 2024-09-14T09:05:38.079 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.079 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.079 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.079 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.396143+0000", 2024-09-14T09:05:38.079 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000157878 2024-09-14T09:05:38.079 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.079 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.080 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:38.080 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.396346+0000", 2024-09-14T09:05:38.080 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020270499999999999 2024-09-14T09:05:38.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.080 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.080 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.080 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401328+0000", 2024-09-14T09:05:38.080 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0049818379999999997 2024-09-14T09:05:38.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.080 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.080 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:38.080 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401367+0000", 2024-09-14T09:05:38.080 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8926e-05 2024-09-14T09:05:38.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.081 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.081 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:38.081 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401541+0000", 2024-09-14T09:05:38.081 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017380600000000001 2024-09-14T09:05:38.081 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.081 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.081 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.081 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401656+0000", 2024-09-14T09:05:38.081 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011553900000000001 2024-09-14T09:05:38.081 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.081 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.081 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.081 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.081 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.081 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:38.082 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379246+0000", 2024-09-14T09:05:38.082 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.697960118000001, 2024-09-14T09:05:38.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.023032166, 2024-09-14T09:05:38.082 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.082 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.082 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.082 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:38.082 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:38.082 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:38.082 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49 2024-09-14T09:05:38.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.082 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.082 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.082 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.082 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379246+0000", 2024-09-14T09:05:38.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379246+0000", 2024-09-14T09:05:38.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379247+0000", 2024-09-14T09:05:38.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7300000000000005e-07 2024-09-14T09:05:38.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379253+0000", 2024-09-14T09:05:38.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.28e-06 2024-09-14T09:05:38.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.084 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379267+0000", 2024-09-14T09:05:38.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4087000000000001e-05 2024-09-14T09:05:38.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.084 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379274+0000", 2024-09-14T09:05:38.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8929999999999999e-06 2024-09-14T09:05:38.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.397561+0000", 2024-09-14T09:05:38.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018286903 2024-09-14T09:05:38.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.397798+0000", 2024-09-14T09:05:38.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000236989 2024-09-14T09:05:38.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:38.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.398157+0000", 2024-09-14T09:05:38.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035888700000000001 2024-09-14T09:05:38.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.086 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.086 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401799+0000", 2024-09-14T09:05:38.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0036428139999999999 2024-09-14T09:05:38.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.086 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:38.086 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402029+0000", 2024-09-14T09:05:38.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022967800000000001 2024-09-14T09:05:38.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.086 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:38.086 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402165+0000", 2024-09-14T09:05:38.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013627600000000001 2024-09-14T09:05:38.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.087 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.087 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402278+0000", 2024-09-14T09:05:38.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011248599999999999 2024-09-14T09:05:38.087 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.087 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.087 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.087 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:38.087 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379312+0000", 2024-09-14T09:05:38.087 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.697893342, 2024-09-14T09:05:38.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.024727301, 2024-09-14T09:05:38.087 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.088 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.088 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.088 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:38.088 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:38.088 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:38.088 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 50 2024-09-14T09:05:38.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.088 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379312+0000", 2024-09-14T09:05:38.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379312+0000", 2024-09-14T09:05:38.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379314+0000", 2024-09-14T09:05:38.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.17e-06 2024-09-14T09:05:38.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379408+0000", 2024-09-14T09:05:38.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4562999999999996e-05 2024-09-14T09:05:38.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379429+0000", 2024-09-14T09:05:38.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1189e-05 2024-09-14T09:05:38.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379448+0000", 2024-09-14T09:05:38.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8734000000000001e-05 2024-09-14T09:05:38.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.399051+0000", 2024-09-14T09:05:38.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.019602441000000002 2024-09-14T09:05:38.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.399218+0000", 2024-09-14T09:05:38.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000167095 2024-09-14T09:05:38.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:38.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.399438+0000", 2024-09-14T09:05:38.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022046500000000001 2024-09-14T09:05:38.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401991+0000", 2024-09-14T09:05:38.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025529250000000002 2024-09-14T09:05:38.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:38.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.403780+0000", 2024-09-14T09:05:38.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017893939999999999 2024-09-14T09:05:38.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:38.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.403915+0000", 2024-09-14T09:05:38.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013478200000000001 2024-09-14T09:05:38.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.404040+0000", 2024-09-14T09:05:38.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012454299999999999 2024-09-14T09:05:38.092 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.093 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.093 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.093 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:38.093 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379481+0000", 2024-09-14T09:05:38.093 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.697724409999999, 2024-09-14T09:05:38.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.026050659, 2024-09-14T09:05:38.093 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.093 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.093 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.093 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:38.093 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:38.093 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:38.093 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 51 2024-09-14T09:05:38.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.094 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379481+0000", 2024-09-14T09:05:38.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379481+0000", 2024-09-14T09:05:38.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379483+0000", 2024-09-14T09:05:38.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3799999999999999e-06 2024-09-14T09:05:38.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379493+0000", 2024-09-14T09:05:38.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0164e-05 2024-09-14T09:05:38.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379535+0000", 2024-09-14T09:05:38.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1672999999999999e-05 2024-09-14T09:05:38.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379552+0000", 2024-09-14T09:05:38.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7615999999999999e-05 2024-09-14T09:05:38.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402365+0000", 2024-09-14T09:05:38.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.022812857999999998 2024-09-14T09:05:38.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402601+0000", 2024-09-14T09:05:38.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000235648 2024-09-14T09:05:38.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:38.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402966+0000", 2024-09-14T09:05:38.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036486100000000002 2024-09-14T09:05:38.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405223+0000", 2024-09-14T09:05:38.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002257144 2024-09-14T09:05:38.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:38.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405271+0000", 2024-09-14T09:05:38.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7769999999999998e-05 2024-09-14T09:05:38.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:38.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405410+0000", 2024-09-14T09:05:38.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013921499999999999 2024-09-14T09:05:38.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405532+0000", 2024-09-14T09:05:38.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012233 2024-09-14T09:05:38.098 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.098 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.098 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.098 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:38.098 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379582+0000", 2024-09-14T09:05:38.099 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.697623701000001, 2024-09-14T09:05:38.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.026468416000000002, 2024-09-14T09:05:38.099 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.099 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.099 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.099 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:38.099 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:38.099 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:38.099 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 52 2024-09-14T09:05:38.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.099 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379582+0000", 2024-09-14T09:05:38.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379582+0000", 2024-09-14T09:05:38.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379584+0000", 2024-09-14T09:05:38.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6139999999999999e-06 2024-09-14T09:05:38.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379596+0000", 2024-09-14T09:05:38.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1956e-05 2024-09-14T09:05:38.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379614+0000", 2024-09-14T09:05:38.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8099999999999999e-05 2024-09-14T09:05:38.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379638+0000", 2024-09-14T09:05:38.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4136999999999999e-05 2024-09-14T09:05:38.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.404134+0000", 2024-09-14T09:05:38.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.024496418999999998 2024-09-14T09:05:38.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.404400+0000", 2024-09-14T09:05:38.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026532500000000001 2024-09-14T09:05:38.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:38.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.404616+0000", 2024-09-14T09:05:38.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021586500000000001 2024-09-14T09:05:38.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:38.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405703+0000", 2024-09-14T09:05:38.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010879220000000001 2024-09-14T09:05:38.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405887+0000", 2024-09-14T09:05:38.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018396499999999999 2024-09-14T09:05:38.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:38.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405985+0000", 2024-09-14T09:05:38.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8033000000000001e-05 2024-09-14T09:05:38.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.406051+0000", 2024-09-14T09:05:38.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5079999999999994e-05 2024-09-14T09:05:38.103 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.647322+0000", 2024-09-14T09:05:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.429883671999999, 2024-09-14T09:05:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021121719000000001, 2024-09-14T09:05:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:38.104 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 75 2024-09-14T09:05:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647322+0000", 2024-09-14T09:05:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647322+0000", 2024-09-14T09:05:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647323+0000", 2024-09-14T09:05:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.04e-06 2024-09-14T09:05:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647407+0000", 2024-09-14T09:05:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3652999999999993e-05 2024-09-14T09:05:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647416+0000", 2024-09-14T09:05:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2310000000000002e-06 2024-09-14T09:05:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647424+0000", 2024-09-14T09:05:38.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3240000000000003e-06 2024-09-14T09:05:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.661338+0000", 2024-09-14T09:05:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013913114000000001 2024-09-14T09:05:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.661461+0000", 2024-09-14T09:05:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012374200000000001 2024-09-14T09:05:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.661653+0000", 2024-09-14T09:05:38.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000191872 2024-09-14T09:05:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.664371+0000", 2024-09-14T09:05:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027177600000000001 2024-09-14T09:05:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668357+0000", 2024-09-14T09:05:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039863229999999999 2024-09-14T09:05:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:38.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668403+0000", 2024-09-14T09:05:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6276000000000002e-05 2024-09-14T09:05:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668444+0000", 2024-09-14T09:05:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0383999999999998e-05 2024-09-14T09:05:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.647440+0000", 2024-09-14T09:05:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.429765355000001, 2024-09-14T09:05:38.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021273368000000001, 2024-09-14T09:05:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 76 2024-09-14T09:05:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647440+0000", 2024-09-14T09:05:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647440+0000", 2024-09-14T09:05:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647441+0000", 2024-09-14T09:05:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1500000000000003e-07 2024-09-14T09:05:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647448+0000", 2024-09-14T09:05:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4420000000000003e-06 2024-09-14T09:05:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647457+0000", 2024-09-14T09:05:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:05:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647465+0000", 2024-09-14T09:05:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2250000000000003e-06 2024-09-14T09:05:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.663329+0000", 2024-09-14T09:05:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015863931000000001 2024-09-14T09:05:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.663474+0000", 2024-09-14T09:05:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014509200000000001 2024-09-14T09:05:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.663695+0000", 2024-09-14T09:05:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022155499999999999 2024-09-14T09:05:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:38.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.664823+0000", 2024-09-14T09:05:38.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001128046 2024-09-14T09:05:38.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668550+0000", 2024-09-14T09:05:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003726493 2024-09-14T09:05:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668638+0000", 2024-09-14T09:05:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8282999999999995e-05 2024-09-14T09:05:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668714+0000", 2024-09-14T09:05:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5856e-05 2024-09-14T09:05:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.116 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:654 3.11 3:8c8e358b:::rbd_object_map.115e1e95bf99:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.852458+0000", 2024-09-14T09:05:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.2247476930000003, 2024-09-14T09:05:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018532317999999999, 2024-09-14T09:05:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 654 2024-09-14T09:05:38.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.852458+0000", 2024-09-14T09:05:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.852458+0000", 2024-09-14T09:05:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.852461+0000", 2024-09-14T09:05:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3270000000000001e-06 2024-09-14T09:05:38.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.852476+0000", 2024-09-14T09:05:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4543e-05 2024-09-14T09:05:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.852509+0000", 2024-09-14T09:05:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3219000000000002e-05 2024-09-14T09:05:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.852528+0000", 2024-09-14T09:05:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8386999999999999e-05 2024-09-14T09:05:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.852705+0000", 2024-09-14T09:05:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000177269 2024-09-14T09:05:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op must wait for map", 2024-09-14T09:05:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.852729+0000", 2024-09-14T09:05:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4513000000000001e-05 2024-09-14T09:05:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.861522+0000", 2024-09-14T09:05:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087921749999999993 2024-09-14T09:05:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.861731+0000", 2024-09-14T09:05:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000209748 2024-09-14T09:05:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.862486+0000", 2024-09-14T09:05:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000754888 2024-09-14T09:05:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.866665+0000", 2024-09-14T09:05:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041788529999999997 2024-09-14T09:05:38.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.870620+0000", 2024-09-14T09:05:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039549190000000003 2024-09-14T09:05:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.870863+0000", 2024-09-14T09:05:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024262099999999999 2024-09-14T09:05:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.870990+0000", 2024-09-14T09:05:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000127856 2024-09-14T09:05:38.122 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:655 3.11 3:8c8e358b:::rbd_object_map.115e1e95bf99:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.853314+0000", 2024-09-14T09:05:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.2238919729999997, 2024-09-14T09:05:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.019496724999999999, 2024-09-14T09:05:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:38.123 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 655 2024-09-14T09:05:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.853314+0000", 2024-09-14T09:05:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.853314+0000", 2024-09-14T09:05:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.853317+0000", 2024-09-14T09:05:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.207e-06 2024-09-14T09:05:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.853335+0000", 2024-09-14T09:05:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8009999999999999e-05 2024-09-14T09:05:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.853401+0000", 2024-09-14T09:05:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5463000000000007e-05 2024-09-14T09:05:38.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.853440+0000", 2024-09-14T09:05:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9789999999999997e-05 2024-09-14T09:05:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.853610+0000", 2024-09-14T09:05:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016973800000000001 2024-09-14T09:05:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:05:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.853630+0000", 2024-09-14T09:05:38.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0210000000000001e-05 2024-09-14T09:05:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863967+0000", 2024-09-14T09:05:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010336942999999999 2024-09-14T09:05:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.864264+0000", 2024-09-14T09:05:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029657799999999998 2024-09-14T09:05:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.865441+0000", 2024-09-14T09:05:38.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001176959 2024-09-14T09:05:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.868947+0000", 2024-09-14T09:05:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0035066849999999998 2024-09-14T09:05:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.872650+0000", 2024-09-14T09:05:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037025399999999998 2024-09-14T09:05:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.872739+0000", 2024-09-14T09:05:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9525999999999996e-05 2024-09-14T09:05:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.872811+0000", 2024-09-14T09:05:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1075999999999995e-05 2024-09-14T09:05:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.129 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:792 3.11 3:8c8e358b:::rbd_object_map.115e1e95bf99:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.893351+0000", 2024-09-14T09:05:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.183854406, 2024-09-14T09:05:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.024882419999999999, 2024-09-14T09:05:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 792 2024-09-14T09:05:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.893351+0000", 2024-09-14T09:05:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.893351+0000", 2024-09-14T09:05:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.893353+0000", 2024-09-14T09:05:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.53e-06 2024-09-14T09:05:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.893361+0000", 2024-09-14T09:05:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6000000000000001e-06 2024-09-14T09:05:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.893379+0000", 2024-09-14T09:05:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8283000000000001e-05 2024-09-14T09:05:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.893393+0000", 2024-09-14T09:05:38.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4365999999999999e-05 2024-09-14T09:05:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.895595+0000", 2024-09-14T09:05:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002201518 2024-09-14T09:05:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.895894+0000", 2024-09-14T09:05:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029924800000000001 2024-09-14T09:05:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.896899+0000", 2024-09-14T09:05:38.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010047960000000001 2024-09-14T09:05:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.908656+0000", 2024-09-14T09:05:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011756902 2024-09-14T09:05:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.918060+0000", 2024-09-14T09:05:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0094038809999999994 2024-09-14T09:05:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.918143+0000", 2024-09-14T09:05:38.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3236000000000001e-05 2024-09-14T09:05:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.918234+0000", 2024-09-14T09:05:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1059999999999999e-05 2024-09-14T09:05:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:821 3.e 3:740ec7e3:::rbd_data.115e724e1d99.0000000000000006:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.895061+0000", 2024-09-14T09:05:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.1821447420000002, 2024-09-14T09:05:38.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.020606620999999999, 2024-09-14T09:05:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:05:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 821 2024-09-14T09:05:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.895061+0000", 2024-09-14T09:05:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.895061+0000", 2024-09-14T09:05:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.895063+0000", 2024-09-14T09:05:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5430000000000001e-06 2024-09-14T09:05:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.895071+0000", 2024-09-14T09:05:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0569999999999993e-06 2024-09-14T09:05:38.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.895092+0000", 2024-09-14T09:05:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1126000000000001e-05 2024-09-14T09:05:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.895108+0000", 2024-09-14T09:05:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6123000000000001e-05 2024-09-14T09:05:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.915239+0000", 2024-09-14T09:05:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.020130674000000001 2024-09-14T09:05:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.915423+0000", 2024-09-14T09:05:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000184222 2024-09-14T09:05:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.915668+0000", 2024-09-14T09:05:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024487599999999999 2024-09-14T09:05:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:829 3.13 3:cc36a41f:::rbd_data.115e1e95bf99.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13581651~3195565] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.899824+0000", 2024-09-14T09:05:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.1773821760000001, 2024-09-14T09:05:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.026145906999999999, 2024-09-14T09:05:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 829 2024-09-14T09:05:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.140 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.899824+0000", 2024-09-14T09:05:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.899824+0000", 2024-09-14T09:05:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.899826+0000", 2024-09-14T09:05:38.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4729999999999999e-06 2024-09-14T09:05:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900269+0000", 2024-09-14T09:05:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00044249300000000002 2024-09-14T09:05:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900300+0000", 2024-09-14T09:05:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1016999999999998e-05 2024-09-14T09:05:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900321+0000", 2024-09-14T09:05:38.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.126e-05 2024-09-14T09:05:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900462+0000", 2024-09-14T09:05:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014153200000000001 2024-09-14T09:05:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900766+0000", 2024-09-14T09:05:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000303424 2024-09-14T09:05:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:38.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901279+0000", 2024-09-14T09:05:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00051300499999999997 2024-09-14T09:05:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921167+0000", 2024-09-14T09:05:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.019888408999999999 2024-09-14T09:05:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.925809+0000", 2024-09-14T09:05:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004641895 2024-09-14T09:05:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:38.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.925897+0000", 2024-09-14T09:05:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7965000000000002e-05 2024-09-14T09:05:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.925970+0000", 2024-09-14T09:05:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2434000000000005e-05 2024-09-14T09:05:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:830 3.11 3:8c8e358b:::rbd_object_map.115e1e95bf99:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.900383+0000", 2024-09-14T09:05:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.1768223769999997, 2024-09-14T09:05:38.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021339981000000001, 2024-09-14T09:05:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 830 2024-09-14T09:05:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900383+0000", 2024-09-14T09:05:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900383+0000", 2024-09-14T09:05:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900385+0000", 2024-09-14T09:05:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.934e-06 2024-09-14T09:05:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900395+0000", 2024-09-14T09:05:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4190000000000008e-06 2024-09-14T09:05:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900418+0000", 2024-09-14T09:05:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2776999999999999e-05 2024-09-14T09:05:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900434+0000", 2024-09-14T09:05:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5996000000000001e-05 2024-09-14T09:05:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900545+0000", 2024-09-14T09:05:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000111601 2024-09-14T09:05:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900740+0000", 2024-09-14T09:05:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000194976 2024-09-14T09:05:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901654+0000", 2024-09-14T09:05:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091431599999999998 2024-09-14T09:05:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.908747+0000", 2024-09-14T09:05:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0070929920000000002 2024-09-14T09:05:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921507+0000", 2024-09-14T09:05:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.012759645999999999 2024-09-14T09:05:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921601+0000", 2024-09-14T09:05:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3844999999999993e-05 2024-09-14T09:05:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921723+0000", 2024-09-14T09:05:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012247899999999999 2024-09-14T09:05:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.150 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:831 3.11 3:8c8e358b:::rbd_object_map.115e1e95bf99:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.900470+0000", 2024-09-14T09:05:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.1767362209999996, 2024-09-14T09:05:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021390532, 2024-09-14T09:05:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 831 2024-09-14T09:05:38.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900470+0000", 2024-09-14T09:05:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900470+0000", 2024-09-14T09:05:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900471+0000", 2024-09-14T09:05:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5370000000000001e-06 2024-09-14T09:05:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900479+0000", 2024-09-14T09:05:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0029999999999994e-06 2024-09-14T09:05:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900499+0000", 2024-09-14T09:05:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0047e-05 2024-09-14T09:05:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900518+0000", 2024-09-14T09:05:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.929e-05 2024-09-14T09:05:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902843+0000", 2024-09-14T09:05:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023242670000000001 2024-09-14T09:05:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.903053+0000", 2024-09-14T09:05:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021007799999999999 2024-09-14T09:05:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.903951+0000", 2024-09-14T09:05:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00089844499999999999 2024-09-14T09:05:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.908830+0000", 2024-09-14T09:05:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048782840000000001 2024-09-14T09:05:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921741+0000", 2024-09-14T09:05:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.012911618999999999 2024-09-14T09:05:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921803+0000", 2024-09-14T09:05:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2238999999999998e-05 2024-09-14T09:05:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921860+0000", 2024-09-14T09:05:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6722999999999999e-05 2024-09-14T09:05:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.156 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:832 3.11 3:8c8e358b:::rbd_object_map.115e1e95bf99:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.900557+0000", 2024-09-14T09:05:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.1766484049999999, 2024-09-14T09:05:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021419134999999999, 2024-09-14T09:05:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 832 2024-09-14T09:05:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900557+0000", 2024-09-14T09:05:38.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900557+0000", 2024-09-14T09:05:38.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900559+0000", 2024-09-14T09:05:38.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.733e-06 2024-09-14T09:05:38.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900567+0000", 2024-09-14T09:05:38.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7270000000000007e-06 2024-09-14T09:05:38.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900586+0000", 2024-09-14T09:05:38.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9233e-05 2024-09-14T09:05:38.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900600+0000", 2024-09-14T09:05:38.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4229999999999999e-05 2024-09-14T09:05:38.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.904848+0000", 2024-09-14T09:05:38.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042476019999999996 2024-09-14T09:05:38.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.905051+0000", 2024-09-14T09:05:38.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020333300000000001 2024-09-14T09:05:38.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:38.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.906440+0000", 2024-09-14T09:05:38.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001388638 2024-09-14T09:05:38.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.911832+0000", 2024-09-14T09:05:38.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053925079999999999 2024-09-14T09:05:38.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:38.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921872+0000", 2024-09-14T09:05:38.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010040083999999999 2024-09-14T09:05:38.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:38.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921925+0000", 2024-09-14T09:05:38.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2524000000000001e-05 2024-09-14T09:05:38.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921977+0000", 2024-09-14T09:05:38.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1523000000000001e-05 2024-09-14T09:05:38.162 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.162 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.162 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.162 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:833 3.11 3:8c8e358b:::rbd_object_map.115e1e95bf99:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:38.162 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.900622+0000", 2024-09-14T09:05:38.162 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.1765837450000003, 2024-09-14T09:05:38.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021467374000000001, 2024-09-14T09:05:38.162 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.162 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.163 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.163 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:38.163 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:38.163 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.163 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 833 2024-09-14T09:05:38.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.163 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900622+0000", 2024-09-14T09:05:38.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900622+0000", 2024-09-14T09:05:38.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900624+0000", 2024-09-14T09:05:38.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4929999999999999e-06 2024-09-14T09:05:38.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900681+0000", 2024-09-14T09:05:38.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7859999999999998e-05 2024-09-14T09:05:38.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900715+0000", 2024-09-14T09:05:38.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3402999999999999e-05 2024-09-14T09:05:38.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900731+0000", 2024-09-14T09:05:38.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5773e-05 2024-09-14T09:05:38.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.908869+0000", 2024-09-14T09:05:38.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0081381809999999995 2024-09-14T09:05:38.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.909003+0000", 2024-09-14T09:05:38.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013468900000000001 2024-09-14T09:05:38.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:38.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.909599+0000", 2024-09-14T09:05:38.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00059570800000000002 2024-09-14T09:05:38.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.915364+0000", 2024-09-14T09:05:38.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057650109999999996 2024-09-14T09:05:38.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:38.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921988+0000", 2024-09-14T09:05:38.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006624217 2024-09-14T09:05:38.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:38.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922040+0000", 2024-09-14T09:05:38.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1749999999999997e-05 2024-09-14T09:05:38.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922089+0000", 2024-09-14T09:05:38.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9289000000000002e-05 2024-09-14T09:05:38.167 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.167 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.167 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.168 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:834 3.11 3:8c8e358b:::rbd_object_map.115e1e95bf99:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:38.168 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.902188+0000", 2024-09-14T09:05:38.168 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.1750173659999996, 2024-09-14T09:05:38.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.020043834, 2024-09-14T09:05:38.168 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.168 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.168 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.168 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:38.168 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:38.168 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.168 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 834 2024-09-14T09:05:38.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.169 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902188+0000", 2024-09-14T09:05:38.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902188+0000", 2024-09-14T09:05:38.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902191+0000", 2024-09-14T09:05:38.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6469999999999998e-06 2024-09-14T09:05:38.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902205+0000", 2024-09-14T09:05:38.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3890000000000001e-05 2024-09-14T09:05:38.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902237+0000", 2024-09-14T09:05:38.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1606000000000002e-05 2024-09-14T09:05:38.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902259+0000", 2024-09-14T09:05:38.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2543000000000001e-05 2024-09-14T09:05:38.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.910278+0000", 2024-09-14T09:05:38.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0080190130000000002 2024-09-14T09:05:38.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.910449+0000", 2024-09-14T09:05:38.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170483 2024-09-14T09:05:38.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:38.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.911071+0000", 2024-09-14T09:05:38.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00062188300000000005 2024-09-14T09:05:38.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.915493+0000", 2024-09-14T09:05:38.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044225100000000002 2024-09-14T09:05:38.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:38.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922118+0000", 2024-09-14T09:05:38.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066246050000000004 2024-09-14T09:05:38.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:38.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922180+0000", 2024-09-14T09:05:38.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2477000000000005e-05 2024-09-14T09:05:38.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922232+0000", 2024-09-14T09:05:38.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2176999999999999e-05 2024-09-14T09:05:38.173 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.173 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.173 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.174 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:835 3.11 3:8c8e358b:::rbd_object_map.115e1e95bf99:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:38.174 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.903632+0000", 2024-09-14T09:05:38.174 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.1735738089999996, 2024-09-14T09:05:38.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018702719, 2024-09-14T09:05:38.174 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.174 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.174 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:38.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:38.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.174 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 835 2024-09-14T09:05:38.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.174 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.903632+0000", 2024-09-14T09:05:38.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.903632+0000", 2024-09-14T09:05:38.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.903634+0000", 2024-09-14T09:05:38.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.216e-06 2024-09-14T09:05:38.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.903647+0000", 2024-09-14T09:05:38.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2979999999999999e-05 2024-09-14T09:05:38.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.903678+0000", 2024-09-14T09:05:38.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0607e-05 2024-09-14T09:05:38.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.903699+0000", 2024-09-14T09:05:38.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1189999999999999e-05 2024-09-14T09:05:38.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.911909+0000", 2024-09-14T09:05:38.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0082102470000000004 2024-09-14T09:05:38.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.912070+0000", 2024-09-14T09:05:38.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016051100000000001 2024-09-14T09:05:38.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:38.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.912751+0000", 2024-09-14T09:05:38.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068099800000000004 2024-09-14T09:05:38.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.915587+0000", 2024-09-14T09:05:38.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028365399999999998 2024-09-14T09:05:38.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:38.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922245+0000", 2024-09-14T09:05:38.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066577449999999996 2024-09-14T09:05:38.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:38.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922297+0000", 2024-09-14T09:05:38.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1854999999999997e-05 2024-09-14T09:05:38.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922335+0000", 2024-09-14T09:05:38.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7830000000000002e-05 2024-09-14T09:05:38.179 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.179 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.179 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.179 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:10026 3.f 3:f6f4e415:::rbd_data.115eadec874.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:38.179 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.713901+0000", 2024-09-14T09:05:38.179 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.36330462000000002, 2024-09-14T09:05:38.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.045312305999999997, 2024-09-14T09:05:38.179 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.179 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.179 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.179 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:38.180 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:38.180 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.180 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 10026 2024-09-14T09:05:38.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.180 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713901+0000", 2024-09-14T09:05:38.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713901+0000", 2024-09-14T09:05:38.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713903+0000", 2024-09-14T09:05:38.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4750000000000001e-06 2024-09-14T09:05:38.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713908+0000", 2024-09-14T09:05:38.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.242e-06 2024-09-14T09:05:38.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713920+0000", 2024-09-14T09:05:38.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2456000000000001e-05 2024-09-14T09:05:38.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713927+0000", 2024-09-14T09:05:38.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0849999999999999e-06 2024-09-14T09:05:38.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713973+0000", 2024-09-14T09:05:38.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5179000000000002e-05 2024-09-14T09:05:38.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.714023+0000", 2024-09-14T09:05:38.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0469000000000001e-05 2024-09-14T09:05:38.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:38.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.715791+0000", 2024-09-14T09:05:38.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017676580000000001 2024-09-14T09:05:38.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.759031+0000", 2024-09-14T09:05:38.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.043240439999999998 2024-09-14T09:05:38.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:38.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.759058+0000", 2024-09-14T09:05:38.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6395999999999999e-05 2024-09-14T09:05:38.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:38.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.759141+0000", 2024-09-14T09:05:38.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3026000000000001e-05 2024-09-14T09:05:38.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.759213+0000", 2024-09-14T09:05:38.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2879999999999993e-05 2024-09-14T09:05:38.184 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.184 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.184 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.185 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:10026 3.f e32/23)", 2024-09-14T09:05:38.185 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.722702+0000", 2024-09-14T09:05:38.185 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.35450375499999998, 2024-09-14T09:05:38.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.036342688999999997, 2024-09-14T09:05:38.185 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:38.185 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:38.185 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:05:38.185 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:38.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:38.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.722702+0000", 2024-09-14T09:05:38.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:38.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.722702+0000", 2024-09-14T09:05:38.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:38.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:38.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.722703+0000", 2024-09-14T09:05:38.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.046e-06 2024-09-14T09:05:38.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:38.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.722708+0000", 2024-09-14T09:05:38.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2429999999999996e-06 2024-09-14T09:05:38.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:38.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.722725+0000", 2024-09-14T09:05:38.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6748e-05 2024-09-14T09:05:38.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.722734+0000", 2024-09-14T09:05:38.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3589999999999997e-06 2024-09-14T09:05:38.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:38.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.759004+0000", 2024-09-14T09:05:38.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.036269606000000003 2024-09-14T09:05:38.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:38.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.759021+0000", 2024-09-14T09:05:38.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6883000000000001e-05 2024-09-14T09:05:38.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:38.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:38.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:38.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.759045+0000", 2024-09-14T09:05:38.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3804000000000001e-05 2024-09-14T09:05:38.188 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.188 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.188 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.189 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:38.189 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:38.189 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:05:38.191 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:05:38.192 INFO:teuthology.orchestra.run.smithi176.stdout:402 write 0x665b511 thru 0x665f9a0 (0x4490 bytes) 2024-09-14T09:05:38.192 INFO:teuthology.orchestra.run.smithi176.stdout:403 read 0x1dd99a6 thru 0x1de342e (0x9a89 bytes) 2024-09-14T09:05:38.192 INFO:teuthology.orchestra.run.smithi176.stdout:404 read 0x25ea641 thru 0x25ebdd3 (0x1793 bytes) 2024-09-14T09:05:38.192 INFO:teuthology.orchestra.run.smithi176.stdout:405 read 0x4ddecb7 thru 0x4ddfd2a (0x1074 bytes) 2024-09-14T09:05:38.192 INFO:teuthology.orchestra.run.smithi176.stdout:406 write 0xa5e7929 thru 0xa5ee73c (0x6e14 bytes) 2024-09-14T09:05:38.192 INFO:teuthology.orchestra.run.smithi176.stdout:408 write 0x904852a thru 0x904dc97 (0x576e bytes) 2024-09-14T09:05:38.193 INFO:teuthology.orchestra.run.smithi176.stdout:410 write 0xbb67ef thru 0xbc605e (0xf870 bytes) 2024-09-14T09:05:38.193 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:38.193 INFO:teuthology.orchestra.run.smithi176.stdout:412 write 0xc133b34 thru 0xc139098 (0x5565 bytes) 2024-09-14T09:05:38.193 INFO:teuthology.orchestra.run.smithi176.stdout:413 read 0x690bd92 thru 0x690fa64 (0x3cd3 bytes) 2024-09-14T09:05:38.193 INFO:teuthology.orchestra.run.smithi176.stdout:414 write 0xc55ec82 thru 0xc55f69b (0xa1a bytes) 2024-09-14T09:05:38.193 INFO:teuthology.orchestra.run.smithi176.stdout:415 write 0x82f39d3 thru 0x82f78c3 (0x3ef1 bytes) 2024-09-14T09:05:38.193 INFO:teuthology.orchestra.run.smithi176.stdout:416 read 0xa1c5747 thru 0xa1c8ce1 (0x359b bytes) 2024-09-14T09:05:38.193 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:38.193 INFO:teuthology.orchestra.run.smithi176.stdout:418 trunc from 0xc55f69c to 0x5e4318f 2024-09-14T09:05:38.193 INFO:teuthology.orchestra.run.smithi176.stdout:419 read 0x29b2c00 thru 0x29b6543 (0x3944 bytes) 2024-09-14T09:05:38.193 INFO:teuthology.orchestra.run.smithi176.stdout:420 punch from 0xd3974b to 0xd3e486, (0x4d3b bytes) 2024-09-14T09:05:38.193 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:38.193 INFO:teuthology.orchestra.run.smithi176.stdout:422 read 0x32a5214 thru 0x32b2f59 (0xdd46 bytes) 2024-09-14T09:05:38.193 INFO:teuthology.orchestra.run.smithi176.stdout:423 write 0x52d67bd thru 0x52e2434 (0xbc78 bytes) 2024-09-14T09:05:38.194 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:38.188+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:38.203 INFO:teuthology.orchestra.run.smithi176.stdout:424 writesame 0xb2cb1b3 thru 0xb2d98ee data_size 0x6(0xe73c bytes) 2024-09-14T09:05:38.214 INFO:teuthology.orchestra.run.smithi176.stdout:425 trunc from 0xb2d98ef to 0x626cdb3 2024-09-14T09:05:38.229 INFO:teuthology.orchestra.run.smithi176.stdout:426 clone 4 order 24 su 131072 sc 2 2024-09-14T09:05:38.291 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:38.291+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:38.319 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:38.319 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:38.319 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:38.319 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:38.333 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:05:38.392 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:38.391+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:38.493 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:38.488+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:38.497 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:38.497 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:38.497 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:38.497 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:38.498 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:38.512 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:05:38.594 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:38.595+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:38.679 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:38.679 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:38.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:38.679 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:38.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.680 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-14T09:05:38.680 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.274102+0000", 2024-09-14T09:05:38.680 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.404037445, 2024-09-14T09:05:38.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003545763, 2024-09-14T09:05:38.680 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.680 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.680 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.680 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274102+0000", 2024-09-14T09:05:38.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274102+0000", 2024-09-14T09:05:38.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274103+0000", 2024-09-14T09:05:38.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4670000000000001e-06 2024-09-14T09:05:38.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274328+0000", 2024-09-14T09:05:38.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022502500000000001 2024-09-14T09:05:38.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274354+0000", 2024-09-14T09:05:38.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5993e-05 2024-09-14T09:05:38.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274372+0000", 2024-09-14T09:05:38.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8117e-05 2024-09-14T09:05:38.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274455+0000", 2024-09-14T09:05:38.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2785000000000001e-05 2024-09-14T09:05:38.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274486+0000", 2024-09-14T09:05:38.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0864000000000001e-05 2024-09-14T09:05:38.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:38.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277538+0000", 2024-09-14T09:05:38.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0030521860000000001 2024-09-14T09:05:38.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277647+0000", 2024-09-14T09:05:38.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109326 2024-09-14T09:05:38.683 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.684 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.684 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.684 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-14T09:05:38.684 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.275860+0000", 2024-09-14T09:05:38.684 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.402279271000001, 2024-09-14T09:05:38.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003513562, 2024-09-14T09:05:38.684 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.684 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.684 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.684 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.275860+0000", 2024-09-14T09:05:38.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.275860+0000", 2024-09-14T09:05:38.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.275861+0000", 2024-09-14T09:05:38.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.513e-06 2024-09-14T09:05:38.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276039+0000", 2024-09-14T09:05:38.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017758199999999999 2024-09-14T09:05:38.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276060+0000", 2024-09-14T09:05:38.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0956000000000001e-05 2024-09-14T09:05:38.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276076+0000", 2024-09-14T09:05:38.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6137000000000002e-05 2024-09-14T09:05:38.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276190+0000", 2024-09-14T09:05:38.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011359200000000001 2024-09-14T09:05:38.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276226+0000", 2024-09-14T09:05:38.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6300000000000001e-05 2024-09-14T09:05:38.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:38.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.279274+0000", 2024-09-14T09:05:38.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00304813 2024-09-14T09:05:38.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.279373+0000", 2024-09-14T09:05:38.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9351999999999993e-05 2024-09-14T09:05:38.688 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.688 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.688 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.688 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:72 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'50, mlcod=17'45)", 2024-09-14T09:05:38.688 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.657508+0000", 2024-09-14T09:05:38.688 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.020630958000002, 2024-09-14T09:05:38.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0033585239999999999, 2024-09-14T09:05:38.688 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.688 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.688 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.688 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657508+0000", 2024-09-14T09:05:38.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657508+0000", 2024-09-14T09:05:38.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657510+0000", 2024-09-14T09:05:38.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8899999999999999e-06 2024-09-14T09:05:38.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657625+0000", 2024-09-14T09:05:38.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115216 2024-09-14T09:05:38.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657649+0000", 2024-09-14T09:05:38.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4043e-05 2024-09-14T09:05:38.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657669+0000", 2024-09-14T09:05:38.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9579999999999999e-05 2024-09-14T09:05:38.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657790+0000", 2024-09-14T09:05:38.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012084200000000001 2024-09-14T09:05:38.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657828+0000", 2024-09-14T09:05:38.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8019999999999999e-05 2024-09-14T09:05:38.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:38.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660805+0000", 2024-09-14T09:05:38.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002976874 2024-09-14T09:05:38.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660867+0000", 2024-09-14T09:05:38.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2058999999999997e-05 2024-09-14T09:05:38.692 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.692 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.692 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.692 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:73 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'51, mlcod=17'45)", 2024-09-14T09:05:38.692 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.658866+0000", 2024-09-14T09:05:38.692 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.019273520999999, 2024-09-14T09:05:38.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035579930000000002, 2024-09-14T09:05:38.692 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.692 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.692 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.693 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658866+0000", 2024-09-14T09:05:38.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658866+0000", 2024-09-14T09:05:38.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658867+0000", 2024-09-14T09:05:38.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.886e-06 2024-09-14T09:05:38.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658880+0000", 2024-09-14T09:05:38.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2204e-05 2024-09-14T09:05:38.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658908+0000", 2024-09-14T09:05:38.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8555999999999999e-05 2024-09-14T09:05:38.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658921+0000", 2024-09-14T09:05:38.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2566e-05 2024-09-14T09:05:38.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659682+0000", 2024-09-14T09:05:38.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000760865 2024-09-14T09:05:38.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659710+0000", 2024-09-14T09:05:38.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8612e-05 2024-09-14T09:05:38.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:38.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662349+0000", 2024-09-14T09:05:38.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0026388370000000002 2024-09-14T09:05:38.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662424+0000", 2024-09-14T09:05:38.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4467000000000004e-05 2024-09-14T09:05:38.696 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.696 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.696 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.696 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:74 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'52, mlcod=17'47)", 2024-09-14T09:05:38.696 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.659878+0000", 2024-09-14T09:05:38.696 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.018260952999999, 2024-09-14T09:05:38.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0040591689999999996, 2024-09-14T09:05:38.697 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.697 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.697 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.697 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659878+0000", 2024-09-14T09:05:38.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659878+0000", 2024-09-14T09:05:38.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659892+0000", 2024-09-14T09:05:38.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3487e-05 2024-09-14T09:05:38.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659902+0000", 2024-09-14T09:05:38.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.062e-05 2024-09-14T09:05:38.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659928+0000", 2024-09-14T09:05:38.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6060000000000001e-05 2024-09-14T09:05:38.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659941+0000", 2024-09-14T09:05:38.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2649999999999999e-05 2024-09-14T09:05:38.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660304+0000", 2024-09-14T09:05:38.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036354699999999998 2024-09-14T09:05:38.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660349+0000", 2024-09-14T09:05:38.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4938999999999998e-05 2024-09-14T09:05:38.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:38.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.663847+0000", 2024-09-14T09:05:38.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0034973339999999999 2024-09-14T09:05:38.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.663937+0000", 2024-09-14T09:05:38.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0532000000000007e-05 2024-09-14T09:05:38.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.700 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.700 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'47)", 2024-09-14T09:05:38.700 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.660695+0000", 2024-09-14T09:05:38.700 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.017443801999999, 2024-09-14T09:05:38.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0062206520000000001, 2024-09-14T09:05:38.701 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.701 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.701 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.701 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660695+0000", 2024-09-14T09:05:38.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660695+0000", 2024-09-14T09:05:38.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660697+0000", 2024-09-14T09:05:38.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.897e-06 2024-09-14T09:05:38.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660816+0000", 2024-09-14T09:05:38.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118442 2024-09-14T09:05:38.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660839+0000", 2024-09-14T09:05:38.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3790000000000001e-05 2024-09-14T09:05:38.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660852+0000", 2024-09-14T09:05:38.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-05 2024-09-14T09:05:38.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660967+0000", 2024-09-14T09:05:38.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115276 2024-09-14T09:05:38.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.661016+0000", 2024-09-14T09:05:38.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8785999999999997e-05 2024-09-14T09:05:38.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:38.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.666825+0000", 2024-09-14T09:05:38.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0058087549999999996 2024-09-14T09:05:38.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.666916+0000", 2024-09-14T09:05:38.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1325999999999999e-05 2024-09-14T09:05:38.704 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.704 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.704 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.704 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'49)", 2024-09-14T09:05:38.704 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.662683+0000", 2024-09-14T09:05:38.704 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.015456489999998, 2024-09-14T09:05:38.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0042817899999999997, 2024-09-14T09:05:38.705 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.705 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.705 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.705 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662683+0000", 2024-09-14T09:05:38.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662683+0000", 2024-09-14T09:05:38.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662684+0000", 2024-09-14T09:05:38.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6559999999999999e-06 2024-09-14T09:05:38.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662696+0000", 2024-09-14T09:05:38.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1454000000000001e-05 2024-09-14T09:05:38.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662724+0000", 2024-09-14T09:05:38.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8535999999999999e-05 2024-09-14T09:05:38.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662752+0000", 2024-09-14T09:05:38.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7503000000000001e-05 2024-09-14T09:05:38.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662867+0000", 2024-09-14T09:05:38.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011500199999999999 2024-09-14T09:05:38.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662899+0000", 2024-09-14T09:05:38.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2122999999999997e-05 2024-09-14T09:05:38.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:38.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.666919+0000", 2024-09-14T09:05:38.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004020633 2024-09-14T09:05:38.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.666964+0000", 2024-09-14T09:05:38.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4882999999999997e-05 2024-09-14T09:05:38.708 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.708 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.708 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.709 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:77 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'55, mlcod=17'53)", 2024-09-14T09:05:38.709 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.668436+0000", 2024-09-14T09:05:38.709 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.009702729000001, 2024-09-14T09:05:38.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035848659999999999, 2024-09-14T09:05:38.709 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.709 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.709 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.709 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668436+0000", 2024-09-14T09:05:38.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668436+0000", 2024-09-14T09:05:38.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668438+0000", 2024-09-14T09:05:38.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.81e-06 2024-09-14T09:05:38.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668551+0000", 2024-09-14T09:05:38.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112726 2024-09-14T09:05:38.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668580+0000", 2024-09-14T09:05:38.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8702999999999999e-05 2024-09-14T09:05:38.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668592+0000", 2024-09-14T09:05:38.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.276e-05 2024-09-14T09:05:38.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668767+0000", 2024-09-14T09:05:38.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017485600000000001 2024-09-14T09:05:38.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668808+0000", 2024-09-14T09:05:38.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1189000000000002e-05 2024-09-14T09:05:38.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:38.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.671940+0000", 2024-09-14T09:05:38.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0031317329999999998 2024-09-14T09:05:38.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672021+0000", 2024-09-14T09:05:38.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1088999999999996e-05 2024-09-14T09:05:38.712 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.713 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.713 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.713 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:78 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'56, mlcod=17'53)", 2024-09-14T09:05:38.713 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.670356+0000", 2024-09-14T09:05:38.713 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.007782943999999, 2024-09-14T09:05:38.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0036555289999999998, 2024-09-14T09:05:38.713 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.713 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.713 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.713 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670356+0000", 2024-09-14T09:05:38.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670356+0000", 2024-09-14T09:05:38.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670358+0000", 2024-09-14T09:05:38.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.79e-06 2024-09-14T09:05:38.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670469+0000", 2024-09-14T09:05:38.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111273 2024-09-14T09:05:38.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670495+0000", 2024-09-14T09:05:38.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5516000000000001e-05 2024-09-14T09:05:38.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670508+0000", 2024-09-14T09:05:38.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2986999999999999e-05 2024-09-14T09:05:38.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670679+0000", 2024-09-14T09:05:38.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017094199999999999 2024-09-14T09:05:38.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670717+0000", 2024-09-14T09:05:38.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8322999999999999e-05 2024-09-14T09:05:38.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:38.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.673942+0000", 2024-09-14T09:05:38.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032255489999999999 2024-09-14T09:05:38.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.674012+0000", 2024-09-14T09:05:38.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9148999999999998e-05 2024-09-14T09:05:38.717 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.717 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.717 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.717 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:79 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'57, mlcod=17'53)", 2024-09-14T09:05:38.717 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.672051+0000", 2024-09-14T09:05:38.717 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.006088243000001, 2024-09-14T09:05:38.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0033465410000000002, 2024-09-14T09:05:38.717 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.717 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.718 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.718 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672051+0000", 2024-09-14T09:05:38.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672051+0000", 2024-09-14T09:05:38.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672053+0000", 2024-09-14T09:05:38.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0159999999999998e-06 2024-09-14T09:05:38.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672128+0000", 2024-09-14T09:05:38.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4909999999999999e-05 2024-09-14T09:05:38.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672162+0000", 2024-09-14T09:05:38.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4725999999999999e-05 2024-09-14T09:05:38.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672181+0000", 2024-09-14T09:05:38.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8357000000000001e-05 2024-09-14T09:05:38.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672311+0000", 2024-09-14T09:05:38.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012992200000000001 2024-09-14T09:05:38.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672342+0000", 2024-09-14T09:05:38.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1473000000000002e-05 2024-09-14T09:05:38.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:38.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.675298+0000", 2024-09-14T09:05:38.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0029560910000000001 2024-09-14T09:05:38.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.675397+0000", 2024-09-14T09:05:38.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9046e-05 2024-09-14T09:05:38.721 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.721 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.721 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.722 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T09:05:38.722 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:28.835461+0000", 2024-09-14T09:05:38.722 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8426778640000006, 2024-09-14T09:05:38.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050182409999999997, 2024-09-14T09:05:38.722 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.722 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.722 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.722 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:38.722 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:38.722 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.722 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 11 2024-09-14T09:05:38.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.722 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835461+0000", 2024-09-14T09:05:38.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835461+0000", 2024-09-14T09:05:38.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835463+0000", 2024-09-14T09:05:38.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9570000000000001e-06 2024-09-14T09:05:38.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835474+0000", 2024-09-14T09:05:38.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0949e-05 2024-09-14T09:05:38.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835495+0000", 2024-09-14T09:05:38.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1325e-05 2024-09-14T09:05:38.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835506+0000", 2024-09-14T09:05:38.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0854e-05 2024-09-14T09:05:38.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835579+0000", 2024-09-14T09:05:38.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2743000000000005e-05 2024-09-14T09:05:38.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T09:05:38.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835585+0000", 2024-09-14T09:05:38.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9519999999999999e-06 2024-09-14T09:05:38.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.838428+0000", 2024-09-14T09:05:38.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0028425830000000001 2024-09-14T09:05:38.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.838557+0000", 2024-09-14T09:05:38.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012985000000000001 2024-09-14T09:05:38.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:05:38.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.838743+0000", 2024-09-14T09:05:38.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018533199999999999 2024-09-14T09:05:38.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:38.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.839964+0000", 2024-09-14T09:05:38.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001221387 2024-09-14T09:05:38.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.840333+0000", 2024-09-14T09:05:38.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036902899999999999 2024-09-14T09:05:38.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:38.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.840410+0000", 2024-09-14T09:05:38.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6860999999999995e-05 2024-09-14T09:05:38.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.840479+0000", 2024-09-14T09:05:38.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9418999999999994e-05 2024-09-14T09:05:38.727 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.727 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.727 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.728 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:657 3.6 3:65b8693c:::rbd_data.115e1e95bf99.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 524288~51261] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:38.728 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.871474+0000", 2024-09-14T09:05:38.728 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8066653620000004, 2024-09-14T09:05:38.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0082168940000000006, 2024-09-14T09:05:38.728 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.728 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.728 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.728 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:38.728 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:38.728 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.728 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 657 2024-09-14T09:05:38.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.729 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871474+0000", 2024-09-14T09:05:38.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871474+0000", 2024-09-14T09:05:38.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871476+0000", 2024-09-14T09:05:38.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4109999999999999e-06 2024-09-14T09:05:38.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871588+0000", 2024-09-14T09:05:38.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111891 2024-09-14T09:05:38.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871609+0000", 2024-09-14T09:05:38.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0856000000000002e-05 2024-09-14T09:05:38.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871622+0000", 2024-09-14T09:05:38.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3175e-05 2024-09-14T09:05:38.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871755+0000", 2024-09-14T09:05:38.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013305599999999999 2024-09-14T09:05:38.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872022+0000", 2024-09-14T09:05:38.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026700600000000003 2024-09-14T09:05:38.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:05:38.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872297+0000", 2024-09-14T09:05:38.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027538299999999998 2024-09-14T09:05:38.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.879290+0000", 2024-09-14T09:05:38.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069921779999999999 2024-09-14T09:05:38.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:38.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.879580+0000", 2024-09-14T09:05:38.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029007000000000002 2024-09-14T09:05:38.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:38.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.879619+0000", 2024-09-14T09:05:38.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9311999999999999e-05 2024-09-14T09:05:38.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.879691+0000", 2024-09-14T09:05:38.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1556000000000004e-05 2024-09-14T09:05:38.733 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.733 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.733 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.733 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:673 3.6 3:65b8693c:::rbd_data.115e1e95bf99.0000000000000002:head [stat,zero 13369344~3230342] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:38.734 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.885224+0000", 2024-09-14T09:05:38.734 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.7929149390000001, 2024-09-14T09:05:38.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0046878689999999999, 2024-09-14T09:05:38.734 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.734 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.734 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:38.734 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:38.734 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:38.734 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.734 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 673 2024-09-14T09:05:38.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.734 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885224+0000", 2024-09-14T09:05:38.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885224+0000", 2024-09-14T09:05:38.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885225+0000", 2024-09-14T09:05:38.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2600000000000002e-07 2024-09-14T09:05:38.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885229+0000", 2024-09-14T09:05:38.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5220000000000001e-06 2024-09-14T09:05:38.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885238+0000", 2024-09-14T09:05:38.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5019999999999992e-06 2024-09-14T09:05:38.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885245+0000", 2024-09-14T09:05:38.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7909999999999999e-06 2024-09-14T09:05:38.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885481+0000", 2024-09-14T09:05:38.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023645 2024-09-14T09:05:38.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.886357+0000", 2024-09-14T09:05:38.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00087546399999999995 2024-09-14T09:05:38.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.889912+0000", 2024-09-14T09:05:38.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035554139999999998 2024-09-14T09:05:38.737 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.738 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.738 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.738 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:9947 3.17 3:ecdf2631:::rbd_data.115eadec874.0000000000000007:head [stat,zero 6291456~10485760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:38.738 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.551084+0000", 2024-09-14T09:05:38.738 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.1270546829999999, 2024-09-14T09:05:38.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032994859999999999, 2024-09-14T09:05:38.738 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.738 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.738 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:38.738 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:38.738 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:38.738 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.738 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 9947 2024-09-14T09:05:38.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.739 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551084+0000", 2024-09-14T09:05:38.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551084+0000", 2024-09-14T09:05:38.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551085+0000", 2024-09-14T09:05:38.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0999999999999998e-07 2024-09-14T09:05:38.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551088+0000", 2024-09-14T09:05:38.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3249999999999999e-06 2024-09-14T09:05:38.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551103+0000", 2024-09-14T09:05:38.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4216000000000001e-05 2024-09-14T09:05:38.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551113+0000", 2024-09-14T09:05:38.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0569e-05 2024-09-14T09:05:38.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551918+0000", 2024-09-14T09:05:38.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000804935 2024-09-14T09:05:38.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.552010+0000", 2024-09-14T09:05:38.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1829999999999993e-05 2024-09-14T09:05:38.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.554384+0000", 2024-09-14T09:05:38.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0023740010000000001 2024-09-14T09:05:38.742 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.742 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.742 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.742 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:9950 3.17 3:eb19e090:::rbd_data.115eadec874.000000000000000a:head [stat,zero 5379443~11397773] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:38.742 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.551136+0000", 2024-09-14T09:05:38.742 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.127003194, 2024-09-14T09:05:38.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0039621170000000002, 2024-09-14T09:05:38.742 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.742 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.742 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:38.742 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:38.743 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:38.743 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.743 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 9950 2024-09-14T09:05:38.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.743 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551136+0000", 2024-09-14T09:05:38.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551136+0000", 2024-09-14T09:05:38.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551136+0000", 2024-09-14T09:05:38.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3000000000000001e-07 2024-09-14T09:05:38.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551139+0000", 2024-09-14T09:05:38.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0010000000000002e-06 2024-09-14T09:05:38.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551147+0000", 2024-09-14T09:05:38.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7379999999999994e-06 2024-09-14T09:05:38.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551153+0000", 2024-09-14T09:05:38.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5300000000000004e-06 2024-09-14T09:05:38.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.552372+0000", 2024-09-14T09:05:38.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001219094 2024-09-14T09:05:38.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.552498+0000", 2024-09-14T09:05:38.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000126453 2024-09-14T09:05:38.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.555098+0000", 2024-09-14T09:05:38.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0025997709999999999 2024-09-14T09:05:38.746 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.746 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.746 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.746 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:9987 3.1 3:81ad4cda:::rbd_data.115eadec874.0000000000000002:head [stat,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:38.746 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.663795+0000", 2024-09-14T09:05:38.746 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.014344189, 2024-09-14T09:05:38.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0053806239999999997, 2024-09-14T09:05:38.747 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.747 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.747 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.747 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:38.747 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:38.747 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.747 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 9987 2024-09-14T09:05:38.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.747 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663795+0000", 2024-09-14T09:05:38.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663795+0000", 2024-09-14T09:05:38.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663798+0000", 2024-09-14T09:05:38.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.01e-06 2024-09-14T09:05:38.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663812+0000", 2024-09-14T09:05:38.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4175999999999999e-05 2024-09-14T09:05:38.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663842+0000", 2024-09-14T09:05:38.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9461000000000001e-05 2024-09-14T09:05:38.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663855+0000", 2024-09-14T09:05:38.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3925e-05 2024-09-14T09:05:38.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663965+0000", 2024-09-14T09:05:38.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109437 2024-09-14T09:05:38.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664066+0000", 2024-09-14T09:05:38.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010084 2024-09-14T09:05:38.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:38.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664218+0000", 2024-09-14T09:05:38.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015209500000000001 2024-09-14T09:05:38.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:38.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.665572+0000", 2024-09-14T09:05:38.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0013543610000000001 2024-09-14T09:05:38.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.669120+0000", 2024-09-14T09:05:38.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035477999999999998 2024-09-14T09:05:38.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:38.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.669146+0000", 2024-09-14T09:05:38.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.618e-05 2024-09-14T09:05:38.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.669175+0000", 2024-09-14T09:05:38.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9339000000000001e-05 2024-09-14T09:05:38.752 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.752 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.752 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.752 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:9992 3.17 3:ecdf2631:::rbd_data.115eadec874.0000000000000007:head [stat,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:38.752 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.663907+0000", 2024-09-14T09:05:38.752 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.0142323369999999, 2024-09-14T09:05:38.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003508129, 2024-09-14T09:05:38.752 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.752 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.753 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.753 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:38.753 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:38.753 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.753 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 9992 2024-09-14T09:05:38.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.753 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663907+0000", 2024-09-14T09:05:38.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663907+0000", 2024-09-14T09:05:38.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663907+0000", 2024-09-14T09:05:38.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.13e-07 2024-09-14T09:05:38.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663912+0000", 2024-09-14T09:05:38.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0239999999999997e-06 2024-09-14T09:05:38.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663921+0000", 2024-09-14T09:05:38.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0860000000000008e-06 2024-09-14T09:05:38.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663927+0000", 2024-09-14T09:05:38.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7490000000000004e-06 2024-09-14T09:05:38.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664065+0000", 2024-09-14T09:05:38.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013774100000000001 2024-09-14T09:05:38.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664274+0000", 2024-09-14T09:05:38.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000209435 2024-09-14T09:05:38.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:05:38.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664664+0000", 2024-09-14T09:05:38.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00038968999999999999 2024-09-14T09:05:38.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:38.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.666044+0000", 2024-09-14T09:05:38.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00138035 2024-09-14T09:05:38.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.667331+0000", 2024-09-14T09:05:38.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012866990000000001 2024-09-14T09:05:38.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:38.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.667378+0000", 2024-09-14T09:05:38.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6594999999999997e-05 2024-09-14T09:05:38.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.667415+0000", 2024-09-14T09:05:38.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7246999999999998e-05 2024-09-14T09:05:38.758 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.758 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.758 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.758 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:9995 3.17 3:eb19e090:::rbd_data.115eadec874.000000000000000a:head [stat,truncate 1195683] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:38.758 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.663952+0000", 2024-09-14T09:05:38.758 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.014187256, 2024-09-14T09:05:38.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00472325, 2024-09-14T09:05:38.758 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.758 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.758 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:38.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:38.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.759 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 9995 2024-09-14T09:05:38.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.759 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663952+0000", 2024-09-14T09:05:38.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663952+0000", 2024-09-14T09:05:38.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663952+0000", 2024-09-14T09:05:38.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4700000000000001e-07 2024-09-14T09:05:38.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663955+0000", 2024-09-14T09:05:38.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5849999999999998e-06 2024-09-14T09:05:38.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663962+0000", 2024-09-14T09:05:38.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9319999999999999e-06 2024-09-14T09:05:38.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663967+0000", 2024-09-14T09:05:38.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.853e-06 2024-09-14T09:05:38.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.665507+0000", 2024-09-14T09:05:38.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001540521 2024-09-14T09:05:38.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.665571+0000", 2024-09-14T09:05:38.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3965000000000002e-05 2024-09-14T09:05:38.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:05:38.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.665686+0000", 2024-09-14T09:05:38.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011476699999999999 2024-09-14T09:05:38.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.667743+0000", 2024-09-14T09:05:38.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0020566360000000001 2024-09-14T09:05:38.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:38.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.668605+0000", 2024-09-14T09:05:38.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00086282899999999996 2024-09-14T09:05:38.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:38.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.668642+0000", 2024-09-14T09:05:38.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6681999999999999e-05 2024-09-14T09:05:38.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.668675+0000", 2024-09-14T09:05:38.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2932999999999998e-05 2024-09-14T09:05:38.763 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.763 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.763 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.763 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:10053 3.17 3:ecdf2631:::rbd_data.115eadec874.0000000000000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-14T09:05:38.764 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.959492+0000", 2024-09-14T09:05:38.764 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.71864682599999996, 2024-09-14T09:05:38.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0037815129999999998, 2024-09-14T09:05:38.764 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.764 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.764 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.764 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:38.764 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:38.764 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.764 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 10053 2024-09-14T09:05:38.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.764 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.959492+0000", 2024-09-14T09:05:38.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.959492+0000", 2024-09-14T09:05:38.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.959494+0000", 2024-09-14T09:05:38.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4410000000000001e-06 2024-09-14T09:05:38.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.959500+0000", 2024-09-14T09:05:38.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9370000000000001e-06 2024-09-14T09:05:38.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.959517+0000", 2024-09-14T09:05:38.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7039999999999999e-05 2024-09-14T09:05:38.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.959525+0000", 2024-09-14T09:05:38.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3289999999999994e-06 2024-09-14T09:05:38.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.959607+0000", 2024-09-14T09:05:38.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2033000000000006e-05 2024-09-14T09:05:38.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.959736+0000", 2024-09-14T09:05:38.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000129009 2024-09-14T09:05:38.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:05:38.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.959910+0000", 2024-09-14T09:05:38.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000173516 2024-09-14T09:05:38.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:38.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.960869+0000", 2024-09-14T09:05:38.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00095911100000000001 2024-09-14T09:05:38.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.963164+0000", 2024-09-14T09:05:38.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0022955710000000002 2024-09-14T09:05:38.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:38.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.963226+0000", 2024-09-14T09:05:38.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1952999999999995e-05 2024-09-14T09:05:38.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.963274+0000", 2024-09-14T09:05:38.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7573e-05 2024-09-14T09:05:38.769 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.769 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.769 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.769 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:10057 3.17 3:eb19e090:::rbd_data.115eadec874.000000000000000a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-14T09:05:38.769 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.962477+0000", 2024-09-14T09:05:38.769 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.71566202300000004, 2024-09-14T09:05:38.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0085064589999999992, 2024-09-14T09:05:38.769 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.769 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.769 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:38.770 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:38.770 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:38.770 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.770 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 10057 2024-09-14T09:05:38.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.770 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.962477+0000", 2024-09-14T09:05:38.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.962477+0000", 2024-09-14T09:05:38.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.962478+0000", 2024-09-14T09:05:38.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0899999999999999e-06 2024-09-14T09:05:38.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.962483+0000", 2024-09-14T09:05:38.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6539999999999998e-06 2024-09-14T09:05:38.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.962498+0000", 2024-09-14T09:05:38.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4819e-05 2024-09-14T09:05:38.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.962505+0000", 2024-09-14T09:05:38.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.807e-06 2024-09-14T09:05:38.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.962564+0000", 2024-09-14T09:05:38.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8912000000000001e-05 2024-09-14T09:05:38.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:38.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.962632+0000", 2024-09-14T09:05:38.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8153999999999998e-05 2024-09-14T09:05:38.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:05:38.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.962709+0000", 2024-09-14T09:05:38.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6625000000000006e-05 2024-09-14T09:05:38.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:38.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.963541+0000", 2024-09-14T09:05:38.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00083169600000000004 2024-09-14T09:05:38.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:38.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.970881+0000", 2024-09-14T09:05:38.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0073404990000000003 2024-09-14T09:05:38.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:38.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.970937+0000", 2024-09-14T09:05:38.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5486999999999999e-05 2024-09-14T09:05:38.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:38.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.970983+0000", 2024-09-14T09:05:38.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6715999999999998e-05 2024-09-14T09:05:38.774 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.774 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.774 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.775 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.775 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.775 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:38.776 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:05:38.777 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:38.772+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:38.877 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:38.872+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:38.889 INFO:teuthology.orchestra.run.smithi176.stderr:reweighted osd.4 to 0.158181 (287e) 2024-09-14T09:05:38.951 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:38.951 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:38.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.951 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:10226 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T09:05:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:38.917408+0000", 2024-09-14T09:05:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.035587256999999997, 2024-09-14T09:05:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035609130000000003, 2024-09-14T09:05:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "waiting for peered", 2024-09-14T09:05:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 10226 2024-09-14T09:05:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.917408+0000", 2024-09-14T09:05:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.917408+0000", 2024-09-14T09:05:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.917410+0000", 2024-09-14T09:05:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6309999999999999e-06 2024-09-14T09:05:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.917422+0000", 2024-09-14T09:05:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1942e-05 2024-09-14T09:05:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.917444+0000", 2024-09-14T09:05:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1506999999999999e-05 2024-09-14T09:05:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.917455+0000", 2024-09-14T09:05:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0730000000000001e-05 2024-09-14T09:05:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.917545+0000", 2024-09-14T09:05:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0328999999999995e-05 2024-09-14T09:05:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:05:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.917556+0000", 2024-09-14T09:05:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1411e-05 2024-09-14T09:05:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:05:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 1 2024-09-14T09:05:38.955 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:38.965 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:05:39.010 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:39.008+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:39.079 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:39.079+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:39.129 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:39.129 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:39.129 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:39.129 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:39.129 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:39.143 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:05:39.194 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:39.192+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:39.263 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:39.263 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:39.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:39.263 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:39.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.264 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4214.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T09:05:39.264 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:18.300171+0000", 2024-09-14T09:05:39.264 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.962184979, 2024-09-14T09:05:39.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018859, 2024-09-14T09:05:39.264 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.264 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.264 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:39.264 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.300171+0000", 2024-09-14T09:05:39.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.300171+0000", 2024-09-14T09:05:39.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.300171+0000", 2024-09-14T09:05:39.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.06e-07 2024-09-14T09:05:39.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.300180+0000", 2024-09-14T09:05:39.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0240000000000003e-06 2024-09-14T09:05:39.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.300225+0000", 2024-09-14T09:05:39.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.456e-05 2024-09-14T09:05:39.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.300251+0000", 2024-09-14T09:05:39.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6679e-05 2024-09-14T09:05:39.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.300445+0000", 2024-09-14T09:05:39.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019341200000000001 2024-09-14T09:05:39.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:39.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.300518+0000", 2024-09-14T09:05:39.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2723000000000002e-05 2024-09-14T09:05:39.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:39.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.301977+0000", 2024-09-14T09:05:39.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0014591649999999999 2024-09-14T09:05:39.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:18.302056+0000", 2024-09-14T09:05:39.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9730999999999999e-05 2024-09-14T09:05:39.267 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.268 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.268 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.268 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:657 3.6 e29/23 3:65b8693c:::rbd_data.115e1e95bf99.0000000000000002:head v 29'2, mlcod=0'0)", 2024-09-14T09:05:39.268 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.872567+0000", 2024-09-14T09:05:39.268 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.389788544, 2024-09-14T09:05:39.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0056247060000000002, 2024-09-14T09:05:39.268 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.268 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.268 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:39.268 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872567+0000", 2024-09-14T09:05:39.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872567+0000", 2024-09-14T09:05:39.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872568+0000", 2024-09-14T09:05:39.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T09:05:39.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872772+0000", 2024-09-14T09:05:39.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020363799999999999 2024-09-14T09:05:39.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872799+0000", 2024-09-14T09:05:39.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6917e-05 2024-09-14T09:05:39.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872815+0000", 2024-09-14T09:05:39.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5812999999999999e-05 2024-09-14T09:05:39.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872928+0000", 2024-09-14T09:05:39.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011381499999999999 2024-09-14T09:05:39.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:39.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872957+0000", 2024-09-14T09:05:39.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9118000000000001e-05 2024-09-14T09:05:39.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:39.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.878076+0000", 2024-09-14T09:05:39.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0051180690000000003 2024-09-14T09:05:39.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.878192+0000", 2024-09-14T09:05:39.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116123 2024-09-14T09:05:39.272 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.272 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.272 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.272 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:963 3.0 3:04479396:::rbd_id.image_client.0-clone2:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T09:05:39.272 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:34.957370+0000", 2024-09-14T09:05:39.272 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.3049856320000002, 2024-09-14T09:05:39.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004759152, 2024-09-14T09:05:39.272 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.272 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.272 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:39.273 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:39.273 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:39.273 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:39.273 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 963 2024-09-14T09:05:39.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.273 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957370+0000", 2024-09-14T09:05:39.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957370+0000", 2024-09-14T09:05:39.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957372+0000", 2024-09-14T09:05:39.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.475e-06 2024-09-14T09:05:39.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957382+0000", 2024-09-14T09:05:39.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9760000000000007e-06 2024-09-14T09:05:39.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957405+0000", 2024-09-14T09:05:39.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2860000000000001e-05 2024-09-14T09:05:39.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957416+0000", 2024-09-14T09:05:39.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.048e-05 2024-09-14T09:05:39.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957529+0000", 2024-09-14T09:05:39.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112849 2024-09-14T09:05:39.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T09:05:39.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957538+0000", 2024-09-14T09:05:39.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.234e-06 2024-09-14T09:05:39.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.960213+0000", 2024-09-14T09:05:39.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0026753749999999998 2024-09-14T09:05:39.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:39.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.960327+0000", 2024-09-14T09:05:39.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114188 2024-09-14T09:05:39.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 2", 2024-09-14T09:05:39.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.960491+0000", 2024-09-14T09:05:39.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016362700000000001 2024-09-14T09:05:39.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:39.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.961472+0000", 2024-09-14T09:05:39.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00098095600000000006 2024-09-14T09:05:39.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:39.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.961990+0000", 2024-09-14T09:05:39.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000518137 2024-09-14T09:05:39.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:39.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.962072+0000", 2024-09-14T09:05:39.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2108999999999996e-05 2024-09-14T09:05:39.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.962129+0000", 2024-09-14T09:05:39.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6885999999999997e-05 2024-09-14T09:05:39.278 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.278 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.278 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.278 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:980 3.6 e30/23 3:67d9142c:::rbd_header.115ed0fcd179:head v 30'11, mlcod=30'9)", 2024-09-14T09:05:39.278 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:34.981610+0000", 2024-09-14T09:05:39.278 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.280745563, 2024-09-14T09:05:39.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0019037710000000001, 2024-09-14T09:05:39.279 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.279 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.279 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:39.279 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.981610+0000", 2024-09-14T09:05:39.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.981610+0000", 2024-09-14T09:05:39.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.981611+0000", 2024-09-14T09:05:39.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6499999999999998e-07 2024-09-14T09:05:39.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.981617+0000", 2024-09-14T09:05:39.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7470000000000002e-06 2024-09-14T09:05:39.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.981631+0000", 2024-09-14T09:05:39.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4803999999999999e-05 2024-09-14T09:05:39.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.981641+0000", 2024-09-14T09:05:39.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4739999999999994e-06 2024-09-14T09:05:39.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.981717+0000", 2024-09-14T09:05:39.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6403999999999996e-05 2024-09-14T09:05:39.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:39.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.981741+0000", 2024-09-14T09:05:39.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3958999999999999e-05 2024-09-14T09:05:39.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:39.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.983459+0000", 2024-09-14T09:05:39.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001717863 2024-09-14T09:05:39.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.983514+0000", 2024-09-14T09:05:39.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4654999999999998e-05 2024-09-14T09:05:39.282 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.282 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.282 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.282 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:9944 3.4 3:245432af:::rbd_data.115eadec874.0000000000000004:head [stat,zero 6291456~12041020] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:39.282 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.550976+0000", 2024-09-14T09:05:39.282 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.71137936, 2024-09-14T09:05:39.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018867420000000001, 2024-09-14T09:05:39.283 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.283 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.283 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:39.283 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:39.283 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:39.283 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:39.283 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 9944 2024-09-14T09:05:39.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.283 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.550976+0000", 2024-09-14T09:05:39.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.550976+0000", 2024-09-14T09:05:39.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.550979+0000", 2024-09-14T09:05:39.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6970000000000002e-06 2024-09-14T09:05:39.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.550989+0000", 2024-09-14T09:05:39.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9329999999999996e-06 2024-09-14T09:05:39.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551019+0000", 2024-09-14T09:05:39.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9740999999999999e-05 2024-09-14T09:05:39.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551034+0000", 2024-09-14T09:05:39.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5406000000000001e-05 2024-09-14T09:05:39.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551119+0000", 2024-09-14T09:05:39.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5277000000000004e-05 2024-09-14T09:05:39.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:39.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551244+0000", 2024-09-14T09:05:39.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012489900000000001 2024-09-14T09:05:39.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.552863+0000", 2024-09-14T09:05:39.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001618789 2024-09-14T09:05:39.286 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.286 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.286 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.286 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:9948 3.19 3:9ed4ec6a:::rbd_data.115eadec874.0000000000000008:head [stat,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:39.286 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.551083+0000", 2024-09-14T09:05:39.287 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.711272935, 2024-09-14T09:05:39.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0021132999999999998, 2024-09-14T09:05:39.287 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.287 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.287 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:39.287 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:39.287 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:39.287 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:39.287 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 9948 2024-09-14T09:05:39.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.287 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551083+0000", 2024-09-14T09:05:39.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551083+0000", 2024-09-14T09:05:39.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551083+0000", 2024-09-14T09:05:39.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1299999999999998e-07 2024-09-14T09:05:39.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551088+0000", 2024-09-14T09:05:39.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3830000000000002e-06 2024-09-14T09:05:39.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551099+0000", 2024-09-14T09:05:39.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1473e-05 2024-09-14T09:05:39.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551111+0000", 2024-09-14T09:05:39.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1401e-05 2024-09-14T09:05:39.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551181+0000", 2024-09-14T09:05:39.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0114e-05 2024-09-14T09:05:39.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:39.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551275+0000", 2024-09-14T09:05:39.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4573000000000004e-05 2024-09-14T09:05:39.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:05:39.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551443+0000", 2024-09-14T09:05:39.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016756200000000001 2024-09-14T09:05:39.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:39.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.553047+0000", 2024-09-14T09:05:39.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001604463 2024-09-14T09:05:39.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:39.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.553126+0000", 2024-09-14T09:05:39.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8887999999999994e-05 2024-09-14T09:05:39.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:39.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.553158+0000", 2024-09-14T09:05:39.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1754000000000001e-05 2024-09-14T09:05:39.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.553196+0000", 2024-09-14T09:05:39.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7775999999999999e-05 2024-09-14T09:05:39.291 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.291 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.292 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.292 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:9982 3.1a e32/23 3:5af3e476:::rbd_data.115eadec874.000000000000000c:head v 32'20, mlcod=32'18)", 2024-09-14T09:05:39.292 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.603737+0000", 2024-09-14T09:05:39.292 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.6586187379999999, 2024-09-14T09:05:39.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001966958, 2024-09-14T09:05:39.292 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.292 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.292 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:39.292 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.603737+0000", 2024-09-14T09:05:39.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.603737+0000", 2024-09-14T09:05:39.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.603738+0000", 2024-09-14T09:05:39.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3600000000000002e-07 2024-09-14T09:05:39.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.604456+0000", 2024-09-14T09:05:39.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00071852600000000004 2024-09-14T09:05:39.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.604471+0000", 2024-09-14T09:05:39.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4602e-05 2024-09-14T09:05:39.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.604481+0000", 2024-09-14T09:05:39.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0564000000000001e-05 2024-09-14T09:05:39.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.604579+0000", 2024-09-14T09:05:39.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7775999999999994e-05 2024-09-14T09:05:39.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:39.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.604596+0000", 2024-09-14T09:05:39.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6524e-05 2024-09-14T09:05:39.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:39.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.605671+0000", 2024-09-14T09:05:39.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001075735 2024-09-14T09:05:39.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.605704+0000", 2024-09-14T09:05:39.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2295000000000002e-05 2024-09-14T09:05:39.296 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.296 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.296 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.296 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:9983 3.1a e32/23 3:5a62ac37:::rbd_data.115eadec874.000000000000000d:head v 32'21, mlcod=32'19)", 2024-09-14T09:05:39.296 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.639585+0000", 2024-09-14T09:05:39.296 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.62277089, 2024-09-14T09:05:39.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0046068979999999999, 2024-09-14T09:05:39.296 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.296 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.296 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:39.296 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639585+0000", 2024-09-14T09:05:39.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639585+0000", 2024-09-14T09:05:39.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639586+0000", 2024-09-14T09:05:39.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6559999999999999e-06 2024-09-14T09:05:39.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639629+0000", 2024-09-14T09:05:39.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2713000000000003e-05 2024-09-14T09:05:39.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639657+0000", 2024-09-14T09:05:39.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7569999999999999e-05 2024-09-14T09:05:39.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639675+0000", 2024-09-14T09:05:39.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8647e-05 2024-09-14T09:05:39.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639791+0000", 2024-09-14T09:05:39.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116229 2024-09-14T09:05:39.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:39.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639829+0000", 2024-09-14T09:05:39.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7530000000000002e-05 2024-09-14T09:05:39.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:39.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.644164+0000", 2024-09-14T09:05:39.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0043347530000000002 2024-09-14T09:05:39.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.644192+0000", 2024-09-14T09:05:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7800000000000001e-05 2024-09-14T09:05:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:9989 3.4 3:245432af:::rbd_data.115eadec874.0000000000000004:head [stat,zero 2097152~9278230] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.663820+0000", 2024-09-14T09:05:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.5985357439999999, 2024-09-14T09:05:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0021431390000000001, 2024-09-14T09:05:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:39.300 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:39.301 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:39.301 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:39.301 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 9989 2024-09-14T09:05:39.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.301 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663820+0000", 2024-09-14T09:05:39.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663820+0000", 2024-09-14T09:05:39.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663823+0000", 2024-09-14T09:05:39.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0189999999999998e-06 2024-09-14T09:05:39.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663832+0000", 2024-09-14T09:05:39.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0669999999999998e-06 2024-09-14T09:05:39.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663864+0000", 2024-09-14T09:05:39.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2498e-05 2024-09-14T09:05:39.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663875+0000", 2024-09-14T09:05:39.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0878e-05 2024-09-14T09:05:39.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664009+0000", 2024-09-14T09:05:39.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000133831 2024-09-14T09:05:39.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:39.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664083+0000", 2024-09-14T09:05:39.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3509000000000004e-05 2024-09-14T09:05:39.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.665963+0000", 2024-09-14T09:05:39.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018803369999999999 2024-09-14T09:05:39.304 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.304 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.304 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.304 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:9993 3.19 3:9ed4ec6a:::rbd_data.115eadec874.0000000000000008:head [stat,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:39.304 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.663928+0000", 2024-09-14T09:05:39.304 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.598427351, 2024-09-14T09:05:39.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0021940670000000001, 2024-09-14T09:05:39.304 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.304 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.305 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:39.305 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:39.305 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:39.305 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:39.305 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 9993 2024-09-14T09:05:39.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.305 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663928+0000", 2024-09-14T09:05:39.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663928+0000", 2024-09-14T09:05:39.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663929+0000", 2024-09-14T09:05:39.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2200000000000004e-07 2024-09-14T09:05:39.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663932+0000", 2024-09-14T09:05:39.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9529999999999999e-06 2024-09-14T09:05:39.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663940+0000", 2024-09-14T09:05:39.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9319999999999996e-06 2024-09-14T09:05:39.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663946+0000", 2024-09-14T09:05:39.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8599999999999998e-06 2024-09-14T09:05:39.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664029+0000", 2024-09-14T09:05:39.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3504999999999994e-05 2024-09-14T09:05:39.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:39.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664103+0000", 2024-09-14T09:05:39.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3763999999999994e-05 2024-09-14T09:05:39.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:05:39.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664228+0000", 2024-09-14T09:05:39.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012475999999999999 2024-09-14T09:05:39.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:39.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.665460+0000", 2024-09-14T09:05:39.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001232431 2024-09-14T09:05:39.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:39.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.666046+0000", 2024-09-14T09:05:39.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00058552799999999998 2024-09-14T09:05:39.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:39.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.666087+0000", 2024-09-14T09:05:39.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1393999999999997e-05 2024-09-14T09:05:39.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.666122+0000", 2024-09-14T09:05:39.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5318000000000003e-05 2024-09-14T09:05:39.309 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.309 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.309 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.309 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:10029 3.1a e32/23 3:5a62ac37:::rbd_data.115eadec874.000000000000000d:head v 32'25, mlcod=32'23)", 2024-09-14T09:05:39.310 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.762577+0000", 2024-09-14T09:05:39.310 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.4997781909999999, 2024-09-14T09:05:39.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002748028, 2024-09-14T09:05:39.310 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.310 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.310 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:39.310 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.762577+0000", 2024-09-14T09:05:39.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.762577+0000", 2024-09-14T09:05:39.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.762580+0000", 2024-09-14T09:05:39.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4269999999999998e-06 2024-09-14T09:05:39.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.762681+0000", 2024-09-14T09:05:39.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100869 2024-09-14T09:05:39.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.762709+0000", 2024-09-14T09:05:39.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8243e-05 2024-09-14T09:05:39.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.762722+0000", 2024-09-14T09:05:39.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2935999999999999e-05 2024-09-14T09:05:39.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.762863+0000", 2024-09-14T09:05:39.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014083300000000001 2024-09-14T09:05:39.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:39.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.762922+0000", 2024-09-14T09:05:39.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9373000000000002e-05 2024-09-14T09:05:39.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:39.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.765221+0000", 2024-09-14T09:05:39.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0022990179999999999 2024-09-14T09:05:39.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.765325+0000", 2024-09-14T09:05:39.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000104329 2024-09-14T09:05:39.313 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.313 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.313 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.314 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:10032 3.19 3:9ed4ec6a:::rbd_data.115eadec874.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:39.314 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.775706+0000", 2024-09-14T09:05:39.314 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.4866491829999999, 2024-09-14T09:05:39.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126247939, 2024-09-14T09:05:39.314 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.314 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.314 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:39.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:39.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:39.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:39.314 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 10032 2024-09-14T09:05:39.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.314 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775706+0000", 2024-09-14T09:05:39.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775706+0000", 2024-09-14T09:05:39.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775709+0000", 2024-09-14T09:05:39.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0400000000000001e-06 2024-09-14T09:05:39.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775721+0000", 2024-09-14T09:05:39.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2119999999999999e-05 2024-09-14T09:05:39.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775752+0000", 2024-09-14T09:05:39.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0139999999999999e-05 2024-09-14T09:05:39.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775766+0000", 2024-09-14T09:05:39.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4497e-05 2024-09-14T09:05:39.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775937+0000", 2024-09-14T09:05:39.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000171022 2024-09-14T09:05:39.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:39.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.776205+0000", 2024-09-14T09:05:39.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026749100000000002 2024-09-14T09:05:39.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:05:39.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.784970+0000", 2024-09-14T09:05:39.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0087648789999999997 2024-09-14T09:05:39.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:39.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901870+0000", 2024-09-14T09:05:39.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11690028 2024-09-14T09:05:39.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:39.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901878+0000", 2024-09-14T09:05:39.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7020000000000007e-06 2024-09-14T09:05:39.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:39.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901913+0000", 2024-09-14T09:05:39.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4706000000000003e-05 2024-09-14T09:05:39.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901954+0000", 2024-09-14T09:05:39.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1062000000000001e-05 2024-09-14T09:05:39.319 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.319 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.319 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.319 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:10032 3.19 e32/23 ondisk, result = 0)", 2024-09-14T09:05:39.319 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.809095+0000", 2024-09-14T09:05:39.319 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.453260904, 2024-09-14T09:05:39.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092866664000000002, 2024-09-14T09:05:39.320 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.320 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.320 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:39.320 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809095+0000", 2024-09-14T09:05:39.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809095+0000", 2024-09-14T09:05:39.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809096+0000", 2024-09-14T09:05:39.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.64e-06 2024-09-14T09:05:39.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809104+0000", 2024-09-14T09:05:39.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7330000000000003e-06 2024-09-14T09:05:39.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809132+0000", 2024-09-14T09:05:39.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7880000000000001e-05 2024-09-14T09:05:39.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809146+0000", 2024-09-14T09:05:39.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4514e-05 2024-09-14T09:05:39.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901858+0000", 2024-09-14T09:05:39.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092711713000000001 2024-09-14T09:05:39.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:39.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901866+0000", 2024-09-14T09:05:39.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4760000000000001e-06 2024-09-14T09:05:39.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901961+0000", 2024-09-14T09:05:39.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5707999999999999e-05 2024-09-14T09:05:39.323 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.323 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.323 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.323 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:10035 3.4 3:245432af:::rbd_data.115eadec874.0000000000000004:head [stat,write 9104681~23750] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-14T09:05:39.323 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.907563+0000", 2024-09-14T09:05:39.323 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.354792314, 2024-09-14T09:05:39.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0029432270000000001, 2024-09-14T09:05:39.323 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.323 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.323 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:39.323 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:39.324 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:39.324 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:39.324 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 10035 2024-09-14T09:05:39.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.324 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.907563+0000", 2024-09-14T09:05:39.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.907563+0000", 2024-09-14T09:05:39.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.907566+0000", 2024-09-14T09:05:39.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6089999999999999e-06 2024-09-14T09:05:39.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.907601+0000", 2024-09-14T09:05:39.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4882000000000002e-05 2024-09-14T09:05:39.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.907624+0000", 2024-09-14T09:05:39.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3090000000000001e-05 2024-09-14T09:05:39.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.907635+0000", 2024-09-14T09:05:39.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1583000000000001e-05 2024-09-14T09:05:39.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.907721+0000", 2024-09-14T09:05:39.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6099000000000004e-05 2024-09-14T09:05:39.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:39.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.907837+0000", 2024-09-14T09:05:39.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011521799999999999 2024-09-14T09:05:39.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 2", 2024-09-14T09:05:39.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.908022+0000", 2024-09-14T09:05:39.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018563100000000001 2024-09-14T09:05:39.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:39.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.910209+0000", 2024-09-14T09:05:39.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002186871 2024-09-14T09:05:39.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:39.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.910439+0000", 2024-09-14T09:05:39.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000230163 2024-09-14T09:05:39.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:39.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.910475+0000", 2024-09-14T09:05:39.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5867000000000001e-05 2024-09-14T09:05:39.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.910506+0000", 2024-09-14T09:05:39.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1214000000000003e-05 2024-09-14T09:05:39.328 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.328 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.328 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.328 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:10043 3.4 3:245432af:::rbd_data.115eadec874.0000000000000004:head [stat,truncate 232003] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-14T09:05:39.328 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.949037+0000", 2024-09-14T09:05:39.328 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.313318204, 2024-09-14T09:05:39.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0019111200000000001, 2024-09-14T09:05:39.329 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.329 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.329 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:39.329 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:39.329 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:39.329 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:39.329 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 10043 2024-09-14T09:05:39.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.329 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.949037+0000", 2024-09-14T09:05:39.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.949037+0000", 2024-09-14T09:05:39.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.949040+0000", 2024-09-14T09:05:39.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6359999999999998e-06 2024-09-14T09:05:39.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.949053+0000", 2024-09-14T09:05:39.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3055e-05 2024-09-14T09:05:39.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.949078+0000", 2024-09-14T09:05:39.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5151999999999999e-05 2024-09-14T09:05:39.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.949089+0000", 2024-09-14T09:05:39.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0757000000000001e-05 2024-09-14T09:05:39.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.949205+0000", 2024-09-14T09:05:39.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011572400000000001 2024-09-14T09:05:39.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:39.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.949295+0000", 2024-09-14T09:05:39.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0320000000000003e-05 2024-09-14T09:05:39.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 2", 2024-09-14T09:05:39.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.949447+0000", 2024-09-14T09:05:39.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000152304 2024-09-14T09:05:39.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:39.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.950633+0000", 2024-09-14T09:05:39.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0011855069999999999 2024-09-14T09:05:39.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:39.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.950811+0000", 2024-09-14T09:05:39.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017793999999999999 2024-09-14T09:05:39.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:39.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.950887+0000", 2024-09-14T09:05:39.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6746e-05 2024-09-14T09:05:39.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.950948+0000", 2024-09-14T09:05:39.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0979e-05 2024-09-14T09:05:39.333 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.334 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.334 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.334 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:10166 3.4 3:245432af:::rbd_data.115eadec874.0000000000000004:head [stat,zero 8388608~6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-14T09:05:39.334 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:38.157947+0000", 2024-09-14T09:05:39.334 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.104408944, 2024-09-14T09:05:39.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0028609999999999998, 2024-09-14T09:05:39.334 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.334 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.334 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:39.334 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:39.334 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:39.335 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:39.335 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 10166 2024-09-14T09:05:39.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.335 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.157947+0000", 2024-09-14T09:05:39.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.157947+0000", 2024-09-14T09:05:39.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.157949+0000", 2024-09-14T09:05:39.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5330000000000001e-06 2024-09-14T09:05:39.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.157958+0000", 2024-09-14T09:05:39.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3430000000000002e-06 2024-09-14T09:05:39.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.157986+0000", 2024-09-14T09:05:39.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7719e-05 2024-09-14T09:05:39.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158010+0000", 2024-09-14T09:05:39.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4009e-05 2024-09-14T09:05:39.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158163+0000", 2024-09-14T09:05:39.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000153051 2024-09-14T09:05:39.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:39.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158379+0000", 2024-09-14T09:05:39.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021535799999999999 2024-09-14T09:05:39.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.160808+0000", 2024-09-14T09:05:39.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0024289870000000001 2024-09-14T09:05:39.338 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.338 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.338 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.338 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:10170 3.1a e33/23 3:5af3e476:::rbd_data.115eadec874.000000000000000c:head v 33'29, mlcod=33'27)", 2024-09-14T09:05:39.339 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:38.158629+0000", 2024-09-14T09:05:39.339 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.103726626, 2024-09-14T09:05:39.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0036077679999999999, 2024-09-14T09:05:39.339 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.339 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.339 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:39.339 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158629+0000", 2024-09-14T09:05:39.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158629+0000", 2024-09-14T09:05:39.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158631+0000", 2024-09-14T09:05:39.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5170000000000002e-06 2024-09-14T09:05:39.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158648+0000", 2024-09-14T09:05:39.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6520000000000001e-05 2024-09-14T09:05:39.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158678+0000", 2024-09-14T09:05:39.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.006e-05 2024-09-14T09:05:39.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158697+0000", 2024-09-14T09:05:39.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9145999999999999e-05 2024-09-14T09:05:39.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158911+0000", 2024-09-14T09:05:39.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021399799999999999 2024-09-14T09:05:39.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:39.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158964+0000", 2024-09-14T09:05:39.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2846999999999997e-05 2024-09-14T09:05:39.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:39.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.162172+0000", 2024-09-14T09:05:39.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032080289999999998 2024-09-14T09:05:39.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.162237+0000", 2024-09-14T09:05:39.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4651000000000002e-05 2024-09-14T09:05:39.342 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.343 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.343 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.343 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:10182 3.1a e33/23 3:5a62ac37:::rbd_data.115eadec874.000000000000000d:head v 33'30, mlcod=33'28)", 2024-09-14T09:05:39.343 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:38.175853+0000", 2024-09-14T09:05:39.343 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.0865029559999999, 2024-09-14T09:05:39.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0061618740000000003, 2024-09-14T09:05:39.343 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.343 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.343 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:39.343 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175853+0000", 2024-09-14T09:05:39.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175853+0000", 2024-09-14T09:05:39.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175854+0000", 2024-09-14T09:05:39.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8169999999999999e-06 2024-09-14T09:05:39.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175914+0000", 2024-09-14T09:05:39.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9812999999999998e-05 2024-09-14T09:05:39.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175941+0000", 2024-09-14T09:05:39.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6786000000000001e-05 2024-09-14T09:05:39.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175955+0000", 2024-09-14T09:05:39.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.45e-05 2024-09-14T09:05:39.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.176049+0000", 2024-09-14T09:05:39.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3801000000000005e-05 2024-09-14T09:05:39.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:39.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.176083+0000", 2024-09-14T09:05:39.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.362e-05 2024-09-14T09:05:39.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:39.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.181952+0000", 2024-09-14T09:05:39.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0058688170000000001 2024-09-14T09:05:39.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.182014+0000", 2024-09-14T09:05:39.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2719999999999996e-05 2024-09-14T09:05:39.347 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.347 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.347 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.347 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "pg_update_log_missing(3.1a epoch 33/23 rep_tid 178 entries 33'31 (0'0) error 3:5af3e476:::rbd_data.115eadec874.000000000000000c:head by client.4446.0:10201 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 33'29)", 2024-09-14T09:05:39.347 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:38.217229+0000", 2024-09-14T09:05:39.347 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.045126644, 2024-09-14T09:05:39.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0029007220000000001, 2024-09-14T09:05:39.347 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.347 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.347 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:39.347 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217229+0000", 2024-09-14T09:05:39.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217229+0000", 2024-09-14T09:05:39.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217230+0000", 2024-09-14T09:05:39.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.049e-06 2024-09-14T09:05:39.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217240+0000", 2024-09-14T09:05:39.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9159999999999996e-06 2024-09-14T09:05:39.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217269+0000", 2024-09-14T09:05:39.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9377e-05 2024-09-14T09:05:39.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217285+0000", 2024-09-14T09:05:39.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6212e-05 2024-09-14T09:05:39.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217399+0000", 2024-09-14T09:05:39.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000113989 2024-09-14T09:05:39.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.220130+0000", 2024-09-14T09:05:39.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0027301790000000001 2024-09-14T09:05:39.350 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.350 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.350 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.350 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "pg_update_log_missing(3.1a epoch 33/23 rep_tid 180 entries 33'32 (0'0) error 3:5a62ac37:::rbd_data.115eadec874.000000000000000d:head by client.4446.0:10202 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 33'29)", 2024-09-14T09:05:39.350 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:38.217605+0000", 2024-09-14T09:05:39.351 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.0447500599999999, 2024-09-14T09:05:39.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002797384, 2024-09-14T09:05:39.351 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:39.351 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:39.351 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:39.351 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:39.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:39.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217605+0000", 2024-09-14T09:05:39.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:39.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217605+0000", 2024-09-14T09:05:39.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:39.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:39.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217606+0000", 2024-09-14T09:05:39.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5000000000000002e-07 2024-09-14T09:05:39.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:39.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217614+0000", 2024-09-14T09:05:39.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7440000000000007e-06 2024-09-14T09:05:39.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:39.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217639+0000", 2024-09-14T09:05:39.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4811e-05 2024-09-14T09:05:39.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:39.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217654+0000", 2024-09-14T09:05:39.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5169e-05 2024-09-14T09:05:39.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:39.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217822+0000", 2024-09-14T09:05:39.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016851 2024-09-14T09:05:39.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:39.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:39.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:39.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.220403+0000", 2024-09-14T09:05:39.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0025804999999999999 2024-09-14T09:05:39.353 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.353 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.354 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.354 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:39.354 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:39.354 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:39.355 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:05:39.355 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:39.352+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:39.456 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:39.452+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:39.527 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:39.527 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:39.541 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:05:39.556 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:39.559+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:39.657 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:39.652+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:39.669 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:05:39.708 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:39.708 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:39.708 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:39.708 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:39.708 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:39.723 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:05:39.766 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:39.764+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:39.813 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:05:39.813 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:05:39.830 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:05:39.830 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:05:39.847 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:05:39.847 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:05:39.858 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:39.859+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:39.863 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:05:39.864 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:05:39.884 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:05:39.884 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:05:39.890 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:39.891 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:39.912 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:05:39.914 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:05:39.915 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:05:39.916 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T09:05:39.947 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:05:39.947 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:05:39.975 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:05:39.976 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:05:39.976 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:39.972+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:40.039 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone3 from 0x626cdb3 (overlap 0x838a43) to 0x1a4eda 2024-09-14T09:05:40.046 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:40.046 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:40.060 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:40.056+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:40.064 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:05:40.161 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:40.156+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:40.185 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-14T09:05:40.226 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:40.227 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:40.245 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:05:40.288 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:40.284+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:40.363 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:40.360+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:40.465 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:40.468+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:40.497 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:40.498 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:40.522 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:05:40.566 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:40.564+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:40.641 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:40.641 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:40.656 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:05:40.666 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:40.664+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:40.767 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:40.768+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:40.781 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:40.781 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:40.795 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:05:40.880 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:40.884+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:40.904 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is set 2024-09-14T09:05:40.967 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:40.967 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:40.969 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:40.972+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:40.981 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:05:41.070 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:41.072+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:41.142 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:41.142 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:41.156 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:05:41.171 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:41.168+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:41.221 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:41.221 INFO:teuthology.orchestra.run.smithi176.stdout:428 read 0xb8dd96 thru 0xb967b6 (0x8a21 bytes) 2024-09-14T09:05:41.223 INFO:teuthology.orchestra.run.smithi176.stdout:429 punch from 0x4fe8d42 to 0x4fe9a0d, (0xccb bytes) 2024-09-14T09:05:41.248 INFO:teuthology.orchestra.run.smithi176.stdout:430 write 0xd2c7b7d thru 0xd2d7276 (0xf6fa bytes) 2024-09-14T09:05:41.254 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:41.254 INFO:teuthology.orchestra.run.smithi176.stdout:434 read 0x5a2907f thru 0x5a30314 (0x7296 bytes) 2024-09-14T09:05:41.255 INFO:teuthology.orchestra.run.smithi176.stdout:435 read 0x5db8e2b thru 0x5dc69ec (0xdbc2 bytes) 2024-09-14T09:05:41.255 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:41.255 INFO:teuthology.orchestra.run.smithi176.stdout:438 writesame 0x25ccaec thru 0x25d62a1 data_size 0x3(0x97b6 bytes) 2024-09-14T09:05:41.272 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:41.268+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:41.329 INFO:teuthology.orchestra.run.smithi176.stdout:439 writesame 0x51ed9c1 thru 0x51f1b61 data_size 0x1(0x41a1 bytes) 2024-09-14T09:05:41.329 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:41.329 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:41.346 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:05:41.373 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:41.368+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:41.409 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:41.409 INFO:teuthology.orchestra.run.smithi176.stdout:441 write 0x9ad1ae3 thru 0x9ad23b3 (0x8d1 bytes) 2024-09-14T09:05:41.416 INFO:teuthology.orchestra.run.smithi176.stdout:442 writesame 0xa55925c thru 0xa565aff data_size 0x2(0xc8a4 bytes) 2024-09-14T09:05:41.451 INFO:teuthology.orchestra.run.smithi176.stdout:443 write 0xda7a85 thru 0xdabde7 (0x4363 bytes) 2024-09-14T09:05:41.470 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:41.470 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:41.474 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:41.476+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:41.487 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:05:41.525 INFO:teuthology.orchestra.run.smithi176.stdout:444 write 0x1d0000f thru 0x1d0be8f (0xbe81 bytes) 2024-09-14T09:05:41.574 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:41.576+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:41.603 INFO:teuthology.orchestra.run.smithi176.stdout:445 read 0x1e9bb88 thru 0x1ea16ec (0x5b65 bytes) 2024-09-14T09:05:41.604 INFO:teuthology.orchestra.run.smithi176.stdout:447 write 0x6f3c8ae thru 0x6f4aa34 (0xe187 bytes) 2024-09-14T09:05:41.618 INFO:teuthology.orchestra.run.smithi176.stdout:448 writesame 0x5829bd6 thru 0x5831cdd data_size 0x4(0x8108 bytes) 2024-09-14T09:05:41.635 INFO:teuthology.orchestra.run.smithi176.stdout:449 trunc from 0xd2d7277 to 0xcaacf1f 2024-09-14T09:05:41.649 INFO:teuthology.orchestra.run.smithi176.stdout:452 read 0x595b6ed thru 0x59601e5 (0x4af9 bytes) 2024-09-14T09:05:41.650 INFO:teuthology.orchestra.run.smithi176.stdout:453 read 0xabcc8a thru 0xaca2ee (0xd665 bytes) 2024-09-14T09:05:41.651 INFO:teuthology.orchestra.run.smithi176.stdout:454 write 0xca96940 thru 0xcaa6344 (0xfa05 bytes) 2024-09-14T09:05:41.658 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:41.658 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:41.661 INFO:teuthology.orchestra.run.smithi176.stdout:455 read 0xa67258 thru 0xa77111 (0xfeba bytes) 2024-09-14T09:05:41.673 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:05:41.676 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:41.672+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:41.679 INFO:teuthology.orchestra.run.smithi176.stdout:457 write 0xe7f9f15 thru 0xe802458 (0x8544 bytes) 2024-09-14T09:05:41.687 INFO:teuthology.orchestra.run.smithi176.stdout:458 read 0xa85bea2 thru 0xa85e35e (0x24bd bytes) 2024-09-14T09:05:41.688 INFO:teuthology.orchestra.run.smithi176.stdout:459 write 0x7c8afd5 thru 0x7c8d72a (0x2756 bytes) 2024-09-14T09:05:41.692 INFO:teuthology.orchestra.run.smithi176.stdout:460 writesame 0x1d02c9b thru 0x1d0e6c3 data_size 0x1(0xba29 bytes) 2024-09-14T09:05:41.777 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:41.772+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:41.786 INFO:teuthology.orchestra.run.smithi176.stdout:461 read 0x18ccea9 thru 0x18dbc0c (0xed64 bytes) 2024-09-14T09:05:41.787 INFO:teuthology.orchestra.run.smithi176.stdout:463 punch from 0x2dc6419 to 0x2dce621, (0x8208 bytes) 2024-09-14T09:05:41.787 INFO:teuthology.orchestra.run.smithi176.stdout:464 write 0xdcb7cb8 thru 0xdcbed63 (0x70ac bytes) 2024-09-14T09:05:41.793 INFO:teuthology.orchestra.run.smithi176.stdout:465 write 0xe36393d thru 0xe371fab (0xe66f bytes) 2024-09-14T09:05:41.803 INFO:teuthology.orchestra.run.smithi176.stdout:467 write 0x6d4bb3 thru 0x6d7e5b (0x32a9 bytes) 2024-09-14T09:05:41.807 INFO:teuthology.orchestra.run.smithi176.stdout:468 punch from 0x262dfcf to 0x2638806, (0xa837 bytes) 2024-09-14T09:05:41.807 INFO:teuthology.orchestra.run.smithi176.stdout:470 punch from 0x1823468 to 0x1825cc9, (0x2861 bytes) 2024-09-14T09:05:41.807 INFO:teuthology.orchestra.run.smithi176.stdout:471 write 0x6af77f0 thru 0x6b05fae (0xe7bf bytes) 2024-09-14T09:05:41.817 INFO:teuthology.orchestra.run.smithi176.stdout:472 punch from 0x66fc1c5 to 0x67037e6, (0x7621 bytes) 2024-09-14T09:05:41.817 INFO:teuthology.orchestra.run.smithi176.stdout:473 read 0x9b7efc3 thru 0x9b83854 (0x4892 bytes) 2024-09-14T09:05:41.817 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:41.817 INFO:teuthology.orchestra.run.smithi176.stdout:477 read 0x8ee680d thru 0x8eebfed (0x57e1 bytes) 2024-09-14T09:05:41.833 INFO:teuthology.orchestra.run.smithi176.stdout:482 trunc from 0xe802459 to 0x8d9e3ef 2024-09-14T09:05:41.848 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:41.848 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:41.863 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:05:41.867 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:41.867 INFO:teuthology.orchestra.run.smithi176.stdout:484 write 0x7781b5b thru 0x7787e44 (0x62ea bytes) 2024-09-14T09:05:41.892 INFO:teuthology.orchestra.run.smithi176.stdout:486 writesame 0xa6bdb17 thru 0xa6c5755 data_size 0x1(0x7c3f bytes) 2024-09-14T09:05:41.892 INFO:teuthology.orchestra.run.smithi176.stdout:488 read 0x1a88f3a thru 0x1a94586 (0xb64d bytes) 2024-09-14T09:05:41.895 INFO:teuthology.orchestra.run.smithi176.stdout:489 trunc from 0xa6c5756 to 0x9cb481b 2024-09-14T09:05:41.912 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:41.916+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:41.915 INFO:teuthology.orchestra.run.smithi176.stdout:491 read 0x4ef08dd thru 0x4ef19a3 (0x10c7 bytes) 2024-09-14T09:05:41.916 INFO:teuthology.orchestra.run.smithi176.stdout:492 write 0x6adeb7d thru 0x6ae6ddb (0x825f bytes) 2024-09-14T09:05:41.920 INFO:teuthology.orchestra.run.smithi176.stdout:493 punch from 0x25625c2 to 0x256459f, (0x1fdd bytes) 2024-09-14T09:05:41.920 INFO:teuthology.orchestra.run.smithi176.stdout:494 read 0x27aa31f thru 0x27b8173 (0xde55 bytes) 2024-09-14T09:05:41.921 INFO:teuthology.orchestra.run.smithi176.stdout:498 writesame 0x9769af thru 0x980652 data_size 0x19(0x9ca4 bytes) 2024-09-14T09:05:41.931 INFO:teuthology.orchestra.run.smithi176.stdout:499 trunc from 0x9cb481b to 0x7fd4046 2024-09-14T09:05:41.980 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:41.980+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:41.989 INFO:teuthology.orchestra.run.smithi176.stdout:500 write 0xd2129f5 thru 0xd214d51 (0x235d bytes) 2024-09-14T09:05:41.993 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:41.993 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:41.996 INFO:teuthology.orchestra.run.smithi176.stdout:502 read 0x3ed1904 thru 0x3edbf44 (0xa641 bytes) 2024-09-14T09:05:41.997 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:42.001 INFO:teuthology.orchestra.run.smithi176.stdout:506 trunc from 0xd214d52 to 0xb9849b5 2024-09-14T09:05:42.006 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:05:42.007 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:05:42.022 INFO:teuthology.orchestra.run.smithi176.stdout:507 write 0x6479e04 thru 0x6485431 (0xb62e bytes) 2024-09-14T09:05:42.026 INFO:teuthology.orchestra.run.smithi176.stdout:509 read 0x37c7057 thru 0x37cbe9d (0x4e47 bytes) 2024-09-14T09:05:42.027 INFO:teuthology.orchestra.run.smithi176.stdout:511 read 0x3b199dc thru 0x3b24535 (0xab5a bytes) 2024-09-14T09:05:42.034 INFO:teuthology.orchestra.run.smithi176.stdout:512 trunc from 0xb9849b5 to 0x97bc03f 2024-09-14T09:05:42.045 INFO:teuthology.orchestra.run.smithi176.stdout:513 punch from 0x6322b75 to 0x632f131, (0xc5bc bytes) 2024-09-14T09:05:42.045 INFO:teuthology.orchestra.run.smithi176.stdout:515 write 0x307dd9 thru 0x311881 (0x9aa9 bytes) 2024-09-14T09:05:42.052 INFO:teuthology.orchestra.run.smithi176.stdout:516 punch from 0x218633f to 0x218ce37, (0x6af8 bytes) 2024-09-14T09:05:42.052 INFO:teuthology.orchestra.run.smithi176.stdout:517 read 0x6e37d04 thru 0x6e3b1c1 (0x34be bytes) 2024-09-14T09:05:42.053 INFO:teuthology.orchestra.run.smithi176.stdout:518 write 0xb3a2f thru 0xbefbf (0xb591 bytes) 2024-09-14T09:05:42.061 INFO:teuthology.orchestra.run.smithi176.stdout:519 writesame 0x833fe21 thru 0x8348148 data_size 0xc(0x8328 bytes) 2024-09-14T09:05:42.070 INFO:teuthology.orchestra.run.smithi176.stdout:520 writesame 0x5075d1b thru 0x508283b data_size 0x1(0xcb21 bytes) 2024-09-14T09:05:42.081 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:42.076+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:42.173 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:42.173 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:42.181 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:05:42.182 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:05:42.182 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:42.180+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:42.187 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:05:42.189 INFO:teuthology.orchestra.run.smithi176.stdout:521 trunc from 0x97bc03f to 0x2ced2ad 2024-09-14T09:05:42.202 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:05:42.202 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:05:42.216 INFO:teuthology.orchestra.run.smithi176.stdout:523 trunc from 0x2ced2ad to 0x47fccc0 2024-09-14T09:05:42.221 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:05:42.221 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:05:42.223 INFO:teuthology.orchestra.run.smithi176.stdout:525 punch from 0x4689320 to 0x468e992, (0x5672 bytes) 2024-09-14T09:05:42.232 INFO:teuthology.orchestra.run.smithi176.stdout:526 write 0x63793a3 thru 0x637b4cc (0x212a bytes) 2024-09-14T09:05:42.236 INFO:teuthology.orchestra.run.smithi176.stdout:528 punch from 0x35221cf to 0x352f11d, (0xcf4e bytes) 2024-09-14T09:05:42.242 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:05:42.242 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:05:42.253 INFO:teuthology.orchestra.run.smithi176.stdout:530 write 0xc82120f thru 0xc82b3bf (0xa1b1 bytes) 2024-09-14T09:05:42.257 INFO:teuthology.orchestra.run.smithi176.stdout:531 punch from 0x923de67 to 0x924b4ab, (0xd644 bytes) 2024-09-14T09:05:42.257 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:42.257 INFO:teuthology.orchestra.run.smithi176.stdout:534 read 0x44aeb0 thru 0x454745 (0x9896 bytes) 2024-09-14T09:05:42.261 INFO:teuthology.orchestra.run.smithi176.stdout:535 trunc from 0xc82b3c0 to 0xe5cd31f 2024-09-14T09:05:42.266 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:05:42.267 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:05:42.278 INFO:teuthology.orchestra.run.smithi176.stdout:536 trunc from 0xe5cd31f to 0x7010cfe 2024-09-14T09:05:42.282 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:42.280+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:42.289 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:05:42.289 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:05:42.295 INFO:teuthology.orchestra.run.smithi176.stdout:537 read 0x2081b49 thru 0x2083211 (0x16c9 bytes) 2024-09-14T09:05:42.309 INFO:teuthology.orchestra.run.smithi176.stdout:540 writesame 0xaf4ab26 thru 0xaf4fb82 data_size 0x1(0x505d bytes) 2024-09-14T09:05:42.309 INFO:teuthology.orchestra.run.smithi176.stdout:541 punch from 0x174c21b to 0x1759be1, (0xd9c6 bytes) 2024-09-14T09:05:42.309 INFO:teuthology.orchestra.run.smithi176.stdout:542 write 0x9b21dec thru 0x9b261be (0x43d3 bytes) 2024-09-14T09:05:42.311 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:05:42.311 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:05:42.315 INFO:teuthology.orchestra.run.smithi176.stdout:544 write 0x1cecb3d thru 0x1cef9cb (0x2e8f bytes) 2024-09-14T09:05:42.317 INFO:teuthology.orchestra.run.smithi176.stdout:truncating to largest ever: 0xebd11da 2024-09-14T09:05:42.318 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:42.318 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:42.323 INFO:teuthology.orchestra.run.smithi176.stdout:545 trunc from 0xaf4fb83 to 0xebd11da 2024-09-14T09:05:42.330 INFO:teuthology.orchestra.run.smithi176.stdout:546 clone 5 order 21 su 131072 sc 4 2024-09-14T09:05:42.332 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:05:42.332 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:05:42.333 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:05:42.394 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:42.392+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:42.459 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:05:42.459 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:05:42.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:05:42.459 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:05:42.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.460 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:42.460 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379206+0000", 2024-09-14T09:05:42.460 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.080100808000001, 2024-09-14T09:05:42.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.022450290000000001, 2024-09-14T09:05:42.460 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.460 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.460 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:42.460 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:42.460 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:42.460 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:42.460 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 48 2024-09-14T09:05:42.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.461 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379206+0000", 2024-09-14T09:05:42.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379206+0000", 2024-09-14T09:05:42.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379207+0000", 2024-09-14T09:05:42.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.167e-06 2024-09-14T09:05:42.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379216+0000", 2024-09-14T09:05:42.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1630000000000002e-06 2024-09-14T09:05:42.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379226+0000", 2024-09-14T09:05:42.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9029999999999999e-06 2024-09-14T09:05:42.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379233+0000", 2024-09-14T09:05:42.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.17e-06 2024-09-14T09:05:42.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.395986+0000", 2024-09-14T09:05:42.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016752195000000001 2024-09-14T09:05:42.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.396143+0000", 2024-09-14T09:05:42.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000157878 2024-09-14T09:05:42.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:42.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.396346+0000", 2024-09-14T09:05:42.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020270499999999999 2024-09-14T09:05:42.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:42.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401328+0000", 2024-09-14T09:05:42.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0049818379999999997 2024-09-14T09:05:42.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:42.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401367+0000", 2024-09-14T09:05:42.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8926e-05 2024-09-14T09:05:42.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:42.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401541+0000", 2024-09-14T09:05:42.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017380600000000001 2024-09-14T09:05:42.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401656+0000", 2024-09-14T09:05:42.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011553900000000001 2024-09-14T09:05:42.465 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.465 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.465 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.465 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:42.465 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379246+0000", 2024-09-14T09:05:42.465 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.080061024999999, 2024-09-14T09:05:42.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.023032166, 2024-09-14T09:05:42.466 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.466 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.466 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:42.466 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:42.466 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:42.466 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:42.466 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49 2024-09-14T09:05:42.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.466 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379246+0000", 2024-09-14T09:05:42.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379246+0000", 2024-09-14T09:05:42.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379247+0000", 2024-09-14T09:05:42.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7300000000000005e-07 2024-09-14T09:05:42.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379253+0000", 2024-09-14T09:05:42.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.28e-06 2024-09-14T09:05:42.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379267+0000", 2024-09-14T09:05:42.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4087000000000001e-05 2024-09-14T09:05:42.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379274+0000", 2024-09-14T09:05:42.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8929999999999999e-06 2024-09-14T09:05:42.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.397561+0000", 2024-09-14T09:05:42.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018286903 2024-09-14T09:05:42.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.397798+0000", 2024-09-14T09:05:42.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000236989 2024-09-14T09:05:42.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:42.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.398157+0000", 2024-09-14T09:05:42.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035888700000000001 2024-09-14T09:05:42.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:42.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401799+0000", 2024-09-14T09:05:42.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0036428139999999999 2024-09-14T09:05:42.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:42.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402029+0000", 2024-09-14T09:05:42.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022967800000000001 2024-09-14T09:05:42.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:42.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402165+0000", 2024-09-14T09:05:42.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013627600000000001 2024-09-14T09:05:42.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402278+0000", 2024-09-14T09:05:42.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011248599999999999 2024-09-14T09:05:42.471 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.471 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.471 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.471 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:42.471 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379312+0000", 2024-09-14T09:05:42.471 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.079994248999999, 2024-09-14T09:05:42.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.024727301, 2024-09-14T09:05:42.471 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.471 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.471 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:42.471 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:42.471 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:42.471 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:42.472 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 50 2024-09-14T09:05:42.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.472 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379312+0000", 2024-09-14T09:05:42.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379312+0000", 2024-09-14T09:05:42.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379314+0000", 2024-09-14T09:05:42.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.17e-06 2024-09-14T09:05:42.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379408+0000", 2024-09-14T09:05:42.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4562999999999996e-05 2024-09-14T09:05:42.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379429+0000", 2024-09-14T09:05:42.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1189e-05 2024-09-14T09:05:42.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379448+0000", 2024-09-14T09:05:42.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8734000000000001e-05 2024-09-14T09:05:42.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.399051+0000", 2024-09-14T09:05:42.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.019602441000000002 2024-09-14T09:05:42.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.399218+0000", 2024-09-14T09:05:42.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000167095 2024-09-14T09:05:42.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:42.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.399438+0000", 2024-09-14T09:05:42.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022046500000000001 2024-09-14T09:05:42.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:42.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.401991+0000", 2024-09-14T09:05:42.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025529250000000002 2024-09-14T09:05:42.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:42.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.403780+0000", 2024-09-14T09:05:42.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017893939999999999 2024-09-14T09:05:42.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:42.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.403915+0000", 2024-09-14T09:05:42.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013478200000000001 2024-09-14T09:05:42.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.404040+0000", 2024-09-14T09:05:42.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012454299999999999 2024-09-14T09:05:42.476 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.476 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.476 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.477 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:42.477 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379481+0000", 2024-09-14T09:05:42.477 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.079825317000001, 2024-09-14T09:05:42.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.026050659, 2024-09-14T09:05:42.477 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.477 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.477 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:42.477 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:42.477 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:42.477 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:42.477 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 51 2024-09-14T09:05:42.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.478 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379481+0000", 2024-09-14T09:05:42.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379481+0000", 2024-09-14T09:05:42.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379483+0000", 2024-09-14T09:05:42.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3799999999999999e-06 2024-09-14T09:05:42.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379493+0000", 2024-09-14T09:05:42.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0164e-05 2024-09-14T09:05:42.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379535+0000", 2024-09-14T09:05:42.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1672999999999999e-05 2024-09-14T09:05:42.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379552+0000", 2024-09-14T09:05:42.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7615999999999999e-05 2024-09-14T09:05:42.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402365+0000", 2024-09-14T09:05:42.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.022812857999999998 2024-09-14T09:05:42.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402601+0000", 2024-09-14T09:05:42.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000235648 2024-09-14T09:05:42.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:42.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.402966+0000", 2024-09-14T09:05:42.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036486100000000002 2024-09-14T09:05:42.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:42.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405223+0000", 2024-09-14T09:05:42.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002257144 2024-09-14T09:05:42.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:42.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405271+0000", 2024-09-14T09:05:42.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7769999999999998e-05 2024-09-14T09:05:42.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:42.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405410+0000", 2024-09-14T09:05:42.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013921499999999999 2024-09-14T09:05:42.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405532+0000", 2024-09-14T09:05:42.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012233 2024-09-14T09:05:42.482 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.482 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.482 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.482 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:42.482 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.379582+0000", 2024-09-14T09:05:42.482 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.079724607999999, 2024-09-14T09:05:42.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.026468416000000002, 2024-09-14T09:05:42.483 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.483 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.483 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:42.483 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:42.483 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:42.483 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:42.483 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 52 2024-09-14T09:05:42.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.483 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379582+0000", 2024-09-14T09:05:42.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379582+0000", 2024-09-14T09:05:42.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379584+0000", 2024-09-14T09:05:42.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6139999999999999e-06 2024-09-14T09:05:42.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379596+0000", 2024-09-14T09:05:42.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1956e-05 2024-09-14T09:05:42.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379614+0000", 2024-09-14T09:05:42.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8099999999999999e-05 2024-09-14T09:05:42.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.379638+0000", 2024-09-14T09:05:42.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4136999999999999e-05 2024-09-14T09:05:42.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.404134+0000", 2024-09-14T09:05:42.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.024496418999999998 2024-09-14T09:05:42.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.404400+0000", 2024-09-14T09:05:42.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026532500000000001 2024-09-14T09:05:42.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:42.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.404616+0000", 2024-09-14T09:05:42.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021586500000000001 2024-09-14T09:05:42.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:42.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405703+0000", 2024-09-14T09:05:42.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010879220000000001 2024-09-14T09:05:42.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:42.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405887+0000", 2024-09-14T09:05:42.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018396499999999999 2024-09-14T09:05:42.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:42.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.405985+0000", 2024-09-14T09:05:42.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8033000000000001e-05 2024-09-14T09:05:42.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.406051+0000", 2024-09-14T09:05:42.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5079999999999994e-05 2024-09-14T09:05:42.488 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.488 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.488 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.488 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:42.488 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.647322+0000", 2024-09-14T09:05:42.488 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 28.811984579000001, 2024-09-14T09:05:42.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021121719000000001, 2024-09-14T09:05:42.488 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.488 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.489 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:42.489 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:42.489 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:42.489 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:42.489 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 75 2024-09-14T09:05:42.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.489 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647322+0000", 2024-09-14T09:05:42.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647322+0000", 2024-09-14T09:05:42.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647323+0000", 2024-09-14T09:05:42.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.04e-06 2024-09-14T09:05:42.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647407+0000", 2024-09-14T09:05:42.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3652999999999993e-05 2024-09-14T09:05:42.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647416+0000", 2024-09-14T09:05:42.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2310000000000002e-06 2024-09-14T09:05:42.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647424+0000", 2024-09-14T09:05:42.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3240000000000003e-06 2024-09-14T09:05:42.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.661338+0000", 2024-09-14T09:05:42.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013913114000000001 2024-09-14T09:05:42.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.661461+0000", 2024-09-14T09:05:42.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012374200000000001 2024-09-14T09:05:42.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:42.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.661653+0000", 2024-09-14T09:05:42.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000191872 2024-09-14T09:05:42.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:42.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.664371+0000", 2024-09-14T09:05:42.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027177600000000001 2024-09-14T09:05:42.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:42.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668357+0000", 2024-09-14T09:05:42.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039863229999999999 2024-09-14T09:05:42.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:42.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668403+0000", 2024-09-14T09:05:42.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6276000000000002e-05 2024-09-14T09:05:42.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668444+0000", 2024-09-14T09:05:42.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0383999999999998e-05 2024-09-14T09:05:42.494 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.494 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.494 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.494 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4207.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T09:05:42.494 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:13.647440+0000", 2024-09-14T09:05:42.494 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 28.811866261999999, 2024-09-14T09:05:42.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021273368000000001, 2024-09-14T09:05:42.494 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.494 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.494 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:42.494 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:42.494 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4207", 2024-09-14T09:05:42.495 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/3989307032", 2024-09-14T09:05:42.495 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 76 2024-09-14T09:05:42.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.495 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647440+0000", 2024-09-14T09:05:42.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647440+0000", 2024-09-14T09:05:42.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647441+0000", 2024-09-14T09:05:42.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1500000000000003e-07 2024-09-14T09:05:42.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647448+0000", 2024-09-14T09:05:42.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4420000000000003e-06 2024-09-14T09:05:42.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647457+0000", 2024-09-14T09:05:42.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:05:42.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.647465+0000", 2024-09-14T09:05:42.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2250000000000003e-06 2024-09-14T09:05:42.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.663329+0000", 2024-09-14T09:05:42.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015863931000000001 2024-09-14T09:05:42.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.663474+0000", 2024-09-14T09:05:42.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014509200000000001 2024-09-14T09:05:42.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:05:42.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.663695+0000", 2024-09-14T09:05:42.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022155499999999999 2024-09-14T09:05:42.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:42.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.664823+0000", 2024-09-14T09:05:42.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001128046 2024-09-14T09:05:42.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:42.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668550+0000", 2024-09-14T09:05:42.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003726493 2024-09-14T09:05:42.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:42.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668638+0000", 2024-09-14T09:05:42.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8282999999999995e-05 2024-09-14T09:05:42.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:13.668714+0000", 2024-09-14T09:05:42.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5856e-05 2024-09-14T09:05:42.499 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.499 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.499 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.499 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:654 3.11 3:8c8e358b:::rbd_object_map.115e1e95bf99:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:42.500 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.852458+0000", 2024-09-14T09:05:42.500 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.6068485999999993, 2024-09-14T09:05:42.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018532317999999999, 2024-09-14T09:05:42.500 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.500 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.500 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:42.500 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:42.500 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:42.500 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:42.500 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 654 2024-09-14T09:05:42.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.500 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.852458+0000", 2024-09-14T09:05:42.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.852458+0000", 2024-09-14T09:05:42.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.852461+0000", 2024-09-14T09:05:42.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3270000000000001e-06 2024-09-14T09:05:42.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.852476+0000", 2024-09-14T09:05:42.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4543e-05 2024-09-14T09:05:42.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.852509+0000", 2024-09-14T09:05:42.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3219000000000002e-05 2024-09-14T09:05:42.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.852528+0000", 2024-09-14T09:05:42.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8386999999999999e-05 2024-09-14T09:05:42.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.852705+0000", 2024-09-14T09:05:42.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000177269 2024-09-14T09:05:42.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op must wait for map", 2024-09-14T09:05:42.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.852729+0000", 2024-09-14T09:05:42.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4513000000000001e-05 2024-09-14T09:05:42.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.861522+0000", 2024-09-14T09:05:42.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087921749999999993 2024-09-14T09:05:42.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.861731+0000", 2024-09-14T09:05:42.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000209748 2024-09-14T09:05:42.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:42.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.862486+0000", 2024-09-14T09:05:42.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000754888 2024-09-14T09:05:42.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:42.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.866665+0000", 2024-09-14T09:05:42.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041788529999999997 2024-09-14T09:05:42.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:42.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.870620+0000", 2024-09-14T09:05:42.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039549190000000003 2024-09-14T09:05:42.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:42.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.870863+0000", 2024-09-14T09:05:42.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024262099999999999 2024-09-14T09:05:42.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.870990+0000", 2024-09-14T09:05:42.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000127856 2024-09-14T09:05:42.506 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.506 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.506 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.506 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:655 3.11 3:8c8e358b:::rbd_object_map.115e1e95bf99:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:42.506 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.853314+0000", 2024-09-14T09:05:42.506 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.6059928800000005, 2024-09-14T09:05:42.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.019496724999999999, 2024-09-14T09:05:42.506 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.506 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.507 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:42.507 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:42.507 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:42.507 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:42.507 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 655 2024-09-14T09:05:42.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.507 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.853314+0000", 2024-09-14T09:05:42.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.853314+0000", 2024-09-14T09:05:42.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.853317+0000", 2024-09-14T09:05:42.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.207e-06 2024-09-14T09:05:42.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.853335+0000", 2024-09-14T09:05:42.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8009999999999999e-05 2024-09-14T09:05:42.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.853401+0000", 2024-09-14T09:05:42.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5463000000000007e-05 2024-09-14T09:05:42.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.853440+0000", 2024-09-14T09:05:42.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9789999999999997e-05 2024-09-14T09:05:42.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.853610+0000", 2024-09-14T09:05:42.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016973800000000001 2024-09-14T09:05:42.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:05:42.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.853630+0000", 2024-09-14T09:05:42.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0210000000000001e-05 2024-09-14T09:05:42.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863967+0000", 2024-09-14T09:05:42.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010336942999999999 2024-09-14T09:05:42.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.864264+0000", 2024-09-14T09:05:42.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029657799999999998 2024-09-14T09:05:42.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:42.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.865441+0000", 2024-09-14T09:05:42.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001176959 2024-09-14T09:05:42.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:42.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.868947+0000", 2024-09-14T09:05:42.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0035066849999999998 2024-09-14T09:05:42.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:42.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.872650+0000", 2024-09-14T09:05:42.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037025399999999998 2024-09-14T09:05:42.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:42.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.872739+0000", 2024-09-14T09:05:42.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9525999999999996e-05 2024-09-14T09:05:42.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.872811+0000", 2024-09-14T09:05:42.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1075999999999995e-05 2024-09-14T09:05:42.512 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.512 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.512 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.512 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:792 3.11 3:8c8e358b:::rbd_object_map.115e1e95bf99:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:42.512 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.893351+0000", 2024-09-14T09:05:42.513 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.5659553129999999, 2024-09-14T09:05:42.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.024882419999999999, 2024-09-14T09:05:42.513 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.513 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.513 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:42.513 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:42.513 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:42.513 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:42.513 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 792 2024-09-14T09:05:42.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.513 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.893351+0000", 2024-09-14T09:05:42.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.893351+0000", 2024-09-14T09:05:42.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.893353+0000", 2024-09-14T09:05:42.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.53e-06 2024-09-14T09:05:42.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.893361+0000", 2024-09-14T09:05:42.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6000000000000001e-06 2024-09-14T09:05:42.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.893379+0000", 2024-09-14T09:05:42.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8283000000000001e-05 2024-09-14T09:05:42.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.893393+0000", 2024-09-14T09:05:42.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4365999999999999e-05 2024-09-14T09:05:42.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.895595+0000", 2024-09-14T09:05:42.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002201518 2024-09-14T09:05:42.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.895894+0000", 2024-09-14T09:05:42.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029924800000000001 2024-09-14T09:05:42.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:42.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.896899+0000", 2024-09-14T09:05:42.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010047960000000001 2024-09-14T09:05:42.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:42.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.908656+0000", 2024-09-14T09:05:42.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011756902 2024-09-14T09:05:42.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:42.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.918060+0000", 2024-09-14T09:05:42.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0094038809999999994 2024-09-14T09:05:42.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:42.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.918143+0000", 2024-09-14T09:05:42.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3236000000000001e-05 2024-09-14T09:05:42.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.918234+0000", 2024-09-14T09:05:42.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1059999999999999e-05 2024-09-14T09:05:42.518 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.518 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.518 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.518 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:821 3.e 3:740ec7e3:::rbd_data.115e724e1d99.0000000000000006:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:42.518 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.895061+0000", 2024-09-14T09:05:42.518 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.5642456490000001, 2024-09-14T09:05:42.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.020606620999999999, 2024-09-14T09:05:42.519 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.519 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.519 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:05:42.519 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:42.519 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:42.519 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:42.519 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 821 2024-09-14T09:05:42.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.519 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.895061+0000", 2024-09-14T09:05:42.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.895061+0000", 2024-09-14T09:05:42.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.895063+0000", 2024-09-14T09:05:42.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5430000000000001e-06 2024-09-14T09:05:42.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.895071+0000", 2024-09-14T09:05:42.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0569999999999993e-06 2024-09-14T09:05:42.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.895092+0000", 2024-09-14T09:05:42.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1126000000000001e-05 2024-09-14T09:05:42.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.895108+0000", 2024-09-14T09:05:42.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6123000000000001e-05 2024-09-14T09:05:42.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.915239+0000", 2024-09-14T09:05:42.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.020130674000000001 2024-09-14T09:05:42.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.915423+0000", 2024-09-14T09:05:42.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000184222 2024-09-14T09:05:42.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.915668+0000", 2024-09-14T09:05:42.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024487599999999999 2024-09-14T09:05:42.522 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.522 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.522 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.523 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:829 3.13 3:cc36a41f:::rbd_data.115e1e95bf99.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13581651~3195565] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:42.523 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.899824+0000", 2024-09-14T09:05:42.523 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.5594830829999999, 2024-09-14T09:05:42.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.026145906999999999, 2024-09-14T09:05:42.523 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.523 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.523 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:42.523 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:42.523 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:42.523 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:42.523 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 829 2024-09-14T09:05:42.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.523 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.899824+0000", 2024-09-14T09:05:42.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.899824+0000", 2024-09-14T09:05:42.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.899826+0000", 2024-09-14T09:05:42.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4729999999999999e-06 2024-09-14T09:05:42.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900269+0000", 2024-09-14T09:05:42.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00044249300000000002 2024-09-14T09:05:42.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900300+0000", 2024-09-14T09:05:42.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1016999999999998e-05 2024-09-14T09:05:42.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900321+0000", 2024-09-14T09:05:42.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.126e-05 2024-09-14T09:05:42.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900462+0000", 2024-09-14T09:05:42.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014153200000000001 2024-09-14T09:05:42.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900766+0000", 2024-09-14T09:05:42.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000303424 2024-09-14T09:05:42.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:42.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901279+0000", 2024-09-14T09:05:42.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00051300499999999997 2024-09-14T09:05:42.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:42.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921167+0000", 2024-09-14T09:05:42.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.019888408999999999 2024-09-14T09:05:42.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:42.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.925809+0000", 2024-09-14T09:05:42.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004641895 2024-09-14T09:05:42.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:42.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.925897+0000", 2024-09-14T09:05:42.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7965000000000002e-05 2024-09-14T09:05:42.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.925970+0000", 2024-09-14T09:05:42.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2434000000000005e-05 2024-09-14T09:05:42.528 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.528 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.528 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.529 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:830 3.11 3:8c8e358b:::rbd_object_map.115e1e95bf99:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:42.529 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.900383+0000", 2024-09-14T09:05:42.529 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.5589232840000005, 2024-09-14T09:05:42.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021339981000000001, 2024-09-14T09:05:42.529 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.529 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.529 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:42.529 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:42.529 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:42.529 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:42.529 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 830 2024-09-14T09:05:42.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.529 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900383+0000", 2024-09-14T09:05:42.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900383+0000", 2024-09-14T09:05:42.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900385+0000", 2024-09-14T09:05:42.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.934e-06 2024-09-14T09:05:42.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900395+0000", 2024-09-14T09:05:42.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4190000000000008e-06 2024-09-14T09:05:42.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900418+0000", 2024-09-14T09:05:42.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2776999999999999e-05 2024-09-14T09:05:42.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900434+0000", 2024-09-14T09:05:42.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5996000000000001e-05 2024-09-14T09:05:42.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900545+0000", 2024-09-14T09:05:42.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000111601 2024-09-14T09:05:42.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900740+0000", 2024-09-14T09:05:42.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000194976 2024-09-14T09:05:42.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:42.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901654+0000", 2024-09-14T09:05:42.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091431599999999998 2024-09-14T09:05:42.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:42.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.908747+0000", 2024-09-14T09:05:42.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0070929920000000002 2024-09-14T09:05:42.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:42.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921507+0000", 2024-09-14T09:05:42.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.012759645999999999 2024-09-14T09:05:42.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:42.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921601+0000", 2024-09-14T09:05:42.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3844999999999993e-05 2024-09-14T09:05:42.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921723+0000", 2024-09-14T09:05:42.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012247899999999999 2024-09-14T09:05:42.534 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.534 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.534 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.534 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:831 3.11 3:8c8e358b:::rbd_object_map.115e1e95bf99:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:42.534 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.900470+0000", 2024-09-14T09:05:42.534 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.5588371280000004, 2024-09-14T09:05:42.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021390532, 2024-09-14T09:05:42.535 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.535 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.535 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:42.535 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:42.535 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:42.535 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:42.535 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 831 2024-09-14T09:05:42.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.535 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900470+0000", 2024-09-14T09:05:42.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900470+0000", 2024-09-14T09:05:42.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900471+0000", 2024-09-14T09:05:42.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5370000000000001e-06 2024-09-14T09:05:42.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900479+0000", 2024-09-14T09:05:42.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0029999999999994e-06 2024-09-14T09:05:42.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900499+0000", 2024-09-14T09:05:42.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0047e-05 2024-09-14T09:05:42.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900518+0000", 2024-09-14T09:05:42.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.929e-05 2024-09-14T09:05:42.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902843+0000", 2024-09-14T09:05:42.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023242670000000001 2024-09-14T09:05:42.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.903053+0000", 2024-09-14T09:05:42.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021007799999999999 2024-09-14T09:05:42.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:42.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.903951+0000", 2024-09-14T09:05:42.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00089844499999999999 2024-09-14T09:05:42.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:42.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.908830+0000", 2024-09-14T09:05:42.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048782840000000001 2024-09-14T09:05:42.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:42.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921741+0000", 2024-09-14T09:05:42.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.012911618999999999 2024-09-14T09:05:42.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:42.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921803+0000", 2024-09-14T09:05:42.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2238999999999998e-05 2024-09-14T09:05:42.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921860+0000", 2024-09-14T09:05:42.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6722999999999999e-05 2024-09-14T09:05:42.540 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.540 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.540 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.540 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:832 3.11 3:8c8e358b:::rbd_object_map.115e1e95bf99:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:42.540 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.900557+0000", 2024-09-14T09:05:42.540 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.5587493119999998, 2024-09-14T09:05:42.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021419134999999999, 2024-09-14T09:05:42.540 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.540 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.541 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:42.541 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:42.541 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:42.541 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:42.541 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 832 2024-09-14T09:05:42.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.541 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900557+0000", 2024-09-14T09:05:42.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900557+0000", 2024-09-14T09:05:42.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900559+0000", 2024-09-14T09:05:42.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.733e-06 2024-09-14T09:05:42.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900567+0000", 2024-09-14T09:05:42.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7270000000000007e-06 2024-09-14T09:05:42.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900586+0000", 2024-09-14T09:05:42.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9233e-05 2024-09-14T09:05:42.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900600+0000", 2024-09-14T09:05:42.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4229999999999999e-05 2024-09-14T09:05:42.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.904848+0000", 2024-09-14T09:05:42.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042476019999999996 2024-09-14T09:05:42.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.905051+0000", 2024-09-14T09:05:42.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020333300000000001 2024-09-14T09:05:42.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:42.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.906440+0000", 2024-09-14T09:05:42.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001388638 2024-09-14T09:05:42.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:42.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.911832+0000", 2024-09-14T09:05:42.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053925079999999999 2024-09-14T09:05:42.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:42.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921872+0000", 2024-09-14T09:05:42.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010040083999999999 2024-09-14T09:05:42.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:42.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921925+0000", 2024-09-14T09:05:42.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2524000000000001e-05 2024-09-14T09:05:42.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921977+0000", 2024-09-14T09:05:42.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1523000000000001e-05 2024-09-14T09:05:42.546 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.546 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.546 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.546 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:833 3.11 3:8c8e358b:::rbd_object_map.115e1e95bf99:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:42.546 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.900622+0000", 2024-09-14T09:05:42.546 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.5586846520000002, 2024-09-14T09:05:42.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021467374000000001, 2024-09-14T09:05:42.546 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.546 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.546 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:42.546 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:42.546 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:42.547 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:42.547 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 833 2024-09-14T09:05:42.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.547 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900622+0000", 2024-09-14T09:05:42.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900622+0000", 2024-09-14T09:05:42.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900624+0000", 2024-09-14T09:05:42.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4929999999999999e-06 2024-09-14T09:05:42.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900681+0000", 2024-09-14T09:05:42.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7859999999999998e-05 2024-09-14T09:05:42.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900715+0000", 2024-09-14T09:05:42.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3402999999999999e-05 2024-09-14T09:05:42.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.900731+0000", 2024-09-14T09:05:42.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5773e-05 2024-09-14T09:05:42.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.908869+0000", 2024-09-14T09:05:42.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0081381809999999995 2024-09-14T09:05:42.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.909003+0000", 2024-09-14T09:05:42.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013468900000000001 2024-09-14T09:05:42.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:42.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.909599+0000", 2024-09-14T09:05:42.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00059570800000000002 2024-09-14T09:05:42.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:42.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.915364+0000", 2024-09-14T09:05:42.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057650109999999996 2024-09-14T09:05:42.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:42.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.921988+0000", 2024-09-14T09:05:42.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006624217 2024-09-14T09:05:42.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:42.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922040+0000", 2024-09-14T09:05:42.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1749999999999997e-05 2024-09-14T09:05:42.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922089+0000", 2024-09-14T09:05:42.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9289000000000002e-05 2024-09-14T09:05:42.552 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.552 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.552 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.552 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:834 3.11 3:8c8e358b:::rbd_object_map.115e1e95bf99:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:42.552 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.902188+0000", 2024-09-14T09:05:42.552 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.5571182730000004, 2024-09-14T09:05:42.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.020043834, 2024-09-14T09:05:42.553 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.553 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.553 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:42.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:42.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:42.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:42.553 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 834 2024-09-14T09:05:42.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.553 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902188+0000", 2024-09-14T09:05:42.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902188+0000", 2024-09-14T09:05:42.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902191+0000", 2024-09-14T09:05:42.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6469999999999998e-06 2024-09-14T09:05:42.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902205+0000", 2024-09-14T09:05:42.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3890000000000001e-05 2024-09-14T09:05:42.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902237+0000", 2024-09-14T09:05:42.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1606000000000002e-05 2024-09-14T09:05:42.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902259+0000", 2024-09-14T09:05:42.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2543000000000001e-05 2024-09-14T09:05:42.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.910278+0000", 2024-09-14T09:05:42.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0080190130000000002 2024-09-14T09:05:42.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.910449+0000", 2024-09-14T09:05:42.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170483 2024-09-14T09:05:42.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:42.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.911071+0000", 2024-09-14T09:05:42.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00062188300000000005 2024-09-14T09:05:42.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:42.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.915493+0000", 2024-09-14T09:05:42.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044225100000000002 2024-09-14T09:05:42.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:42.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922118+0000", 2024-09-14T09:05:42.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066246050000000004 2024-09-14T09:05:42.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:42.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922180+0000", 2024-09-14T09:05:42.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2477000000000005e-05 2024-09-14T09:05:42.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922232+0000", 2024-09-14T09:05:42.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2176999999999999e-05 2024-09-14T09:05:42.558 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.558 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.558 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.558 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:835 3.11 3:8c8e358b:::rbd_object_map.115e1e95bf99:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:42.558 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.903632+0000", 2024-09-14T09:05:42.558 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.5556747160000004, 2024-09-14T09:05:42.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018702719, 2024-09-14T09:05:42.559 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.559 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.559 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:42.559 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:42.559 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:42.559 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:42.559 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 835 2024-09-14T09:05:42.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.559 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.903632+0000", 2024-09-14T09:05:42.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.903632+0000", 2024-09-14T09:05:42.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.903634+0000", 2024-09-14T09:05:42.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.216e-06 2024-09-14T09:05:42.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.903647+0000", 2024-09-14T09:05:42.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2979999999999999e-05 2024-09-14T09:05:42.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.903678+0000", 2024-09-14T09:05:42.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0607e-05 2024-09-14T09:05:42.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.903699+0000", 2024-09-14T09:05:42.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1189999999999999e-05 2024-09-14T09:05:42.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.911909+0000", 2024-09-14T09:05:42.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0082102470000000004 2024-09-14T09:05:42.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.912070+0000", 2024-09-14T09:05:42.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016051100000000001 2024-09-14T09:05:42.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:42.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.912751+0000", 2024-09-14T09:05:42.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068099800000000004 2024-09-14T09:05:42.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:42.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.915587+0000", 2024-09-14T09:05:42.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028365399999999998 2024-09-14T09:05:42.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:42.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922245+0000", 2024-09-14T09:05:42.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066577449999999996 2024-09-14T09:05:42.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:42.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922297+0000", 2024-09-14T09:05:42.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1854999999999997e-05 2024-09-14T09:05:42.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922335+0000", 2024-09-14T09:05:42.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7830000000000002e-05 2024-09-14T09:05:42.564 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.564 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.564 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.564 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:10026 3.f 3:f6f4e415:::rbd_data.115eadec874.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:42.564 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.713901+0000", 2024-09-14T09:05:42.564 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.745405527, 2024-09-14T09:05:42.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.045312305999999997, 2024-09-14T09:05:42.565 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.565 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.565 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:42.565 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:05:42.565 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:05:42.565 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:42.565 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 10026 2024-09-14T09:05:42.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.565 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713901+0000", 2024-09-14T09:05:42.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713901+0000", 2024-09-14T09:05:42.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713903+0000", 2024-09-14T09:05:42.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4750000000000001e-06 2024-09-14T09:05:42.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713908+0000", 2024-09-14T09:05:42.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.242e-06 2024-09-14T09:05:42.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713920+0000", 2024-09-14T09:05:42.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2456000000000001e-05 2024-09-14T09:05:42.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713927+0000", 2024-09-14T09:05:42.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0849999999999999e-06 2024-09-14T09:05:42.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.713973+0000", 2024-09-14T09:05:42.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5179000000000002e-05 2024-09-14T09:05:42.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.714023+0000", 2024-09-14T09:05:42.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0469000000000001e-05 2024-09-14T09:05:42.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:42.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.715791+0000", 2024-09-14T09:05:42.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017676580000000001 2024-09-14T09:05:42.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.759031+0000", 2024-09-14T09:05:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.043240439999999998 2024-09-14T09:05:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:05:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.759058+0000", 2024-09-14T09:05:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6395999999999999e-05 2024-09-14T09:05:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:05:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.759141+0000", 2024-09-14T09:05:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3026000000000001e-05 2024-09-14T09:05:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.759213+0000", 2024-09-14T09:05:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2879999999999993e-05 2024-09-14T09:05:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:10026 3.f e32/23)", 2024-09-14T09:05:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.722702+0000", 2024-09-14T09:05:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.7366046619999995, 2024-09-14T09:05:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.036342688999999997, 2024-09-14T09:05:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:05:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:05:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:05:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:05:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:05:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.722702+0000", 2024-09-14T09:05:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:05:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.722702+0000", 2024-09-14T09:05:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:05:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:05:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.722703+0000", 2024-09-14T09:05:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.046e-06 2024-09-14T09:05:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:05:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.722708+0000", 2024-09-14T09:05:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2429999999999996e-06 2024-09-14T09:05:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:05:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.722725+0000", 2024-09-14T09:05:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6748e-05 2024-09-14T09:05:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:05:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.722734+0000", 2024-09-14T09:05:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3589999999999997e-06 2024-09-14T09:05:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:05:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.759004+0000", 2024-09-14T09:05:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.036269606000000003 2024-09-14T09:05:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:05:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.759021+0000", 2024-09-14T09:05:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6883000000000001e-05 2024-09-14T09:05:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:05:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:05:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:05:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.759045+0000", 2024-09-14T09:05:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3804000000000001e-05 2024-09-14T09:05:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:05:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:05:42.574 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:05:42.576 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:05:42.576 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:42.576+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:42.677 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:42.672+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:42.708 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:42.708 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:42.709 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:42.709 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:42.722 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:05:42.778 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:42.780+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:42.879 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:42.876+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:42.901 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:42.901 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:42.901 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:42.901 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:42.901 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:42.917 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:05:42.929 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T09:05:42.975 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone4 from 0xebd11da (overlap 0x2ced2ad) to 0x0 2024-09-14T09:05:42.980 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:42.984+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:43.081 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:43.076+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:43.096 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:43.096 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:43.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:43.096 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:43.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.096 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-14T09:05:43.096 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.274102+0000", 2024-09-14T09:05:43.096 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.822106886, 2024-09-14T09:05:43.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003545763, 2024-09-14T09:05:43.097 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.097 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.097 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.097 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274102+0000", 2024-09-14T09:05:43.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274102+0000", 2024-09-14T09:05:43.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274103+0000", 2024-09-14T09:05:43.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4670000000000001e-06 2024-09-14T09:05:43.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274328+0000", 2024-09-14T09:05:43.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022502500000000001 2024-09-14T09:05:43.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274354+0000", 2024-09-14T09:05:43.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5993e-05 2024-09-14T09:05:43.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274372+0000", 2024-09-14T09:05:43.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8117e-05 2024-09-14T09:05:43.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274455+0000", 2024-09-14T09:05:43.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2785000000000001e-05 2024-09-14T09:05:43.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.274486+0000", 2024-09-14T09:05:43.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0864000000000001e-05 2024-09-14T09:05:43.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277538+0000", 2024-09-14T09:05:43.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0030521860000000001 2024-09-14T09:05:43.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.277647+0000", 2024-09-14T09:05:43.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109326 2024-09-14T09:05:43.100 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.100 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.100 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.101 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-14T09:05:43.101 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.275860+0000", 2024-09-14T09:05:43.101 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.820348712000001, 2024-09-14T09:05:43.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003513562, 2024-09-14T09:05:43.101 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.101 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.101 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.101 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.275860+0000", 2024-09-14T09:05:43.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.275860+0000", 2024-09-14T09:05:43.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.275861+0000", 2024-09-14T09:05:43.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.513e-06 2024-09-14T09:05:43.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276039+0000", 2024-09-14T09:05:43.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017758199999999999 2024-09-14T09:05:43.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276060+0000", 2024-09-14T09:05:43.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0956000000000001e-05 2024-09-14T09:05:43.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276076+0000", 2024-09-14T09:05:43.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6137000000000002e-05 2024-09-14T09:05:43.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276190+0000", 2024-09-14T09:05:43.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011359200000000001 2024-09-14T09:05:43.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.276226+0000", 2024-09-14T09:05:43.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6300000000000001e-05 2024-09-14T09:05:43.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.279274+0000", 2024-09-14T09:05:43.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00304813 2024-09-14T09:05:43.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.279373+0000", 2024-09-14T09:05:43.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9351999999999993e-05 2024-09-14T09:05:43.105 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.105 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.105 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.105 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:72 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'50, mlcod=17'45)", 2024-09-14T09:05:43.105 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.657508+0000", 2024-09-14T09:05:43.105 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.438700398999998, 2024-09-14T09:05:43.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0033585239999999999, 2024-09-14T09:05:43.105 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.105 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.105 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.105 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657508+0000", 2024-09-14T09:05:43.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657508+0000", 2024-09-14T09:05:43.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657510+0000", 2024-09-14T09:05:43.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8899999999999999e-06 2024-09-14T09:05:43.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657625+0000", 2024-09-14T09:05:43.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115216 2024-09-14T09:05:43.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657649+0000", 2024-09-14T09:05:43.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4043e-05 2024-09-14T09:05:43.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657669+0000", 2024-09-14T09:05:43.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9579999999999999e-05 2024-09-14T09:05:43.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657790+0000", 2024-09-14T09:05:43.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012084200000000001 2024-09-14T09:05:43.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.657828+0000", 2024-09-14T09:05:43.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8019999999999999e-05 2024-09-14T09:05:43.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660805+0000", 2024-09-14T09:05:43.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002976874 2024-09-14T09:05:43.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660867+0000", 2024-09-14T09:05:43.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2058999999999997e-05 2024-09-14T09:05:43.109 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.109 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.109 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.109 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:73 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'51, mlcod=17'45)", 2024-09-14T09:05:43.109 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.658866+0000", 2024-09-14T09:05:43.109 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.437342961999999, 2024-09-14T09:05:43.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035579930000000002, 2024-09-14T09:05:43.109 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.109 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.110 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.110 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658866+0000", 2024-09-14T09:05:43.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658866+0000", 2024-09-14T09:05:43.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658867+0000", 2024-09-14T09:05:43.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.886e-06 2024-09-14T09:05:43.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658880+0000", 2024-09-14T09:05:43.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2204e-05 2024-09-14T09:05:43.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658908+0000", 2024-09-14T09:05:43.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8555999999999999e-05 2024-09-14T09:05:43.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.658921+0000", 2024-09-14T09:05:43.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2566e-05 2024-09-14T09:05:43.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659682+0000", 2024-09-14T09:05:43.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000760865 2024-09-14T09:05:43.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659710+0000", 2024-09-14T09:05:43.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8612e-05 2024-09-14T09:05:43.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662349+0000", 2024-09-14T09:05:43.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0026388370000000002 2024-09-14T09:05:43.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662424+0000", 2024-09-14T09:05:43.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4467000000000004e-05 2024-09-14T09:05:43.113 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.113 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.113 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.113 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:74 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'52, mlcod=17'47)", 2024-09-14T09:05:43.113 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.659878+0000", 2024-09-14T09:05:43.113 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.436330393999999, 2024-09-14T09:05:43.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0040591689999999996, 2024-09-14T09:05:43.114 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.114 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.114 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.114 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659878+0000", 2024-09-14T09:05:43.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659878+0000", 2024-09-14T09:05:43.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659892+0000", 2024-09-14T09:05:43.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3487e-05 2024-09-14T09:05:43.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659902+0000", 2024-09-14T09:05:43.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.062e-05 2024-09-14T09:05:43.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659928+0000", 2024-09-14T09:05:43.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6060000000000001e-05 2024-09-14T09:05:43.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.659941+0000", 2024-09-14T09:05:43.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2649999999999999e-05 2024-09-14T09:05:43.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660304+0000", 2024-09-14T09:05:43.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036354699999999998 2024-09-14T09:05:43.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660349+0000", 2024-09-14T09:05:43.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4938999999999998e-05 2024-09-14T09:05:43.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.663847+0000", 2024-09-14T09:05:43.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0034973339999999999 2024-09-14T09:05:43.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.663937+0000", 2024-09-14T09:05:43.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0532000000000007e-05 2024-09-14T09:05:43.117 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.117 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.117 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.118 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'47)", 2024-09-14T09:05:43.118 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.660695+0000", 2024-09-14T09:05:43.118 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.435513242999999, 2024-09-14T09:05:43.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0062206520000000001, 2024-09-14T09:05:43.118 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.118 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.118 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.118 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660695+0000", 2024-09-14T09:05:43.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660695+0000", 2024-09-14T09:05:43.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660697+0000", 2024-09-14T09:05:43.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.897e-06 2024-09-14T09:05:43.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660816+0000", 2024-09-14T09:05:43.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118442 2024-09-14T09:05:43.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660839+0000", 2024-09-14T09:05:43.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3790000000000001e-05 2024-09-14T09:05:43.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660852+0000", 2024-09-14T09:05:43.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-05 2024-09-14T09:05:43.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.660967+0000", 2024-09-14T09:05:43.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115276 2024-09-14T09:05:43.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.661016+0000", 2024-09-14T09:05:43.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8785999999999997e-05 2024-09-14T09:05:43.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.666825+0000", 2024-09-14T09:05:43.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0058087549999999996 2024-09-14T09:05:43.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.666916+0000", 2024-09-14T09:05:43.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1325999999999999e-05 2024-09-14T09:05:43.121 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.121 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.122 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.122 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'49)", 2024-09-14T09:05:43.122 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.662683+0000", 2024-09-14T09:05:43.122 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.433525930999998, 2024-09-14T09:05:43.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0042817899999999997, 2024-09-14T09:05:43.122 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.122 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.122 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.122 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662683+0000", 2024-09-14T09:05:43.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662683+0000", 2024-09-14T09:05:43.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662684+0000", 2024-09-14T09:05:43.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6559999999999999e-06 2024-09-14T09:05:43.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662696+0000", 2024-09-14T09:05:43.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1454000000000001e-05 2024-09-14T09:05:43.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662724+0000", 2024-09-14T09:05:43.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8535999999999999e-05 2024-09-14T09:05:43.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662752+0000", 2024-09-14T09:05:43.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7503000000000001e-05 2024-09-14T09:05:43.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662867+0000", 2024-09-14T09:05:43.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011500199999999999 2024-09-14T09:05:43.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.662899+0000", 2024-09-14T09:05:43.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2122999999999997e-05 2024-09-14T09:05:43.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.666919+0000", 2024-09-14T09:05:43.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004020633 2024-09-14T09:05:43.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.666964+0000", 2024-09-14T09:05:43.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4882999999999997e-05 2024-09-14T09:05:43.126 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.126 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.126 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.126 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:77 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'55, mlcod=17'53)", 2024-09-14T09:05:43.126 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.668436+0000", 2024-09-14T09:05:43.126 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.427772170000001, 2024-09-14T09:05:43.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035848659999999999, 2024-09-14T09:05:43.126 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.126 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.127 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.127 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668436+0000", 2024-09-14T09:05:43.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668436+0000", 2024-09-14T09:05:43.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668438+0000", 2024-09-14T09:05:43.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.81e-06 2024-09-14T09:05:43.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668551+0000", 2024-09-14T09:05:43.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112726 2024-09-14T09:05:43.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668580+0000", 2024-09-14T09:05:43.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8702999999999999e-05 2024-09-14T09:05:43.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668592+0000", 2024-09-14T09:05:43.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.276e-05 2024-09-14T09:05:43.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668767+0000", 2024-09-14T09:05:43.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017485600000000001 2024-09-14T09:05:43.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.668808+0000", 2024-09-14T09:05:43.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1189000000000002e-05 2024-09-14T09:05:43.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.671940+0000", 2024-09-14T09:05:43.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0031317329999999998 2024-09-14T09:05:43.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.672021+0000", 2024-09-14T09:05:43.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1088999999999996e-05 2024-09-14T09:05:43.130 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.130 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.130 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.130 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4207.0:78 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'56, mlcod=17'53)", 2024-09-14T09:05:43.130 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:13.670356+0000", 2024-09-14T09:05:43.130 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.425852384999999, 2024-09-14T09:05:43.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0036555289999999998, 2024-09-14T09:05:43.131 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.131 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.131 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.131 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670356+0000", 2024-09-14T09:05:43.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670356+0000", 2024-09-14T09:05:43.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670358+0000", 2024-09-14T09:05:43.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.79e-06 2024-09-14T09:05:43.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670469+0000", 2024-09-14T09:05:43.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111273 2024-09-14T09:05:43.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670495+0000", 2024-09-14T09:05:43.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5516000000000001e-05 2024-09-14T09:05:43.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670508+0000", 2024-09-14T09:05:43.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2986999999999999e-05 2024-09-14T09:05:43.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670679+0000", 2024-09-14T09:05:43.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017094199999999999 2024-09-14T09:05:43.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.670717+0000", 2024-09-14T09:05:43.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8322999999999999e-05 2024-09-14T09:05:43.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.673942+0000", 2024-09-14T09:05:43.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032255489999999999 2024-09-14T09:05:43.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:13.674012+0000", 2024-09-14T09:05:43.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9148999999999998e-05 2024-09-14T09:05:43.134 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.134 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.134 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.134 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T09:05:43.134 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:28.835461+0000", 2024-09-14T09:05:43.134 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.260747305000001, 2024-09-14T09:05:43.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050182409999999997, 2024-09-14T09:05:43.135 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.135 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.135 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:43.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:43.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:43.135 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 11 2024-09-14T09:05:43.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.135 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835461+0000", 2024-09-14T09:05:43.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835461+0000", 2024-09-14T09:05:43.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835463+0000", 2024-09-14T09:05:43.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9570000000000001e-06 2024-09-14T09:05:43.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835474+0000", 2024-09-14T09:05:43.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0949e-05 2024-09-14T09:05:43.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835495+0000", 2024-09-14T09:05:43.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1325e-05 2024-09-14T09:05:43.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835506+0000", 2024-09-14T09:05:43.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0854e-05 2024-09-14T09:05:43.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835579+0000", 2024-09-14T09:05:43.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2743000000000005e-05 2024-09-14T09:05:43.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T09:05:43.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.835585+0000", 2024-09-14T09:05:43.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9519999999999999e-06 2024-09-14T09:05:43.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.838428+0000", 2024-09-14T09:05:43.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0028425830000000001 2024-09-14T09:05:43.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.838557+0000", 2024-09-14T09:05:43.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012985000000000001 2024-09-14T09:05:43.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:05:43.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.838743+0000", 2024-09-14T09:05:43.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018533199999999999 2024-09-14T09:05:43.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:43.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.839964+0000", 2024-09-14T09:05:43.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001221387 2024-09-14T09:05:43.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:43.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.840333+0000", 2024-09-14T09:05:43.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036902899999999999 2024-09-14T09:05:43.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.840410+0000", 2024-09-14T09:05:43.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6860999999999995e-05 2024-09-14T09:05:43.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:28.840479+0000", 2024-09-14T09:05:43.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9418999999999994e-05 2024-09-14T09:05:43.140 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.140 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.140 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.141 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:657 3.6 3:65b8693c:::rbd_data.115e1e95bf99.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 524288~51261] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:43.141 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.871474+0000", 2024-09-14T09:05:43.141 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.2247348030000005, 2024-09-14T09:05:43.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0082168940000000006, 2024-09-14T09:05:43.141 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.141 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.141 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.141 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:43.141 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:43.141 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:43.141 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 657 2024-09-14T09:05:43.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.141 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871474+0000", 2024-09-14T09:05:43.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871474+0000", 2024-09-14T09:05:43.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871476+0000", 2024-09-14T09:05:43.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4109999999999999e-06 2024-09-14T09:05:43.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871588+0000", 2024-09-14T09:05:43.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111891 2024-09-14T09:05:43.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871609+0000", 2024-09-14T09:05:43.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0856000000000002e-05 2024-09-14T09:05:43.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871622+0000", 2024-09-14T09:05:43.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3175e-05 2024-09-14T09:05:43.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.871755+0000", 2024-09-14T09:05:43.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013305599999999999 2024-09-14T09:05:43.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872022+0000", 2024-09-14T09:05:43.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026700600000000003 2024-09-14T09:05:43.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:05:43.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872297+0000", 2024-09-14T09:05:43.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027538299999999998 2024-09-14T09:05:43.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:43.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.879290+0000", 2024-09-14T09:05:43.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069921779999999999 2024-09-14T09:05:43.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:43.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.879580+0000", 2024-09-14T09:05:43.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029007000000000002 2024-09-14T09:05:43.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.879619+0000", 2024-09-14T09:05:43.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9311999999999999e-05 2024-09-14T09:05:43.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.879691+0000", 2024-09-14T09:05:43.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1556000000000004e-05 2024-09-14T09:05:43.146 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.146 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.146 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.146 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:673 3.6 3:65b8693c:::rbd_data.115e1e95bf99.0000000000000002:head [stat,zero 13369344~3230342] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:05:43.146 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.885224+0000", 2024-09-14T09:05:43.146 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.2109843799999993, 2024-09-14T09:05:43.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0046878689999999999, 2024-09-14T09:05:43.147 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.147 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.147 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:43.147 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:43.147 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:43.147 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:43.147 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 673 2024-09-14T09:05:43.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.147 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885224+0000", 2024-09-14T09:05:43.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885224+0000", 2024-09-14T09:05:43.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885225+0000", 2024-09-14T09:05:43.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2600000000000002e-07 2024-09-14T09:05:43.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885229+0000", 2024-09-14T09:05:43.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5220000000000001e-06 2024-09-14T09:05:43.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885238+0000", 2024-09-14T09:05:43.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5019999999999992e-06 2024-09-14T09:05:43.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885245+0000", 2024-09-14T09:05:43.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7909999999999999e-06 2024-09-14T09:05:43.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.885481+0000", 2024-09-14T09:05:43.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023645 2024-09-14T09:05:43.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.886357+0000", 2024-09-14T09:05:43.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00087546399999999995 2024-09-14T09:05:43.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.889912+0000", 2024-09-14T09:05:43.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035554139999999998 2024-09-14T09:05:43.151 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.151 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.151 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.151 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:9950 3.17 3:eb19e090:::rbd_data.115eadec874.000000000000000a:head [stat,zero 5379443~11397773] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:43.151 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.551136+0000", 2024-09-14T09:05:43.151 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.5450726350000004, 2024-09-14T09:05:43.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0039621170000000002, 2024-09-14T09:05:43.151 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.152 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.152 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:43.152 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:43.152 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:43.152 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:43.152 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 9950 2024-09-14T09:05:43.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.152 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551136+0000", 2024-09-14T09:05:43.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551136+0000", 2024-09-14T09:05:43.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551136+0000", 2024-09-14T09:05:43.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3000000000000001e-07 2024-09-14T09:05:43.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551139+0000", 2024-09-14T09:05:43.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0010000000000002e-06 2024-09-14T09:05:43.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551147+0000", 2024-09-14T09:05:43.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7379999999999994e-06 2024-09-14T09:05:43.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551153+0000", 2024-09-14T09:05:43.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5300000000000004e-06 2024-09-14T09:05:43.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.552372+0000", 2024-09-14T09:05:43.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001219094 2024-09-14T09:05:43.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.552498+0000", 2024-09-14T09:05:43.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000126453 2024-09-14T09:05:43.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.555098+0000", 2024-09-14T09:05:43.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0025997709999999999 2024-09-14T09:05:43.155 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.155 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.155 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:9987 3.1 3:81ad4cda:::rbd_data.115eadec874.0000000000000002:head [stat,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.663795+0000", 2024-09-14T09:05:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.4324136300000001, 2024-09-14T09:05:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0053806239999999997, 2024-09-14T09:05:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 9987 2024-09-14T09:05:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663795+0000", 2024-09-14T09:05:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663795+0000", 2024-09-14T09:05:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663798+0000", 2024-09-14T09:05:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.01e-06 2024-09-14T09:05:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663812+0000", 2024-09-14T09:05:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4175999999999999e-05 2024-09-14T09:05:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663842+0000", 2024-09-14T09:05:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9461000000000001e-05 2024-09-14T09:05:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663855+0000", 2024-09-14T09:05:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3925e-05 2024-09-14T09:05:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663965+0000", 2024-09-14T09:05:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109437 2024-09-14T09:05:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664066+0000", 2024-09-14T09:05:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010084 2024-09-14T09:05:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664218+0000", 2024-09-14T09:05:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015209500000000001 2024-09-14T09:05:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.665572+0000", 2024-09-14T09:05:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0013543610000000001 2024-09-14T09:05:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.669120+0000", 2024-09-14T09:05:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035477999999999998 2024-09-14T09:05:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.669146+0000", 2024-09-14T09:05:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.618e-05 2024-09-14T09:05:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.669175+0000", 2024-09-14T09:05:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9339000000000001e-05 2024-09-14T09:05:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:9992 3.17 3:ecdf2631:::rbd_data.115eadec874.0000000000000007:head [stat,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.663907+0000", 2024-09-14T09:05:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.4323017780000002, 2024-09-14T09:05:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003508129, 2024-09-14T09:05:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 9992 2024-09-14T09:05:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663907+0000", 2024-09-14T09:05:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663907+0000", 2024-09-14T09:05:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663907+0000", 2024-09-14T09:05:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.13e-07 2024-09-14T09:05:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663912+0000", 2024-09-14T09:05:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0239999999999997e-06 2024-09-14T09:05:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663921+0000", 2024-09-14T09:05:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0860000000000008e-06 2024-09-14T09:05:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663927+0000", 2024-09-14T09:05:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7490000000000004e-06 2024-09-14T09:05:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664065+0000", 2024-09-14T09:05:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013774100000000001 2024-09-14T09:05:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664274+0000", 2024-09-14T09:05:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000209435 2024-09-14T09:05:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:05:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664664+0000", 2024-09-14T09:05:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00038968999999999999 2024-09-14T09:05:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.666044+0000", 2024-09-14T09:05:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00138035 2024-09-14T09:05:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.667331+0000", 2024-09-14T09:05:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012866990000000001 2024-09-14T09:05:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.667378+0000", 2024-09-14T09:05:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6594999999999997e-05 2024-09-14T09:05:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.667415+0000", 2024-09-14T09:05:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7246999999999998e-05 2024-09-14T09:05:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:9995 3.17 3:eb19e090:::rbd_data.115eadec874.000000000000000a:head [stat,truncate 1195683] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.663952+0000", 2024-09-14T09:05:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.4322566969999997, 2024-09-14T09:05:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00472325, 2024-09-14T09:05:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 9995 2024-09-14T09:05:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663952+0000", 2024-09-14T09:05:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663952+0000", 2024-09-14T09:05:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663952+0000", 2024-09-14T09:05:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4700000000000001e-07 2024-09-14T09:05:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663955+0000", 2024-09-14T09:05:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5849999999999998e-06 2024-09-14T09:05:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663962+0000", 2024-09-14T09:05:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9319999999999999e-06 2024-09-14T09:05:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663967+0000", 2024-09-14T09:05:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.853e-06 2024-09-14T09:05:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.665507+0000", 2024-09-14T09:05:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001540521 2024-09-14T09:05:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.665571+0000", 2024-09-14T09:05:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3965000000000002e-05 2024-09-14T09:05:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:05:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.665686+0000", 2024-09-14T09:05:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011476699999999999 2024-09-14T09:05:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.667743+0000", 2024-09-14T09:05:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0020566360000000001 2024-09-14T09:05:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.668605+0000", 2024-09-14T09:05:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00086282899999999996 2024-09-14T09:05:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.668642+0000", 2024-09-14T09:05:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6681999999999999e-05 2024-09-14T09:05:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.668675+0000", 2024-09-14T09:05:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2932999999999998e-05 2024-09-14T09:05:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:10053 3.17 3:ecdf2631:::rbd_data.115eadec874.0000000000000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-14T09:05:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.959492+0000", 2024-09-14T09:05:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.1367162669999997, 2024-09-14T09:05:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0037815129999999998, 2024-09-14T09:05:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 10053 2024-09-14T09:05:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.959492+0000", 2024-09-14T09:05:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.959492+0000", 2024-09-14T09:05:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.959494+0000", 2024-09-14T09:05:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4410000000000001e-06 2024-09-14T09:05:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.959500+0000", 2024-09-14T09:05:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9370000000000001e-06 2024-09-14T09:05:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.959517+0000", 2024-09-14T09:05:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7039999999999999e-05 2024-09-14T09:05:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.959525+0000", 2024-09-14T09:05:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3289999999999994e-06 2024-09-14T09:05:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.959607+0000", 2024-09-14T09:05:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2033000000000006e-05 2024-09-14T09:05:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.959736+0000", 2024-09-14T09:05:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000129009 2024-09-14T09:05:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:05:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.959910+0000", 2024-09-14T09:05:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000173516 2024-09-14T09:05:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.960869+0000", 2024-09-14T09:05:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00095911100000000001 2024-09-14T09:05:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.963164+0000", 2024-09-14T09:05:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0022955710000000002 2024-09-14T09:05:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.963226+0000", 2024-09-14T09:05:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1952999999999995e-05 2024-09-14T09:05:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.963274+0000", 2024-09-14T09:05:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7573e-05 2024-09-14T09:05:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:10057 3.17 3:eb19e090:::rbd_data.115eadec874.000000000000000a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-14T09:05:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.962477+0000", 2024-09-14T09:05:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.1337314640000002, 2024-09-14T09:05:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0085064589999999992, 2024-09-14T09:05:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 10057 2024-09-14T09:05:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.962477+0000", 2024-09-14T09:05:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.962477+0000", 2024-09-14T09:05:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.962478+0000", 2024-09-14T09:05:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0899999999999999e-06 2024-09-14T09:05:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.962483+0000", 2024-09-14T09:05:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6539999999999998e-06 2024-09-14T09:05:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.962498+0000", 2024-09-14T09:05:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4819e-05 2024-09-14T09:05:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.962505+0000", 2024-09-14T09:05:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.807e-06 2024-09-14T09:05:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.962564+0000", 2024-09-14T09:05:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8912000000000001e-05 2024-09-14T09:05:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.962632+0000", 2024-09-14T09:05:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8153999999999998e-05 2024-09-14T09:05:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:05:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.962709+0000", 2024-09-14T09:05:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6625000000000006e-05 2024-09-14T09:05:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.963541+0000", 2024-09-14T09:05:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00083169600000000004 2024-09-14T09:05:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.970881+0000", 2024-09-14T09:05:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0073404990000000003 2024-09-14T09:05:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.970937+0000", 2024-09-14T09:05:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5486999999999999e-05 2024-09-14T09:05:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.970983+0000", 2024-09-14T09:05:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6715999999999998e-05 2024-09-14T09:05:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18085 3.12 3:4be43b13:::rbd_header.115e564e8eff:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:05:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:42.816378+0000", 2024-09-14T09:05:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.279830933, 2024-09-14T09:05:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0077118919999999997, 2024-09-14T09:05:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18085 2024-09-14T09:05:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.816378+0000", 2024-09-14T09:05:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.816378+0000", 2024-09-14T09:05:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.816380+0000", 2024-09-14T09:05:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4389999999999999e-06 2024-09-14T09:05:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.816389+0000", 2024-09-14T09:05:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.037e-06 2024-09-14T09:05:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.816410+0000", 2024-09-14T09:05:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0842000000000001e-05 2024-09-14T09:05:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.816423+0000", 2024-09-14T09:05:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3426e-05 2024-09-14T09:05:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.816510+0000", 2024-09-14T09:05:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.674e-05 2024-09-14T09:05:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T09:05:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.816519+0000", 2024-09-14T09:05:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5560000000000008e-06 2024-09-14T09:05:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.821635+0000", 2024-09-14T09:05:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0051163729999999996 2024-09-14T09:05:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.821747+0000", 2024-09-14T09:05:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111915 2024-09-14T09:05:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:05:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.822187+0000", 2024-09-14T09:05:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043986399999999999 2024-09-14T09:05:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.822933+0000", 2024-09-14T09:05:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00074601699999999997 2024-09-14T09:05:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.824012+0000", 2024-09-14T09:05:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010789040000000001 2024-09-14T09:05:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.824056+0000", 2024-09-14T09:05:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4802999999999998e-05 2024-09-14T09:05:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.824089+0000", 2024-09-14T09:05:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2975999999999997e-05 2024-09-14T09:05:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18209 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:05:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:42.980249+0000", 2024-09-14T09:05:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.115959171, 2024-09-14T09:05:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0035183929999999999, 2024-09-14T09:05:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18209 2024-09-14T09:05:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.980249+0000", 2024-09-14T09:05:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.980249+0000", 2024-09-14T09:05:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.980251+0000", 2024-09-14T09:05:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.75e-06 2024-09-14T09:05:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.980258+0000", 2024-09-14T09:05:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1470000000000004e-06 2024-09-14T09:05:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.980279+0000", 2024-09-14T09:05:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0843e-05 2024-09-14T09:05:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.980289+0000", 2024-09-14T09:05:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0288e-05 2024-09-14T09:05:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.980358+0000", 2024-09-14T09:05:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9010999999999994e-05 2024-09-14T09:05:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.980457+0000", 2024-09-14T09:05:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8325000000000005e-05 2024-09-14T09:05:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:05:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.980651+0000", 2024-09-14T09:05:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000194298 2024-09-14T09:05:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.981717+0000", 2024-09-14T09:05:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010664299999999999 2024-09-14T09:05:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.983660+0000", 2024-09-14T09:05:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0019425740000000001 2024-09-14T09:05:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.983720+0000", 2024-09-14T09:05:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0343999999999997e-05 2024-09-14T09:05:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.983768+0000", 2024-09-14T09:05:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7382999999999997e-05 2024-09-14T09:05:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.197 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:43.199 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:05:43.199 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-14T09:05:43.199 INFO:teuthology.orchestra.run.smithi176.stdout:547 writesame 0x6559446 thru 0x656153e data_size 0x1(0x80f9 bytes) 2024-09-14T09:05:43.200 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:43.200+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:43.213 INFO:teuthology.orchestra.run.smithi176.stdout:548 read 0x769d81d thru 0x76a9ec5 (0xc6a9 bytes) 2024-09-14T09:05:43.215 INFO:teuthology.orchestra.run.smithi176.stdout:549 read 0x4372a5e thru 0x437564a (0x2bed bytes) 2024-09-14T09:05:43.215 INFO:teuthology.orchestra.run.smithi176.stdout:550 read 0x479ddf9 thru 0x479e324 (0x52c bytes) 2024-09-14T09:05:43.215 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:43.215 INFO:teuthology.orchestra.run.smithi176.stdout:553 writesame 0x413252a thru 0x413ab31 data_size 0x2(0x8608 bytes) 2024-09-14T09:05:43.260 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:43.260 INFO:teuthology.orchestra.run.smithi176.stdout:555 punch from 0x8dd5c97 to 0x8dd6cdf, (0x1048 bytes) 2024-09-14T09:05:43.262 INFO:teuthology.orchestra.run.smithi176.stdout:556 trunc from 0xebd11da to 0xd59f57f 2024-09-14T09:05:43.281 INFO:teuthology.orchestra.run.smithi176.stdout:557 write 0x6f77676 thru 0x6f78098 (0xa23 bytes) 2024-09-14T09:05:43.299 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:43.300+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:43.301 INFO:teuthology.orchestra.run.smithi176.stdout:558 read 0x9daf310 thru 0x9dbce97 (0xdb88 bytes) 2024-09-14T09:05:43.317 INFO:teuthology.orchestra.run.smithi176.stdout:559 trunc from 0xd59f57f to 0x6e352c1 2024-09-14T09:05:43.345 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:43.345 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:43.346 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:43.346 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:43.350 INFO:teuthology.orchestra.run.smithi176.stdout:560 clone 6 order 25 su 4194304 sc 11 2024-09-14T09:05:43.360 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:05:43.406 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:43.404+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:43.477 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:43.477 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:43.477 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:43.477 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:43.478 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:43.491 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:05:43.500 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:43.504+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:43.602 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:43.604+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:43.661 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:43.661 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:43.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:43.661 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:43.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.661 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:657 3.6 e29/23 3:65b8693c:::rbd_data.115e1e95bf99.0000000000000002:head v 29'2, mlcod=0'0)", 2024-09-14T09:05:43.661 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.872567+0000", 2024-09-14T09:05:43.661 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.7869088840000007, 2024-09-14T09:05:43.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0056247060000000002, 2024-09-14T09:05:43.661 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.662 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.662 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.662 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872567+0000", 2024-09-14T09:05:43.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872567+0000", 2024-09-14T09:05:43.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872568+0000", 2024-09-14T09:05:43.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T09:05:43.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872772+0000", 2024-09-14T09:05:43.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020363799999999999 2024-09-14T09:05:43.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872799+0000", 2024-09-14T09:05:43.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6917e-05 2024-09-14T09:05:43.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872815+0000", 2024-09-14T09:05:43.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5812999999999999e-05 2024-09-14T09:05:43.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872928+0000", 2024-09-14T09:05:43.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011381499999999999 2024-09-14T09:05:43.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872957+0000", 2024-09-14T09:05:43.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9118000000000001e-05 2024-09-14T09:05:43.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.878076+0000", 2024-09-14T09:05:43.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0051180690000000003 2024-09-14T09:05:43.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.878192+0000", 2024-09-14T09:05:43.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116123 2024-09-14T09:05:43.665 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.665 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.666 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.666 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:963 3.0 3:04479396:::rbd_id.image_client.0-clone2:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T09:05:43.666 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:34.957370+0000", 2024-09-14T09:05:43.666 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.702105972, 2024-09-14T09:05:43.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004759152, 2024-09-14T09:05:43.666 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.666 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.666 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.666 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:43.666 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:43.666 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:43.667 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 963 2024-09-14T09:05:43.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.667 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957370+0000", 2024-09-14T09:05:43.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957370+0000", 2024-09-14T09:05:43.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957372+0000", 2024-09-14T09:05:43.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.475e-06 2024-09-14T09:05:43.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957382+0000", 2024-09-14T09:05:43.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9760000000000007e-06 2024-09-14T09:05:43.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957405+0000", 2024-09-14T09:05:43.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2860000000000001e-05 2024-09-14T09:05:43.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957416+0000", 2024-09-14T09:05:43.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.048e-05 2024-09-14T09:05:43.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957529+0000", 2024-09-14T09:05:43.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112849 2024-09-14T09:05:43.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T09:05:43.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957538+0000", 2024-09-14T09:05:43.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.234e-06 2024-09-14T09:05:43.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.960213+0000", 2024-09-14T09:05:43.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0026753749999999998 2024-09-14T09:05:43.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.960327+0000", 2024-09-14T09:05:43.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114188 2024-09-14T09:05:43.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 2", 2024-09-14T09:05:43.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.960491+0000", 2024-09-14T09:05:43.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016362700000000001 2024-09-14T09:05:43.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:43.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.961472+0000", 2024-09-14T09:05:43.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00098095600000000006 2024-09-14T09:05:43.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:43.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.961990+0000", 2024-09-14T09:05:43.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000518137 2024-09-14T09:05:43.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.962072+0000", 2024-09-14T09:05:43.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2108999999999996e-05 2024-09-14T09:05:43.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.962129+0000", 2024-09-14T09:05:43.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6885999999999997e-05 2024-09-14T09:05:43.672 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.672 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.672 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.672 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:9948 3.19 3:9ed4ec6a:::rbd_data.115eadec874.0000000000000008:head [stat,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:43.672 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.551083+0000", 2024-09-14T09:05:43.673 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.1083932750000001, 2024-09-14T09:05:43.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0021132999999999998, 2024-09-14T09:05:43.673 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.673 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.673 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.673 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:43.673 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:43.673 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:43.673 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 9948 2024-09-14T09:05:43.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.673 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551083+0000", 2024-09-14T09:05:43.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551083+0000", 2024-09-14T09:05:43.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551083+0000", 2024-09-14T09:05:43.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1299999999999998e-07 2024-09-14T09:05:43.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551088+0000", 2024-09-14T09:05:43.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3830000000000002e-06 2024-09-14T09:05:43.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551099+0000", 2024-09-14T09:05:43.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1473e-05 2024-09-14T09:05:43.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551111+0000", 2024-09-14T09:05:43.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1401e-05 2024-09-14T09:05:43.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551181+0000", 2024-09-14T09:05:43.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0114e-05 2024-09-14T09:05:43.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551275+0000", 2024-09-14T09:05:43.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4573000000000004e-05 2024-09-14T09:05:43.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:05:43.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.551443+0000", 2024-09-14T09:05:43.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016756200000000001 2024-09-14T09:05:43.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:43.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.553047+0000", 2024-09-14T09:05:43.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001604463 2024-09-14T09:05:43.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:43.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.553126+0000", 2024-09-14T09:05:43.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8887999999999994e-05 2024-09-14T09:05:43.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.553158+0000", 2024-09-14T09:05:43.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1754000000000001e-05 2024-09-14T09:05:43.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.553196+0000", 2024-09-14T09:05:43.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7775999999999999e-05 2024-09-14T09:05:43.678 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.678 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.678 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.678 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:9982 3.1a e32/23 3:5af3e476:::rbd_data.115eadec874.000000000000000c:head v 32'20, mlcod=32'18)", 2024-09-14T09:05:43.678 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.603737+0000", 2024-09-14T09:05:43.678 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.0557390780000002, 2024-09-14T09:05:43.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001966958, 2024-09-14T09:05:43.679 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.679 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.679 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.679 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.603737+0000", 2024-09-14T09:05:43.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.603737+0000", 2024-09-14T09:05:43.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.603738+0000", 2024-09-14T09:05:43.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3600000000000002e-07 2024-09-14T09:05:43.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.604456+0000", 2024-09-14T09:05:43.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00071852600000000004 2024-09-14T09:05:43.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.604471+0000", 2024-09-14T09:05:43.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4602e-05 2024-09-14T09:05:43.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.604481+0000", 2024-09-14T09:05:43.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0564000000000001e-05 2024-09-14T09:05:43.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.604579+0000", 2024-09-14T09:05:43.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7775999999999994e-05 2024-09-14T09:05:43.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.604596+0000", 2024-09-14T09:05:43.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6524e-05 2024-09-14T09:05:43.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.605671+0000", 2024-09-14T09:05:43.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001075735 2024-09-14T09:05:43.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.605704+0000", 2024-09-14T09:05:43.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2295000000000002e-05 2024-09-14T09:05:43.682 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.683 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.683 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.683 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:9983 3.1a e32/23 3:5a62ac37:::rbd_data.115eadec874.000000000000000d:head v 32'21, mlcod=32'19)", 2024-09-14T09:05:43.683 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.639585+0000", 2024-09-14T09:05:43.683 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.0198912299999998, 2024-09-14T09:05:43.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0046068979999999999, 2024-09-14T09:05:43.683 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.683 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.683 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.683 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639585+0000", 2024-09-14T09:05:43.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639585+0000", 2024-09-14T09:05:43.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639586+0000", 2024-09-14T09:05:43.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6559999999999999e-06 2024-09-14T09:05:43.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639629+0000", 2024-09-14T09:05:43.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2713000000000003e-05 2024-09-14T09:05:43.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639657+0000", 2024-09-14T09:05:43.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7569999999999999e-05 2024-09-14T09:05:43.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639675+0000", 2024-09-14T09:05:43.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8647e-05 2024-09-14T09:05:43.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639791+0000", 2024-09-14T09:05:43.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116229 2024-09-14T09:05:43.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639829+0000", 2024-09-14T09:05:43.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7530000000000002e-05 2024-09-14T09:05:43.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.644164+0000", 2024-09-14T09:05:43.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0043347530000000002 2024-09-14T09:05:43.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.644192+0000", 2024-09-14T09:05:43.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7800000000000001e-05 2024-09-14T09:05:43.687 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.687 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.687 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.687 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:9989 3.4 3:245432af:::rbd_data.115eadec874.0000000000000004:head [stat,zero 2097152~9278230] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:43.687 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.663820+0000", 2024-09-14T09:05:43.688 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.9956560840000002, 2024-09-14T09:05:43.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0021431390000000001, 2024-09-14T09:05:43.688 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.688 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.688 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:43.688 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:43.688 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:43.688 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:43.688 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 9989 2024-09-14T09:05:43.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.688 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663820+0000", 2024-09-14T09:05:43.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663820+0000", 2024-09-14T09:05:43.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663823+0000", 2024-09-14T09:05:43.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0189999999999998e-06 2024-09-14T09:05:43.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663832+0000", 2024-09-14T09:05:43.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0669999999999998e-06 2024-09-14T09:05:43.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663864+0000", 2024-09-14T09:05:43.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2498e-05 2024-09-14T09:05:43.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663875+0000", 2024-09-14T09:05:43.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0878e-05 2024-09-14T09:05:43.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664009+0000", 2024-09-14T09:05:43.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000133831 2024-09-14T09:05:43.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664083+0000", 2024-09-14T09:05:43.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3509000000000004e-05 2024-09-14T09:05:43.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.665963+0000", 2024-09-14T09:05:43.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018803369999999999 2024-09-14T09:05:43.692 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.692 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.692 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.692 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:9993 3.19 3:9ed4ec6a:::rbd_data.115eadec874.0000000000000008:head [stat,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:43.692 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.663928+0000", 2024-09-14T09:05:43.692 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.9955476909999996, 2024-09-14T09:05:43.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0021940670000000001, 2024-09-14T09:05:43.692 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.692 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.693 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.693 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:43.693 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:43.693 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:43.693 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 9993 2024-09-14T09:05:43.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.693 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663928+0000", 2024-09-14T09:05:43.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663928+0000", 2024-09-14T09:05:43.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663929+0000", 2024-09-14T09:05:43.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2200000000000004e-07 2024-09-14T09:05:43.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663932+0000", 2024-09-14T09:05:43.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9529999999999999e-06 2024-09-14T09:05:43.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663940+0000", 2024-09-14T09:05:43.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9319999999999996e-06 2024-09-14T09:05:43.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.663946+0000", 2024-09-14T09:05:43.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8599999999999998e-06 2024-09-14T09:05:43.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664029+0000", 2024-09-14T09:05:43.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3504999999999994e-05 2024-09-14T09:05:43.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664103+0000", 2024-09-14T09:05:43.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3763999999999994e-05 2024-09-14T09:05:43.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:05:43.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.664228+0000", 2024-09-14T09:05:43.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012475999999999999 2024-09-14T09:05:43.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:43.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.665460+0000", 2024-09-14T09:05:43.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001232431 2024-09-14T09:05:43.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:43.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.666046+0000", 2024-09-14T09:05:43.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00058552799999999998 2024-09-14T09:05:43.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.666087+0000", 2024-09-14T09:05:43.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1393999999999997e-05 2024-09-14T09:05:43.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.666122+0000", 2024-09-14T09:05:43.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5318000000000003e-05 2024-09-14T09:05:43.698 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.698 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.698 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.698 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:10029 3.1a e32/23 3:5a62ac37:::rbd_data.115eadec874.000000000000000d:head v 32'25, mlcod=32'23)", 2024-09-14T09:05:43.698 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.762577+0000", 2024-09-14T09:05:43.698 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.8968985309999997, 2024-09-14T09:05:43.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002748028, 2024-09-14T09:05:43.698 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.698 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.698 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.698 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.762577+0000", 2024-09-14T09:05:43.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.762577+0000", 2024-09-14T09:05:43.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.762580+0000", 2024-09-14T09:05:43.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4269999999999998e-06 2024-09-14T09:05:43.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.762681+0000", 2024-09-14T09:05:43.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100869 2024-09-14T09:05:43.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.762709+0000", 2024-09-14T09:05:43.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8243e-05 2024-09-14T09:05:43.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.762722+0000", 2024-09-14T09:05:43.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2935999999999999e-05 2024-09-14T09:05:43.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.762863+0000", 2024-09-14T09:05:43.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014083300000000001 2024-09-14T09:05:43.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.762922+0000", 2024-09-14T09:05:43.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9373000000000002e-05 2024-09-14T09:05:43.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.765221+0000", 2024-09-14T09:05:43.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0022990179999999999 2024-09-14T09:05:43.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.765325+0000", 2024-09-14T09:05:43.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000104329 2024-09-14T09:05:43.702 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.702 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.702 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.702 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:10032 3.19 3:9ed4ec6a:::rbd_data.115eadec874.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:43.702 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.775706+0000", 2024-09-14T09:05:43.702 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.8837695229999998, 2024-09-14T09:05:43.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126247939, 2024-09-14T09:05:43.703 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.703 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.703 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.703 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:43.703 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:43.703 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:43.703 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 10032 2024-09-14T09:05:43.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.703 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775706+0000", 2024-09-14T09:05:43.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775706+0000", 2024-09-14T09:05:43.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775709+0000", 2024-09-14T09:05:43.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0400000000000001e-06 2024-09-14T09:05:43.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775721+0000", 2024-09-14T09:05:43.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2119999999999999e-05 2024-09-14T09:05:43.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775752+0000", 2024-09-14T09:05:43.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0139999999999999e-05 2024-09-14T09:05:43.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775766+0000", 2024-09-14T09:05:43.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4497e-05 2024-09-14T09:05:43.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775937+0000", 2024-09-14T09:05:43.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000171022 2024-09-14T09:05:43.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.776205+0000", 2024-09-14T09:05:43.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026749100000000002 2024-09-14T09:05:43.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:05:43.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.784970+0000", 2024-09-14T09:05:43.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0087648789999999997 2024-09-14T09:05:43.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:43.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901870+0000", 2024-09-14T09:05:43.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11690028 2024-09-14T09:05:43.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:43.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901878+0000", 2024-09-14T09:05:43.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7020000000000007e-06 2024-09-14T09:05:43.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901913+0000", 2024-09-14T09:05:43.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4706000000000003e-05 2024-09-14T09:05:43.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901954+0000", 2024-09-14T09:05:43.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1062000000000001e-05 2024-09-14T09:05:43.708 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.708 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.708 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.708 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:10032 3.19 e32/23 ondisk, result = 0)", 2024-09-14T09:05:43.708 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.809095+0000", 2024-09-14T09:05:43.708 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.8503812440000003, 2024-09-14T09:05:43.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092866664000000002, 2024-09-14T09:05:43.708 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.708 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.709 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:43.709 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809095+0000", 2024-09-14T09:05:43.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809095+0000", 2024-09-14T09:05:43.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809096+0000", 2024-09-14T09:05:43.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.64e-06 2024-09-14T09:05:43.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809104+0000", 2024-09-14T09:05:43.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7330000000000003e-06 2024-09-14T09:05:43.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809132+0000", 2024-09-14T09:05:43.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7880000000000001e-05 2024-09-14T09:05:43.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809146+0000", 2024-09-14T09:05:43.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4514e-05 2024-09-14T09:05:43.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901858+0000", 2024-09-14T09:05:43.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092711713000000001 2024-09-14T09:05:43.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901866+0000", 2024-09-14T09:05:43.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4760000000000001e-06 2024-09-14T09:05:43.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901961+0000", 2024-09-14T09:05:43.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5707999999999999e-05 2024-09-14T09:05:43.712 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.712 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.712 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.712 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:10035 3.4 3:245432af:::rbd_data.115eadec874.0000000000000004:head [stat,write 9104681~23750] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-14T09:05:43.712 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.907563+0000", 2024-09-14T09:05:43.712 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.7519126539999998, 2024-09-14T09:05:43.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0029432270000000001, 2024-09-14T09:05:43.713 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.713 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.713 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.713 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:43.713 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:43.713 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:43.713 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 10035 2024-09-14T09:05:43.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.713 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.907563+0000", 2024-09-14T09:05:43.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.907563+0000", 2024-09-14T09:05:43.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.907566+0000", 2024-09-14T09:05:43.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6089999999999999e-06 2024-09-14T09:05:43.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.907601+0000", 2024-09-14T09:05:43.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4882000000000002e-05 2024-09-14T09:05:43.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.907624+0000", 2024-09-14T09:05:43.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3090000000000001e-05 2024-09-14T09:05:43.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.907635+0000", 2024-09-14T09:05:43.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1583000000000001e-05 2024-09-14T09:05:43.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.907721+0000", 2024-09-14T09:05:43.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6099000000000004e-05 2024-09-14T09:05:43.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.907837+0000", 2024-09-14T09:05:43.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011521799999999999 2024-09-14T09:05:43.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 2", 2024-09-14T09:05:43.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.908022+0000", 2024-09-14T09:05:43.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018563100000000001 2024-09-14T09:05:43.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:43.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.910209+0000", 2024-09-14T09:05:43.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002186871 2024-09-14T09:05:43.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:43.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.910439+0000", 2024-09-14T09:05:43.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000230163 2024-09-14T09:05:43.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.910475+0000", 2024-09-14T09:05:43.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5867000000000001e-05 2024-09-14T09:05:43.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.910506+0000", 2024-09-14T09:05:43.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1214000000000003e-05 2024-09-14T09:05:43.718 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.718 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.718 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.718 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:10166 3.4 3:245432af:::rbd_data.115eadec874.0000000000000004:head [stat,zero 8388608~6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-14T09:05:43.718 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:38.157947+0000", 2024-09-14T09:05:43.718 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.5015292840000001, 2024-09-14T09:05:43.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0028609999999999998, 2024-09-14T09:05:43.718 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.719 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.719 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:43.719 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:43.719 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:43.719 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:43.719 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 10166 2024-09-14T09:05:43.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.719 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.157947+0000", 2024-09-14T09:05:43.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.157947+0000", 2024-09-14T09:05:43.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.157949+0000", 2024-09-14T09:05:43.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5330000000000001e-06 2024-09-14T09:05:43.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.157958+0000", 2024-09-14T09:05:43.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3430000000000002e-06 2024-09-14T09:05:43.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.157986+0000", 2024-09-14T09:05:43.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7719e-05 2024-09-14T09:05:43.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158010+0000", 2024-09-14T09:05:43.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4009e-05 2024-09-14T09:05:43.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158163+0000", 2024-09-14T09:05:43.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000153051 2024-09-14T09:05:43.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158379+0000", 2024-09-14T09:05:43.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021535799999999999 2024-09-14T09:05:43.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.160808+0000", 2024-09-14T09:05:43.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0024289870000000001 2024-09-14T09:05:43.722 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.723 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.723 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.723 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:10170 3.1a e33/23 3:5af3e476:::rbd_data.115eadec874.000000000000000c:head v 33'29, mlcod=33'27)", 2024-09-14T09:05:43.723 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:38.158629+0000", 2024-09-14T09:05:43.723 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.5008469660000001, 2024-09-14T09:05:43.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0036077679999999999, 2024-09-14T09:05:43.723 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.723 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.723 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.723 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158629+0000", 2024-09-14T09:05:43.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158629+0000", 2024-09-14T09:05:43.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158631+0000", 2024-09-14T09:05:43.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5170000000000002e-06 2024-09-14T09:05:43.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158648+0000", 2024-09-14T09:05:43.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6520000000000001e-05 2024-09-14T09:05:43.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158678+0000", 2024-09-14T09:05:43.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.006e-05 2024-09-14T09:05:43.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158697+0000", 2024-09-14T09:05:43.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9145999999999999e-05 2024-09-14T09:05:43.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158911+0000", 2024-09-14T09:05:43.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021399799999999999 2024-09-14T09:05:43.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.158964+0000", 2024-09-14T09:05:43.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2846999999999997e-05 2024-09-14T09:05:43.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.162172+0000", 2024-09-14T09:05:43.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032080289999999998 2024-09-14T09:05:43.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.162237+0000", 2024-09-14T09:05:43.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4651000000000002e-05 2024-09-14T09:05:43.727 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.727 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.727 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.727 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:10182 3.1a e33/23 3:5a62ac37:::rbd_data.115eadec874.000000000000000d:head v 33'30, mlcod=33'28)", 2024-09-14T09:05:43.727 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:38.175853+0000", 2024-09-14T09:05:43.727 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.4836232960000002, 2024-09-14T09:05:43.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0061618740000000003, 2024-09-14T09:05:43.728 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.728 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.728 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.728 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175853+0000", 2024-09-14T09:05:43.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175853+0000", 2024-09-14T09:05:43.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175854+0000", 2024-09-14T09:05:43.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8169999999999999e-06 2024-09-14T09:05:43.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175914+0000", 2024-09-14T09:05:43.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9812999999999998e-05 2024-09-14T09:05:43.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175941+0000", 2024-09-14T09:05:43.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6786000000000001e-05 2024-09-14T09:05:43.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175955+0000", 2024-09-14T09:05:43.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.45e-05 2024-09-14T09:05:43.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.176049+0000", 2024-09-14T09:05:43.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3801000000000005e-05 2024-09-14T09:05:43.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.176083+0000", 2024-09-14T09:05:43.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.362e-05 2024-09-14T09:05:43.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.181952+0000", 2024-09-14T09:05:43.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0058688170000000001 2024-09-14T09:05:43.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.182014+0000", 2024-09-14T09:05:43.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2719999999999996e-05 2024-09-14T09:05:43.731 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.732 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.732 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.732 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "pg_update_log_missing(3.1a epoch 33/23 rep_tid 178 entries 33'31 (0'0) error 3:5af3e476:::rbd_data.115eadec874.000000000000000c:head by client.4446.0:10201 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 33'29)", 2024-09-14T09:05:43.732 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:38.217229+0000", 2024-09-14T09:05:43.732 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.4422469839999996, 2024-09-14T09:05:43.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0029007220000000001, 2024-09-14T09:05:43.732 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.732 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.732 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:43.732 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217229+0000", 2024-09-14T09:05:43.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217229+0000", 2024-09-14T09:05:43.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217230+0000", 2024-09-14T09:05:43.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.049e-06 2024-09-14T09:05:43.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217240+0000", 2024-09-14T09:05:43.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9159999999999996e-06 2024-09-14T09:05:43.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217269+0000", 2024-09-14T09:05:43.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9377e-05 2024-09-14T09:05:43.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217285+0000", 2024-09-14T09:05:43.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6212e-05 2024-09-14T09:05:43.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217399+0000", 2024-09-14T09:05:43.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000113989 2024-09-14T09:05:43.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.220130+0000", 2024-09-14T09:05:43.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0027301790000000001 2024-09-14T09:05:43.735 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.735 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.736 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.736 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "pg_update_log_missing(3.1a epoch 33/23 rep_tid 180 entries 33'32 (0'0) error 3:5a62ac37:::rbd_data.115eadec874.000000000000000d:head by client.4446.0:10202 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 33'29)", 2024-09-14T09:05:43.736 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:38.217605+0000", 2024-09-14T09:05:43.736 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.4418704, 2024-09-14T09:05:43.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002797384, 2024-09-14T09:05:43.736 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.736 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.736 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:43.736 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217605+0000", 2024-09-14T09:05:43.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217605+0000", 2024-09-14T09:05:43.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217606+0000", 2024-09-14T09:05:43.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5000000000000002e-07 2024-09-14T09:05:43.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217614+0000", 2024-09-14T09:05:43.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7440000000000007e-06 2024-09-14T09:05:43.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217639+0000", 2024-09-14T09:05:43.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4811e-05 2024-09-14T09:05:43.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217654+0000", 2024-09-14T09:05:43.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5169e-05 2024-09-14T09:05:43.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.217822+0000", 2024-09-14T09:05:43.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016851 2024-09-14T09:05:43.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.220403+0000", 2024-09-14T09:05:43.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0025804999999999999 2024-09-14T09:05:43.739 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.739 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.739 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.740 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:18071 3.18 e37/28 3:1b44cc8b:::rbd_data.115e564e8eff.0000000000000009:head v 37'21, mlcod=37'19)", 2024-09-14T09:05:43.740 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:42.314712+0000", 2024-09-14T09:05:43.740 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.3447640279999999, 2024-09-14T09:05:43.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0019777620000000001, 2024-09-14T09:05:43.740 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.740 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.740 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.740 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.314712+0000", 2024-09-14T09:05:43.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.314712+0000", 2024-09-14T09:05:43.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.314713+0000", 2024-09-14T09:05:43.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1340000000000001e-06 2024-09-14T09:05:43.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.314726+0000", 2024-09-14T09:05:43.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3536000000000001e-05 2024-09-14T09:05:43.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.314740+0000", 2024-09-14T09:05:43.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3147e-05 2024-09-14T09:05:43.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.314746+0000", 2024-09-14T09:05:43.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2809999999999997e-06 2024-09-14T09:05:43.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.314814+0000", 2024-09-14T09:05:43.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8124e-05 2024-09-14T09:05:43.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.314838+0000", 2024-09-14T09:05:43.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4459000000000001e-05 2024-09-14T09:05:43.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.316655+0000", 2024-09-14T09:05:43.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001816345 2024-09-14T09:05:43.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.316690+0000", 2024-09-14T09:05:43.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4736000000000001e-05 2024-09-14T09:05:43.744 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.744 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.744 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.744 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:18171 3.1c e38/34 3:3831950c:::rbd_directory:head v 38'51, mlcod=31'49)", 2024-09-14T09:05:43.745 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:42.927360+0000", 2024-09-14T09:05:43.745 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.73211632100000001, 2024-09-14T09:05:43.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0022107540000000001, 2024-09-14T09:05:43.745 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.745 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.745 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.745 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.927360+0000", 2024-09-14T09:05:43.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.927360+0000", 2024-09-14T09:05:43.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.927360+0000", 2024-09-14T09:05:43.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2999999999999999e-07 2024-09-14T09:05:43.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.927368+0000", 2024-09-14T09:05:43.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.67e-06 2024-09-14T09:05:43.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.927379+0000", 2024-09-14T09:05:43.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0417e-05 2024-09-14T09:05:43.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.927393+0000", 2024-09-14T09:05:43.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4453999999999999e-05 2024-09-14T09:05:43.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.927536+0000", 2024-09-14T09:05:43.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014308999999999999 2024-09-14T09:05:43.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.927585+0000", 2024-09-14T09:05:43.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8350000000000003e-05 2024-09-14T09:05:43.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.929405+0000", 2024-09-14T09:05:43.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001820428 2024-09-14T09:05:43.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.929570+0000", 2024-09-14T09:05:43.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016511499999999999 2024-09-14T09:05:43.749 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.749 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.749 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.749 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:18207 3.e e38/34 3:777671c3:::rbd_data.115e564e8eff.0000000000000000:head v 38'24, mlcod=37'21)", 2024-09-14T09:05:43.749 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:42.981217+0000", 2024-09-14T09:05:43.749 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.678258417, 2024-09-14T09:05:43.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0037743220000000001, 2024-09-14T09:05:43.749 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.750 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.750 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.750 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.981217+0000", 2024-09-14T09:05:43.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.981217+0000", 2024-09-14T09:05:43.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.981218+0000", 2024-09-14T09:05:43.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.97e-07 2024-09-14T09:05:43.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.981225+0000", 2024-09-14T09:05:43.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9940000000000003e-06 2024-09-14T09:05:43.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.981250+0000", 2024-09-14T09:05:43.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5602999999999999e-05 2024-09-14T09:05:43.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.981258+0000", 2024-09-14T09:05:43.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9310000000000008e-06 2024-09-14T09:05:43.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.981320+0000", 2024-09-14T09:05:43.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1620999999999998e-05 2024-09-14T09:05:43.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.981345+0000", 2024-09-14T09:05:43.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5228999999999998e-05 2024-09-14T09:05:43.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.984960+0000", 2024-09-14T09:05:43.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0036145919999999998 2024-09-14T09:05:43.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.984992+0000", 2024-09-14T09:05:43.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2155e-05 2024-09-14T09:05:43.754 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.754 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.754 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.754 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:18208 3.a e38/34 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:head v 38'30, mlcod=37'27)", 2024-09-14T09:05:43.754 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:42.981435+0000", 2024-09-14T09:05:43.754 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.67804072100000001, 2024-09-14T09:05:43.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0042290139999999997, 2024-09-14T09:05:43.754 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:43.754 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:43.755 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:43.755 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:43.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:43.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.981435+0000", 2024-09-14T09:05:43.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:43.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.981435+0000", 2024-09-14T09:05:43.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:43.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:43.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.981436+0000", 2024-09-14T09:05:43.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.23e-07 2024-09-14T09:05:43.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:43.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.981443+0000", 2024-09-14T09:05:43.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9940000000000003e-06 2024-09-14T09:05:43.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:43.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.981456+0000", 2024-09-14T09:05:43.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3190000000000001e-05 2024-09-14T09:05:43.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:43.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.981464+0000", 2024-09-14T09:05:43.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3720000000000005e-06 2024-09-14T09:05:43.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:43.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.981523+0000", 2024-09-14T09:05:43.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8891000000000002e-05 2024-09-14T09:05:43.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:43.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.981542+0000", 2024-09-14T09:05:43.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8807000000000001e-05 2024-09-14T09:05:43.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:43.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.985637+0000", 2024-09-14T09:05:43.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0040945950000000004 2024-09-14T09:05:43.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:43.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:43.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:43.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.985664+0000", 2024-09-14T09:05:43.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7342e-05 2024-09-14T09:05:43.758 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.758 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.758 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.758 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:43.758 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:43.759 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:43.760 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:05:43.760 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:43.756+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:43.861 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:43.864+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:43.916 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:05:43.916 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:05:43.916 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-14T09:05:43.916 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T09:05:43.916 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T09:05:43.916 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-14T09:05:43.923 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is unset 2024-09-14T09:05:43.933 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:43.933 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:43.933 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:43.933 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:43.952 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:05:43.961 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:43.964+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:44.055 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:05:44.055 INFO:teuthology.orchestra.run.smithi183.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T09:05:44.055 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:05:44.062 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone5 from 0x6e352c1 (overlap 0x6e352c1) to 0x210ff39 2024-09-14T09:05:44.062 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:44.060+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:44.088 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:44.088 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:44.088 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:44.088 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:44.088 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:44.110 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:05:44.202 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:44.200+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:44.264 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:44.260+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:44.270 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-14T09:05:44.273 INFO:teuthology.orchestra.run.smithi176.stdout:562 read 0x3e2f098 thru 0x3e37a7b (0x89e4 bytes) 2024-09-14T09:05:44.289 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:44.289 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:44.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:44.289 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:44.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.290 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:845 3.9 e29/23 3:928dad78:::rbd_data.115e1e95bf99.0000000000000007:head v 29'3, mlcod=26'1)", 2024-09-14T09:05:44.290 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.921987+0000", 2024-09-14T09:05:44.290 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.365792129999999, 2024-09-14T09:05:44.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0020543100000000002, 2024-09-14T09:05:44.290 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.290 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.290 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:44.290 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.921987+0000", 2024-09-14T09:05:44.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.921987+0000", 2024-09-14T09:05:44.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.921989+0000", 2024-09-14T09:05:44.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8700000000000001e-06 2024-09-14T09:05:44.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.921999+0000", 2024-09-14T09:05:44.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9310000000000003e-06 2024-09-14T09:05:44.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.922020+0000", 2024-09-14T09:05:44.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1080000000000001e-05 2024-09-14T09:05:44.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.922031+0000", 2024-09-14T09:05:44.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1374e-05 2024-09-14T09:05:44.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.922161+0000", 2024-09-14T09:05:44.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013039400000000001 2024-09-14T09:05:44.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:44.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.922197+0000", 2024-09-14T09:05:44.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5497999999999998e-05 2024-09-14T09:05:44.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:44.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.923987+0000", 2024-09-14T09:05:44.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0017899120000000001 2024-09-14T09:05:44.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.924041+0000", 2024-09-14T09:05:44.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4250999999999999e-05 2024-09-14T09:05:44.294 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.294 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.294 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.294 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:10035 3.4 e33/23 3:245432af:::rbd_data.115eadec874.0000000000000004:head v 33'13, mlcod=32'11)", 2024-09-14T09:05:44.294 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.908213+0000", 2024-09-14T09:05:44.294 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.3795659069999999, 2024-09-14T09:05:44.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002085834, 2024-09-14T09:05:44.295 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.295 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.295 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:44.295 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.908213+0000", 2024-09-14T09:05:44.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.908213+0000", 2024-09-14T09:05:44.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.908213+0000", 2024-09-14T09:05:44.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8900000000000001e-07 2024-09-14T09:05:44.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.908232+0000", 2024-09-14T09:05:44.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9057000000000001e-05 2024-09-14T09:05:44.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.908252+0000", 2024-09-14T09:05:44.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9548999999999999e-05 2024-09-14T09:05:44.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.908262+0000", 2024-09-14T09:05:44.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0543000000000001e-05 2024-09-14T09:05:44.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.908333+0000", 2024-09-14T09:05:44.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0413000000000005e-05 2024-09-14T09:05:44.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:44.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.908362+0000", 2024-09-14T09:05:44.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8963999999999999e-05 2024-09-14T09:05:44.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:44.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.910263+0000", 2024-09-14T09:05:44.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0019008600000000001 2024-09-14T09:05:44.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.910299+0000", 2024-09-14T09:05:44.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6158999999999997e-05 2024-09-14T09:05:44.298 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.298 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.298 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.298 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:18175 3.1f e38/28 3:fbcf3b0f:::rbd_object_map.115e3abd3b2c:head v 38'10, mlcod=29'8)", 2024-09-14T09:05:44.299 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:42.939142+0000", 2024-09-14T09:05:44.299 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.3486369069999999, 2024-09-14T09:05:44.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001602951, 2024-09-14T09:05:44.299 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.299 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.299 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:44.299 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.939142+0000", 2024-09-14T09:05:44.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.939142+0000", 2024-09-14T09:05:44.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.939143+0000", 2024-09-14T09:05:44.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8999999999999996e-07 2024-09-14T09:05:44.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.939150+0000", 2024-09-14T09:05:44.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8199999999999997e-06 2024-09-14T09:05:44.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.939178+0000", 2024-09-14T09:05:44.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7203e-05 2024-09-14T09:05:44.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.939198+0000", 2024-09-14T09:05:44.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0775999999999999e-05 2024-09-14T09:05:44.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.939273+0000", 2024-09-14T09:05:44.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4323e-05 2024-09-14T09:05:44.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:44.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.939288+0000", 2024-09-14T09:05:44.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5268999999999999e-05 2024-09-14T09:05:44.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:44.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.940620+0000", 2024-09-14T09:05:44.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0013317680000000001 2024-09-14T09:05:44.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:42.940745+0000", 2024-09-14T09:05:44.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012510200000000001 2024-09-14T09:05:44.302 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.303 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.303 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.303 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18383 3.9 3:9252f7c9:::rbd_data.115e3abd3b2c.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:05:44.303 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:05:44.303 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.1293570500000001, 2024-09-14T09:05:44.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056433830999999997, 2024-09-14T09:05:44.303 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.303 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.303 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:44.303 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:44.303 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:44.303 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:44.304 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18383 2024-09-14T09:05:44.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.304 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:05:44.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:05:44.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158423+0000", 2024-09-14T09:05:44.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T09:05:44.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158429+0000", 2024-09-14T09:05:44.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0179999999999998e-06 2024-09-14T09:05:44.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158440+0000", 2024-09-14T09:05:44.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0769e-05 2024-09-14T09:05:44.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158445+0000", 2024-09-14T09:05:44.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5840000000000002e-06 2024-09-14T09:05:44.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158495+0000", 2024-09-14T09:05:44.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9129999999999999e-05 2024-09-14T09:05:44.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:44.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158598+0000", 2024-09-14T09:05:44.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102938 2024-09-14T09:05:44.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:05:44.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.161048+0000", 2024-09-14T09:05:44.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002450336 2024-09-14T09:05:44.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:44.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214605+0000", 2024-09-14T09:05:44.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053557526000000001 2024-09-14T09:05:44.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:44.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214634+0000", 2024-09-14T09:05:44.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8483000000000002e-05 2024-09-14T09:05:44.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:44.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214731+0000", 2024-09-14T09:05:44.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7026000000000002e-05 2024-09-14T09:05:44.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214856+0000", 2024-09-14T09:05:44.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012474799999999999 2024-09-14T09:05:44.309 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.309 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.309 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.309 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18383 3.9 e38/34 ondisk, result = 0)", 2024-09-14T09:05:44.309 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:05:44.309 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.119040569, 2024-09-14T09:05:44.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045880575999999999, 2024-09-14T09:05:44.310 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.310 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.310 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:44.310 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:05:44.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:05:44.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168739+0000", 2024-09-14T09:05:44.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3900000000000004e-07 2024-09-14T09:05:44.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168744+0000", 2024-09-14T09:05:44.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5929999999999999e-06 2024-09-14T09:05:44.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168754+0000", 2024-09-14T09:05:44.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0686999999999999e-05 2024-09-14T09:05:44.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168761+0000", 2024-09-14T09:05:44.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6340000000000003e-06 2024-09-14T09:05:44.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214578+0000", 2024-09-14T09:05:44.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045817443999999999 2024-09-14T09:05:44.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:44.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214596+0000", 2024-09-14T09:05:44.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7159e-05 2024-09-14T09:05:44.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214619+0000", 2024-09-14T09:05:44.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.332e-05 2024-09-14T09:05:44.313 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.313 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.313 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.313 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:18387 3.1f e38/28 3:fbcf3b0f:::rbd_object_map.115e3abd3b2c:head v 38'14, mlcod=38'12)", 2024-09-14T09:05:44.313 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.217716+0000", 2024-09-14T09:05:44.314 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.0700627869999999, 2024-09-14T09:05:44.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018755989999999999, 2024-09-14T09:05:44.314 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.314 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.314 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:44.314 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.217716+0000", 2024-09-14T09:05:44.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.217716+0000", 2024-09-14T09:05:44.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.217718+0000", 2024-09-14T09:05:44.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7600000000000001e-06 2024-09-14T09:05:44.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.217729+0000", 2024-09-14T09:05:44.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0888e-05 2024-09-14T09:05:44.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.217746+0000", 2024-09-14T09:05:44.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7153999999999999e-05 2024-09-14T09:05:44.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.217756+0000", 2024-09-14T09:05:44.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0259e-05 2024-09-14T09:05:44.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.217914+0000", 2024-09-14T09:05:44.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015825100000000001 2024-09-14T09:05:44.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:44.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.217980+0000", 2024-09-14T09:05:44.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5895999999999994e-05 2024-09-14T09:05:44.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:44.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.219456+0000", 2024-09-14T09:05:44.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0014753290000000001 2024-09-14T09:05:44.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.219592+0000", 2024-09-14T09:05:44.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000136062 2024-09-14T09:05:44.317 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.317 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.318 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.318 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:18422 3.1f e38/28 3:fbcf3b0f:::rbd_object_map.115e3abd3b2c:head v 38'20, mlcod=38'18)", 2024-09-14T09:05:44.318 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.290659+0000", 2024-09-14T09:05:44.318 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.99711998599999996, 2024-09-14T09:05:44.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088581270000000004, 2024-09-14T09:05:44.318 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.318 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.318 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:44.318 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.290659+0000", 2024-09-14T09:05:44.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.290659+0000", 2024-09-14T09:05:44.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.290661+0000", 2024-09-14T09:05:44.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2450000000000001e-06 2024-09-14T09:05:44.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.290672+0000", 2024-09-14T09:05:44.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1361e-05 2024-09-14T09:05:44.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.290694+0000", 2024-09-14T09:05:44.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1243e-05 2024-09-14T09:05:44.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.290707+0000", 2024-09-14T09:05:44.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3261999999999999e-05 2024-09-14T09:05:44.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.290796+0000", 2024-09-14T09:05:44.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8894999999999995e-05 2024-09-14T09:05:44.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:44.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.290829+0000", 2024-09-14T09:05:44.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3519999999999998e-05 2024-09-14T09:05:44.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:44.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.299457+0000", 2024-09-14T09:05:44.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0086277290000000006 2024-09-14T09:05:44.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.299517+0000", 2024-09-14T09:05:44.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9871999999999999e-05 2024-09-14T09:05:44.322 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.322 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.322 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.322 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:18600 3.1f e39/28 3:fbcf3b0f:::rbd_object_map.115e3abd3b2c:head v 39'30, mlcod=38'28)", 2024-09-14T09:05:44.322 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.065496+0000", 2024-09-14T09:05:44.322 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.222282433, 2024-09-14T09:05:44.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0016409650000000001, 2024-09-14T09:05:44.323 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.323 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.323 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:44.323 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.065496+0000", 2024-09-14T09:05:44.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.065496+0000", 2024-09-14T09:05:44.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.065497+0000", 2024-09-14T09:05:44.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.02e-07 2024-09-14T09:05:44.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.065504+0000", 2024-09-14T09:05:44.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0029999999999997e-06 2024-09-14T09:05:44.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.065516+0000", 2024-09-14T09:05:44.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1493e-05 2024-09-14T09:05:44.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.065522+0000", 2024-09-14T09:05:44.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1700000000000002e-06 2024-09-14T09:05:44.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.065657+0000", 2024-09-14T09:05:44.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013522499999999999 2024-09-14T09:05:44.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:44.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.065721+0000", 2024-09-14T09:05:44.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4074999999999999e-05 2024-09-14T09:05:44.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:44.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.067094+0000", 2024-09-14T09:05:44.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0013729980000000001 2024-09-14T09:05:44.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.067137+0000", 2024-09-14T09:05:44.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3099000000000002e-05 2024-09-14T09:05:44.326 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.326 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.326 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.327 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18609 3.9 3:919bb0a3:::rbd_data.115e3abd3b2c.000000000000001c:head [stat,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:44.327 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.068381+0000", 2024-09-14T09:05:44.327 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.219398285, 2024-09-14T09:05:44.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0042100339999999997, 2024-09-14T09:05:44.327 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.327 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.327 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:44.327 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:44.327 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:44.327 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:44.327 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18609 2024-09-14T09:05:44.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.327 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.068381+0000", 2024-09-14T09:05:44.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.068381+0000", 2024-09-14T09:05:44.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.068382+0000", 2024-09-14T09:05:44.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.781e-06 2024-09-14T09:05:44.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.068389+0000", 2024-09-14T09:05:44.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5300000000000002e-06 2024-09-14T09:05:44.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.068407+0000", 2024-09-14T09:05:44.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8053000000000001e-05 2024-09-14T09:05:44.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.068416+0000", 2024-09-14T09:05:44.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8529999999999998e-06 2024-09-14T09:05:44.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.068546+0000", 2024-09-14T09:05:44.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000130371 2024-09-14T09:05:44.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:44.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.068803+0000", 2024-09-14T09:05:44.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025695999999999998 2024-09-14T09:05:44.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.072591+0000", 2024-09-14T09:05:44.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0037874860000000001 2024-09-14T09:05:44.331 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.331 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.331 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.332 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "pg_update_log_missing(3.13 epoch 39/34 rep_tid 200 entries 39'8 (0'0) error 3:c8d5a11c:::rbd_data.115e3abd3b2c.0000000000000019:head by client.4446.0:18606 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 36'5)", 2024-09-14T09:05:44.332 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.070317+0000", 2024-09-14T09:05:44.332 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.21746204699999999, 2024-09-14T09:05:44.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0015385150000000001, 2024-09-14T09:05:44.332 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.332 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.332 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:44.332 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.070317+0000", 2024-09-14T09:05:44.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.070317+0000", 2024-09-14T09:05:44.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.070317+0000", 2024-09-14T09:05:44.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3499999999999996e-07 2024-09-14T09:05:44.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.070323+0000", 2024-09-14T09:05:44.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.835e-06 2024-09-14T09:05:44.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.070336+0000", 2024-09-14T09:05:44.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3224000000000001e-05 2024-09-14T09:05:44.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.070350+0000", 2024-09-14T09:05:44.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3841e-05 2024-09-14T09:05:44.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.070586+0000", 2024-09-14T09:05:44.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000235407 2024-09-14T09:05:44.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.071855+0000", 2024-09-14T09:05:44.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001269573 2024-09-14T09:05:44.335 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.335 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.335 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.335 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18829 3.9 3:919bb0a3:::rbd_data.115e3abd3b2c.000000000000001c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:44.335 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.091474+0000", 2024-09-14T09:05:44.336 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.1963048, 2024-09-14T09:05:44.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0022268819999999999, 2024-09-14T09:05:44.336 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.336 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.336 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:44.336 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:44.336 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:44.336 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:44.336 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18829 2024-09-14T09:05:44.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.336 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091474+0000", 2024-09-14T09:05:44.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091474+0000", 2024-09-14T09:05:44.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091476+0000", 2024-09-14T09:05:44.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4339999999999999e-06 2024-09-14T09:05:44.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091486+0000", 2024-09-14T09:05:44.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8300000000000008e-06 2024-09-14T09:05:44.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091512+0000", 2024-09-14T09:05:44.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5599999999999999e-05 2024-09-14T09:05:44.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091524+0000", 2024-09-14T09:05:44.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1764e-05 2024-09-14T09:05:44.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091602+0000", 2024-09-14T09:05:44.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8308999999999999e-05 2024-09-14T09:05:44.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:44.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091726+0000", 2024-09-14T09:05:44.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012428499999999999 2024-09-14T09:05:44.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:05:44.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091982+0000", 2024-09-14T09:05:44.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025607000000000001 2024-09-14T09:05:44.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:44.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.093265+0000", 2024-09-14T09:05:44.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001282367 2024-09-14T09:05:44.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:44.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.093580+0000", 2024-09-14T09:05:44.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031529300000000002 2024-09-14T09:05:44.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:44.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.093640+0000", 2024-09-14T09:05:44.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9877000000000003e-05 2024-09-14T09:05:44.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.093701+0000", 2024-09-14T09:05:44.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1053e-05 2024-09-14T09:05:44.341 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.341 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.341 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.342 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18833 3.9 3:928e32ea:::rbd_data.115e3abd3b2c.000000000000001f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:44.342 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.095528+0000", 2024-09-14T09:05:44.342 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.19225087099999999, 2024-09-14T09:05:44.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003845827, 2024-09-14T09:05:44.342 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.342 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.342 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:44.342 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:44.342 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:44.342 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:44.342 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18833 2024-09-14T09:05:44.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.342 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095528+0000", 2024-09-14T09:05:44.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095528+0000", 2024-09-14T09:05:44.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095529+0000", 2024-09-14T09:05:44.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.491e-06 2024-09-14T09:05:44.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095536+0000", 2024-09-14T09:05:44.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5749999999999997e-06 2024-09-14T09:05:44.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095549+0000", 2024-09-14T09:05:44.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3427999999999999e-05 2024-09-14T09:05:44.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095558+0000", 2024-09-14T09:05:44.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6600000000000001e-06 2024-09-14T09:05:44.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095639+0000", 2024-09-14T09:05:44.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1199000000000006e-05 2024-09-14T09:05:44.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:44.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095726+0000", 2024-09-14T09:05:44.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7169999999999994e-05 2024-09-14T09:05:44.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:05:44.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095937+0000", 2024-09-14T09:05:44.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000210764 2024-09-14T09:05:44.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:44.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097000+0000", 2024-09-14T09:05:44.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001062552 2024-09-14T09:05:44.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:44.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099262+0000", 2024-09-14T09:05:44.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0022622699999999998 2024-09-14T09:05:44.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:44.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099318+0000", 2024-09-14T09:05:44.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6085999999999998e-05 2024-09-14T09:05:44.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099374+0000", 2024-09-14T09:05:44.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5631999999999999e-05 2024-09-14T09:05:44.347 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.347 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.347 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.347 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "pg_update_log_missing(3.1a epoch 39/34 rep_tid 63 entries 39'42 (0'0) error 3:5d7331e3:::rbd_data.115e3abd3b2c.000000000000002c:head by client.4446.0:18835 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 37'39)", 2024-09-14T09:05:44.347 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.096185+0000", 2024-09-14T09:05:44.348 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.19159430699999999, 2024-09-14T09:05:44.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0058811840000000002, 2024-09-14T09:05:44.348 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.348 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.348 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:44.348 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096185+0000", 2024-09-14T09:05:44.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096185+0000", 2024-09-14T09:05:44.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096186+0000", 2024-09-14T09:05:44.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0440000000000001e-06 2024-09-14T09:05:44.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096199+0000", 2024-09-14T09:05:44.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.293e-05 2024-09-14T09:05:44.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096225+0000", 2024-09-14T09:05:44.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6364e-05 2024-09-14T09:05:44.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096247+0000", 2024-09-14T09:05:44.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2101000000000001e-05 2024-09-14T09:05:44.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096330+0000", 2024-09-14T09:05:44.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3394999999999997e-05 2024-09-14T09:05:44.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102066+0000", 2024-09-14T09:05:44.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0057353500000000002 2024-09-14T09:05:44.351 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.351 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.351 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.351 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "pg_update_log_missing(3.1f epoch 39/28 rep_tid 256 entries 39'31 (0'0) error 3:ff6538e6:::rbd_data.115e3abd3b2c.000000000000002f:head by client.4446.0:18840 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 39'29)", 2024-09-14T09:05:44.351 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.100059+0000", 2024-09-14T09:05:44.351 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.187720263, 2024-09-14T09:05:44.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0049658159999999996, 2024-09-14T09:05:44.351 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.351 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.352 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:44.352 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.100059+0000", 2024-09-14T09:05:44.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.100059+0000", 2024-09-14T09:05:44.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.100060+0000", 2024-09-14T09:05:44.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.234e-06 2024-09-14T09:05:44.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.100070+0000", 2024-09-14T09:05:44.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0662000000000001e-05 2024-09-14T09:05:44.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.100098+0000", 2024-09-14T09:05:44.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7634999999999999e-05 2024-09-14T09:05:44.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.100116+0000", 2024-09-14T09:05:44.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8060999999999999e-05 2024-09-14T09:05:44.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.100214+0000", 2024-09-14T09:05:44.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8183999999999994e-05 2024-09-14T09:05:44.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105024+0000", 2024-09-14T09:05:44.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0048100399999999998 2024-09-14T09:05:44.355 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.355 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.355 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.355 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:18841 3.1a e39/34 3:5bcb7d4f:::rbd_data.115e3abd3b2c.0000000000000028:head v 39'43, mlcod=37'40)", 2024-09-14T09:05:44.356 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.100355+0000", 2024-09-14T09:05:44.356 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.187424007, 2024-09-14T09:05:44.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0047141090000000002, 2024-09-14T09:05:44.356 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.356 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.356 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:44.356 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.100355+0000", 2024-09-14T09:05:44.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.100355+0000", 2024-09-14T09:05:44.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.100356+0000", 2024-09-14T09:05:44.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0210000000000001e-06 2024-09-14T09:05:44.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.100365+0000", 2024-09-14T09:05:44.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5829999999999996e-06 2024-09-14T09:05:44.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.100379+0000", 2024-09-14T09:05:44.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3487e-05 2024-09-14T09:05:44.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.100390+0000", 2024-09-14T09:05:44.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.097e-05 2024-09-14T09:05:44.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.100448+0000", 2024-09-14T09:05:44.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7782999999999999e-05 2024-09-14T09:05:44.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:44.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.100469+0000", 2024-09-14T09:05:44.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1662e-05 2024-09-14T09:05:44.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:44.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105027+0000", 2024-09-14T09:05:44.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0045580409999999997 2024-09-14T09:05:44.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105069+0000", 2024-09-14T09:05:44.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1561999999999999e-05 2024-09-14T09:05:44.360 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.360 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.360 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.361 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:18848 3.1a e39/34 3:5d7331e3:::rbd_data.115e3abd3b2c.000000000000002c:head v 39'44, mlcod=39'41)", 2024-09-14T09:05:44.361 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.103165+0000", 2024-09-14T09:05:44.361 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.18461377600000001, 2024-09-14T09:05:44.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0049281450000000001, 2024-09-14T09:05:44.361 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.361 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.361 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:44.361 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103165+0000", 2024-09-14T09:05:44.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103165+0000", 2024-09-14T09:05:44.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103166+0000", 2024-09-14T09:05:44.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0640000000000001e-06 2024-09-14T09:05:44.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103173+0000", 2024-09-14T09:05:44.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9569999999999996e-06 2024-09-14T09:05:44.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103189+0000", 2024-09-14T09:05:44.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5951999999999999e-05 2024-09-14T09:05:44.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103198+0000", 2024-09-14T09:05:44.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2580000000000001e-06 2024-09-14T09:05:44.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103298+0000", 2024-09-14T09:05:44.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9913000000000004e-05 2024-09-14T09:05:44.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:44.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103322+0000", 2024-09-14T09:05:44.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.385e-05 2024-09-14T09:05:44.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:44.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108045+0000", 2024-09-14T09:05:44.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0047234180000000001 2024-09-14T09:05:44.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108093+0000", 2024-09-14T09:05:44.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7732999999999999e-05 2024-09-14T09:05:44.365 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.365 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.365 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.365 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18850 3.9 3:9252f7c9:::rbd_data.115e3abd3b2c.0000000000000032:head [stat,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:44.365 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.105156+0000", 2024-09-14T09:05:44.365 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.18262328899999999, 2024-09-14T09:05:44.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0033757969999999998, 2024-09-14T09:05:44.366 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.366 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.366 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:44.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:44.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:44.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:44.366 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18850 2024-09-14T09:05:44.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.366 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105156+0000", 2024-09-14T09:05:44.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105156+0000", 2024-09-14T09:05:44.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105157+0000", 2024-09-14T09:05:44.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.691e-06 2024-09-14T09:05:44.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105164+0000", 2024-09-14T09:05:44.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.737e-06 2024-09-14T09:05:44.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105177+0000", 2024-09-14T09:05:44.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2554999999999999e-05 2024-09-14T09:05:44.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105187+0000", 2024-09-14T09:05:44.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0329999999999999e-05 2024-09-14T09:05:44.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105264+0000", 2024-09-14T09:05:44.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7033000000000006e-05 2024-09-14T09:05:44.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:44.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105355+0000", 2024-09-14T09:05:44.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0700999999999998e-05 2024-09-14T09:05:44.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:05:44.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105538+0000", 2024-09-14T09:05:44.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000183871 2024-09-14T09:05:44.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:44.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.107835+0000", 2024-09-14T09:05:44.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0022961380000000001 2024-09-14T09:05:44.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:44.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108391+0000", 2024-09-14T09:05:44.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00055638699999999998 2024-09-14T09:05:44.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:44.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108460+0000", 2024-09-14T09:05:44.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8596000000000005e-05 2024-09-14T09:05:44.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108531+0000", 2024-09-14T09:05:44.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1758e-05 2024-09-14T09:05:44.371 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.371 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.371 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.371 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:18851 3.1f e39/28 3:ff6538e6:::rbd_data.115e3abd3b2c.000000000000002f:head v 39'32, mlcod=39'30)", 2024-09-14T09:05:44.371 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.106186+0000", 2024-09-14T09:05:44.372 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.181593119, 2024-09-14T09:05:44.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0022561069999999998, 2024-09-14T09:05:44.372 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.372 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.372 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:44.372 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.106186+0000", 2024-09-14T09:05:44.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.106186+0000", 2024-09-14T09:05:44.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.106187+0000", 2024-09-14T09:05:44.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3399999999999998e-07 2024-09-14T09:05:44.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.106193+0000", 2024-09-14T09:05:44.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7299999999999999e-06 2024-09-14T09:05:44.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.106206+0000", 2024-09-14T09:05:44.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3033e-05 2024-09-14T09:05:44.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.106214+0000", 2024-09-14T09:05:44.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4420000000000001e-06 2024-09-14T09:05:44.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.106296+0000", 2024-09-14T09:05:44.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2445000000000001e-05 2024-09-14T09:05:44.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:44.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.106314+0000", 2024-09-14T09:05:44.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8005000000000001e-05 2024-09-14T09:05:44.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:44.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108405+0000", 2024-09-14T09:05:44.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0020912560000000001 2024-09-14T09:05:44.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108442+0000", 2024-09-14T09:05:44.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6362000000000002e-05 2024-09-14T09:05:44.376 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.376 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.376 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.376 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:18873 3.1f e39/28 3:fa950ac4:::rbd_data.115e3abd3b2c.0000000000000037:head v 39'34, mlcod=39'30)", 2024-09-14T09:05:44.376 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.109262+0000", 2024-09-14T09:05:44.376 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.17851700000000001, 2024-09-14T09:05:44.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002106597, 2024-09-14T09:05:44.376 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.376 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.376 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:44.376 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.109262+0000", 2024-09-14T09:05:44.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.109262+0000", 2024-09-14T09:05:44.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.109263+0000", 2024-09-14T09:05:44.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:05:44.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.109269+0000", 2024-09-14T09:05:44.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5390000000000004e-06 2024-09-14T09:05:44.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.109283+0000", 2024-09-14T09:05:44.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3331e-05 2024-09-14T09:05:44.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.109291+0000", 2024-09-14T09:05:44.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6680000000000007e-06 2024-09-14T09:05:44.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.109362+0000", 2024-09-14T09:05:44.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1182999999999999e-05 2024-09-14T09:05:44.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:44.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.109376+0000", 2024-09-14T09:05:44.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3971000000000001e-05 2024-09-14T09:05:44.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:44.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.111317+0000", 2024-09-14T09:05:44.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001940453 2024-09-14T09:05:44.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.111368+0000", 2024-09-14T09:05:44.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1603e-05 2024-09-14T09:05:44.380 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.380 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.380 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.381 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:18946 3.1f e39/28 3:fbcf3b0f:::rbd_object_map.115e3abd3b2c:head v 39'39, mlcod=39'36)", 2024-09-14T09:05:44.381 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.127790+0000", 2024-09-14T09:05:44.381 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.15998906199999999, 2024-09-14T09:05:44.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002003126, 2024-09-14T09:05:44.381 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:44.381 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:44.381 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:44.381 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:44.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:44.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.127790+0000", 2024-09-14T09:05:44.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:44.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.127790+0000", 2024-09-14T09:05:44.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:44.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:44.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.127790+0000", 2024-09-14T09:05:44.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6400000000000002e-07 2024-09-14T09:05:44.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:44.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.127795+0000", 2024-09-14T09:05:44.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1219999999999997e-06 2024-09-14T09:05:44.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:44.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.127804+0000", 2024-09-14T09:05:44.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4380000000000001e-06 2024-09-14T09:05:44.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:44.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.127809+0000", 2024-09-14T09:05:44.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9130000000000002e-06 2024-09-14T09:05:44.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:44.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.127852+0000", 2024-09-14T09:05:44.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3532000000000003e-05 2024-09-14T09:05:44.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:44.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.127865+0000", 2024-09-14T09:05:44.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2712e-05 2024-09-14T09:05:44.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:44.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.129757+0000", 2024-09-14T09:05:44.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018920090000000001 2024-09-14T09:05:44.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:44.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:44.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:44.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.129793+0000", 2024-09-14T09:05:44.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5735999999999998e-05 2024-09-14T09:05:44.384 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.385 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.385 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.385 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:44.385 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:44.385 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:44.385 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:05:44.386 INFO:teuthology.orchestra.run.smithi176.stdout:563 write 0x8580cd2 thru 0x8582fbd (0x22ec bytes) 2024-09-14T09:05:44.387 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:05:44.388 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:44.384+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:44.449 INFO:teuthology.orchestra.run.smithi176.stdout:565 read 0x2ea0feb thru 0x2eaf08d (0xe0a3 bytes) 2024-09-14T09:05:44.450 INFO:teuthology.orchestra.run.smithi176.stdout:567 punch from 0x604dc2a to 0x6054f48, (0x731e bytes) 2024-09-14T09:05:44.462 INFO:teuthology.orchestra.run.smithi176.stdout:568 trunc from 0x8582fbe to 0x1c620e2 2024-09-14T09:05:44.487 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:44.484+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:44.575 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:44.575 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:44.575 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:44.575 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:44.577 INFO:teuthology.orchestra.run.smithi176.stdout:569 read 0xa0b54c thru 0xa1591c (0xa3d1 bytes) 2024-09-14T09:05:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:570 write 0x1bb537c thru 0x1bb9077 (0x3cfc bytes) 2024-09-14T09:05:44.581 INFO:teuthology.orchestra.run.smithi176.stdout:571 read 0x156477e thru 0x156ef44 (0xa7c7 bytes) 2024-09-14T09:05:44.581 INFO:teuthology.orchestra.run.smithi176.stdout:572 write 0x1b0a4d thru 0x1be541 (0xdaf5 bytes) 2024-09-14T09:05:44.586 INFO:teuthology.orchestra.run.smithi176.stdout:573 read 0xc873d2 thru 0xc94f54 (0xdb83 bytes) 2024-09-14T09:05:44.588 INFO:teuthology.orchestra.run.smithi176.stdout:575 read 0x1abff8c thru 0x1ace2e8 (0xe35d bytes) 2024-09-14T09:05:44.588 INFO:teuthology.orchestra.run.smithi176.stdout:576 read 0xd29c06 thru 0xd2c641 (0x2a3c bytes) 2024-09-14T09:05:44.588 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:44.584+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:44.588 INFO:teuthology.orchestra.run.smithi176.stdout:578 read 0xcd858a thru 0xcd928c (0xd03 bytes) 2024-09-14T09:05:44.591 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:05:44.596 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:05:44.597 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:05:44.607 INFO:teuthology.orchestra.run.smithi176.stdout:580 write 0x7ecee4b thru 0x7eddce0 (0xee96 bytes) 2024-09-14T09:05:44.611 INFO:teuthology.orchestra.run.smithi176.stdout:584 punch from 0x33f1177 to 0x33f383a, (0x26c3 bytes) 2024-09-14T09:05:44.611 INFO:teuthology.orchestra.run.smithi176.stdout:585 read 0x34e29ee thru 0x34e9288 (0x689b bytes) 2024-09-14T09:05:44.615 INFO:teuthology.orchestra.run.smithi176.stdout:586 trunc from 0x7eddce1 to 0xab35e3f 2024-09-14T09:05:44.617 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:05:44.618 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:05:44.622 INFO:teuthology.orchestra.run.smithi176.stdout:588 read 0x67ff261 thru 0x680b175 (0xbf15 bytes) 2024-09-14T09:05:44.627 INFO:teuthology.orchestra.run.smithi176.stdout:589 trunc from 0xab35e3f to 0xdf5c37e 2024-09-14T09:05:44.634 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:44.634 INFO:teuthology.orchestra.run.smithi176.stdout:591 punch from 0x658aa93 to 0x658ce01, (0x236e bytes) 2024-09-14T09:05:44.634 INFO:teuthology.orchestra.run.smithi176.stdout:592 write 0x6473d83 thru 0x647b8c4 (0x7b42 bytes) 2024-09-14T09:05:44.635 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:05:44.636 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:05:44.637 INFO:teuthology.orchestra.run.smithi176.stdout:593 read 0x324404f thru 0x3253a3c (0xf9ee bytes) 2024-09-14T09:05:44.638 INFO:teuthology.orchestra.run.smithi176.stdout:594 write 0x901f1c0 thru 0x9021d6d (0x2bae bytes) 2024-09-14T09:05:44.654 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:05:44.654 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:05:44.659 INFO:teuthology.orchestra.run.smithi176.stdout:595 trunc from 0xdf5c37e to 0x9ec128 2024-09-14T09:05:44.680 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:05:44.680 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:05:44.689 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:44.684+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:44.690 INFO:teuthology.orchestra.run.smithi176.stdout:597 trunc from 0x9ec128 to 0x3c7d31d 2024-09-14T09:05:44.700 INFO:teuthology.orchestra.run.smithi176.stdout:598 writesame 0x2fc0d55 thru 0x2fcbf0f data_size 0x1(0xb1bb bytes) 2024-09-14T09:05:44.707 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:05:44.707 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:05:44.731 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:44.731 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:44.742 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:05:44.742 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:05:44.748 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:05:44.779 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:05:44.822 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:05:44.823 INFO:teuthology.orchestra.run.smithi176.stdout:599 trunc from 0x3c7d31d to 0xdf9c542 2024-09-14T09:05:44.834 INFO:teuthology.orchestra.run.smithi176.stdout:601 write 0x5a263ec thru 0x5a29372 (0x2f87 bytes) 2024-09-14T09:05:44.834 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:44.832+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:44.838 INFO:teuthology.orchestra.run.smithi176.stdout:602 writesame 0x3e84735 thru 0x3e8adb0 data_size 0xe(0x667c bytes) 2024-09-14T09:05:44.851 INFO:teuthology.orchestra.run.smithi176.stdout:603 write 0xa4a88f thru 0xa50c73 (0x63e5 bytes) 2024-09-14T09:05:44.864 INFO:teuthology.orchestra.run.smithi176.stdout:604 trunc from 0xdf9c542 to 0x7e86cbf 2024-09-14T09:05:44.878 INFO:teuthology.orchestra.run.smithi176.stdout:605 read 0x45a5bfb thru 0x45b3726 (0xdb2c bytes) 2024-09-14T09:05:44.878 INFO:teuthology.orchestra.run.smithi176.stdout:606 writesame 0xe4afa7 thru 0xe531f0 data_size 0x33(0x824a bytes) 2024-09-14T09:05:44.886 INFO:teuthology.orchestra.run.smithi176.stdout:607 read 0x5a3e64 thru 0x5a858e (0x472b bytes) 2024-09-14T09:05:44.896 INFO:teuthology.orchestra.run.smithi176.stdout:609 trunc from 0x7e86cbf to 0x2c9bfa8 2024-09-14T09:05:44.910 INFO:teuthology.orchestra.run.smithi176.stdout:610 read 0x20cc753 thru 0x20d6529 (0x9dd7 bytes) 2024-09-14T09:05:44.924 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:44.928+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:44.936 INFO:teuthology.orchestra.run.smithi176.stdout:611 writesame 0xc9dd94f thru 0xc9e9b63 data_size 0x5(0xc215 bytes) 2024-09-14T09:05:44.936 INFO:teuthology.orchestra.run.smithi176.stdout:614 write 0x9035bb8 thru 0x903e1e7 (0x8630 bytes) 2024-09-14T09:05:44.947 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:44.947 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:44.956 INFO:teuthology.orchestra.run.smithi176.stdout:615 writesame 0xe4afd7f thru 0xe4b33a4 data_size 0x1d(0x3626 bytes) 2024-09-14T09:05:44.958 INFO:teuthology.orchestra.run.smithi176.stdout:618 read 0xd5727bd thru 0xd582683 (0xfec7 bytes) 2024-09-14T09:05:44.958 INFO:teuthology.orchestra.run.smithi176.stdout:619 punch from 0x4f754a0 to 0x4f7e691, (0x91f1 bytes) 2024-09-14T09:05:44.958 INFO:teuthology.orchestra.run.smithi176.stdout:620 read 0x81cbb02 thru 0x81cbda4 (0x2a3 bytes) 2024-09-14T09:05:44.963 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:05:44.966 INFO:teuthology.orchestra.run.smithi176.stdout:621 trunc from 0xe4b33a5 to 0x8dbe216 2024-09-14T09:05:44.985 INFO:teuthology.orchestra.run.smithi176.stdout:622 trunc from 0x8dbe216 to 0x60606bb 2024-09-14T09:05:44.996 INFO:teuthology.orchestra.run.smithi176.stdout:623 read 0x102c23f thru 0x102cb00 (0x8c2 bytes) 2024-09-14T09:05:45.015 INFO:teuthology.orchestra.run.smithi176.stdout:624 write 0xbb065f2 thru 0xbb07cc1 (0x16d0 bytes) 2024-09-14T09:05:45.017 INFO:teuthology.orchestra.run.smithi176.stdout:626 punch from 0x6575a7 to 0x65935b, (0x1db4 bytes) 2024-09-14T09:05:45.025 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:45.020+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:45.027 INFO:teuthology.orchestra.run.smithi176.stdout:627 write 0xcde3cfc thru 0xcded59a (0x989f bytes) 2024-09-14T09:05:45.032 INFO:teuthology.orchestra.run.smithi176.stdout:628 write 0xaf64c57 thru 0xaf6cabf (0x7e69 bytes) 2024-09-14T09:05:45.037 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:45.037 INFO:teuthology.orchestra.run.smithi176.stdout:631 read 0x994f86 thru 0x9982ee (0x3369 bytes) 2024-09-14T09:05:45.038 INFO:teuthology.orchestra.run.smithi176.stdout:632 write 0xc6b7feb thru 0xc6b962e (0x1644 bytes) 2024-09-14T09:05:45.041 INFO:teuthology.orchestra.run.smithi176.stdout:633 write 0xaa710b6 thru 0xaa74a09 (0x3954 bytes) 2024-09-14T09:05:45.045 INFO:teuthology.orchestra.run.smithi176.stdout:634 read 0x7c47bc1 thru 0x7c5777a (0xfbba bytes) 2024-09-14T09:05:45.046 INFO:teuthology.orchestra.run.smithi176.stdout:635 read 0x203ef2c thru 0x20403b9 (0x148e bytes) 2024-09-14T09:05:45.046 INFO:teuthology.orchestra.run.smithi176.stdout:636 read 0x76e98c1 thru 0x76efd67 (0x64a7 bytes) 2024-09-14T09:05:45.047 INFO:teuthology.orchestra.run.smithi176.stdout:637 punch from 0xc9b3ce9 to 0xc9baef2, (0x7209 bytes) 2024-09-14T09:05:45.047 INFO:teuthology.orchestra.run.smithi176.stdout:638 punch from 0x4d762e1 to 0x4d78319, (0x2038 bytes) 2024-09-14T09:05:45.057 INFO:teuthology.orchestra.run.smithi176.stdout:639 trunc from 0xcded59b to 0x74d9911 2024-09-14T09:05:45.074 INFO:teuthology.orchestra.run.smithi176.stdout:640 read 0x4b900cd thru 0x4b904e9 (0x41d bytes) 2024-09-14T09:05:45.083 INFO:teuthology.orchestra.run.smithi176.stdout:641 trunc from 0x74d9911 to 0x30fbd75 2024-09-14T09:05:45.108 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:45.108 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:45.123 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:05:45.126 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:45.124+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:45.128 INFO:teuthology.orchestra.run.smithi176.stdout:642 write 0x7585de6 thru 0x758fcc7 (0x9ee2 bytes) 2024-09-14T09:05:45.131 INFO:teuthology.orchestra.run.smithi176.stdout:644 write 0x24b9f28 thru 0x24c53a8 (0xb481 bytes) 2024-09-14T09:05:45.140 INFO:teuthology.orchestra.run.smithi176.stdout:648 read 0x103ac1f thru 0x1043654 (0x8a36 bytes) 2024-09-14T09:05:45.140 INFO:teuthology.orchestra.run.smithi176.stdout:649 read 0x2bcc3d5 thru 0x2bd5b80 (0x97ac bytes) 2024-09-14T09:05:45.142 INFO:teuthology.orchestra.run.smithi176.stdout:653 write 0x382aa17 thru 0x3839555 (0xeb3f bytes) 2024-09-14T09:05:45.176 INFO:teuthology.orchestra.run.smithi176.stdout:654 write 0xe388c80 thru 0xe389358 (0x6d9 bytes) 2024-09-14T09:05:45.178 INFO:teuthology.orchestra.run.smithi176.stdout:655 write 0x47bb108 thru 0x47c740a (0xc303 bytes) 2024-09-14T09:05:45.190 INFO:teuthology.orchestra.run.smithi176.stdout:656 writesame 0x8e5c44e thru 0x8e63372 data_size 0x1(0x6f25 bytes) 2024-09-14T09:05:45.227 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:45.228+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:45.252 INFO:teuthology.orchestra.run.smithi176.stdout:658 clone 7 order 18 su 131072 sc 14 2024-09-14T09:05:45.254 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:45.254 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:45.268 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:05:45.328 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:45.328+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:45.429 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:45.432+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:45.433 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:45.433 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:45.449 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:05:45.530 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:45.528+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:45.615 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:45.615 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:45.629 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:05:45.631 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:45.628+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:45.732 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:45.732+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:45.750 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:45.750 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:45.764 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:05:45.852 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:45.856+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:45.925 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:45.925 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:45.934 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:45.936+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:45.941 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:05:45.952 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T09:05:46.034 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:46.032+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:46.107 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:46.108 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:46.122 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:05:46.135 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:46.132+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:46.236 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:46.236+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:46.240 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:46.240 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:46.255 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:05:46.327 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone6 from 0xe389359 (overlap 0x9ec128) to 0x0 2024-09-14T09:05:46.344 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:46.348+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:46.430 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:46.430 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:46.437 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:46.440+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:46.445 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:05:46.538 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:46.536+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:46.614 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-14T09:05:46.619 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:46.619 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:46.633 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:05:46.639 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:46.636+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:46.740 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:46.740+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:46.766 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:46.766 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:46.781 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:05:46.798 INFO:teuthology.orchestra.run.smithi176.stdout:660 punch from 0x7b7cebd to 0x7b8845a, (0xb59d bytes) 2024-09-14T09:05:46.807 INFO:teuthology.orchestra.run.smithi176.stdout:661 punch from 0x971c42 to 0x97d7a6, (0xbb64 bytes) 2024-09-14T09:05:46.807 INFO:teuthology.orchestra.run.smithi176.stdout:662 punch from 0x7d8851d to 0x7d8ff01, (0x79e4 bytes) 2024-09-14T09:05:46.807 INFO:teuthology.orchestra.run.smithi176.stdout:664 write 0x43a489a thru 0x43b459d (0xfd04 bytes) 2024-09-14T09:05:46.821 INFO:teuthology.orchestra.run.smithi176.stdout:665 write 0x78955a5 thru 0x78979c6 (0x2422 bytes) 2024-09-14T09:05:46.824 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:05:46.827 INFO:teuthology.orchestra.run.smithi176.stdout:666 punch from 0x4bb385e to 0x4bbe2d6, (0xaa78 bytes) 2024-09-14T09:05:46.828 INFO:teuthology.orchestra.run.smithi176.stdout:667 write 0x7b86ad0 thru 0x7b9597d (0xeeae bytes) 2024-09-14T09:05:46.836 INFO:teuthology.orchestra.run.smithi176.stdout:670 read 0x46ac498 thru 0x46b0d2c (0x4895 bytes) 2024-09-14T09:05:46.837 INFO:teuthology.orchestra.run.smithi176.stdout:672 read 0x5b4f673 thru 0x5b5bd87 (0xc715 bytes) 2024-09-14T09:05:46.838 INFO:teuthology.orchestra.run.smithi176.stdout:674 punch from 0x986298a to 0x9870c97, (0xe30d bytes) 2024-09-14T09:05:46.838 INFO:teuthology.orchestra.run.smithi176.stdout:675 read 0x62036a thru 0x627d06 (0x799d bytes) 2024-09-14T09:05:46.839 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:46.840 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:46.836+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:46.849 INFO:teuthology.orchestra.run.smithi176.stdout:677 trunc from 0xe389359 to 0x9c7cd4e 2024-09-14T09:05:46.876 INFO:teuthology.orchestra.run.smithi176.stdout:678 writesame 0x8afc2fa thru 0x8b0905d data_size 0x37(0xcd64 bytes) 2024-09-14T09:05:46.885 INFO:teuthology.orchestra.run.smithi176.stdout:679 read 0x55b48c0 thru 0x55c266d (0xddae bytes) 2024-09-14T09:05:46.886 INFO:teuthology.orchestra.run.smithi176.stdout:680 punch from 0x9bf558f to 0x9c00f01, (0xb972 bytes) 2024-09-14T09:05:46.886 INFO:teuthology.orchestra.run.smithi176.stdout:681 write 0x3813f98 thru 0x3814139 (0x1a2 bytes) 2024-09-14T09:05:46.891 INFO:teuthology.orchestra.run.smithi176.stdout:682 write 0x516e38d thru 0x517b887 (0xd4fb bytes) 2024-09-14T09:05:46.898 INFO:teuthology.orchestra.run.smithi176.stdout:683 read 0x7251cb4 thru 0x725f390 (0xd6dd bytes) 2024-09-14T09:05:46.899 INFO:teuthology.orchestra.run.smithi176.stdout:684 read 0x1e7385d thru 0x1e7f239 (0xb9dd bytes) 2024-09-14T09:05:46.900 INFO:teuthology.orchestra.run.smithi176.stdout:685 read 0x7813f12 thru 0x781d233 (0x9322 bytes) 2024-09-14T09:05:46.900 INFO:teuthology.orchestra.run.smithi176.stdout:686 read 0x63d29c3 thru 0x63e015c (0xd79a bytes) 2024-09-14T09:05:46.911 INFO:teuthology.orchestra.run.smithi176.stdout:687 trunc from 0x9c7cd4e to 0x31fcf38 2024-09-14T09:05:46.935 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is unset 2024-09-14T09:05:46.940 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:46.944+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:46.954 INFO:teuthology.orchestra.run.smithi176.stdout:688 read 0x29a0345 thru 0x29aa346 (0xa002 bytes) 2024-09-14T09:05:46.967 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:46.967 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:46.971 INFO:teuthology.orchestra.run.smithi176.stdout:691 writesame 0x8fd24a7 thru 0x8fdadd7 data_size 0x17(0x8931 bytes) 2024-09-14T09:05:46.972 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:46.972 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:46.972 INFO:teuthology.orchestra.run.smithi176.stdout:694 read 0x712065e thru 0x712eb34 (0xe4d7 bytes) 2024-09-14T09:05:46.972 INFO:teuthology.orchestra.run.smithi176.stdout:696 read 0xbb0a20 thru 0xbb5526 (0x4b07 bytes) 2024-09-14T09:05:46.972 INFO:teuthology.orchestra.run.smithi176.stdout:697 read 0x42bfde0 thru 0x42cb8e4 (0xbb05 bytes) 2024-09-14T09:05:46.982 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:05:46.985 INFO:teuthology.orchestra.run.smithi176.stdout:698 trunc from 0x8fdadd8 to 0x6aa7c4 2024-09-14T09:05:46.999 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:05:46.999 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:05:47.009 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:47.009 INFO:teuthology.orchestra.run.smithi176.stdout:701 trunc from 0x6aa7c4 to 0x417b36 2024-09-14T09:05:47.019 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:05:47.019 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:05:47.038 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:05:47.038 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:05:47.041 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:47.036+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:47.055 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:05:47.056 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:05:47.063 INFO:teuthology.orchestra.run.smithi176.stdout:702 write 0x94f3758 thru 0x94f7986 (0x422f bytes) 2024-09-14T09:05:47.070 INFO:teuthology.orchestra.run.smithi176.stdout:703 read 0x6358c8a thru 0x635bb1e (0x2e95 bytes) 2024-09-14T09:05:47.070 INFO:teuthology.orchestra.run.smithi176.stdout:704 read 0x2d524ac thru 0x2d5cc11 (0xa766 bytes) 2024-09-14T09:05:47.070 INFO:teuthology.orchestra.run.smithi176.stdout:707 read 0x8f0a2d7 thru 0x8f11ff4 (0x7d1e bytes) 2024-09-14T09:05:47.074 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:05:47.074 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:05:47.086 INFO:teuthology.orchestra.run.smithi176.stdout:708 write 0xd5f6a18 thru 0xd606185 (0xf76e bytes) 2024-09-14T09:05:47.094 INFO:teuthology.orchestra.run.smithi176.stdout:711 read 0xc008fe6 thru 0xc01311e (0xa139 bytes) 2024-09-14T09:05:47.094 INFO:teuthology.orchestra.run.smithi176.stdout:712 writesame 0x9fca5b6 thru 0x9fcfb51 data_size 0x2(0x559c bytes) 2024-09-14T09:05:47.096 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:05:47.097 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:05:47.111 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:47.111 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:47.111 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:47.111 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:47.124 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:47.124 INFO:teuthology.orchestra.run.smithi176.stdout:714 write 0x8b91164 thru 0x8b9d67c (0xc519 bytes) 2024-09-14T09:05:47.127 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:05:47.130 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:05:47.130 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:05:47.147 INFO:teuthology.orchestra.run.smithi176.stdout:715 writesame 0xeb76b65 thru 0xeb7b30a data_size 0x3(0x47a6 bytes) 2024-09-14T09:05:47.147 INFO:teuthology.orchestra.run.smithi176.stdout:716 punch from 0x886a47f to 0x8874a1f, (0xa5a0 bytes) 2024-09-14T09:05:47.147 INFO:teuthology.orchestra.run.smithi176.stdout:717 read 0x232f518 thru 0x233f4a0 (0xff89 bytes) 2024-09-14T09:05:47.147 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:47.147 INFO:teuthology.orchestra.run.smithi176.stdout:719 read 0x1568402 thru 0x156cac1 (0x46c0 bytes) 2024-09-14T09:05:47.147 INFO:teuthology.orchestra.run.smithi176.stdout:720 write 0x6276ca thru 0x62833b (0xc72 bytes) 2024-09-14T09:05:47.147 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:47.144+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:47.153 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:05:47.153 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:05:47.166 INFO:teuthology.orchestra.run.smithi176.stdout:721 trunc from 0xeb7b30b to 0x5aaedc1 2024-09-14T09:05:47.196 INFO:teuthology.orchestra.run.smithi176.stdout:722 trunc from 0x5aaedc1 to 0x53c78f 2024-09-14T09:05:47.235 INFO:teuthology.orchestra.run.smithi176.stdout:723 write 0x73d3ddb thru 0x73de62b (0xa851 bytes) 2024-09-14T09:05:47.242 INFO:teuthology.orchestra.run.smithi176.stdout:724 writesame 0x5cc449d thru 0x5cd09ce data_size 0x2b(0xc532 bytes) 2024-09-14T09:05:47.243 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:47.240+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:47.257 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:47.257 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:47.257 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:47.257 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:47.257 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:47.262 INFO:teuthology.orchestra.run.smithi176.stdout:725 trunc from 0x73de62c to 0x1bee481 2024-09-14T09:05:47.275 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:05:47.300 INFO:teuthology.orchestra.run.smithi176.stdout:726 trunc from 0x1bee481 to 0xba5e570 2024-09-14T09:05:47.312 INFO:teuthology.orchestra.run.smithi176.stdout:729 writesame 0x68b32fb thru 0x68b3f11 data_size 0x1(0xc17 bytes) 2024-09-14T09:05:47.319 INFO:teuthology.orchestra.run.smithi176.stdout:730 read 0x1de8d41 thru 0x1dec5a1 (0x3861 bytes) 2024-09-14T09:05:47.320 INFO:teuthology.orchestra.run.smithi176.stdout:732 punch from 0x5ac8e80 to 0x5ace961, (0x5ae1 bytes) 2024-09-14T09:05:47.320 INFO:teuthology.orchestra.run.smithi176.stdout:734 read 0xdf0cf9 thru 0xdfe7d7 (0xdadf bytes) 2024-09-14T09:05:47.334 INFO:teuthology.orchestra.run.smithi176.stdout:736 trunc from 0xba5e570 to 0x15fe8df 2024-09-14T09:05:47.345 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:47.348+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:47.354 INFO:teuthology.orchestra.run.smithi176.stdout:737 read 0xaf36b thru 0xaf969 (0x5ff bytes) 2024-09-14T09:05:47.374 INFO:teuthology.orchestra.run.smithi176.stdout:738 write 0x6804b55 thru 0x68144db (0xf987 bytes) 2024-09-14T09:05:47.381 INFO:teuthology.orchestra.run.smithi176.stdout:739 write 0x495c711 thru 0x4967fe6 (0xb8d6 bytes) 2024-09-14T09:05:47.415 INFO:teuthology.orchestra.run.smithi176.stdout:741 writesame 0xc6e0177 thru 0xc6e99ad data_size 0x1f(0x9837 bytes) 2024-09-14T09:05:47.415 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:47.415 INFO:teuthology.orchestra.run.smithi176.stdout:744 write 0x5c193a1 thru 0x5c1f35c (0x5fbc bytes) 2024-09-14T09:05:47.423 INFO:teuthology.orchestra.run.smithi176.stdout:745 writesame 0x5753568 thru 0x5757ca4 data_size 0x1(0x473d bytes) 2024-09-14T09:05:47.427 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:47.427 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:47.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:47.427 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:47.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.427 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18753 3.12 3.81006c52 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.427 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.076789+0000", 2024-09-14T09:05:47.427 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.350064368, 2024-09-14T09:05:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088781859999999997, 2024-09-14T09:05:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18753 2024-09-14T09:05:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076789+0000", 2024-09-14T09:05:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076789+0000", 2024-09-14T09:05:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076790+0000", 2024-09-14T09:05:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2699999999999998e-07 2024-09-14T09:05:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076796+0000", 2024-09-14T09:05:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3210000000000004e-06 2024-09-14T09:05:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076806+0000", 2024-09-14T09:05:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9199999999999999e-06 2024-09-14T09:05:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076816+0000", 2024-09-14T09:05:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9000000000000001e-06 2024-09-14T09:05:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.085658+0000", 2024-09-14T09:05:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088426210000000002 2024-09-14T09:05:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.085668+0000", 2024-09-14T09:05:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4970000000000007e-06 2024-09-14T09:05:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18754 3.12 3.81006c52 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.076833+0000", 2024-09-14T09:05:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.3500205090000001, 2024-09-14T09:05:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088813820000000002, 2024-09-14T09:05:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18754 2024-09-14T09:05:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076833+0000", 2024-09-14T09:05:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076833+0000", 2024-09-14T09:05:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076834+0000", 2024-09-14T09:05:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1999999999999999e-07 2024-09-14T09:05:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076838+0000", 2024-09-14T09:05:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6799999999999999e-06 2024-09-14T09:05:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076856+0000", 2024-09-14T09:05:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8691000000000001e-05 2024-09-14T09:05:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076865+0000", 2024-09-14T09:05:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9849999999999995e-06 2024-09-14T09:05:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.085704+0000", 2024-09-14T09:05:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088383660000000003 2024-09-14T09:05:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.085715+0000", 2024-09-14T09:05:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1039999999999999e-05 2024-09-14T09:05:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18755 3.12 3.81006c52 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.076877+0000", 2024-09-14T09:05:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.3499765290000001, 2024-09-14T09:05:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088906469999999998, 2024-09-14T09:05:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18755 2024-09-14T09:05:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076877+0000", 2024-09-14T09:05:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076877+0000", 2024-09-14T09:05:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076878+0000", 2024-09-14T09:05:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8100000000000002e-07 2024-09-14T09:05:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076881+0000", 2024-09-14T09:05:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2899999999999998e-06 2024-09-14T09:05:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076890+0000", 2024-09-14T09:05:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7960000000000001e-06 2024-09-14T09:05:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076898+0000", 2024-09-14T09:05:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8280000000000003e-06 2024-09-14T09:05:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.085757+0000", 2024-09-14T09:05:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088595879999999998 2024-09-14T09:05:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.085768+0000", 2024-09-14T09:05:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0464e-05 2024-09-14T09:05:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18756 3.12 3.81006c52 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.076908+0000", 2024-09-14T09:05:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.34994594, 2024-09-14T09:05:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0089070020000000007, 2024-09-14T09:05:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18756 2024-09-14T09:05:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076908+0000", 2024-09-14T09:05:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076908+0000", 2024-09-14T09:05:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076909+0000", 2024-09-14T09:05:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4799999999999998e-07 2024-09-14T09:05:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076911+0000", 2024-09-14T09:05:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.875e-06 2024-09-14T09:05:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076928+0000", 2024-09-14T09:05:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6557000000000001e-05 2024-09-14T09:05:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076935+0000", 2024-09-14T09:05:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0260000000000002e-06 2024-09-14T09:05:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.085805+0000", 2024-09-14T09:05:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088700570000000006 2024-09-14T09:05:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.085815+0000", 2024-09-14T09:05:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8390000000000002e-06 2024-09-14T09:05:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18757 3.12 3.81006c52 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.076946+0000", 2024-09-14T09:05:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.3499077289999999, 2024-09-14T09:05:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0089136730000000004, 2024-09-14T09:05:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18757 2024-09-14T09:05:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076946+0000", 2024-09-14T09:05:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076946+0000", 2024-09-14T09:05:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076947+0000", 2024-09-14T09:05:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.68e-07 2024-09-14T09:05:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076950+0000", 2024-09-14T09:05:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9179999999999998e-06 2024-09-14T09:05:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076959+0000", 2024-09-14T09:05:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9530000000000005e-06 2024-09-14T09:05:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076965+0000", 2024-09-14T09:05:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7449999999999998e-06 2024-09-14T09:05:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.085850+0000", 2024-09-14T09:05:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088847969999999998 2024-09-14T09:05:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.085860+0000", 2024-09-14T09:05:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6919999999999997e-06 2024-09-14T09:05:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18758 3.12 3.81006c52 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.076975+0000", 2024-09-14T09:05:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.349878592, 2024-09-14T09:05:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0089275369999999993, 2024-09-14T09:05:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18758 2024-09-14T09:05:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076975+0000", 2024-09-14T09:05:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076975+0000", 2024-09-14T09:05:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076976+0000", 2024-09-14T09:05:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7899999999999998e-07 2024-09-14T09:05:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076979+0000", 2024-09-14T09:05:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7300000000000001e-06 2024-09-14T09:05:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076987+0000", 2024-09-14T09:05:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4980000000000006e-06 2024-09-14T09:05:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076993+0000", 2024-09-14T09:05:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4160000000000001e-06 2024-09-14T09:05:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.085890+0000", 2024-09-14T09:05:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088969119999999999 2024-09-14T09:05:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.085903+0000", 2024-09-14T09:05:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2401999999999999e-05 2024-09-14T09:05:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18779 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 14024704~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087321+0000", 2024-09-14T09:05:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.3395331559999999, 2024-09-14T09:05:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0090439219999999994, 2024-09-14T09:05:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18779 2024-09-14T09:05:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087321+0000", 2024-09-14T09:05:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087321+0000", 2024-09-14T09:05:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087321+0000", 2024-09-14T09:05:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5299999999999999e-07 2024-09-14T09:05:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087329+0000", 2024-09-14T09:05:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2640000000000008e-06 2024-09-14T09:05:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087336+0000", 2024-09-14T09:05:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4389999999999997e-06 2024-09-14T09:05:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087340+0000", 2024-09-14T09:05:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6290000000000001e-06 2024-09-14T09:05:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096065+0000", 2024-09-14T09:05:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0087245410000000006 2024-09-14T09:05:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096245+0000", 2024-09-14T09:05:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017988800000000001 2024-09-14T09:05:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096365+0000", 2024-09-14T09:05:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119708 2024-09-14T09:05:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18780 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 14286848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087348+0000", 2024-09-14T09:05:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.3395053579999998, 2024-09-14T09:05:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0094481670000000004, 2024-09-14T09:05:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18780 2024-09-14T09:05:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087348+0000", 2024-09-14T09:05:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087348+0000", 2024-09-14T09:05:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087349+0000", 2024-09-14T09:05:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0900000000000002e-07 2024-09-14T09:05:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087351+0000", 2024-09-14T09:05:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1009999999999999e-06 2024-09-14T09:05:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087357+0000", 2024-09-14T09:05:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2659999999999998e-06 2024-09-14T09:05:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087362+0000", 2024-09-14T09:05:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6650000000000002e-06 2024-09-14T09:05:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096433+0000", 2024-09-14T09:05:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0090705649999999992 2024-09-14T09:05:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096645+0000", 2024-09-14T09:05:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000212733 2024-09-14T09:05:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096797+0000", 2024-09-14T09:05:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000151328 2024-09-14T09:05:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18781 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 14548992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087369+0000", 2024-09-14T09:05:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.339484696, 2024-09-14T09:05:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0096976600000000003, 2024-09-14T09:05:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18781 2024-09-14T09:05:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087369+0000", 2024-09-14T09:05:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087369+0000", 2024-09-14T09:05:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087370+0000", 2024-09-14T09:05:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9000000000000002e-07 2024-09-14T09:05:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087372+0000", 2024-09-14T09:05:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4399999999999999e-06 2024-09-14T09:05:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087379+0000", 2024-09-14T09:05:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6749999999999996e-06 2024-09-14T09:05:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087383+0000", 2024-09-14T09:05:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5630000000000002e-06 2024-09-14T09:05:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096852+0000", 2024-09-14T09:05:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0094688429999999994 2024-09-14T09:05:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096973+0000", 2024-09-14T09:05:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000121066 2024-09-14T09:05:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097067+0000", 2024-09-14T09:05:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3683000000000004e-05 2024-09-14T09:05:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18782 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 14811136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087391+0000", 2024-09-14T09:05:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.3394632930000001, 2024-09-14T09:05:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0098848249999999999, 2024-09-14T09:05:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18782 2024-09-14T09:05:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087391+0000", 2024-09-14T09:05:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087391+0000", 2024-09-14T09:05:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087391+0000", 2024-09-14T09:05:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9400000000000001e-07 2024-09-14T09:05:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087393+0000", 2024-09-14T09:05:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1619999999999998e-06 2024-09-14T09:05:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087401+0000", 2024-09-14T09:05:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3950000000000001e-06 2024-09-14T09:05:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087414+0000", 2024-09-14T09:05:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2794e-05 2024-09-14T09:05:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097115+0000", 2024-09-14T09:05:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0097009920000000003 2024-09-14T09:05:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097191+0000", 2024-09-14T09:05:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6205000000000006e-05 2024-09-14T09:05:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097275+0000", 2024-09-14T09:05:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3882999999999996e-05 2024-09-14T09:05:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18783 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 15073280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087423+0000", 2024-09-14T09:05:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.3394310250000001, 2024-09-14T09:05:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010039031, 2024-09-14T09:05:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18783 2024-09-14T09:05:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087423+0000", 2024-09-14T09:05:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087423+0000", 2024-09-14T09:05:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087423+0000", 2024-09-14T09:05:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1799999999999995e-07 2024-09-14T09:05:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087426+0000", 2024-09-14T09:05:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.79e-06 2024-09-14T09:05:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087433+0000", 2024-09-14T09:05:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9310000000000002e-06 2024-09-14T09:05:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087438+0000", 2024-09-14T09:05:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1379999999999998e-06 2024-09-14T09:05:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097321+0000", 2024-09-14T09:05:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.009882877 2024-09-14T09:05:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097388+0000", 2024-09-14T09:05:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6958000000000005e-05 2024-09-14T09:05:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097462+0000", 2024-09-14T09:05:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3819000000000006e-05 2024-09-14T09:05:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18784 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 15335424~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087445+0000", 2024-09-14T09:05:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.3394083640000001, 2024-09-14T09:05:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010293808999999999, 2024-09-14T09:05:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18784 2024-09-14T09:05:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087445+0000", 2024-09-14T09:05:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087445+0000", 2024-09-14T09:05:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087446+0000", 2024-09-14T09:05:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7e-07 2024-09-14T09:05:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087448+0000", 2024-09-14T09:05:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.013e-06 2024-09-14T09:05:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087454+0000", 2024-09-14T09:05:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.088e-06 2024-09-14T09:05:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087460+0000", 2024-09-14T09:05:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.117e-06 2024-09-14T09:05:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097513+0000", 2024-09-14T09:05:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010052901 2024-09-14T09:05:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097613+0000", 2024-09-14T09:05:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9596e-05 2024-09-14T09:05:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097739+0000", 2024-09-14T09:05:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012662400000000001 2024-09-14T09:05:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18785 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 15597568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087467+0000", 2024-09-14T09:05:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.3393870099999998, 2024-09-14T09:05:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010499494, 2024-09-14T09:05:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18785 2024-09-14T09:05:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087467+0000", 2024-09-14T09:05:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087467+0000", 2024-09-14T09:05:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087467+0000", 2024-09-14T09:05:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7599999999999998e-07 2024-09-14T09:05:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087475+0000", 2024-09-14T09:05:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9929999999999995e-06 2024-09-14T09:05:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087482+0000", 2024-09-14T09:05:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8369999999999999e-06 2024-09-14T09:05:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087487+0000", 2024-09-14T09:05:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0919999999999997e-06 2024-09-14T09:05:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097794+0000", 2024-09-14T09:05:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010306553 2024-09-14T09:05:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097894+0000", 2024-09-14T09:05:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100077 2024-09-14T09:05:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097966+0000", 2024-09-14T09:05:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2565999999999996e-05 2024-09-14T09:05:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18786 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 15859712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087495+0000", 2024-09-14T09:05:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.3393592910000001, 2024-09-14T09:05:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010652643, 2024-09-14T09:05:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18786 2024-09-14T09:05:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087495+0000", 2024-09-14T09:05:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087495+0000", 2024-09-14T09:05:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087495+0000", 2024-09-14T09:05:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9400000000000005e-07 2024-09-14T09:05:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087497+0000", 2024-09-14T09:05:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9069999999999999e-06 2024-09-14T09:05:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087503+0000", 2024-09-14T09:05:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0759999999999999e-06 2024-09-14T09:05:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087508+0000", 2024-09-14T09:05:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6929999999999998e-06 2024-09-14T09:05:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098005+0000", 2024-09-14T09:05:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010497390000000001 2024-09-14T09:05:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098071+0000", 2024-09-14T09:05:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6021999999999999e-05 2024-09-14T09:05:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098147+0000", 2024-09-14T09:05:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5961e-05 2024-09-14T09:05:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18787 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 16121856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087514+0000", 2024-09-14T09:05:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.3393396119999998, 2024-09-14T09:05:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010782221, 2024-09-14T09:05:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18787 2024-09-14T09:05:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087514+0000", 2024-09-14T09:05:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087514+0000", 2024-09-14T09:05:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087515+0000", 2024-09-14T09:05:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9499999999999998e-07 2024-09-14T09:05:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087516+0000", 2024-09-14T09:05:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7880000000000001e-06 2024-09-14T09:05:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087523+0000", 2024-09-14T09:05:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1340000000000001e-06 2024-09-14T09:05:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087532+0000", 2024-09-14T09:05:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8130000000000007e-06 2024-09-14T09:05:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098178+0000", 2024-09-14T09:05:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010646136 2024-09-14T09:05:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098235+0000", 2024-09-14T09:05:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6415000000000001e-05 2024-09-14T09:05:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098296+0000", 2024-09-14T09:05:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1539999999999997e-05 2024-09-14T09:05:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18788 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 16384000~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087539+0000", 2024-09-14T09:05:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.339314941, 2024-09-14T09:05:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010916135, 2024-09-14T09:05:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18788 2024-09-14T09:05:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087539+0000", 2024-09-14T09:05:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087539+0000", 2024-09-14T09:05:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087539+0000", 2024-09-14T09:05:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9800000000000004e-07 2024-09-14T09:05:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087541+0000", 2024-09-14T09:05:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9879999999999999e-06 2024-09-14T09:05:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087547+0000", 2024-09-14T09:05:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7579999999999998e-06 2024-09-14T09:05:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087551+0000", 2024-09-14T09:05:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0709999999999996e-06 2024-09-14T09:05:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098329+0000", 2024-09-14T09:05:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010777406999999999 2024-09-14T09:05:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098385+0000", 2024-09-14T09:05:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6514000000000001e-05 2024-09-14T09:05:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098455+0000", 2024-09-14T09:05:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9899000000000003e-05 2024-09-14T09:05:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18789 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 16646144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087557+0000", 2024-09-14T09:05:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.339296408, 2024-09-14T09:05:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011043562, 2024-09-14T09:05:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18789 2024-09-14T09:05:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087557+0000", 2024-09-14T09:05:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087557+0000", 2024-09-14T09:05:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087558+0000", 2024-09-14T09:05:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.03e-07 2024-09-14T09:05:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087560+0000", 2024-09-14T09:05:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1459999999999999e-06 2024-09-14T09:05:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087566+0000", 2024-09-14T09:05:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2870000000000001e-06 2024-09-14T09:05:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087571+0000", 2024-09-14T09:05:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5390000000000001e-06 2024-09-14T09:05:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098485+0000", 2024-09-14T09:05:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010914561999999999 2024-09-14T09:05:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098541+0000", 2024-09-14T09:05:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5384999999999999e-05 2024-09-14T09:05:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098601+0000", 2024-09-14T09:05:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0239999999999999e-05 2024-09-14T09:05:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19431 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.472845+0000", 2024-09-14T09:05:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 2.9540087050000001, 2024-09-14T09:05:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0089211680000000002, 2024-09-14T09:05:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19431 2024-09-14T09:05:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472845+0000", 2024-09-14T09:05:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472845+0000", 2024-09-14T09:05:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472846+0000", 2024-09-14T09:05:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0900000000000001e-07 2024-09-14T09:05:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472848+0000", 2024-09-14T09:05:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.063e-06 2024-09-14T09:05:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472857+0000", 2024-09-14T09:05:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9369999999999993e-06 2024-09-14T09:05:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472863+0000", 2024-09-14T09:05:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9780000000000002e-06 2024-09-14T09:05:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.481674+0000", 2024-09-14T09:05:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088108709999999996 2024-09-14T09:05:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.481714+0000", 2024-09-14T09:05:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0757999999999998e-05 2024-09-14T09:05:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.481766+0000", 2024-09-14T09:05:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1952e-05 2024-09-14T09:05:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19432 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.472871+0000", 2024-09-14T09:05:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 2.9539830550000001, 2024-09-14T09:05:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0090042509999999996, 2024-09-14T09:05:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19432 2024-09-14T09:05:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472871+0000", 2024-09-14T09:05:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472871+0000", 2024-09-14T09:05:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472871+0000", 2024-09-14T09:05:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9299999999999998e-07 2024-09-14T09:05:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472874+0000", 2024-09-14T09:05:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.278e-06 2024-09-14T09:05:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472881+0000", 2024-09-14T09:05:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7570000000000004e-06 2024-09-14T09:05:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472887+0000", 2024-09-14T09:05:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0059999999999998e-06 2024-09-14T09:05:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.481784+0000", 2024-09-14T09:05:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088970879999999992 2024-09-14T09:05:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.481825+0000", 2024-09-14T09:05:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1083e-05 2024-09-14T09:05:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.481875+0000", 2024-09-14T09:05:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9546000000000003e-05 2024-09-14T09:05:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19433 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.472895+0000", 2024-09-14T09:05:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 2.953959212, 2024-09-14T09:05:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.009104816, 2024-09-14T09:05:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19433 2024-09-14T09:05:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472895+0000", 2024-09-14T09:05:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472895+0000", 2024-09-14T09:05:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472895+0000", 2024-09-14T09:05:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2200000000000004e-07 2024-09-14T09:05:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472897+0000", 2024-09-14T09:05:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.097e-06 2024-09-14T09:05:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472904+0000", 2024-09-14T09:05:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4150000000000004e-06 2024-09-14T09:05:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472909+0000", 2024-09-14T09:05:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0860000000000001e-06 2024-09-14T09:05:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.481900+0000", 2024-09-14T09:05:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.008991064 2024-09-14T09:05:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.481943+0000", 2024-09-14T09:05:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3470999999999998e-05 2024-09-14T09:05:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.481999+0000", 2024-09-14T09:05:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6161e-05 2024-09-14T09:05:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:47.511 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:47.513 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:05:47.514 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:47.514 INFO:teuthology.orchestra.run.smithi176.stdout:747 read 0xa08b567 thru 0xa08cb30 (0x15ca bytes) 2024-09-14T09:05:47.514 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:47.514 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:47.514 INFO:teuthology.orchestra.run.smithi176.stdout:750 write 0x77a29c1 thru 0x77ab770 (0x8db0 bytes) 2024-09-14T09:05:47.514 INFO:teuthology.orchestra.run.smithi176.stdout:751 read 0x81c5f7c thru 0x81c995a (0x39df bytes) 2024-09-14T09:05:47.514 INFO:teuthology.orchestra.run.smithi176.stdout:752 writesame 0x5e2f7ab thru 0x5e390c0 data_size 0x5(0x9916 bytes) 2024-09-14T09:05:47.514 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:47.516+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:47.529 INFO:teuthology.orchestra.run.smithi176.stdout:753 read 0x7db0afa thru 0x7db6d48 (0x624f bytes) 2024-09-14T09:05:47.529 INFO:teuthology.orchestra.run.smithi176.stdout:754 write 0x1a4fc5 thru 0x1a87e5 (0x3821 bytes) 2024-09-14T09:05:47.550 INFO:teuthology.orchestra.run.smithi176.stdout:755 write 0x586d8bc thru 0x587561c (0x7d61 bytes) 2024-09-14T09:05:47.561 INFO:teuthology.orchestra.run.smithi176.stdout:756 write 0x30c3f17 thru 0x30c8d60 (0x4e4a bytes) 2024-09-14T09:05:47.569 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:47.569 INFO:teuthology.orchestra.run.smithi176.stdout:758 read 0xad82a3e thru 0xad91cdc (0xf29f bytes) 2024-09-14T09:05:47.569 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:47.569 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:47.585 INFO:teuthology.orchestra.run.smithi176.stdout:762 trunc from 0xc6e99ae to 0x4350795 2024-09-14T09:05:47.611 INFO:teuthology.orchestra.run.smithi176.stdout:763 write 0x1d8d3d4 thru 0x1d9cdc8 (0xf9f5 bytes) 2024-09-14T09:05:47.613 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:47.608+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:47.625 INFO:teuthology.orchestra.run.smithi176.stdout:764 punch from 0x37aa2c7 to 0x37b5766, (0xb49f bytes) 2024-09-14T09:05:47.630 INFO:teuthology.orchestra.run.smithi176.stdout:766 trunc from 0x4350795 to 0xf61a59 2024-09-14T09:05:47.656 INFO:teuthology.orchestra.run.smithi176.stdout:767 trunc from 0xf61a59 to 0x1d7c5f 2024-09-14T09:05:47.678 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:47.678 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:47.678 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:47.678 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:47.695 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:05:47.742 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:47.740+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:47.755 INFO:teuthology.orchestra.run.smithi176.stdout:768 writesame 0x50014d1 thru 0x500513d data_size 0x1(0x3c6d bytes) 2024-09-14T09:05:47.755 INFO:teuthology.orchestra.run.smithi176.stdout:769 write 0x1708e4 thru 0x17587c (0x4f99 bytes) 2024-09-14T09:05:47.779 INFO:teuthology.orchestra.run.smithi176.stdout:771 write 0x9e784ff thru 0x9e7889d (0x39f bytes) 2024-09-14T09:05:47.798 INFO:teuthology.orchestra.run.smithi176.stdout:772 writesame 0xacf6b44 thru 0xacfe179 data_size 0x1(0x7636 bytes) 2024-09-14T09:05:47.798 INFO:teuthology.orchestra.run.smithi176.stdout:773 read 0x49a2da6 thru 0x49ad80f (0xaa6a bytes) 2024-09-14T09:05:47.798 INFO:teuthology.orchestra.run.smithi176.stdout:774 clone 8 order 18 su 65536 sc 15 2024-09-14T09:05:47.814 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:47.812+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:47.867 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:47.867 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:47.867 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:47.867 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:47.867 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:47.881 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:05:47.922 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:47.920+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:48.016 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:48.012+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:48.050 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:48.050 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:657 3.6 e29/23 3:65b8693c:::rbd_data.115e1e95bf99.0000000000000002:head v 29'2, mlcod=0'0)", 2024-09-14T09:05:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.872567+0000", 2024-09-14T09:05:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.178155192, 2024-09-14T09:05:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0056247060000000002, 2024-09-14T09:05:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872567+0000", 2024-09-14T09:05:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872567+0000", 2024-09-14T09:05:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872568+0000", 2024-09-14T09:05:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T09:05:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872772+0000", 2024-09-14T09:05:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020363799999999999 2024-09-14T09:05:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872799+0000", 2024-09-14T09:05:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6917e-05 2024-09-14T09:05:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872815+0000", 2024-09-14T09:05:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5812999999999999e-05 2024-09-14T09:05:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872928+0000", 2024-09-14T09:05:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011381499999999999 2024-09-14T09:05:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872957+0000", 2024-09-14T09:05:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9118000000000001e-05 2024-09-14T09:05:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.878076+0000", 2024-09-14T09:05:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0051180690000000003 2024-09-14T09:05:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.878192+0000", 2024-09-14T09:05:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116123 2024-09-14T09:05:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:963 3.0 3:04479396:::rbd_id.image_client.0-clone2:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T09:05:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:34.957370+0000", 2024-09-14T09:05:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 13.09335228, 2024-09-14T09:05:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004759152, 2024-09-14T09:05:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 963 2024-09-14T09:05:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957370+0000", 2024-09-14T09:05:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957370+0000", 2024-09-14T09:05:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957372+0000", 2024-09-14T09:05:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.475e-06 2024-09-14T09:05:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957382+0000", 2024-09-14T09:05:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9760000000000007e-06 2024-09-14T09:05:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957405+0000", 2024-09-14T09:05:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2860000000000001e-05 2024-09-14T09:05:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957416+0000", 2024-09-14T09:05:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.048e-05 2024-09-14T09:05:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957529+0000", 2024-09-14T09:05:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112849 2024-09-14T09:05:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T09:05:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.957538+0000", 2024-09-14T09:05:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.234e-06 2024-09-14T09:05:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.960213+0000", 2024-09-14T09:05:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0026753749999999998 2024-09-14T09:05:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.960327+0000", 2024-09-14T09:05:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000114188 2024-09-14T09:05:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 2", 2024-09-14T09:05:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.960491+0000", 2024-09-14T09:05:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016362700000000001 2024-09-14T09:05:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.961472+0000", 2024-09-14T09:05:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00098095600000000006 2024-09-14T09:05:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.961990+0000", 2024-09-14T09:05:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000518137 2024-09-14T09:05:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.962072+0000", 2024-09-14T09:05:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2108999999999996e-05 2024-09-14T09:05:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:34.962129+0000", 2024-09-14T09:05:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6885999999999997e-05 2024-09-14T09:05:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:9983 3.1a e32/23 3:5a62ac37:::rbd_data.115eadec874.000000000000000d:head v 32'21, mlcod=32'19)", 2024-09-14T09:05:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.639585+0000", 2024-09-14T09:05:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.411137538, 2024-09-14T09:05:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0046068979999999999, 2024-09-14T09:05:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639585+0000", 2024-09-14T09:05:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639585+0000", 2024-09-14T09:05:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639586+0000", 2024-09-14T09:05:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6559999999999999e-06 2024-09-14T09:05:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639629+0000", 2024-09-14T09:05:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2713000000000003e-05 2024-09-14T09:05:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639657+0000", 2024-09-14T09:05:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7569999999999999e-05 2024-09-14T09:05:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639675+0000", 2024-09-14T09:05:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8647e-05 2024-09-14T09:05:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639791+0000", 2024-09-14T09:05:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116229 2024-09-14T09:05:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.639829+0000", 2024-09-14T09:05:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7530000000000002e-05 2024-09-14T09:05:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.644164+0000", 2024-09-14T09:05:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0043347530000000002 2024-09-14T09:05:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.644192+0000", 2024-09-14T09:05:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7800000000000001e-05 2024-09-14T09:05:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:10032 3.19 3:9ed4ec6a:::rbd_data.115eadec874.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.775706+0000", 2024-09-14T09:05:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.275015830999999, 2024-09-14T09:05:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126247939, 2024-09-14T09:05:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 10032 2024-09-14T09:05:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775706+0000", 2024-09-14T09:05:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775706+0000", 2024-09-14T09:05:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775709+0000", 2024-09-14T09:05:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0400000000000001e-06 2024-09-14T09:05:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775721+0000", 2024-09-14T09:05:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2119999999999999e-05 2024-09-14T09:05:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775752+0000", 2024-09-14T09:05:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0139999999999999e-05 2024-09-14T09:05:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775766+0000", 2024-09-14T09:05:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4497e-05 2024-09-14T09:05:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775937+0000", 2024-09-14T09:05:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000171022 2024-09-14T09:05:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.776205+0000", 2024-09-14T09:05:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026749100000000002 2024-09-14T09:05:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:05:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.784970+0000", 2024-09-14T09:05:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0087648789999999997 2024-09-14T09:05:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901870+0000", 2024-09-14T09:05:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11690028 2024-09-14T09:05:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901878+0000", 2024-09-14T09:05:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7020000000000007e-06 2024-09-14T09:05:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901913+0000", 2024-09-14T09:05:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4706000000000003e-05 2024-09-14T09:05:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901954+0000", 2024-09-14T09:05:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1062000000000001e-05 2024-09-14T09:05:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:10032 3.19 e32/23 ondisk, result = 0)", 2024-09-14T09:05:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.809095+0000", 2024-09-14T09:05:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.241627552000001, 2024-09-14T09:05:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092866664000000002, 2024-09-14T09:05:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809095+0000", 2024-09-14T09:05:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809095+0000", 2024-09-14T09:05:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809096+0000", 2024-09-14T09:05:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.64e-06 2024-09-14T09:05:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809104+0000", 2024-09-14T09:05:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7330000000000003e-06 2024-09-14T09:05:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809132+0000", 2024-09-14T09:05:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7880000000000001e-05 2024-09-14T09:05:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809146+0000", 2024-09-14T09:05:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4514e-05 2024-09-14T09:05:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901858+0000", 2024-09-14T09:05:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092711713000000001 2024-09-14T09:05:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901866+0000", 2024-09-14T09:05:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4760000000000001e-06 2024-09-14T09:05:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901961+0000", 2024-09-14T09:05:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5707999999999999e-05 2024-09-14T09:05:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:10182 3.1a e33/23 3:5a62ac37:::rbd_data.115eadec874.000000000000000d:head v 33'30, mlcod=33'28)", 2024-09-14T09:05:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:38.175853+0000", 2024-09-14T09:05:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8748696040000006, 2024-09-14T09:05:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0061618740000000003, 2024-09-14T09:05:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175853+0000", 2024-09-14T09:05:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175853+0000", 2024-09-14T09:05:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175854+0000", 2024-09-14T09:05:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8169999999999999e-06 2024-09-14T09:05:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175914+0000", 2024-09-14T09:05:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9812999999999998e-05 2024-09-14T09:05:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175941+0000", 2024-09-14T09:05:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6786000000000001e-05 2024-09-14T09:05:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175955+0000", 2024-09-14T09:05:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.45e-05 2024-09-14T09:05:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.176049+0000", 2024-09-14T09:05:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3801000000000005e-05 2024-09-14T09:05:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.176083+0000", 2024-09-14T09:05:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.362e-05 2024-09-14T09:05:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.181952+0000", 2024-09-14T09:05:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0058688170000000001 2024-09-14T09:05:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.182014+0000", 2024-09-14T09:05:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2719999999999996e-05 2024-09-14T09:05:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18826 3.19 3:9c0d60c2:::rbd_data.115e3abd3b2c.0000000000000025:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.091231+0000", 2024-09-14T09:05:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.959491136, 2024-09-14T09:05:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0082287890000000002, 2024-09-14T09:05:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18826 2024-09-14T09:05:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091231+0000", 2024-09-14T09:05:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091231+0000", 2024-09-14T09:05:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091232+0000", 2024-09-14T09:05:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.437e-06 2024-09-14T09:05:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091239+0000", 2024-09-14T09:05:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6479999999999997e-06 2024-09-14T09:05:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091255+0000", 2024-09-14T09:05:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5727e-05 2024-09-14T09:05:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091264+0000", 2024-09-14T09:05:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1989999999999995e-06 2024-09-14T09:05:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091339+0000", 2024-09-14T09:05:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5117999999999995e-05 2024-09-14T09:05:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091466+0000", 2024-09-14T09:05:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000126811 2024-09-14T09:05:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:05:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091772+0000", 2024-09-14T09:05:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030571299999999999 2024-09-14T09:05:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096709+0000", 2024-09-14T09:05:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0049374960000000004 2024-09-14T09:05:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099352+0000", 2024-09-14T09:05:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0026423750000000002 2024-09-14T09:05:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099416+0000", 2024-09-14T09:05:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4731000000000001e-05 2024-09-14T09:05:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099460+0000", 2024-09-14T09:05:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3534e-05 2024-09-14T09:05:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18828 3.1a 3:5bcb7d4f:::rbd_data.115e3abd3b2c.0000000000000028:head [stat,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.091433+0000", 2024-09-14T09:05:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.959289064, 2024-09-14T09:05:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0080150039999999992, 2024-09-14T09:05:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18828 2024-09-14T09:05:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091433+0000", 2024-09-14T09:05:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091433+0000", 2024-09-14T09:05:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091434+0000", 2024-09-14T09:05:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.054e-06 2024-09-14T09:05:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091440+0000", 2024-09-14T09:05:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5249999999999996e-06 2024-09-14T09:05:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091453+0000", 2024-09-14T09:05:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2860999999999999e-05 2024-09-14T09:05:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091460+0000", 2024-09-14T09:05:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9559999999999997e-06 2024-09-14T09:05:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091522+0000", 2024-09-14T09:05:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1433999999999995e-05 2024-09-14T09:05:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091673+0000", 2024-09-14T09:05:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015112599999999999 2024-09-14T09:05:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099448+0000", 2024-09-14T09:05:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0077750479999999997 2024-09-14T09:05:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18835 3.1a 3:5d7331e3:::rbd_data.115e3abd3b2c.000000000000002c:head [stat,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.095572+0000", 2024-09-14T09:05:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.9551500069999999, 2024-09-14T09:05:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0068495320000000002, 2024-09-14T09:05:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18835 2024-09-14T09:05:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095572+0000", 2024-09-14T09:05:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095572+0000", 2024-09-14T09:05:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095574+0000", 2024-09-14T09:05:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9769999999999999e-06 2024-09-14T09:05:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095582+0000", 2024-09-14T09:05:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6410000000000002e-06 2024-09-14T09:05:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095601+0000", 2024-09-14T09:05:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9034000000000001e-05 2024-09-14T09:05:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095611+0000", 2024-09-14T09:05:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0431999999999999e-05 2024-09-14T09:05:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095680+0000", 2024-09-14T09:05:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8753999999999999e-05 2024-09-14T09:05:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095811+0000", 2024-09-14T09:05:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013112399999999999 2024-09-14T09:05:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102422+0000", 2024-09-14T09:05:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0066105699999999996 2024-09-14T09:05:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "pg_update_log_missing(3.f epoch 39/34 rep_tid 211 entries 39'21 (0'0) error 3:f32f8323:::rbd_data.115e3abd3b2c.000000000000002a:head by client.4446.0:18831 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 39'19)", 2024-09-14T09:05:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.096732+0000", 2024-09-14T09:05:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.953989687, 2024-09-14T09:05:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0054062889999999999, 2024-09-14T09:05:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096732+0000", 2024-09-14T09:05:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096732+0000", 2024-09-14T09:05:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096733+0000", 2024-09-14T09:05:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5700000000000002e-07 2024-09-14T09:05:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096742+0000", 2024-09-14T09:05:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3030000000000005e-06 2024-09-14T09:05:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096767+0000", 2024-09-14T09:05:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4621999999999999e-05 2024-09-14T09:05:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096788+0000", 2024-09-14T09:05:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0791000000000001e-05 2024-09-14T09:05:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096872+0000", 2024-09-14T09:05:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3949999999999994e-05 2024-09-14T09:05:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102139+0000", 2024-09-14T09:05:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0052669659999999997 2024-09-14T09:05:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18836 3.19 3:981038f8:::rbd_data.115e3abd3b2c.000000000000002d:head [stat,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.097080+0000", 2024-09-14T09:05:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.9536418769999999, 2024-09-14T09:05:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0053201890000000003, 2024-09-14T09:05:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18836 2024-09-14T09:05:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097080+0000", 2024-09-14T09:05:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097080+0000", 2024-09-14T09:05:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097082+0000", 2024-09-14T09:05:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3999999999999999e-06 2024-09-14T09:05:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097088+0000", 2024-09-14T09:05:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.998e-06 2024-09-14T09:05:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097104+0000", 2024-09-14T09:05:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6291999999999999e-05 2024-09-14T09:05:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097111+0000", 2024-09-14T09:05:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7490000000000001e-06 2024-09-14T09:05:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097167+0000", 2024-09-14T09:05:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6427e-05 2024-09-14T09:05:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097311+0000", 2024-09-14T09:05:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014356800000000001 2024-09-14T09:05:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102400+0000", 2024-09-14T09:05:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050897549999999996 2024-09-14T09:05:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18841 3.1a 3:5bcb7d4f:::rbd_data.115e3abd3b2c.0000000000000028:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.099711+0000", 2024-09-14T09:05:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.9510108019999999, 2024-09-14T09:05:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0060169799999999999, 2024-09-14T09:05:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18841 2024-09-14T09:05:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099711+0000", 2024-09-14T09:05:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099711+0000", 2024-09-14T09:05:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099713+0000", 2024-09-14T09:05:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5880000000000001e-06 2024-09-14T09:05:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099719+0000", 2024-09-14T09:05:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4790000000000001e-06 2024-09-14T09:05:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099735+0000", 2024-09-14T09:05:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5671999999999998e-05 2024-09-14T09:05:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099744+0000", 2024-09-14T09:05:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5210000000000002e-06 2024-09-14T09:05:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099824+0000", 2024-09-14T09:05:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0442e-05 2024-09-14T09:05:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099955+0000", 2024-09-14T09:05:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013079300000000001 2024-09-14T09:05:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 2", 2024-09-14T09:05:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.100251+0000", 2024-09-14T09:05:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029648899999999999 2024-09-14T09:05:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105631+0000", 2024-09-14T09:05:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0053793019999999999 2024-09-14T09:05:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105640+0000", 2024-09-14T09:05:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7419999999999992e-06 2024-09-14T09:05:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105683+0000", 2024-09-14T09:05:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3040999999999997e-05 2024-09-14T09:05:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105728+0000", 2024-09-14T09:05:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4910999999999997e-05 2024-09-14T09:05:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18847 3.19 3:981038f8:::rbd_data.115e3abd3b2c.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.102651+0000", 2024-09-14T09:05:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.9480706990000001, 2024-09-14T09:05:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0060367759999999998, 2024-09-14T09:05:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18847 2024-09-14T09:05:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102651+0000", 2024-09-14T09:05:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102651+0000", 2024-09-14T09:05:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102653+0000", 2024-09-14T09:05:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3799999999999999e-06 2024-09-14T09:05:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102660+0000", 2024-09-14T09:05:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9480000000000002e-06 2024-09-14T09:05:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102675+0000", 2024-09-14T09:05:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5046e-05 2024-09-14T09:05:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102681+0000", 2024-09-14T09:05:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8030000000000002e-06 2024-09-14T09:05:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102733+0000", 2024-09-14T09:05:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2846000000000002e-05 2024-09-14T09:05:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102803+0000", 2024-09-14T09:05:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9296999999999997e-05 2024-09-14T09:05:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:05:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102917+0000", 2024-09-14T09:05:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011461200000000001 2024-09-14T09:05:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108240+0000", 2024-09-14T09:05:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0053221459999999998 2024-09-14T09:05:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108584+0000", 2024-09-14T09:05:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000344006 2024-09-14T09:05:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108642+0000", 2024-09-14T09:05:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8004999999999998e-05 2024-09-14T09:05:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108688+0000", 2024-09-14T09:05:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6687000000000002e-05 2024-09-14T09:05:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18848 3.1a 3:5d7331e3:::rbd_data.115e3abd3b2c.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.102710+0000", 2024-09-14T09:05:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.948012329, 2024-09-14T09:05:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0061127430000000003, 2024-09-14T09:05:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18848 2024-09-14T09:05:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102710+0000", 2024-09-14T09:05:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102710+0000", 2024-09-14T09:05:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102711+0000", 2024-09-14T09:05:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0299999999999998e-07 2024-09-14T09:05:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102715+0000", 2024-09-14T09:05:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4360000000000004e-06 2024-09-14T09:05:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102723+0000", 2024-09-14T09:05:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7330000000000003e-06 2024-09-14T09:05:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102728+0000", 2024-09-14T09:05:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4940000000000003e-06 2024-09-14T09:05:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102793+0000", 2024-09-14T09:05:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5037999999999996e-05 2024-09-14T09:05:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102872+0000", 2024-09-14T09:05:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8616999999999996e-05 2024-09-14T09:05:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 2", 2024-09-14T09:05:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103061+0000", 2024-09-14T09:05:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018863500000000001 2024-09-14T09:05:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108275+0000", 2024-09-14T09:05:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0052146759999999997 2024-09-14T09:05:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108737+0000", 2024-09-14T09:05:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046148899999999999 2024-09-14T09:05:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108786+0000", 2024-09-14T09:05:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9191999999999999e-05 2024-09-14T09:05:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108823+0000", 2024-09-14T09:05:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.663e-05 2024-09-14T09:05:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:18846 3.f e39/34 3:f32f8323:::rbd_data.115e3abd3b2c.000000000000002a:head v 39'22, mlcod=39'20)", 2024-09-14T09:05:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.103489+0000", 2024-09-14T09:05:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.9472331700000001, 2024-09-14T09:05:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0052005020000000001, 2024-09-14T09:05:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103489+0000", 2024-09-14T09:05:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103489+0000", 2024-09-14T09:05:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103490+0000", 2024-09-14T09:05:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.091e-06 2024-09-14T09:05:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103499+0000", 2024-09-14T09:05:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1260000000000004e-06 2024-09-14T09:05:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103516+0000", 2024-09-14T09:05:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6354e-05 2024-09-14T09:05:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103526+0000", 2024-09-14T09:05:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0876000000000001e-05 2024-09-14T09:05:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103586+0000", 2024-09-14T09:05:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9301e-05 2024-09-14T09:05:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103613+0000", 2024-09-14T09:05:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6778e-05 2024-09-14T09:05:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108637+0000", 2024-09-14T09:05:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050246989999999997 2024-09-14T09:05:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108689+0000", 2024-09-14T09:05:48.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2277000000000001e-05 2024-09-14T09:05:48.123 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.123 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.123 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.123 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:19545 3.f e39/34 3:f12afce6:::rbd_data.115e4c71d6e8.0000000000000003:head v 39'24, mlcod=39'22)", 2024-09-14T09:05:48.123 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.529754+0000", 2024-09-14T09:05:48.123 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.5209680150000002, 2024-09-14T09:05:48.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037365357000000002, 2024-09-14T09:05:48.123 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.123 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.123 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:48.123 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.529754+0000", 2024-09-14T09:05:48.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.529754+0000", 2024-09-14T09:05:48.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.529756+0000", 2024-09-14T09:05:48.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.804e-06 2024-09-14T09:05:48.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537071+0000", 2024-09-14T09:05:48.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0073151120000000004 2024-09-14T09:05:48.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537111+0000", 2024-09-14T09:05:48.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0315999999999998e-05 2024-09-14T09:05:48.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537133+0000", 2024-09-14T09:05:48.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2098000000000001e-05 2024-09-14T09:05:48.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537274+0000", 2024-09-14T09:05:48.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014045400000000001 2024-09-14T09:05:48.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537312+0000", 2024-09-14T09:05:48.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7761e-05 2024-09-14T09:05:48.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:48.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.567050+0000", 2024-09-14T09:05:48.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029738547000000001 2024-09-14T09:05:48.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.567119+0000", 2024-09-14T09:05:48.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9264999999999995e-05 2024-09-14T09:05:48.127 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.127 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.127 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.127 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20695 3.4 3:2204856a:::rbd_data.115e889d4809.000000000000001b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.127 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.022232+0000", 2024-09-14T09:05:48.128 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.028490235, 2024-09-14T09:05:48.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01001865, 2024-09-14T09:05:48.128 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.128 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.128 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:48.128 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.128 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.128 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.128 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20695 2024-09-14T09:05:48.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.128 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022232+0000", 2024-09-14T09:05:48.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022232+0000", 2024-09-14T09:05:48.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022233+0000", 2024-09-14T09:05:48.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0020000000000001e-06 2024-09-14T09:05:48.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022238+0000", 2024-09-14T09:05:48.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3279999999999996e-06 2024-09-14T09:05:48.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022251+0000", 2024-09-14T09:05:48.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2573e-05 2024-09-14T09:05:48.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022258+0000", 2024-09-14T09:05:48.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3819999999999998e-06 2024-09-14T09:05:48.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022323+0000", 2024-09-14T09:05:48.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4902999999999998e-05 2024-09-14T09:05:48.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022385+0000", 2024-09-14T09:05:48.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1933999999999994e-05 2024-09-14T09:05:48.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 2", 2024-09-14T09:05:48.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022451+0000", 2024-09-14T09:05:48.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5529000000000002e-05 2024-09-14T09:05:48.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:48.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022975+0000", 2024-09-14T09:05:48.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00052479100000000004 2024-09-14T09:05:48.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:48.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.032141+0000", 2024-09-14T09:05:48.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0091651990000000006 2024-09-14T09:05:48.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:48.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.032201+0000", 2024-09-14T09:05:48.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0727999999999999e-05 2024-09-14T09:05:48.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.032251+0000", 2024-09-14T09:05:48.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9280999999999998e-05 2024-09-14T09:05:48.132 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.133 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.133 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.133 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20862 3.1a 3:580a70cb:::rbd_data.115e889d4809.0000000000000177:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.133 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.497482+0000", 2024-09-14T09:05:48.133 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.55323978299999998, 2024-09-14T09:05:48.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.032797795999999997, 2024-09-14T09:05:48.133 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.133 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.133 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:48.133 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.133 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.133 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.134 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20862 2024-09-14T09:05:48.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.134 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497482+0000", 2024-09-14T09:05:48.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497482+0000", 2024-09-14T09:05:48.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497485+0000", 2024-09-14T09:05:48.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8569999999999999e-06 2024-09-14T09:05:48.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497496+0000", 2024-09-14T09:05:48.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0277e-05 2024-09-14T09:05:48.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497519+0000", 2024-09-14T09:05:48.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3513e-05 2024-09-14T09:05:48.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497533+0000", 2024-09-14T09:05:48.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4343e-05 2024-09-14T09:05:48.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497642+0000", 2024-09-14T09:05:48.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010840099999999999 2024-09-14T09:05:48.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497840+0000", 2024-09-14T09:05:48.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000198192 2024-09-14T09:05:48.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 2", 2024-09-14T09:05:48.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.499243+0000", 2024-09-14T09:05:48.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0014027550000000001 2024-09-14T09:05:48.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:48.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530087+0000", 2024-09-14T09:05:48.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030844086999999999 2024-09-14T09:05:48.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:48.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530120+0000", 2024-09-14T09:05:48.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3309000000000002e-05 2024-09-14T09:05:48.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:48.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530207+0000", 2024-09-14T09:05:48.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6536e-05 2024-09-14T09:05:48.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530280+0000", 2024-09-14T09:05:48.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3526e-05 2024-09-14T09:05:48.140 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.140 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.140 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.141 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:20862 3.1a e41/34 ondisk, result = 0)", 2024-09-14T09:05:48.141 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.503269+0000", 2024-09-14T09:05:48.141 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.54745277800000003, 2024-09-14T09:05:48.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026831292999999999, 2024-09-14T09:05:48.141 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.141 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.141 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.141 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503269+0000", 2024-09-14T09:05:48.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503269+0000", 2024-09-14T09:05:48.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503271+0000", 2024-09-14T09:05:48.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3820000000000001e-06 2024-09-14T09:05:48.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503276+0000", 2024-09-14T09:05:48.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6400000000000002e-06 2024-09-14T09:05:48.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503293+0000", 2024-09-14T09:05:48.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6404000000000001e-05 2024-09-14T09:05:48.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503304+0000", 2024-09-14T09:05:48.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0905e-05 2024-09-14T09:05:48.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530061+0000", 2024-09-14T09:05:48.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026757019 2024-09-14T09:05:48.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530076+0000", 2024-09-14T09:05:48.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5526999999999999e-05 2024-09-14T09:05:48.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530101+0000", 2024-09-14T09:05:48.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4416000000000001e-05 2024-09-14T09:05:48.144 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.144 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.144 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.145 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20892 3.19 3:9c9d5d4e:::rbd_data.115e889d4809.000000000000007c:head [set-alloc-hint object_size 262144 write_size 262144,write 54228~63989] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.145 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.617095+0000", 2024-09-14T09:05:48.145 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.433627286, 2024-09-14T09:05:48.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0096848479999999994, 2024-09-14T09:05:48.145 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.145 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.145 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:48.145 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.145 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.145 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.145 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20892 2024-09-14T09:05:48.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.145 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.617095+0000", 2024-09-14T09:05:48.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.617095+0000", 2024-09-14T09:05:48.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.617097+0000", 2024-09-14T09:05:48.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.627e-06 2024-09-14T09:05:48.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.617150+0000", 2024-09-14T09:05:48.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2528999999999998e-05 2024-09-14T09:05:48.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.617181+0000", 2024-09-14T09:05:48.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1279999999999999e-05 2024-09-14T09:05:48.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.617197+0000", 2024-09-14T09:05:48.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5843000000000001e-05 2024-09-14T09:05:48.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.617341+0000", 2024-09-14T09:05:48.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014422799999999999 2024-09-14T09:05:48.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.617583+0000", 2024-09-14T09:05:48.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024192800000000001 2024-09-14T09:05:48.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:05:48.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.617808+0000", 2024-09-14T09:05:48.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022461700000000001 2024-09-14T09:05:48.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:48.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.625752+0000", 2024-09-14T09:05:48.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0079439809999999993 2024-09-14T09:05:48.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:48.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.626621+0000", 2024-09-14T09:05:48.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00086927700000000001 2024-09-14T09:05:48.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:48.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.626710+0000", 2024-09-14T09:05:48.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9145000000000001e-05 2024-09-14T09:05:48.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.626780+0000", 2024-09-14T09:05:48.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9393000000000005e-05 2024-09-14T09:05:48.150 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.150 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.150 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.150 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.150 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.150 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:48.151 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:05:48.152 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:48.148+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:48.252 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:48.256+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:48.325 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:48.325 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:48.325 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:48.326 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:48.340 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:05:48.352 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:48.356+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:48.383 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone7 from 0xacfe17a (overlap 0x1d7c5f) to 0x1a8988 2024-09-14T09:05:48.453 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:48.448+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:48.470 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:48.470 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:48.470 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:48.470 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:48.470 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:48.485 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:05:48.554 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:48.556+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:48.642 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-14T09:05:48.654 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:48.652+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:48.660 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18383 3.9 3:9252f7c9:::rbd_data.115e3abd3b2c.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:05:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:05:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.500403189, 2024-09-14T09:05:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056433830999999997, 2024-09-14T09:05:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18383 2024-09-14T09:05:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:05:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:05:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158423+0000", 2024-09-14T09:05:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T09:05:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158429+0000", 2024-09-14T09:05:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0179999999999998e-06 2024-09-14T09:05:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158440+0000", 2024-09-14T09:05:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0769e-05 2024-09-14T09:05:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158445+0000", 2024-09-14T09:05:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5840000000000002e-06 2024-09-14T09:05:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158495+0000", 2024-09-14T09:05:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9129999999999999e-05 2024-09-14T09:05:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158598+0000", 2024-09-14T09:05:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102938 2024-09-14T09:05:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:05:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.161048+0000", 2024-09-14T09:05:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002450336 2024-09-14T09:05:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214605+0000", 2024-09-14T09:05:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053557526000000001 2024-09-14T09:05:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214634+0000", 2024-09-14T09:05:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8483000000000002e-05 2024-09-14T09:05:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214731+0000", 2024-09-14T09:05:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7026000000000002e-05 2024-09-14T09:05:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214856+0000", 2024-09-14T09:05:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012474799999999999 2024-09-14T09:05:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18383 3.9 e38/34 ondisk, result = 0)", 2024-09-14T09:05:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:05:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.4900867079999998, 2024-09-14T09:05:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045880575999999999, 2024-09-14T09:05:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:05:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:05:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168739+0000", 2024-09-14T09:05:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3900000000000004e-07 2024-09-14T09:05:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168744+0000", 2024-09-14T09:05:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5929999999999999e-06 2024-09-14T09:05:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168754+0000", 2024-09-14T09:05:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0686999999999999e-05 2024-09-14T09:05:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168761+0000", 2024-09-14T09:05:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6340000000000003e-06 2024-09-14T09:05:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214578+0000", 2024-09-14T09:05:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045817443999999999 2024-09-14T09:05:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214596+0000", 2024-09-14T09:05:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7159e-05 2024-09-14T09:05:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214619+0000", 2024-09-14T09:05:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.332e-05 2024-09-14T09:05:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19152 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14159058~8940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:05:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.3361742520000002, 2024-09-14T09:05:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12891994500000001, 2024-09-14T09:05:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19152 2024-09-14T09:05:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:05:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:05:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322653+0000", 2024-09-14T09:05:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3750000000000001e-06 2024-09-14T09:05:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329647+0000", 2024-09-14T09:05:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069941960000000003 2024-09-14T09:05:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329673+0000", 2024-09-14T09:05:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5760000000000001e-05 2024-09-14T09:05:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329684+0000", 2024-09-14T09:05:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0474e-05 2024-09-14T09:05:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329776+0000", 2024-09-14T09:05:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2467999999999996e-05 2024-09-14T09:05:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329987+0000", 2024-09-14T09:05:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021133 2024-09-14T09:05:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:05:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.331303+0000", 2024-09-14T09:05:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001315629 2024-09-14T09:05:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.411090+0000", 2024-09-14T09:05:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079787477999999995 2024-09-14T09:05:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451463+0000", 2024-09-14T09:05:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040372312 2024-09-14T09:05:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451523+0000", 2024-09-14T09:05:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0420000000000001e-05 2024-09-14T09:05:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451571+0000", 2024-09-14T09:05:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7503000000000002e-05 2024-09-14T09:05:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20923 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2359296~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671847+0000", 2024-09-14T09:05:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.98697811700000004, 2024-09-14T09:05:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013320925000000001, 2024-09-14T09:05:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20923 2024-09-14T09:05:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671847+0000", 2024-09-14T09:05:48.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671847+0000", 2024-09-14T09:05:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671848+0000", 2024-09-14T09:05:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1299999999999999e-07 2024-09-14T09:05:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671851+0000", 2024-09-14T09:05:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.49e-06 2024-09-14T09:05:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671862+0000", 2024-09-14T09:05:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1236999999999999e-05 2024-09-14T09:05:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671869+0000", 2024-09-14T09:05:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9530000000000002e-06 2024-09-14T09:05:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.683496+0000", 2024-09-14T09:05:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011626965 2024-09-14T09:05:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.683754+0000", 2024-09-14T09:05:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025779399999999999 2024-09-14T09:05:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.685168+0000", 2024-09-14T09:05:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0014137729999999999 2024-09-14T09:05:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20924 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 655360~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671905+0000", 2024-09-14T09:05:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.98691953099999996, 2024-09-14T09:05:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015888248000000001, 2024-09-14T09:05:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20924 2024-09-14T09:05:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671905+0000", 2024-09-14T09:05:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671905+0000", 2024-09-14T09:05:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671907+0000", 2024-09-14T09:05:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.136e-06 2024-09-14T09:05:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671912+0000", 2024-09-14T09:05:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0599999999999998e-06 2024-09-14T09:05:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671925+0000", 2024-09-14T09:05:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3103e-05 2024-09-14T09:05:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671936+0000", 2024-09-14T09:05:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0794e-05 2024-09-14T09:05:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.685304+0000", 2024-09-14T09:05:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013368011000000001 2024-09-14T09:05:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.685562+0000", 2024-09-14T09:05:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025813700000000003 2024-09-14T09:05:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.687794+0000", 2024-09-14T09:05:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0022320069999999998 2024-09-14T09:05:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20925 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:05:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.98687547200000003, 2024-09-14T09:05:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017084327999999999, 2024-09-14T09:05:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20925 2024-09-14T09:05:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:05:48.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:05:48.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:05:48.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3e-07 2024-09-14T09:05:48.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671953+0000", 2024-09-14T09:05:48.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0299999999999998e-06 2024-09-14T09:05:48.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671964+0000", 2024-09-14T09:05:48.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1e-05 2024-09-14T09:05:48.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671971+0000", 2024-09-14T09:05:48.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9929999999999998e-06 2024-09-14T09:05:48.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.687941+0000", 2024-09-14T09:05:48.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015969357 2024-09-14T09:05:48.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.688070+0000", 2024-09-14T09:05:48.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000129858 2024-09-14T09:05:48.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689034+0000", 2024-09-14T09:05:48.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00096336000000000002 2024-09-14T09:05:48.694 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.694 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.695 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.695 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20926 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.695 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:05:48.695 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.98684225199999998, 2024-09-14T09:05:48.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018572207, 2024-09-14T09:05:48.695 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.695 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.695 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.695 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.696 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.696 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.696 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20926 2024-09-14T09:05:48.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.696 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:05:48.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:05:48.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:05:48.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:05:48.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671990+0000", 2024-09-14T09:05:48.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5769999999999999e-06 2024-09-14T09:05:48.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672001+0000", 2024-09-14T09:05:48.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1293e-05 2024-09-14T09:05:48.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672008+0000", 2024-09-14T09:05:48.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9099999999999999e-06 2024-09-14T09:05:48.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689147+0000", 2024-09-14T09:05:48.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017139060000000001 2024-09-14T09:05:48.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689324+0000", 2024-09-14T09:05:48.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017640200000000001 2024-09-14T09:05:48.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690555+0000", 2024-09-14T09:05:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001231212 2024-09-14T09:05:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20927 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2621440~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:05:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.98680528599999995, 2024-09-14T09:05:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020002227000000001, 2024-09-14T09:05:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20927 2024-09-14T09:05:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:05:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:05:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:05:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9699999999999995e-07 2024-09-14T09:05:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672024+0000", 2024-09-14T09:05:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:05:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672036+0000", 2024-09-14T09:05:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.119e-05 2024-09-14T09:05:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672044+0000", 2024-09-14T09:05:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8370000000000003e-06 2024-09-14T09:05:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690602+0000", 2024-09-14T09:05:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018557648 2024-09-14T09:05:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690805+0000", 2024-09-14T09:05:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020282899999999999 2024-09-14T09:05:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692022+0000", 2024-09-14T09:05:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012170429999999999 2024-09-14T09:05:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20928 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 917504~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:05:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.98676914599999999, 2024-09-14T09:05:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021623738, 2024-09-14T09:05:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20928 2024-09-14T09:05:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:05:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:05:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672057+0000", 2024-09-14T09:05:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:05:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672059+0000", 2024-09-14T09:05:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9270000000000001e-06 2024-09-14T09:05:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672070+0000", 2024-09-14T09:05:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0773e-05 2024-09-14T09:05:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672077+0000", 2024-09-14T09:05:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9600000000000003e-06 2024-09-14T09:05:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692154+0000", 2024-09-14T09:05:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020076482 2024-09-14T09:05:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692419+0000", 2024-09-14T09:05:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026517400000000002 2024-09-14T09:05:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693680+0000", 2024-09-14T09:05:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012607390000000001 2024-09-14T09:05:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20929 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2752512~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:05:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.986736104, 2024-09-14T09:05:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022898728, 2024-09-14T09:05:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20929 2024-09-14T09:05:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:05:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:05:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672090+0000", 2024-09-14T09:05:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:05:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672092+0000", 2024-09-14T09:05:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8940000000000002e-06 2024-09-14T09:05:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672103+0000", 2024-09-14T09:05:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0815999999999999e-05 2024-09-14T09:05:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672110+0000", 2024-09-14T09:05:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9469999999999997e-06 2024-09-14T09:05:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693737+0000", 2024-09-14T09:05:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021626467999999999 2024-09-14T09:05:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693962+0000", 2024-09-14T09:05:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225474 2024-09-14T09:05:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.694988+0000", 2024-09-14T09:05:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001025469 2024-09-14T09:05:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20930 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:05:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.98670301000000005, 2024-09-14T09:05:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024303286, 2024-09-14T09:05:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20930 2024-09-14T09:05:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:05:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:05:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672123+0000", 2024-09-14T09:05:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:05:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672125+0000", 2024-09-14T09:05:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7300000000000001e-06 2024-09-14T09:05:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672134+0000", 2024-09-14T09:05:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7930000000000003e-06 2024-09-14T09:05:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672141+0000", 2024-09-14T09:05:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9029999999999998e-06 2024-09-14T09:05:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695087+0000", 2024-09-14T09:05:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022946013000000001 2024-09-14T09:05:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695350+0000", 2024-09-14T09:05:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026266000000000001 2024-09-14T09:05:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696425+0000", 2024-09-14T09:05:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001075527 2024-09-14T09:05:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20931 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:05:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.98666978100000002, 2024-09-14T09:05:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025331519, 2024-09-14T09:05:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20931 2024-09-14T09:05:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:05:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:05:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672156+0000", 2024-09-14T09:05:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4300000000000002e-07 2024-09-14T09:05:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672159+0000", 2024-09-14T09:05:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T09:05:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672170+0000", 2024-09-14T09:05:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1192999999999999e-05 2024-09-14T09:05:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672177+0000", 2024-09-14T09:05:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9059999999999996e-06 2024-09-14T09:05:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696532+0000", 2024-09-14T09:05:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024355291000000001 2024-09-14T09:05:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696755+0000", 2024-09-14T09:05:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022297699999999999 2024-09-14T09:05:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697487+0000", 2024-09-14T09:05:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00073171499999999997 2024-09-14T09:05:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20932 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1179648~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:05:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.98663633799999995, 2024-09-14T09:05:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026104083, 2024-09-14T09:05:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20932 2024-09-14T09:05:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:05:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:05:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:05:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0699999999999996e-07 2024-09-14T09:05:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672192+0000", 2024-09-14T09:05:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7700000000000002e-06 2024-09-14T09:05:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672203+0000", 2024-09-14T09:05:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0956e-05 2024-09-14T09:05:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672211+0000", 2024-09-14T09:05:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3769999999999996e-06 2024-09-14T09:05:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697530+0000", 2024-09-14T09:05:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025318411999999998 2024-09-14T09:05:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697640+0000", 2024-09-14T09:05:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011045299999999999 2024-09-14T09:05:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698293+0000", 2024-09-14T09:05:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065240800000000005 2024-09-14T09:05:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20933 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3014656~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:05:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.98660213500000005, 2024-09-14T09:05:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026558875999999999, 2024-09-14T09:05:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20933 2024-09-14T09:05:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:05:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:05:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672224+0000", 2024-09-14T09:05:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2399999999999997e-07 2024-09-14T09:05:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672234+0000", 2024-09-14T09:05:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0793e-05 2024-09-14T09:05:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672246+0000", 2024-09-14T09:05:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659e-05 2024-09-14T09:05:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672255+0000", 2024-09-14T09:05:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5730000000000008e-06 2024-09-14T09:05:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698326+0000", 2024-09-14T09:05:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026071414000000001 2024-09-14T09:05:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698374+0000", 2024-09-14T09:05:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8032999999999999e-05 2024-09-14T09:05:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698782+0000", 2024-09-14T09:05:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040768 2024-09-14T09:05:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20934 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1310720~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:05:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.98656025899999999, 2024-09-14T09:05:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027292067999999999, 2024-09-14T09:05:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20934 2024-09-14T09:05:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:05:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:05:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672266+0000", 2024-09-14T09:05:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6299999999999993e-07 2024-09-14T09:05:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672269+0000", 2024-09-14T09:05:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8569999999999999e-06 2024-09-14T09:05:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672280+0000", 2024-09-14T09:05:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1236999999999999e-05 2024-09-14T09:05:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672287+0000", 2024-09-14T09:05:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8260000000000004e-06 2024-09-14T09:05:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698820+0000", 2024-09-14T09:05:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026533055999999999 2024-09-14T09:05:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698933+0000", 2024-09-14T09:05:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011291 2024-09-14T09:05:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699557+0000", 2024-09-14T09:05:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062421899999999997 2024-09-14T09:05:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20935 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:05:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.986526499, 2024-09-14T09:05:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028108610999999999, 2024-09-14T09:05:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20935 2024-09-14T09:05:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:05:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:05:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672299+0000", 2024-09-14T09:05:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T09:05:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672302+0000", 2024-09-14T09:05:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7640000000000001e-06 2024-09-14T09:05:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672313+0000", 2024-09-14T09:05:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1080000000000001e-05 2024-09-14T09:05:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672321+0000", 2024-09-14T09:05:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6490000000000008e-06 2024-09-14T09:05:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699587+0000", 2024-09-14T09:05:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027265884000000001 2024-09-14T09:05:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699631+0000", 2024-09-14T09:05:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4947999999999997e-05 2024-09-14T09:05:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700407+0000", 2024-09-14T09:05:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00077559299999999999 2024-09-14T09:05:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20936 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1441792~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:05:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.986492323, 2024-09-14T09:05:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028784750000000001, 2024-09-14T09:05:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20936 2024-09-14T09:05:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:05:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:05:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:05:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4700000000000001e-07 2024-09-14T09:05:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672336+0000", 2024-09-14T09:05:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6730000000000001e-06 2024-09-14T09:05:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672347+0000", 2024-09-14T09:05:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0710000000000001e-05 2024-09-14T09:05:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672354+0000", 2024-09-14T09:05:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4630000000000004e-06 2024-09-14T09:05:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700442+0000", 2024-09-14T09:05:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028087951 2024-09-14T09:05:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700557+0000", 2024-09-14T09:05:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011461800000000001 2024-09-14T09:05:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701117+0000", 2024-09-14T09:05:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056058799999999999 2024-09-14T09:05:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20937 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:05:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.98645919699999995, 2024-09-14T09:05:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029493028000000001, 2024-09-14T09:05:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20937 2024-09-14T09:05:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:05:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:05:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:05:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T09:05:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672384+0000", 2024-09-14T09:05:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7127000000000001e-05 2024-09-14T09:05:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672395+0000", 2024-09-14T09:05:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759999999999999e-05 2024-09-14T09:05:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672402+0000", 2024-09-14T09:05:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0960000000000003e-06 2024-09-14T09:05:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701147+0000", 2024-09-14T09:05:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028744241 2024-09-14T09:05:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701246+0000", 2024-09-14T09:05:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9031999999999996e-05 2024-09-14T09:05:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701859+0000", 2024-09-14T09:05:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061310499999999997 2024-09-14T09:05:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20938 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:05:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.98641203700000002, 2024-09-14T09:05:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030109219999999999, 2024-09-14T09:05:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20938 2024-09-14T09:05:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:05:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:05:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672414+0000", 2024-09-14T09:05:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-07 2024-09-14T09:05:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672417+0000", 2024-09-14T09:05:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:05:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672428+0000", 2024-09-14T09:05:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0873000000000001e-05 2024-09-14T09:05:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672435+0000", 2024-09-14T09:05:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:05:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701884+0000", 2024-09-14T09:05:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029449811999999999 2024-09-14T09:05:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701934+0000", 2024-09-14T09:05:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9703000000000002e-05 2024-09-14T09:05:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702522+0000", 2024-09-14T09:05:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00058811200000000001 2024-09-14T09:05:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20939 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:05:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.98637990399999997, 2024-09-14T09:05:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030779259, 2024-09-14T09:05:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20939 2024-09-14T09:05:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:05:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:05:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672446+0000", 2024-09-14T09:05:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:05:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672449+0000", 2024-09-14T09:05:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7930000000000002e-06 2024-09-14T09:05:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672461+0000", 2024-09-14T09:05:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T09:05:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672471+0000", 2024-09-14T09:05:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0807e-05 2024-09-14T09:05:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702551+0000", 2024-09-14T09:05:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030079228 2024-09-14T09:05:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702664+0000", 2024-09-14T09:05:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011351800000000001 2024-09-14T09:05:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.703224+0000", 2024-09-14T09:05:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056026999999999997 2024-09-14T09:05:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.768 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.768 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.768 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.768 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:48.770 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:05:48.771 INFO:teuthology.orchestra.run.smithi176.stdout:777 write 0xd754b1b thru 0xd75d42d (0x8913 bytes) 2024-09-14T09:05:48.771 INFO:teuthology.orchestra.run.smithi176.stdout:778 writesame 0x30557d2 thru 0x3059e02 data_size 0x11(0x4631 bytes) 2024-09-14T09:05:48.771 INFO:teuthology.orchestra.run.smithi176.stdout:779 punch from 0xaf42e62 to 0xaf50e6d, (0xe00b bytes) 2024-09-14T09:05:48.772 INFO:teuthology.orchestra.run.smithi176.stdout:781 write 0x45cb8ba thru 0x45cbc51 (0x398 bytes) 2024-09-14T09:05:48.772 INFO:teuthology.orchestra.run.smithi176.stdout:782 trunc from 0xd75d42e to 0x3828c2a 2024-09-14T09:05:48.772 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:48.764+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:48.772 INFO:teuthology.orchestra.run.smithi176.stdout:783 writesame 0x717cde5 thru 0x718b3af data_size 0x1(0xe5cb bytes) 2024-09-14T09:05:48.783 INFO:teuthology.orchestra.run.smithi176.stdout:784 trunc from 0x718b3b0 to 0xbc7fb12 2024-09-14T09:05:48.797 INFO:teuthology.orchestra.run.smithi176.stdout:785 read 0x1538501 thru 0x1542507 (0xa007 bytes) 2024-09-14T09:05:48.797 INFO:teuthology.orchestra.run.smithi176.stdout:786 read 0xabee9d1 thru 0xabf8bcb (0xa1fb bytes) 2024-09-14T09:05:48.797 INFO:teuthology.orchestra.run.smithi176.stdout:787 punch from 0x1f869d2 to 0x1f8e6fd, (0x7d2b bytes) 2024-09-14T09:05:48.797 INFO:teuthology.orchestra.run.smithi176.stdout:790 punch from 0x9546d74 to 0x9554c24, (0xdeb0 bytes) 2024-09-14T09:05:48.797 INFO:teuthology.orchestra.run.smithi176.stdout:791 read 0x54f9916 thru 0x54f9ccc (0x3b7 bytes) 2024-09-14T09:05:48.798 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:48.798 INFO:teuthology.orchestra.run.smithi176.stdout:793 punch from 0xb4f0279 to 0xb4f5433, (0x51ba bytes) 2024-09-14T09:05:48.798 INFO:teuthology.orchestra.run.smithi176.stdout:796 read 0xb71b937 thru 0xb71d2f7 (0x19c1 bytes) 2024-09-14T09:05:48.798 INFO:teuthology.orchestra.run.smithi176.stdout:797 punch from 0x6acc523 to 0x6acf12d, (0x2c0a bytes) 2024-09-14T09:05:48.798 INFO:teuthology.orchestra.run.smithi176.stdout:798 punch from 0xa498633 to 0xa4a1dc8, (0x9795 bytes) 2024-09-14T09:05:48.798 INFO:teuthology.orchestra.run.smithi176.stdout:799 write 0x595a2b5 thru 0x5963eec (0x9c38 bytes) 2024-09-14T09:05:48.821 INFO:teuthology.orchestra.run.smithi176.stdout:800 write 0xd14ee67 thru 0xd15358b (0x4725 bytes) 2024-09-14T09:05:48.837 INFO:teuthology.orchestra.run.smithi176.stdout:801 trunc from 0xd15358c to 0x8fcac16 2024-09-14T09:05:48.861 INFO:teuthology.orchestra.run.smithi176.stdout:802 punch from 0x672078a to 0x672c1e1, (0xba57 bytes) 2024-09-14T09:05:48.871 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:48.868+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:48.875 INFO:teuthology.orchestra.run.smithi176.stdout:803 trunc from 0x8fcac16 to 0x1f3956 2024-09-14T09:05:48.925 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:48.925 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:48.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.926 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:21521 3.18 3:1a13cf78:::rbd_data.115e9d49ac1.000000000000000a:head [call rbd.sparse_copyup in=24b,set-alloc-hint object_size 262144 write_size 262144,truncate 131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-14T09:05:48.926 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:48.924948+0000", 2024-09-14T09:05:48.926 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.001902265, 2024-09-14T09:05:48.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001979537, 2024-09-14T09:05:48.926 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:48.926 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:48.926 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "waiting for sub ops", 2024-09-14T09:05:48.926 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:48.926 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:48.926 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:48.926 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 21521 2024-09-14T09:05:48.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.926 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:48.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:48.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.924948+0000", 2024-09-14T09:05:48.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:48.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.924948+0000", 2024-09-14T09:05:48.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:48.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:48.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.924950+0000", 2024-09-14T09:05:48.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0370000000000001e-06 2024-09-14T09:05:48.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:48.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.924959+0000", 2024-09-14T09:05:48.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6530000000000006e-06 2024-09-14T09:05:48.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:48.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.924986+0000", 2024-09-14T09:05:48.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6225999999999999e-05 2024-09-14T09:05:48.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:48.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.925003+0000", 2024-09-14T09:05:48.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7762999999999999e-05 2024-09-14T09:05:48.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:48.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.925083+0000", 2024-09-14T09:05:48.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0043000000000006e-05 2024-09-14T09:05:48.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:48.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.925207+0000", 2024-09-14T09:05:48.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012383399999999999 2024-09-14T09:05:48.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:05:48.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.925386+0000", 2024-09-14T09:05:48.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017918099999999999 2024-09-14T09:05:48.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:48.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:48.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:48.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.926760+0000", 2024-09-14T09:05:48.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001373179 2024-09-14T09:05:48.930 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.930 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:48.930 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.930 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:48.930 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:05:48.930 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 1 2024-09-14T09:05:48.931 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:48.943 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:05:48.955 INFO:teuthology.orchestra.run.smithi176.stdout:804 read 0x1788db thru 0x18056e (0x7c94 bytes) 2024-09-14T09:05:48.968 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T09:05:48.972 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:48.972+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:48.973 INFO:teuthology.orchestra.run.smithi176.stdout:805 write 0x380e2bb thru 0x38180a3 (0x9de9 bytes) 2024-09-14T09:05:48.982 INFO:teuthology.orchestra.run.smithi176.stdout:806 punch from 0x381ab4 to 0x3873b8, (0x5904 bytes) 2024-09-14T09:05:48.983 INFO:teuthology.orchestra.run.smithi176.stdout:807 read 0xbb1192 thru 0xbb40cd (0x2f3c bytes) 2024-09-14T09:05:48.983 INFO:teuthology.orchestra.run.smithi176.stdout:808 punch from 0x25c56f2 to 0x25cc3c6, (0x6cd4 bytes) 2024-09-14T09:05:48.983 INFO:teuthology.orchestra.run.smithi176.stdout:809 punch from 0x26ac405 to 0x26bb889, (0xf484 bytes) 2024-09-14T09:05:48.988 INFO:teuthology.orchestra.run.smithi176.stdout:813 trunc from 0x38180a4 to 0x70804bd 2024-09-14T09:05:49.017 INFO:teuthology.orchestra.run.smithi176.stdout:814 writesame 0xc4ed6a8 thru 0xc4fc790 data_size 0x1(0xf0e9 bytes) 2024-09-14T09:05:49.018 INFO:teuthology.orchestra.run.smithi176.stdout:816 read 0x5dd4fd1 thru 0x5de3684 (0xe6b4 bytes) 2024-09-14T09:05:49.032 INFO:teuthology.orchestra.run.smithi176.stdout:817 writesame 0xeca651d thru 0xecb328c data_size 0x26(0xcd70 bytes) 2024-09-14T09:05:49.032 INFO:teuthology.orchestra.run.smithi176.stdout:819 writesame 0xd44e113 thru 0xd44e664 data_size 0x3(0x552 bytes) 2024-09-14T09:05:49.038 INFO:teuthology.orchestra.run.smithi176.stdout:820 write 0x12f41fe thru 0x12f648c (0x228f bytes) 2024-09-14T09:05:49.054 INFO:teuthology.orchestra.run.smithi176.stdout:821 trunc from 0xecb328d to 0x96a74c4 2024-09-14T09:05:49.071 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:05:49.071 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:05:49.071 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-14T09:05:49.072 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-14T09:05:49.074 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:49.068+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:49.076 INFO:teuthology.orchestra.run.smithi176.stdout:822 punch from 0x7674e65 to 0x7683614, (0xe7af bytes) 2024-09-14T09:05:49.086 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:49.086 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:49.086 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:49.086 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:49.086 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:49.091 INFO:teuthology.orchestra.run.smithi176.stdout:823 writesame 0xae8d186 thru 0xae99d64 data_size 0x9(0xcbdf bytes) 2024-09-14T09:05:49.091 INFO:teuthology.orchestra.run.smithi176.stdout:824 punch from 0xaa10a34 to 0xaa123c5, (0x1991 bytes) 2024-09-14T09:05:49.091 INFO:teuthology.orchestra.run.smithi176.stdout:826 write 0x982e99d thru 0x983a73d (0xbda1 bytes) 2024-09-14T09:05:49.103 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:05:49.106 INFO:teuthology.orchestra.run.smithi176.stdout:828 punch from 0x58ae698 to 0x58b3714, (0x507c bytes) 2024-09-14T09:05:49.106 INFO:teuthology.orchestra.run.smithi176.stdout:832 read 0x47d5ff8 thru 0x47d66d8 (0x6e1 bytes) 2024-09-14T09:05:49.106 INFO:teuthology.orchestra.run.smithi176.stdout:833 read 0x8c1b01c thru 0x8c1c056 (0x103b bytes) 2024-09-14T09:05:49.106 INFO:teuthology.orchestra.run.smithi176.stdout:834 read 0x7cfe00f thru 0x7d050eb (0x70dd bytes) 2024-09-14T09:05:49.106 INFO:teuthology.orchestra.run.smithi176.stdout:837 write 0x185267f thru 0x1854f4b (0x28cd bytes) 2024-09-14T09:05:49.112 INFO:teuthology.orchestra.run.smithi176.stdout:838 read 0x44e30 thru 0x5370e (0xe8df bytes) 2024-09-14T09:05:49.113 INFO:teuthology.orchestra.run.smithi176.stdout:839 punch from 0xa06b35d to 0xa07738a, (0xc02d bytes) 2024-09-14T09:05:49.113 INFO:teuthology.orchestra.run.smithi176.stdout:840 read 0xaac87d5 thru 0xaac9b3c (0x1368 bytes) 2024-09-14T09:05:49.113 INFO:teuthology.orchestra.run.smithi176.stdout:842 write 0xa9e77dd thru 0xa9f7493 (0xfcb7 bytes) 2024-09-14T09:05:49.125 INFO:teuthology.orchestra.run.smithi176.stdout:truncating to largest ever: 0xed616fc 2024-09-14T09:05:49.131 INFO:teuthology.orchestra.run.smithi176.stdout:843 trunc from 0xae99d65 to 0xed616fc 2024-09-14T09:05:49.146 INFO:teuthology.orchestra.run.smithi176.stdout:845 write 0x4a30ea5 thru 0x4a3e945 (0xdaa1 bytes) 2024-09-14T09:05:49.156 INFO:teuthology.orchestra.run.smithi176.stdout:847 write 0x29341be thru 0x2937d57 (0x3b9a bytes) 2024-09-14T09:05:49.164 INFO:teuthology.orchestra.run.smithi176.stdout:848 read 0xaaf534e thru 0xaafb146 (0x5df9 bytes) 2024-09-14T09:05:49.164 INFO:teuthology.orchestra.run.smithi176.stdout:849 clone 9 order 20 su 4096 sc 12 2024-09-14T09:05:49.176 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:49.176+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:49.181 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:05:49.233 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:49.233 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:49.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:49.234 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:49.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.234 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:05:49.234 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:05:49.234 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.149125444999999, 2024-09-14T09:05:49.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:05:49.234 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.234 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.234 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:49.234 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:49.234 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:49.234 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:49.234 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:05:49.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.235 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:05:49.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:05:49.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:05:49.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:05:49.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:05:49.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:05:49.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:05:49.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:05:49.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:05:49.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:05:49.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:05:49.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:05:49.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:05:49.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:05:49.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:05:49.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:05:49.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:05:49.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:05:49.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:05:49.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:05:49.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:05:49.238 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.238 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.239 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.239 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:05:49.239 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:05:49.239 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.149070786999999, 2024-09-14T09:05:49.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:05:49.239 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.239 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.239 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:49.239 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:49.239 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:49.239 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:49.239 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:05:49.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.240 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:05:49.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:05:49.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:05:49.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:05:49.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:05:49.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:05:49.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:05:49.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:05:49.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:05:49.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:05:49.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:05:49.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:05:49.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:05:49.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:05:49.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:05:49.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:05:49.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:05:49.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:05:49.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:05:49.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:05:49.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:05:49.243 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.243 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.243 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.244 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:05:49.244 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:05:49.244 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.148965412999999, 2024-09-14T09:05:49.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:05:49.244 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.244 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.244 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:49.244 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:49.244 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:49.244 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:49.244 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:05:49.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.245 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:05:49.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:05:49.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:05:49.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:05:49.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:05:49.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:05:49.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:05:49.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:05:49.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:05:49.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:05:49.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:05:49.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:05:49.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:05:49.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:05:49.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:05:49.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:05:49.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:05:49.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:05:49.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:05:49.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:05:49.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:05:49.248 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.248 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.249 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.249 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:05:49.249 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:05:49.249 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.148608649, 2024-09-14T09:05:49.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:05:49.249 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.249 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.249 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:49.249 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:49.249 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:49.249 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:49.250 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:05:49.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.250 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:05:49.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:05:49.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:05:49.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:05:49.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:05:49.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:05:49.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:05:49.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:05:49.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:05:49.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:05:49.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:05:49.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:05:49.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:05:49.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:05:49.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:05:49.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:05:49.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:05:49.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:05:49.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:05:49.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:05:49.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:05:49.254 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.254 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.254 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.254 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:05:49.254 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:05:49.254 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.148523911000002, 2024-09-14T09:05:49.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:05:49.254 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.254 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.254 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:49.255 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:49.255 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:49.255 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:49.255 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:05:49.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.255 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:05:49.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:05:49.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:05:49.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:05:49.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:05:49.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:05:49.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:05:49.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:05:49.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:05:49.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:05:49.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:05:49.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:05:49.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:05:49.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:05:49.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:05:49.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:05:49.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:05:49.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:05:49.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:05:49.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:05:49.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:05:49.259 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.259 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.259 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.259 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:05:49.259 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:05:49.259 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.148436225000001, 2024-09-14T09:05:49.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:05:49.259 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.259 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.259 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:49.259 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:49.259 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:49.260 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:49.260 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:05:49.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.260 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:05:49.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:05:49.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:05:49.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:05:49.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:05:49.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:05:49.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:05:49.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:05:49.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:05:49.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:05:49.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:05:49.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:05:49.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:05:49.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:05:49.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:05:49.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:05:49.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:05:49.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:05:49.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:05:49.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:05:49.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:05:49.264 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.264 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.264 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.265 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18388 3.18 3:19e2be55:::rbd_data.115e3abd3b2c.0000000000000021:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:05:49.265 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:05:49.265 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.0125229139999998, 2024-09-14T09:05:49.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042293648000000003, 2024-09-14T09:05:49.265 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.265 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.265 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:49.265 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:49.265 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:49.265 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:49.266 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18388 2024-09-14T09:05:49.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.266 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:05:49.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:05:49.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220236+0000", 2024-09-14T09:05:49.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1339999999999998e-06 2024-09-14T09:05:49.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220244+0000", 2024-09-14T09:05:49.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1950000000000003e-06 2024-09-14T09:05:49.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220262+0000", 2024-09-14T09:05:49.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8017e-05 2024-09-14T09:05:49.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220270+0000", 2024-09-14T09:05:49.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9480000000000008e-06 2024-09-14T09:05:49.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220357+0000", 2024-09-14T09:05:49.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7185e-05 2024-09-14T09:05:49.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220576+0000", 2024-09-14T09:05:49.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021912200000000001 2024-09-14T09:05:49.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:05:49.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.222435+0000", 2024-09-14T09:05:49.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018592350000000001 2024-09-14T09:05:49.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:49.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262439+0000", 2024-09-14T09:05:49.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040003368999999997 2024-09-14T09:05:49.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:49.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262452+0000", 2024-09-14T09:05:49.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2941e-05 2024-09-14T09:05:49.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:49.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262490+0000", 2024-09-14T09:05:49.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8478e-05 2024-09-14T09:05:49.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262527+0000", 2024-09-14T09:05:49.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7024000000000003e-05 2024-09-14T09:05:49.270 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.270 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.270 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.271 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18388 3.18 e38/28 ondisk, result = 0)", 2024-09-14T09:05:49.271 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.227795+0000", 2024-09-14T09:05:49.271 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.004961271, 2024-09-14T09:05:49.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034648832999999997, 2024-09-14T09:05:49.271 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.271 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.271 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:49.271 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227795+0000", 2024-09-14T09:05:49.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227795+0000", 2024-09-14T09:05:49.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227796+0000", 2024-09-14T09:05:49.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.879999999999999e-07 2024-09-14T09:05:49.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227801+0000", 2024-09-14T09:05:49.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4909999999999999e-06 2024-09-14T09:05:49.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227815+0000", 2024-09-14T09:05:49.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4905e-05 2024-09-14T09:05:49.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227824+0000", 2024-09-14T09:05:49.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7770000000000008e-06 2024-09-14T09:05:49.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262425+0000", 2024-09-14T09:05:49.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034601109999999997 2024-09-14T09:05:49.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262434+0000", 2024-09-14T09:05:49.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5250000000000005e-06 2024-09-14T09:05:49.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262444+0000", 2024-09-14T09:05:49.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0037000000000001e-05 2024-09-14T09:05:49.275 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.275 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.275 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.275 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18762 3.2 3:45fd9b2e:::rbd_data.115e564e8eff.0000000000000003:7 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:49.275 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.073609+0000", 2024-09-14T09:05:49.275 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.159147484, 2024-09-14T09:05:49.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015012897000000001, 2024-09-14T09:05:49.275 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.275 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.275 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:49.275 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:49.275 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:49.275 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:49.275 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18762 2024-09-14T09:05:49.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.276 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073609+0000", 2024-09-14T09:05:49.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073609+0000", 2024-09-14T09:05:49.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073609+0000", 2024-09-14T09:05:49.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2300000000000001e-07 2024-09-14T09:05:49.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073612+0000", 2024-09-14T09:05:49.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4899999999999999e-06 2024-09-14T09:05:49.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073618+0000", 2024-09-14T09:05:49.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7490000000000004e-06 2024-09-14T09:05:49.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073623+0000", 2024-09-14T09:05:49.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.152e-06 2024-09-14T09:05:49.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.088449+0000", 2024-09-14T09:05:49.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014826672000000001 2024-09-14T09:05:49.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.088524+0000", 2024-09-14T09:05:49.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4259999999999997e-05 2024-09-14T09:05:49.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.088622+0000", 2024-09-14T09:05:49.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8051e-05 2024-09-14T09:05:49.279 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.279 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.279 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.279 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18763 3.2 3:45fd9b2e:::rbd_data.115e564e8eff.0000000000000003:7 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:49.279 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.073631+0000", 2024-09-14T09:05:49.279 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.1591248260000002, 2024-09-14T09:05:49.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015216986, 2024-09-14T09:05:49.279 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.279 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.280 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:49.280 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:49.280 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:49.280 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:49.280 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18763 2024-09-14T09:05:49.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.280 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073631+0000", 2024-09-14T09:05:49.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073631+0000", 2024-09-14T09:05:49.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073632+0000", 2024-09-14T09:05:49.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4400000000000001e-07 2024-09-14T09:05:49.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073634+0000", 2024-09-14T09:05:49.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3089999999999998e-06 2024-09-14T09:05:49.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073640+0000", 2024-09-14T09:05:49.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1680000000000001e-06 2024-09-14T09:05:49.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073645+0000", 2024-09-14T09:05:49.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.028e-06 2024-09-14T09:05:49.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.088653+0000", 2024-09-14T09:05:49.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01500783 2024-09-14T09:05:49.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.088730+0000", 2024-09-14T09:05:49.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6849999999999998e-05 2024-09-14T09:05:49.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.088848+0000", 2024-09-14T09:05:49.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011835700000000001 2024-09-14T09:05:49.283 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.283 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.283 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.283 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18764 3.2 3:45fd9b2e:::rbd_data.115e564e8eff.0000000000000003:7 [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:49.283 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.073652+0000", 2024-09-14T09:05:49.283 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.1591039810000003, 2024-09-14T09:05:49.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015437263, 2024-09-14T09:05:49.283 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.283 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.284 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:49.284 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:49.284 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:49.284 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:49.284 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18764 2024-09-14T09:05:49.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.284 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073652+0000", 2024-09-14T09:05:49.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073652+0000", 2024-09-14T09:05:49.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073653+0000", 2024-09-14T09:05:49.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.89e-07 2024-09-14T09:05:49.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073655+0000", 2024-09-14T09:05:49.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3489999999999999e-06 2024-09-14T09:05:49.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073661+0000", 2024-09-14T09:05:49.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8080000000000001e-06 2024-09-14T09:05:49.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073666+0000", 2024-09-14T09:05:49.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0370000000000002e-06 2024-09-14T09:05:49.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.088878+0000", 2024-09-14T09:05:49.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01521218 2024-09-14T09:05:49.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.088986+0000", 2024-09-14T09:05:49.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107425 2024-09-14T09:05:49.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.089090+0000", 2024-09-14T09:05:49.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000104075 2024-09-14T09:05:49.287 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.287 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.287 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.287 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18765 3.2 3:45fd9b2e:::rbd_data.115e564e8eff.0000000000000003:7 [sparse-read 15204352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:49.287 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.073672+0000", 2024-09-14T09:05:49.287 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.1590839219999998, 2024-09-14T09:05:49.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015686662000000001, 2024-09-14T09:05:49.287 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.287 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.287 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:49.288 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:49.288 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:49.288 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:49.288 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18765 2024-09-14T09:05:49.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.288 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073672+0000", 2024-09-14T09:05:49.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073672+0000", 2024-09-14T09:05:49.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073673+0000", 2024-09-14T09:05:49.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7000000000000004e-07 2024-09-14T09:05:49.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073675+0000", 2024-09-14T09:05:49.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.413e-06 2024-09-14T09:05:49.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073681+0000", 2024-09-14T09:05:49.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9750000000000004e-06 2024-09-14T09:05:49.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073686+0000", 2024-09-14T09:05:49.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:05:49.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.089119+0000", 2024-09-14T09:05:49.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015432934000000001 2024-09-14T09:05:49.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.089202+0000", 2024-09-14T09:05:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3380999999999993e-05 2024-09-14T09:05:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.089359+0000", 2024-09-14T09:05:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000156909 2024-09-14T09:05:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18766 3.2 3:45fd9b2e:::rbd_data.115e564e8eff.0000000000000003:7 [sparse-read 15466496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.073715+0000", 2024-09-14T09:05:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.1590415089999997, 2024-09-14T09:05:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015828563, 2024-09-14T09:05:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18766 2024-09-14T09:05:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073715+0000", 2024-09-14T09:05:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073715+0000", 2024-09-14T09:05:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073715+0000", 2024-09-14T09:05:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1399999999999997e-07 2024-09-14T09:05:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073718+0000", 2024-09-14T09:05:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.666e-06 2024-09-14T09:05:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073725+0000", 2024-09-14T09:05:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4629999999999998e-06 2024-09-14T09:05:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073732+0000", 2024-09-14T09:05:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.024e-06 2024-09-14T09:05:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.089396+0000", 2024-09-14T09:05:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015664563999999999 2024-09-14T09:05:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.089464+0000", 2024-09-14T09:05:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7837999999999997e-05 2024-09-14T09:05:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.089543+0000", 2024-09-14T09:05:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9394000000000006e-05 2024-09-14T09:05:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18767 3.2 3:45fd9b2e:::rbd_data.115e564e8eff.0000000000000003:7 [sparse-read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.073740+0000", 2024-09-14T09:05:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.1590168140000001, 2024-09-14T09:05:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015982881000000001, 2024-09-14T09:05:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18767 2024-09-14T09:05:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073740+0000", 2024-09-14T09:05:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073740+0000", 2024-09-14T09:05:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073740+0000", 2024-09-14T09:05:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1399999999999997e-07 2024-09-14T09:05:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073742+0000", 2024-09-14T09:05:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2120000000000002e-06 2024-09-14T09:05:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073749+0000", 2024-09-14T09:05:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2650000000000002e-06 2024-09-14T09:05:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073754+0000", 2024-09-14T09:05:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9450000000000001e-06 2024-09-14T09:05:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.089572+0000", 2024-09-14T09:05:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015818189999999999 2024-09-14T09:05:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.089643+0000", 2024-09-14T09:05:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1198999999999994e-05 2024-09-14T09:05:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.089722+0000", 2024-09-14T09:05:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9456000000000006e-05 2024-09-14T09:05:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18768 3.2 3:45fd9b2e:::rbd_data.115e564e8eff.0000000000000003:7 [sparse-read 15990784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.073760+0000", 2024-09-14T09:05:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.1589964320000004, 2024-09-14T09:05:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016145425000000001, 2024-09-14T09:05:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18768 2024-09-14T09:05:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073760+0000", 2024-09-14T09:05:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073760+0000", 2024-09-14T09:05:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073760+0000", 2024-09-14T09:05:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6899999999999998e-07 2024-09-14T09:05:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073763+0000", 2024-09-14T09:05:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4279999999999999e-06 2024-09-14T09:05:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073769+0000", 2024-09-14T09:05:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0549999999999996e-06 2024-09-14T09:05:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073774+0000", 2024-09-14T09:05:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9609999999999996e-06 2024-09-14T09:05:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.089748+0000", 2024-09-14T09:05:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015974214 2024-09-14T09:05:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.089827+0000", 2024-09-14T09:05:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8713000000000003e-05 2024-09-14T09:05:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.089905+0000", 2024-09-14T09:05:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8684999999999996e-05 2024-09-14T09:05:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18769 3.2 3:45fd9b2e:::rbd_data.115e564e8eff.0000000000000003:7 [sparse-read 16252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.073780+0000", 2024-09-14T09:05:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.15897615, 2024-09-14T09:05:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019189386999999999, 2024-09-14T09:05:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18769 2024-09-14T09:05:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073780+0000", 2024-09-14T09:05:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073780+0000", 2024-09-14T09:05:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073781+0000", 2024-09-14T09:05:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.96e-07 2024-09-14T09:05:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073783+0000", 2024-09-14T09:05:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0779999999999998e-06 2024-09-14T09:05:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073788+0000", 2024-09-14T09:05:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6400000000000002e-06 2024-09-14T09:05:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073793+0000", 2024-09-14T09:05:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1880000000000001e-06 2024-09-14T09:05:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.089932+0000", 2024-09-14T09:05:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016138144 2024-09-14T09:05:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.090002+0000", 2024-09-14T09:05:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9974999999999993e-05 2024-09-14T09:05:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.092970+0000", 2024-09-14T09:05:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0029679659999999998 2024-09-14T09:05:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18770 3.2 3:45fd9b2e:::rbd_data.115e564e8eff.0000000000000003:7 [sparse-read 16515072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.073800+0000", 2024-09-14T09:05:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.1589563390000004, 2024-09-14T09:05:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016728957999999999, 2024-09-14T09:05:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18770 2024-09-14T09:05:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073800+0000", 2024-09-14T09:05:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073800+0000", 2024-09-14T09:05:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073800+0000", 2024-09-14T09:05:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9900000000000001e-07 2024-09-14T09:05:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073806+0000", 2024-09-14T09:05:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.31e-06 2024-09-14T09:05:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073812+0000", 2024-09-14T09:05:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1469999999999998e-06 2024-09-14T09:05:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.073817+0000", 2024-09-14T09:05:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7129999999999996e-06 2024-09-14T09:05:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.090136+0000", 2024-09-14T09:05:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016319799999999999 2024-09-14T09:05:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.090425+0000", 2024-09-14T09:05:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028861799999999998 2024-09-14T09:05:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.090529+0000", 2024-09-14T09:05:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000103971 2024-09-14T09:05:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:19543 3.5 e39/23 3:a536ff9b:::rbd_data.115e4c71d6e8.0000000000000002:head v 39'8, mlcod=39'6)", 2024-09-14T09:05:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:05:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.7096682540000003, 2024-09-14T09:05:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.039280019999999999, 2024-09-14T09:05:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:05:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:05:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523090+0000", 2024-09-14T09:05:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.773e-06 2024-09-14T09:05:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537290+0000", 2024-09-14T09:05:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014200007000000001 2024-09-14T09:05:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537317+0000", 2024-09-14T09:05:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6704999999999999e-05 2024-09-14T09:05:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537328+0000", 2024-09-14T09:05:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1299e-05 2024-09-14T09:05:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537417+0000", 2024-09-14T09:05:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8869999999999994e-05 2024-09-14T09:05:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537445+0000", 2024-09-14T09:05:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8377999999999998e-05 2024-09-14T09:05:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562324+0000", 2024-09-14T09:05:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024879307999999999 2024-09-14T09:05:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562368+0000", 2024-09-14T09:05:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3680000000000002e-05 2024-09-14T09:05:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20736 3.2 3:45030d14:::rbd_data.115e889d4809.000000000000027a:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.099577+0000", 2024-09-14T09:05:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 2.1331792109999999, 2024-09-14T09:05:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026646762000000001, 2024-09-14T09:05:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20736 2024-09-14T09:05:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099577+0000", 2024-09-14T09:05:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099577+0000", 2024-09-14T09:05:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099579+0000", 2024-09-14T09:05:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5009999999999999e-06 2024-09-14T09:05:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099585+0000", 2024-09-14T09:05:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2870000000000001e-06 2024-09-14T09:05:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099602+0000", 2024-09-14T09:05:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7176e-05 2024-09-14T09:05:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099611+0000", 2024-09-14T09:05:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7010000000000002e-06 2024-09-14T09:05:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099664+0000", 2024-09-14T09:05:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3183000000000002e-05 2024-09-14T09:05:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099793+0000", 2024-09-14T09:05:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012945099999999999 2024-09-14T09:05:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:05:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.100819+0000", 2024-09-14T09:05:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010252149999999999 2024-09-14T09:05:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.125929+0000", 2024-09-14T09:05:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025110072000000001 2024-09-14T09:05:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.126034+0000", 2024-09-14T09:05:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105655 2024-09-14T09:05:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.126141+0000", 2024-09-14T09:05:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107069 2024-09-14T09:05:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.126224+0000", 2024-09-14T09:05:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2452000000000003e-05 2024-09-14T09:05:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:20736 3.2 e41/23 ondisk, result = 0)", 2024-09-14T09:05:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.104726+0000", 2024-09-14T09:05:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 2.128030613, 2024-09-14T09:05:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021513264000000001, 2024-09-14T09:05:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.104726+0000", 2024-09-14T09:05:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.104726+0000", 2024-09-14T09:05:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.104727+0000", 2024-09-14T09:05:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3099999999999996e-07 2024-09-14T09:05:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.104731+0000", 2024-09-14T09:05:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8339999999999998e-06 2024-09-14T09:05:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.104747+0000", 2024-09-14T09:05:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5294e-05 2024-09-14T09:05:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.104755+0000", 2024-09-14T09:05:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1640000000000001e-06 2024-09-14T09:05:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.126005+0000", 2024-09-14T09:05:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021250331000000001 2024-09-14T09:05:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.126022+0000", 2024-09-14T09:05:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7203000000000001e-05 2024-09-14T09:05:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.126239+0000", 2024-09-14T09:05:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021650700000000001 2024-09-14T09:05:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:49.327 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:49.329 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:05:49.329 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:05:49.330 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:05:49.330 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:49.332+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:49.337 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:05:49.337 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:05:49.353 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:05:49.353 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:05:49.370 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:05:49.371 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:05:49.391 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:05:49.391 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:05:49.395 INFO:teuthology.orchestra.run.smithi176.stdout:moved 0 / 82 (0%) 2024-09-14T09:05:49.395 INFO:teuthology.orchestra.run.smithi176.stdout:avg 10.25 2024-09-14T09:05:49.395 INFO:teuthology.orchestra.run.smithi176.stdout:stddev 4.4088 -> 4.4088 (expected baseline 2.99479) 2024-09-14T09:05:49.395 INFO:teuthology.orchestra.run.smithi176.stdout:min osd.4 with 2 -> 2 pgs (0.195122 -> 0.195122 * mean) 2024-09-14T09:05:49.395 INFO:teuthology.orchestra.run.smithi176.stdout:max osd.1 with 16 -> 16 pgs (1.56098 -> 1.56098 * mean) 2024-09-14T09:05:49.395 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:05:49.396 INFO:teuthology.orchestra.run.smithi176.stdout:oload 1000 2024-09-14T09:05:49.396 INFO:teuthology.orchestra.run.smithi176.stdout:max_change 1 2024-09-14T09:05:49.396 INFO:teuthology.orchestra.run.smithi176.stdout:max_change_osds 4 2024-09-14T09:05:49.396 INFO:teuthology.orchestra.run.smithi176.stdout:average_utilization 0.0003 2024-09-14T09:05:49.396 INFO:teuthology.orchestra.run.smithi176.stdout:overload_utilization 0.0034 2024-09-14T09:05:49.397 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 weight 0.1582 -> 0.1590 2024-09-14T09:05:49.397 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 weight 0.7646 -> 0.7656 2024-09-14T09:05:49.397 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-09-14T09:05:49.396+0000 7f321da70640 -1 mgr.server reply reply (2) No such file or directory no change 2024-09-14T09:05:49.397 INFO:teuthology.orchestra.run.smithi176.stderr:no change 2024-09-14T09:05:49.411 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:05:49.411 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:05:49.413 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-14T09:05:49.414 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2024-09-14T09:05:49.432 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:05:49.432 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:05:49.432 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:49.428+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:49.452 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:05:49.453 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:05:49.461 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:49.461 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:49.475 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:05:49.530 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:49.532+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:49.593 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:49.593 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:49.608 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:05:49.631 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:49.628+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:49.732 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:49.728+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:49.775 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:49.775 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:49.789 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:05:49.833 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:49.828+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:49.908 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:49.908 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:49.922 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:05:49.967 INFO:teuthology.orchestra.run.smithi176.stdout:moved 0 / 82 (0%) 2024-09-14T09:05:49.967 INFO:teuthology.orchestra.run.smithi176.stdout:avg 10.25 2024-09-14T09:05:49.967 INFO:teuthology.orchestra.run.smithi176.stdout:stddev 4.4088 -> 4.4088 (expected baseline 2.99479) 2024-09-14T09:05:49.967 INFO:teuthology.orchestra.run.smithi176.stdout:min osd.4 with 2 -> 2 pgs (0.195122 -> 0.195122 * mean) 2024-09-14T09:05:49.967 INFO:teuthology.orchestra.run.smithi176.stdout:max osd.1 with 16 -> 16 pgs (1.56098 -> 1.56098 * mean) 2024-09-14T09:05:49.967 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:05:49.967 INFO:teuthology.orchestra.run.smithi176.stdout:oload 1000 2024-09-14T09:05:49.967 INFO:teuthology.orchestra.run.smithi176.stdout:max_change 1 2024-09-14T09:05:49.968 INFO:teuthology.orchestra.run.smithi176.stdout:max_change_osds 4 2024-09-14T09:05:49.968 INFO:teuthology.orchestra.run.smithi176.stdout:average_utilization 0.0003 2024-09-14T09:05:49.968 INFO:teuthology.orchestra.run.smithi176.stdout:overload_utilization 0.0034 2024-09-14T09:05:49.968 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 weight 0.1582 -> 0.1590 2024-09-14T09:05:49.968 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 weight 0.7646 -> 0.7656 2024-09-14T09:05:49.968 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:49.972+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:49.974 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is set 2024-09-14T09:05:49.997 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-14T09:05:49.997 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-14T09:05:50.043 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:50.040+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:50.062 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:50.062 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:50.079 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:05:50.127 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone8 from 0xed616fc (overlap 0x1f3956) to 0x31f55 2024-09-14T09:05:50.164 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:50.168+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:50.237 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:50.232+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:50.257 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:50.257 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:50.273 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:05:50.364 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:50.368+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:50.430 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-14T09:05:50.439 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:50.440+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:50.445 INFO:teuthology.orchestra.run.smithi176.stdout:850 punch from 0x9ae5db0 to 0x9ae7331, (0x1581 bytes) 2024-09-14T09:05:50.447 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:50.447 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:50.456 INFO:teuthology.orchestra.run.smithi176.stdout:853 read 0xdaebfe1 thru 0xdaec553 (0x573 bytes) 2024-09-14T09:05:50.456 INFO:teuthology.orchestra.run.smithi176.stdout:855 punch from 0x5a78cf1 to 0x5a7952b, (0x83a bytes) 2024-09-14T09:05:50.456 INFO:teuthology.orchestra.run.smithi176.stdout:857 write 0xd4022d0 thru 0xd4105ae (0xe2df bytes) 2024-09-14T09:05:50.464 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:05:50.482 INFO:teuthology.orchestra.run.smithi176.stdout:860 writesame 0xd040975 thru 0xd04faf4 data_size 0x8a(0xf180 bytes) 2024-09-14T09:05:50.490 INFO:teuthology.orchestra.run.smithi176.stdout:862 read 0xd2afe13 thru 0xd2bcc3b (0xce29 bytes) 2024-09-14T09:05:50.492 INFO:teuthology.orchestra.run.smithi176.stdout:863 read 0xd7a3960 thru 0xd7afae7 (0xc188 bytes) 2024-09-14T09:05:50.494 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:50.494 INFO:teuthology.orchestra.run.smithi176.stdout:866 read 0x53cb9d7 thru 0x53d194a (0x5f74 bytes) 2024-09-14T09:05:50.512 INFO:teuthology.orchestra.run.smithi176.stdout:867 trunc from 0xed616fc to 0x6c50dd8 2024-09-14T09:05:50.556 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:50.559+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:50.558 INFO:teuthology.orchestra.run.smithi176.stdout:868 read 0x4e85f98 thru 0x4e95798 (0xf801 bytes) 2024-09-14T09:05:50.558 INFO:teuthology.orchestra.run.smithi176.stdout:869 punch from 0x2659241 to 0x26645d0, (0xb38f bytes) 2024-09-14T09:05:50.558 INFO:teuthology.orchestra.run.smithi176.stdout:870 read 0x69ad383 thru 0x69b53bd (0x803b bytes) 2024-09-14T09:05:50.558 INFO:teuthology.orchestra.run.smithi176.stdout:873 read 0x28cdb thru 0x2dc8e (0x4fb4 bytes) 2024-09-14T09:05:50.585 INFO:teuthology.orchestra.run.smithi176.stdout:874 write 0xbd16adb thru 0xbd176ef (0xc15 bytes) 2024-09-14T09:05:50.593 INFO:teuthology.orchestra.run.smithi176.stdout:875 write 0x752f220 thru 0x75379a9 (0x878a bytes) 2024-09-14T09:05:50.612 INFO:teuthology.orchestra.run.smithi176.stdout:876 trunc from 0xbd176f0 to 0x6803921 2024-09-14T09:05:50.641 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:50.636+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:50.647 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:50.647 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:50.662 INFO:teuthology.orchestra.run.smithi176.stdout:877 punch from 0x1f20ca8 to 0x1f2cedf, (0xc237 bytes) 2024-09-14T09:05:50.662 INFO:teuthology.orchestra.run.smithi176.stdout:878 read 0x56329b7 thru 0x5634c40 (0x228a bytes) 2024-09-14T09:05:50.662 INFO:teuthology.orchestra.run.smithi176.stdout:879 read 0x1f9a995 thru 0x1f9b782 (0xdee bytes) 2024-09-14T09:05:50.662 INFO:teuthology.orchestra.run.smithi176.stdout:880 write 0xfce40c thru 0xfd1d56 (0x394b bytes) 2024-09-14T09:05:50.664 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:05:50.687 INFO:teuthology.orchestra.run.smithi176.stdout:881 trunc from 0x6803921 to 0x98c0f07 2024-09-14T09:05:50.714 INFO:teuthology.orchestra.run.smithi176.stdout:883 write 0xb0f40fd thru 0xb0ffb23 (0xba27 bytes) 2024-09-14T09:05:50.741 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:50.736+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:50.743 INFO:teuthology.orchestra.run.smithi176.stdout:884 read 0xa156f97 thru 0xa15f9c2 (0x8a2c bytes) 2024-09-14T09:05:50.743 INFO:teuthology.orchestra.run.smithi176.stdout:887 read 0xb0b4eb2 thru 0xb0bbc7a (0x6dc9 bytes) 2024-09-14T09:05:50.745 INFO:teuthology.orchestra.run.smithi176.stdout:888 punch from 0x677ed89 to 0x6786536, (0x77ad bytes) 2024-09-14T09:05:50.745 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:50.745 INFO:teuthology.orchestra.run.smithi176.stdout:890 writesame 0x8181c6a thru 0x818450e data_size 0x1(0x28a5 bytes) 2024-09-14T09:05:50.767 INFO:teuthology.orchestra.run.smithi176.stdout:891 trunc from 0xb0ffb24 to 0x763ce5f 2024-09-14T09:05:50.814 INFO:teuthology.orchestra.run.smithi176.stdout:894 writesame 0xbc4735e thru 0xbc4f997 data_size 0x8a(0x863a bytes) 2024-09-14T09:05:50.815 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:50.815 INFO:teuthology.orchestra.run.smithi176.stdout:896 trunc from 0xbc4f998 to 0xb75dda9 2024-09-14T09:05:50.835 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:50.835 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:50.842 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:50.843+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:50.850 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:05:50.852 INFO:teuthology.orchestra.run.smithi176.stdout:897 trunc from 0xb75dda9 to 0x11b8b20 2024-09-14T09:05:50.907 INFO:teuthology.orchestra.run.smithi176.stdout:898 read 0xcd9dcf thru 0xce68f9 (0xcb2b bytes) 2024-09-14T09:05:50.909 INFO:teuthology.orchestra.run.smithi176.stdout:899 read 0xcd42af thru 0xcdbf3a (0x7c8c bytes) 2024-09-14T09:05:50.911 INFO:teuthology.orchestra.run.smithi176.stdout:900 read 0x672d7e thru 0x67b9aa (0x8c2d bytes) 2024-09-14T09:05:50.911 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:50.911 INFO:teuthology.orchestra.run.smithi176.stdout:902 read 0x3f05ac thru 0x3f1aee (0x1543 bytes) 2024-09-14T09:05:50.911 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:50.939 INFO:teuthology.orchestra.run.smithi176.stdout:905 write 0x7c6b689 thru 0x7c780e4 (0xca5c bytes) 2024-09-14T09:05:50.942 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:50.940+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:50.961 INFO:teuthology.orchestra.run.smithi176.stdout:906 punch from 0x47dae21 to 0x47e6eec, (0xc0cb bytes) 2024-09-14T09:05:50.961 INFO:teuthology.orchestra.run.smithi176.stdout:907 read 0x258d4e5 thru 0x259cc52 (0xf76e bytes) 2024-09-14T09:05:50.970 INFO:teuthology.orchestra.run.smithi176.stdout:moved 0 / 82 (0%) 2024-09-14T09:05:50.970 INFO:teuthology.orchestra.run.smithi176.stdout:avg 10.25 2024-09-14T09:05:50.970 INFO:teuthology.orchestra.run.smithi176.stdout:stddev 4.4088 -> 4.4088 (expected baseline 2.99479) 2024-09-14T09:05:50.970 INFO:teuthology.orchestra.run.smithi176.stdout:min osd.4 with 2 -> 2 pgs (0.195122 -> 0.195122 * mean) 2024-09-14T09:05:50.970 INFO:teuthology.orchestra.run.smithi176.stdout:max osd.1 with 16 -> 16 pgs (1.56098 -> 1.56098 * mean) 2024-09-14T09:05:50.970 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:05:50.970 INFO:teuthology.orchestra.run.smithi176.stdout:oload 110 2024-09-14T09:05:50.970 INFO:teuthology.orchestra.run.smithi176.stdout:max_change 3 2024-09-14T09:05:50.970 INFO:teuthology.orchestra.run.smithi176.stdout:max_change_osds 4 2024-09-14T09:05:50.970 INFO:teuthology.orchestra.run.smithi176.stdout:average_utilization 0.0003 2024-09-14T09:05:50.971 INFO:teuthology.orchestra.run.smithi176.stdout:overload_utilization 0.0004 2024-09-14T09:05:50.971 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 weight 0.1590 -> 0.1599 2024-09-14T09:05:50.971 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 weight 0.7656 -> 0.7667 2024-09-14T09:05:50.983 INFO:teuthology.orchestra.run.smithi176.stdout:908 write 0xaaaf71a thru 0xaabb483 (0xbd6a bytes) 2024-09-14T09:05:51.001 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-14T09:05:51.001 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-14T09:05:51.002 INFO:teuthology.orchestra.run.smithi176.stdout:909 writesame 0x1a402e5 thru 0x1a4675b data_size 0x1(0x6477 bytes) 2024-09-14T09:05:51.020 INFO:teuthology.orchestra.run.smithi176.stdout:910 read 0x16bb6eb thru 0x16bbda4 (0x6ba bytes) 2024-09-14T09:05:51.029 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:51.029 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:51.034 INFO:teuthology.orchestra.run.smithi176.stdout:913 write 0xc3907be thru 0xc395d0d (0x5550 bytes) 2024-09-14T09:05:51.043 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:51.043+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:51.046 INFO:teuthology.orchestra.run.smithi176.stdout:914 trunc from 0xc395d0e to 0xce7e790 2024-09-14T09:05:51.047 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:05:51.053 INFO:teuthology.orchestra.run.smithi176.stdout:915 punch from 0x79cd0b9 to 0x79d91a3, (0xc0ea bytes) 2024-09-14T09:05:51.053 INFO:teuthology.orchestra.run.smithi176.stdout:917 writesame 0xb845929 thru 0xb850b0d data_size 0x1(0xb1e5 bytes) 2024-09-14T09:05:51.073 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:51.074 INFO:teuthology.orchestra.run.smithi176.stdout:921 write 0x254416a thru 0x254af28 (0x6dbf bytes) 2024-09-14T09:05:51.089 INFO:teuthology.orchestra.run.smithi176.stdout:922 writesame 0x11bf125 thru 0x11c44c0 data_size 0x2(0x539c bytes) 2024-09-14T09:05:51.096 INFO:teuthology.orchestra.run.smithi176.stdout:923 write 0x7e980a6 thru 0x7e98af0 (0xa4b bytes) 2024-09-14T09:05:51.098 INFO:teuthology.orchestra.run.smithi176.stdout:924 write 0x390a710 thru 0x3919001 (0xe8f2 bytes) 2024-09-14T09:05:51.115 INFO:teuthology.orchestra.run.smithi176.stdout:925 read 0xa2d62cf thru 0xa2daa12 (0x4744 bytes) 2024-09-14T09:05:51.115 INFO:teuthology.orchestra.run.smithi176.stdout:926 write 0xb5cfb32 thru 0xb5d3653 (0x3b22 bytes) 2024-09-14T09:05:51.117 INFO:teuthology.orchestra.run.smithi176.stdout:927 read 0x2182b6c thru 0x218e3ef (0xb884 bytes) 2024-09-14T09:05:51.117 INFO:teuthology.orchestra.run.smithi176.stdout:928 write 0x73d831a thru 0x73e4a77 (0xc75e bytes) 2024-09-14T09:05:51.136 INFO:teuthology.orchestra.run.smithi176.stdout:929 flatten 2024-09-14T09:05:51.150 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:51.148+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:51.208 INFO:teuthology.orchestra.run.smithi176.stdout:931 write 0x8ac764d thru 0x8ad3524 (0xbed8 bytes) 2024-09-14T09:05:51.221 INFO:teuthology.orchestra.run.smithi176.stdout:932 write 0x5e3cbd thru 0x5e3fc5 (0x309 bytes) 2024-09-14T09:05:51.228 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:51.228 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:51.234 INFO:teuthology.orchestra.run.smithi176.stdout:933 trunc from 0xce7e790 to 0x4c1e739 2024-09-14T09:05:51.244 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:51.247+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:51.245 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:05:51.260 INFO:teuthology.orchestra.run.smithi176.stdout:934 read 0x270f3d1 thru 0x271130c (0x1f3c bytes) 2024-09-14T09:05:51.281 INFO:teuthology.orchestra.run.smithi176.stdout:935 write 0xe54f02d thru 0xe551daf (0x2d83 bytes) 2024-09-14T09:05:51.286 INFO:teuthology.orchestra.run.smithi176.stdout:936 writesame 0x8a65df4 thru 0x8a6be02 data_size 0x7(0x600f bytes) 2024-09-14T09:05:51.294 INFO:teuthology.orchestra.run.smithi176.stdout:938 write 0x1526111 thru 0x15291bc (0x30ac bytes) 2024-09-14T09:05:51.296 INFO:teuthology.orchestra.run.smithi176.stdout:939 write 0x47d6013 thru 0x47ded79 (0x8d67 bytes) 2024-09-14T09:05:51.306 INFO:teuthology.orchestra.run.smithi176.stdout:940 read 0x7dfa926 thru 0x7e02f8d (0x8668 bytes) 2024-09-14T09:05:51.306 INFO:teuthology.orchestra.run.smithi176.stdout:941 read 0x229a02a thru 0x22a8520 (0xe4f7 bytes) 2024-09-14T09:05:51.307 INFO:teuthology.orchestra.run.smithi176.stdout:942 writesame 0x6183cf7 thru 0x618ca46 data_size 0x88(0x8d50 bytes) 2024-09-14T09:05:51.320 INFO:teuthology.orchestra.run.smithi176.stdout:943 write 0xe37ebd6 thru 0xe38c6ef (0xdb1a bytes) 2024-09-14T09:05:51.336 INFO:teuthology.orchestra.run.smithi176.stdout:944 punch from 0x4e0ea6c to 0x4e1dba2, (0xf136 bytes) 2024-09-14T09:05:51.336 INFO:teuthology.orchestra.run.smithi176.stdout:945 writesame 0xada9c37 thru 0xadac56e data_size 0x4(0x2938 bytes) 2024-09-14T09:05:51.345 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:51.340+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:51.357 INFO:teuthology.orchestra.run.smithi176.stdout:946 trunc from 0xe551db0 to 0x66210c6 2024-09-14T09:05:51.396 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-09-14T09:05:51.399+0000 7f321da70640 -1 mgr.server reply reply (2) No such file or directory no change 2024-09-14T09:05:51.396 INFO:teuthology.orchestra.run.smithi176.stdout:moved 0 / 82 (0%) 2024-09-14T09:05:51.396 INFO:teuthology.orchestra.run.smithi176.stdout:avg 10.25 2024-09-14T09:05:51.397 INFO:teuthology.orchestra.run.smithi176.stdout:stddev 4.4088 -> 4.4088 (expected baseline 2.99479) 2024-09-14T09:05:51.397 INFO:teuthology.orchestra.run.smithi176.stdout:min osd.4 with 2 -> 2 pgs (0.195122 -> 0.195122 * mean) 2024-09-14T09:05:51.397 INFO:teuthology.orchestra.run.smithi176.stdout:max osd.1 with 16 -> 16 pgs (1.56098 -> 1.56098 * mean) 2024-09-14T09:05:51.397 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:05:51.397 INFO:teuthology.orchestra.run.smithi176.stdout:oload 110 2024-09-14T09:05:51.397 INFO:teuthology.orchestra.run.smithi176.stdout:max_change 0.05 2024-09-14T09:05:51.397 INFO:teuthology.orchestra.run.smithi176.stdout:max_change_osds 4 2024-09-14T09:05:51.397 INFO:teuthology.orchestra.run.smithi176.stdout:average_utilization 0.0003 2024-09-14T09:05:51.397 INFO:teuthology.orchestra.run.smithi176.stdout:overload_utilization 0.0004 2024-09-14T09:05:51.397 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 weight 0.1599 -> 0.1608 2024-09-14T09:05:51.397 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 weight 0.7667 -> 0.7677 2024-09-14T09:05:51.397 INFO:teuthology.orchestra.run.smithi176.stderr:no change 2024-09-14T09:05:51.402 INFO:teuthology.orchestra.run.smithi176.stdout:947 write 0xb759245 thru 0xb75f4e3 (0x629f bytes) 2024-09-14T09:05:51.419 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-14T09:05:51.419 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-14T09:05:51.420 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:51.420 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:51.426 INFO:teuthology.orchestra.run.smithi176.stdout:950 write 0xe8bdf32 thru 0xe8c7ed7 (0x9fa6 bytes) 2024-09-14T09:05:51.435 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:05:51.447 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:51.447+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:51.450 INFO:teuthology.orchestra.run.smithi176.stdout:951 trunc from 0xe8c7ed8 to 0xb100731 2024-09-14T09:05:51.467 INFO:teuthology.orchestra.run.smithi176.stdout:952 punch from 0x579bfb2 to 0x57a49e9, (0x8a37 bytes) 2024-09-14T09:05:51.467 INFO:teuthology.orchestra.run.smithi176.stdout:953 punch from 0x1729efe to 0x173575d, (0xb85f bytes) 2024-09-14T09:05:51.467 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:51.467 INFO:teuthology.orchestra.run.smithi176.stdout:955 read 0x77729b thru 0x78693f (0xf6a5 bytes) 2024-09-14T09:05:51.468 INFO:teuthology.orchestra.run.smithi176.stdout:956 read 0x63a3d89 thru 0x63af058 (0xb2d0 bytes) 2024-09-14T09:05:51.470 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:05:51.480 INFO:teuthology.orchestra.run.smithi176.stdout:957 trunc from 0xb100731 to 0x3289f93 2024-09-14T09:05:51.531 INFO:teuthology.orchestra.run.smithi176.stdout:959 write 0x8396480 thru 0x83a4419 (0xdf9a bytes) 2024-09-14T09:05:51.547 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:51.544+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:51.562 INFO:teuthology.orchestra.run.smithi176.stdout:960 punch from 0x73a3f44 to 0x73aa5c5, (0x6681 bytes) 2024-09-14T09:05:51.572 INFO:teuthology.orchestra.run.smithi176.stdout:962 trunc from 0x83a441a to 0x16ef0f9 2024-09-14T09:05:51.597 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:51.597 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:51.616 INFO:teuthology.orchestra.run.smithi176.stdout:963 read 0x374971 thru 0x376672 (0x1d02 bytes) 2024-09-14T09:05:51.619 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:05:51.635 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:05:51.636 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:05:51.644 INFO:teuthology.orchestra.run.smithi176.stdout:964 write 0xec7c24a thru 0xec890d7 (0xce8e bytes) 2024-09-14T09:05:51.648 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:51.644+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:51.654 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:05:51.655 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:05:51.687 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:05:51.688 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:05:51.688 INFO:teuthology.orchestra.run.smithi176.stdout:965 trunc from 0xec890d8 to 0x6ec91f0 2024-09-14T09:05:51.710 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:05:51.710 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:05:51.710 INFO:teuthology.orchestra.run.smithi176.stdout:967 read 0x4d19911 thru 0x4d1ae2a (0x151a bytes) 2024-09-14T09:05:51.711 INFO:teuthology.orchestra.run.smithi176.stdout:970 punch from 0x10847fe to 0x1087cd0, (0x34d2 bytes) 2024-09-14T09:05:51.711 INFO:teuthology.orchestra.run.smithi176.stdout:971 read 0x477a191 thru 0x478250a (0x837a bytes) 2024-09-14T09:05:51.711 INFO:teuthology.orchestra.run.smithi176.stdout:972 clone 10 order 19 su 2048 sc 7 2024-09-14T09:05:51.730 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:05:51.730 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:05:51.749 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:51.744+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:51.751 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:05:51.751 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:05:51.769 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:51.769 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:51.769 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:51.769 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:51.769 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:51.771 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:05:51.771 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:05:51.780 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-09-14T09:05:51.783+0000 7f321da70640 -1 mgr.server reply reply (2) No such file or directory no change 2024-09-14T09:05:51.781 INFO:teuthology.orchestra.run.smithi176.stdout:moved 0 / 82 (0%) 2024-09-14T09:05:51.781 INFO:teuthology.orchestra.run.smithi176.stdout:avg 10.25 2024-09-14T09:05:51.781 INFO:teuthology.orchestra.run.smithi176.stdout:stddev 4.4088 -> 4.4088 (expected baseline 2.99479) 2024-09-14T09:05:51.781 INFO:teuthology.orchestra.run.smithi176.stdout:min osd.4 with 2 -> 2 pgs (0.195122 -> 0.195122 * mean) 2024-09-14T09:05:51.781 INFO:teuthology.orchestra.run.smithi176.stdout:max osd.1 with 16 -> 16 pgs (1.56098 -> 1.56098 * mean) 2024-09-14T09:05:51.781 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:05:51.781 INFO:teuthology.orchestra.run.smithi176.stdout:oload 110 2024-09-14T09:05:51.781 INFO:teuthology.orchestra.run.smithi176.stdout:max_change 1 2024-09-14T09:05:51.781 INFO:teuthology.orchestra.run.smithi176.stdout:max_change_osds 4 2024-09-14T09:05:51.781 INFO:teuthology.orchestra.run.smithi176.stdout:average_utilization 0.0003 2024-09-14T09:05:51.781 INFO:teuthology.orchestra.run.smithi176.stdout:overload_utilization 0.0004 2024-09-14T09:05:51.782 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 weight 0.1599 -> 0.1608 2024-09-14T09:05:51.782 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 weight 0.7667 -> 0.7677 2024-09-14T09:05:51.782 INFO:teuthology.orchestra.run.smithi176.stderr:no change 2024-09-14T09:05:51.783 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:05:51.792 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:05:51.793 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:05:51.858 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:51.856+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:51.911 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18756 3.12 3.81006c52 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.076908+0000", 2024-09-14T09:05:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.8328466930000005, 2024-09-14T09:05:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0089070020000000007, 2024-09-14T09:05:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18756 2024-09-14T09:05:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076908+0000", 2024-09-14T09:05:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076908+0000", 2024-09-14T09:05:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076909+0000", 2024-09-14T09:05:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4799999999999998e-07 2024-09-14T09:05:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076911+0000", 2024-09-14T09:05:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.875e-06 2024-09-14T09:05:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076928+0000", 2024-09-14T09:05:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6557000000000001e-05 2024-09-14T09:05:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076935+0000", 2024-09-14T09:05:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0260000000000002e-06 2024-09-14T09:05:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.085805+0000", 2024-09-14T09:05:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088700570000000006 2024-09-14T09:05:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:51.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.085815+0000", 2024-09-14T09:05:51.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8390000000000002e-06 2024-09-14T09:05:51.916 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.916 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:51.916 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.916 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18757 3.12 3.81006c52 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:51.916 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.076946+0000", 2024-09-14T09:05:51.916 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.8328084819999999, 2024-09-14T09:05:51.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0089136730000000004, 2024-09-14T09:05:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18757 2024-09-14T09:05:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076946+0000", 2024-09-14T09:05:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076946+0000", 2024-09-14T09:05:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076947+0000", 2024-09-14T09:05:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.68e-07 2024-09-14T09:05:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076950+0000", 2024-09-14T09:05:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9179999999999998e-06 2024-09-14T09:05:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076959+0000", 2024-09-14T09:05:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9530000000000005e-06 2024-09-14T09:05:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076965+0000", 2024-09-14T09:05:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7449999999999998e-06 2024-09-14T09:05:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.085850+0000", 2024-09-14T09:05:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088847969999999998 2024-09-14T09:05:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.085860+0000", 2024-09-14T09:05:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6919999999999997e-06 2024-09-14T09:05:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18758 3.12 3.81006c52 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.076975+0000", 2024-09-14T09:05:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.8327793449999996, 2024-09-14T09:05:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0089275369999999993, 2024-09-14T09:05:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18758 2024-09-14T09:05:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076975+0000", 2024-09-14T09:05:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076975+0000", 2024-09-14T09:05:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076976+0000", 2024-09-14T09:05:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7899999999999998e-07 2024-09-14T09:05:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076979+0000", 2024-09-14T09:05:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7300000000000001e-06 2024-09-14T09:05:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076987+0000", 2024-09-14T09:05:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4980000000000006e-06 2024-09-14T09:05:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.076993+0000", 2024-09-14T09:05:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4160000000000001e-06 2024-09-14T09:05:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.085890+0000", 2024-09-14T09:05:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088969119999999999 2024-09-14T09:05:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.085903+0000", 2024-09-14T09:05:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2401999999999999e-05 2024-09-14T09:05:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18779 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 14024704~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087321+0000", 2024-09-14T09:05:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.8224339089999999, 2024-09-14T09:05:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0090439219999999994, 2024-09-14T09:05:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18779 2024-09-14T09:05:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087321+0000", 2024-09-14T09:05:51.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:51.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087321+0000", 2024-09-14T09:05:51.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:51.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087321+0000", 2024-09-14T09:05:51.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5299999999999999e-07 2024-09-14T09:05:51.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:51.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087329+0000", 2024-09-14T09:05:51.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2640000000000008e-06 2024-09-14T09:05:51.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:51.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087336+0000", 2024-09-14T09:05:51.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4389999999999997e-06 2024-09-14T09:05:51.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:51.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087340+0000", 2024-09-14T09:05:51.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6290000000000001e-06 2024-09-14T09:05:51.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:51.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096065+0000", 2024-09-14T09:05:51.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0087245410000000006 2024-09-14T09:05:51.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:51.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096245+0000", 2024-09-14T09:05:51.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017988800000000001 2024-09-14T09:05:51.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:51.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096365+0000", 2024-09-14T09:05:51.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000119708 2024-09-14T09:05:51.931 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.931 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:51.931 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.931 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18780 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 14286848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:51.932 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087348+0000", 2024-09-14T09:05:51.932 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.8224061110000003, 2024-09-14T09:05:51.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0094481670000000004, 2024-09-14T09:05:51.932 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:51.932 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:51.932 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:51.932 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:51.932 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:51.932 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:51.932 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18780 2024-09-14T09:05:51.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.932 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:51.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:51.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087348+0000", 2024-09-14T09:05:51.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:51.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087348+0000", 2024-09-14T09:05:51.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:51.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087349+0000", 2024-09-14T09:05:51.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0900000000000002e-07 2024-09-14T09:05:51.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:51.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087351+0000", 2024-09-14T09:05:51.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1009999999999999e-06 2024-09-14T09:05:51.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:51.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087357+0000", 2024-09-14T09:05:51.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2659999999999998e-06 2024-09-14T09:05:51.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:51.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087362+0000", 2024-09-14T09:05:51.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6650000000000002e-06 2024-09-14T09:05:51.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:51.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096433+0000", 2024-09-14T09:05:51.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0090705649999999992 2024-09-14T09:05:51.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:51.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096645+0000", 2024-09-14T09:05:51.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000212733 2024-09-14T09:05:51.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:51.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096797+0000", 2024-09-14T09:05:51.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000151328 2024-09-14T09:05:51.936 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.936 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:51.936 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.937 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18781 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 14548992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:51.937 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087369+0000", 2024-09-14T09:05:51.937 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.8223854490000004, 2024-09-14T09:05:51.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0096976600000000003, 2024-09-14T09:05:51.937 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:51.937 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:51.937 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:51.937 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:51.937 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:51.937 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:51.938 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18781 2024-09-14T09:05:51.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.938 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:51.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:51.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087369+0000", 2024-09-14T09:05:51.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:51.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087369+0000", 2024-09-14T09:05:51.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:51.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087370+0000", 2024-09-14T09:05:51.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9000000000000002e-07 2024-09-14T09:05:51.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:51.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087372+0000", 2024-09-14T09:05:51.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4399999999999999e-06 2024-09-14T09:05:51.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:51.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087379+0000", 2024-09-14T09:05:51.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6749999999999996e-06 2024-09-14T09:05:51.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:51.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087383+0000", 2024-09-14T09:05:51.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5630000000000002e-06 2024-09-14T09:05:51.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:51.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096852+0000", 2024-09-14T09:05:51.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0094688429999999994 2024-09-14T09:05:51.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:51.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096973+0000", 2024-09-14T09:05:51.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000121066 2024-09-14T09:05:51.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:51.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097067+0000", 2024-09-14T09:05:51.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3683000000000004e-05 2024-09-14T09:05:51.941 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.941 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:51.941 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.942 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18782 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 14811136~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:51.942 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087391+0000", 2024-09-14T09:05:51.942 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.8223640459999997, 2024-09-14T09:05:51.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0098848249999999999, 2024-09-14T09:05:51.942 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:51.942 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:51.942 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:51.942 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:51.942 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:51.942 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:51.943 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18782 2024-09-14T09:05:51.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.943 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:51.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:51.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087391+0000", 2024-09-14T09:05:51.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:51.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087391+0000", 2024-09-14T09:05:51.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:51.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087391+0000", 2024-09-14T09:05:51.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9400000000000001e-07 2024-09-14T09:05:51.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:51.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087393+0000", 2024-09-14T09:05:51.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1619999999999998e-06 2024-09-14T09:05:51.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:51.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087401+0000", 2024-09-14T09:05:51.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3950000000000001e-06 2024-09-14T09:05:51.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:51.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087414+0000", 2024-09-14T09:05:51.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2794e-05 2024-09-14T09:05:51.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:51.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097115+0000", 2024-09-14T09:05:51.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0097009920000000003 2024-09-14T09:05:51.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:51.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097191+0000", 2024-09-14T09:05:51.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6205000000000006e-05 2024-09-14T09:05:51.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:51.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097275+0000", 2024-09-14T09:05:51.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3882999999999996e-05 2024-09-14T09:05:51.947 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.947 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:51.947 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.947 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18783 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 15073280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:51.947 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087423+0000", 2024-09-14T09:05:51.947 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.8223317779999997, 2024-09-14T09:05:51.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010039031, 2024-09-14T09:05:51.947 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:51.947 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:51.948 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:51.948 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:51.948 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:51.948 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:51.948 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18783 2024-09-14T09:05:51.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.948 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:51.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:51.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087423+0000", 2024-09-14T09:05:51.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:51.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087423+0000", 2024-09-14T09:05:51.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:51.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087423+0000", 2024-09-14T09:05:51.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1799999999999995e-07 2024-09-14T09:05:51.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:51.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087426+0000", 2024-09-14T09:05:51.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.79e-06 2024-09-14T09:05:51.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:51.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087433+0000", 2024-09-14T09:05:51.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9310000000000002e-06 2024-09-14T09:05:51.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:51.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087438+0000", 2024-09-14T09:05:51.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1379999999999998e-06 2024-09-14T09:05:51.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:51.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097321+0000", 2024-09-14T09:05:51.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.009882877 2024-09-14T09:05:51.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:51.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097388+0000", 2024-09-14T09:05:51.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6958000000000005e-05 2024-09-14T09:05:51.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:51.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097462+0000", 2024-09-14T09:05:51.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3819000000000006e-05 2024-09-14T09:05:51.952 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.952 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:51.952 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.952 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18784 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 15335424~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:51.952 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087445+0000", 2024-09-14T09:05:51.952 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.8223091169999996, 2024-09-14T09:05:51.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010293808999999999, 2024-09-14T09:05:51.953 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:51.953 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:51.953 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:51.953 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:51.953 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:51.953 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:51.953 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18784 2024-09-14T09:05:51.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.953 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:51.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:51.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087445+0000", 2024-09-14T09:05:51.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:51.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087445+0000", 2024-09-14T09:05:51.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:51.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087446+0000", 2024-09-14T09:05:51.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7e-07 2024-09-14T09:05:51.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:51.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087448+0000", 2024-09-14T09:05:51.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.013e-06 2024-09-14T09:05:51.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:51.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087454+0000", 2024-09-14T09:05:51.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.088e-06 2024-09-14T09:05:51.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:51.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087460+0000", 2024-09-14T09:05:51.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.117e-06 2024-09-14T09:05:51.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:51.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097513+0000", 2024-09-14T09:05:51.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010052901 2024-09-14T09:05:51.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:51.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097613+0000", 2024-09-14T09:05:51.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9596e-05 2024-09-14T09:05:51.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:51.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097739+0000", 2024-09-14T09:05:51.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012662400000000001 2024-09-14T09:05:51.957 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.957 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:51.957 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.958 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18785 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 15597568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:51.958 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087467+0000", 2024-09-14T09:05:51.958 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.8222877630000003, 2024-09-14T09:05:51.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010499494, 2024-09-14T09:05:51.958 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:51.958 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:51.958 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:51.958 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:51.958 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:51.958 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:51.958 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18785 2024-09-14T09:05:51.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.959 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:51.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:51.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087467+0000", 2024-09-14T09:05:51.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:51.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087467+0000", 2024-09-14T09:05:51.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:51.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087467+0000", 2024-09-14T09:05:51.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7599999999999998e-07 2024-09-14T09:05:51.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:51.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087475+0000", 2024-09-14T09:05:51.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9929999999999995e-06 2024-09-14T09:05:51.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:51.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087482+0000", 2024-09-14T09:05:51.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8369999999999999e-06 2024-09-14T09:05:51.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:51.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087487+0000", 2024-09-14T09:05:51.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0919999999999997e-06 2024-09-14T09:05:51.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:51.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097794+0000", 2024-09-14T09:05:51.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010306553 2024-09-14T09:05:51.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:51.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097894+0000", 2024-09-14T09:05:51.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100077 2024-09-14T09:05:51.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:51.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097966+0000", 2024-09-14T09:05:51.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2565999999999996e-05 2024-09-14T09:05:51.962 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.963 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:51.963 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.963 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18786 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 15859712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:51.963 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087495+0000", 2024-09-14T09:05:51.963 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.8222600440000001, 2024-09-14T09:05:51.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010652643, 2024-09-14T09:05:51.963 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:51.963 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:51.963 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:51.964 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:51.964 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:51.964 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:51.964 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18786 2024-09-14T09:05:51.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.964 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:51.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:51.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087495+0000", 2024-09-14T09:05:51.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:51.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087495+0000", 2024-09-14T09:05:51.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:51.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087495+0000", 2024-09-14T09:05:51.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9400000000000005e-07 2024-09-14T09:05:51.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:51.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087497+0000", 2024-09-14T09:05:51.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9069999999999999e-06 2024-09-14T09:05:51.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:51.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087503+0000", 2024-09-14T09:05:51.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0759999999999999e-06 2024-09-14T09:05:51.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:51.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087508+0000", 2024-09-14T09:05:51.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6929999999999998e-06 2024-09-14T09:05:51.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:51.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098005+0000", 2024-09-14T09:05:51.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010497390000000001 2024-09-14T09:05:51.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:51.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098071+0000", 2024-09-14T09:05:51.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6021999999999999e-05 2024-09-14T09:05:51.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:51.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098147+0000", 2024-09-14T09:05:51.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5961e-05 2024-09-14T09:05:51.968 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.968 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:51.968 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.968 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18787 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 16121856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:51.968 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087514+0000", 2024-09-14T09:05:51.968 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.8222403649999999, 2024-09-14T09:05:51.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010782221, 2024-09-14T09:05:51.969 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:51.969 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:51.969 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:51.969 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:51.969 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:51.969 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:51.969 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18787 2024-09-14T09:05:51.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.969 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:51.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:51.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087514+0000", 2024-09-14T09:05:51.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:51.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087514+0000", 2024-09-14T09:05:51.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:51.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087515+0000", 2024-09-14T09:05:51.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9499999999999998e-07 2024-09-14T09:05:51.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:51.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087516+0000", 2024-09-14T09:05:51.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7880000000000001e-06 2024-09-14T09:05:51.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:51.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087523+0000", 2024-09-14T09:05:51.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1340000000000001e-06 2024-09-14T09:05:51.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:51.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087532+0000", 2024-09-14T09:05:51.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8130000000000007e-06 2024-09-14T09:05:51.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:51.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098178+0000", 2024-09-14T09:05:51.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010646136 2024-09-14T09:05:51.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:51.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098235+0000", 2024-09-14T09:05:51.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6415000000000001e-05 2024-09-14T09:05:51.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:51.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098296+0000", 2024-09-14T09:05:51.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1539999999999997e-05 2024-09-14T09:05:51.973 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.973 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:51.973 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.974 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18788 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 16384000~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:51.974 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087539+0000", 2024-09-14T09:05:51.974 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.8222156940000005, 2024-09-14T09:05:51.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010916135, 2024-09-14T09:05:51.974 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:51.974 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:51.974 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:51.974 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:51.974 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:51.974 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:51.974 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18788 2024-09-14T09:05:51.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.975 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:51.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:51.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087539+0000", 2024-09-14T09:05:51.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:51.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087539+0000", 2024-09-14T09:05:51.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:51.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087539+0000", 2024-09-14T09:05:51.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9800000000000004e-07 2024-09-14T09:05:51.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:51.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087541+0000", 2024-09-14T09:05:51.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9879999999999999e-06 2024-09-14T09:05:51.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:51.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087547+0000", 2024-09-14T09:05:51.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7579999999999998e-06 2024-09-14T09:05:51.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:51.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087551+0000", 2024-09-14T09:05:51.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0709999999999996e-06 2024-09-14T09:05:51.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:51.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098329+0000", 2024-09-14T09:05:51.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010777406999999999 2024-09-14T09:05:51.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:51.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098385+0000", 2024-09-14T09:05:51.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6514000000000001e-05 2024-09-14T09:05:51.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:51.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098455+0000", 2024-09-14T09:05:51.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9899000000000003e-05 2024-09-14T09:05:51.978 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.979 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:51.979 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.979 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18789 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 16646144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:51.979 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.087557+0000", 2024-09-14T09:05:51.979 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.8221971610000001, 2024-09-14T09:05:51.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011043562, 2024-09-14T09:05:51.979 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:51.979 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:51.979 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:51.979 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:51.980 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:51.980 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:51.980 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18789 2024-09-14T09:05:51.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.980 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:51.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:51.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087557+0000", 2024-09-14T09:05:51.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:51.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087557+0000", 2024-09-14T09:05:51.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:51.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087558+0000", 2024-09-14T09:05:51.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.03e-07 2024-09-14T09:05:51.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:51.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087560+0000", 2024-09-14T09:05:51.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1459999999999999e-06 2024-09-14T09:05:51.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:51.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087566+0000", 2024-09-14T09:05:51.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2870000000000001e-06 2024-09-14T09:05:51.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:51.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.087571+0000", 2024-09-14T09:05:51.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5390000000000001e-06 2024-09-14T09:05:51.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:51.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098485+0000", 2024-09-14T09:05:51.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010914561999999999 2024-09-14T09:05:51.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:51.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098541+0000", 2024-09-14T09:05:51.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5384999999999999e-05 2024-09-14T09:05:51.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:51.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.098601+0000", 2024-09-14T09:05:51.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0239999999999999e-05 2024-09-14T09:05:51.984 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.984 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:51.984 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.985 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19431 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:51.985 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.472845+0000", 2024-09-14T09:05:51.985 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.4369094579999997, 2024-09-14T09:05:51.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0089211680000000002, 2024-09-14T09:05:51.985 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:51.985 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:51.985 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:51.985 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:51.985 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:51.985 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:51.985 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19431 2024-09-14T09:05:51.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.986 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:51.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:51.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472845+0000", 2024-09-14T09:05:51.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:51.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472845+0000", 2024-09-14T09:05:51.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:51.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472846+0000", 2024-09-14T09:05:51.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0900000000000001e-07 2024-09-14T09:05:51.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:51.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472848+0000", 2024-09-14T09:05:51.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.063e-06 2024-09-14T09:05:51.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:51.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472857+0000", 2024-09-14T09:05:51.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9369999999999993e-06 2024-09-14T09:05:51.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:51.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472863+0000", 2024-09-14T09:05:51.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9780000000000002e-06 2024-09-14T09:05:51.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:51.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.481674+0000", 2024-09-14T09:05:51.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088108709999999996 2024-09-14T09:05:51.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:51.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.481714+0000", 2024-09-14T09:05:51.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0757999999999998e-05 2024-09-14T09:05:51.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:51.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.481766+0000", 2024-09-14T09:05:51.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1952e-05 2024-09-14T09:05:51.989 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.989 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:51.990 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.990 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19432 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:51.990 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.472871+0000", 2024-09-14T09:05:51.990 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.4368838080000002, 2024-09-14T09:05:51.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0090042509999999996, 2024-09-14T09:05:51.990 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:51.990 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:51.990 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:51.990 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:51.990 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:51.990 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:51.991 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19432 2024-09-14T09:05:51.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.991 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:51.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:51.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472871+0000", 2024-09-14T09:05:51.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:51.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472871+0000", 2024-09-14T09:05:51.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:51.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472871+0000", 2024-09-14T09:05:51.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9299999999999998e-07 2024-09-14T09:05:51.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:51.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472874+0000", 2024-09-14T09:05:51.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.278e-06 2024-09-14T09:05:51.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:51.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472881+0000", 2024-09-14T09:05:51.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7570000000000004e-06 2024-09-14T09:05:51.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:51.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472887+0000", 2024-09-14T09:05:51.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0059999999999998e-06 2024-09-14T09:05:51.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:51.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.481784+0000", 2024-09-14T09:05:51.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088970879999999992 2024-09-14T09:05:51.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:51.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.481825+0000", 2024-09-14T09:05:51.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1083e-05 2024-09-14T09:05:51.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:51.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.481875+0000", 2024-09-14T09:05:51.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9546000000000003e-05 2024-09-14T09:05:51.994 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.994 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:51.995 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:51.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.995 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19433 3.12 3:4a360081:::rbd_data.115e564e8eff.0000000000000002:7 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:51.995 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.472895+0000", 2024-09-14T09:05:51.995 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.436859965, 2024-09-14T09:05:51.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.009104816, 2024-09-14T09:05:51.995 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:51.995 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:51.995 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:51.995 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:51.995 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:51.996 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:51.996 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19433 2024-09-14T09:05:51.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.996 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:51.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:51.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472895+0000", 2024-09-14T09:05:51.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:51.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472895+0000", 2024-09-14T09:05:51.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:51.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:51.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472895+0000", 2024-09-14T09:05:51.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2200000000000004e-07 2024-09-14T09:05:51.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:51.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472897+0000", 2024-09-14T09:05:51.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.097e-06 2024-09-14T09:05:51.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:51.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472904+0000", 2024-09-14T09:05:51.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4150000000000004e-06 2024-09-14T09:05:51.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:51.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.472909+0000", 2024-09-14T09:05:51.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0860000000000001e-06 2024-09-14T09:05:51.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:51.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.481900+0000", 2024-09-14T09:05:51.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.008991064 2024-09-14T09:05:51.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:51.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.481943+0000", 2024-09-14T09:05:51.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3470999999999998e-05 2024-09-14T09:05:51.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:51.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:51.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:51.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.481999+0000", 2024-09-14T09:05:51.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6161e-05 2024-09-14T09:05:51.999 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.000 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.000 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.000 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20858 3.17 3:ed5052d1:::rbd_data.115e889d4809.000000000000015c:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:52.000 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:05:52.000 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4804066149999997, 2024-09-14T09:05:52.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053381794000000003, 2024-09-14T09:05:52.000 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.000 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.000 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:52.000 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.001 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.001 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.001 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20858 2024-09-14T09:05:52.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.001 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:05:52.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:05:52.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429351+0000", 2024-09-14T09:05:52.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1149999999999998e-06 2024-09-14T09:05:52.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429363+0000", 2024-09-14T09:05:52.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1819e-05 2024-09-14T09:05:52.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429389+0000", 2024-09-14T09:05:52.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5605e-05 2024-09-14T09:05:52.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429402+0000", 2024-09-14T09:05:52.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3175e-05 2024-09-14T09:05:52.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429492+0000", 2024-09-14T09:05:52.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.988e-05 2024-09-14T09:05:52.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429597+0000", 2024-09-14T09:05:52.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010564 2024-09-14T09:05:52.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:05:52.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.433465+0000", 2024-09-14T09:05:52.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0038674249999999999 2024-09-14T09:05:52.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:52.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482483+0000", 2024-09-14T09:05:52.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049017999999999999 2024-09-14T09:05:52.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:52.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482625+0000", 2024-09-14T09:05:52.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000142447 2024-09-14T09:05:52.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:52.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482684+0000", 2024-09-14T09:05:52.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8522e-05 2024-09-14T09:05:52.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482730+0000", 2024-09-14T09:05:52.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6165999999999998e-05 2024-09-14T09:05:52.006 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.006 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.006 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.007 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:20858 3.17 e41/23 ondisk, result = 0)", 2024-09-14T09:05:52.007 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:05:52.007 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4656471719999997, 2024-09-14T09:05:52.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038631402000000002, 2024-09-14T09:05:52.007 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.007 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.007 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:52.007 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:05:52.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:05:52.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444109+0000", 2024-09-14T09:05:52.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.331e-06 2024-09-14T09:05:52.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444114+0000", 2024-09-14T09:05:52.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3709999999999999e-06 2024-09-14T09:05:52.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444130+0000", 2024-09-14T09:05:52.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6079e-05 2024-09-14T09:05:52.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444144+0000", 2024-09-14T09:05:52.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3331e-05 2024-09-14T09:05:52.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482538+0000", 2024-09-14T09:05:52.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038394878 2024-09-14T09:05:52.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482548+0000", 2024-09-14T09:05:52.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0093e-05 2024-09-14T09:05:52.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482739+0000", 2024-09-14T09:05:52.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019031899999999999 2024-09-14T09:05:52.011 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.011 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.011 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.011 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:21523 3.3 3:c5644c39:::rbd_data.115e9d49ac1.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-14T09:05:52.011 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:48.930748+0000", 2024-09-14T09:05:52.011 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 2.9790060970000001, 2024-09-14T09:05:52.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0092026680000000007, 2024-09-14T09:05:52.012 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.012 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.012 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:52.012 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.012 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.012 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.012 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 21523 2024-09-14T09:05:52.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.012 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.930748+0000", 2024-09-14T09:05:52.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.930748+0000", 2024-09-14T09:05:52.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.930752+0000", 2024-09-14T09:05:52.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.27e-06 2024-09-14T09:05:52.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.930767+0000", 2024-09-14T09:05:52.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5074e-05 2024-09-14T09:05:52.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.930802+0000", 2024-09-14T09:05:52.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5179000000000003e-05 2024-09-14T09:05:52.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.930820+0000", 2024-09-14T09:05:52.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7620000000000001e-05 2024-09-14T09:05:52.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.930983+0000", 2024-09-14T09:05:52.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016344800000000001 2024-09-14T09:05:52.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.931216+0000", 2024-09-14T09:05:52.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000233367 2024-09-14T09:05:52.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:05:52.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.931633+0000", 2024-09-14T09:05:52.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00041671200000000002 2024-09-14T09:05:52.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:52.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.933538+0000", 2024-09-14T09:05:52.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0019051949999999999 2024-09-14T09:05:52.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:52.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.939809+0000", 2024-09-14T09:05:52.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0062705119999999998 2024-09-14T09:05:52.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:52.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.939872+0000", 2024-09-14T09:05:52.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3205000000000002e-05 2024-09-14T09:05:52.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.939951+0000", 2024-09-14T09:05:52.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9085999999999995e-05 2024-09-14T09:05:52.018 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.018 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.018 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.018 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.018 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.018 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:52.019 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T09:05:52.020 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:52.019+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:52.021 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:05:52.121 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:52.123+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:52.191 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:52.191 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:52.191 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:52.191 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:52.209 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:05:52.259 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:52.256+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:52.323 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:52.323+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:52.336 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:52.337 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:52.337 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:52.337 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:52.337 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:52.350 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-14T09:05:52.352 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:05:52.357 INFO:teuthology.orchestra.run.smithi176.stdout:973 read 0x331920b thru 0x331adb5 (0x1bab bytes) 2024-09-14T09:05:52.365 INFO:teuthology.orchestra.run.smithi176.stdout:974 trunc from 0x6ec91f0 to 0x275e6d9 2024-09-14T09:05:52.399 INFO:teuthology.orchestra.run.smithi176.stdout:975 trunc from 0x275e6d9 to 0x269b6f4 2024-09-14T09:05:52.422 INFO:teuthology.orchestra.run.smithi176.stdout:977 trunc from 0x269b6f4 to 0x9e21f11 2024-09-14T09:05:52.423 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:52.420+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:52.443 INFO:teuthology.orchestra.run.smithi176.stdout:979 write 0xbe93f10 thru 0xbe9c880 (0x8971 bytes) 2024-09-14T09:05:52.461 INFO:teuthology.orchestra.run.smithi176.stdout:980 read 0x8db45ee thru 0x8db7286 (0x2c99 bytes) 2024-09-14T09:05:52.461 INFO:teuthology.orchestra.run.smithi176.stdout:981 write 0x609a499 thru 0x609b9bb (0x1523 bytes) 2024-09-14T09:05:52.475 INFO:teuthology.orchestra.run.smithi176.stdout:982 read 0xa7c079a thru 0xa7c7a90 (0x72f7 bytes) 2024-09-14T09:05:52.494 INFO:teuthology.orchestra.run.smithi176.stdout:983 trunc from 0xbe9c881 to 0x2a91126 2024-09-14T09:05:52.506 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:657 3.6 e29/23 3:65b8693c:::rbd_data.115e1e95bf99.0000000000000002:head v 29'2, mlcod=0'0)", 2024-09-14T09:05:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.872567+0000", 2024-09-14T09:05:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.633484077999999, 2024-09-14T09:05:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0056247060000000002, 2024-09-14T09:05:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872567+0000", 2024-09-14T09:05:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872567+0000", 2024-09-14T09:05:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872568+0000", 2024-09-14T09:05:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2130000000000001e-06 2024-09-14T09:05:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872772+0000", 2024-09-14T09:05:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020363799999999999 2024-09-14T09:05:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872799+0000", 2024-09-14T09:05:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6917e-05 2024-09-14T09:05:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872815+0000", 2024-09-14T09:05:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5812999999999999e-05 2024-09-14T09:05:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872928+0000", 2024-09-14T09:05:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011381499999999999 2024-09-14T09:05:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.872957+0000", 2024-09-14T09:05:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9118000000000001e-05 2024-09-14T09:05:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.878076+0000", 2024-09-14T09:05:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0051180690000000003 2024-09-14T09:05:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.878192+0000", 2024-09-14T09:05:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116123 2024-09-14T09:05:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:10032 3.19 3:9ed4ec6a:::rbd_data.115eadec874.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.775706+0000", 2024-09-14T09:05:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.730344716999999, 2024-09-14T09:05:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126247939, 2024-09-14T09:05:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 10032 2024-09-14T09:05:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775706+0000", 2024-09-14T09:05:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775706+0000", 2024-09-14T09:05:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775709+0000", 2024-09-14T09:05:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0400000000000001e-06 2024-09-14T09:05:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775721+0000", 2024-09-14T09:05:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2119999999999999e-05 2024-09-14T09:05:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775752+0000", 2024-09-14T09:05:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0139999999999999e-05 2024-09-14T09:05:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775766+0000", 2024-09-14T09:05:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4497e-05 2024-09-14T09:05:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775937+0000", 2024-09-14T09:05:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000171022 2024-09-14T09:05:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.776205+0000", 2024-09-14T09:05:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026749100000000002 2024-09-14T09:05:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:05:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.784970+0000", 2024-09-14T09:05:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0087648789999999997 2024-09-14T09:05:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901870+0000", 2024-09-14T09:05:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11690028 2024-09-14T09:05:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901878+0000", 2024-09-14T09:05:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7020000000000007e-06 2024-09-14T09:05:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901913+0000", 2024-09-14T09:05:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4706000000000003e-05 2024-09-14T09:05:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901954+0000", 2024-09-14T09:05:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1062000000000001e-05 2024-09-14T09:05:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:10032 3.19 e32/23 ondisk, result = 0)", 2024-09-14T09:05:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.809095+0000", 2024-09-14T09:05:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.696956438000001, 2024-09-14T09:05:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092866664000000002, 2024-09-14T09:05:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809095+0000", 2024-09-14T09:05:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809095+0000", 2024-09-14T09:05:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809096+0000", 2024-09-14T09:05:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.64e-06 2024-09-14T09:05:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809104+0000", 2024-09-14T09:05:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7330000000000003e-06 2024-09-14T09:05:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809132+0000", 2024-09-14T09:05:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7880000000000001e-05 2024-09-14T09:05:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809146+0000", 2024-09-14T09:05:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4514e-05 2024-09-14T09:05:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901858+0000", 2024-09-14T09:05:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092711713000000001 2024-09-14T09:05:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901866+0000", 2024-09-14T09:05:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4760000000000001e-06 2024-09-14T09:05:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901961+0000", 2024-09-14T09:05:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5707999999999999e-05 2024-09-14T09:05:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:10182 3.1a e33/23 3:5a62ac37:::rbd_data.115eadec874.000000000000000d:head v 33'30, mlcod=33'28)", 2024-09-14T09:05:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:38.175853+0000", 2024-09-14T09:05:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.330198490000001, 2024-09-14T09:05:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0061618740000000003, 2024-09-14T09:05:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175853+0000", 2024-09-14T09:05:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175853+0000", 2024-09-14T09:05:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175854+0000", 2024-09-14T09:05:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8169999999999999e-06 2024-09-14T09:05:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175914+0000", 2024-09-14T09:05:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9812999999999998e-05 2024-09-14T09:05:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175941+0000", 2024-09-14T09:05:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6786000000000001e-05 2024-09-14T09:05:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.175955+0000", 2024-09-14T09:05:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.45e-05 2024-09-14T09:05:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.176049+0000", 2024-09-14T09:05:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3801000000000005e-05 2024-09-14T09:05:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.176083+0000", 2024-09-14T09:05:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.362e-05 2024-09-14T09:05:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.181952+0000", 2024-09-14T09:05:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0058688170000000001 2024-09-14T09:05:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:38.182014+0000", 2024-09-14T09:05:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2719999999999996e-05 2024-09-14T09:05:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18826 3.19 3:9c0d60c2:::rbd_data.115e3abd3b2c.0000000000000025:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.091231+0000", 2024-09-14T09:05:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.4148200220000007, 2024-09-14T09:05:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0082287890000000002, 2024-09-14T09:05:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18826 2024-09-14T09:05:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091231+0000", 2024-09-14T09:05:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091231+0000", 2024-09-14T09:05:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091232+0000", 2024-09-14T09:05:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.437e-06 2024-09-14T09:05:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091239+0000", 2024-09-14T09:05:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6479999999999997e-06 2024-09-14T09:05:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091255+0000", 2024-09-14T09:05:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5727e-05 2024-09-14T09:05:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091264+0000", 2024-09-14T09:05:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1989999999999995e-06 2024-09-14T09:05:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091339+0000", 2024-09-14T09:05:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5117999999999995e-05 2024-09-14T09:05:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091466+0000", 2024-09-14T09:05:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000126811 2024-09-14T09:05:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:05:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091772+0000", 2024-09-14T09:05:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030571299999999999 2024-09-14T09:05:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096709+0000", 2024-09-14T09:05:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0049374960000000004 2024-09-14T09:05:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099352+0000", 2024-09-14T09:05:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0026423750000000002 2024-09-14T09:05:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099416+0000", 2024-09-14T09:05:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4731000000000001e-05 2024-09-14T09:05:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099460+0000", 2024-09-14T09:05:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3534e-05 2024-09-14T09:05:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18828 3.1a 3:5bcb7d4f:::rbd_data.115e3abd3b2c.0000000000000028:head [stat,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.091433+0000", 2024-09-14T09:05:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.4146179500000002, 2024-09-14T09:05:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0080150039999999992, 2024-09-14T09:05:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18828 2024-09-14T09:05:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091433+0000", 2024-09-14T09:05:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091433+0000", 2024-09-14T09:05:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091434+0000", 2024-09-14T09:05:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.054e-06 2024-09-14T09:05:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091440+0000", 2024-09-14T09:05:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5249999999999996e-06 2024-09-14T09:05:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091453+0000", 2024-09-14T09:05:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2860999999999999e-05 2024-09-14T09:05:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091460+0000", 2024-09-14T09:05:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9559999999999997e-06 2024-09-14T09:05:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091522+0000", 2024-09-14T09:05:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1433999999999995e-05 2024-09-14T09:05:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.091673+0000", 2024-09-14T09:05:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015112599999999999 2024-09-14T09:05:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099448+0000", 2024-09-14T09:05:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0077750479999999997 2024-09-14T09:05:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18835 3.1a 3:5d7331e3:::rbd_data.115e3abd3b2c.000000000000002c:head [stat,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.095572+0000", 2024-09-14T09:05:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.4104788930000005, 2024-09-14T09:05:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0068495320000000002, 2024-09-14T09:05:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18835 2024-09-14T09:05:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095572+0000", 2024-09-14T09:05:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095572+0000", 2024-09-14T09:05:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095574+0000", 2024-09-14T09:05:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9769999999999999e-06 2024-09-14T09:05:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095582+0000", 2024-09-14T09:05:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6410000000000002e-06 2024-09-14T09:05:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095601+0000", 2024-09-14T09:05:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9034000000000001e-05 2024-09-14T09:05:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095611+0000", 2024-09-14T09:05:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0431999999999999e-05 2024-09-14T09:05:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095680+0000", 2024-09-14T09:05:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8753999999999999e-05 2024-09-14T09:05:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.095811+0000", 2024-09-14T09:05:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013112399999999999 2024-09-14T09:05:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102422+0000", 2024-09-14T09:05:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0066105699999999996 2024-09-14T09:05:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "pg_update_log_missing(3.f epoch 39/34 rep_tid 211 entries 39'21 (0'0) error 3:f32f8323:::rbd_data.115e3abd3b2c.000000000000002a:head by client.4446.0:18831 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 39'19)", 2024-09-14T09:05:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.096732+0000", 2024-09-14T09:05:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.4093185730000002, 2024-09-14T09:05:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0054062889999999999, 2024-09-14T09:05:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096732+0000", 2024-09-14T09:05:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096732+0000", 2024-09-14T09:05:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096733+0000", 2024-09-14T09:05:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5700000000000002e-07 2024-09-14T09:05:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096742+0000", 2024-09-14T09:05:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3030000000000005e-06 2024-09-14T09:05:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096767+0000", 2024-09-14T09:05:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4621999999999999e-05 2024-09-14T09:05:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096788+0000", 2024-09-14T09:05:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0791000000000001e-05 2024-09-14T09:05:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.096872+0000", 2024-09-14T09:05:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3949999999999994e-05 2024-09-14T09:05:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102139+0000", 2024-09-14T09:05:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0052669659999999997 2024-09-14T09:05:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.543 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18836 3.19 3:981038f8:::rbd_data.115e3abd3b2c.000000000000002d:head [stat,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.097080+0000", 2024-09-14T09:05:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.4089707629999992, 2024-09-14T09:05:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0053201890000000003, 2024-09-14T09:05:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18836 2024-09-14T09:05:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097080+0000", 2024-09-14T09:05:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097080+0000", 2024-09-14T09:05:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097082+0000", 2024-09-14T09:05:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3999999999999999e-06 2024-09-14T09:05:52.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097088+0000", 2024-09-14T09:05:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.998e-06 2024-09-14T09:05:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097104+0000", 2024-09-14T09:05:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6291999999999999e-05 2024-09-14T09:05:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097111+0000", 2024-09-14T09:05:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7490000000000001e-06 2024-09-14T09:05:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097167+0000", 2024-09-14T09:05:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6427e-05 2024-09-14T09:05:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.097311+0000", 2024-09-14T09:05:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014356800000000001 2024-09-14T09:05:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102400+0000", 2024-09-14T09:05:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050897549999999996 2024-09-14T09:05:52.547 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18841 3.1a 3:5bcb7d4f:::rbd_data.115e3abd3b2c.0000000000000028:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.099711+0000", 2024-09-14T09:05:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.4063396879999992, 2024-09-14T09:05:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0060169799999999999, 2024-09-14T09:05:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.548 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18841 2024-09-14T09:05:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099711+0000", 2024-09-14T09:05:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099711+0000", 2024-09-14T09:05:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099713+0000", 2024-09-14T09:05:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5880000000000001e-06 2024-09-14T09:05:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099719+0000", 2024-09-14T09:05:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4790000000000001e-06 2024-09-14T09:05:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099735+0000", 2024-09-14T09:05:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5671999999999998e-05 2024-09-14T09:05:52.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099744+0000", 2024-09-14T09:05:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5210000000000002e-06 2024-09-14T09:05:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099824+0000", 2024-09-14T09:05:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0442e-05 2024-09-14T09:05:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.099955+0000", 2024-09-14T09:05:52.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013079300000000001 2024-09-14T09:05:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 2", 2024-09-14T09:05:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.100251+0000", 2024-09-14T09:05:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029648899999999999 2024-09-14T09:05:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105631+0000", 2024-09-14T09:05:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0053793019999999999 2024-09-14T09:05:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105640+0000", 2024-09-14T09:05:52.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7419999999999992e-06 2024-09-14T09:05:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105683+0000", 2024-09-14T09:05:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3040999999999997e-05 2024-09-14T09:05:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.105728+0000", 2024-09-14T09:05:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4910999999999997e-05 2024-09-14T09:05:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18847 3.19 3:981038f8:::rbd_data.115e3abd3b2c.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.102651+0000", 2024-09-14T09:05:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.4033995850000007, 2024-09-14T09:05:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0060367759999999998, 2024-09-14T09:05:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18847 2024-09-14T09:05:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102651+0000", 2024-09-14T09:05:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102651+0000", 2024-09-14T09:05:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102653+0000", 2024-09-14T09:05:52.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3799999999999999e-06 2024-09-14T09:05:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102660+0000", 2024-09-14T09:05:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9480000000000002e-06 2024-09-14T09:05:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102675+0000", 2024-09-14T09:05:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5046e-05 2024-09-14T09:05:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102681+0000", 2024-09-14T09:05:52.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8030000000000002e-06 2024-09-14T09:05:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102733+0000", 2024-09-14T09:05:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2846000000000002e-05 2024-09-14T09:05:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102803+0000", 2024-09-14T09:05:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9296999999999997e-05 2024-09-14T09:05:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:05:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102917+0000", 2024-09-14T09:05:52.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011461200000000001 2024-09-14T09:05:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108240+0000", 2024-09-14T09:05:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0053221459999999998 2024-09-14T09:05:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108584+0000", 2024-09-14T09:05:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000344006 2024-09-14T09:05:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108642+0000", 2024-09-14T09:05:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8004999999999998e-05 2024-09-14T09:05:52.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108688+0000", 2024-09-14T09:05:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6687000000000002e-05 2024-09-14T09:05:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18848 3.1a 3:5d7331e3:::rbd_data.115e3abd3b2c.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.102710+0000", 2024-09-14T09:05:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.4033412149999993, 2024-09-14T09:05:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0061127430000000003, 2024-09-14T09:05:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.559 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18848 2024-09-14T09:05:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102710+0000", 2024-09-14T09:05:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102710+0000", 2024-09-14T09:05:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102711+0000", 2024-09-14T09:05:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0299999999999998e-07 2024-09-14T09:05:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102715+0000", 2024-09-14T09:05:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4360000000000004e-06 2024-09-14T09:05:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102723+0000", 2024-09-14T09:05:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7330000000000003e-06 2024-09-14T09:05:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102728+0000", 2024-09-14T09:05:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4940000000000003e-06 2024-09-14T09:05:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102793+0000", 2024-09-14T09:05:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5037999999999996e-05 2024-09-14T09:05:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.102872+0000", 2024-09-14T09:05:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8616999999999996e-05 2024-09-14T09:05:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 2", 2024-09-14T09:05:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103061+0000", 2024-09-14T09:05:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018863500000000001 2024-09-14T09:05:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108275+0000", 2024-09-14T09:05:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0052146759999999997 2024-09-14T09:05:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108737+0000", 2024-09-14T09:05:52.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00046148899999999999 2024-09-14T09:05:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108786+0000", 2024-09-14T09:05:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9191999999999999e-05 2024-09-14T09:05:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108823+0000", 2024-09-14T09:05:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.663e-05 2024-09-14T09:05:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:18846 3.f e39/34 3:f32f8323:::rbd_data.115e3abd3b2c.000000000000002a:head v 39'22, mlcod=39'20)", 2024-09-14T09:05:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.103489+0000", 2024-09-14T09:05:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.4025620560000007, 2024-09-14T09:05:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0052005020000000001, 2024-09-14T09:05:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103489+0000", 2024-09-14T09:05:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103489+0000", 2024-09-14T09:05:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103490+0000", 2024-09-14T09:05:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.091e-06 2024-09-14T09:05:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103499+0000", 2024-09-14T09:05:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1260000000000004e-06 2024-09-14T09:05:52.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103516+0000", 2024-09-14T09:05:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6354e-05 2024-09-14T09:05:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103526+0000", 2024-09-14T09:05:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0876000000000001e-05 2024-09-14T09:05:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103586+0000", 2024-09-14T09:05:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9301e-05 2024-09-14T09:05:52.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.103613+0000", 2024-09-14T09:05:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6778e-05 2024-09-14T09:05:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108637+0000", 2024-09-14T09:05:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050246989999999997 2024-09-14T09:05:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.108689+0000", 2024-09-14T09:05:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2277000000000001e-05 2024-09-14T09:05:52.568 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:19545 3.f e39/34 3:f12afce6:::rbd_data.115e4c71d6e8.0000000000000003:head v 39'24, mlcod=39'22)", 2024-09-14T09:05:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.529754+0000", 2024-09-14T09:05:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.9762969009999996, 2024-09-14T09:05:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037365357000000002, 2024-09-14T09:05:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.529754+0000", 2024-09-14T09:05:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.529754+0000", 2024-09-14T09:05:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.529756+0000", 2024-09-14T09:05:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.804e-06 2024-09-14T09:05:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537071+0000", 2024-09-14T09:05:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0073151120000000004 2024-09-14T09:05:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537111+0000", 2024-09-14T09:05:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0315999999999998e-05 2024-09-14T09:05:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537133+0000", 2024-09-14T09:05:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2098000000000001e-05 2024-09-14T09:05:52.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537274+0000", 2024-09-14T09:05:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014045400000000001 2024-09-14T09:05:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537312+0000", 2024-09-14T09:05:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7761e-05 2024-09-14T09:05:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.567050+0000", 2024-09-14T09:05:52.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029738547000000001 2024-09-14T09:05:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.567119+0000", 2024-09-14T09:05:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9264999999999995e-05 2024-09-14T09:05:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20695 3.4 3:2204856a:::rbd_data.115e889d4809.000000000000001b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.022232+0000", 2024-09-14T09:05:52.573 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.4838191209999998, 2024-09-14T09:05:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01001865, 2024-09-14T09:05:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20695 2024-09-14T09:05:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022232+0000", 2024-09-14T09:05:52.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022232+0000", 2024-09-14T09:05:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022233+0000", 2024-09-14T09:05:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0020000000000001e-06 2024-09-14T09:05:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022238+0000", 2024-09-14T09:05:52.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3279999999999996e-06 2024-09-14T09:05:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022251+0000", 2024-09-14T09:05:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2573e-05 2024-09-14T09:05:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022258+0000", 2024-09-14T09:05:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3819999999999998e-06 2024-09-14T09:05:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022323+0000", 2024-09-14T09:05:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4902999999999998e-05 2024-09-14T09:05:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022385+0000", 2024-09-14T09:05:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1933999999999994e-05 2024-09-14T09:05:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 2", 2024-09-14T09:05:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022451+0000", 2024-09-14T09:05:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5529000000000002e-05 2024-09-14T09:05:52.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.022975+0000", 2024-09-14T09:05:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00052479100000000004 2024-09-14T09:05:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.032141+0000", 2024-09-14T09:05:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0091651990000000006 2024-09-14T09:05:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.032201+0000", 2024-09-14T09:05:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0727999999999999e-05 2024-09-14T09:05:52.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.032251+0000", 2024-09-14T09:05:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9280999999999998e-05 2024-09-14T09:05:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20862 3.1a 3:580a70cb:::rbd_data.115e889d4809.0000000000000177:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.497482+0000", 2024-09-14T09:05:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.0085686689999998, 2024-09-14T09:05:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.032797795999999997, 2024-09-14T09:05:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.579 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20862 2024-09-14T09:05:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497482+0000", 2024-09-14T09:05:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497482+0000", 2024-09-14T09:05:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497485+0000", 2024-09-14T09:05:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8569999999999999e-06 2024-09-14T09:05:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497496+0000", 2024-09-14T09:05:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0277e-05 2024-09-14T09:05:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497519+0000", 2024-09-14T09:05:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3513e-05 2024-09-14T09:05:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497533+0000", 2024-09-14T09:05:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4343e-05 2024-09-14T09:05:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497642+0000", 2024-09-14T09:05:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010840099999999999 2024-09-14T09:05:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497840+0000", 2024-09-14T09:05:52.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000198192 2024-09-14T09:05:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 2", 2024-09-14T09:05:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.499243+0000", 2024-09-14T09:05:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0014027550000000001 2024-09-14T09:05:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530087+0000", 2024-09-14T09:05:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030844086999999999 2024-09-14T09:05:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530120+0000", 2024-09-14T09:05:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3309000000000002e-05 2024-09-14T09:05:52.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530207+0000", 2024-09-14T09:05:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6536e-05 2024-09-14T09:05:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530280+0000", 2024-09-14T09:05:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3526e-05 2024-09-14T09:05:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.584 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:20862 3.1a e41/34 ondisk, result = 0)", 2024-09-14T09:05:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.503269+0000", 2024-09-14T09:05:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.0027816639999996, 2024-09-14T09:05:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026831292999999999, 2024-09-14T09:05:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503269+0000", 2024-09-14T09:05:52.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503269+0000", 2024-09-14T09:05:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503271+0000", 2024-09-14T09:05:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3820000000000001e-06 2024-09-14T09:05:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503276+0000", 2024-09-14T09:05:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6400000000000002e-06 2024-09-14T09:05:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503293+0000", 2024-09-14T09:05:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6404000000000001e-05 2024-09-14T09:05:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503304+0000", 2024-09-14T09:05:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0905e-05 2024-09-14T09:05:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530061+0000", 2024-09-14T09:05:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026757019 2024-09-14T09:05:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530076+0000", 2024-09-14T09:05:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5526999999999999e-05 2024-09-14T09:05:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530101+0000", 2024-09-14T09:05:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4416000000000001e-05 2024-09-14T09:05:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.588 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20892 3.19 3:9c9d5d4e:::rbd_data.115e889d4809.000000000000007c:head [set-alloc-hint object_size 262144 write_size 262144,write 54228~63989] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.617095+0000", 2024-09-14T09:05:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8889561720000003, 2024-09-14T09:05:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0096848479999999994, 2024-09-14T09:05:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:52.589 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20892 2024-09-14T09:05:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.617095+0000", 2024-09-14T09:05:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.617095+0000", 2024-09-14T09:05:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.617097+0000", 2024-09-14T09:05:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.627e-06 2024-09-14T09:05:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.617150+0000", 2024-09-14T09:05:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2528999999999998e-05 2024-09-14T09:05:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.617181+0000", 2024-09-14T09:05:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1279999999999999e-05 2024-09-14T09:05:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.617197+0000", 2024-09-14T09:05:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5843000000000001e-05 2024-09-14T09:05:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.617341+0000", 2024-09-14T09:05:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014422799999999999 2024-09-14T09:05:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.617583+0000", 2024-09-14T09:05:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024192800000000001 2024-09-14T09:05:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:05:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.617808+0000", 2024-09-14T09:05:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022461700000000001 2024-09-14T09:05:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.625752+0000", 2024-09-14T09:05:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0079439809999999993 2024-09-14T09:05:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.626621+0000", 2024-09-14T09:05:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00086927700000000001 2024-09-14T09:05:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.626710+0000", 2024-09-14T09:05:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9145000000000001e-05 2024-09-14T09:05:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.626780+0000", 2024-09-14T09:05:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9393000000000005e-05 2024-09-14T09:05:52.594 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:21522 3.0 3:0039c8e9:::rbd_data.115e9d49ac1.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-14T09:05:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:48.930598+0000", 2024-09-14T09:05:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.5754533190000002, 2024-09-14T09:05:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014757594000000001, 2024-09-14T09:05:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 21522 2024-09-14T09:05:52.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.930598+0000", 2024-09-14T09:05:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.930598+0000", 2024-09-14T09:05:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.930601+0000", 2024-09-14T09:05:52.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.163e-06 2024-09-14T09:05:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.930861+0000", 2024-09-14T09:05:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026049999999999999 2024-09-14T09:05:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.930900+0000", 2024-09-14T09:05:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8843000000000001e-05 2024-09-14T09:05:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.930922+0000", 2024-09-14T09:05:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1733e-05 2024-09-14T09:05:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.931018+0000", 2024-09-14T09:05:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6269999999999998e-05 2024-09-14T09:05:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.931184+0000", 2024-09-14T09:05:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016556000000000001 2024-09-14T09:05:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 2", 2024-09-14T09:05:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.931420+0000", 2024-09-14T09:05:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023652300000000001 2024-09-14T09:05:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.936558+0000", 2024-09-14T09:05:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0051379119999999997 2024-09-14T09:05:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.945286+0000", 2024-09-14T09:05:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0087279290000000006 2024-09-14T09:05:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.945326+0000", 2024-09-14T09:05:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9456999999999998e-05 2024-09-14T09:05:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.945355+0000", 2024-09-14T09:05:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9703999999999999e-05 2024-09-14T09:05:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:21525 3.8 e42/23 3:13cecb79:::rbd_data.115e9d49ac1.000000000000000c:head v 42'11, mlcod=41'9)", 2024-09-14T09:05:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:48.933346+0000", 2024-09-14T09:05:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.572704715, 2024-09-14T09:05:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011545616, 2024-09-14T09:05:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.933346+0000", 2024-09-14T09:05:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.933346+0000", 2024-09-14T09:05:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.933348+0000", 2024-09-14T09:05:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9870000000000002e-06 2024-09-14T09:05:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.933510+0000", 2024-09-14T09:05:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000161965 2024-09-14T09:05:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.933556+0000", 2024-09-14T09:05:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6215999999999999e-05 2024-09-14T09:05:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.933577+0000", 2024-09-14T09:05:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0689000000000001e-05 2024-09-14T09:05:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.933705+0000", 2024-09-14T09:05:52.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012807899999999999 2024-09-14T09:05:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.933796+0000", 2024-09-14T09:05:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0696e-05 2024-09-14T09:05:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.944842+0000", 2024-09-14T09:05:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011046149 2024-09-14T09:05:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:52.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:48.944892+0000", 2024-09-14T09:05:52.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9835e-05 2024-09-14T09:05:52.605 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.605 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.605 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.605 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.605 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.605 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:52.606 INFO:teuthology.orchestra.run.smithi176.stdout:984 write 0x1935106 thru 0x1944b8c (0xfa87 bytes) 2024-09-14T09:05:52.607 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:05:52.607 INFO:teuthology.orchestra.run.smithi176.stdout:985 writesame 0x62fff2 thru 0x638299 data_size 0x71(0x82a8 bytes) 2024-09-14T09:05:52.607 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:52.604+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:52.649 INFO:teuthology.orchestra.run.smithi176.stdout:986 punch from 0x1276b61 to 0x127bfea, (0x5489 bytes) 2024-09-14T09:05:52.651 INFO:teuthology.orchestra.run.smithi176.stdout:987 trunc from 0x2a91126 to 0x179dfa1 2024-09-14T09:05:52.707 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:52.707+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:52.763 INFO:teuthology.orchestra.run.smithi176.stdout:988 write 0x9342f75 thru 0x9347e36 (0x4ec2 bytes) 2024-09-14T09:05:52.777 INFO:teuthology.orchestra.run.smithi176.stdout:989 read 0x8fef840 thru 0x8ff0b86 (0x1347 bytes) 2024-09-14T09:05:52.777 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:52.777 INFO:teuthology.orchestra.run.smithi176.stdout:991 write 0x72a3528 thru 0x72a73ae (0x3e87 bytes) 2024-09-14T09:05:52.791 INFO:teuthology.orchestra.run.smithi176.stdout:992 write 0x129b337 thru 0x12a0d87 (0x5a51 bytes) 2024-09-14T09:05:52.807 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:52.807+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:52.828 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:52.828 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:52.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.829 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29064 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.829 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819027+0000", 2024-09-14T09:05:52.829 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0090198499999999994, 2024-09-14T09:05:52.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0090466060000000004, 2024-09-14T09:05:52.829 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.829 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.829 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.829 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.829 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.829 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.829 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29064 2024-09-14T09:05:52.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.830 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819027+0000", 2024-09-14T09:05:52.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819027+0000", 2024-09-14T09:05:52.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819027+0000", 2024-09-14T09:05:52.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9900000000000001e-07 2024-09-14T09:05:52.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819029+0000", 2024-09-14T09:05:52.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.874e-06 2024-09-14T09:05:52.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819036+0000", 2024-09-14T09:05:52.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4239999999999998e-06 2024-09-14T09:05:52.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819041+0000", 2024-09-14T09:05:52.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9559999999999996e-06 2024-09-14T09:05:52.832 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.832 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.832 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.832 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29065 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.832 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819049+0000", 2024-09-14T09:05:52.833 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0089980979999999995, 2024-09-14T09:05:52.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0090883969999999998, 2024-09-14T09:05:52.833 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.833 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.833 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.833 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.833 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.833 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.833 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29065 2024-09-14T09:05:52.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.834 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819049+0000", 2024-09-14T09:05:52.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819049+0000", 2024-09-14T09:05:52.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819049+0000", 2024-09-14T09:05:52.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0499999999999999e-07 2024-09-14T09:05:52.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819051+0000", 2024-09-14T09:05:52.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.84e-06 2024-09-14T09:05:52.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819059+0000", 2024-09-14T09:05:52.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7139999999999993e-06 2024-09-14T09:05:52.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819070+0000", 2024-09-14T09:05:52.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1600000000000001e-05 2024-09-14T09:05:52.836 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.836 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.836 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.837 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29301 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.837 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819078+0000", 2024-09-14T09:05:52.837 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0089692569999999996, 2024-09-14T09:05:52.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0091139380000000002, 2024-09-14T09:05:52.837 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.837 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.837 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.837 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.837 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.837 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.838 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29301 2024-09-14T09:05:52.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.838 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819078+0000", 2024-09-14T09:05:52.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819078+0000", 2024-09-14T09:05:52.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819078+0000", 2024-09-14T09:05:52.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6400000000000003e-07 2024-09-14T09:05:52.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819083+0000", 2024-09-14T09:05:52.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.49e-06 2024-09-14T09:05:52.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819092+0000", 2024-09-14T09:05:52.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1829999999999994e-06 2024-09-14T09:05:52.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819096+0000", 2024-09-14T09:05:52.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7650000000000001e-06 2024-09-14T09:05:52.840 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.840 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.840 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.841 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29302 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.841 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819103+0000", 2024-09-14T09:05:52.841 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0089433220000000001, 2024-09-14T09:05:52.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0091409070000000002, 2024-09-14T09:05:52.841 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.841 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.841 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.841 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.841 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.841 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.841 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29302 2024-09-14T09:05:52.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.842 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819103+0000", 2024-09-14T09:05:52.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819103+0000", 2024-09-14T09:05:52.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819104+0000", 2024-09-14T09:05:52.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2500000000000001e-07 2024-09-14T09:05:52.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819106+0000", 2024-09-14T09:05:52.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.903e-06 2024-09-14T09:05:52.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819114+0000", 2024-09-14T09:05:52.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0940000000000008e-06 2024-09-14T09:05:52.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819119+0000", 2024-09-14T09:05:52.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.831e-06 2024-09-14T09:05:52.844 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.844 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.844 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.844 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29303 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.845 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819125+0000", 2024-09-14T09:05:52.845 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0089215240000000001, 2024-09-14T09:05:52.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0091710060000000006, 2024-09-14T09:05:52.845 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.845 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.845 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.845 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.845 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.845 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.845 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29303 2024-09-14T09:05:52.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.846 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819125+0000", 2024-09-14T09:05:52.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819125+0000", 2024-09-14T09:05:52.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819126+0000", 2024-09-14T09:05:52.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6400000000000003e-07 2024-09-14T09:05:52.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819128+0000", 2024-09-14T09:05:52.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.906e-06 2024-09-14T09:05:52.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819134+0000", 2024-09-14T09:05:52.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0499999999999997e-06 2024-09-14T09:05:52.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819138+0000", 2024-09-14T09:05:52.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6990000000000002e-06 2024-09-14T09:05:52.848 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.848 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.848 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.848 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29304 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.848 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819145+0000", 2024-09-14T09:05:52.849 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0089019240000000003, 2024-09-14T09:05:52.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0092048270000000005, 2024-09-14T09:05:52.849 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.849 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.849 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.849 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.849 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.849 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.849 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29304 2024-09-14T09:05:52.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.849 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819145+0000", 2024-09-14T09:05:52.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819145+0000", 2024-09-14T09:05:52.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819145+0000", 2024-09-14T09:05:52.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1800000000000001e-07 2024-09-14T09:05:52.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819152+0000", 2024-09-14T09:05:52.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8120000000000002e-06 2024-09-14T09:05:52.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819159+0000", 2024-09-14T09:05:52.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5309999999999998e-06 2024-09-14T09:05:52.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819163+0000", 2024-09-14T09:05:52.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7960000000000003e-06 2024-09-14T09:05:52.852 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.852 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.852 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.852 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29305 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.853 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819170+0000", 2024-09-14T09:05:52.853 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.008876419, 2024-09-14T09:05:52.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0092303469999999999, 2024-09-14T09:05:52.853 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.853 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.853 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.853 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.853 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.853 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.853 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29305 2024-09-14T09:05:52.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.853 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819170+0000", 2024-09-14T09:05:52.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819170+0000", 2024-09-14T09:05:52.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819171+0000", 2024-09-14T09:05:52.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3000000000000001e-07 2024-09-14T09:05:52.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819173+0000", 2024-09-14T09:05:52.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9240000000000001e-06 2024-09-14T09:05:52.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819179+0000", 2024-09-14T09:05:52.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9909999999999999e-06 2024-09-14T09:05:52.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819183+0000", 2024-09-14T09:05:52.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7500000000000003e-06 2024-09-14T09:05:52.856 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.856 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.856 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.857 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29306 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.857 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819190+0000", 2024-09-14T09:05:52.857 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0088566059999999995, 2024-09-14T09:05:52.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0092628550000000004, 2024-09-14T09:05:52.857 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.857 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.857 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.857 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.857 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.857 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.857 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29306 2024-09-14T09:05:52.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.858 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819190+0000", 2024-09-14T09:05:52.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819190+0000", 2024-09-14T09:05:52.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819191+0000", 2024-09-14T09:05:52.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6899999999999998e-07 2024-09-14T09:05:52.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819193+0000", 2024-09-14T09:05:52.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.04e-06 2024-09-14T09:05:52.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819200+0000", 2024-09-14T09:05:52.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9460000000000001e-06 2024-09-14T09:05:52.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819210+0000", 2024-09-14T09:05:52.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0438999999999999e-05 2024-09-14T09:05:52.860 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.860 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.860 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.860 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29307 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.860 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819218+0000", 2024-09-14T09:05:52.861 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0088288370000000008, 2024-09-14T09:05:52.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0092854029999999994, 2024-09-14T09:05:52.861 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.861 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.861 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.861 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.861 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.861 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.861 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29307 2024-09-14T09:05:52.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.861 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819218+0000", 2024-09-14T09:05:52.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819218+0000", 2024-09-14T09:05:52.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819218+0000", 2024-09-14T09:05:52.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9800000000000004e-07 2024-09-14T09:05:52.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819221+0000", 2024-09-14T09:05:52.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1150000000000001e-06 2024-09-14T09:05:52.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819227+0000", 2024-09-14T09:05:52.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8580000000000002e-06 2024-09-14T09:05:52.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819232+0000", 2024-09-14T09:05:52.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8999999999999997e-06 2024-09-14T09:05:52.864 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.864 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.864 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.864 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29308 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.864 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819239+0000", 2024-09-14T09:05:52.865 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0088075489999999996, 2024-09-14T09:05:52.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0093201510000000005, 2024-09-14T09:05:52.865 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.865 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.865 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.865 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.865 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.865 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.865 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29308 2024-09-14T09:05:52.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.865 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819239+0000", 2024-09-14T09:05:52.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819239+0000", 2024-09-14T09:05:52.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819240+0000", 2024-09-14T09:05:52.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.75e-07 2024-09-14T09:05:52.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819249+0000", 2024-09-14T09:05:52.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8489999999999995e-06 2024-09-14T09:05:52.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819255+0000", 2024-09-14T09:05:52.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7259999999999996e-06 2024-09-14T09:05:52.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819260+0000", 2024-09-14T09:05:52.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8659999999999997e-06 2024-09-14T09:05:52.868 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.868 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.868 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.868 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29309 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.868 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819267+0000", 2024-09-14T09:05:52.868 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0087796899999999997, 2024-09-14T09:05:52.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0093437499999999996, 2024-09-14T09:05:52.869 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.869 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.869 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.869 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.869 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.869 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.869 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29309 2024-09-14T09:05:52.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.869 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819267+0000", 2024-09-14T09:05:52.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819267+0000", 2024-09-14T09:05:52.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819268+0000", 2024-09-14T09:05:52.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7899999999999999e-07 2024-09-14T09:05:52.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819269+0000", 2024-09-14T09:05:52.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8220000000000001e-06 2024-09-14T09:05:52.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819276+0000", 2024-09-14T09:05:52.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1319999999999999e-06 2024-09-14T09:05:52.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819280+0000", 2024-09-14T09:05:52.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7040000000000002e-06 2024-09-14T09:05:52.872 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.872 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.872 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.872 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29310 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.872 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819286+0000", 2024-09-14T09:05:52.872 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0087603799999999999, 2024-09-14T09:05:52.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0093758569999999996, 2024-09-14T09:05:52.873 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.873 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.873 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.873 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.873 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.873 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.873 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29310 2024-09-14T09:05:52.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.873 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819286+0000", 2024-09-14T09:05:52.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819286+0000", 2024-09-14T09:05:52.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819287+0000", 2024-09-14T09:05:52.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9799999999999999e-07 2024-09-14T09:05:52.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819289+0000", 2024-09-14T09:05:52.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.733e-06 2024-09-14T09:05:52.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819294+0000", 2024-09-14T09:05:52.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9560000000000002e-06 2024-09-14T09:05:52.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819299+0000", 2024-09-14T09:05:52.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6940000000000003e-06 2024-09-14T09:05:52.876 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.876 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.876 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.876 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29311 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.876 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819305+0000", 2024-09-14T09:05:52.876 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0087414339999999993, 2024-09-14T09:05:52.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0094073430000000003, 2024-09-14T09:05:52.877 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.877 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.877 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.877 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.877 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.877 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.877 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29311 2024-09-14T09:05:52.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.877 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819305+0000", 2024-09-14T09:05:52.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819305+0000", 2024-09-14T09:05:52.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819306+0000", 2024-09-14T09:05:52.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7800000000000002e-07 2024-09-14T09:05:52.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819307+0000", 2024-09-14T09:05:52.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.654e-06 2024-09-14T09:05:52.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819313+0000", 2024-09-14T09:05:52.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.122e-06 2024-09-14T09:05:52.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819318+0000", 2024-09-14T09:05:52.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7350000000000004e-06 2024-09-14T09:05:52.880 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.880 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.880 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.880 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29312 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.880 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819325+0000", 2024-09-14T09:05:52.880 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0087221199999999999, 2024-09-14T09:05:52.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0094402280000000002, 2024-09-14T09:05:52.880 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.881 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.881 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.881 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.881 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.881 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.881 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29312 2024-09-14T09:05:52.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.881 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819325+0000", 2024-09-14T09:05:52.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819325+0000", 2024-09-14T09:05:52.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819325+0000", 2024-09-14T09:05:52.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.89e-07 2024-09-14T09:05:52.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819327+0000", 2024-09-14T09:05:52.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.792e-06 2024-09-14T09:05:52.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819333+0000", 2024-09-14T09:05:52.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9660000000000001e-06 2024-09-14T09:05:52.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819337+0000", 2024-09-14T09:05:52.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6469999999999997e-06 2024-09-14T09:05:52.884 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.884 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.884 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.884 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29313 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.884 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819344+0000", 2024-09-14T09:05:52.885 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.008703166, 2024-09-14T09:05:52.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0094739400000000001, 2024-09-14T09:05:52.885 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.885 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.885 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.885 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.885 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.885 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.885 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29313 2024-09-14T09:05:52.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.885 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819344+0000", 2024-09-14T09:05:52.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819344+0000", 2024-09-14T09:05:52.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819344+0000", 2024-09-14T09:05:52.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1100000000000001e-07 2024-09-14T09:05:52.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819349+0000", 2024-09-14T09:05:52.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2959999999999998e-06 2024-09-14T09:05:52.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819356+0000", 2024-09-14T09:05:52.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5629999999999997e-06 2024-09-14T09:05:52.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819360+0000", 2024-09-14T09:05:52.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5709999999999999e-06 2024-09-14T09:05:52.888 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.889 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.889 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.889 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29314 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.889 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819367+0000", 2024-09-14T09:05:52.889 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0086793219999999997, 2024-09-14T09:05:52.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0095018269999999991, 2024-09-14T09:05:52.889 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.889 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.889 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.890 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.890 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.890 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.890 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29314 2024-09-14T09:05:52.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.890 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819367+0000", 2024-09-14T09:05:52.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819367+0000", 2024-09-14T09:05:52.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819368+0000", 2024-09-14T09:05:52.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6699999999999999e-07 2024-09-14T09:05:52.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819370+0000", 2024-09-14T09:05:52.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1090000000000001e-06 2024-09-14T09:05:52.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819377+0000", 2024-09-14T09:05:52.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.939e-06 2024-09-14T09:05:52.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819382+0000", 2024-09-14T09:05:52.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1610000000000002e-06 2024-09-14T09:05:52.893 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.893 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.893 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.893 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29315 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.893 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819389+0000", 2024-09-14T09:05:52.893 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0086574180000000001, 2024-09-14T09:05:52.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0095335590000000005, 2024-09-14T09:05:52.893 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.893 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.893 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.894 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.894 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.894 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.894 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29315 2024-09-14T09:05:52.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.894 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819389+0000", 2024-09-14T09:05:52.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819389+0000", 2024-09-14T09:05:52.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819390+0000", 2024-09-14T09:05:52.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2500000000000001e-07 2024-09-14T09:05:52.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819397+0000", 2024-09-14T09:05:52.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6249999999999998e-06 2024-09-14T09:05:52.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819404+0000", 2024-09-14T09:05:52.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6320000000000002e-06 2024-09-14T09:05:52.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819414+0000", 2024-09-14T09:05:52.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8199999999999992e-06 2024-09-14T09:05:52.896 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.897 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.897 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.897 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29316 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.897 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819421+0000", 2024-09-14T09:05:52.897 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.008626079, 2024-09-14T09:05:52.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0095581399999999997, 2024-09-14T09:05:52.897 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.897 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.897 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.898 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.898 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.898 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.898 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29316 2024-09-14T09:05:52.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.898 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819421+0000", 2024-09-14T09:05:52.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819421+0000", 2024-09-14T09:05:52.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819421+0000", 2024-09-14T09:05:52.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7599999999999997e-07 2024-09-14T09:05:52.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819423+0000", 2024-09-14T09:05:52.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9099999999999999e-06 2024-09-14T09:05:52.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819430+0000", 2024-09-14T09:05:52.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4330000000000001e-06 2024-09-14T09:05:52.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819434+0000", 2024-09-14T09:05:52.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.826e-06 2024-09-14T09:05:52.900 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.900 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.900 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.901 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29317 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.901 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819441+0000", 2024-09-14T09:05:52.901 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0086060730000000005, 2024-09-14T09:05:52.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0095925379999999994, 2024-09-14T09:05:52.901 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.901 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.901 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.901 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29317 2024-09-14T09:05:52.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.902 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819441+0000", 2024-09-14T09:05:52.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819441+0000", 2024-09-14T09:05:52.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819441+0000", 2024-09-14T09:05:52.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7599999999999998e-07 2024-09-14T09:05:52.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819443+0000", 2024-09-14T09:05:52.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.826e-06 2024-09-14T09:05:52.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819449+0000", 2024-09-14T09:05:52.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.302e-06 2024-09-14T09:05:52.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819454+0000", 2024-09-14T09:05:52.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6929999999999998e-06 2024-09-14T09:05:52.904 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.904 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.904 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.905 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29318 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.905 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819461+0000", 2024-09-14T09:05:52.905 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0085862409999999997, 2024-09-14T09:05:52.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.009625336, 2024-09-14T09:05:52.905 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.905 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.905 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.905 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.905 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.905 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.906 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29318 2024-09-14T09:05:52.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.906 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819461+0000", 2024-09-14T09:05:52.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819461+0000", 2024-09-14T09:05:52.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819461+0000", 2024-09-14T09:05:52.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5100000000000001e-07 2024-09-14T09:05:52.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819463+0000", 2024-09-14T09:05:52.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7630000000000001e-06 2024-09-14T09:05:52.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819469+0000", 2024-09-14T09:05:52.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.93e-06 2024-09-14T09:05:52.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819479+0000", 2024-09-14T09:05:52.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0508e-05 2024-09-14T09:05:52.908 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.908 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.909 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.909 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29319 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.909 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819486+0000", 2024-09-14T09:05:52.909 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0085608769999999997, 2024-09-14T09:05:52.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0096518690000000004, 2024-09-14T09:05:52.909 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.909 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.909 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.910 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.910 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.910 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.910 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29319 2024-09-14T09:05:52.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.910 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819486+0000", 2024-09-14T09:05:52.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819486+0000", 2024-09-14T09:05:52.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819486+0000", 2024-09-14T09:05:52.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6100000000000001e-07 2024-09-14T09:05:52.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819488+0000", 2024-09-14T09:05:52.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9580000000000001e-06 2024-09-14T09:05:52.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819495+0000", 2024-09-14T09:05:52.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3269999999999997e-06 2024-09-14T09:05:52.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819500+0000", 2024-09-14T09:05:52.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9259999999999999e-06 2024-09-14T09:05:52.912 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.912 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.913 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.913 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29320 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.913 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819507+0000", 2024-09-14T09:05:52.913 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0085395499999999999, 2024-09-14T09:05:52.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0096821689999999992, 2024-09-14T09:05:52.913 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.913 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.913 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.913 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.913 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.914 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.914 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29320 2024-09-14T09:05:52.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.914 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819507+0000", 2024-09-14T09:05:52.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819507+0000", 2024-09-14T09:05:52.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819508+0000", 2024-09-14T09:05:52.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.06e-07 2024-09-14T09:05:52.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819510+0000", 2024-09-14T09:05:52.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1289999999999999e-06 2024-09-14T09:05:52.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819516+0000", 2024-09-14T09:05:52.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.63e-06 2024-09-14T09:05:52.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819521+0000", 2024-09-14T09:05:52.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8409999999999999e-06 2024-09-14T09:05:52.916 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.917 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.917 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.917 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29321 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.917 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:05:52.917 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0085183659999999994, 2024-09-14T09:05:52.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0097126499999999998, 2024-09-14T09:05:52.917 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.917 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.917 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.918 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29321 2024-09-14T09:05:52.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.918 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:05:52.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:05:52.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819529+0000", 2024-09-14T09:05:52.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8800000000000003e-07 2024-09-14T09:05:52.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819531+0000", 2024-09-14T09:05:52.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9439999999999999e-06 2024-09-14T09:05:52.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819537+0000", 2024-09-14T09:05:52.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.037e-06 2024-09-14T09:05:52.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819542+0000", 2024-09-14T09:05:52.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7670000000000003e-06 2024-09-14T09:05:52.921 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.921 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.921 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.921 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29322 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.921 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:05:52.921 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0084988560000000008, 2024-09-14T09:05:52.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0097441779999999992, 2024-09-14T09:05:52.921 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.921 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.922 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.922 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29322 2024-09-14T09:05:52.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.922 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:05:52.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:05:52.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:05:52.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9499999999999998e-07 2024-09-14T09:05:52.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819550+0000", 2024-09-14T09:05:52.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8309999999999999e-06 2024-09-14T09:05:52.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819556+0000", 2024-09-14T09:05:52.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1299999999999998e-06 2024-09-14T09:05:52.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819561+0000", 2024-09-14T09:05:52.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.707e-06 2024-09-14T09:05:52.925 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.925 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.925 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.925 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29061 3.9 3:90303d3a:::rbd_data.115ebc3c67d1.0000000000000014:c [read 720896~2048 out=2048b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.925 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.818950+0000", 2024-09-14T09:05:52.925 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0090970249999999999, 2024-09-14T09:05:52.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010400378, 2024-09-14T09:05:52.926 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.926 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.926 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:52.926 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.926 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.926 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.926 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29061 2024-09-14T09:05:52.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.926 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.818950+0000", 2024-09-14T09:05:52.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.818950+0000", 2024-09-14T09:05:52.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.818950+0000", 2024-09-14T09:05:52.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2899999999999999e-07 2024-09-14T09:05:52.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.818952+0000", 2024-09-14T09:05:52.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.801e-06 2024-09-14T09:05:52.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.818960+0000", 2024-09-14T09:05:52.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6000000000000001e-06 2024-09-14T09:05:52.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.818965+0000", 2024-09-14T09:05:52.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9590000000000003e-06 2024-09-14T09:05:52.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.829206+0000", 2024-09-14T09:05:52.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01024167 2024-09-14T09:05:52.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:52.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.829284+0000", 2024-09-14T09:05:52.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7919999999999999e-05 2024-09-14T09:05:52.930 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.930 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.930 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.930 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29062 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.930 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.818973+0000", 2024-09-14T09:05:52.930 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0090741729999999996, 2024-09-14T09:05:52.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010444126, 2024-09-14T09:05:52.930 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.930 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.931 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:05:52.931 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.931 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.931 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.931 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29062 2024-09-14T09:05:52.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.931 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.818973+0000", 2024-09-14T09:05:52.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.818973+0000", 2024-09-14T09:05:52.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.818973+0000", 2024-09-14T09:05:52.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4499999999999997e-07 2024-09-14T09:05:52.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.818975+0000", 2024-09-14T09:05:52.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2120000000000002e-06 2024-09-14T09:05:52.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.818984+0000", 2024-09-14T09:05:52.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2020000000000004e-06 2024-09-14T09:05:52.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.818996+0000", 2024-09-14T09:05:52.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2136999999999999e-05 2024-09-14T09:05:52.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:52.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.829411+0000", 2024-09-14T09:05:52.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010415432000000001 2024-09-14T09:05:52.934 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.934 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.934 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.935 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29063 3.9 3.5cbc0c09 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:52.935 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819004+0000", 2024-09-14T09:05:52.935 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0090429059999999999, 2024-09-14T09:05:52.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010467761000000001, 2024-09-14T09:05:52.935 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:52.935 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:52.935 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:05:52.935 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:52.935 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:52.935 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:52.936 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29063 2024-09-14T09:05:52.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.936 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:52.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:52.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819004+0000", 2024-09-14T09:05:52.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:52.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819004+0000", 2024-09-14T09:05:52.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:52.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:52.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819004+0000", 2024-09-14T09:05:52.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2799999999999996e-07 2024-09-14T09:05:52.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:52.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819007+0000", 2024-09-14T09:05:52.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3099999999999999e-06 2024-09-14T09:05:52.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:52.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819015+0000", 2024-09-14T09:05:52.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1920000000000005e-06 2024-09-14T09:05:52.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:52.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:52.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:52.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819020+0000", 2024-09-14T09:05:52.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9420000000000003e-06 2024-09-14T09:05:52.938 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.938 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:52.939 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.939 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:52.939 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:05:52.939 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 27 2024-09-14T09:05:52.939 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:52.941 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:05:52.942 INFO:teuthology.orchestra.run.smithi176.stdout:993 punch from 0x52d99e9 to 0x52e863e, (0xec55 bytes) 2024-09-14T09:05:52.942 INFO:teuthology.orchestra.run.smithi176.stdout:994 write 0x6d434f1 thru 0x6d44296 (0xda6 bytes) 2024-09-14T09:05:52.942 INFO:teuthology.orchestra.run.smithi176.stdout:995 punch from 0x36d584b to 0x36de6e4, (0x8e99 bytes) 2024-09-14T09:05:52.942 INFO:teuthology.orchestra.run.smithi176.stdout:997 read 0x1b429c thru 0x1b9f9f (0x5d04 bytes) 2024-09-14T09:05:52.942 INFO:teuthology.orchestra.run.smithi176.stdout:998 trunc from 0x9347e37 to 0xce2f157 2024-09-14T09:05:52.942 INFO:teuthology.orchestra.run.smithi176.stdout:1000 write 0xcbf02cb thru 0xcbfe80a (0xe540 bytes) 2024-09-14T09:05:52.942 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:52.936+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:52.950 INFO:teuthology.orchestra.run.smithi176.stdout:1001 punch from 0xadb56a9 to 0xadc48e6, (0xf23d bytes) 2024-09-14T09:05:52.958 INFO:teuthology.orchestra.run.smithi176.stdout:1002 writesame 0xe89ac41 thru 0xe89f45c data_size 0xd(0x481c bytes) 2024-09-14T09:05:52.968 INFO:teuthology.orchestra.run.smithi176.stdout:1003 trunc from 0xe89f45d to 0x7ccb8d8 2024-09-14T09:05:52.987 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is set 2024-09-14T09:05:52.993 INFO:teuthology.orchestra.run.smithi176.stdout:1005 writesame 0x5bb9e4a thru 0x5bbc32e data_size 0x1(0x24e5 bytes) 2024-09-14T09:05:53.013 INFO:teuthology.orchestra.run.smithi176.stdout:1006 read 0x7667a50 thru 0x766fa9b (0x804c bytes) 2024-09-14T09:05:53.013 INFO:teuthology.orchestra.run.smithi176.stdout:1007 punch from 0xa5e66a to 0xa61b8f, (0x3525 bytes) 2024-09-14T09:05:53.013 INFO:teuthology.orchestra.run.smithi176.stdout:1009 read 0x5814a86 thru 0x581644a (0x19c5 bytes) 2024-09-14T09:05:53.014 INFO:teuthology.orchestra.run.smithi176.stdout:1010 read 0x28a9de7 thru 0x28b82bf (0xe4d9 bytes) 2024-09-14T09:05:53.014 INFO:teuthology.orchestra.run.smithi176.stdout:1011 read 0x1e84ca3 thru 0x1e8c729 (0x7a87 bytes) 2024-09-14T09:05:53.029 INFO:teuthology.orchestra.run.smithi176.stdout:1012 write 0xd928b74 thru 0xd92cab4 (0x3f41 bytes) 2024-09-14T09:05:53.041 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:53.043+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:53.048 INFO:teuthology.orchestra.run.smithi176.stdout:1014 writesame 0xc7926 thru 0xd2cf1 data_size 0x4a(0xb3cc bytes) 2024-09-14T09:05:53.103 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:53.103 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:53.103 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:53.104 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:53.104 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:53.122 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:05:53.166 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:53.164+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:53.197 INFO:teuthology.orchestra.run.smithi176.stdout:1015 punch from 0x8bd643b to 0x8bd75fd, (0x11c2 bytes) 2024-09-14T09:05:53.197 INFO:teuthology.orchestra.run.smithi176.stdout:1016 write 0x140165d thru 0x140ce2e (0xb7d2 bytes) 2024-09-14T09:05:53.200 INFO:teuthology.orchestra.run.smithi176.stdout:1017 read 0x1f6d5c3 thru 0x1f76fd7 (0x9a15 bytes) 2024-09-14T09:05:53.200 INFO:teuthology.orchestra.run.smithi176.stdout:1018 writesame 0xa342506 thru 0xa348132 data_size 0x1(0x5c2d bytes) 2024-09-14T09:05:53.210 INFO:teuthology.orchestra.run.smithi176.stdout:1019 read 0x7c3aa11 thru 0x7c47382 (0xc972 bytes) 2024-09-14T09:05:53.221 INFO:teuthology.orchestra.run.smithi176.stdout:1020 trunc from 0xd92cab5 to 0x566dd1a 2024-09-14T09:05:53.240 INFO:teuthology.orchestra.run.smithi176.stdout:1021 read 0x4bdb1cb thru 0x4be327e (0x80b4 bytes) 2024-09-14T09:05:53.243 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:53.240+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:53.252 INFO:teuthology.orchestra.run.smithi176.stdout:1023 writesame 0x98df4b3 thru 0x98e1d6d data_size 0x1(0x28bb bytes) 2024-09-14T09:05:53.259 INFO:teuthology.orchestra.run.smithi176.stdout:1024 writesame 0x9cf778c thru 0x9d00463 data_size 0x4(0x8cd8 bytes) 2024-09-14T09:05:53.259 INFO:teuthology.orchestra.run.smithi176.stdout:1025 read 0x1f3bde0 thru 0x1f3cf3f (0x1160 bytes) 2024-09-14T09:05:53.259 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:53.271 INFO:teuthology.orchestra.run.smithi176.stdout:1029 write 0xe4b120e thru 0xe4b74c5 (0x62b8 bytes) 2024-09-14T09:05:53.281 INFO:teuthology.orchestra.run.smithi176.stdout:1030 write 0x417b82f thru 0x4184ca1 (0x9473 bytes) 2024-09-14T09:05:53.299 INFO:teuthology.orchestra.run.smithi176.stdout:1031 read 0x1c3218f thru 0x1c41250 (0xf0c2 bytes) 2024-09-14T09:05:53.299 INFO:teuthology.orchestra.run.smithi176.stdout:1033 write 0x6230e81 thru 0x623feb7 (0xf037 bytes) 2024-09-14T09:05:53.303 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:53.303 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:53.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:53.303 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:53.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.303 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18383 3.9 3:9252f7c9:::rbd_data.115e3abd3b2c.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:05:53.303 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:05:53.304 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.145430914, 2024-09-14T09:05:53.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056433830999999997, 2024-09-14T09:05:53.304 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.304 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.304 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:53.304 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.304 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.304 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.304 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18383 2024-09-14T09:05:53.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.304 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:05:53.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:05:53.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158423+0000", 2024-09-14T09:05:53.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T09:05:53.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158429+0000", 2024-09-14T09:05:53.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0179999999999998e-06 2024-09-14T09:05:53.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158440+0000", 2024-09-14T09:05:53.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0769e-05 2024-09-14T09:05:53.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158445+0000", 2024-09-14T09:05:53.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5840000000000002e-06 2024-09-14T09:05:53.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158495+0000", 2024-09-14T09:05:53.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9129999999999999e-05 2024-09-14T09:05:53.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158598+0000", 2024-09-14T09:05:53.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102938 2024-09-14T09:05:53.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:05:53.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.161048+0000", 2024-09-14T09:05:53.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002450336 2024-09-14T09:05:53.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:53.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214605+0000", 2024-09-14T09:05:53.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053557526000000001 2024-09-14T09:05:53.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:53.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214634+0000", 2024-09-14T09:05:53.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8483000000000002e-05 2024-09-14T09:05:53.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:53.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214731+0000", 2024-09-14T09:05:53.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7026000000000002e-05 2024-09-14T09:05:53.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214856+0000", 2024-09-14T09:05:53.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012474799999999999 2024-09-14T09:05:53.309 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.309 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.309 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.309 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18383 3.9 e38/34 ondisk, result = 0)", 2024-09-14T09:05:53.309 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:05:53.310 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.135114433, 2024-09-14T09:05:53.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045880575999999999, 2024-09-14T09:05:53.310 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.310 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.310 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.310 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:05:53.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:05:53.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168739+0000", 2024-09-14T09:05:53.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3900000000000004e-07 2024-09-14T09:05:53.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168744+0000", 2024-09-14T09:05:53.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5929999999999999e-06 2024-09-14T09:05:53.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168754+0000", 2024-09-14T09:05:53.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0686999999999999e-05 2024-09-14T09:05:53.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168761+0000", 2024-09-14T09:05:53.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6340000000000003e-06 2024-09-14T09:05:53.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214578+0000", 2024-09-14T09:05:53.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045817443999999999 2024-09-14T09:05:53.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214596+0000", 2024-09-14T09:05:53.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7159e-05 2024-09-14T09:05:53.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214619+0000", 2024-09-14T09:05:53.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.332e-05 2024-09-14T09:05:53.313 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.313 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.313 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.313 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19152 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14159058~8940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:53.313 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:05:53.313 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.9812019769999996, 2024-09-14T09:05:53.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12891994500000001, 2024-09-14T09:05:53.314 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.314 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.314 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:53.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.314 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19152 2024-09-14T09:05:53.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.314 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:05:53.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:05:53.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322653+0000", 2024-09-14T09:05:53.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3750000000000001e-06 2024-09-14T09:05:53.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329647+0000", 2024-09-14T09:05:53.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069941960000000003 2024-09-14T09:05:53.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329673+0000", 2024-09-14T09:05:53.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5760000000000001e-05 2024-09-14T09:05:53.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329684+0000", 2024-09-14T09:05:53.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0474e-05 2024-09-14T09:05:53.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329776+0000", 2024-09-14T09:05:53.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2467999999999996e-05 2024-09-14T09:05:53.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329987+0000", 2024-09-14T09:05:53.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021133 2024-09-14T09:05:53.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:05:53.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.331303+0000", 2024-09-14T09:05:53.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001315629 2024-09-14T09:05:53.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:53.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.411090+0000", 2024-09-14T09:05:53.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079787477999999995 2024-09-14T09:05:53.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:53.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451463+0000", 2024-09-14T09:05:53.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040372312 2024-09-14T09:05:53.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:53.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451523+0000", 2024-09-14T09:05:53.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0420000000000001e-05 2024-09-14T09:05:53.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451571+0000", 2024-09-14T09:05:53.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7503000000000002e-05 2024-09-14T09:05:53.318 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.318 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.319 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.319 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20925 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:53.319 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:05:53.319 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6319031969999998, 2024-09-14T09:05:53.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017084327999999999, 2024-09-14T09:05:53.319 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.319 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.319 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.319 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.319 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.319 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.320 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20925 2024-09-14T09:05:53.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.320 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:05:53.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:05:53.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:05:53.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3e-07 2024-09-14T09:05:53.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671953+0000", 2024-09-14T09:05:53.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0299999999999998e-06 2024-09-14T09:05:53.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671964+0000", 2024-09-14T09:05:53.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1e-05 2024-09-14T09:05:53.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671971+0000", 2024-09-14T09:05:53.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9929999999999998e-06 2024-09-14T09:05:53.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.687941+0000", 2024-09-14T09:05:53.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015969357 2024-09-14T09:05:53.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.688070+0000", 2024-09-14T09:05:53.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000129858 2024-09-14T09:05:53.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689034+0000", 2024-09-14T09:05:53.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00096336000000000002 2024-09-14T09:05:53.323 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.323 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.323 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.323 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20926 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:53.323 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:05:53.323 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.631869977, 2024-09-14T09:05:53.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018572207, 2024-09-14T09:05:53.323 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.323 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.323 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.324 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.324 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.324 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.324 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20926 2024-09-14T09:05:53.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.324 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:05:53.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:05:53.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:05:53.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:05:53.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671990+0000", 2024-09-14T09:05:53.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5769999999999999e-06 2024-09-14T09:05:53.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672001+0000", 2024-09-14T09:05:53.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1293e-05 2024-09-14T09:05:53.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672008+0000", 2024-09-14T09:05:53.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9099999999999999e-06 2024-09-14T09:05:53.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689147+0000", 2024-09-14T09:05:53.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017139060000000001 2024-09-14T09:05:53.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689324+0000", 2024-09-14T09:05:53.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017640200000000001 2024-09-14T09:05:53.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690555+0000", 2024-09-14T09:05:53.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001231212 2024-09-14T09:05:53.327 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.327 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.327 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.327 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20927 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2621440~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:53.328 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:05:53.328 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6318330110000003, 2024-09-14T09:05:53.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020002227000000001, 2024-09-14T09:05:53.328 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.328 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.328 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.328 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.328 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.328 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.328 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20927 2024-09-14T09:05:53.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.328 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:05:53.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:05:53.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:05:53.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9699999999999995e-07 2024-09-14T09:05:53.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672024+0000", 2024-09-14T09:05:53.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:05:53.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672036+0000", 2024-09-14T09:05:53.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.119e-05 2024-09-14T09:05:53.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672044+0000", 2024-09-14T09:05:53.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8370000000000003e-06 2024-09-14T09:05:53.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690602+0000", 2024-09-14T09:05:53.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018557648 2024-09-14T09:05:53.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690805+0000", 2024-09-14T09:05:53.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020282899999999999 2024-09-14T09:05:53.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692022+0000", 2024-09-14T09:05:53.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012170429999999999 2024-09-14T09:05:53.331 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.332 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.332 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.332 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20928 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 917504~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:53.332 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:05:53.332 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6317968709999997, 2024-09-14T09:05:53.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021623738, 2024-09-14T09:05:53.332 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.332 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.332 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.332 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.332 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.332 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.333 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20928 2024-09-14T09:05:53.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.333 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:05:53.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:05:53.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672057+0000", 2024-09-14T09:05:53.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:05:53.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672059+0000", 2024-09-14T09:05:53.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9270000000000001e-06 2024-09-14T09:05:53.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672070+0000", 2024-09-14T09:05:53.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0773e-05 2024-09-14T09:05:53.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672077+0000", 2024-09-14T09:05:53.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9600000000000003e-06 2024-09-14T09:05:53.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692154+0000", 2024-09-14T09:05:53.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020076482 2024-09-14T09:05:53.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692419+0000", 2024-09-14T09:05:53.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026517400000000002 2024-09-14T09:05:53.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693680+0000", 2024-09-14T09:05:53.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012607390000000001 2024-09-14T09:05:53.336 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.336 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.336 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.336 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20929 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2752512~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:53.336 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:05:53.336 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6317638290000005, 2024-09-14T09:05:53.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022898728, 2024-09-14T09:05:53.336 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.336 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.336 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.337 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.337 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.337 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.337 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20929 2024-09-14T09:05:53.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.337 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:05:53.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:05:53.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672090+0000", 2024-09-14T09:05:53.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:05:53.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672092+0000", 2024-09-14T09:05:53.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8940000000000002e-06 2024-09-14T09:05:53.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672103+0000", 2024-09-14T09:05:53.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0815999999999999e-05 2024-09-14T09:05:53.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672110+0000", 2024-09-14T09:05:53.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9469999999999997e-06 2024-09-14T09:05:53.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693737+0000", 2024-09-14T09:05:53.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021626467999999999 2024-09-14T09:05:53.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693962+0000", 2024-09-14T09:05:53.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225474 2024-09-14T09:05:53.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.694988+0000", 2024-09-14T09:05:53.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001025469 2024-09-14T09:05:53.340 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.340 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.340 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.340 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20930 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:53.340 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:05:53.340 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6317307349999997, 2024-09-14T09:05:53.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024303286, 2024-09-14T09:05:53.340 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.341 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.341 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.341 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.341 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.341 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.341 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20930 2024-09-14T09:05:53.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.341 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:05:53.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:05:53.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672123+0000", 2024-09-14T09:05:53.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:05:53.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672125+0000", 2024-09-14T09:05:53.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7300000000000001e-06 2024-09-14T09:05:53.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672134+0000", 2024-09-14T09:05:53.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7930000000000003e-06 2024-09-14T09:05:53.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672141+0000", 2024-09-14T09:05:53.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9029999999999998e-06 2024-09-14T09:05:53.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695087+0000", 2024-09-14T09:05:53.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022946013000000001 2024-09-14T09:05:53.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695350+0000", 2024-09-14T09:05:53.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026266000000000001 2024-09-14T09:05:53.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696425+0000", 2024-09-14T09:05:53.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001075527 2024-09-14T09:05:53.344 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.344 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.344 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.344 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20931 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:53.344 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:05:53.345 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6316975060000001, 2024-09-14T09:05:53.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025331519, 2024-09-14T09:05:53.345 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.345 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.345 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.345 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.345 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.345 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.345 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20931 2024-09-14T09:05:53.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.345 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:05:53.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:05:53.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672156+0000", 2024-09-14T09:05:53.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4300000000000002e-07 2024-09-14T09:05:53.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672159+0000", 2024-09-14T09:05:53.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T09:05:53.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672170+0000", 2024-09-14T09:05:53.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1192999999999999e-05 2024-09-14T09:05:53.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672177+0000", 2024-09-14T09:05:53.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9059999999999996e-06 2024-09-14T09:05:53.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696532+0000", 2024-09-14T09:05:53.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024355291000000001 2024-09-14T09:05:53.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696755+0000", 2024-09-14T09:05:53.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022297699999999999 2024-09-14T09:05:53.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697487+0000", 2024-09-14T09:05:53.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00073171499999999997 2024-09-14T09:05:53.348 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.348 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.348 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.348 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20932 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1179648~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:53.349 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:05:53.349 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6316640629999997, 2024-09-14T09:05:53.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026104083, 2024-09-14T09:05:53.349 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.349 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.349 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.349 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.349 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.349 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.349 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20932 2024-09-14T09:05:53.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.349 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:05:53.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:05:53.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:05:53.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0699999999999996e-07 2024-09-14T09:05:53.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672192+0000", 2024-09-14T09:05:53.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7700000000000002e-06 2024-09-14T09:05:53.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672203+0000", 2024-09-14T09:05:53.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0956e-05 2024-09-14T09:05:53.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672211+0000", 2024-09-14T09:05:53.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3769999999999996e-06 2024-09-14T09:05:53.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697530+0000", 2024-09-14T09:05:53.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025318411999999998 2024-09-14T09:05:53.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697640+0000", 2024-09-14T09:05:53.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011045299999999999 2024-09-14T09:05:53.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698293+0000", 2024-09-14T09:05:53.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065240800000000005 2024-09-14T09:05:53.353 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.353 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.353 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.353 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20933 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3014656~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:53.353 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:05:53.353 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6316298600000003, 2024-09-14T09:05:53.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026558875999999999, 2024-09-14T09:05:53.353 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.353 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.353 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.353 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.353 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.353 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.354 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20933 2024-09-14T09:05:53.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.354 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:05:53.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:05:53.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672224+0000", 2024-09-14T09:05:53.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2399999999999997e-07 2024-09-14T09:05:53.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672234+0000", 2024-09-14T09:05:53.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0793e-05 2024-09-14T09:05:53.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672246+0000", 2024-09-14T09:05:53.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659e-05 2024-09-14T09:05:53.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672255+0000", 2024-09-14T09:05:53.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5730000000000008e-06 2024-09-14T09:05:53.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698326+0000", 2024-09-14T09:05:53.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026071414000000001 2024-09-14T09:05:53.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698374+0000", 2024-09-14T09:05:53.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8032999999999999e-05 2024-09-14T09:05:53.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698782+0000", 2024-09-14T09:05:53.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040768 2024-09-14T09:05:53.357 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.357 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.357 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.357 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20934 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1310720~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:53.357 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:05:53.357 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6315879840000003, 2024-09-14T09:05:53.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027292067999999999, 2024-09-14T09:05:53.357 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.357 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.357 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.358 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.358 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.358 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.358 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20934 2024-09-14T09:05:53.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.358 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:05:53.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:05:53.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672266+0000", 2024-09-14T09:05:53.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6299999999999993e-07 2024-09-14T09:05:53.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672269+0000", 2024-09-14T09:05:53.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8569999999999999e-06 2024-09-14T09:05:53.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672280+0000", 2024-09-14T09:05:53.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1236999999999999e-05 2024-09-14T09:05:53.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672287+0000", 2024-09-14T09:05:53.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8260000000000004e-06 2024-09-14T09:05:53.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698820+0000", 2024-09-14T09:05:53.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026533055999999999 2024-09-14T09:05:53.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698933+0000", 2024-09-14T09:05:53.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011291 2024-09-14T09:05:53.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699557+0000", 2024-09-14T09:05:53.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062421899999999997 2024-09-14T09:05:53.361 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.361 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.361 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.361 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20935 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:53.361 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:05:53.361 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6315542240000003, 2024-09-14T09:05:53.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028108610999999999, 2024-09-14T09:05:53.362 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.362 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.362 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.362 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.362 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.362 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.362 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20935 2024-09-14T09:05:53.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.362 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:05:53.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:05:53.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672299+0000", 2024-09-14T09:05:53.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T09:05:53.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672302+0000", 2024-09-14T09:05:53.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7640000000000001e-06 2024-09-14T09:05:53.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672313+0000", 2024-09-14T09:05:53.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1080000000000001e-05 2024-09-14T09:05:53.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672321+0000", 2024-09-14T09:05:53.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6490000000000008e-06 2024-09-14T09:05:53.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699587+0000", 2024-09-14T09:05:53.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027265884000000001 2024-09-14T09:05:53.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699631+0000", 2024-09-14T09:05:53.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4947999999999997e-05 2024-09-14T09:05:53.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700407+0000", 2024-09-14T09:05:53.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00077559299999999999 2024-09-14T09:05:53.365 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.366 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.366 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.366 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20936 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1441792~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:53.366 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:05:53.366 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6315200479999996, 2024-09-14T09:05:53.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028784750000000001, 2024-09-14T09:05:53.366 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.366 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.366 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.367 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.367 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20936 2024-09-14T09:05:53.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.367 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:05:53.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:05:53.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:05:53.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4700000000000001e-07 2024-09-14T09:05:53.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672336+0000", 2024-09-14T09:05:53.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6730000000000001e-06 2024-09-14T09:05:53.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672347+0000", 2024-09-14T09:05:53.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0710000000000001e-05 2024-09-14T09:05:53.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672354+0000", 2024-09-14T09:05:53.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4630000000000004e-06 2024-09-14T09:05:53.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700442+0000", 2024-09-14T09:05:53.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028087951 2024-09-14T09:05:53.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700557+0000", 2024-09-14T09:05:53.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011461800000000001 2024-09-14T09:05:53.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701117+0000", 2024-09-14T09:05:53.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056058799999999999 2024-09-14T09:05:53.370 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.370 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.370 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.370 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20937 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:53.370 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:05:53.371 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6314869219999997, 2024-09-14T09:05:53.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029493028000000001, 2024-09-14T09:05:53.371 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.371 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.371 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.371 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.371 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.371 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.371 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20937 2024-09-14T09:05:53.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.371 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:05:53.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:05:53.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:05:53.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T09:05:53.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672384+0000", 2024-09-14T09:05:53.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7127000000000001e-05 2024-09-14T09:05:53.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672395+0000", 2024-09-14T09:05:53.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759999999999999e-05 2024-09-14T09:05:53.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672402+0000", 2024-09-14T09:05:53.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0960000000000003e-06 2024-09-14T09:05:53.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701147+0000", 2024-09-14T09:05:53.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028744241 2024-09-14T09:05:53.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701246+0000", 2024-09-14T09:05:53.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9031999999999996e-05 2024-09-14T09:05:53.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701859+0000", 2024-09-14T09:05:53.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061310499999999997 2024-09-14T09:05:53.375 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.375 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.375 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.375 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20938 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:53.375 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:05:53.375 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6314397620000003, 2024-09-14T09:05:53.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030109219999999999, 2024-09-14T09:05:53.375 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.375 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.375 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.375 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.375 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.376 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.376 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20938 2024-09-14T09:05:53.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.376 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:05:53.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:05:53.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672414+0000", 2024-09-14T09:05:53.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-07 2024-09-14T09:05:53.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672417+0000", 2024-09-14T09:05:53.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:05:53.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672428+0000", 2024-09-14T09:05:53.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0873000000000001e-05 2024-09-14T09:05:53.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672435+0000", 2024-09-14T09:05:53.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:05:53.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701884+0000", 2024-09-14T09:05:53.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029449811999999999 2024-09-14T09:05:53.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701934+0000", 2024-09-14T09:05:53.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9703000000000002e-05 2024-09-14T09:05:53.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702522+0000", 2024-09-14T09:05:53.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00058811200000000001 2024-09-14T09:05:53.379 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.379 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.379 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.379 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20939 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:53.379 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:05:53.379 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.6314076289999999, 2024-09-14T09:05:53.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030779259, 2024-09-14T09:05:53.379 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.379 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.380 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.380 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20939 2024-09-14T09:05:53.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.380 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:05:53.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:05:53.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672446+0000", 2024-09-14T09:05:53.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:05:53.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672449+0000", 2024-09-14T09:05:53.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7930000000000002e-06 2024-09-14T09:05:53.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672461+0000", 2024-09-14T09:05:53.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T09:05:53.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672471+0000", 2024-09-14T09:05:53.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0807e-05 2024-09-14T09:05:53.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702551+0000", 2024-09-14T09:05:53.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030079228 2024-09-14T09:05:53.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702664+0000", 2024-09-14T09:05:53.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011351800000000001 2024-09-14T09:05:53.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.703224+0000", 2024-09-14T09:05:53.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056026999999999997 2024-09-14T09:05:53.383 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.383 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.383 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.383 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29321 3.9 3:90303d3a:::rbd_data.115ebc3c67d1.0000000000000014:c [read 770048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:53.383 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:05:53.383 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.48432430399999998, 2024-09-14T09:05:53.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015942787, 2024-09-14T09:05:53.383 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.384 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.384 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.384 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.384 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.384 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.384 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29321 2024-09-14T09:05:53.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.384 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:05:53.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:05:53.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819529+0000", 2024-09-14T09:05:53.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8800000000000003e-07 2024-09-14T09:05:53.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819531+0000", 2024-09-14T09:05:53.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9439999999999999e-06 2024-09-14T09:05:53.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819537+0000", 2024-09-14T09:05:53.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.037e-06 2024-09-14T09:05:53.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819542+0000", 2024-09-14T09:05:53.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7670000000000003e-06 2024-09-14T09:05:53.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835320+0000", 2024-09-14T09:05:53.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015778119 2024-09-14T09:05:53.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835384+0000", 2024-09-14T09:05:53.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3897000000000002e-05 2024-09-14T09:05:53.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835471+0000", 2024-09-14T09:05:53.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7534999999999995e-05 2024-09-14T09:05:53.387 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.387 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.387 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.388 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29322 3.9 3:90303d3a:::rbd_data.115ebc3c67d1.0000000000000014:c [read 780288~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:53.388 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:05:53.388 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.48430479399999998, 2024-09-14T09:05:53.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016069904999999999, 2024-09-14T09:05:53.388 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.388 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.388 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.388 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.388 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.388 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.388 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29322 2024-09-14T09:05:53.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.389 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:05:53.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:05:53.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:05:53.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9499999999999998e-07 2024-09-14T09:05:53.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819550+0000", 2024-09-14T09:05:53.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8309999999999999e-06 2024-09-14T09:05:53.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819556+0000", 2024-09-14T09:05:53.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1299999999999998e-06 2024-09-14T09:05:53.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819561+0000", 2024-09-14T09:05:53.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.707e-06 2024-09-14T09:05:53.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835500+0000", 2024-09-14T09:05:53.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015939456000000001 2024-09-14T09:05:53.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835566+0000", 2024-09-14T09:05:53.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5285999999999998e-05 2024-09-14T09:05:53.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835618+0000", 2024-09-14T09:05:53.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2099999999999999e-05 2024-09-14T09:05:53.392 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.392 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.392 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.392 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.392 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.392 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:53.394 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:05:53.395 INFO:teuthology.orchestra.run.smithi176.stdout:1035 punch from 0x66f8fd6 to 0x6700f87, (0x7fb1 bytes) 2024-09-14T09:05:53.395 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:53.396 INFO:teuthology.orchestra.run.smithi176.stdout:1038 punch from 0x4138e7b to 0x413bf77, (0x30fc bytes) 2024-09-14T09:05:53.396 INFO:teuthology.orchestra.run.smithi176.stdout:1039 writesame 0xd850c88 thru 0xd859f2e data_size 0x1(0x92a7 bytes) 2024-09-14T09:05:53.396 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:53.396 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:53.396 INFO:teuthology.orchestra.run.smithi176.stdout:1043 read 0xb765d20 thru 0xb76b58f (0x5870 bytes) 2024-09-14T09:05:53.396 INFO:teuthology.orchestra.run.smithi176.stdout:1044 punch from 0x1b5a44 to 0x1bdcad, (0x8269 bytes) 2024-09-14T09:05:53.396 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:53.396 INFO:teuthology.orchestra.run.smithi176.stdout:1047 trunc from 0xe4b74c6 to 0xbcc01dd 2024-09-14T09:05:53.396 INFO:teuthology.orchestra.run.smithi176.stdout:1049 write 0x1994d19 thru 0x199f581 (0xa869 bytes) 2024-09-14T09:05:53.396 INFO:teuthology.orchestra.run.smithi176.stdout:1050 writesame 0x3aaef37 thru 0x3aba3eb data_size 0x1(0xb4b5 bytes) 2024-09-14T09:05:53.397 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:53.395+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:53.425 INFO:teuthology.orchestra.run.smithi176.stdout:1052 writesame 0x13e1214 thru 0x13e7ba6 data_size 0x27(0x6993 bytes) 2024-09-14T09:05:53.430 INFO:teuthology.orchestra.run.smithi176.stdout:1053 write 0x9b7942a thru 0x9b7b2bf (0x1e96 bytes) 2024-09-14T09:05:53.441 INFO:teuthology.orchestra.run.smithi176.stdout:1054 writesame 0xbb3acb6 thru 0xbb457a1 data_size 0x2(0xaaec bytes) 2024-09-14T09:05:53.459 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:53.459 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:53.459 INFO:teuthology.orchestra.run.smithi176.stdout:1058 read 0x15e4fd2 thru 0x15e8d36 (0x3d65 bytes) 2024-09-14T09:05:53.469 INFO:teuthology.orchestra.run.smithi176.stdout:1059 writesame 0xd23598f thru 0xd241846 data_size 0x44(0xbeb8 bytes) 2024-09-14T09:05:53.469 INFO:teuthology.orchestra.run.smithi176.stdout:1061 write 0xc581e64 thru 0xc58d0e1 (0xb27e bytes) 2024-09-14T09:05:53.489 INFO:teuthology.orchestra.run.smithi176.stdout:1062 write 0x2524b63 thru 0x2527550 (0x29ee bytes) 2024-09-14T09:05:53.494 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:53.492+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:53.503 INFO:teuthology.orchestra.run.smithi176.stdout:1063 punch from 0xe48e73 to 0xe5459d, (0xb72a bytes) 2024-09-14T09:05:53.503 INFO:teuthology.orchestra.run.smithi176.stdout:1064 read 0x8db031c thru 0x8db037b (0x60 bytes) 2024-09-14T09:05:53.516 INFO:teuthology.orchestra.run.smithi176.stdout:1065 trunc from 0xd241847 to 0x2743279 2024-09-14T09:05:53.533 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:53.534 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:53.534 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:53.534 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:53.541 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:53.541 INFO:teuthology.orchestra.run.smithi176.stdout:1068 read 0x10d9c47 thru 0x10e91f9 (0xf5b3 bytes) 2024-09-14T09:05:53.543 INFO:teuthology.orchestra.run.smithi176.stdout:1069 read 0x2221b8c thru 0x222bc25 (0xa09a bytes) 2024-09-14T09:05:53.551 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:05:53.562 INFO:teuthology.orchestra.run.smithi176.stdout:1074 write 0xb76ec05 thru 0xb7729b3 (0x3daf bytes) 2024-09-14T09:05:53.587 INFO:teuthology.orchestra.run.smithi176.stdout:1075 trunc from 0xb7729b4 to 0x20a3761 2024-09-14T09:05:53.595 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:53.592+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:53.605 INFO:teuthology.orchestra.run.smithi176.stdout:1076 read 0xa88a41 thru 0xa8ba96 (0x3056 bytes) 2024-09-14T09:05:53.621 INFO:teuthology.orchestra.run.smithi176.stdout:1077 trunc from 0x20a3761 to 0xce79060 2024-09-14T09:05:53.628 INFO:teuthology.orchestra.run.smithi176.stdout:1079 punch from 0xc92edbe to 0xc939028, (0xa26a bytes) 2024-09-14T09:05:53.644 INFO:teuthology.orchestra.run.smithi176.stdout:1080 trunc from 0xce79060 to 0x4b846e 2024-09-14T09:05:53.677 INFO:teuthology.orchestra.run.smithi176.stdout:1081 read 0x34e2aa thru 0x356b18 (0x886f bytes) 2024-09-14T09:05:53.690 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:53.690 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:53.690 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:53.691 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:53.691 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:53.691 INFO:teuthology.orchestra.run.smithi176.stdout:1083 writesame 0x2e21361 thru 0x2e2b04e data_size 0x35(0x9cee bytes) 2024-09-14T09:05:53.695 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:53.695+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:53.707 INFO:teuthology.orchestra.run.smithi176.stdout:1084 write 0x7b7942f thru 0x7b79c1a (0x7ec bytes) 2024-09-14T09:05:53.709 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:05:53.715 INFO:teuthology.orchestra.run.smithi176.stdout:1085 read 0x356b040 thru 0x35745b4 (0x9575 bytes) 2024-09-14T09:05:53.719 INFO:teuthology.orchestra.run.smithi176.stdout:1087 trunc from 0x7b79c1b to 0x4795902 2024-09-14T09:05:53.741 INFO:teuthology.orchestra.run.smithi176.stdout:1088 trunc from 0x4795902 to 0x8e5918d 2024-09-14T09:05:53.758 INFO:teuthology.orchestra.run.smithi176.stdout:1089 writesame 0xb472ddd thru 0xb482be5 data_size 0x1(0xfe09 bytes) 2024-09-14T09:05:53.768 INFO:teuthology.orchestra.run.smithi176.stdout:1090 write 0xcf9ad19 thru 0xcfa2439 (0x7721 bytes) 2024-09-14T09:05:53.789 INFO:teuthology.orchestra.run.smithi176.stdout:1093 trunc from 0xcfa243a to 0xb5b12a1 2024-09-14T09:05:53.796 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:53.799+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:53.807 INFO:teuthology.orchestra.run.smithi176.stdout:1095 write 0xba6507 thru 0xbac4a8 (0x5fa2 bytes) 2024-09-14T09:05:53.813 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:05:53.831 INFO:teuthology.orchestra.run.smithi176.stdout:1096 trunc from 0xb5b12a1 to 0xd0c028d 2024-09-14T09:05:53.838 INFO:teuthology.orchestra.run.smithi176.stdout:1097 read 0x285059f thru 0x285b653 (0xb0b5 bytes) 2024-09-14T09:05:53.845 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:53.845 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:05:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:05:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.762299687999999, 2024-09-14T09:05:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:05:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.846 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.847 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:05:53.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.847 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:05:53.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:05:53.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:05:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:05:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:05:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:05:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:05:53.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:05:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:05:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:05:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:05:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:05:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:05:53.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:05:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:05:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:05:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:05:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:05:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:05:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:05:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:05:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:05:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:05:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.762245029999999, 2024-09-14T09:05:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:05:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.851 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:05:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:05:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:05:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:05:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:05:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:05:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:05:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:05:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:05:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:05:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:05:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:05:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:05:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:05:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:05:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:05:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:05:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:05:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:05:53.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:05:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:05:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:05:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:05:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:05:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.762139655999999, 2024-09-14T09:05:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:05:53.856 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:05:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:05:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:05:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:05:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:05:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:05:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:05:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:05:53.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:05:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:05:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:05:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:05:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:05:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:05:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:05:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:05:53.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:05:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:05:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:05:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:05:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:05:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:05:53.860 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:05:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:05:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.761782891999999, 2024-09-14T09:05:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:05:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.861 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:05:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:05:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:05:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:05:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:05:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:05:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:05:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:05:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:05:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:05:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:05:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:05:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:05:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:05:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:05:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:05:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:05:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:05:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:05:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:05:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:05:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:05:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.865 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:05:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:05:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.761698154000001, 2024-09-14T09:05:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:05:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:05:53.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:05:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:05:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:05:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:05:53.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:05:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:05:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:05:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:05:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:05:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:05:53.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:05:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:05:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:05:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:05:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:05:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:05:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:05:53.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:05:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:05:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:05:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:05:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:05:53.870 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:05:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.761610468000001, 2024-09-14T09:05:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:05:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:05:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:05:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:05:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:05:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:05:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:05:53.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:05:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:05:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:05:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:05:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:05:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:05:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:05:53.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:05:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:05:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:05:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:05:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:05:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:05:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:05:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:05:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:05:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18388 3.18 3:19e2be55:::rbd_data.115e3abd3b2c.0000000000000021:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:05:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:05:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.625697156999999, 2024-09-14T09:05:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042293648000000003, 2024-09-14T09:05:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.875 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18388 2024-09-14T09:05:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:05:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:05:53.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220236+0000", 2024-09-14T09:05:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1339999999999998e-06 2024-09-14T09:05:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220244+0000", 2024-09-14T09:05:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1950000000000003e-06 2024-09-14T09:05:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220262+0000", 2024-09-14T09:05:53.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8017e-05 2024-09-14T09:05:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220270+0000", 2024-09-14T09:05:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9480000000000008e-06 2024-09-14T09:05:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220357+0000", 2024-09-14T09:05:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7185e-05 2024-09-14T09:05:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220576+0000", 2024-09-14T09:05:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021912200000000001 2024-09-14T09:05:53.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:05:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.222435+0000", 2024-09-14T09:05:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018592350000000001 2024-09-14T09:05:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262439+0000", 2024-09-14T09:05:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040003368999999997 2024-09-14T09:05:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262452+0000", 2024-09-14T09:05:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2941e-05 2024-09-14T09:05:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262490+0000", 2024-09-14T09:05:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8478e-05 2024-09-14T09:05:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262527+0000", 2024-09-14T09:05:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7024000000000003e-05 2024-09-14T09:05:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18388 3.18 e38/28 ondisk, result = 0)", 2024-09-14T09:05:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.227795+0000", 2024-09-14T09:05:53.880 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.618135514, 2024-09-14T09:05:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034648832999999997, 2024-09-14T09:05:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227795+0000", 2024-09-14T09:05:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227795+0000", 2024-09-14T09:05:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227796+0000", 2024-09-14T09:05:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.879999999999999e-07 2024-09-14T09:05:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227801+0000", 2024-09-14T09:05:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4909999999999999e-06 2024-09-14T09:05:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227815+0000", 2024-09-14T09:05:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4905e-05 2024-09-14T09:05:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227824+0000", 2024-09-14T09:05:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7770000000000008e-06 2024-09-14T09:05:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262425+0000", 2024-09-14T09:05:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034601109999999997 2024-09-14T09:05:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262434+0000", 2024-09-14T09:05:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5250000000000005e-06 2024-09-14T09:05:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262444+0000", 2024-09-14T09:05:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0037000000000001e-05 2024-09-14T09:05:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:19543 3.5 e39/23 3:a536ff9b:::rbd_data.115e4c71d6e8.0000000000000002:head v 39'8, mlcod=39'6)", 2024-09-14T09:05:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:05:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.3228424969999999, 2024-09-14T09:05:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.039280019999999999, 2024-09-14T09:05:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:05:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:05:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523090+0000", 2024-09-14T09:05:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.773e-06 2024-09-14T09:05:53.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537290+0000", 2024-09-14T09:05:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014200007000000001 2024-09-14T09:05:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537317+0000", 2024-09-14T09:05:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6704999999999999e-05 2024-09-14T09:05:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537328+0000", 2024-09-14T09:05:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1299e-05 2024-09-14T09:05:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537417+0000", 2024-09-14T09:05:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8869999999999994e-05 2024-09-14T09:05:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537445+0000", 2024-09-14T09:05:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8377999999999998e-05 2024-09-14T09:05:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562324+0000", 2024-09-14T09:05:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024879307999999999 2024-09-14T09:05:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562368+0000", 2024-09-14T09:05:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3680000000000002e-05 2024-09-14T09:05:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20736 3.2 3:45030d14:::rbd_data.115e889d4809.000000000000027a:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.099577+0000", 2024-09-14T09:05:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.7463534540000003, 2024-09-14T09:05:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026646762000000001, 2024-09-14T09:05:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:53.888 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20736 2024-09-14T09:05:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099577+0000", 2024-09-14T09:05:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099577+0000", 2024-09-14T09:05:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099579+0000", 2024-09-14T09:05:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5009999999999999e-06 2024-09-14T09:05:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099585+0000", 2024-09-14T09:05:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2870000000000001e-06 2024-09-14T09:05:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099602+0000", 2024-09-14T09:05:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7176e-05 2024-09-14T09:05:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099611+0000", 2024-09-14T09:05:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7010000000000002e-06 2024-09-14T09:05:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099664+0000", 2024-09-14T09:05:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3183000000000002e-05 2024-09-14T09:05:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099793+0000", 2024-09-14T09:05:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012945099999999999 2024-09-14T09:05:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:05:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.100819+0000", 2024-09-14T09:05:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010252149999999999 2024-09-14T09:05:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.125929+0000", 2024-09-14T09:05:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025110072000000001 2024-09-14T09:05:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.126034+0000", 2024-09-14T09:05:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105655 2024-09-14T09:05:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.126141+0000", 2024-09-14T09:05:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107069 2024-09-14T09:05:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.126224+0000", 2024-09-14T09:05:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2452000000000003e-05 2024-09-14T09:05:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31015 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.062742+0000", 2024-09-14T09:05:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.78318812599999998, 2024-09-14T09:05:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027690059999999999, 2024-09-14T09:05:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.894 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31015 2024-09-14T09:05:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062742+0000", 2024-09-14T09:05:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062742+0000", 2024-09-14T09:05:53.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062743+0000", 2024-09-14T09:05:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.51e-07 2024-09-14T09:05:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062745+0000", 2024-09-14T09:05:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0480000000000001e-06 2024-09-14T09:05:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062751+0000", 2024-09-14T09:05:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3929999999999996e-06 2024-09-14T09:05:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062757+0000", 2024-09-14T09:05:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5130000000000004e-06 2024-09-14T09:05:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.088137+0000", 2024-09-14T09:05:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025380494999999999 2024-09-14T09:05:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.088178+0000", 2024-09-14T09:05:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0963000000000001e-05 2024-09-14T09:05:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.090433+0000", 2024-09-14T09:05:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0022541969999999999 2024-09-14T09:05:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31020 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.062866+0000", 2024-09-14T09:05:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.78306484600000004, 2024-09-14T09:05:53.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026470701999999999, 2024-09-14T09:05:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31020 2024-09-14T09:05:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062866+0000", 2024-09-14T09:05:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062866+0000", 2024-09-14T09:05:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062866+0000", 2024-09-14T09:05:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8500000000000002e-07 2024-09-14T09:05:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062868+0000", 2024-09-14T09:05:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.12e-06 2024-09-14T09:05:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062875+0000", 2024-09-14T09:05:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4139999999999999e-06 2024-09-14T09:05:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062880+0000", 2024-09-14T09:05:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2499999999999997e-06 2024-09-14T09:05:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089197+0000", 2024-09-14T09:05:53.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026317317 2024-09-14T09:05:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089264+0000", 2024-09-14T09:05:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6224999999999997e-05 2024-09-14T09:05:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089336+0000", 2024-09-14T09:05:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2891000000000004e-05 2024-09-14T09:05:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.902 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31021 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.062888+0000", 2024-09-14T09:05:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.78304304800000002, 2024-09-14T09:05:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026614331000000001, 2024-09-14T09:05:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31021 2024-09-14T09:05:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062888+0000", 2024-09-14T09:05:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062888+0000", 2024-09-14T09:05:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062888+0000", 2024-09-14T09:05:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9900000000000001e-07 2024-09-14T09:05:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062890+0000", 2024-09-14T09:05:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1500000000000002e-06 2024-09-14T09:05:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062905+0000", 2024-09-14T09:05:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5170999999999999e-05 2024-09-14T09:05:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062911+0000", 2024-09-14T09:05:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4539999999999998e-06 2024-09-14T09:05:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089360+0000", 2024-09-14T09:05:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026448979000000001 2024-09-14T09:05:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089424+0000", 2024-09-14T09:05:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4725000000000001e-05 2024-09-14T09:05:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089502+0000", 2024-09-14T09:05:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7353000000000003e-05 2024-09-14T09:05:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31022 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.062919+0000", 2024-09-14T09:05:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.78301140800000002, 2024-09-14T09:05:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026754692, 2024-09-14T09:05:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31022 2024-09-14T09:05:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062919+0000", 2024-09-14T09:05:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062919+0000", 2024-09-14T09:05:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062920+0000", 2024-09-14T09:05:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2099999999999997e-07 2024-09-14T09:05:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062922+0000", 2024-09-14T09:05:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.452e-06 2024-09-14T09:05:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062929+0000", 2024-09-14T09:05:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4439999999999996e-06 2024-09-14T09:05:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062958+0000", 2024-09-14T09:05:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8943999999999999e-05 2024-09-14T09:05:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089526+0000", 2024-09-14T09:05:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026568556 2024-09-14T09:05:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089600+0000", 2024-09-14T09:05:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3506999999999999e-05 2024-09-14T09:05:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089674+0000", 2024-09-14T09:05:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4268000000000001e-05 2024-09-14T09:05:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31023 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 264192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.062978+0000", 2024-09-14T09:05:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.78295219500000002, 2024-09-14T09:05:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026871328999999999, 2024-09-14T09:05:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31023 2024-09-14T09:05:53.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062978+0000", 2024-09-14T09:05:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062978+0000", 2024-09-14T09:05:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062979+0000", 2024-09-14T09:05:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6499999999999999e-07 2024-09-14T09:05:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062982+0000", 2024-09-14T09:05:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8650000000000001e-06 2024-09-14T09:05:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062990+0000", 2024-09-14T09:05:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9889999999999992e-06 2024-09-14T09:05:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063028+0000", 2024-09-14T09:05:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8154000000000001e-05 2024-09-14T09:05:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089700+0000", 2024-09-14T09:05:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026672548000000001 2024-09-14T09:05:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089772+0000", 2024-09-14T09:05:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1829999999999995e-05 2024-09-14T09:05:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089850+0000", 2024-09-14T09:05:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7378000000000004e-05 2024-09-14T09:05:53.914 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31024 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 282624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.063039+0000", 2024-09-14T09:05:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.78289149999999996, 2024-09-14T09:05:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027079282, 2024-09-14T09:05:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31024 2024-09-14T09:05:53.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063039+0000", 2024-09-14T09:05:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063039+0000", 2024-09-14T09:05:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063040+0000", 2024-09-14T09:05:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8299999999999997e-07 2024-09-14T09:05:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063043+0000", 2024-09-14T09:05:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9610000000000001e-06 2024-09-14T09:05:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063058+0000", 2024-09-14T09:05:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5529999999999999e-05 2024-09-14T09:05:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063065+0000", 2024-09-14T09:05:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.596e-06 2024-09-14T09:05:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089877+0000", 2024-09-14T09:05:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026812373 2024-09-14T09:05:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089949+0000", 2024-09-14T09:05:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2014000000000005e-05 2024-09-14T09:05:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.090118+0000", 2024-09-14T09:05:53.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016922500000000001 2024-09-14T09:05:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31025 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 292864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.063074+0000", 2024-09-14T09:05:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.78285685900000002, 2024-09-14T09:05:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027206443, 2024-09-14T09:05:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.919 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31025 2024-09-14T09:05:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063074+0000", 2024-09-14T09:05:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063074+0000", 2024-09-14T09:05:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063074+0000", 2024-09-14T09:05:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.68e-07 2024-09-14T09:05:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063077+0000", 2024-09-14T09:05:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.853e-06 2024-09-14T09:05:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063084+0000", 2024-09-14T09:05:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0419999999999997e-06 2024-09-14T09:05:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063089+0000", 2024-09-14T09:05:53.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1150000000000002e-06 2024-09-14T09:05:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.090144+0000", 2024-09-14T09:05:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027054974999999998 2024-09-14T09:05:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.090211+0000", 2024-09-14T09:05:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6832e-05 2024-09-14T09:05:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.090280+0000", 2024-09-14T09:05:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9058000000000002e-05 2024-09-14T09:05:53.922 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31278 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.072728+0000", 2024-09-14T09:05:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.77320223799999999, 2024-09-14T09:05:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026492094000000001, 2024-09-14T09:05:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31278 2024-09-14T09:05:53.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072728+0000", 2024-09-14T09:05:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072728+0000", 2024-09-14T09:05:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072729+0000", 2024-09-14T09:05:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.06e-07 2024-09-14T09:05:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072732+0000", 2024-09-14T09:05:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.269e-06 2024-09-14T09:05:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072740+0000", 2024-09-14T09:05:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5780000000000001e-06 2024-09-14T09:05:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072744+0000", 2024-09-14T09:05:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5909999999999998e-06 2024-09-14T09:05:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.098888+0000", 2024-09-14T09:05:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026143816 2024-09-14T09:05:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.099097+0000", 2024-09-14T09:05:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000208624 2024-09-14T09:05:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.099220+0000", 2024-09-14T09:05:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012350999999999999 2024-09-14T09:05:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31279 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 274432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.072774+0000", 2024-09-14T09:05:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.77315659599999997, 2024-09-14T09:05:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026662447999999998, 2024-09-14T09:05:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.927 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31279 2024-09-14T09:05:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072774+0000", 2024-09-14T09:05:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072774+0000", 2024-09-14T09:05:53.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072775+0000", 2024-09-14T09:05:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0900000000000001e-07 2024-09-14T09:05:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072778+0000", 2024-09-14T09:05:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7340000000000002e-06 2024-09-14T09:05:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072789+0000", 2024-09-14T09:05:53.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0287e-05 2024-09-14T09:05:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072795+0000", 2024-09-14T09:05:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7390000000000002e-06 2024-09-14T09:05:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.099278+0000", 2024-09-14T09:05:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026482887 2024-09-14T09:05:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.099353+0000", 2024-09-14T09:05:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5013999999999997e-05 2024-09-14T09:05:53.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.099436+0000", 2024-09-14T09:05:53.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3078000000000007e-05 2024-09-14T09:05:53.931 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.931 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.931 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.931 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31280 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 284672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:53.931 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.073228+0000", 2024-09-14T09:05:53.931 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.77270265299999996, 2024-09-14T09:05:53.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026419582, 2024-09-14T09:05:53.931 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:53.931 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:53.932 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:53.932 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:53.932 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:53.932 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:53.932 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31280 2024-09-14T09:05:53.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.932 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:53.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:53.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.073228+0000", 2024-09-14T09:05:53.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:53.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.073228+0000", 2024-09-14T09:05:53.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:53.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:53.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.073229+0000", 2024-09-14T09:05:53.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.063e-06 2024-09-14T09:05:53.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:53.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.073234+0000", 2024-09-14T09:05:53.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.78e-06 2024-09-14T09:05:53.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:53.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.073246+0000", 2024-09-14T09:05:53.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1753000000000001e-05 2024-09-14T09:05:53.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:53.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.073252+0000", 2024-09-14T09:05:53.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3879999999999997e-06 2024-09-14T09:05:53.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:53.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.099490+0000", 2024-09-14T09:05:53.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026238227999999999 2024-09-14T09:05:53.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:53.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.099565+0000", 2024-09-14T09:05:53.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5238e-05 2024-09-14T09:05:53.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:53.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:53.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:53.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.099647+0000", 2024-09-14T09:05:53.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2132000000000006e-05 2024-09-14T09:05:53.935 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.935 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.935 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.935 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:53.935 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:53.935 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:53.936 INFO:teuthology.orchestra.run.smithi176.stdout:1098 write 0xde59b99 thru 0xde5d459 (0x38c1 bytes) 2024-09-14T09:05:53.937 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:05:53.938 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:53.938 INFO:teuthology.orchestra.run.smithi176.stdout:1102 writesame 0xeb2e06 thru 0xebad75 data_size 0x8(0x7f70 bytes) 2024-09-14T09:05:53.938 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:53.938 INFO:teuthology.orchestra.run.smithi176.stdout:1104 punch from 0x1191f7c to 0x1197b87, (0x5c0b bytes) 2024-09-14T09:05:53.938 INFO:teuthology.orchestra.run.smithi176.stdout:1105 trunc from 0xde5d45a to 0x5fdfbd7 2024-09-14T09:05:53.938 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:53.938 INFO:teuthology.orchestra.run.smithi176.stdout:1108 write 0x90790e3 thru 0x9086fee (0xdf0c bytes) 2024-09-14T09:05:53.939 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:53.939+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:53.945 INFO:teuthology.orchestra.run.smithi176.stdout:1109 read 0x8995547 thru 0x89a0f64 (0xba1e bytes) 2024-09-14T09:05:53.946 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:53.946 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:53.946 INFO:teuthology.orchestra.run.smithi176.stdout:1112 writesame 0x886849a thru 0x88752a8 data_size 0x1d(0xce0f bytes) 2024-09-14T09:05:53.965 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:05:53.965 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:05:53.965 INFO:teuthology.orchestra.run.smithi176.stdout:1114 read 0x2da005c thru 0x2da79b3 (0x7958 bytes) 2024-09-14T09:05:53.969 INFO:teuthology.orchestra.run.smithi176.stdout:1115 trunc from 0x9086fef to 0x6b4279d 2024-09-14T09:05:53.983 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:53.983 INFO:teuthology.orchestra.run.smithi176.stdout:1117 writesame 0x276d010 thru 0x277aff8 data_size 0x63(0xdfe9 bytes) 2024-09-14T09:05:53.985 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:05:53.985 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:05:53.997 INFO:teuthology.orchestra.run.smithi176.stdout:1118 write 0x612cd72 thru 0x613ca7d (0xfd0c bytes) 2024-09-14T09:05:54.004 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:05:54.004 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:05:54.014 INFO:teuthology.orchestra.run.smithi176.stdout:1120 read 0x49ce19 thru 0x4a85d1 (0xb7b9 bytes) 2024-09-14T09:05:54.015 INFO:teuthology.orchestra.run.smithi176.stdout:1122 punch from 0x314388a to 0x31516c6, (0xde3c bytes) 2024-09-14T09:05:54.015 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:54.015 INFO:teuthology.orchestra.run.smithi176.stdout:1124 read 0x1d3966c thru 0x1d410d0 (0x7a65 bytes) 2024-09-14T09:05:54.016 INFO:teuthology.orchestra.run.smithi176.stdout:1127 writesame 0x5192bb thru 0x522562 data_size 0x68(0x92a8 bytes) 2024-09-14T09:05:54.020 INFO:teuthology.orchestra.run.smithi176.stdout:1128 read 0x5e337cb thru 0x5e3d77c (0x9fb2 bytes) 2024-09-14T09:05:54.020 INFO:teuthology.orchestra.run.smithi176.stdout:1130 read 0x1a7125d thru 0x1a76a6b (0x580f bytes) 2024-09-14T09:05:54.020 INFO:teuthology.orchestra.run.smithi176.stdout:1131 writesame 0x2e08ae2 thru 0x2e151ee data_size 0x1(0xc70d bytes) 2024-09-14T09:05:54.025 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:05:54.025 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:05:54.037 INFO:teuthology.orchestra.run.smithi176.stdout:1132 read 0x3aa2ecf thru 0x3aa9087 (0x61b9 bytes) 2024-09-14T09:05:54.037 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:54.032+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:54.049 INFO:teuthology.orchestra.run.smithi176.stdout:1134 trunc from 0x6b4279d to 0xe876ce6 2024-09-14T09:05:54.052 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:05:54.052 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:05:54.055 INFO:teuthology.orchestra.run.smithi176.stdout:1136 write 0xd36e032 thru 0xd3787a9 (0xa778 bytes) 2024-09-14T09:05:54.065 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:54.065 INFO:teuthology.orchestra.run.smithi176.stdout:1138 read 0x4088bf1 thru 0x4093c1e (0xb02e bytes) 2024-09-14T09:05:54.065 INFO:teuthology.orchestra.run.smithi176.stdout:1139 writesame 0x8600c08 thru 0x8603d87 data_size 0x60(0x3180 bytes) 2024-09-14T09:05:54.073 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:05:54.073 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:05:54.074 INFO:teuthology.orchestra.run.smithi176.stdout:1140 write 0x5dbd188 thru 0x5dbf0e4 (0x1f5d bytes) 2024-09-14T09:05:54.086 INFO:teuthology.orchestra.run.smithi176.stdout:1141 writesame 0x2be8230 thru 0x2bf450c data_size 0xb(0xc2dd bytes) 2024-09-14T09:05:54.089 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:54.089 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:54.093 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:05:54.094 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:05:54.097 INFO:teuthology.orchestra.run.smithi176.stdout:1142 write 0xe1c9163 thru 0xe1d0545 (0x73e3 bytes) 2024-09-14T09:05:54.106 INFO:teuthology.orchestra.run.smithi176.stdout:1145 write 0x35b93cd thru 0x35c1639 (0x826d bytes) 2024-09-14T09:05:54.108 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:05:54.114 INFO:teuthology.orchestra.run.smithi176.stdout:1146 punch from 0x4d13e17 to 0x4d1a1f7, (0x63e0 bytes) 2024-09-14T09:05:54.115 INFO:teuthology.orchestra.run.smithi176.stdout:1148 read 0x3b803c9 thru 0x3b80b32 (0x76a bytes) 2024-09-14T09:05:54.115 INFO:teuthology.orchestra.run.smithi176.stdout:1149 write 0xa8f9b8e thru 0xa900544 (0x69b7 bytes) 2024-09-14T09:05:54.116 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:05:54.117 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:05:54.129 INFO:teuthology.orchestra.run.smithi176.stdout:1150 read 0x4434222 thru 0x443eb0a (0xa8e9 bytes) 2024-09-14T09:05:54.129 INFO:teuthology.orchestra.run.smithi176.stdout:1151 write 0x4ebd5e3 thru 0x4ec7101 (0x9b1f bytes) 2024-09-14T09:05:54.140 INFO:teuthology.orchestra.run.smithi176.stdout:1152 trunc from 0xe876ce6 to 0xea14a82 2024-09-14T09:05:54.147 INFO:teuthology.orchestra.run.smithi176.stdout:1153 trunc from 0xea14a82 to 0xd6893c7 2024-09-14T09:05:54.156 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:54.159+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:54.158 INFO:teuthology.orchestra.run.smithi176.stdout:1154 write 0x65c6943 thru 0x65cd87c (0x6f3a bytes) 2024-09-14T09:05:54.168 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:54.169 INFO:teuthology.orchestra.run.smithi176.stdout:1156 read 0xb7e0f3 thru 0xb899db (0xb8e9 bytes) 2024-09-14T09:05:54.170 INFO:teuthology.orchestra.run.smithi176.stdout:1160 punch from 0xad54f3d to 0xad627f6, (0xd8b9 bytes) 2024-09-14T09:05:54.170 INFO:teuthology.orchestra.run.smithi176.stdout:1161 read 0x26781ab thru 0x268298f (0xa7e5 bytes) 2024-09-14T09:05:54.171 INFO:teuthology.orchestra.run.smithi176.stdout:1162 read 0x3441232 thru 0x344f230 (0xdfff bytes) 2024-09-14T09:05:54.171 INFO:teuthology.orchestra.run.smithi176.stdout:1164 read 0xf5737c thru 0xf5ca61 (0x56e6 bytes) 2024-09-14T09:05:54.171 INFO:teuthology.orchestra.run.smithi176.stdout:1165 write 0xbb60c5f thru 0xbb6bf2f (0xb2d1 bytes) 2024-09-14T09:05:54.181 INFO:teuthology.orchestra.run.smithi176.stdout:1166 write 0x3478532 thru 0x347a15c (0x1c2b bytes) 2024-09-14T09:05:54.187 INFO:teuthology.orchestra.run.smithi176.stdout:1167 read 0xd4feb96 thru 0xd50127d (0x26e8 bytes) 2024-09-14T09:05:54.187 INFO:teuthology.orchestra.run.smithi176.stdout:1168 read 0x9e64475 thru 0x9e705db (0xc167 bytes) 2024-09-14T09:05:54.196 INFO:teuthology.orchestra.run.smithi176.stdout:1169 writesame 0xe4a0470 thru 0xe4ad4a5 data_size 0x1d(0xd036 bytes) 2024-09-14T09:05:54.196 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:54.196 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:54.196 INFO:teuthology.orchestra.run.smithi176.stdout:1172 punch from 0xd1b4c72 to 0xd1bdfdf, (0x936d bytes) 2024-09-14T09:05:54.196 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:54.196 INFO:teuthology.orchestra.run.smithi176.stdout:1174 writesame 0x6773b82 thru 0x678255b data_size 0x25(0xe9da bytes) 2024-09-14T09:05:54.213 INFO:teuthology.orchestra.run.smithi176.stdout:1175 trunc from 0xe4ad4a6 to 0x50e047e 2024-09-14T09:05:54.239 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:54.239+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:54.246 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:54.246 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:54.249 INFO:teuthology.orchestra.run.smithi176.stdout:1176 write 0x92b2643 thru 0x92ba74f (0x810d bytes) 2024-09-14T09:05:54.258 INFO:teuthology.orchestra.run.smithi176.stdout:1177 clone 11 order 23 su 65536 sc 14 2024-09-14T09:05:54.263 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:05:54.352 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:54.355+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:54.420 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:54.420 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:54.434 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:05:54.476 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:54.479+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:54.541 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:54.543+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:54.599 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:54.600 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:54.614 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:05:54.642 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:54.640+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:54.743 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:54.744+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:54.779 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:54.779 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:54.793 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:05:54.844 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:54.840+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:54.916 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:54.916 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:54.930 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:05:54.976 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:54.979+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:55.011 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T09:05:55.051 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:55.051 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:55.058 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:55.056+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:55.066 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:05:55.110 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone10 from 0x92ba750 (overlap 0x4b846e) to 0x0 2024-09-14T09:05:55.152 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:55.155+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:55.199 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-14T09:05:55.238 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:55.238 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:55.248 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:55.244+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:55.253 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:05:55.349 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:55.351+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:55.412 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:55.412 INFO:teuthology.orchestra.run.smithi176.stdout:1179 write 0x7b7e4c9 thru 0x7b80622 (0x215a bytes) 2024-09-14T09:05:55.420 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:55.420 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:55.429 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:55.434 INFO:teuthology.orchestra.run.smithi176.stdout:1182 trunc from 0x92ba750 to 0x64e21b7 2024-09-14T09:05:55.436 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:05:55.449 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:55.444+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:55.550 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:55.551+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:55.650 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:55.648+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:55.656 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:55.656 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:55.659 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:55.659 INFO:teuthology.orchestra.run.smithi176.stdout:1184 punch from 0x1ce425f to 0x1cf0173, (0xbf14 bytes) 2024-09-14T09:05:55.660 INFO:teuthology.orchestra.run.smithi176.stdout:1185 punch from 0x2b3d5e6 to 0x2b4206b, (0x4a85 bytes) 2024-09-14T09:05:55.667 INFO:teuthology.orchestra.run.smithi176.stdout:1186 trunc from 0x64e21b7 to 0xbf3410 2024-09-14T09:05:55.672 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:05:55.687 INFO:teuthology.orchestra.run.smithi176.stdout:1187 write 0x230b699 thru 0x230cb4e (0x14b6 bytes) 2024-09-14T09:05:55.709 INFO:teuthology.orchestra.run.smithi176.stdout:1188 write 0xb0bf67f thru 0xb0cc65f (0xcfe1 bytes) 2024-09-14T09:05:55.713 INFO:teuthology.orchestra.run.smithi176.stdout:1189 write 0x1b9b16f thru 0x1ba3e3f (0x8cd1 bytes) 2024-09-14T09:05:55.732 INFO:teuthology.orchestra.run.smithi176.stdout:1190 trunc from 0xb0cc660 to 0x1735f22 2024-09-14T09:05:55.760 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:55.763+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:55.769 INFO:teuthology.orchestra.run.smithi176.stdout:1191 write 0x68e6df7 thru 0x68f4715 (0xd91f bytes) 2024-09-14T09:05:55.793 INFO:teuthology.orchestra.run.smithi176.stdout:1192 write 0xc9276f7 thru 0xc92d50d (0x5e17 bytes) 2024-09-14T09:05:55.798 INFO:teuthology.orchestra.run.smithi176.stdout:1193 read 0x5ae5ae6 thru 0x5af482a (0xed45 bytes) 2024-09-14T09:05:55.820 INFO:teuthology.orchestra.run.smithi176.stdout:1194 trunc from 0xc92d50e to 0x1ecd8b1 2024-09-14T09:05:55.846 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:55.846 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:55.852 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:55.848+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:55.860 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:05:55.872 INFO:teuthology.orchestra.run.smithi176.stdout:1195 write 0x9283204 thru 0x928dd44 (0xab41 bytes) 2024-09-14T09:05:55.877 INFO:teuthology.orchestra.run.smithi176.stdout:1196 punch from 0x585973e to 0x585f2a0, (0x5b62 bytes) 2024-09-14T09:05:55.892 INFO:teuthology.orchestra.run.smithi176.stdout:1197 writesame 0xacbac2c thru 0xacbb0d4 data_size 0x1(0x4a9 bytes) 2024-09-14T09:05:55.892 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:55.907 INFO:teuthology.orchestra.run.smithi176.stdout:1199 trunc from 0xacbb0d5 to 0x42e20ad 2024-09-14T09:05:55.933 INFO:teuthology.orchestra.run.smithi176.stdout:1201 punch from 0x68db57 to 0x6910ba, (0x3563 bytes) 2024-09-14T09:05:55.933 INFO:teuthology.orchestra.run.smithi176.stdout:1202 read 0x32a4018 thru 0x32a43c7 (0x3b0 bytes) 2024-09-14T09:05:55.934 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:55.934 INFO:teuthology.orchestra.run.smithi176.stdout:1204 punch from 0xa82b40 to 0xa8cce1, (0xa1a1 bytes) 2024-09-14T09:05:55.934 INFO:teuthology.orchestra.run.smithi176.stdout:1205 read 0x351ebdb thru 0x352b39b (0xc7c1 bytes) 2024-09-14T09:05:55.934 INFO:teuthology.orchestra.run.smithi176.stdout:1206 read 0x14d06ba thru 0x14dc0eb (0xba32 bytes) 2024-09-14T09:05:55.935 INFO:teuthology.orchestra.run.smithi176.stdout:1208 read 0x1e62f70 thru 0x1e6d216 (0xa2a7 bytes) 2024-09-14T09:05:55.953 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:55.948+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:55.956 INFO:teuthology.orchestra.run.smithi176.stdout:1209 write 0x84f1f07 thru 0x84f267a (0x774 bytes) 2024-09-14T09:05:55.962 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:55.962 INFO:teuthology.orchestra.run.smithi176.stdout:1212 writesame 0x366090a thru 0x36631a8 data_size 0x1(0x289f bytes) 2024-09-14T09:05:55.987 INFO:teuthology.orchestra.run.smithi176.stdout:1214 write 0x79f076c thru 0x79f6082 (0x5917 bytes) 2024-09-14T09:05:55.994 INFO:teuthology.orchestra.run.smithi176.stdout:1215 punch from 0x560ec21 to 0x561beb4, (0xd293 bytes) 2024-09-14T09:05:55.994 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:55.995 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:56.009 INFO:teuthology.orchestra.run.smithi176.stdout:1218 trunc from 0x84f267b to 0x331fd61 2024-09-14T09:05:56.032 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is unset 2024-09-14T09:05:56.034 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:56.034 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:56.040 INFO:teuthology.orchestra.run.smithi176.stdout:1220 read 0x187a650 thru 0x187a992 (0x343 bytes) 2024-09-14T09:05:56.043 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:56.051 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:05:56.053 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:56.055+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:56.054 INFO:teuthology.orchestra.run.smithi176.stdout:1223 write 0x39b15b6 thru 0x39bcaa2 (0xb4ed bytes) 2024-09-14T09:05:56.057 INFO:teuthology.orchestra.run.smithi176.stdout:1225 write 0x544107 thru 0x547258 (0x3152 bytes) 2024-09-14T09:05:56.061 INFO:teuthology.orchestra.run.smithi176.stdout:1226 write 0x274a4e4 thru 0x2751507 (0x7024 bytes) 2024-09-14T09:05:56.064 INFO:teuthology.orchestra.run.smithi176.stdout:1227 punch from 0x10b915c to 0x10c4258, (0xb0fc bytes) 2024-09-14T09:05:56.064 INFO:teuthology.orchestra.run.smithi176.stdout:1228 read 0x3082d69 thru 0x3084aba (0x1d52 bytes) 2024-09-14T09:05:56.066 INFO:teuthology.orchestra.run.smithi176.stdout:1229 punch from 0x14a49cf to 0x14b30ab, (0xe6dc bytes) 2024-09-14T09:05:56.066 INFO:teuthology.orchestra.run.smithi176.stdout:1231 writesame 0x13282bf thru 0x133380f data_size 0x13(0xb551 bytes) 2024-09-14T09:05:56.070 INFO:teuthology.orchestra.run.smithi176.stdout:1234 write 0x307db2 thru 0x310eff (0x914e bytes) 2024-09-14T09:05:56.091 INFO:teuthology.orchestra.run.smithi176.stdout:1235 writesame 0x96da245 thru 0x96dfd39 data_size 0x1(0x5af5 bytes) 2024-09-14T09:05:56.091 INFO:teuthology.orchestra.run.smithi176.stdout:1236 punch from 0x77146b9 to 0x771ef43, (0xa88a bytes) 2024-09-14T09:05:56.091 INFO:teuthology.orchestra.run.smithi176.stdout:1237 write 0x5728499 thru 0x572c544 (0x40ac bytes) 2024-09-14T09:05:56.100 INFO:teuthology.orchestra.run.smithi176.stdout:1240 trunc from 0x96dfd3a to 0xd34c92d 2024-09-14T09:05:56.119 INFO:teuthology.orchestra.run.smithi176.stdout:1241 trunc from 0xd34c92d to 0x55c050d 2024-09-14T09:05:56.138 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:05:56.148 INFO:teuthology.orchestra.run.smithi176.stdout:1242 write 0xa28572e thru 0xa289a98 (0x436b bytes) 2024-09-14T09:05:56.154 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:56.155+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:56.160 INFO:teuthology.orchestra.run.smithi176.stdout:1243 trunc from 0xa289a99 to 0x6feefdc 2024-09-14T09:05:56.190 INFO:teuthology.orchestra.run.smithi176.stdout:1246 writesame 0xa4d19e1 thru 0xa4d33c0 data_size 0x30(0x19e0 bytes) 2024-09-14T09:05:56.190 INFO:teuthology.orchestra.run.smithi176.stdout:1248 write 0x8957156 thru 0x89578a9 (0x754 bytes) 2024-09-14T09:05:56.195 INFO:teuthology.orchestra.run.smithi176.stdout:1249 writesame 0x49d1b9e thru 0x49dea98 data_size 0x1(0xcefb bytes) 2024-09-14T09:05:56.196 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:56.196 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:56.211 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:05:56.254 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:56.252+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:56.287 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:05:56.287 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:05:56.304 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:05:56.304 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:05:56.320 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:05:56.320 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:05:56.337 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:05:56.338 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:05:56.355 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:56.352+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:56.358 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:05:56.358 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:05:56.364 INFO:teuthology.orchestra.run.smithi176.stdout:1250 write 0xb1f3b7c thru 0xb1fe1a4 (0xa629 bytes) 2024-09-14T09:05:56.370 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:56.370 INFO:teuthology.orchestra.run.smithi176.stdout:1254 write 0x96c7701 thru 0x96d2ad1 (0xb3d1 bytes) 2024-09-14T09:05:56.371 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:56.371 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:56.371 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:56.371 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:56.371 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:56.381 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:05:56.381 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:05:56.387 INFO:teuthology.orchestra.run.smithi176.stdout:1258 trunc from 0xb1fe1a5 to 0x52fc484 2024-09-14T09:05:56.389 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:05:56.405 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:05:56.405 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:05:56.419 INFO:teuthology.orchestra.run.smithi176.stdout:1259 writesame 0x8f3c3a3 thru 0x8f4028e data_size 0x2(0x3eec bytes) 2024-09-14T09:05:56.434 INFO:teuthology.orchestra.run.smithi176.stdout:1260 write 0xe070a3d thru 0xe071258 (0x81c bytes) 2024-09-14T09:05:56.436 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:05:56.436 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:05:56.438 INFO:teuthology.orchestra.run.smithi176.stdout:1262 writesame 0x3b6a747 thru 0x3b758d5 data_size 0x1(0xb18f bytes) 2024-09-14T09:05:56.456 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:56.459+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:56.530 INFO:teuthology.orchestra.run.smithi176.stdout:1263 writesame 0x4c37b4e thru 0x4c42702 data_size 0x1(0xabb5 bytes) 2024-09-14T09:05:56.535 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:56.535 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:56.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:56.535 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:56.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.535 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20858 3.17 3:ed5052d1:::rbd_data.115e889d4809.000000000000015c:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:56.535 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:05:56.535 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.1054567479999999, 2024-09-14T09:05:56.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053381794000000003, 2024-09-14T09:05:56.535 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.535 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.535 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:56.535 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:56.535 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:56.536 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:56.536 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20858 2024-09-14T09:05:56.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.536 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:05:56.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:05:56.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429351+0000", 2024-09-14T09:05:56.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1149999999999998e-06 2024-09-14T09:05:56.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429363+0000", 2024-09-14T09:05:56.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1819e-05 2024-09-14T09:05:56.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429389+0000", 2024-09-14T09:05:56.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5605e-05 2024-09-14T09:05:56.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429402+0000", 2024-09-14T09:05:56.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3175e-05 2024-09-14T09:05:56.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429492+0000", 2024-09-14T09:05:56.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.988e-05 2024-09-14T09:05:56.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429597+0000", 2024-09-14T09:05:56.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010564 2024-09-14T09:05:56.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:05:56.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.433465+0000", 2024-09-14T09:05:56.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0038674249999999999 2024-09-14T09:05:56.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:56.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482483+0000", 2024-09-14T09:05:56.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049017999999999999 2024-09-14T09:05:56.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:56.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482625+0000", 2024-09-14T09:05:56.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000142447 2024-09-14T09:05:56.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:56.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482684+0000", 2024-09-14T09:05:56.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8522e-05 2024-09-14T09:05:56.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482730+0000", 2024-09-14T09:05:56.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6165999999999998e-05 2024-09-14T09:05:56.541 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.541 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.541 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.541 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:20858 3.17 e41/23 ondisk, result = 0)", 2024-09-14T09:05:56.541 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:05:56.541 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.0906973050000008, 2024-09-14T09:05:56.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038631402000000002, 2024-09-14T09:05:56.541 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.542 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.542 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:56.542 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:05:56.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:05:56.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444109+0000", 2024-09-14T09:05:56.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.331e-06 2024-09-14T09:05:56.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444114+0000", 2024-09-14T09:05:56.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3709999999999999e-06 2024-09-14T09:05:56.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444130+0000", 2024-09-14T09:05:56.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6079e-05 2024-09-14T09:05:56.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444144+0000", 2024-09-14T09:05:56.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3331e-05 2024-09-14T09:05:56.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482538+0000", 2024-09-14T09:05:56.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038394878 2024-09-14T09:05:56.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482548+0000", 2024-09-14T09:05:56.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0093e-05 2024-09-14T09:05:56.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482739+0000", 2024-09-14T09:05:56.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019031899999999999 2024-09-14T09:05:56.545 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.545 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.545 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.545 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:27419 3.3 3:c5d5ef46:::rbd_data.115ebc3c67d1.000000000000000d:c [read 970752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:56.545 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.687665+0000", 2024-09-14T09:05:56.545 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.847139882, 2024-09-14T09:05:56.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029720004000000001, 2024-09-14T09:05:56.546 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.546 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.546 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:56.546 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:56.546 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:56.546 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:56.546 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 27419 2024-09-14T09:05:56.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.546 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687665+0000", 2024-09-14T09:05:56.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687665+0000", 2024-09-14T09:05:56.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687665+0000", 2024-09-14T09:05:56.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.01e-07 2024-09-14T09:05:56.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687668+0000", 2024-09-14T09:05:56.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9069999999999999e-06 2024-09-14T09:05:56.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687674+0000", 2024-09-14T09:05:56.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1279999999999996e-06 2024-09-14T09:05:56.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687679+0000", 2024-09-14T09:05:56.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3680000000000004e-06 2024-09-14T09:05:56.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.717253+0000", 2024-09-14T09:05:56.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029574251999999999 2024-09-14T09:05:56.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.717319+0000", 2024-09-14T09:05:56.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5790999999999994e-05 2024-09-14T09:05:56.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.717385+0000", 2024-09-14T09:05:56.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6156999999999997e-05 2024-09-14T09:05:56.549 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.549 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.549 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.549 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:27420 3.3 3:c5d5ef46:::rbd_data.115ebc3c67d1.000000000000000d:c [read 980992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:56.550 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.687687+0000", 2024-09-14T09:05:56.550 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.8471181589999999, 2024-09-14T09:05:56.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029872780000000002, 2024-09-14T09:05:56.550 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.550 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.550 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:56.550 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:56.550 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:56.550 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:56.550 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 27420 2024-09-14T09:05:56.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.550 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687687+0000", 2024-09-14T09:05:56.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687687+0000", 2024-09-14T09:05:56.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687687+0000", 2024-09-14T09:05:56.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3799999999999998e-07 2024-09-14T09:05:56.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687689+0000", 2024-09-14T09:05:56.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1019999999999999e-06 2024-09-14T09:05:56.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687696+0000", 2024-09-14T09:05:56.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9079999999999998e-06 2024-09-14T09:05:56.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687701+0000", 2024-09-14T09:05:56.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8550000000000001e-06 2024-09-14T09:05:56.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.717410+0000", 2024-09-14T09:05:56.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029708906 2024-09-14T09:05:56.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.717480+0000", 2024-09-14T09:05:56.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0716000000000005e-05 2024-09-14T09:05:56.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.717559+0000", 2024-09-14T09:05:56.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8855000000000003e-05 2024-09-14T09:05:56.553 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.553 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.554 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.554 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:27421 3.3 3:c5d5ef46:::rbd_data.115ebc3c67d1.000000000000000d:c [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:56.554 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.687708+0000", 2024-09-14T09:05:56.554 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.847097046, 2024-09-14T09:05:56.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030024248, 2024-09-14T09:05:56.554 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.554 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.554 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:56.554 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:56.554 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:56.554 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:56.555 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 27421 2024-09-14T09:05:56.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.555 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687708+0000", 2024-09-14T09:05:56.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687708+0000", 2024-09-14T09:05:56.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687708+0000", 2024-09-14T09:05:56.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6600000000000002e-07 2024-09-14T09:05:56.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687710+0000", 2024-09-14T09:05:56.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3020000000000002e-06 2024-09-14T09:05:56.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687717+0000", 2024-09-14T09:05:56.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1190000000000002e-06 2024-09-14T09:05:56.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687722+0000", 2024-09-14T09:05:56.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9710000000000003e-06 2024-09-14T09:05:56.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.717583+0000", 2024-09-14T09:05:56.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029861795 2024-09-14T09:05:56.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.717654+0000", 2024-09-14T09:05:56.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0376999999999994e-05 2024-09-14T09:05:56.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.717732+0000", 2024-09-14T09:05:56.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8218000000000002e-05 2024-09-14T09:05:56.558 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.558 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.558 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.558 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:27422 3.3 3:c5d5ef46:::rbd_data.115ebc3c67d1.000000000000000d:c [read 1009664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:56.558 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.687728+0000", 2024-09-14T09:05:56.558 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.8470766510000001, 2024-09-14T09:05:56.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030177169, 2024-09-14T09:05:56.558 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.558 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.559 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:56.559 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:56.559 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:56.559 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:56.559 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 27422 2024-09-14T09:05:56.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.559 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687728+0000", 2024-09-14T09:05:56.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687728+0000", 2024-09-14T09:05:56.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687729+0000", 2024-09-14T09:05:56.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0999999999999998e-07 2024-09-14T09:05:56.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687731+0000", 2024-09-14T09:05:56.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8309999999999999e-06 2024-09-14T09:05:56.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687737+0000", 2024-09-14T09:05:56.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2580000000000001e-06 2024-09-14T09:05:56.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687741+0000", 2024-09-14T09:05:56.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2520000000000001e-06 2024-09-14T09:05:56.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.717764+0000", 2024-09-14T09:05:56.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030022508999999999 2024-09-14T09:05:56.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.717833+0000", 2024-09-14T09:05:56.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9119e-05 2024-09-14T09:05:56.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.717905+0000", 2024-09-14T09:05:56.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2490000000000006e-05 2024-09-14T09:05:56.562 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.562 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.562 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.562 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:27423 3.3 3:c5d5ef46:::rbd_data.115ebc3c67d1.000000000000000d:c [read 1028096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:56.562 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.687748+0000", 2024-09-14T09:05:56.562 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.847056646, 2024-09-14T09:05:56.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030337435999999999, 2024-09-14T09:05:56.562 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.563 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.563 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:56.563 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:56.563 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:56.563 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:56.563 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 27423 2024-09-14T09:05:56.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.563 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687748+0000", 2024-09-14T09:05:56.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687748+0000", 2024-09-14T09:05:56.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687749+0000", 2024-09-14T09:05:56.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.75e-07 2024-09-14T09:05:56.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687755+0000", 2024-09-14T09:05:56.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2480000000000001e-06 2024-09-14T09:05:56.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687761+0000", 2024-09-14T09:05:56.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2820000000000002e-06 2024-09-14T09:05:56.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687766+0000", 2024-09-14T09:05:56.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0660000000000003e-06 2024-09-14T09:05:56.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.717933+0000", 2024-09-14T09:05:56.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030166938000000001 2024-09-14T09:05:56.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.718007+0000", 2024-09-14T09:05:56.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4332999999999995e-05 2024-09-14T09:05:56.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.566 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.718086+0000", 2024-09-14T09:05:56.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8094000000000001e-05 2024-09-14T09:05:56.566 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.566 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.566 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.566 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:27424 3.3 3:c5d5ef46:::rbd_data.115ebc3c67d1.000000000000000d:c [read 1038336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:56.566 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.687773+0000", 2024-09-14T09:05:56.566 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.8470314769999998, 2024-09-14T09:05:56.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030476916999999999, 2024-09-14T09:05:56.566 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.567 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.567 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:56.567 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:56.567 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:56.567 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:56.567 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 27424 2024-09-14T09:05:56.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.567 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687773+0000", 2024-09-14T09:05:56.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687773+0000", 2024-09-14T09:05:56.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687774+0000", 2024-09-14T09:05:56.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2099999999999996e-07 2024-09-14T09:05:56.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687776+0000", 2024-09-14T09:05:56.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1550000000000001e-06 2024-09-14T09:05:56.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687783+0000", 2024-09-14T09:05:56.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8909999999999998e-06 2024-09-14T09:05:56.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.687788+0000", 2024-09-14T09:05:56.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0599999999999998e-06 2024-09-14T09:05:56.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.718115+0000", 2024-09-14T09:05:56.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030327155000000001 2024-09-14T09:05:56.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.718194+0000", 2024-09-14T09:05:56.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8690999999999996e-05 2024-09-14T09:05:56.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.718250+0000", 2024-09-14T09:05:56.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6344000000000001e-05 2024-09-14T09:05:56.570 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.570 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.570 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.570 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31056 3.17 3:ec451343:::rbd_data.115ebc3c67d1.0000000000000000:c [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:56.570 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.064210+0000", 2024-09-14T09:05:56.570 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.4705947720000001, 2024-09-14T09:05:56.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029737801000000001, 2024-09-14T09:05:56.570 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.570 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.571 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:56.571 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:56.571 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:56.571 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:56.571 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31056 2024-09-14T09:05:56.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.571 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064210+0000", 2024-09-14T09:05:56.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064210+0000", 2024-09-14T09:05:56.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064210+0000", 2024-09-14T09:05:56.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7800000000000002e-07 2024-09-14T09:05:56.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064212+0000", 2024-09-14T09:05:56.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.694e-06 2024-09-14T09:05:56.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064218+0000", 2024-09-14T09:05:56.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7470000000000002e-06 2024-09-14T09:05:56.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064222+0000", 2024-09-14T09:05:56.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4499999999999997e-06 2024-09-14T09:05:56.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093272+0000", 2024-09-14T09:05:56.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029049564 2024-09-14T09:05:56.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093342+0000", 2024-09-14T09:05:56.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0050999999999997e-05 2024-09-14T09:05:56.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093948+0000", 2024-09-14T09:05:56.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060591699999999998 2024-09-14T09:05:56.574 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.574 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.574 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.574 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31057 3.17 3:ec451343:::rbd_data.115ebc3c67d1.0000000000000000:c [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:56.574 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.064228+0000", 2024-09-14T09:05:56.574 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.4705765309999999, 2024-09-14T09:05:56.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030167363999999999, 2024-09-14T09:05:56.574 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.574 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.574 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:56.575 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:56.575 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:56.575 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:56.575 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31057 2024-09-14T09:05:56.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.575 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064228+0000", 2024-09-14T09:05:56.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064228+0000", 2024-09-14T09:05:56.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064229+0000", 2024-09-14T09:05:56.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8299999999999997e-07 2024-09-14T09:05:56.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064231+0000", 2024-09-14T09:05:56.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8470000000000001e-06 2024-09-14T09:05:56.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064236+0000", 2024-09-14T09:05:56.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6189999999999999e-06 2024-09-14T09:05:56.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064241+0000", 2024-09-14T09:05:56.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5000000000000001e-06 2024-09-14T09:05:56.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093980+0000", 2024-09-14T09:05:56.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029739853 2024-09-14T09:05:56.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.094053+0000", 2024-09-14T09:05:56.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2304000000000004e-05 2024-09-14T09:05:56.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.094396+0000", 2024-09-14T09:05:56.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034275799999999997 2024-09-14T09:05:56.578 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.578 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.578 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.578 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31058 3.17 3:ec451343:::rbd_data.115ebc3c67d1.0000000000000000:c [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:56.578 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.064246+0000", 2024-09-14T09:05:56.578 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.4705583130000002, 2024-09-14T09:05:56.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030653671, 2024-09-14T09:05:56.578 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.578 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.578 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:56.579 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:56.579 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:56.579 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:56.579 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31058 2024-09-14T09:05:56.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.579 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064246+0000", 2024-09-14T09:05:56.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064246+0000", 2024-09-14T09:05:56.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064247+0000", 2024-09-14T09:05:56.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6199999999999999e-07 2024-09-14T09:05:56.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064253+0000", 2024-09-14T09:05:56.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1580000000000002e-06 2024-09-14T09:05:56.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064259+0000", 2024-09-14T09:05:56.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1319999999999999e-06 2024-09-14T09:05:56.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064264+0000", 2024-09-14T09:05:56.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.685e-06 2024-09-14T09:05:56.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.094461+0000", 2024-09-14T09:05:56.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030196806999999999 2024-09-14T09:05:56.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.094511+0000", 2024-09-14T09:05:56.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0657e-05 2024-09-14T09:05:56.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.094900+0000", 2024-09-14T09:05:56.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00038886999999999998 2024-09-14T09:05:56.582 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.582 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.582 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.583 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31059 3.17 3:ec451343:::rbd_data.115ebc3c67d1.0000000000000000:c [read 178176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:56.583 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.064270+0000", 2024-09-14T09:05:56.583 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.4705348840000001, 2024-09-14T09:05:56.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030820295000000001, 2024-09-14T09:05:56.583 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.583 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.583 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:56.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:56.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:56.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:56.583 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31059 2024-09-14T09:05:56.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.583 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064270+0000", 2024-09-14T09:05:56.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064270+0000", 2024-09-14T09:05:56.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064270+0000", 2024-09-14T09:05:56.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5900000000000003e-07 2024-09-14T09:05:56.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064272+0000", 2024-09-14T09:05:56.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9309999999999998e-06 2024-09-14T09:05:56.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064277+0000", 2024-09-14T09:05:56.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.259e-06 2024-09-14T09:05:56.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064282+0000", 2024-09-14T09:05:56.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5789999999999997e-06 2024-09-14T09:05:56.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.094937+0000", 2024-09-14T09:05:56.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030654825 2024-09-14T09:05:56.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.095018+0000", 2024-09-14T09:05:56.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1017e-05 2024-09-14T09:05:56.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.095090+0000", 2024-09-14T09:05:56.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2324999999999996e-05 2024-09-14T09:05:56.587 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.587 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.587 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.587 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31060 3.17 3:ec451343:::rbd_data.115ebc3c67d1.0000000000000000:c [read 196608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:56.587 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.064289+0000", 2024-09-14T09:05:56.587 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.4705162010000001, 2024-09-14T09:05:56.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030974154, 2024-09-14T09:05:56.587 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.587 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.587 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31060 2024-09-14T09:05:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064289+0000", 2024-09-14T09:05:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064289+0000", 2024-09-14T09:05:56.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064289+0000", 2024-09-14T09:05:56.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2100000000000002e-07 2024-09-14T09:05:56.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064291+0000", 2024-09-14T09:05:56.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.057e-06 2024-09-14T09:05:56.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064298+0000", 2024-09-14T09:05:56.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.126e-06 2024-09-14T09:05:56.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064303+0000", 2024-09-14T09:05:56.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9869999999999999e-06 2024-09-14T09:05:56.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.095129+0000", 2024-09-14T09:05:56.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030826388 2024-09-14T09:05:56.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.095198+0000", 2024-09-14T09:05:56.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8658999999999994e-05 2024-09-14T09:05:56.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.095263+0000", 2024-09-14T09:05:56.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4516000000000004e-05 2024-09-14T09:05:56.591 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.591 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.591 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.592 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31061 3.17 3:ec451343:::rbd_data.115ebc3c67d1.0000000000000000:c [read 206848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:56.592 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.064310+0000", 2024-09-14T09:05:56.592 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.4704947310000001, 2024-09-14T09:05:56.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.031146745, 2024-09-14T09:05:56.592 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.592 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.592 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:56.592 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:56.592 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:56.592 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:56.592 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31061 2024-09-14T09:05:56.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.592 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064310+0000", 2024-09-14T09:05:56.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064310+0000", 2024-09-14T09:05:56.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064310+0000", 2024-09-14T09:05:56.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.46e-07 2024-09-14T09:05:56.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064313+0000", 2024-09-14T09:05:56.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5739999999999998e-06 2024-09-14T09:05:56.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064320+0000", 2024-09-14T09:05:56.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0199999999999997e-06 2024-09-14T09:05:56.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064325+0000", 2024-09-14T09:05:56.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9239999999999998e-06 2024-09-14T09:05:56.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.095306+0000", 2024-09-14T09:05:56.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030981379999999999 2024-09-14T09:05:56.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.095379+0000", 2024-09-14T09:05:56.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2255999999999994e-05 2024-09-14T09:05:56.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.095457+0000", 2024-09-14T09:05:56.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8145000000000005e-05 2024-09-14T09:05:56.595 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.596 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.596 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.596 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31062 3.17 3:ec451343:::rbd_data.115ebc3c67d1.0000000000000000:c [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:56.596 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.064332+0000", 2024-09-14T09:05:56.596 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.470472762, 2024-09-14T09:05:56.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.031397869000000002, 2024-09-14T09:05:56.596 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.596 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.596 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:56.596 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:56.596 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:56.596 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:56.596 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31062 2024-09-14T09:05:56.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.597 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064332+0000", 2024-09-14T09:05:56.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064332+0000", 2024-09-14T09:05:56.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064332+0000", 2024-09-14T09:05:56.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1399999999999997e-07 2024-09-14T09:05:56.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064334+0000", 2024-09-14T09:05:56.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0499999999999999e-06 2024-09-14T09:05:56.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064341+0000", 2024-09-14T09:05:56.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.64e-06 2024-09-14T09:05:56.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064346+0000", 2024-09-14T09:05:56.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1059999999999999e-06 2024-09-14T09:05:56.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.095488+0000", 2024-09-14T09:05:56.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.031141433999999999 2024-09-14T09:05:56.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.095649+0000", 2024-09-14T09:05:56.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000161788 2024-09-14T09:05:56.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.095730+0000", 2024-09-14T09:05:56.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0437000000000003e-05 2024-09-14T09:05:56.599 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.599 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.600 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.600 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31063 3.17 3:ec451343:::rbd_data.115ebc3c67d1.0000000000000000:c [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:56.600 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.064353+0000", 2024-09-14T09:05:56.600 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.4704513499999998, 2024-09-14T09:05:56.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.031610920000000001, 2024-09-14T09:05:56.600 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.600 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.600 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:56.600 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:56.600 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:56.600 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:56.601 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31063 2024-09-14T09:05:56.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.601 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064353+0000", 2024-09-14T09:05:56.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064353+0000", 2024-09-14T09:05:56.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064354+0000", 2024-09-14T09:05:56.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2599999999999998e-07 2024-09-14T09:05:56.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064356+0000", 2024-09-14T09:05:56.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4339999999999999e-06 2024-09-14T09:05:56.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064363+0000", 2024-09-14T09:05:56.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.781e-06 2024-09-14T09:05:56.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064368+0000", 2024-09-14T09:05:56.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9810000000000003e-06 2024-09-14T09:05:56.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.095797+0000", 2024-09-14T09:05:56.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.031429186999999997 2024-09-14T09:05:56.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.095870+0000", 2024-09-14T09:05:56.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3003000000000006e-05 2024-09-14T09:05:56.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.095964+0000", 2024-09-14T09:05:56.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4108000000000001e-05 2024-09-14T09:05:56.604 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.605 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.605 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.605 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31064 3.17 3:ec451343:::rbd_data.115ebc3c67d1.0000000000000000:c [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:56.605 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.064375+0000", 2024-09-14T09:05:56.605 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.4704292300000001, 2024-09-14T09:05:56.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.031767618999999997, 2024-09-14T09:05:56.605 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.605 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.605 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:56.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:56.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:56.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:56.606 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31064 2024-09-14T09:05:56.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.606 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064375+0000", 2024-09-14T09:05:56.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064375+0000", 2024-09-14T09:05:56.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064376+0000", 2024-09-14T09:05:56.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4000000000000002e-07 2024-09-14T09:05:56.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064378+0000", 2024-09-14T09:05:56.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0590000000000001e-06 2024-09-14T09:05:56.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064384+0000", 2024-09-14T09:05:56.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3319999999999997e-06 2024-09-14T09:05:56.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064389+0000", 2024-09-14T09:05:56.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8080000000000004e-06 2024-09-14T09:05:56.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.095993+0000", 2024-09-14T09:05:56.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.031604275000000001 2024-09-14T09:05:56.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.096068+0000", 2024-09-14T09:05:56.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4809999999999997e-05 2024-09-14T09:05:56.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.096143+0000", 2024-09-14T09:05:56.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4894999999999994e-05 2024-09-14T09:05:56.609 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.609 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.609 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.609 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31065 3.17 3:ec451343:::rbd_data.115ebc3c67d1.0000000000000000:c [read 264192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:56.609 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.064433+0000", 2024-09-14T09:05:56.609 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.4703716610000002, 2024-09-14T09:05:56.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.032011280000000003, 2024-09-14T09:05:56.609 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.609 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.609 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:56.609 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:56.610 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:56.610 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:56.610 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31065 2024-09-14T09:05:56.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.610 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064433+0000", 2024-09-14T09:05:56.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064433+0000", 2024-09-14T09:05:56.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064434+0000", 2024-09-14T09:05:56.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T09:05:56.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064437+0000", 2024-09-14T09:05:56.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8710000000000001e-06 2024-09-14T09:05:56.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064445+0000", 2024-09-14T09:05:56.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0579999999999998e-06 2024-09-14T09:05:56.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064465+0000", 2024-09-14T09:05:56.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0264000000000001e-05 2024-09-14T09:05:56.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.096307+0000", 2024-09-14T09:05:56.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.031841819 2024-09-14T09:05:56.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.096349+0000", 2024-09-14T09:05:56.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2663999999999997e-05 2024-09-14T09:05:56.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.096444+0000", 2024-09-14T09:05:56.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4937000000000003e-05 2024-09-14T09:05:56.613 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.613 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.613 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.613 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31066 3.17 3:ec451343:::rbd_data.115ebc3c67d1.0000000000000000:c [read 282624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:56.613 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.064474+0000", 2024-09-14T09:05:56.613 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.4703308509999999, 2024-09-14T09:05:56.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.032177247999999999, 2024-09-14T09:05:56.613 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.613 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.614 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:56.614 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:56.614 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:56.614 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:56.614 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31066 2024-09-14T09:05:56.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.614 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064474+0000", 2024-09-14T09:05:56.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064474+0000", 2024-09-14T09:05:56.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064474+0000", 2024-09-14T09:05:56.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8100000000000003e-07 2024-09-14T09:05:56.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064477+0000", 2024-09-14T09:05:56.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5500000000000001e-06 2024-09-14T09:05:56.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064483+0000", 2024-09-14T09:05:56.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1410000000000002e-06 2024-09-14T09:05:56.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064489+0000", 2024-09-14T09:05:56.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4759999999999997e-06 2024-09-14T09:05:56.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.096507+0000", 2024-09-14T09:05:56.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.032018641 2024-09-14T09:05:56.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.096579+0000", 2024-09-14T09:05:56.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1833000000000002e-05 2024-09-14T09:05:56.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.096651+0000", 2024-09-14T09:05:56.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2026000000000005e-05 2024-09-14T09:05:56.617 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.617 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.617 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.617 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31067 3.17 3:ec451343:::rbd_data.115ebc3c67d1.0000000000000000:c [read 292864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:56.617 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.064495+0000", 2024-09-14T09:05:56.617 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.4703099059999998, 2024-09-14T09:05:56.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.032341073999999997, 2024-09-14T09:05:56.617 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:56.618 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:56.618 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:56.618 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:56.618 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:56.618 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:56.618 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31067 2024-09-14T09:05:56.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.618 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:56.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:56.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064495+0000", 2024-09-14T09:05:56.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:56.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064495+0000", 2024-09-14T09:05:56.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:56.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:56.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064495+0000", 2024-09-14T09:05:56.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2100000000000002e-07 2024-09-14T09:05:56.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:56.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064497+0000", 2024-09-14T09:05:56.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.221e-06 2024-09-14T09:05:56.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:56.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064503+0000", 2024-09-14T09:05:56.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9259999999999997e-06 2024-09-14T09:05:56.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:56.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.064508+0000", 2024-09-14T09:05:56.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6700000000000002e-06 2024-09-14T09:05:56.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:56.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.096688+0000", 2024-09-14T09:05:56.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.032179963999999998 2024-09-14T09:05:56.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:56.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.096761+0000", 2024-09-14T09:05:56.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3207999999999994e-05 2024-09-14T09:05:56.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:56.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:56.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:56.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.096836+0000", 2024-09-14T09:05:56.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4664000000000002e-05 2024-09-14T09:05:56.621 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.621 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.621 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.621 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:56.621 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:56.621 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:56.622 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:05:56.623 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:56.620+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:56.660 INFO:teuthology.orchestra.run.smithi176.stdout:1264 punch from 0x5b4c7cc to 0x5b57a79, (0xb2ad bytes) 2024-09-14T09:05:56.660 INFO:teuthology.orchestra.run.smithi176.stdout:1268 writesame 0x8150db7 thru 0x81570ad data_size 0x9(0x62f7 bytes) 2024-09-14T09:05:56.688 INFO:teuthology.orchestra.run.smithi176.stdout:1269 write 0xedb617f thru 0xedbcb89 (0x6a0b bytes) 2024-09-14T09:05:56.708 INFO:teuthology.orchestra.run.smithi176.stdout:1271 trunc from 0xedbcb8a to 0x810c2f9 2024-09-14T09:05:56.724 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:56.723+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:56.729 INFO:teuthology.orchestra.run.smithi176.stdout:1272 read 0x3b45c73 thru 0x3b480b2 (0x2440 bytes) 2024-09-14T09:05:56.731 INFO:teuthology.orchestra.run.smithi176.stdout:1273 read 0x157de78 thru 0x1583a7e (0x5c07 bytes) 2024-09-14T09:05:56.732 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:56.735 INFO:teuthology.orchestra.run.smithi176.stdout:1277 trunc from 0x810c2f9 to 0x60f2682 2024-09-14T09:05:56.776 INFO:teuthology.orchestra.run.smithi176.stdout:1278 write 0xbe58cef thru 0xbe5d189 (0x449b bytes) 2024-09-14T09:05:56.781 INFO:teuthology.orchestra.run.smithi176.stdout:1279 writesame 0x31ec4fb thru 0x31f4394 data_size 0x23(0x7e9a bytes) 2024-09-14T09:05:56.786 INFO:teuthology.orchestra.run.smithi176.stdout:1280 read 0x42bd0af thru 0x42bfb46 (0x2a98 bytes) 2024-09-14T09:05:56.787 INFO:teuthology.orchestra.run.smithi176.stdout:1281 read 0x51501bd thru 0x515bd62 (0xbba6 bytes) 2024-09-14T09:05:56.788 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:56.788 INFO:teuthology.orchestra.run.smithi176.stdout:1284 read 0x19d8a2b thru 0x19dbc81 (0x3257 bytes) 2024-09-14T09:05:56.789 INFO:teuthology.orchestra.run.smithi176.stdout:1285 write 0xb5bc79d thru 0xb5bda22 (0x1286 bytes) 2024-09-14T09:05:56.795 INFO:teuthology.orchestra.run.smithi176.stdout:1286 read 0x6b5a586 thru 0x6b68b65 (0xe5e0 bytes) 2024-09-14T09:05:56.796 INFO:teuthology.orchestra.run.smithi176.stdout:1287 read 0x2d10980 thru 0x2d1a732 (0x9db3 bytes) 2024-09-14T09:05:56.797 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:56.797 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:56.798 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:56.798 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:56.798 INFO:teuthology.orchestra.run.smithi176.stdout:1288 read 0x632da14 thru 0x6337063 (0x9650 bytes) 2024-09-14T09:05:56.798 INFO:teuthology.orchestra.run.smithi176.stdout:1289 read 0x2d74c39 thru 0x2d76a5a (0x1e22 bytes) 2024-09-14T09:05:56.799 INFO:teuthology.orchestra.run.smithi176.stdout:1291 write 0x15851b9 thru 0x158f9d8 (0xa820 bytes) 2024-09-14T09:05:56.801 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:05:56.802 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:05:56.802 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-14T09:05:56.802 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-14T09:05:56.807 INFO:teuthology.orchestra.run.smithi176.stdout:1292 write 0xffee99 thru 0x10047fc (0x5964 bytes) 2024-09-14T09:05:56.813 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:05:56.821 INFO:teuthology.orchestra.run.smithi176.stdout:1293 write 0xd312d86 thru 0xd317b54 (0x4dcf bytes) 2024-09-14T09:05:56.827 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:56.827 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:56.824+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:56.827 INFO:teuthology.orchestra.run.smithi176.stdout:1295 trunc from 0xd317b55 to 0xc9dd9ee 2024-09-14T09:05:56.837 INFO:teuthology.orchestra.run.smithi176.stdout:1297 punch from 0x4ed6d1 to 0x4f7603, (0x9f32 bytes) 2024-09-14T09:05:56.838 INFO:teuthology.orchestra.run.smithi176.stdout:1298 punch from 0x7dc2913 to 0x7dc5ef0, (0x35dd bytes) 2024-09-14T09:05:56.838 INFO:teuthology.orchestra.run.smithi176.stdout:1300 writesame 0x4b5e41d thru 0x4b5f5f6 data_size 0x5(0x11da bytes) 2024-09-14T09:05:56.840 INFO:teuthology.orchestra.run.smithi176.stdout:1303 punch from 0x67fce00 to 0x680c286, (0xf486 bytes) 2024-09-14T09:05:56.857 INFO:teuthology.orchestra.run.smithi176.stdout:1305 trunc from 0xc9dd9ee to 0x2b458c7 2024-09-14T09:05:56.880 INFO:teuthology.orchestra.run.smithi176.stdout:1306 read 0xb35d59 thru 0xb42163 (0xc40b bytes) 2024-09-14T09:05:56.884 INFO:teuthology.orchestra.run.smithi176.stdout:1307 trunc from 0x2b458c7 to 0x4a04f12 2024-09-14T09:05:56.901 INFO:teuthology.orchestra.run.smithi176.stdout:1308 writesame 0x669e18e thru 0x66a45ff data_size 0x2e(0x6472 bytes) 2024-09-14T09:05:56.904 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:56.904 INFO:teuthology.orchestra.run.smithi176.stdout:1310 flatten 2024-09-14T09:05:56.912 INFO:teuthology.orchestra.run.smithi176.stdout:1311 write 0x5b01a4a thru 0x5b050a9 (0x3660 bytes) 2024-09-14T09:05:56.919 INFO:teuthology.orchestra.run.smithi176.stdout:1312 trunc from 0x66a4600 to 0x8517218 2024-09-14T09:05:56.924 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:56.920+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:56.930 INFO:teuthology.orchestra.run.smithi176.stdout:1313 trunc from 0x8517218 to 0x56d4137 2024-09-14T09:05:56.943 INFO:teuthology.orchestra.run.smithi176.stdout:1314 read 0x1d1777e thru 0x1d1d710 (0x5f93 bytes) 2024-09-14T09:05:56.959 INFO:teuthology.orchestra.run.smithi176.stdout:1316 write 0xa93b9a0 thru 0xa9404c4 (0x4b25 bytes) 2024-09-14T09:05:56.967 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:56.967 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:56.967 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:56.967 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:56.967 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:56.967 INFO:teuthology.orchestra.run.smithi176.stdout:1317 write 0x4895bad thru 0x48a3217 (0xd66b bytes) 2024-09-14T09:05:56.985 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:05:56.986 INFO:teuthology.orchestra.run.smithi176.stdout:1318 write 0xcd0cbe1 thru 0xcd179ef (0xae0f bytes) 2024-09-14T09:05:56.991 INFO:teuthology.orchestra.run.smithi176.stdout:1321 read 0x69bfc52 thru 0x69c5e87 (0x6236 bytes) 2024-09-14T09:05:56.991 INFO:teuthology.orchestra.run.smithi176.stdout:1322 punch from 0x3a54bd0 to 0x3a62983, (0xddb3 bytes) 2024-09-14T09:05:56.996 INFO:teuthology.orchestra.run.smithi176.stdout:1324 trunc from 0xcd179f0 to 0x98355c6 2024-09-14T09:05:57.012 INFO:teuthology.orchestra.run.smithi176.stdout:1325 trunc from 0x98355c6 to 0xda552ce 2024-09-14T09:05:57.019 INFO:teuthology.orchestra.run.smithi176.stdout:1327 writesame 0x185b224 thru 0x185e2ed data_size 0x5(0x30ca bytes) 2024-09-14T09:05:57.022 INFO:teuthology.orchestra.run.smithi176.stdout:1328 write 0xa3f5c5c thru 0xa3f6ae3 (0xe88 bytes) 2024-09-14T09:05:57.025 INFO:teuthology.orchestra.run.smithi176.stdout:1329 read 0x8d34d27 thru 0x8d3d888 (0x8b62 bytes) 2024-09-14T09:05:57.025 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:57.025 INFO:teuthology.orchestra.run.smithi176.stdout:1331 punch from 0x5d293de to 0x5d2b608, (0x222a bytes) 2024-09-14T09:05:57.025 INFO:teuthology.orchestra.run.smithi176.stdout:1332 writesame 0xa2e3953 thru 0xa2ea7c7 data_size 0x1(0x6e75 bytes) 2024-09-14T09:05:57.026 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:57.027+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:57.097 INFO:teuthology.orchestra.run.smithi176.stdout:1333 trunc from 0xda552ce to 0xe0986cc 2024-09-14T09:05:57.105 INFO:teuthology.orchestra.run.smithi176.stdout:1334 punch from 0x6ead755 to 0x6ebce94, (0xf73f bytes) 2024-09-14T09:05:57.105 INFO:teuthology.orchestra.run.smithi176.stdout:1335 write 0x9c00afb thru 0x9c0f5fd (0xeb03 bytes) 2024-09-14T09:05:57.111 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:57.111 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:57.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:57.111 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:57.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.111 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:10032 3.19 3:9ed4ec6a:::rbd_data.115eadec874.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:05:57.111 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.775706+0000", 2024-09-14T09:05:57.111 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.335513862999999, 2024-09-14T09:05:57.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126247939, 2024-09-14T09:05:57.111 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.112 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.112 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:57.112 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.112 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.112 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.112 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 10032 2024-09-14T09:05:57.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.112 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775706+0000", 2024-09-14T09:05:57.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775706+0000", 2024-09-14T09:05:57.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775709+0000", 2024-09-14T09:05:57.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0400000000000001e-06 2024-09-14T09:05:57.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775721+0000", 2024-09-14T09:05:57.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2119999999999999e-05 2024-09-14T09:05:57.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775752+0000", 2024-09-14T09:05:57.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0139999999999999e-05 2024-09-14T09:05:57.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775766+0000", 2024-09-14T09:05:57.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4497e-05 2024-09-14T09:05:57.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775937+0000", 2024-09-14T09:05:57.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000171022 2024-09-14T09:05:57.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.776205+0000", 2024-09-14T09:05:57.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026749100000000002 2024-09-14T09:05:57.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:05:57.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.784970+0000", 2024-09-14T09:05:57.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0087648789999999997 2024-09-14T09:05:57.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:57.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901870+0000", 2024-09-14T09:05:57.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11690028 2024-09-14T09:05:57.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:57.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901878+0000", 2024-09-14T09:05:57.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7020000000000007e-06 2024-09-14T09:05:57.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:57.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901913+0000", 2024-09-14T09:05:57.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4706000000000003e-05 2024-09-14T09:05:57.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901954+0000", 2024-09-14T09:05:57.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1062000000000001e-05 2024-09-14T09:05:57.117 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.117 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.117 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.117 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:10032 3.19 e32/23 ondisk, result = 0)", 2024-09-14T09:05:57.117 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.809095+0000", 2024-09-14T09:05:57.117 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.302125583999999, 2024-09-14T09:05:57.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092866664000000002, 2024-09-14T09:05:57.117 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.117 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.118 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.118 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809095+0000", 2024-09-14T09:05:57.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809095+0000", 2024-09-14T09:05:57.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809096+0000", 2024-09-14T09:05:57.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.64e-06 2024-09-14T09:05:57.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809104+0000", 2024-09-14T09:05:57.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7330000000000003e-06 2024-09-14T09:05:57.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809132+0000", 2024-09-14T09:05:57.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7880000000000001e-05 2024-09-14T09:05:57.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809146+0000", 2024-09-14T09:05:57.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4514e-05 2024-09-14T09:05:57.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901858+0000", 2024-09-14T09:05:57.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092711713000000001 2024-09-14T09:05:57.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901866+0000", 2024-09-14T09:05:57.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4760000000000001e-06 2024-09-14T09:05:57.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901961+0000", 2024-09-14T09:05:57.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5707999999999999e-05 2024-09-14T09:05:57.120 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.121 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.121 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.121 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:19545 3.f e39/34 3:f12afce6:::rbd_data.115e4c71d6e8.0000000000000003:head v 39'24, mlcod=39'22)", 2024-09-14T09:05:57.121 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.529754+0000", 2024-09-14T09:05:57.121 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.581466046999999, 2024-09-14T09:05:57.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037365357000000002, 2024-09-14T09:05:57.121 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.121 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.121 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:57.121 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.529754+0000", 2024-09-14T09:05:57.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.529754+0000", 2024-09-14T09:05:57.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.529756+0000", 2024-09-14T09:05:57.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.804e-06 2024-09-14T09:05:57.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537071+0000", 2024-09-14T09:05:57.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0073151120000000004 2024-09-14T09:05:57.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537111+0000", 2024-09-14T09:05:57.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0315999999999998e-05 2024-09-14T09:05:57.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537133+0000", 2024-09-14T09:05:57.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2098000000000001e-05 2024-09-14T09:05:57.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537274+0000", 2024-09-14T09:05:57.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014045400000000001 2024-09-14T09:05:57.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537312+0000", 2024-09-14T09:05:57.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7761e-05 2024-09-14T09:05:57.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:57.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.567050+0000", 2024-09-14T09:05:57.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029738547000000001 2024-09-14T09:05:57.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.567119+0000", 2024-09-14T09:05:57.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9264999999999995e-05 2024-09-14T09:05:57.125 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.125 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.125 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.125 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20862 3.1a 3:580a70cb:::rbd_data.115e889d4809.0000000000000177:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:57.125 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.497482+0000", 2024-09-14T09:05:57.126 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.6137378150000004, 2024-09-14T09:05:57.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.032797795999999997, 2024-09-14T09:05:57.126 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.126 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.126 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:57.126 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.126 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.126 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.126 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20862 2024-09-14T09:05:57.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.126 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497482+0000", 2024-09-14T09:05:57.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497482+0000", 2024-09-14T09:05:57.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497485+0000", 2024-09-14T09:05:57.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8569999999999999e-06 2024-09-14T09:05:57.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497496+0000", 2024-09-14T09:05:57.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0277e-05 2024-09-14T09:05:57.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497519+0000", 2024-09-14T09:05:57.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3513e-05 2024-09-14T09:05:57.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497533+0000", 2024-09-14T09:05:57.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4343e-05 2024-09-14T09:05:57.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497642+0000", 2024-09-14T09:05:57.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010840099999999999 2024-09-14T09:05:57.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497840+0000", 2024-09-14T09:05:57.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000198192 2024-09-14T09:05:57.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 2", 2024-09-14T09:05:57.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.499243+0000", 2024-09-14T09:05:57.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0014027550000000001 2024-09-14T09:05:57.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:57.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530087+0000", 2024-09-14T09:05:57.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030844086999999999 2024-09-14T09:05:57.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:57.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530120+0000", 2024-09-14T09:05:57.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3309000000000002e-05 2024-09-14T09:05:57.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:57.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530207+0000", 2024-09-14T09:05:57.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6536e-05 2024-09-14T09:05:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530280+0000", 2024-09-14T09:05:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3526e-05 2024-09-14T09:05:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:20862 3.1a e41/34 ondisk, result = 0)", 2024-09-14T09:05:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.503269+0000", 2024-09-14T09:05:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.6079508100000002, 2024-09-14T09:05:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026831292999999999, 2024-09-14T09:05:57.131 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503269+0000", 2024-09-14T09:05:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503269+0000", 2024-09-14T09:05:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503271+0000", 2024-09-14T09:05:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3820000000000001e-06 2024-09-14T09:05:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503276+0000", 2024-09-14T09:05:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6400000000000002e-06 2024-09-14T09:05:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503293+0000", 2024-09-14T09:05:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6404000000000001e-05 2024-09-14T09:05:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503304+0000", 2024-09-14T09:05:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0905e-05 2024-09-14T09:05:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530061+0000", 2024-09-14T09:05:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026757019 2024-09-14T09:05:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530076+0000", 2024-09-14T09:05:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5526999999999999e-05 2024-09-14T09:05:57.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530101+0000", 2024-09-14T09:05:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4416000000000001e-05 2024-09-14T09:05:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:30959 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 262144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.061752+0000", 2024-09-14T09:05:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.0494683450000002, 2024-09-14T09:05:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023312566999999999, 2024-09-14T09:05:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.135 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30959 2024-09-14T09:05:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061752+0000", 2024-09-14T09:05:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061752+0000", 2024-09-14T09:05:57.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061752+0000", 2024-09-14T09:05:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2699999999999999e-07 2024-09-14T09:05:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061755+0000", 2024-09-14T09:05:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2850000000000001e-06 2024-09-14T09:05:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061761+0000", 2024-09-14T09:05:57.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7769999999999997e-06 2024-09-14T09:05:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061766+0000", 2024-09-14T09:05:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7090000000000001e-06 2024-09-14T09:05:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.084892+0000", 2024-09-14T09:05:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023125759999999999 2024-09-14T09:05:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.084967+0000", 2024-09-14T09:05:57.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5444000000000005e-05 2024-09-14T09:05:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.085064+0000", 2024-09-14T09:05:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7065000000000007e-05 2024-09-14T09:05:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:30960 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 272384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.061773+0000", 2024-09-14T09:05:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.0494470419999997, 2024-09-14T09:05:57.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023509044999999999, 2024-09-14T09:05:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30960 2024-09-14T09:05:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061773+0000", 2024-09-14T09:05:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061773+0000", 2024-09-14T09:05:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061774+0000", 2024-09-14T09:05:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3700000000000001e-07 2024-09-14T09:05:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061776+0000", 2024-09-14T09:05:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0990000000000001e-06 2024-09-14T09:05:57.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061782+0000", 2024-09-14T09:05:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2829999999999998e-06 2024-09-14T09:05:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061786+0000", 2024-09-14T09:05:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4970000000000003e-06 2024-09-14T09:05:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.085106+0000", 2024-09-14T09:05:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023319617000000001 2024-09-14T09:05:57.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.085182+0000", 2024-09-14T09:05:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5512000000000005e-05 2024-09-14T09:05:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.085282+0000", 2024-09-14T09:05:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010060000000000001 2024-09-14T09:05:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:30961 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 290816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.061793+0000", 2024-09-14T09:05:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.049426918, 2024-09-14T09:05:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023756772999999998, 2024-09-14T09:05:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30961 2024-09-14T09:05:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061793+0000", 2024-09-14T09:05:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061793+0000", 2024-09-14T09:05:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061794+0000", 2024-09-14T09:05:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1800000000000001e-07 2024-09-14T09:05:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061796+0000", 2024-09-14T09:05:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0480000000000001e-06 2024-09-14T09:05:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061802+0000", 2024-09-14T09:05:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2129999999999996e-06 2024-09-14T09:05:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061807+0000", 2024-09-14T09:05:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.617e-06 2024-09-14T09:05:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.085339+0000", 2024-09-14T09:05:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023532761999999999 2024-09-14T09:05:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.085416+0000", 2024-09-14T09:05:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6951000000000002e-05 2024-09-14T09:05:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.085550+0000", 2024-09-14T09:05:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000133764 2024-09-14T09:05:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:30962 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 301056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.061813+0000", 2024-09-14T09:05:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.0494068959999998, 2024-09-14T09:05:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023993381000000001, 2024-09-14T09:05:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30962 2024-09-14T09:05:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061813+0000", 2024-09-14T09:05:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061813+0000", 2024-09-14T09:05:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061814+0000", 2024-09-14T09:05:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4799999999999999e-07 2024-09-14T09:05:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061816+0000", 2024-09-14T09:05:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9400000000000001e-06 2024-09-14T09:05:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061822+0000", 2024-09-14T09:05:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7050000000000002e-06 2024-09-14T09:05:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061827+0000", 2024-09-14T09:05:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5909999999999998e-06 2024-09-14T09:05:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.085612+0000", 2024-09-14T09:05:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023785404999999999 2024-09-14T09:05:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.085695+0000", 2024-09-14T09:05:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2661e-05 2024-09-14T09:05:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.085807+0000", 2024-09-14T09:05:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111631 2024-09-14T09:05:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.151 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.152 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.152 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31209 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:57.152 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.068892+0000", 2024-09-14T09:05:57.152 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.0423282269999996, 2024-09-14T09:05:57.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023518807999999999, 2024-09-14T09:05:57.152 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.152 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.152 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.152 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.152 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.152 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31209 2024-09-14T09:05:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068892+0000", 2024-09-14T09:05:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068892+0000", 2024-09-14T09:05:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068893+0000", 2024-09-14T09:05:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0100000000000005e-07 2024-09-14T09:05:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068895+0000", 2024-09-14T09:05:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0389999999999999e-06 2024-09-14T09:05:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068901+0000", 2024-09-14T09:05:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4069999999999998e-06 2024-09-14T09:05:57.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068913+0000", 2024-09-14T09:05:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1753000000000001e-05 2024-09-14T09:05:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092223+0000", 2024-09-14T09:05:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023310581 2024-09-14T09:05:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092313+0000", 2024-09-14T09:05:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9399000000000002e-05 2024-09-14T09:05:57.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092411+0000", 2024-09-14T09:05:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8028000000000004e-05 2024-09-14T09:05:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31210 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.068920+0000", 2024-09-14T09:05:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.0422998090000002, 2024-09-14T09:05:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023661033000000001, 2024-09-14T09:05:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.156 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31210 2024-09-14T09:05:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068920+0000", 2024-09-14T09:05:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068920+0000", 2024-09-14T09:05:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068921+0000", 2024-09-14T09:05:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4900000000000001e-07 2024-09-14T09:05:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068924+0000", 2024-09-14T09:05:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2890000000000002e-06 2024-09-14T09:05:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068930+0000", 2024-09-14T09:05:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1020000000000002e-06 2024-09-14T09:05:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068936+0000", 2024-09-14T09:05:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1260000000000003e-06 2024-09-14T09:05:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092449+0000", 2024-09-14T09:05:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023512488000000002 2024-09-14T09:05:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092501+0000", 2024-09-14T09:05:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2385000000000001e-05 2024-09-14T09:05:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092581+0000", 2024-09-14T09:05:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0193999999999998e-05 2024-09-14T09:05:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31211 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:57.160 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.068944+0000", 2024-09-14T09:05:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.042276599, 2024-09-14T09:05:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023854895000000001, 2024-09-14T09:05:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31211 2024-09-14T09:05:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068944+0000", 2024-09-14T09:05:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068944+0000", 2024-09-14T09:05:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068944+0000", 2024-09-14T09:05:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9900000000000001e-07 2024-09-14T09:05:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068946+0000", 2024-09-14T09:05:57.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1579999999999999e-06 2024-09-14T09:05:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068953+0000", 2024-09-14T09:05:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3219999999999998e-06 2024-09-14T09:05:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068958+0000", 2024-09-14T09:05:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0390000000000004e-06 2024-09-14T09:05:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092630+0000", 2024-09-14T09:05:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023672397000000001 2024-09-14T09:05:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092702+0000", 2024-09-14T09:05:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2006000000000001e-05 2024-09-14T09:05:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092798+0000", 2024-09-14T09:05:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6473999999999998e-05 2024-09-14T09:05:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.164 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31212 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.068964+0000", 2024-09-14T09:05:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.0422561310000003, 2024-09-14T09:05:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024056381000000002, 2024-09-14T09:05:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.165 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31212 2024-09-14T09:05:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068964+0000", 2024-09-14T09:05:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068964+0000", 2024-09-14T09:05:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068964+0000", 2024-09-14T09:05:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.34e-07 2024-09-14T09:05:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068966+0000", 2024-09-14T09:05:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.826e-06 2024-09-14T09:05:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068972+0000", 2024-09-14T09:05:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9070000000000004e-06 2024-09-14T09:05:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068977+0000", 2024-09-14T09:05:57.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6020000000000002e-06 2024-09-14T09:05:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092832+0000", 2024-09-14T09:05:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023854843000000001 2024-09-14T09:05:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092903+0000", 2024-09-14T09:05:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0982000000000006e-05 2024-09-14T09:05:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093020+0000", 2024-09-14T09:05:57.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117787 2024-09-14T09:05:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31213 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 235520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.068983+0000", 2024-09-14T09:05:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.0422366839999997, 2024-09-14T09:05:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024207487, 2024-09-14T09:05:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.169 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31213 2024-09-14T09:05:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068983+0000", 2024-09-14T09:05:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068983+0000", 2024-09-14T09:05:57.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068984+0000", 2024-09-14T09:05:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2399999999999999e-07 2024-09-14T09:05:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068986+0000", 2024-09-14T09:05:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1409999999999999e-06 2024-09-14T09:05:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068992+0000", 2024-09-14T09:05:57.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3419999999999996e-06 2024-09-14T09:05:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068998+0000", 2024-09-14T09:05:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1359999999999996e-06 2024-09-14T09:05:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093051+0000", 2024-09-14T09:05:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024053741 2024-09-14T09:05:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093120+0000", 2024-09-14T09:05:57.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8714000000000007e-05 2024-09-14T09:05:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093191+0000", 2024-09-14T09:05:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0988999999999994e-05 2024-09-14T09:05:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31214 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.069004+0000", 2024-09-14T09:05:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.0422156869999997, 2024-09-14T09:05:57.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024362755, 2024-09-14T09:05:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31214 2024-09-14T09:05:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069004+0000", 2024-09-14T09:05:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069004+0000", 2024-09-14T09:05:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069005+0000", 2024-09-14T09:05:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3099999999999998e-07 2024-09-14T09:05:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069013+0000", 2024-09-14T09:05:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2330000000000006e-06 2024-09-14T09:05:57.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069020+0000", 2024-09-14T09:05:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9e-06 2024-09-14T09:05:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069025+0000", 2024-09-14T09:05:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0479999999999998e-06 2024-09-14T09:05:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093219+0000", 2024-09-14T09:05:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024193429999999998 2024-09-14T09:05:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093283+0000", 2024-09-14T09:05:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4449000000000006e-05 2024-09-14T09:05:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093367+0000", 2024-09-14T09:05:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4264000000000005e-05 2024-09-14T09:05:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31215 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 264192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.069033+0000", 2024-09-14T09:05:57.177 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.0421870589999997, 2024-09-14T09:05:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024520486000000001, 2024-09-14T09:05:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31215 2024-09-14T09:05:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069033+0000", 2024-09-14T09:05:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069033+0000", 2024-09-14T09:05:57.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069034+0000", 2024-09-14T09:05:57.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6199999999999998e-07 2024-09-14T09:05:57.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069036+0000", 2024-09-14T09:05:57.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.283e-06 2024-09-14T09:05:57.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069043+0000", 2024-09-14T09:05:57.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9330000000000004e-06 2024-09-14T09:05:57.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069047+0000", 2024-09-14T09:05:57.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6940000000000003e-06 2024-09-14T09:05:57.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093396+0000", 2024-09-14T09:05:57.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024348452 2024-09-14T09:05:57.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093464+0000", 2024-09-14T09:05:57.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7879999999999994e-05 2024-09-14T09:05:57.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093554+0000", 2024-09-14T09:05:57.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9782000000000002e-05 2024-09-14T09:05:57.181 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.181 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.181 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.181 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31216 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 282624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:57.182 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.069086+0000", 2024-09-14T09:05:57.182 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.0421343270000003, 2024-09-14T09:05:57.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024651639999999999, 2024-09-14T09:05:57.182 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.182 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.182 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.182 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.182 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.182 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.182 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31216 2024-09-14T09:05:57.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.182 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069086+0000", 2024-09-14T09:05:57.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069086+0000", 2024-09-14T09:05:57.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069086+0000", 2024-09-14T09:05:57.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4499999999999997e-07 2024-09-14T09:05:57.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069089+0000", 2024-09-14T09:05:57.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4040000000000002e-06 2024-09-14T09:05:57.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069097+0000", 2024-09-14T09:05:57.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7770000000000002e-06 2024-09-14T09:05:57.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069102+0000", 2024-09-14T09:05:57.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6359999999999999e-06 2024-09-14T09:05:57.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093581+0000", 2024-09-14T09:05:57.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024478771999999999 2024-09-14T09:05:57.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093648+0000", 2024-09-14T09:05:57.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6824999999999998e-05 2024-09-14T09:05:57.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093737+0000", 2024-09-14T09:05:57.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9580999999999995e-05 2024-09-14T09:05:57.186 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.186 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.186 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.186 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31217 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:57.186 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.069110+0000", 2024-09-14T09:05:57.186 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.0421106360000003, 2024-09-14T09:05:57.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024872781, 2024-09-14T09:05:57.186 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.186 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.186 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.186 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31217 2024-09-14T09:05:57.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.187 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069110+0000", 2024-09-14T09:05:57.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069110+0000", 2024-09-14T09:05:57.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069110+0000", 2024-09-14T09:05:57.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5899999999999996e-07 2024-09-14T09:05:57.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069112+0000", 2024-09-14T09:05:57.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7460000000000001e-06 2024-09-14T09:05:57.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069118+0000", 2024-09-14T09:05:57.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6810000000000003e-06 2024-09-14T09:05:57.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069128+0000", 2024-09-14T09:05:57.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0675e-05 2024-09-14T09:05:57.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093816+0000", 2024-09-14T09:05:57.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024687333999999998 2024-09-14T09:05:57.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093887+0000", 2024-09-14T09:05:57.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1614999999999998e-05 2024-09-14T09:05:57.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093982+0000", 2024-09-14T09:05:57.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5171000000000001e-05 2024-09-14T09:05:57.190 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.190 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.190 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.190 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:37541 3.19 3:9e83327d:::rbd_data.115eb07a8abd.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:05:57.190 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:56.440900+0000", 2024-09-14T09:05:57.190 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.67032041099999995, 2024-09-14T09:05:57.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.044952620999999998, 2024-09-14T09:05:57.190 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.190 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.190 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:57.190 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.191 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 37541 2024-09-14T09:05:57.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.191 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.440900+0000", 2024-09-14T09:05:57.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.440900+0000", 2024-09-14T09:05:57.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.440902+0000", 2024-09-14T09:05:57.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7680000000000001e-06 2024-09-14T09:05:57.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.440908+0000", 2024-09-14T09:05:57.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5880000000000003e-06 2024-09-14T09:05:57.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.440926+0000", 2024-09-14T09:05:57.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7982000000000002e-05 2024-09-14T09:05:57.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.440936+0000", 2024-09-14T09:05:57.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9389999999999992e-06 2024-09-14T09:05:57.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.440995+0000", 2024-09-14T09:05:57.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8981999999999999e-05 2024-09-14T09:05:57.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.441058+0000", 2024-09-14T09:05:57.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2686000000000003e-05 2024-09-14T09:05:57.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:05:57.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.443155+0000", 2024-09-14T09:05:57.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0020971190000000002 2024-09-14T09:05:57.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:57.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.485711+0000", 2024-09-14T09:05:57.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042556121000000002 2024-09-14T09:05:57.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:57.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.485775+0000", 2024-09-14T09:05:57.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4268000000000002e-05 2024-09-14T09:05:57.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:57.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.485817+0000", 2024-09-14T09:05:57.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1743000000000003e-05 2024-09-14T09:05:57.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.485852+0000", 2024-09-14T09:05:57.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5425e-05 2024-09-14T09:05:57.195 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.196 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.196 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.196 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:37541 3.19 e48/23 ondisk, result = 0)", 2024-09-14T09:05:57.196 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:56.449519+0000", 2024-09-14T09:05:57.196 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.66170075500000003, 2024-09-14T09:05:57.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036340787999999999, 2024-09-14T09:05:57.196 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.196 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.196 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.196 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.449519+0000", 2024-09-14T09:05:57.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.449519+0000", 2024-09-14T09:05:57.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.449521+0000", 2024-09-14T09:05:57.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.702e-06 2024-09-14T09:05:57.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.449529+0000", 2024-09-14T09:05:57.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3559999999999993e-06 2024-09-14T09:05:57.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.449553+0000", 2024-09-14T09:05:57.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3337e-05 2024-09-14T09:05:57.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.449567+0000", 2024-09-14T09:05:57.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.416e-05 2024-09-14T09:05:57.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.485762+0000", 2024-09-14T09:05:57.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036194573000000001 2024-09-14T09:05:57.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.485770+0000", 2024-09-14T09:05:57.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4789999999999996e-06 2024-09-14T09:05:57.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.485860+0000", 2024-09-14T09:05:57.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0180999999999996e-05 2024-09-14T09:05:57.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.199 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.199 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.199 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:57.200 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-09-14T09:05:57.171+0000 7f321da70640 -1 mgr.server reply reply (2) No such file or directory no change 2024-09-14T09:05:57.201 INFO:teuthology.orchestra.run.smithi176.stdout:1339 punch from 0x29c0866 to 0x29d05e3, (0xfd7d bytes) 2024-09-14T09:05:57.201 INFO:teuthology.orchestra.run.smithi176.stdout:1341 write 0x3e54b32 thru 0x3e64813 (0xfce2 bytes) 2024-09-14T09:05:57.201 INFO:teuthology.orchestra.run.smithi176.stdout:1343 write 0x7703b39 thru 0x770a33d (0x6805 bytes) 2024-09-14T09:05:57.202 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:05:57.203 INFO:teuthology.orchestra.run.smithi176.stdout:1344 writesame 0x8510c7 thru 0x855782 data_size 0x12(0x46bc bytes) 2024-09-14T09:05:57.203 INFO:teuthology.orchestra.run.smithi176.stdout:1345 write 0xec92d11 thru 0xec9d47b (0xa76b bytes) 2024-09-14T09:05:57.203 INFO:teuthology.orchestra.run.smithi176.stdout:1346 write 0x6903fe0 thru 0x6905339 (0x135a bytes) 2024-09-14T09:05:57.203 INFO:teuthology.orchestra.run.smithi176.stdout:1348 trunc from 0xec9d47c to 0x8b96263 2024-09-14T09:05:57.204 INFO:teuthology.orchestra.run.smithi176.stdout:moved 0 / 82 (0%) 2024-09-14T09:05:57.204 INFO:teuthology.orchestra.run.smithi176.stdout:avg 10.25 2024-09-14T09:05:57.204 INFO:teuthology.orchestra.run.smithi176.stdout:stddev 4.4088 -> 4.4088 (expected baseline 2.99479) 2024-09-14T09:05:57.204 INFO:teuthology.orchestra.run.smithi176.stdout:min osd.4 with 2 -> 2 pgs (0.195122 -> 0.195122 * mean) 2024-09-14T09:05:57.204 INFO:teuthology.orchestra.run.smithi176.stdout:max osd.1 with 16 -> 16 pgs (1.56098 -> 1.56098 * mean) 2024-09-14T09:05:57.204 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:05:57.204 INFO:teuthology.orchestra.run.smithi176.stdout:oload 110 2024-09-14T09:05:57.204 INFO:teuthology.orchestra.run.smithi176.stdout:max_change 0.05 2024-09-14T09:05:57.204 INFO:teuthology.orchestra.run.smithi176.stdout:max_change_osds 4 2024-09-14T09:05:57.204 INFO:teuthology.orchestra.run.smithi176.stdout:average_utilization 0.0003 2024-09-14T09:05:57.204 INFO:teuthology.orchestra.run.smithi176.stdout:overload_utilization 0.0004 2024-09-14T09:05:57.204 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 weight 0.1599 -> 0.1635 2024-09-14T09:05:57.205 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 weight 0.7667 -> 0.7785 2024-09-14T09:05:57.205 INFO:teuthology.orchestra.run.smithi176.stderr:no change 2024-09-14T09:05:57.205 INFO:teuthology.orchestra.run.smithi176.stdout:1349 read 0x1fc4378 thru 0x1fc876f (0x43f8 bytes) 2024-09-14T09:05:57.205 INFO:teuthology.orchestra.run.smithi176.stdout:1351 read 0x2ed3dad thru 0x2ed4bc9 (0xe1d bytes) 2024-09-14T09:05:57.206 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:57.206 INFO:teuthology.orchestra.run.smithi176.stdout:1353 trunc from 0x8b96263 to 0x7e81971 2024-09-14T09:05:57.207 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-14T09:05:57.207 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-14T09:05:57.207 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:57.196+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:57.208 INFO:teuthology.orchestra.run.smithi176.stdout:1354 writesame 0xd171429 thru 0xd17a708 data_size 0xa0(0x92e0 bytes) 2024-09-14T09:05:57.208 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:57.221 INFO:teuthology.orchestra.run.smithi176.stdout:1356 trunc from 0xd17a709 to 0x4049bba 2024-09-14T09:05:57.255 INFO:teuthology.orchestra.run.smithi176.stdout:1357 write 0x722ae77 thru 0x7239b4f (0xecd9 bytes) 2024-09-14T09:05:57.261 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:57.268 INFO:teuthology.orchestra.run.smithi176.stdout:1359 trunc from 0x7239b50 to 0xc770e4c 2024-09-14T09:05:57.275 INFO:teuthology.orchestra.run.smithi176.stdout:1360 punch from 0xb578759 to 0xb57dda0, (0x5647 bytes) 2024-09-14T09:05:57.275 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:57.284 INFO:teuthology.orchestra.run.smithi176.stdout:1363 write 0xd3ce6ac thru 0xd3da639 (0xbf8e bytes) 2024-09-14T09:05:57.290 INFO:teuthology.orchestra.run.smithi176.stdout:1364 writesame 0x5403d2c thru 0x540541d data_size 0x21(0x16f2 bytes) 2024-09-14T09:05:57.299 INFO:teuthology.orchestra.run.smithi176.stdout:1366 writesame 0xd87c297 thru 0xd884622 data_size 0x2(0x838c bytes) 2024-09-14T09:05:57.301 INFO:teuthology.orchestra.run.smithi176.stdout:1367 write 0x916a428 thru 0x91752ea (0xaec3 bytes) 2024-09-14T09:05:57.301 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:57.303+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:57.317 INFO:teuthology.orchestra.run.smithi176.stdout:1368 trunc from 0xd884623 to 0x9152419 2024-09-14T09:05:57.328 INFO:teuthology.orchestra.run.smithi176.stdout:1369 read 0x1b94c01 thru 0x1b97c29 (0x3029 bytes) 2024-09-14T09:05:57.339 INFO:teuthology.orchestra.run.smithi176.stdout:1370 writesame 0xbfdbcd1 thru 0xbfeb5a5 data_size 0x1(0xf8d5 bytes) 2024-09-14T09:05:57.341 INFO:teuthology.orchestra.run.smithi176.stdout:1371 punch from 0x68301cd to 0x683cf35, (0xcd68 bytes) 2024-09-14T09:05:57.342 INFO:teuthology.orchestra.run.smithi176.stdout:1373 read 0x67b85fc thru 0x67bfd0e (0x7713 bytes) 2024-09-14T09:05:57.342 INFO:teuthology.orchestra.run.smithi176.stdout:1374 read 0x24ccbb5 thru 0x24cd965 (0xdb1 bytes) 2024-09-14T09:05:57.343 INFO:teuthology.orchestra.run.smithi176.stdout:1375 write 0x735309f thru 0x735ccd2 (0x9c34 bytes) 2024-09-14T09:05:57.344 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:57.344 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:57.344 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:57.344 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:57.360 INFO:teuthology.orchestra.run.smithi176.stdout:1376 writesame 0xcc1127a thru 0xcc15906 data_size 0x1(0x468d bytes) 2024-09-14T09:05:57.362 INFO:teuthology.orchestra.run.smithi176.stdout:1378 read 0x67a9255 thru 0x67b486a (0xb616 bytes) 2024-09-14T09:05:57.363 INFO:teuthology.orchestra.run.smithi176.stdout:1379 write 0x4fde311 thru 0x4fe677e (0x846e bytes) 2024-09-14T09:05:57.364 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:05:57.368 INFO:teuthology.orchestra.run.smithi176.stdout:1380 write 0x8140d19 thru 0x814c02e (0xb316 bytes) 2024-09-14T09:05:57.378 INFO:teuthology.orchestra.run.smithi176.stdout:1381 read 0x94469c4 thru 0x9450d91 (0xa3ce bytes) 2024-09-14T09:05:57.378 INFO:teuthology.orchestra.run.smithi176.stdout:1382 write 0x97b5e26 thru 0x97ba956 (0x4b31 bytes) 2024-09-14T09:05:57.386 INFO:teuthology.orchestra.run.smithi176.stdout:1384 read 0x4c58f0f thru 0x4c5d54c (0x463e bytes) 2024-09-14T09:05:57.387 INFO:teuthology.orchestra.run.smithi176.stdout:1385 writesame 0x6ab8898 thru 0x6ac71a8 data_size 0x1(0xe911 bytes) 2024-09-14T09:05:57.402 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:57.400+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:57.484 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:57.484 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:57.484 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:57.484 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:57.484 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:57.497 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:05:57.505 INFO:teuthology.orchestra.run.smithi176.stdout:1386 writesame 0x56234a thru 0x570f63 data_size 0x2f(0xec1a bytes) 2024-09-14T09:05:57.505 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:57.500+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:57.523 INFO:teuthology.orchestra.run.smithi176.stdout:1387 write 0xeb9d395 thru 0xeba7720 (0xa38c bytes) 2024-09-14T09:05:57.531 INFO:teuthology.orchestra.run.smithi176.stdout:1388 read 0x1b4f44 thru 0x1bfded (0xaeaa bytes) 2024-09-14T09:05:57.533 INFO:teuthology.orchestra.run.smithi176.stdout:1389 writesame 0x2349d21 thru 0x2352c15 data_size 0xb(0x8ef5 bytes) 2024-09-14T09:05:57.539 INFO:teuthology.orchestra.run.smithi176.stdout:1390 trunc from 0xeba7721 to 0xd712272 2024-09-14T09:05:57.559 INFO:teuthology.orchestra.run.smithi176.stdout:1393 write 0x3e7cfef thru 0x3e7ee40 (0x1e52 bytes) 2024-09-14T09:05:57.580 INFO:teuthology.orchestra.run.smithi176.stdout:1395 trunc from 0xd712272 to 0x2f286af 2024-09-14T09:05:57.604 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:57.604 INFO:teuthology.orchestra.run.smithi176.stdout:1397 read 0x50097a thru 0x50f99c (0xf023 bytes) 2024-09-14T09:05:57.605 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:57.607+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:57.605 INFO:teuthology.orchestra.run.smithi176.stdout:1398 punch from 0xb818dc to 0xb8cb83, (0xb2a7 bytes) 2024-09-14T09:05:57.605 INFO:teuthology.orchestra.run.smithi176.stdout:1399 read 0x1334385 thru 0x1341fa3 (0xdc1f bytes) 2024-09-14T09:05:57.606 INFO:teuthology.orchestra.run.smithi176.stdout:1400 punch from 0x22b0811 to 0x22b2a53, (0x2242 bytes) 2024-09-14T09:05:57.633 INFO:teuthology.orchestra.run.smithi176.stdout:1401 write 0xe4ee7f4 thru 0xe4f525d (0x6a6a bytes) 2024-09-14T09:05:57.640 INFO:teuthology.orchestra.run.smithi176.stdout:1403 writesame 0x9196635 thru 0x91a20a5 data_size 0x1(0xba71 bytes) 2024-09-14T09:05:57.662 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:57.662 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:57.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18383 3.9 3:9252f7c9:::rbd_data.115e3abd3b2c.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:05:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:05:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.503445737, 2024-09-14T09:05:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056433830999999997, 2024-09-14T09:05:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18383 2024-09-14T09:05:57.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:05:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:05:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158423+0000", 2024-09-14T09:05:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T09:05:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158429+0000", 2024-09-14T09:05:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0179999999999998e-06 2024-09-14T09:05:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158440+0000", 2024-09-14T09:05:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0769e-05 2024-09-14T09:05:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158445+0000", 2024-09-14T09:05:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5840000000000002e-06 2024-09-14T09:05:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158495+0000", 2024-09-14T09:05:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9129999999999999e-05 2024-09-14T09:05:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158598+0000", 2024-09-14T09:05:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102938 2024-09-14T09:05:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:05:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.161048+0000", 2024-09-14T09:05:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002450336 2024-09-14T09:05:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214605+0000", 2024-09-14T09:05:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053557526000000001 2024-09-14T09:05:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214634+0000", 2024-09-14T09:05:57.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8483000000000002e-05 2024-09-14T09:05:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214731+0000", 2024-09-14T09:05:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7026000000000002e-05 2024-09-14T09:05:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214856+0000", 2024-09-14T09:05:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012474799999999999 2024-09-14T09:05:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18383 3.9 e38/34 ondisk, result = 0)", 2024-09-14T09:05:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:05:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.493129256, 2024-09-14T09:05:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045880575999999999, 2024-09-14T09:05:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:05:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:05:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168739+0000", 2024-09-14T09:05:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3900000000000004e-07 2024-09-14T09:05:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168744+0000", 2024-09-14T09:05:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5929999999999999e-06 2024-09-14T09:05:57.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168754+0000", 2024-09-14T09:05:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0686999999999999e-05 2024-09-14T09:05:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168761+0000", 2024-09-14T09:05:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6340000000000003e-06 2024-09-14T09:05:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214578+0000", 2024-09-14T09:05:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045817443999999999 2024-09-14T09:05:57.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214596+0000", 2024-09-14T09:05:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7159e-05 2024-09-14T09:05:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214619+0000", 2024-09-14T09:05:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.332e-05 2024-09-14T09:05:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19152 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14159058~8940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:05:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:05:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 13.339216799999999, 2024-09-14T09:05:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12891994500000001, 2024-09-14T09:05:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19152 2024-09-14T09:05:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:05:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:05:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322653+0000", 2024-09-14T09:05:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3750000000000001e-06 2024-09-14T09:05:57.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329647+0000", 2024-09-14T09:05:57.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069941960000000003 2024-09-14T09:05:57.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329673+0000", 2024-09-14T09:05:57.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5760000000000001e-05 2024-09-14T09:05:57.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329684+0000", 2024-09-14T09:05:57.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0474e-05 2024-09-14T09:05:57.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329776+0000", 2024-09-14T09:05:57.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2467999999999996e-05 2024-09-14T09:05:57.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329987+0000", 2024-09-14T09:05:57.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021133 2024-09-14T09:05:57.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:05:57.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.331303+0000", 2024-09-14T09:05:57.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001315629 2024-09-14T09:05:57.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:57.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.411090+0000", 2024-09-14T09:05:57.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079787477999999995 2024-09-14T09:05:57.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:57.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451463+0000", 2024-09-14T09:05:57.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040372312 2024-09-14T09:05:57.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:57.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451523+0000", 2024-09-14T09:05:57.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0420000000000001e-05 2024-09-14T09:05:57.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451571+0000", 2024-09-14T09:05:57.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7503000000000002e-05 2024-09-14T09:05:57.678 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.678 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.678 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.678 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20925 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:57.678 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:05:57.678 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.9899180199999993, 2024-09-14T09:05:57.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017084327999999999, 2024-09-14T09:05:57.679 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.679 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.679 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.679 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.679 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.679 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.679 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20925 2024-09-14T09:05:57.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.679 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:05:57.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:05:57.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:05:57.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3e-07 2024-09-14T09:05:57.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671953+0000", 2024-09-14T09:05:57.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0299999999999998e-06 2024-09-14T09:05:57.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671964+0000", 2024-09-14T09:05:57.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1e-05 2024-09-14T09:05:57.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671971+0000", 2024-09-14T09:05:57.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9929999999999998e-06 2024-09-14T09:05:57.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.687941+0000", 2024-09-14T09:05:57.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015969357 2024-09-14T09:05:57.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.688070+0000", 2024-09-14T09:05:57.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000129858 2024-09-14T09:05:57.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689034+0000", 2024-09-14T09:05:57.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00096336000000000002 2024-09-14T09:05:57.682 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.682 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.682 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.682 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20926 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:57.683 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:05:57.683 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.9898848000000005, 2024-09-14T09:05:57.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018572207, 2024-09-14T09:05:57.683 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.683 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.683 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.683 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.683 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.683 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.683 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20926 2024-09-14T09:05:57.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.683 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:05:57.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:05:57.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:05:57.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:05:57.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671990+0000", 2024-09-14T09:05:57.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5769999999999999e-06 2024-09-14T09:05:57.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672001+0000", 2024-09-14T09:05:57.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1293e-05 2024-09-14T09:05:57.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672008+0000", 2024-09-14T09:05:57.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9099999999999999e-06 2024-09-14T09:05:57.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689147+0000", 2024-09-14T09:05:57.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017139060000000001 2024-09-14T09:05:57.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689324+0000", 2024-09-14T09:05:57.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017640200000000001 2024-09-14T09:05:57.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690555+0000", 2024-09-14T09:05:57.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001231212 2024-09-14T09:05:57.686 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.687 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.687 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.687 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20927 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2621440~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:57.687 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:05:57.687 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.9898478340000008, 2024-09-14T09:05:57.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020002227000000001, 2024-09-14T09:05:57.687 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.687 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.687 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.687 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.687 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.687 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.687 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20927 2024-09-14T09:05:57.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.688 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:05:57.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:05:57.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:05:57.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9699999999999995e-07 2024-09-14T09:05:57.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672024+0000", 2024-09-14T09:05:57.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:05:57.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672036+0000", 2024-09-14T09:05:57.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.119e-05 2024-09-14T09:05:57.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672044+0000", 2024-09-14T09:05:57.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8370000000000003e-06 2024-09-14T09:05:57.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690602+0000", 2024-09-14T09:05:57.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018557648 2024-09-14T09:05:57.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690805+0000", 2024-09-14T09:05:57.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020282899999999999 2024-09-14T09:05:57.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692022+0000", 2024-09-14T09:05:57.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012170429999999999 2024-09-14T09:05:57.691 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.691 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.691 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.691 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20928 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 917504~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:57.691 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:05:57.691 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.9898116940000001, 2024-09-14T09:05:57.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021623738, 2024-09-14T09:05:57.691 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.691 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.691 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.692 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.692 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.692 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.692 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20928 2024-09-14T09:05:57.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.692 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:05:57.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:05:57.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672057+0000", 2024-09-14T09:05:57.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:05:57.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672059+0000", 2024-09-14T09:05:57.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9270000000000001e-06 2024-09-14T09:05:57.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672070+0000", 2024-09-14T09:05:57.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0773e-05 2024-09-14T09:05:57.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672077+0000", 2024-09-14T09:05:57.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9600000000000003e-06 2024-09-14T09:05:57.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692154+0000", 2024-09-14T09:05:57.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020076482 2024-09-14T09:05:57.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692419+0000", 2024-09-14T09:05:57.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026517400000000002 2024-09-14T09:05:57.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693680+0000", 2024-09-14T09:05:57.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012607390000000001 2024-09-14T09:05:57.695 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.695 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.695 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.696 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20929 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2752512~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:57.696 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:05:57.696 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.989778652, 2024-09-14T09:05:57.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022898728, 2024-09-14T09:05:57.696 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.696 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.696 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.696 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.696 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.696 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.696 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20929 2024-09-14T09:05:57.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.696 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:05:57.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:05:57.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672090+0000", 2024-09-14T09:05:57.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:05:57.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672092+0000", 2024-09-14T09:05:57.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8940000000000002e-06 2024-09-14T09:05:57.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672103+0000", 2024-09-14T09:05:57.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0815999999999999e-05 2024-09-14T09:05:57.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672110+0000", 2024-09-14T09:05:57.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9469999999999997e-06 2024-09-14T09:05:57.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693737+0000", 2024-09-14T09:05:57.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021626467999999999 2024-09-14T09:05:57.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693962+0000", 2024-09-14T09:05:57.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225474 2024-09-14T09:05:57.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.694988+0000", 2024-09-14T09:05:57.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001025469 2024-09-14T09:05:57.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.700 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.700 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20930 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:57.700 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:05:57.700 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.9897455579999992, 2024-09-14T09:05:57.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024303286, 2024-09-14T09:05:57.700 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.701 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.701 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.701 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20930 2024-09-14T09:05:57.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.701 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:05:57.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:05:57.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672123+0000", 2024-09-14T09:05:57.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:05:57.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672125+0000", 2024-09-14T09:05:57.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7300000000000001e-06 2024-09-14T09:05:57.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672134+0000", 2024-09-14T09:05:57.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7930000000000003e-06 2024-09-14T09:05:57.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672141+0000", 2024-09-14T09:05:57.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9029999999999998e-06 2024-09-14T09:05:57.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695087+0000", 2024-09-14T09:05:57.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022946013000000001 2024-09-14T09:05:57.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695350+0000", 2024-09-14T09:05:57.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026266000000000001 2024-09-14T09:05:57.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696425+0000", 2024-09-14T09:05:57.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001075527 2024-09-14T09:05:57.704 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.704 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.704 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.704 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20931 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:57.704 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:05:57.705 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.9897123289999996, 2024-09-14T09:05:57.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025331519, 2024-09-14T09:05:57.705 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.705 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.705 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.705 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.705 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.705 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.705 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20931 2024-09-14T09:05:57.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.706 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:05:57.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:05:57.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672156+0000", 2024-09-14T09:05:57.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4300000000000002e-07 2024-09-14T09:05:57.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672159+0000", 2024-09-14T09:05:57.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T09:05:57.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672170+0000", 2024-09-14T09:05:57.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1192999999999999e-05 2024-09-14T09:05:57.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672177+0000", 2024-09-14T09:05:57.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9059999999999996e-06 2024-09-14T09:05:57.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696532+0000", 2024-09-14T09:05:57.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024355291000000001 2024-09-14T09:05:57.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696755+0000", 2024-09-14T09:05:57.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022297699999999999 2024-09-14T09:05:57.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697487+0000", 2024-09-14T09:05:57.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00073171499999999997 2024-09-14T09:05:57.709 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.709 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.709 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.709 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20932 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1179648~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:57.709 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:05:57.709 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.9896788860000001, 2024-09-14T09:05:57.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026104083, 2024-09-14T09:05:57.709 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.709 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.710 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.710 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.710 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.710 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.710 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20932 2024-09-14T09:05:57.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.710 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:05:57.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:05:57.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:05:57.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0699999999999996e-07 2024-09-14T09:05:57.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672192+0000", 2024-09-14T09:05:57.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7700000000000002e-06 2024-09-14T09:05:57.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672203+0000", 2024-09-14T09:05:57.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0956e-05 2024-09-14T09:05:57.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672211+0000", 2024-09-14T09:05:57.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3769999999999996e-06 2024-09-14T09:05:57.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697530+0000", 2024-09-14T09:05:57.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025318411999999998 2024-09-14T09:05:57.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697640+0000", 2024-09-14T09:05:57.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011045299999999999 2024-09-14T09:05:57.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698293+0000", 2024-09-14T09:05:57.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065240800000000005 2024-09-14T09:05:57.713 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.713 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.713 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.713 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20933 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3014656~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:57.713 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:05:57.713 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.9896446829999999, 2024-09-14T09:05:57.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026558875999999999, 2024-09-14T09:05:57.714 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.714 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.714 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.714 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.714 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.714 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.714 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20933 2024-09-14T09:05:57.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.714 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:05:57.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:05:57.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672224+0000", 2024-09-14T09:05:57.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2399999999999997e-07 2024-09-14T09:05:57.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672234+0000", 2024-09-14T09:05:57.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0793e-05 2024-09-14T09:05:57.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672246+0000", 2024-09-14T09:05:57.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659e-05 2024-09-14T09:05:57.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672255+0000", 2024-09-14T09:05:57.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5730000000000008e-06 2024-09-14T09:05:57.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698326+0000", 2024-09-14T09:05:57.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026071414000000001 2024-09-14T09:05:57.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698374+0000", 2024-09-14T09:05:57.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8032999999999999e-05 2024-09-14T09:05:57.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698782+0000", 2024-09-14T09:05:57.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040768 2024-09-14T09:05:57.717 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.717 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.717 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.718 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20934 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1310720~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:57.718 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:05:57.718 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.9896028070000007, 2024-09-14T09:05:57.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027292067999999999, 2024-09-14T09:05:57.718 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.718 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.718 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.718 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.718 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.718 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.718 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20934 2024-09-14T09:05:57.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.719 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:05:57.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:05:57.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672266+0000", 2024-09-14T09:05:57.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6299999999999993e-07 2024-09-14T09:05:57.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672269+0000", 2024-09-14T09:05:57.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8569999999999999e-06 2024-09-14T09:05:57.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672280+0000", 2024-09-14T09:05:57.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1236999999999999e-05 2024-09-14T09:05:57.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672287+0000", 2024-09-14T09:05:57.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8260000000000004e-06 2024-09-14T09:05:57.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698820+0000", 2024-09-14T09:05:57.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026533055999999999 2024-09-14T09:05:57.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698933+0000", 2024-09-14T09:05:57.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011291 2024-09-14T09:05:57.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699557+0000", 2024-09-14T09:05:57.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062421899999999997 2024-09-14T09:05:57.721 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.722 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.722 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.722 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20935 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:57.722 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:05:57.722 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.9895690469999998, 2024-09-14T09:05:57.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028108610999999999, 2024-09-14T09:05:57.722 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.722 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.722 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.722 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.722 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.722 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.723 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20935 2024-09-14T09:05:57.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.723 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:05:57.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:05:57.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672299+0000", 2024-09-14T09:05:57.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T09:05:57.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672302+0000", 2024-09-14T09:05:57.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7640000000000001e-06 2024-09-14T09:05:57.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672313+0000", 2024-09-14T09:05:57.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1080000000000001e-05 2024-09-14T09:05:57.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672321+0000", 2024-09-14T09:05:57.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6490000000000008e-06 2024-09-14T09:05:57.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699587+0000", 2024-09-14T09:05:57.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027265884000000001 2024-09-14T09:05:57.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699631+0000", 2024-09-14T09:05:57.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4947999999999997e-05 2024-09-14T09:05:57.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700407+0000", 2024-09-14T09:05:57.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00077559299999999999 2024-09-14T09:05:57.726 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.726 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.726 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.726 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20936 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1441792~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:57.726 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:05:57.726 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.989534871, 2024-09-14T09:05:57.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028784750000000001, 2024-09-14T09:05:57.726 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.726 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.726 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.726 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20936 2024-09-14T09:05:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:05:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:05:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:05:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4700000000000001e-07 2024-09-14T09:05:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672336+0000", 2024-09-14T09:05:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6730000000000001e-06 2024-09-14T09:05:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672347+0000", 2024-09-14T09:05:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0710000000000001e-05 2024-09-14T09:05:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672354+0000", 2024-09-14T09:05:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4630000000000004e-06 2024-09-14T09:05:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700442+0000", 2024-09-14T09:05:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028087951 2024-09-14T09:05:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700557+0000", 2024-09-14T09:05:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011461800000000001 2024-09-14T09:05:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701117+0000", 2024-09-14T09:05:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056058799999999999 2024-09-14T09:05:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20937 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:05:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.9895017450000001, 2024-09-14T09:05:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029493028000000001, 2024-09-14T09:05:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20937 2024-09-14T09:05:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:05:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:05:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:05:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T09:05:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672384+0000", 2024-09-14T09:05:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7127000000000001e-05 2024-09-14T09:05:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672395+0000", 2024-09-14T09:05:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759999999999999e-05 2024-09-14T09:05:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672402+0000", 2024-09-14T09:05:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0960000000000003e-06 2024-09-14T09:05:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701147+0000", 2024-09-14T09:05:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028744241 2024-09-14T09:05:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701246+0000", 2024-09-14T09:05:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9031999999999996e-05 2024-09-14T09:05:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701859+0000", 2024-09-14T09:05:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061310499999999997 2024-09-14T09:05:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20938 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:05:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.9894545850000007, 2024-09-14T09:05:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030109219999999999, 2024-09-14T09:05:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20938 2024-09-14T09:05:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:05:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:05:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672414+0000", 2024-09-14T09:05:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-07 2024-09-14T09:05:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672417+0000", 2024-09-14T09:05:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:05:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672428+0000", 2024-09-14T09:05:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0873000000000001e-05 2024-09-14T09:05:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672435+0000", 2024-09-14T09:05:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:05:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701884+0000", 2024-09-14T09:05:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029449811999999999 2024-09-14T09:05:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701934+0000", 2024-09-14T09:05:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9703000000000002e-05 2024-09-14T09:05:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702522+0000", 2024-09-14T09:05:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00058811200000000001 2024-09-14T09:05:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20939 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:05:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.9894224519999995, 2024-09-14T09:05:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030779259, 2024-09-14T09:05:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20939 2024-09-14T09:05:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:05:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:05:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672446+0000", 2024-09-14T09:05:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:05:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672449+0000", 2024-09-14T09:05:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7930000000000002e-06 2024-09-14T09:05:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672461+0000", 2024-09-14T09:05:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T09:05:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672471+0000", 2024-09-14T09:05:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0807e-05 2024-09-14T09:05:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702551+0000", 2024-09-14T09:05:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030079228 2024-09-14T09:05:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702664+0000", 2024-09-14T09:05:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011351800000000001 2024-09-14T09:05:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.703224+0000", 2024-09-14T09:05:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056026999999999997 2024-09-14T09:05:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29321 3.9 3:90303d3a:::rbd_data.115ebc3c67d1.0000000000000014:c [read 770048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:05:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8423391269999998, 2024-09-14T09:05:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015942787, 2024-09-14T09:05:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29321 2024-09-14T09:05:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:05:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:05:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819529+0000", 2024-09-14T09:05:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8800000000000003e-07 2024-09-14T09:05:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819531+0000", 2024-09-14T09:05:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9439999999999999e-06 2024-09-14T09:05:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819537+0000", 2024-09-14T09:05:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.037e-06 2024-09-14T09:05:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819542+0000", 2024-09-14T09:05:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7670000000000003e-06 2024-09-14T09:05:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835320+0000", 2024-09-14T09:05:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015778119 2024-09-14T09:05:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835384+0000", 2024-09-14T09:05:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3897000000000002e-05 2024-09-14T09:05:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835471+0000", 2024-09-14T09:05:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7534999999999995e-05 2024-09-14T09:05:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29322 3.9 3:90303d3a:::rbd_data.115ebc3c67d1.0000000000000014:c [read 780288~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:05:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:05:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8423196170000002, 2024-09-14T09:05:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016069904999999999, 2024-09-14T09:05:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29322 2024-09-14T09:05:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:05:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:05:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:05:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9499999999999998e-07 2024-09-14T09:05:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819550+0000", 2024-09-14T09:05:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8309999999999999e-06 2024-09-14T09:05:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819556+0000", 2024-09-14T09:05:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1299999999999998e-06 2024-09-14T09:05:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819561+0000", 2024-09-14T09:05:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.707e-06 2024-09-14T09:05:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835500+0000", 2024-09-14T09:05:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015939456000000001 2024-09-14T09:05:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835566+0000", 2024-09-14T09:05:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5285999999999998e-05 2024-09-14T09:05:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835618+0000", 2024-09-14T09:05:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2099999999999999e-05 2024-09-14T09:05:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.753 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:57.753 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:57.753 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:57.754 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:05:57.755 INFO:teuthology.orchestra.run.smithi176.stdout:1405 read 0x5d74f4b thru 0x5d764d9 (0x158f bytes) 2024-09-14T09:05:57.755 INFO:teuthology.orchestra.run.smithi176.stdout:1406 read 0x7cc8c4c thru 0x7cd288c (0x9c41 bytes) 2024-09-14T09:05:57.755 INFO:teuthology.orchestra.run.smithi176.stdout:1407 trunc from 0xe4f525e to 0x6131529 2024-09-14T09:05:57.755 INFO:teuthology.orchestra.run.smithi176.stdout:1408 read 0x34e612f thru 0x34ef02c (0x8efe bytes) 2024-09-14T09:05:57.755 INFO:teuthology.orchestra.run.smithi176.stdout:1409 read 0x4b2b73d thru 0x4b32537 (0x6dfb bytes) 2024-09-14T09:05:57.755 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:57.752+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:57.756 INFO:teuthology.orchestra.run.smithi176.stdout:1410 write 0xe840d07 thru 0xe84af60 (0xa25a bytes) 2024-09-14T09:05:57.762 INFO:teuthology.orchestra.run.smithi176.stdout:1411 read 0x45134f7 thru 0x451aa97 (0x75a1 bytes) 2024-09-14T09:05:57.763 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:57.763 INFO:teuthology.orchestra.run.smithi176.stdout:1414 write 0xa76bc3a thru 0xa770074 (0x443b bytes) 2024-09-14T09:05:57.770 INFO:teuthology.orchestra.run.smithi176.stdout:1415 read 0xacf0012 thru 0xacfd037 (0xd026 bytes) 2024-09-14T09:05:57.770 INFO:teuthology.orchestra.run.smithi176.stdout:1416 read 0x1c7accf thru 0x1c8082b (0x5b5d bytes) 2024-09-14T09:05:57.771 INFO:teuthology.orchestra.run.smithi176.stdout:1418 write 0x4b3465d thru 0x4b41682 (0xd026 bytes) 2024-09-14T09:05:57.778 INFO:teuthology.orchestra.run.smithi176.stdout:1421 read 0x7fd5ced thru 0x7fe0986 (0xac9a bytes) 2024-09-14T09:05:57.779 INFO:teuthology.orchestra.run.smithi176.stdout:1422 punch from 0xac34c79 to 0xac38cb6, (0x403d bytes) 2024-09-14T09:05:57.779 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:57.779 INFO:teuthology.orchestra.run.smithi176.stdout:1425 write 0x81d3a52 thru 0x81da56a (0x6b19 bytes) 2024-09-14T09:05:57.783 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:57.783 INFO:teuthology.orchestra.run.smithi176.stdout:1427 write 0x472b678 thru 0x4732c45 (0x75ce bytes) 2024-09-14T09:05:57.787 INFO:teuthology.orchestra.run.smithi176.stdout:1428 write 0x3314cee thru 0x3318241 (0x3554 bytes) 2024-09-14T09:05:57.791 INFO:teuthology.orchestra.run.smithi176.stdout:1429 write 0x4dfa445 thru 0x4dfdb2a (0x36e6 bytes) 2024-09-14T09:05:57.794 INFO:teuthology.orchestra.run.smithi176.stdout:1431 write 0x49ff04 thru 0x4aced6 (0xcfd3 bytes) 2024-09-14T09:05:57.823 INFO:teuthology.orchestra.run.smithi176.stdout:1432 trunc from 0xe84af61 to 0x1f929de 2024-09-14T09:05:57.854 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:57.852+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:57.865 INFO:teuthology.orchestra.run.smithi176.stdout:1433 write 0x79f89a2 thru 0x7a0722a (0xe889 bytes) 2024-09-14T09:05:57.874 INFO:teuthology.orchestra.run.smithi176.stdout:1434 punch from 0x278b6c0 to 0x2794fb3, (0x98f3 bytes) 2024-09-14T09:05:57.874 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:57.883 INFO:teuthology.orchestra.run.smithi176.stdout:1436 write 0x877e458 thru 0x8785494 (0x703d bytes) 2024-09-14T09:05:57.890 INFO:teuthology.orchestra.run.smithi176.stdout:1438 write 0x6021546 thru 0x602448a (0x2f45 bytes) 2024-09-14T09:05:57.893 INFO:teuthology.orchestra.run.smithi176.stdout:1439 write 0x1211c65 thru 0x1212b11 (0xead bytes) 2024-09-14T09:05:57.895 INFO:teuthology.orchestra.run.smithi176.stdout:1440 read 0x5c9abcf thru 0x5ca14c1 (0x68f3 bytes) 2024-09-14T09:05:57.896 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:57.896 INFO:teuthology.orchestra.run.smithi176.stdout:1443 read 0x2ea42e5 thru 0x2eb3d45 (0xfa61 bytes) 2024-09-14T09:05:57.899 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:57.899 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:57.899 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:05:57.899 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:57.900 INFO:teuthology.orchestra.run.smithi176.stdout:1444 trunc from 0x8785495 to 0x5ffe897 2024-09-14T09:05:57.918 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:05:57.923 INFO:teuthology.orchestra.run.smithi176.stdout:1445 writesame 0x4f0333e thru 0x4f06aff data_size 0x3(0x37c2 bytes) 2024-09-14T09:05:57.928 INFO:teuthology.orchestra.run.smithi176.stdout:1447 read 0x2b0d8b4 thru 0x2b16ea3 (0x95f0 bytes) 2024-09-14T09:05:57.929 INFO:teuthology.orchestra.run.smithi176.stdout:1448 punch from 0x59ba819 to 0x59bdd61, (0x3548 bytes) 2024-09-14T09:05:57.929 INFO:teuthology.orchestra.run.smithi176.stdout:1449 clone 12 order 23 su 65536 sc 2 2024-09-14T09:05:57.956 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:05:57.952+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:05:58.031 INFO:teuthology.orchestra.run.smithi176.stdout:moved 0 / 82 (0%) 2024-09-14T09:05:58.031 INFO:teuthology.orchestra.run.smithi176.stdout:avg 10.25 2024-09-14T09:05:58.031 INFO:teuthology.orchestra.run.smithi176.stdout:stddev 4.4088 -> 4.4088 (expected baseline 2.99479) 2024-09-14T09:05:58.031 INFO:teuthology.orchestra.run.smithi176.stdout:min osd.4 with 2 -> 2 pgs (0.195122 -> 0.195122 * mean) 2024-09-14T09:05:58.031 INFO:teuthology.orchestra.run.smithi176.stdout:max osd.1 with 16 -> 16 pgs (1.56098 -> 1.56098 * mean) 2024-09-14T09:05:58.031 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:05:58.032 INFO:teuthology.orchestra.run.smithi176.stdout:oload 110 2024-09-14T09:05:58.032 INFO:teuthology.orchestra.run.smithi176.stdout:max_change 3 2024-09-14T09:05:58.032 INFO:teuthology.orchestra.run.smithi176.stdout:max_change_osds 4 2024-09-14T09:05:58.032 INFO:teuthology.orchestra.run.smithi176.stdout:average_utilization 0.0003 2024-09-14T09:05:58.032 INFO:teuthology.orchestra.run.smithi176.stdout:overload_utilization 0.0004 2024-09-14T09:05:58.032 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 weight 0.1599 -> 0.1635 2024-09-14T09:05:58.032 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 weight 0.7667 -> 0.7785 2024-09-14T09:05:58.054 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:58.054 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:05:58.054 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:05:58.055 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:05:58.055 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:58.056 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-14T09:05:58.056 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-14T09:05:58.061 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T09:05:58.071 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:05:58.072 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:58.068+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:58.163 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:58.160+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:58.222 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-14T09:05:58.225 INFO:teuthology.orchestra.run.smithi176.stdout:1450 read 0x5e04fcb thru 0x5e0ea50 (0x9a86 bytes) 2024-09-14T09:05:58.228 INFO:teuthology.orchestra.run.smithi176.stdout:1451 write 0x3877487 thru 0x387f556 (0x80d0 bytes) 2024-09-14T09:05:58.242 INFO:teuthology.orchestra.run.smithi176.stdout:1453 read 0x5f4fcae thru 0x5f5a7b3 (0xab06 bytes) 2024-09-14T09:05:58.259 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:58.256+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:58.262 INFO:teuthology.orchestra.run.smithi176.stdout:1454 writesame 0xc8a02bd thru 0xc8a6794 data_size 0xe(0x64d8 bytes) 2024-09-14T09:05:58.262 INFO:teuthology.orchestra.run.smithi176.stdout:1455 writesame 0xab046dd thru 0xab06857 data_size 0x1(0x217b bytes) 2024-09-14T09:05:58.277 INFO:teuthology.orchestra.run.smithi176.stdout:1456 write 0x3be9d0e thru 0x3bef249 (0x553c bytes) 2024-09-14T09:05:58.285 INFO:teuthology.orchestra.run.smithi176.stdout:1457 punch from 0x5ec23b5 to 0x5ecba91, (0x96dc bytes) 2024-09-14T09:05:58.299 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:05:58.299 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:05:58.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:05:58.300 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:05:58.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.300 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:05:58.300 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:05:58.300 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.215944873000002, 2024-09-14T09:05:58.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:05:58.300 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.300 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.300 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:58.300 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:58.300 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:58.300 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:58.300 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:05:58.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.301 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:05:58.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:05:58.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:05:58.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:05:58.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:05:58.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:05:58.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:05:58.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:05:58.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:05:58.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:05:58.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:05:58.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:05:58.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:05:58.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:05:58.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:05:58.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:05:58.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:05:58.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:05:58.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:05:58.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:05:58.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:05:58.304 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.305 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.305 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.305 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:05:58.305 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:05:58.305 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.215890215000002, 2024-09-14T09:05:58.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:05:58.305 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.305 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.305 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:58.305 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:58.305 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:58.305 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:58.306 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:05:58.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.306 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:05:58.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:05:58.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:05:58.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:05:58.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:05:58.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:05:58.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:05:58.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:05:58.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:05:58.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:05:58.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:05:58.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:05:58.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:05:58.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:05:58.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:05:58.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:05:58.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:05:58.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:05:58.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:05:58.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:05:58.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:05:58.311 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.311 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.311 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.311 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:05:58.311 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:05:58.311 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.215784841000001, 2024-09-14T09:05:58.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:05:58.311 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.312 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.312 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:58.312 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:58.312 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:58.312 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:58.312 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:05:58.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.312 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:05:58.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:05:58.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:05:58.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:05:58.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:05:58.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:05:58.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:05:58.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:05:58.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:05:58.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:05:58.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:05:58.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:05:58.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:05:58.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:05:58.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:05:58.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:05:58.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:05:58.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:05:58.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:05:58.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:05:58.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:05:58.316 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.316 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.316 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.317 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:05:58.317 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:05:58.317 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.215428076999999, 2024-09-14T09:05:58.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:05:58.317 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.317 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.317 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:58.317 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:58.317 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:58.317 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:58.318 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:05:58.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.318 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:05:58.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:05:58.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:05:58.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:05:58.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:05:58.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:05:58.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:05:58.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:05:58.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:05:58.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:05:58.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:05:58.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:05:58.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:05:58.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:05:58.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:05:58.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:05:58.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:05:58.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:05:58.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:05:58.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:05:58.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:05:58.322 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.322 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.322 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.322 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:05:58.322 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:05:58.322 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.215343339, 2024-09-14T09:05:58.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:05:58.323 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.323 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.323 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:58.323 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:58.323 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:58.323 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:58.323 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:05:58.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.323 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:05:58.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:05:58.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:05:58.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:05:58.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:05:58.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:05:58.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:05:58.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:05:58.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:05:58.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:05:58.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:05:58.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:05:58.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:05:58.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:05:58.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:05:58.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:05:58.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:05:58.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:05:58.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:05:58.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:05:58.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:05:58.327 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.327 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.327 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.328 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:05:58.328 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:05:58.328 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.215255653, 2024-09-14T09:05:58.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:05:58.328 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.328 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.328 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:58.328 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:58.328 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:58.328 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:58.328 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:05:58.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.328 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:05:58.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:05:58.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:05:58.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:05:58.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:05:58.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:05:58.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:05:58.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:05:58.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:05:58.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:05:58.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:05:58.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:05:58.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:05:58.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:05:58.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:05:58.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:05:58.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:05:58.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:05:58.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:05:58.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:05:58.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:05:58.332 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.332 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.332 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.333 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18388 3.18 3:19e2be55:::rbd_data.115e3abd3b2c.0000000000000021:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:05:58.333 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:05:58.333 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.079342342, 2024-09-14T09:05:58.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042293648000000003, 2024-09-14T09:05:58.333 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.333 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.333 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:58.333 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:58.333 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:58.333 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:58.333 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18388 2024-09-14T09:05:58.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.334 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:05:58.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:05:58.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220236+0000", 2024-09-14T09:05:58.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1339999999999998e-06 2024-09-14T09:05:58.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220244+0000", 2024-09-14T09:05:58.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1950000000000003e-06 2024-09-14T09:05:58.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220262+0000", 2024-09-14T09:05:58.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8017e-05 2024-09-14T09:05:58.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220270+0000", 2024-09-14T09:05:58.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9480000000000008e-06 2024-09-14T09:05:58.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220357+0000", 2024-09-14T09:05:58.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7185e-05 2024-09-14T09:05:58.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220576+0000", 2024-09-14T09:05:58.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021912200000000001 2024-09-14T09:05:58.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:05:58.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.222435+0000", 2024-09-14T09:05:58.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018592350000000001 2024-09-14T09:05:58.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:58.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262439+0000", 2024-09-14T09:05:58.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040003368999999997 2024-09-14T09:05:58.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:58.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262452+0000", 2024-09-14T09:05:58.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2941e-05 2024-09-14T09:05:58.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:58.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262490+0000", 2024-09-14T09:05:58.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8478e-05 2024-09-14T09:05:58.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262527+0000", 2024-09-14T09:05:58.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7024000000000003e-05 2024-09-14T09:05:58.338 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.338 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.338 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.338 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18388 3.18 e38/28 ondisk, result = 0)", 2024-09-14T09:05:58.339 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.227795+0000", 2024-09-14T09:05:58.339 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.071780699, 2024-09-14T09:05:58.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034648832999999997, 2024-09-14T09:05:58.339 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.339 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.339 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:58.339 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227795+0000", 2024-09-14T09:05:58.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227795+0000", 2024-09-14T09:05:58.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227796+0000", 2024-09-14T09:05:58.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.879999999999999e-07 2024-09-14T09:05:58.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227801+0000", 2024-09-14T09:05:58.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4909999999999999e-06 2024-09-14T09:05:58.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227815+0000", 2024-09-14T09:05:58.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4905e-05 2024-09-14T09:05:58.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227824+0000", 2024-09-14T09:05:58.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7770000000000008e-06 2024-09-14T09:05:58.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262425+0000", 2024-09-14T09:05:58.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034601109999999997 2024-09-14T09:05:58.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262434+0000", 2024-09-14T09:05:58.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5250000000000005e-06 2024-09-14T09:05:58.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262444+0000", 2024-09-14T09:05:58.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0037000000000001e-05 2024-09-14T09:05:58.346 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.346 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.346 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.347 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:19543 3.5 e39/23 3:a536ff9b:::rbd_data.115e4c71d6e8.0000000000000002:head v 39'8, mlcod=39'6)", 2024-09-14T09:05:58.347 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:05:58.347 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 13.776487681999999, 2024-09-14T09:05:58.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.039280019999999999, 2024-09-14T09:05:58.347 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.348 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.348 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:58.348 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:05:58.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:05:58.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523090+0000", 2024-09-14T09:05:58.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.773e-06 2024-09-14T09:05:58.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537290+0000", 2024-09-14T09:05:58.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014200007000000001 2024-09-14T09:05:58.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537317+0000", 2024-09-14T09:05:58.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6704999999999999e-05 2024-09-14T09:05:58.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537328+0000", 2024-09-14T09:05:58.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1299e-05 2024-09-14T09:05:58.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537417+0000", 2024-09-14T09:05:58.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8869999999999994e-05 2024-09-14T09:05:58.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537445+0000", 2024-09-14T09:05:58.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8377999999999998e-05 2024-09-14T09:05:58.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:58.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562324+0000", 2024-09-14T09:05:58.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024879307999999999 2024-09-14T09:05:58.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562368+0000", 2024-09-14T09:05:58.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3680000000000002e-05 2024-09-14T09:05:58.355 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.356 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.356 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.356 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20736 3.2 3:45030d14:::rbd_data.115e889d4809.000000000000027a:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:05:58.356 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.099577+0000", 2024-09-14T09:05:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 11.199998639, 2024-09-14T09:05:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026646762000000001, 2024-09-14T09:05:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:58.358 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:58.358 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20736 2024-09-14T09:05:58.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.358 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099577+0000", 2024-09-14T09:05:58.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099577+0000", 2024-09-14T09:05:58.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099579+0000", 2024-09-14T09:05:58.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5009999999999999e-06 2024-09-14T09:05:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099585+0000", 2024-09-14T09:05:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2870000000000001e-06 2024-09-14T09:05:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099602+0000", 2024-09-14T09:05:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7176e-05 2024-09-14T09:05:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099611+0000", 2024-09-14T09:05:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7010000000000002e-06 2024-09-14T09:05:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099664+0000", 2024-09-14T09:05:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3183000000000002e-05 2024-09-14T09:05:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099793+0000", 2024-09-14T09:05:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012945099999999999 2024-09-14T09:05:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:05:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.100819+0000", 2024-09-14T09:05:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010252149999999999 2024-09-14T09:05:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.125929+0000", 2024-09-14T09:05:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025110072000000001 2024-09-14T09:05:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.126034+0000", 2024-09-14T09:05:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105655 2024-09-14T09:05:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.126141+0000", 2024-09-14T09:05:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107069 2024-09-14T09:05:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.126224+0000", 2024-09-14T09:05:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2452000000000003e-05 2024-09-14T09:05:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.369 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31015 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:58.369 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.062742+0000", 2024-09-14T09:05:58.369 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.2368333109999998, 2024-09-14T09:05:58.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027690059999999999, 2024-09-14T09:05:58.369 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.370 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.370 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:58.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:58.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:58.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:58.370 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31015 2024-09-14T09:05:58.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.371 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062742+0000", 2024-09-14T09:05:58.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062742+0000", 2024-09-14T09:05:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062743+0000", 2024-09-14T09:05:58.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.51e-07 2024-09-14T09:05:58.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062745+0000", 2024-09-14T09:05:58.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0480000000000001e-06 2024-09-14T09:05:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062751+0000", 2024-09-14T09:05:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3929999999999996e-06 2024-09-14T09:05:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062757+0000", 2024-09-14T09:05:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5130000000000004e-06 2024-09-14T09:05:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.088137+0000", 2024-09-14T09:05:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025380494999999999 2024-09-14T09:05:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.088178+0000", 2024-09-14T09:05:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0963000000000001e-05 2024-09-14T09:05:58.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.090433+0000", 2024-09-14T09:05:58.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0022541969999999999 2024-09-14T09:05:58.377 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31022 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.062919+0000", 2024-09-14T09:05:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.2366565930000002, 2024-09-14T09:05:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026754692, 2024-09-14T09:05:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:58.380 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31022 2024-09-14T09:05:58.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.380 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062919+0000", 2024-09-14T09:05:58.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062919+0000", 2024-09-14T09:05:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062920+0000", 2024-09-14T09:05:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2099999999999997e-07 2024-09-14T09:05:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062922+0000", 2024-09-14T09:05:58.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.452e-06 2024-09-14T09:05:58.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062929+0000", 2024-09-14T09:05:58.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4439999999999996e-06 2024-09-14T09:05:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062958+0000", 2024-09-14T09:05:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8943999999999999e-05 2024-09-14T09:05:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089526+0000", 2024-09-14T09:05:58.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026568556 2024-09-14T09:05:58.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089600+0000", 2024-09-14T09:05:58.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3506999999999999e-05 2024-09-14T09:05:58.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089674+0000", 2024-09-14T09:05:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4268000000000001e-05 2024-09-14T09:05:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31023 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 264192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.062978+0000", 2024-09-14T09:05:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.2365973800000001, 2024-09-14T09:05:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026871328999999999, 2024-09-14T09:05:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31023 2024-09-14T09:05:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062978+0000", 2024-09-14T09:05:58.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062978+0000", 2024-09-14T09:05:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062979+0000", 2024-09-14T09:05:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6499999999999999e-07 2024-09-14T09:05:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062982+0000", 2024-09-14T09:05:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8650000000000001e-06 2024-09-14T09:05:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062990+0000", 2024-09-14T09:05:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9889999999999992e-06 2024-09-14T09:05:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063028+0000", 2024-09-14T09:05:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8154000000000001e-05 2024-09-14T09:05:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089700+0000", 2024-09-14T09:05:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026672548000000001 2024-09-14T09:05:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089772+0000", 2024-09-14T09:05:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1829999999999995e-05 2024-09-14T09:05:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089850+0000", 2024-09-14T09:05:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7378000000000004e-05 2024-09-14T09:05:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31024 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 282624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.063039+0000", 2024-09-14T09:05:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.2365366849999999, 2024-09-14T09:05:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027079282, 2024-09-14T09:05:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31024 2024-09-14T09:05:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063039+0000", 2024-09-14T09:05:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063039+0000", 2024-09-14T09:05:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063040+0000", 2024-09-14T09:05:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8299999999999997e-07 2024-09-14T09:05:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063043+0000", 2024-09-14T09:05:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9610000000000001e-06 2024-09-14T09:05:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063058+0000", 2024-09-14T09:05:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5529999999999999e-05 2024-09-14T09:05:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063065+0000", 2024-09-14T09:05:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.596e-06 2024-09-14T09:05:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089877+0000", 2024-09-14T09:05:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026812373 2024-09-14T09:05:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089949+0000", 2024-09-14T09:05:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2014000000000005e-05 2024-09-14T09:05:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.090118+0000", 2024-09-14T09:05:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016922500000000001 2024-09-14T09:05:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31025 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 292864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.063074+0000", 2024-09-14T09:05:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.2365020439999999, 2024-09-14T09:05:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027206443, 2024-09-14T09:05:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31025 2024-09-14T09:05:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063074+0000", 2024-09-14T09:05:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063074+0000", 2024-09-14T09:05:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063074+0000", 2024-09-14T09:05:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.68e-07 2024-09-14T09:05:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063077+0000", 2024-09-14T09:05:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.853e-06 2024-09-14T09:05:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063084+0000", 2024-09-14T09:05:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0419999999999997e-06 2024-09-14T09:05:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063089+0000", 2024-09-14T09:05:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1150000000000002e-06 2024-09-14T09:05:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.090144+0000", 2024-09-14T09:05:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027054974999999998 2024-09-14T09:05:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.090211+0000", 2024-09-14T09:05:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6832e-05 2024-09-14T09:05:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.090280+0000", 2024-09-14T09:05:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9058000000000002e-05 2024-09-14T09:05:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31279 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 274432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:05:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.072774+0000", 2024-09-14T09:05:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.2268017809999998, 2024-09-14T09:05:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026662447999999998, 2024-09-14T09:05:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31279 2024-09-14T09:05:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072774+0000", 2024-09-14T09:05:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072774+0000", 2024-09-14T09:05:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072775+0000", 2024-09-14T09:05:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0900000000000001e-07 2024-09-14T09:05:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072778+0000", 2024-09-14T09:05:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7340000000000002e-06 2024-09-14T09:05:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072789+0000", 2024-09-14T09:05:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0287e-05 2024-09-14T09:05:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072795+0000", 2024-09-14T09:05:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7390000000000002e-06 2024-09-14T09:05:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.099278+0000", 2024-09-14T09:05:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026482887 2024-09-14T09:05:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.099353+0000", 2024-09-14T09:05:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5013999999999997e-05 2024-09-14T09:05:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.099436+0000", 2024-09-14T09:05:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3078000000000007e-05 2024-09-14T09:05:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38002 3.1e 3:7df715aa:::rbd_data.115eb07a8abd.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:05:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:05:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.90982443599999996, 2024-09-14T09:05:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.065148304000000004, 2024-09-14T09:05:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38002 2024-09-14T09:05:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:05:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:05:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389752+0000", 2024-09-14T09:05:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1829999999999999e-06 2024-09-14T09:05:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389758+0000", 2024-09-14T09:05:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0919999999999997e-06 2024-09-14T09:05:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389769+0000", 2024-09-14T09:05:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759e-05 2024-09-14T09:05:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389775+0000", 2024-09-14T09:05:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8819999999999998e-06 2024-09-14T09:05:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389824+0000", 2024-09-14T09:05:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8573999999999999e-05 2024-09-14T09:05:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389876+0000", 2024-09-14T09:05:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2643999999999999e-05 2024-09-14T09:05:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:05:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.392640+0000", 2024-09-14T09:05:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0027636409999999998 2024-09-14T09:05:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454672+0000", 2024-09-14T09:05:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062031703000000001 2024-09-14T09:05:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454765+0000", 2024-09-14T09:05:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2862000000000006e-05 2024-09-14T09:05:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454841+0000", 2024-09-14T09:05:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5857000000000002e-05 2024-09-14T09:05:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454900+0000", 2024-09-14T09:05:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9107000000000002e-05 2024-09-14T09:05:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:38002 3.1e e48/23 ondisk, result = 0)", 2024-09-14T09:05:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:05:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.89950784699999997, 2024-09-14T09:05:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054841741999999999, 2024-09-14T09:05:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:05:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:05:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400069+0000", 2024-09-14T09:05:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.339e-06 2024-09-14T09:05:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400076+0000", 2024-09-14T09:05:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7800000000000003e-06 2024-09-14T09:05:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400093+0000", 2024-09-14T09:05:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6915999999999999e-05 2024-09-14T09:05:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400106+0000", 2024-09-14T09:05:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2785e-05 2024-09-14T09:05:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454713+0000", 2024-09-14T09:05:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054607041000000002 2024-09-14T09:05:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454724+0000", 2024-09-14T09:05:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1202e-05 2024-09-14T09:05:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454910+0000", 2024-09-14T09:05:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018567900000000001 2024-09-14T09:05:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38100 3.b 3:d0f67c24:::rbd_data.115eb07a8abd.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:05:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:05:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.653654187, 2024-09-14T09:05:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053701964999999997, 2024-09-14T09:05:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:05:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:05:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:05:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:05:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38100 2024-09-14T09:05:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:05:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:05:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645923+0000", 2024-09-14T09:05:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:05:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645928+0000", 2024-09-14T09:05:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0699999999999997e-06 2024-09-14T09:05:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645942+0000", 2024-09-14T09:05:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4069e-05 2024-09-14T09:05:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645951+0000", 2024-09-14T09:05:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3230000000000004e-06 2024-09-14T09:05:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646008+0000", 2024-09-14T09:05:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6583999999999999e-05 2024-09-14T09:05:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646089+0000", 2024-09-14T09:05:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1557999999999994e-05 2024-09-14T09:05:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:05:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.650355+0000", 2024-09-14T09:05:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004265525 2024-09-14T09:05:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:05:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699439+0000", 2024-09-14T09:05:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049083904999999997 2024-09-14T09:05:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:05:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699467+0000", 2024-09-14T09:05:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8198999999999999e-05 2024-09-14T09:05:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:05:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699552+0000", 2024-09-14T09:05:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T09:05:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699624+0000", 2024-09-14T09:05:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1725999999999997e-05 2024-09-14T09:05:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:38100 3.b e48/28 ondisk, result = 0)", 2024-09-14T09:05:58.425 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.669267+0000", 2024-09-14T09:05:58.425 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.63030855600000002, 2024-09-14T09:05:58.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030186418999999999, 2024-09-14T09:05:58.425 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:05:58.425 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:05:58.425 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:05:58.425 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:05:58.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:05:58.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.669267+0000", 2024-09-14T09:05:58.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:05:58.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.669267+0000", 2024-09-14T09:05:58.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:05:58.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:05:58.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.669270+0000", 2024-09-14T09:05:58.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3269999999999999e-06 2024-09-14T09:05:58.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:05:58.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.669284+0000", 2024-09-14T09:05:58.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4662e-05 2024-09-14T09:05:58.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:05:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.669317+0000", 2024-09-14T09:05:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3080000000000002e-05 2024-09-14T09:05:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:05:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.669334+0000", 2024-09-14T09:05:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6376999999999999e-05 2024-09-14T09:05:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:05:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699411+0000", 2024-09-14T09:05:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030077653999999999 2024-09-14T09:05:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:05:58.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699429+0000", 2024-09-14T09:05:58.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7723e-05 2024-09-14T09:05:58.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:05:58.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:05:58.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:05:58.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699454+0000", 2024-09-14T09:05:58.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4596e-05 2024-09-14T09:05:58.428 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.428 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.428 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.428 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:05:58.428 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:05:58.428 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:05:58.429 INFO:teuthology.orchestra.run.smithi176.stdout:1460 write 0xdc74b27 thru 0xdc7e328 (0x9802 bytes) 2024-09-14T09:05:58.429 INFO:teuthology.orchestra.run.smithi176.stdout:1461 read 0x97aaa3f thru 0x97aff6f (0x5531 bytes) 2024-09-14T09:05:58.429 INFO:teuthology.orchestra.run.smithi176.stdout:1464 write 0xc2ee483 thru 0xc2fcc69 (0xe7e7 bytes) 2024-09-14T09:05:58.429 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:58.429 INFO:teuthology.orchestra.run.smithi176.stdout:1467 write 0x296fad1 thru 0x2970cf6 (0x1226 bytes) 2024-09-14T09:05:58.430 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:05:58.431 INFO:teuthology.orchestra.run.smithi176.stdout:1468 read 0x6db12f0 thru 0x6dbee55 (0xdb66 bytes) 2024-09-14T09:05:58.431 INFO:teuthology.orchestra.run.smithi176.stdout:1469 read 0x81f44fa thru 0x82036f2 (0xf1f9 bytes) 2024-09-14T09:05:58.432 INFO:teuthology.orchestra.run.smithi176.stdout:1471 read 0x334dc3a thru 0x335373c (0x5b03 bytes) 2024-09-14T09:05:58.432 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:58.432 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:58.432 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:58.432 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:58.432 INFO:teuthology.orchestra.run.smithi176.stdout:1481 trunc from 0xdc7e329 to 0x3dd7a66 2024-09-14T09:05:58.432 INFO:teuthology.orchestra.run.smithi176.stdout:1482 punch from 0x2587687 to 0x2592be5, (0xb55e bytes) 2024-09-14T09:05:58.432 INFO:teuthology.orchestra.run.smithi176.stdout:1483 write 0x5130009 thru 0x513139a (0x1392 bytes) 2024-09-14T09:05:58.432 INFO:teuthology.orchestra.run.smithi176.stdout:1484 write 0x9376f87 thru 0x937d82e (0x68a8 bytes) 2024-09-14T09:05:58.432 INFO:teuthology.orchestra.run.smithi176.stdout:1485 writesame 0xa14f47f thru 0xa158699 data_size 0x1(0x921b bytes) 2024-09-14T09:05:58.433 INFO:teuthology.orchestra.run.smithi176.stdout:1486 write 0xe2a7ba0 thru 0xe2a97dc (0x1c3d bytes) 2024-09-14T09:05:58.433 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:58.435+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:58.437 INFO:teuthology.orchestra.run.smithi176.stdout:1487 writesame 0x2f3cae0 thru 0x2f3d0ab data_size 0xe(0x5cc bytes) 2024-09-14T09:05:58.440 INFO:teuthology.orchestra.run.smithi176.stdout:1488 write 0x7fb1fe thru 0x806f52 (0xbd55 bytes) 2024-09-14T09:05:58.457 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:05:58.488 INFO:teuthology.orchestra.run.smithi176.stdout:1489 read 0xb66bf7b thru 0xb679edb (0xdf61 bytes) 2024-09-14T09:05:58.488 INFO:teuthology.orchestra.run.smithi176.stdout:1490 punch from 0x42fe2cc to 0x4306480, (0x81b4 bytes) 2024-09-14T09:05:58.488 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:05:58.488 INFO:teuthology.orchestra.run.smithi176.stdout:1492 writesame 0x11f463 thru 0x12175a data_size 0x8(0x22f8 bytes) 2024-09-14T09:05:58.490 INFO:teuthology.orchestra.run.smithi176.stdout:1495 punch from 0xba57dd9 to 0xba5b43e, (0x3665 bytes) 2024-09-14T09:05:58.490 INFO:teuthology.orchestra.run.smithi176.stdout:1496 write 0x7635237 thru 0x763cfd3 (0x7d9d bytes) 2024-09-14T09:05:58.494 INFO:teuthology.orchestra.run.smithi176.stdout:1499 write 0x864a7ed thru 0x86563b7 (0xbbcb bytes) 2024-09-14T09:05:58.499 INFO:teuthology.orchestra.run.smithi176.stdout:1500 write 0x655b066 thru 0x655c9b9 (0x1954 bytes) 2024-09-14T09:05:58.503 INFO:teuthology.orchestra.run.smithi176.stdout:1501 writesame 0xab1fb1a thru 0xab23c75 data_size 0x59(0x415c bytes) 2024-09-14T09:05:58.507 INFO:teuthology.orchestra.run.smithi176.stdout:1503 clone 13 order 24 su 2097152 sc 4 2024-09-14T09:05:58.534 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:58.535+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:58.570 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:05:58.570 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:05:58.570 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:05:58.570 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:05:58.583 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:05:58.602 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:05:58.602 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:05:58.618 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:05:58.618 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:05:58.636 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:05:58.636 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:05:58.651 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:05:58.652 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:05:58.672 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:05:58.672 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:05:58.676 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:58.679+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:58.690 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:05:58.690 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:05:58.710 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:05:58.711 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:05:58.731 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:05:58.731 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:05:58.735 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:58.732+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:58.747 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:58.747 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:58.763 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:05:58.856 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:58.859+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:58.932 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:58.932 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:58.938 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:58.939+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:58.947 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:05:59.039 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:05:59.039+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:05:59.043 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is unset 2024-09-14T09:05:59.046 INFO:teuthology.orchestra.run.smithi176.stdout:moved 3 / 82 (3.65854%) 2024-09-14T09:05:59.046 INFO:teuthology.orchestra.run.smithi176.stdout:avg 10.25 2024-09-14T09:05:59.047 INFO:teuthology.orchestra.run.smithi176.stdout:stddev 4.4088 -> 4.4088 (expected baseline 2.99479) 2024-09-14T09:05:59.047 INFO:teuthology.orchestra.run.smithi176.stdout:min osd.4 with 2 -> 1 pgs (0.195122 -> 0.097561 * mean) 2024-09-14T09:05:59.047 INFO:teuthology.orchestra.run.smithi176.stdout:max osd.1 with 16 -> 15 pgs (1.56098 -> 1.46341 * mean) 2024-09-14T09:05:59.047 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:05:59.047 INFO:teuthology.orchestra.run.smithi176.stdout:oload 110 2024-09-14T09:05:59.047 INFO:teuthology.orchestra.run.smithi176.stdout:max_change 0.05 2024-09-14T09:05:59.047 INFO:teuthology.orchestra.run.smithi176.stdout:max_change_osds 4 2024-09-14T09:05:59.047 INFO:teuthology.orchestra.run.smithi176.stdout:average_utilization 0.0004 2024-09-14T09:05:59.047 INFO:teuthology.orchestra.run.smithi176.stdout:overload_utilization 0.0004 2024-09-14T09:05:59.047 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 weight 0.7785 -> 0.8247 2024-09-14T09:05:59.047 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 weight 0.1635 -> 0.1724 2024-09-14T09:05:59.078 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-14T09:05:59.079 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-14T09:05:59.117 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:59.117 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:59.131 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:05:59.172 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:05:59.175+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:05:59.240 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:05:59.243+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:05:59.297 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:59.297 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:59.311 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:05:59.352 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:05:59.355+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:05:59.441 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:59.436+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:59.476 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:59.476 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:59.491 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:05:59.543 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:59.540+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:05:59.612 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:59.612 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:59.626 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:05:59.643 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:59.640+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:59.743 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:59.743 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:59.744 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:05:59.744+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:05:59.758 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:05:59.845 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:05:59.840+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:05:59.877 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:05:59.877 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:05:59.893 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:05:59.946 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:05:59.944+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:00.018 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:00.019 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:00.033 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:06:00.047 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:00.044+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:00.051 INFO:teuthology.orchestra.run.smithi176.stdout:moved 1 / 82 (1.21951%) 2024-09-14T09:06:00.052 INFO:teuthology.orchestra.run.smithi176.stdout:avg 10.25 2024-09-14T09:06:00.052 INFO:teuthology.orchestra.run.smithi176.stdout:stddev 4.4088 -> 4.02337 (expected baseline 2.99479) 2024-09-14T09:06:00.052 INFO:teuthology.orchestra.run.smithi176.stdout:min osd.4 with 1 -> 2 pgs (0.097561 -> 0.195122 * mean) 2024-09-14T09:06:00.052 INFO:teuthology.orchestra.run.smithi176.stdout:max osd.7 with 16 -> 16 pgs (1.56098 -> 1.56098 * mean) 2024-09-14T09:06:00.052 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:06:00.052 INFO:teuthology.orchestra.run.smithi176.stdout:oload 110 2024-09-14T09:06:00.052 INFO:teuthology.orchestra.run.smithi176.stdout:max_change 3 2024-09-14T09:06:00.052 INFO:teuthology.orchestra.run.smithi176.stdout:max_change_osds 4 2024-09-14T09:06:00.052 INFO:teuthology.orchestra.run.smithi176.stdout:average_utilization 0.0004 2024-09-14T09:06:00.052 INFO:teuthology.orchestra.run.smithi176.stdout:overload_utilization 0.0004 2024-09-14T09:06:00.052 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 weight 0.8247 -> 0.8737 2024-09-14T09:06:00.052 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 weight 0.1724 -> 0.1817 2024-09-14T09:06:00.078 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-14T09:06:00.079 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-14T09:06:00.160 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:00.160 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:00.170 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:00.168+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:00.175 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:06:00.249 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:00.244+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:00.291 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:00.291 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:00.305 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:06:00.350 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:00.348+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:00.427 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:00.427 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:00.439 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-09-14T09:06:00.439+0000 7f321da70640 -1 mgr.server reply reply (2) No such file or directory no change 2024-09-14T09:06:00.439 INFO:teuthology.orchestra.run.smithi176.stdout:moved 2 / 82 (2.43902%) 2024-09-14T09:06:00.440 INFO:teuthology.orchestra.run.smithi176.stdout:avg 10.25 2024-09-14T09:06:00.440 INFO:teuthology.orchestra.run.smithi176.stdout:stddev 4.02337 -> 4.11552 (expected baseline 2.99479) 2024-09-14T09:06:00.440 INFO:teuthology.orchestra.run.smithi176.stdout:min osd.4 with 2 -> 2 pgs (0.195122 -> 0.195122 * mean) 2024-09-14T09:06:00.440 INFO:teuthology.orchestra.run.smithi176.stdout:max osd.7 with 16 -> 16 pgs (1.56098 -> 1.56098 * mean) 2024-09-14T09:06:00.440 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:06:00.440 INFO:teuthology.orchestra.run.smithi176.stdout:oload 110 2024-09-14T09:06:00.440 INFO:teuthology.orchestra.run.smithi176.stdout:max_change 1 2024-09-14T09:06:00.440 INFO:teuthology.orchestra.run.smithi176.stdout:max_change_osds 4 2024-09-14T09:06:00.440 INFO:teuthology.orchestra.run.smithi176.stdout:average_utilization 0.0004 2024-09-14T09:06:00.440 INFO:teuthology.orchestra.run.smithi176.stdout:overload_utilization 0.0004 2024-09-14T09:06:00.440 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 weight 0.8737 -> 0.9255 2024-09-14T09:06:00.440 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 weight 0.1817 -> 0.1915 2024-09-14T09:06:00.441 INFO:teuthology.orchestra.run.smithi176.stderr:no change 2024-09-14T09:06:00.441 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:06:00.451 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:00.451+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:00.552 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:00.548+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:00.618 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:00.619 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:00.632 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:06:00.653 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:00.655+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:00.749 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:06:00.798 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:00.796+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:00.799 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:00.800 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:00.815 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:06:00.854 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:00.855+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:00.892 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:00.892 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:00.908 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:00.909 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:00.925 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:00.925 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:00.943 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:00.943 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:00.955 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:00.952+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:00.963 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:00.963 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:00.981 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:00.981 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:00.981 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:00.981 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:00.984 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:00.984 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:00.995 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:06:01.005 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:01.005 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:01.026 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:01.026 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:01.076 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T09:06:01.086 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:01.084+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:01.125 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:01.125 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:01.125 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:01.125 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:01.125 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:01.140 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:06:01.156 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:01.159+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:01.257 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:01.252+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:01.304 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:10032 3.19 3:9ed4ec6a:::rbd_data.115eadec874.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:06:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.775706+0000", 2024-09-14T09:06:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.527433590000001, 2024-09-14T09:06:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.126247939, 2024-09-14T09:06:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.305 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 10032 2024-09-14T09:06:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775706+0000", 2024-09-14T09:06:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775706+0000", 2024-09-14T09:06:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775709+0000", 2024-09-14T09:06:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0400000000000001e-06 2024-09-14T09:06:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775721+0000", 2024-09-14T09:06:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2119999999999999e-05 2024-09-14T09:06:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775752+0000", 2024-09-14T09:06:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0139999999999999e-05 2024-09-14T09:06:01.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775766+0000", 2024-09-14T09:06:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4497e-05 2024-09-14T09:06:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.775937+0000", 2024-09-14T09:06:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000171022 2024-09-14T09:06:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.776205+0000", 2024-09-14T09:06:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026749100000000002 2024-09-14T09:06:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:06:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.784970+0000", 2024-09-14T09:06:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0087648789999999997 2024-09-14T09:06:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901870+0000", 2024-09-14T09:06:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11690028 2024-09-14T09:06:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901878+0000", 2024-09-14T09:06:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7020000000000007e-06 2024-09-14T09:06:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901913+0000", 2024-09-14T09:06:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4706000000000003e-05 2024-09-14T09:06:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901954+0000", 2024-09-14T09:06:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1062000000000001e-05 2024-09-14T09:06:01.310 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:10032 3.19 e32/23 ondisk, result = 0)", 2024-09-14T09:06:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:37.809095+0000", 2024-09-14T09:06:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.494045311000001, 2024-09-14T09:06:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092866664000000002, 2024-09-14T09:06:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809095+0000", 2024-09-14T09:06:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809095+0000", 2024-09-14T09:06:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809096+0000", 2024-09-14T09:06:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.64e-06 2024-09-14T09:06:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809104+0000", 2024-09-14T09:06:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7330000000000003e-06 2024-09-14T09:06:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809132+0000", 2024-09-14T09:06:01.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7880000000000001e-05 2024-09-14T09:06:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.809146+0000", 2024-09-14T09:06:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4514e-05 2024-09-14T09:06:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901858+0000", 2024-09-14T09:06:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092711713000000001 2024-09-14T09:06:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901866+0000", 2024-09-14T09:06:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4760000000000001e-06 2024-09-14T09:06:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:37.901961+0000", 2024-09-14T09:06:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5707999999999999e-05 2024-09-14T09:06:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:19545 3.f e39/34 3:f12afce6:::rbd_data.115e4c71d6e8.0000000000000003:head v 39'24, mlcod=39'22)", 2024-09-14T09:06:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.529754+0000", 2024-09-14T09:06:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.773385774000001, 2024-09-14T09:06:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037365357000000002, 2024-09-14T09:06:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:01.316 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.529754+0000", 2024-09-14T09:06:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.529754+0000", 2024-09-14T09:06:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.529756+0000", 2024-09-14T09:06:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.804e-06 2024-09-14T09:06:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537071+0000", 2024-09-14T09:06:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0073151120000000004 2024-09-14T09:06:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537111+0000", 2024-09-14T09:06:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0315999999999998e-05 2024-09-14T09:06:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537133+0000", 2024-09-14T09:06:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2098000000000001e-05 2024-09-14T09:06:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537274+0000", 2024-09-14T09:06:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014045400000000001 2024-09-14T09:06:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537312+0000", 2024-09-14T09:06:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7761e-05 2024-09-14T09:06:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:01.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.567050+0000", 2024-09-14T09:06:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029738547000000001 2024-09-14T09:06:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.567119+0000", 2024-09-14T09:06:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9264999999999995e-05 2024-09-14T09:06:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20862 3.1a 3:580a70cb:::rbd_data.115e889d4809.0000000000000177:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.497482+0000", 2024-09-14T09:06:01.320 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 13.805657542000001, 2024-09-14T09:06:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.032797795999999997, 2024-09-14T09:06:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20862 2024-09-14T09:06:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497482+0000", 2024-09-14T09:06:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497482+0000", 2024-09-14T09:06:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497485+0000", 2024-09-14T09:06:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8569999999999999e-06 2024-09-14T09:06:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497496+0000", 2024-09-14T09:06:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0277e-05 2024-09-14T09:06:01.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497519+0000", 2024-09-14T09:06:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3513e-05 2024-09-14T09:06:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497533+0000", 2024-09-14T09:06:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4343e-05 2024-09-14T09:06:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497642+0000", 2024-09-14T09:06:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010840099999999999 2024-09-14T09:06:01.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.497840+0000", 2024-09-14T09:06:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000198192 2024-09-14T09:06:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 2", 2024-09-14T09:06:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.499243+0000", 2024-09-14T09:06:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0014027550000000001 2024-09-14T09:06:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530087+0000", 2024-09-14T09:06:01.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030844086999999999 2024-09-14T09:06:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530120+0000", 2024-09-14T09:06:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3309000000000002e-05 2024-09-14T09:06:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530207+0000", 2024-09-14T09:06:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6536e-05 2024-09-14T09:06:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530280+0000", 2024-09-14T09:06:01.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3526e-05 2024-09-14T09:06:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:20862 3.1a e41/34 ondisk, result = 0)", 2024-09-14T09:06:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.503269+0000", 2024-09-14T09:06:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 13.799870537, 2024-09-14T09:06:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026831292999999999, 2024-09-14T09:06:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503269+0000", 2024-09-14T09:06:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503269+0000", 2024-09-14T09:06:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503271+0000", 2024-09-14T09:06:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3820000000000001e-06 2024-09-14T09:06:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503276+0000", 2024-09-14T09:06:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6400000000000002e-06 2024-09-14T09:06:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503293+0000", 2024-09-14T09:06:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6404000000000001e-05 2024-09-14T09:06:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.503304+0000", 2024-09-14T09:06:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0905e-05 2024-09-14T09:06:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530061+0000", 2024-09-14T09:06:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026757019 2024-09-14T09:06:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530076+0000", 2024-09-14T09:06:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5526999999999999e-05 2024-09-14T09:06:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.530101+0000", 2024-09-14T09:06:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4416000000000001e-05 2024-09-14T09:06:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.329 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:30960 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 272384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.061773+0000", 2024-09-14T09:06:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2413667690000008, 2024-09-14T09:06:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023509044999999999, 2024-09-14T09:06:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30960 2024-09-14T09:06:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.330 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061773+0000", 2024-09-14T09:06:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061773+0000", 2024-09-14T09:06:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061774+0000", 2024-09-14T09:06:01.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3700000000000001e-07 2024-09-14T09:06:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061776+0000", 2024-09-14T09:06:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0990000000000001e-06 2024-09-14T09:06:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061782+0000", 2024-09-14T09:06:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2829999999999998e-06 2024-09-14T09:06:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061786+0000", 2024-09-14T09:06:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4970000000000003e-06 2024-09-14T09:06:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.085106+0000", 2024-09-14T09:06:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023319617000000001 2024-09-14T09:06:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.085182+0000", 2024-09-14T09:06:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5512000000000005e-05 2024-09-14T09:06:01.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.085282+0000", 2024-09-14T09:06:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010060000000000001 2024-09-14T09:06:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:30961 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 290816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.061793+0000", 2024-09-14T09:06:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2413466450000001, 2024-09-14T09:06:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023756772999999998, 2024-09-14T09:06:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.334 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30961 2024-09-14T09:06:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061793+0000", 2024-09-14T09:06:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061793+0000", 2024-09-14T09:06:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061794+0000", 2024-09-14T09:06:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1800000000000001e-07 2024-09-14T09:06:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061796+0000", 2024-09-14T09:06:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0480000000000001e-06 2024-09-14T09:06:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061802+0000", 2024-09-14T09:06:01.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2129999999999996e-06 2024-09-14T09:06:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061807+0000", 2024-09-14T09:06:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.617e-06 2024-09-14T09:06:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.085339+0000", 2024-09-14T09:06:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023532761999999999 2024-09-14T09:06:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.085416+0000", 2024-09-14T09:06:01.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6951000000000002e-05 2024-09-14T09:06:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.085550+0000", 2024-09-14T09:06:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000133764 2024-09-14T09:06:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:30962 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 301056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.061813+0000", 2024-09-14T09:06:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2413266230000009, 2024-09-14T09:06:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023993381000000001, 2024-09-14T09:06:01.338 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 30962 2024-09-14T09:06:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061813+0000", 2024-09-14T09:06:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061813+0000", 2024-09-14T09:06:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061814+0000", 2024-09-14T09:06:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4799999999999999e-07 2024-09-14T09:06:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061816+0000", 2024-09-14T09:06:01.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9400000000000001e-06 2024-09-14T09:06:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061822+0000", 2024-09-14T09:06:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7050000000000002e-06 2024-09-14T09:06:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.061827+0000", 2024-09-14T09:06:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5909999999999998e-06 2024-09-14T09:06:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.085612+0000", 2024-09-14T09:06:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023785404999999999 2024-09-14T09:06:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.085695+0000", 2024-09-14T09:06:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2661e-05 2024-09-14T09:06:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.085807+0000", 2024-09-14T09:06:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000111631 2024-09-14T09:06:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31209 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.068892+0000", 2024-09-14T09:06:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2342479540000006, 2024-09-14T09:06:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023518807999999999, 2024-09-14T09:06:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31209 2024-09-14T09:06:01.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068892+0000", 2024-09-14T09:06:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068892+0000", 2024-09-14T09:06:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068893+0000", 2024-09-14T09:06:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0100000000000005e-07 2024-09-14T09:06:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068895+0000", 2024-09-14T09:06:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0389999999999999e-06 2024-09-14T09:06:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068901+0000", 2024-09-14T09:06:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4069999999999998e-06 2024-09-14T09:06:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068913+0000", 2024-09-14T09:06:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1753000000000001e-05 2024-09-14T09:06:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092223+0000", 2024-09-14T09:06:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023310581 2024-09-14T09:06:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092313+0000", 2024-09-14T09:06:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9399000000000002e-05 2024-09-14T09:06:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092411+0000", 2024-09-14T09:06:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8028000000000004e-05 2024-09-14T09:06:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31210 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.068920+0000", 2024-09-14T09:06:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2342195359999995, 2024-09-14T09:06:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023661033000000001, 2024-09-14T09:06:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.347 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31210 2024-09-14T09:06:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068920+0000", 2024-09-14T09:06:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068920+0000", 2024-09-14T09:06:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068921+0000", 2024-09-14T09:06:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4900000000000001e-07 2024-09-14T09:06:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068924+0000", 2024-09-14T09:06:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2890000000000002e-06 2024-09-14T09:06:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068930+0000", 2024-09-14T09:06:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1020000000000002e-06 2024-09-14T09:06:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068936+0000", 2024-09-14T09:06:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1260000000000003e-06 2024-09-14T09:06:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092449+0000", 2024-09-14T09:06:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023512488000000002 2024-09-14T09:06:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092501+0000", 2024-09-14T09:06:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2385000000000001e-05 2024-09-14T09:06:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092581+0000", 2024-09-14T09:06:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0193999999999998e-05 2024-09-14T09:06:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31211 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:01.351 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.068944+0000", 2024-09-14T09:06:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2341963259999993, 2024-09-14T09:06:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023854895000000001, 2024-09-14T09:06:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31211 2024-09-14T09:06:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068944+0000", 2024-09-14T09:06:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068944+0000", 2024-09-14T09:06:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068944+0000", 2024-09-14T09:06:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9900000000000001e-07 2024-09-14T09:06:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068946+0000", 2024-09-14T09:06:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1579999999999999e-06 2024-09-14T09:06:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068953+0000", 2024-09-14T09:06:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3219999999999998e-06 2024-09-14T09:06:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068958+0000", 2024-09-14T09:06:01.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0390000000000004e-06 2024-09-14T09:06:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092630+0000", 2024-09-14T09:06:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023672397000000001 2024-09-14T09:06:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092702+0000", 2024-09-14T09:06:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2006000000000001e-05 2024-09-14T09:06:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092798+0000", 2024-09-14T09:06:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6473999999999998e-05 2024-09-14T09:06:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31212 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.068964+0000", 2024-09-14T09:06:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2341758580000004, 2024-09-14T09:06:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024056381000000002, 2024-09-14T09:06:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.356 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31212 2024-09-14T09:06:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068964+0000", 2024-09-14T09:06:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068964+0000", 2024-09-14T09:06:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068964+0000", 2024-09-14T09:06:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.34e-07 2024-09-14T09:06:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068966+0000", 2024-09-14T09:06:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.826e-06 2024-09-14T09:06:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068972+0000", 2024-09-14T09:06:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9070000000000004e-06 2024-09-14T09:06:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068977+0000", 2024-09-14T09:06:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6020000000000002e-06 2024-09-14T09:06:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092832+0000", 2024-09-14T09:06:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023854843000000001 2024-09-14T09:06:01.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.092903+0000", 2024-09-14T09:06:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0982000000000006e-05 2024-09-14T09:06:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093020+0000", 2024-09-14T09:06:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117787 2024-09-14T09:06:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31213 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 235520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.068983+0000", 2024-09-14T09:06:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2341564110000007, 2024-09-14T09:06:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024207487, 2024-09-14T09:06:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31213 2024-09-14T09:06:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.361 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068983+0000", 2024-09-14T09:06:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068983+0000", 2024-09-14T09:06:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068984+0000", 2024-09-14T09:06:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2399999999999999e-07 2024-09-14T09:06:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068986+0000", 2024-09-14T09:06:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1409999999999999e-06 2024-09-14T09:06:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068992+0000", 2024-09-14T09:06:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3419999999999996e-06 2024-09-14T09:06:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.068998+0000", 2024-09-14T09:06:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1359999999999996e-06 2024-09-14T09:06:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093051+0000", 2024-09-14T09:06:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024053741 2024-09-14T09:06:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093120+0000", 2024-09-14T09:06:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8714000000000007e-05 2024-09-14T09:06:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093191+0000", 2024-09-14T09:06:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0988999999999994e-05 2024-09-14T09:06:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31214 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.069004+0000", 2024-09-14T09:06:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2341354140000007, 2024-09-14T09:06:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024362755, 2024-09-14T09:06:01.365 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31214 2024-09-14T09:06:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069004+0000", 2024-09-14T09:06:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069004+0000", 2024-09-14T09:06:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069005+0000", 2024-09-14T09:06:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3099999999999998e-07 2024-09-14T09:06:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069013+0000", 2024-09-14T09:06:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2330000000000006e-06 2024-09-14T09:06:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069020+0000", 2024-09-14T09:06:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9e-06 2024-09-14T09:06:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069025+0000", 2024-09-14T09:06:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0479999999999998e-06 2024-09-14T09:06:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093219+0000", 2024-09-14T09:06:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024193429999999998 2024-09-14T09:06:01.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093283+0000", 2024-09-14T09:06:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4449000000000006e-05 2024-09-14T09:06:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093367+0000", 2024-09-14T09:06:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4264000000000005e-05 2024-09-14T09:06:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31215 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 264192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:01.369 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.069033+0000", 2024-09-14T09:06:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2341067859999999, 2024-09-14T09:06:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024520486000000001, 2024-09-14T09:06:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31215 2024-09-14T09:06:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069033+0000", 2024-09-14T09:06:01.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069033+0000", 2024-09-14T09:06:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069034+0000", 2024-09-14T09:06:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6199999999999998e-07 2024-09-14T09:06:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069036+0000", 2024-09-14T09:06:01.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.283e-06 2024-09-14T09:06:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069043+0000", 2024-09-14T09:06:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9330000000000004e-06 2024-09-14T09:06:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069047+0000", 2024-09-14T09:06:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6940000000000003e-06 2024-09-14T09:06:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093396+0000", 2024-09-14T09:06:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024348452 2024-09-14T09:06:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093464+0000", 2024-09-14T09:06:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7879999999999994e-05 2024-09-14T09:06:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093554+0000", 2024-09-14T09:06:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9782000000000002e-05 2024-09-14T09:06:01.373 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31216 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 282624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.069086+0000", 2024-09-14T09:06:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2340540539999996, 2024-09-14T09:06:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024651639999999999, 2024-09-14T09:06:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31216 2024-09-14T09:06:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069086+0000", 2024-09-14T09:06:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069086+0000", 2024-09-14T09:06:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069086+0000", 2024-09-14T09:06:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4499999999999997e-07 2024-09-14T09:06:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069089+0000", 2024-09-14T09:06:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4040000000000002e-06 2024-09-14T09:06:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069097+0000", 2024-09-14T09:06:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7770000000000002e-06 2024-09-14T09:06:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069102+0000", 2024-09-14T09:06:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6359999999999999e-06 2024-09-14T09:06:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093581+0000", 2024-09-14T09:06:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024478771999999999 2024-09-14T09:06:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093648+0000", 2024-09-14T09:06:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6824999999999998e-05 2024-09-14T09:06:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093737+0000", 2024-09-14T09:06:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9580999999999995e-05 2024-09-14T09:06:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31217 3.0 3:02083ad0:::rbd_data.115ebc3c67d1.0000000000000007:c [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.069110+0000", 2024-09-14T09:06:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2340303630000005, 2024-09-14T09:06:01.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024872781, 2024-09-14T09:06:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31217 2024-09-14T09:06:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069110+0000", 2024-09-14T09:06:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069110+0000", 2024-09-14T09:06:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069110+0000", 2024-09-14T09:06:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5899999999999996e-07 2024-09-14T09:06:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069112+0000", 2024-09-14T09:06:01.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7460000000000001e-06 2024-09-14T09:06:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069118+0000", 2024-09-14T09:06:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6810000000000003e-06 2024-09-14T09:06:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.069128+0000", 2024-09-14T09:06:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0675e-05 2024-09-14T09:06:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093816+0000", 2024-09-14T09:06:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024687333999999998 2024-09-14T09:06:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093887+0000", 2024-09-14T09:06:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1614999999999998e-05 2024-09-14T09:06:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.093982+0000", 2024-09-14T09:06:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5171000000000001e-05 2024-09-14T09:06:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:37541 3.19 3:9e83327d:::rbd_data.115eb07a8abd.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:56.440900+0000", 2024-09-14T09:06:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8622401379999998, 2024-09-14T09:06:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.044952620999999998, 2024-09-14T09:06:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 37541 2024-09-14T09:06:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.383 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.440900+0000", 2024-09-14T09:06:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.440900+0000", 2024-09-14T09:06:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.440902+0000", 2024-09-14T09:06:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7680000000000001e-06 2024-09-14T09:06:01.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.440908+0000", 2024-09-14T09:06:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5880000000000003e-06 2024-09-14T09:06:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.440926+0000", 2024-09-14T09:06:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7982000000000002e-05 2024-09-14T09:06:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.440936+0000", 2024-09-14T09:06:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9389999999999992e-06 2024-09-14T09:06:01.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.440995+0000", 2024-09-14T09:06:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8981999999999999e-05 2024-09-14T09:06:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.441058+0000", 2024-09-14T09:06:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2686000000000003e-05 2024-09-14T09:06:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:06:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.443155+0000", 2024-09-14T09:06:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0020971190000000002 2024-09-14T09:06:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.485711+0000", 2024-09-14T09:06:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042556121000000002 2024-09-14T09:06:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.485775+0000", 2024-09-14T09:06:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4268000000000002e-05 2024-09-14T09:06:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.485817+0000", 2024-09-14T09:06:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1743000000000003e-05 2024-09-14T09:06:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.485852+0000", 2024-09-14T09:06:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5425e-05 2024-09-14T09:06:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:37541 3.19 e48/23 ondisk, result = 0)", 2024-09-14T09:06:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:56.449519+0000", 2024-09-14T09:06:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8536204820000002, 2024-09-14T09:06:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036340787999999999, 2024-09-14T09:06:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.388 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.449519+0000", 2024-09-14T09:06:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.449519+0000", 2024-09-14T09:06:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.449521+0000", 2024-09-14T09:06:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.702e-06 2024-09-14T09:06:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.449529+0000", 2024-09-14T09:06:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3559999999999993e-06 2024-09-14T09:06:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.449553+0000", 2024-09-14T09:06:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3337e-05 2024-09-14T09:06:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.449567+0000", 2024-09-14T09:06:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.416e-05 2024-09-14T09:06:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.485762+0000", 2024-09-14T09:06:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036194573000000001 2024-09-14T09:06:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.485770+0000", 2024-09-14T09:06:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4789999999999996e-06 2024-09-14T09:06:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:56.485860+0000", 2024-09-14T09:06:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0180999999999996e-05 2024-09-14T09:06:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:38524 3.a e49/34 3:563c85eb:::rbd_data.115e9e133b92.0000000000000001:head v 49'144, mlcod=48'142)", 2024-09-14T09:06:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:58.460927+0000", 2024-09-14T09:06:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 2.842212704, 2024-09-14T09:06:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027150429, 2024-09-14T09:06:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.460927+0000", 2024-09-14T09:06:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.460927+0000", 2024-09-14T09:06:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.460930+0000", 2024-09-14T09:06:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.48e-06 2024-09-14T09:06:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.464780+0000", 2024-09-14T09:06:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.003849828 2024-09-14T09:06:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.464803+0000", 2024-09-14T09:06:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3178000000000001e-05 2024-09-14T09:06:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.464816+0000", 2024-09-14T09:06:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2814e-05 2024-09-14T09:06:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.464904+0000", 2024-09-14T09:06:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8646000000000005e-05 2024-09-14T09:06:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.464942+0000", 2024-09-14T09:06:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7827999999999998e-05 2024-09-14T09:06:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.488006+0000", 2024-09-14T09:06:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023064312 2024-09-14T09:06:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.488078+0000", 2024-09-14T09:06:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1342999999999998e-05 2024-09-14T09:06:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.395 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.396 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.396 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:01.397 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:06:01.397 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:01.392+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:01.498 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:01.499+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:01.576 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:01.576 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:01.577 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:01.577 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:01.592 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:06:01.638 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:01.636+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:01.700 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:01.700 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:01.700 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:01.700 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:01.700 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:01.700 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:01.696+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:01.713 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:06:01.802 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:01.803+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:01.814 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is set 2024-09-14T09:06:01.885 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18383 3.9 3:9252f7c9:::rbd_data.115e3abd3b2c.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:06:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.725125858999998, 2024-09-14T09:06:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056433830999999997, 2024-09-14T09:06:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.886 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18383 2024-09-14T09:06:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158423+0000", 2024-09-14T09:06:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T09:06:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158429+0000", 2024-09-14T09:06:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0179999999999998e-06 2024-09-14T09:06:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158440+0000", 2024-09-14T09:06:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0769e-05 2024-09-14T09:06:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158445+0000", 2024-09-14T09:06:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5840000000000002e-06 2024-09-14T09:06:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158495+0000", 2024-09-14T09:06:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9129999999999999e-05 2024-09-14T09:06:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158598+0000", 2024-09-14T09:06:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102938 2024-09-14T09:06:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.161048+0000", 2024-09-14T09:06:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002450336 2024-09-14T09:06:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214605+0000", 2024-09-14T09:06:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053557526000000001 2024-09-14T09:06:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214634+0000", 2024-09-14T09:06:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8483000000000002e-05 2024-09-14T09:06:01.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214731+0000", 2024-09-14T09:06:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7026000000000002e-05 2024-09-14T09:06:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214856+0000", 2024-09-14T09:06:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012474799999999999 2024-09-14T09:06:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18383 3.9 e38/34 ondisk, result = 0)", 2024-09-14T09:06:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.714809377999998, 2024-09-14T09:06:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045880575999999999, 2024-09-14T09:06:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168739+0000", 2024-09-14T09:06:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3900000000000004e-07 2024-09-14T09:06:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168744+0000", 2024-09-14T09:06:01.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5929999999999999e-06 2024-09-14T09:06:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168754+0000", 2024-09-14T09:06:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0686999999999999e-05 2024-09-14T09:06:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168761+0000", 2024-09-14T09:06:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6340000000000003e-06 2024-09-14T09:06:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214578+0000", 2024-09-14T09:06:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045817443999999999 2024-09-14T09:06:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214596+0000", 2024-09-14T09:06:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7159e-05 2024-09-14T09:06:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214619+0000", 2024-09-14T09:06:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.332e-05 2024-09-14T09:06:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19152 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14159058~8940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.560896922000001, 2024-09-14T09:06:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12891994500000001, 2024-09-14T09:06:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19152 2024-09-14T09:06:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322653+0000", 2024-09-14T09:06:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3750000000000001e-06 2024-09-14T09:06:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329647+0000", 2024-09-14T09:06:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069941960000000003 2024-09-14T09:06:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329673+0000", 2024-09-14T09:06:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5760000000000001e-05 2024-09-14T09:06:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329684+0000", 2024-09-14T09:06:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0474e-05 2024-09-14T09:06:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329776+0000", 2024-09-14T09:06:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2467999999999996e-05 2024-09-14T09:06:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329987+0000", 2024-09-14T09:06:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021133 2024-09-14T09:06:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.331303+0000", 2024-09-14T09:06:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001315629 2024-09-14T09:06:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:01.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.411090+0000", 2024-09-14T09:06:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079787477999999995 2024-09-14T09:06:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451463+0000", 2024-09-14T09:06:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040372312 2024-09-14T09:06:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451523+0000", 2024-09-14T09:06:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0420000000000001e-05 2024-09-14T09:06:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451571+0000", 2024-09-14T09:06:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7503000000000002e-05 2024-09-14T09:06:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20925 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:06:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.211598142, 2024-09-14T09:06:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017084327999999999, 2024-09-14T09:06:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20925 2024-09-14T09:06:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:06:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:06:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:06:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3e-07 2024-09-14T09:06:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671953+0000", 2024-09-14T09:06:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0299999999999998e-06 2024-09-14T09:06:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671964+0000", 2024-09-14T09:06:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1e-05 2024-09-14T09:06:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671971+0000", 2024-09-14T09:06:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9929999999999998e-06 2024-09-14T09:06:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.687941+0000", 2024-09-14T09:06:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015969357 2024-09-14T09:06:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.688070+0000", 2024-09-14T09:06:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000129858 2024-09-14T09:06:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689034+0000", 2024-09-14T09:06:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00096336000000000002 2024-09-14T09:06:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20926 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.211564922000001, 2024-09-14T09:06:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018572207, 2024-09-14T09:06:01.905 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20926 2024-09-14T09:06:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:06:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671990+0000", 2024-09-14T09:06:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5769999999999999e-06 2024-09-14T09:06:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672001+0000", 2024-09-14T09:06:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1293e-05 2024-09-14T09:06:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672008+0000", 2024-09-14T09:06:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9099999999999999e-06 2024-09-14T09:06:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689147+0000", 2024-09-14T09:06:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017139060000000001 2024-09-14T09:06:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689324+0000", 2024-09-14T09:06:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017640200000000001 2024-09-14T09:06:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690555+0000", 2024-09-14T09:06:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001231212 2024-09-14T09:06:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20927 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2621440~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:01.909 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.211527955999999, 2024-09-14T09:06:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020002227000000001, 2024-09-14T09:06:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20927 2024-09-14T09:06:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9699999999999995e-07 2024-09-14T09:06:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672024+0000", 2024-09-14T09:06:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:06:01.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672036+0000", 2024-09-14T09:06:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.119e-05 2024-09-14T09:06:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672044+0000", 2024-09-14T09:06:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8370000000000003e-06 2024-09-14T09:06:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690602+0000", 2024-09-14T09:06:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018557648 2024-09-14T09:06:01.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690805+0000", 2024-09-14T09:06:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020282899999999999 2024-09-14T09:06:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692022+0000", 2024-09-14T09:06:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012170429999999999 2024-09-14T09:06:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20928 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 917504~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.211491816000001, 2024-09-14T09:06:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021623738, 2024-09-14T09:06:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20928 2024-09-14T09:06:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.914 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672057+0000", 2024-09-14T09:06:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:06:01.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672059+0000", 2024-09-14T09:06:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9270000000000001e-06 2024-09-14T09:06:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672070+0000", 2024-09-14T09:06:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0773e-05 2024-09-14T09:06:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672077+0000", 2024-09-14T09:06:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9600000000000003e-06 2024-09-14T09:06:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692154+0000", 2024-09-14T09:06:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020076482 2024-09-14T09:06:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692419+0000", 2024-09-14T09:06:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026517400000000002 2024-09-14T09:06:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693680+0000", 2024-09-14T09:06:01.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012607390000000001 2024-09-14T09:06:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20929 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2752512~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.211458774, 2024-09-14T09:06:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022898728, 2024-09-14T09:06:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20929 2024-09-14T09:06:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672090+0000", 2024-09-14T09:06:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:06:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672092+0000", 2024-09-14T09:06:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8940000000000002e-06 2024-09-14T09:06:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672103+0000", 2024-09-14T09:06:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0815999999999999e-05 2024-09-14T09:06:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672110+0000", 2024-09-14T09:06:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9469999999999997e-06 2024-09-14T09:06:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693737+0000", 2024-09-14T09:06:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021626467999999999 2024-09-14T09:06:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693962+0000", 2024-09-14T09:06:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225474 2024-09-14T09:06:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.694988+0000", 2024-09-14T09:06:01.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001025469 2024-09-14T09:06:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20930 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.21142568, 2024-09-14T09:06:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024303286, 2024-09-14T09:06:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.923 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20930 2024-09-14T09:06:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672123+0000", 2024-09-14T09:06:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:06:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672125+0000", 2024-09-14T09:06:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7300000000000001e-06 2024-09-14T09:06:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672134+0000", 2024-09-14T09:06:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7930000000000003e-06 2024-09-14T09:06:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672141+0000", 2024-09-14T09:06:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9029999999999998e-06 2024-09-14T09:06:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695087+0000", 2024-09-14T09:06:01.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022946013000000001 2024-09-14T09:06:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695350+0000", 2024-09-14T09:06:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026266000000000001 2024-09-14T09:06:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696425+0000", 2024-09-14T09:06:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001075527 2024-09-14T09:06:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20931 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.211392451, 2024-09-14T09:06:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025331519, 2024-09-14T09:06:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.928 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20931 2024-09-14T09:06:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672156+0000", 2024-09-14T09:06:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4300000000000002e-07 2024-09-14T09:06:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672159+0000", 2024-09-14T09:06:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T09:06:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672170+0000", 2024-09-14T09:06:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1192999999999999e-05 2024-09-14T09:06:01.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672177+0000", 2024-09-14T09:06:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9059999999999996e-06 2024-09-14T09:06:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696532+0000", 2024-09-14T09:06:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024355291000000001 2024-09-14T09:06:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696755+0000", 2024-09-14T09:06:01.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022297699999999999 2024-09-14T09:06:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697487+0000", 2024-09-14T09:06:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00073171499999999997 2024-09-14T09:06:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20932 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1179648~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.211359008000001, 2024-09-14T09:06:01.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026104083, 2024-09-14T09:06:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20932 2024-09-14T09:06:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0699999999999996e-07 2024-09-14T09:06:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672192+0000", 2024-09-14T09:06:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7700000000000002e-06 2024-09-14T09:06:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672203+0000", 2024-09-14T09:06:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0956e-05 2024-09-14T09:06:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672211+0000", 2024-09-14T09:06:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3769999999999996e-06 2024-09-14T09:06:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697530+0000", 2024-09-14T09:06:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025318411999999998 2024-09-14T09:06:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697640+0000", 2024-09-14T09:06:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011045299999999999 2024-09-14T09:06:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698293+0000", 2024-09-14T09:06:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065240800000000005 2024-09-14T09:06:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.936 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20933 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3014656~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.211324805, 2024-09-14T09:06:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026558875999999999, 2024-09-14T09:06:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20933 2024-09-14T09:06:01.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672224+0000", 2024-09-14T09:06:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2399999999999997e-07 2024-09-14T09:06:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672234+0000", 2024-09-14T09:06:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0793e-05 2024-09-14T09:06:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672246+0000", 2024-09-14T09:06:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659e-05 2024-09-14T09:06:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672255+0000", 2024-09-14T09:06:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5730000000000008e-06 2024-09-14T09:06:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698326+0000", 2024-09-14T09:06:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026071414000000001 2024-09-14T09:06:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698374+0000", 2024-09-14T09:06:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8032999999999999e-05 2024-09-14T09:06:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698782+0000", 2024-09-14T09:06:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040768 2024-09-14T09:06:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20934 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1310720~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.211282928999999, 2024-09-14T09:06:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027292067999999999, 2024-09-14T09:06:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.941 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20934 2024-09-14T09:06:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672266+0000", 2024-09-14T09:06:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6299999999999993e-07 2024-09-14T09:06:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672269+0000", 2024-09-14T09:06:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8569999999999999e-06 2024-09-14T09:06:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672280+0000", 2024-09-14T09:06:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1236999999999999e-05 2024-09-14T09:06:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672287+0000", 2024-09-14T09:06:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8260000000000004e-06 2024-09-14T09:06:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698820+0000", 2024-09-14T09:06:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026533055999999999 2024-09-14T09:06:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698933+0000", 2024-09-14T09:06:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011291 2024-09-14T09:06:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699557+0000", 2024-09-14T09:06:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062421899999999997 2024-09-14T09:06:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20935 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:01.945 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.211249169, 2024-09-14T09:06:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028108610999999999, 2024-09-14T09:06:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20935 2024-09-14T09:06:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672299+0000", 2024-09-14T09:06:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T09:06:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672302+0000", 2024-09-14T09:06:01.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7640000000000001e-06 2024-09-14T09:06:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672313+0000", 2024-09-14T09:06:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1080000000000001e-05 2024-09-14T09:06:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672321+0000", 2024-09-14T09:06:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6490000000000008e-06 2024-09-14T09:06:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699587+0000", 2024-09-14T09:06:01.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027265884000000001 2024-09-14T09:06:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699631+0000", 2024-09-14T09:06:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4947999999999997e-05 2024-09-14T09:06:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700407+0000", 2024-09-14T09:06:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00077559299999999999 2024-09-14T09:06:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20936 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1441792~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.211214993, 2024-09-14T09:06:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028784750000000001, 2024-09-14T09:06:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20936 2024-09-14T09:06:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4700000000000001e-07 2024-09-14T09:06:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672336+0000", 2024-09-14T09:06:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6730000000000001e-06 2024-09-14T09:06:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672347+0000", 2024-09-14T09:06:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0710000000000001e-05 2024-09-14T09:06:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672354+0000", 2024-09-14T09:06:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4630000000000004e-06 2024-09-14T09:06:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700442+0000", 2024-09-14T09:06:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028087951 2024-09-14T09:06:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700557+0000", 2024-09-14T09:06:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011461800000000001 2024-09-14T09:06:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701117+0000", 2024-09-14T09:06:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056058799999999999 2024-09-14T09:06:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20937 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.211181867000001, 2024-09-14T09:06:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029493028000000001, 2024-09-14T09:06:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.954 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20937 2024-09-14T09:06:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T09:06:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672384+0000", 2024-09-14T09:06:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7127000000000001e-05 2024-09-14T09:06:01.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672395+0000", 2024-09-14T09:06:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759999999999999e-05 2024-09-14T09:06:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672402+0000", 2024-09-14T09:06:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0960000000000003e-06 2024-09-14T09:06:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701147+0000", 2024-09-14T09:06:01.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028744241 2024-09-14T09:06:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701246+0000", 2024-09-14T09:06:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9031999999999996e-05 2024-09-14T09:06:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701859+0000", 2024-09-14T09:06:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061310499999999997 2024-09-14T09:06:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20938 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.211134706999999, 2024-09-14T09:06:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030109219999999999, 2024-09-14T09:06:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20938 2024-09-14T09:06:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672414+0000", 2024-09-14T09:06:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-07 2024-09-14T09:06:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672417+0000", 2024-09-14T09:06:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:06:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672428+0000", 2024-09-14T09:06:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0873000000000001e-05 2024-09-14T09:06:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672435+0000", 2024-09-14T09:06:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:06:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701884+0000", 2024-09-14T09:06:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029449811999999999 2024-09-14T09:06:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701934+0000", 2024-09-14T09:06:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9703000000000002e-05 2024-09-14T09:06:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702522+0000", 2024-09-14T09:06:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00058811200000000001 2024-09-14T09:06:01.962 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20939 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.211102574, 2024-09-14T09:06:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030779259, 2024-09-14T09:06:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20939 2024-09-14T09:06:01.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672446+0000", 2024-09-14T09:06:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:06:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672449+0000", 2024-09-14T09:06:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7930000000000002e-06 2024-09-14T09:06:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672461+0000", 2024-09-14T09:06:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T09:06:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672471+0000", 2024-09-14T09:06:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0807e-05 2024-09-14T09:06:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702551+0000", 2024-09-14T09:06:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030079228 2024-09-14T09:06:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702664+0000", 2024-09-14T09:06:01.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011351800000000001 2024-09-14T09:06:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.703224+0000", 2024-09-14T09:06:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056026999999999997 2024-09-14T09:06:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29321 3.9 3:90303d3a:::rbd_data.115ebc3c67d1.0000000000000014:c [read 770048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:06:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:06:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.0640192489999993, 2024-09-14T09:06:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015942787, 2024-09-14T09:06:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29321 2024-09-14T09:06:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:06:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:06:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819529+0000", 2024-09-14T09:06:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8800000000000003e-07 2024-09-14T09:06:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819531+0000", 2024-09-14T09:06:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9439999999999999e-06 2024-09-14T09:06:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819537+0000", 2024-09-14T09:06:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.037e-06 2024-09-14T09:06:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819542+0000", 2024-09-14T09:06:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7670000000000003e-06 2024-09-14T09:06:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835320+0000", 2024-09-14T09:06:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015778119 2024-09-14T09:06:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835384+0000", 2024-09-14T09:06:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3897000000000002e-05 2024-09-14T09:06:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835471+0000", 2024-09-14T09:06:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7534999999999995e-05 2024-09-14T09:06:01.971 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29322 3.9 3:90303d3a:::rbd_data.115ebc3c67d1.0000000000000014:c [read 780288~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:06:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:06:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.0639997389999998, 2024-09-14T09:06:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016069904999999999, 2024-09-14T09:06:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:01.972 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29322 2024-09-14T09:06:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:06:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:06:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:06:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9499999999999998e-07 2024-09-14T09:06:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819550+0000", 2024-09-14T09:06:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8309999999999999e-06 2024-09-14T09:06:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819556+0000", 2024-09-14T09:06:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1299999999999998e-06 2024-09-14T09:06:01.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819561+0000", 2024-09-14T09:06:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.707e-06 2024-09-14T09:06:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835500+0000", 2024-09-14T09:06:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015939456000000001 2024-09-14T09:06:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835566+0000", 2024-09-14T09:06:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5285999999999998e-05 2024-09-14T09:06:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:01.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835618+0000", 2024-09-14T09:06:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2099999999999999e-05 2024-09-14T09:06:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:01.976 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:01.976 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:01.977 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:06:01.978 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:01.979+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:02.078 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:02.079+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:02.108 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:02.109 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:02.109 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:02.109 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:02.123 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:06:02.179 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:02.179+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:02.244 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:02.244 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:02.244 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:02.244 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:02.245 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:02.258 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:06:02.302 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:02.300+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:02.379 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:02.379 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:02.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:02.379 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:02.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.379 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:02.379 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:02.379 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.293689909000001, 2024-09-14T09:06:02.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:06:02.379 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.380 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.380 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:02.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:02.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:02.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.380 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:06:02.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.380 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:02.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:02.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:06:02.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:06:02.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:06:02.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:06:02.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:06:02.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:06:02.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:06:02.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:06:02.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:06:02.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:06:02.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:02.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:06:02.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:06:02.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:06:02.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:06:02.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:06:02.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:06:02.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:06:02.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:06:02.384 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.384 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.384 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.385 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:02.385 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:02.385 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.293635251000001, 2024-09-14T09:06:02.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:06:02.385 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.385 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.385 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:02.385 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:02.385 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:02.385 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.385 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:06:02.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.386 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:02.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:02.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:02.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:06:02.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:06:02.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:06:02.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:06:02.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:06:02.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:06:02.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:06:02.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:06:02.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:06:02.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:02.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:06:02.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:06:02.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:06:02.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:06:02.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:06:02.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:06:02.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:06:02.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:06:02.390 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.390 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.390 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.390 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:02.390 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:02.390 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.293529877000001, 2024-09-14T09:06:02.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:06:02.390 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.390 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.390 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:02.391 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:02.391 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:02.391 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.391 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:06:02.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.391 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:02.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:02.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:06:02.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:06:02.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:06:02.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:06:02.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:06:02.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:06:02.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:06:02.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:06:02.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:06:02.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:06:02.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:02.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:06:02.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:06:02.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:06:02.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:06:02.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:06:02.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:06:02.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:06:02.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:06:02.395 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.395 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.395 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.395 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:02.395 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:02.395 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.293173113000002, 2024-09-14T09:06:02.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:06:02.396 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.396 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.396 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:02.396 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:02.396 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:02.396 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.396 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:06:02.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.396 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:02.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:02.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:06:02.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:06:02.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:06:02.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:06:02.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:06:02.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:06:02.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:06:02.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:06:02.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:06:02.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:06:02.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:02.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:06:02.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:06:02.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:06:02.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:06:02.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:06:02.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:06:02.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:06:02.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:06:02.400 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.401 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.401 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.401 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:02.401 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:02.401 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.293088375, 2024-09-14T09:06:02.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:06:02.401 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.401 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.401 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:02.401 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:02.401 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:02.402 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.402 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:06:02.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.402 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:02.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:02.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:06:02.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:06:02.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:06:02.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:06:02.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:06:02.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:06:02.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:06:02.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:06:02.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:06:02.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:06:02.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:02.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:06:02.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:06:02.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:06:02.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:06:02.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:06:02.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:06:02.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:06:02.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:06:02.406 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.406 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.406 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.406 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:02.406 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:02.406 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.293000688999999, 2024-09-14T09:06:02.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:06:02.406 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.406 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.407 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:02.407 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:02.407 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:02.407 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.407 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:06:02.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.407 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:02.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:02.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:06:02.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:06:02.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:06:02.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:06:02.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:06:02.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:06:02.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:06:02.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:06:02.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:06:02.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:06:02.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:02.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:06:02.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:06:02.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:06:02.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:06:02.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:06:02.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:06:02.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:06:02.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:06:02.411 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.411 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.411 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.411 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18388 3.18 3:19e2be55:::rbd_data.115e3abd3b2c.0000000000000021:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:06:02.411 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:02.411 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.157087378, 2024-09-14T09:06:02.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042293648000000003, 2024-09-14T09:06:02.411 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.412 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.412 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:02.412 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:02.412 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:02.412 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.412 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18388 2024-09-14T09:06:02.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.412 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:02.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:02.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220236+0000", 2024-09-14T09:06:02.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1339999999999998e-06 2024-09-14T09:06:02.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220244+0000", 2024-09-14T09:06:02.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1950000000000003e-06 2024-09-14T09:06:02.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220262+0000", 2024-09-14T09:06:02.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8017e-05 2024-09-14T09:06:02.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220270+0000", 2024-09-14T09:06:02.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9480000000000008e-06 2024-09-14T09:06:02.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220357+0000", 2024-09-14T09:06:02.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7185e-05 2024-09-14T09:06:02.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220576+0000", 2024-09-14T09:06:02.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021912200000000001 2024-09-14T09:06:02.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:02.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.222435+0000", 2024-09-14T09:06:02.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018592350000000001 2024-09-14T09:06:02.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:02.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262439+0000", 2024-09-14T09:06:02.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040003368999999997 2024-09-14T09:06:02.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:02.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262452+0000", 2024-09-14T09:06:02.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2941e-05 2024-09-14T09:06:02.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:02.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262490+0000", 2024-09-14T09:06:02.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8478e-05 2024-09-14T09:06:02.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262527+0000", 2024-09-14T09:06:02.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7024000000000003e-05 2024-09-14T09:06:02.417 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.417 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.417 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.417 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18388 3.18 e38/28 ondisk, result = 0)", 2024-09-14T09:06:02.417 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.227795+0000", 2024-09-14T09:06:02.417 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.149525735000001, 2024-09-14T09:06:02.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034648832999999997, 2024-09-14T09:06:02.417 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.417 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.417 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:02.417 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227795+0000", 2024-09-14T09:06:02.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227795+0000", 2024-09-14T09:06:02.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227796+0000", 2024-09-14T09:06:02.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.879999999999999e-07 2024-09-14T09:06:02.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227801+0000", 2024-09-14T09:06:02.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4909999999999999e-06 2024-09-14T09:06:02.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227815+0000", 2024-09-14T09:06:02.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4905e-05 2024-09-14T09:06:02.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227824+0000", 2024-09-14T09:06:02.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7770000000000008e-06 2024-09-14T09:06:02.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262425+0000", 2024-09-14T09:06:02.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034601109999999997 2024-09-14T09:06:02.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262434+0000", 2024-09-14T09:06:02.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5250000000000005e-06 2024-09-14T09:06:02.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262444+0000", 2024-09-14T09:06:02.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0037000000000001e-05 2024-09-14T09:06:02.421 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.421 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.421 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.421 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:19543 3.5 e39/23 3:a536ff9b:::rbd_data.115e4c71d6e8.0000000000000002:head v 39'8, mlcod=39'6)", 2024-09-14T09:06:02.421 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:02.421 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.854232717999999, 2024-09-14T09:06:02.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.039280019999999999, 2024-09-14T09:06:02.422 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.422 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.422 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:02.422 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:02.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:02.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523090+0000", 2024-09-14T09:06:02.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.773e-06 2024-09-14T09:06:02.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537290+0000", 2024-09-14T09:06:02.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014200007000000001 2024-09-14T09:06:02.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537317+0000", 2024-09-14T09:06:02.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6704999999999999e-05 2024-09-14T09:06:02.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537328+0000", 2024-09-14T09:06:02.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1299e-05 2024-09-14T09:06:02.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537417+0000", 2024-09-14T09:06:02.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8869999999999994e-05 2024-09-14T09:06:02.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537445+0000", 2024-09-14T09:06:02.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8377999999999998e-05 2024-09-14T09:06:02.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:02.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562324+0000", 2024-09-14T09:06:02.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024879307999999999 2024-09-14T09:06:02.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562368+0000", 2024-09-14T09:06:02.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3680000000000002e-05 2024-09-14T09:06:02.426 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.426 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.426 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.426 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20736 3.2 3:45030d14:::rbd_data.115e889d4809.000000000000027a:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:02.426 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.099577+0000", 2024-09-14T09:06:02.426 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.277743675, 2024-09-14T09:06:02.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026646762000000001, 2024-09-14T09:06:02.427 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.427 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.427 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:02.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:02.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:02.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.427 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20736 2024-09-14T09:06:02.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.427 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099577+0000", 2024-09-14T09:06:02.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099577+0000", 2024-09-14T09:06:02.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099579+0000", 2024-09-14T09:06:02.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5009999999999999e-06 2024-09-14T09:06:02.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099585+0000", 2024-09-14T09:06:02.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2870000000000001e-06 2024-09-14T09:06:02.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099602+0000", 2024-09-14T09:06:02.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7176e-05 2024-09-14T09:06:02.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099611+0000", 2024-09-14T09:06:02.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7010000000000002e-06 2024-09-14T09:06:02.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099664+0000", 2024-09-14T09:06:02.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3183000000000002e-05 2024-09-14T09:06:02.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.099793+0000", 2024-09-14T09:06:02.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012945099999999999 2024-09-14T09:06:02.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:02.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.100819+0000", 2024-09-14T09:06:02.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010252149999999999 2024-09-14T09:06:02.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:02.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.125929+0000", 2024-09-14T09:06:02.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025110072000000001 2024-09-14T09:06:02.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:02.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.126034+0000", 2024-09-14T09:06:02.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105655 2024-09-14T09:06:02.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:02.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.126141+0000", 2024-09-14T09:06:02.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107069 2024-09-14T09:06:02.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.126224+0000", 2024-09-14T09:06:02.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2452000000000003e-05 2024-09-14T09:06:02.432 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.432 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.432 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.432 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31015 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:02.432 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.062742+0000", 2024-09-14T09:06:02.432 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.3145783469999994, 2024-09-14T09:06:02.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027690059999999999, 2024-09-14T09:06:02.433 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.433 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.433 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:02.433 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:02.433 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:02.433 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.433 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31015 2024-09-14T09:06:02.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.433 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062742+0000", 2024-09-14T09:06:02.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062742+0000", 2024-09-14T09:06:02.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062743+0000", 2024-09-14T09:06:02.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.51e-07 2024-09-14T09:06:02.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062745+0000", 2024-09-14T09:06:02.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0480000000000001e-06 2024-09-14T09:06:02.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062751+0000", 2024-09-14T09:06:02.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3929999999999996e-06 2024-09-14T09:06:02.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062757+0000", 2024-09-14T09:06:02.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5130000000000004e-06 2024-09-14T09:06:02.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.088137+0000", 2024-09-14T09:06:02.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025380494999999999 2024-09-14T09:06:02.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.088178+0000", 2024-09-14T09:06:02.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0963000000000001e-05 2024-09-14T09:06:02.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.090433+0000", 2024-09-14T09:06:02.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0022541969999999999 2024-09-14T09:06:02.436 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.436 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.436 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.437 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31022 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:02.437 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.062919+0000", 2024-09-14T09:06:02.437 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.3144016290000007, 2024-09-14T09:06:02.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026754692, 2024-09-14T09:06:02.437 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.437 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.437 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:02.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:02.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:02.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.437 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31022 2024-09-14T09:06:02.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.438 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062919+0000", 2024-09-14T09:06:02.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062919+0000", 2024-09-14T09:06:02.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062920+0000", 2024-09-14T09:06:02.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2099999999999997e-07 2024-09-14T09:06:02.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062922+0000", 2024-09-14T09:06:02.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.452e-06 2024-09-14T09:06:02.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062929+0000", 2024-09-14T09:06:02.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4439999999999996e-06 2024-09-14T09:06:02.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062958+0000", 2024-09-14T09:06:02.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8943999999999999e-05 2024-09-14T09:06:02.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089526+0000", 2024-09-14T09:06:02.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026568556 2024-09-14T09:06:02.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089600+0000", 2024-09-14T09:06:02.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3506999999999999e-05 2024-09-14T09:06:02.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089674+0000", 2024-09-14T09:06:02.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4268000000000001e-05 2024-09-14T09:06:02.441 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.441 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.441 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.441 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31023 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 264192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:02.441 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.062978+0000", 2024-09-14T09:06:02.441 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.3143424160000006, 2024-09-14T09:06:02.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026871328999999999, 2024-09-14T09:06:02.442 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.442 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.442 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:02.442 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:02.442 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:02.442 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.442 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31023 2024-09-14T09:06:02.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.442 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062978+0000", 2024-09-14T09:06:02.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062978+0000", 2024-09-14T09:06:02.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062979+0000", 2024-09-14T09:06:02.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6499999999999999e-07 2024-09-14T09:06:02.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062982+0000", 2024-09-14T09:06:02.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8650000000000001e-06 2024-09-14T09:06:02.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062990+0000", 2024-09-14T09:06:02.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9889999999999992e-06 2024-09-14T09:06:02.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063028+0000", 2024-09-14T09:06:02.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8154000000000001e-05 2024-09-14T09:06:02.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089700+0000", 2024-09-14T09:06:02.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026672548000000001 2024-09-14T09:06:02.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089772+0000", 2024-09-14T09:06:02.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1829999999999995e-05 2024-09-14T09:06:02.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089850+0000", 2024-09-14T09:06:02.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7378000000000004e-05 2024-09-14T09:06:02.445 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.446 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.446 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.446 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31024 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 282624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:02.446 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.063039+0000", 2024-09-14T09:06:02.446 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.3142817210000004, 2024-09-14T09:06:02.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027079282, 2024-09-14T09:06:02.446 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.446 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.446 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:02.446 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:02.446 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:02.446 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.447 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31024 2024-09-14T09:06:02.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.447 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063039+0000", 2024-09-14T09:06:02.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063039+0000", 2024-09-14T09:06:02.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063040+0000", 2024-09-14T09:06:02.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8299999999999997e-07 2024-09-14T09:06:02.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063043+0000", 2024-09-14T09:06:02.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9610000000000001e-06 2024-09-14T09:06:02.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063058+0000", 2024-09-14T09:06:02.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5529999999999999e-05 2024-09-14T09:06:02.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063065+0000", 2024-09-14T09:06:02.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.596e-06 2024-09-14T09:06:02.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089877+0000", 2024-09-14T09:06:02.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026812373 2024-09-14T09:06:02.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.089949+0000", 2024-09-14T09:06:02.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2014000000000005e-05 2024-09-14T09:06:02.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.090118+0000", 2024-09-14T09:06:02.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016922500000000001 2024-09-14T09:06:02.450 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.450 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.450 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.450 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31025 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 292864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:02.450 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.063074+0000", 2024-09-14T09:06:02.450 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.3142470799999995, 2024-09-14T09:06:02.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027206443, 2024-09-14T09:06:02.450 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.450 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.451 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:02.451 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:02.451 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:02.451 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.451 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31025 2024-09-14T09:06:02.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.451 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063074+0000", 2024-09-14T09:06:02.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063074+0000", 2024-09-14T09:06:02.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063074+0000", 2024-09-14T09:06:02.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.68e-07 2024-09-14T09:06:02.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063077+0000", 2024-09-14T09:06:02.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.853e-06 2024-09-14T09:06:02.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063084+0000", 2024-09-14T09:06:02.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0419999999999997e-06 2024-09-14T09:06:02.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063089+0000", 2024-09-14T09:06:02.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1150000000000002e-06 2024-09-14T09:06:02.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.090144+0000", 2024-09-14T09:06:02.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027054974999999998 2024-09-14T09:06:02.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.090211+0000", 2024-09-14T09:06:02.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6832e-05 2024-09-14T09:06:02.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.090280+0000", 2024-09-14T09:06:02.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9058000000000002e-05 2024-09-14T09:06:02.454 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.454 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.454 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.454 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31279 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 274432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:02.455 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.072774+0000", 2024-09-14T09:06:02.455 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.3045468170000003, 2024-09-14T09:06:02.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026662447999999998, 2024-09-14T09:06:02.455 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.455 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.455 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:02.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:02.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:02.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.455 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31279 2024-09-14T09:06:02.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.455 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072774+0000", 2024-09-14T09:06:02.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072774+0000", 2024-09-14T09:06:02.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072775+0000", 2024-09-14T09:06:02.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0900000000000001e-07 2024-09-14T09:06:02.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072778+0000", 2024-09-14T09:06:02.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7340000000000002e-06 2024-09-14T09:06:02.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072789+0000", 2024-09-14T09:06:02.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0287e-05 2024-09-14T09:06:02.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.072795+0000", 2024-09-14T09:06:02.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7390000000000002e-06 2024-09-14T09:06:02.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.099278+0000", 2024-09-14T09:06:02.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026482887 2024-09-14T09:06:02.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.099353+0000", 2024-09-14T09:06:02.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5013999999999997e-05 2024-09-14T09:06:02.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.099436+0000", 2024-09-14T09:06:02.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3078000000000007e-05 2024-09-14T09:06:02.459 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.459 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.459 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.459 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38002 3.1e 3:7df715aa:::rbd_data.115eb07a8abd.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:02.459 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:02.459 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.9875694719999997, 2024-09-14T09:06:02.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.065148304000000004, 2024-09-14T09:06:02.459 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.459 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.460 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:02.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:02.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:02.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.460 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38002 2024-09-14T09:06:02.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.460 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:02.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:02.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389752+0000", 2024-09-14T09:06:02.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1829999999999999e-06 2024-09-14T09:06:02.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389758+0000", 2024-09-14T09:06:02.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0919999999999997e-06 2024-09-14T09:06:02.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389769+0000", 2024-09-14T09:06:02.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759e-05 2024-09-14T09:06:02.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389775+0000", 2024-09-14T09:06:02.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8819999999999998e-06 2024-09-14T09:06:02.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389824+0000", 2024-09-14T09:06:02.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8573999999999999e-05 2024-09-14T09:06:02.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389876+0000", 2024-09-14T09:06:02.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2643999999999999e-05 2024-09-14T09:06:02.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:06:02.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.392640+0000", 2024-09-14T09:06:02.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0027636409999999998 2024-09-14T09:06:02.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:02.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454672+0000", 2024-09-14T09:06:02.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062031703000000001 2024-09-14T09:06:02.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:02.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454765+0000", 2024-09-14T09:06:02.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2862000000000006e-05 2024-09-14T09:06:02.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:02.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454841+0000", 2024-09-14T09:06:02.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5857000000000002e-05 2024-09-14T09:06:02.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454900+0000", 2024-09-14T09:06:02.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9107000000000002e-05 2024-09-14T09:06:02.464 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.465 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.465 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.465 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:38002 3.1e e48/23 ondisk, result = 0)", 2024-09-14T09:06:02.465 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:02.465 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.9772528830000002, 2024-09-14T09:06:02.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054841741999999999, 2024-09-14T09:06:02.465 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.465 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.465 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:02.465 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:02.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:02.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400069+0000", 2024-09-14T09:06:02.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.339e-06 2024-09-14T09:06:02.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400076+0000", 2024-09-14T09:06:02.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7800000000000003e-06 2024-09-14T09:06:02.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400093+0000", 2024-09-14T09:06:02.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6915999999999999e-05 2024-09-14T09:06:02.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400106+0000", 2024-09-14T09:06:02.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2785e-05 2024-09-14T09:06:02.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454713+0000", 2024-09-14T09:06:02.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054607041000000002 2024-09-14T09:06:02.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454724+0000", 2024-09-14T09:06:02.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1202e-05 2024-09-14T09:06:02.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454910+0000", 2024-09-14T09:06:02.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018567900000000001 2024-09-14T09:06:02.469 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.469 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.469 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.469 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38100 3.b 3:d0f67c24:::rbd_data.115eb07a8abd.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:02.469 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:02.469 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.7313992230000004, 2024-09-14T09:06:02.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053701964999999997, 2024-09-14T09:06:02.469 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.469 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.469 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:02.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:02.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:02.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.470 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38100 2024-09-14T09:06:02.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.470 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:02.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:02.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645923+0000", 2024-09-14T09:06:02.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:06:02.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645928+0000", 2024-09-14T09:06:02.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0699999999999997e-06 2024-09-14T09:06:02.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645942+0000", 2024-09-14T09:06:02.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4069e-05 2024-09-14T09:06:02.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645951+0000", 2024-09-14T09:06:02.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3230000000000004e-06 2024-09-14T09:06:02.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646008+0000", 2024-09-14T09:06:02.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6583999999999999e-05 2024-09-14T09:06:02.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646089+0000", 2024-09-14T09:06:02.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1557999999999994e-05 2024-09-14T09:06:02.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:02.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.650355+0000", 2024-09-14T09:06:02.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004265525 2024-09-14T09:06:02.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:02.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699439+0000", 2024-09-14T09:06:02.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049083904999999997 2024-09-14T09:06:02.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:02.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699467+0000", 2024-09-14T09:06:02.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8198999999999999e-05 2024-09-14T09:06:02.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:02.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699552+0000", 2024-09-14T09:06:02.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T09:06:02.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699624+0000", 2024-09-14T09:06:02.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1725999999999997e-05 2024-09-14T09:06:02.475 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.475 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.475 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.475 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:38100 3.b e48/28 ondisk, result = 0)", 2024-09-14T09:06:02.475 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.669267+0000", 2024-09-14T09:06:02.475 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.7080535919999997, 2024-09-14T09:06:02.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030186418999999999, 2024-09-14T09:06:02.475 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:02.475 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:02.475 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:02.475 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:02.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:02.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.669267+0000", 2024-09-14T09:06:02.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:02.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.669267+0000", 2024-09-14T09:06:02.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:02.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:02.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.669270+0000", 2024-09-14T09:06:02.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3269999999999999e-06 2024-09-14T09:06:02.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:02.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.669284+0000", 2024-09-14T09:06:02.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4662e-05 2024-09-14T09:06:02.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:02.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.669317+0000", 2024-09-14T09:06:02.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3080000000000002e-05 2024-09-14T09:06:02.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.669334+0000", 2024-09-14T09:06:02.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6376999999999999e-05 2024-09-14T09:06:02.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:02.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699411+0000", 2024-09-14T09:06:02.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030077653999999999 2024-09-14T09:06:02.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:02.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699429+0000", 2024-09-14T09:06:02.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7723e-05 2024-09-14T09:06:02.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:02.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:02.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:02.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699454+0000", 2024-09-14T09:06:02.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4596e-05 2024-09-14T09:06:02.478 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.478 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.479 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.479 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:02.479 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:02.479 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:02.480 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:06:02.481 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:02.480+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:02.581 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:02.583+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:02.614 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:02.614 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:02.614 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:02.614 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:02.628 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:06:02.683 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:02.684+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:02.784 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:02.780+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:02.793 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:02.793 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:02.793 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:02.794 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:02.794 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:02.807 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:06:02.849 INFO:teuthology.orchestra.run.smithi176.stdout:flattening image image_client.0-clone12 2024-09-14T09:06:02.884 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:02.887+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:02.938 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:02.938 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:02.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:02.938 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:02.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.938 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:654 3.11 e29/23 3:8c8e358b:::rbd_object_map.115e1e95bf99:head v 29'3, mlcod=28'1)", 2024-09-14T09:06:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.863095+0000", 2024-09-14T09:06:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.075543388, 2024-09-14T09:06:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0070911430000000003, 2024-09-14T09:06:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863095+0000", 2024-09-14T09:06:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863095+0000", 2024-09-14T09:06:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863096+0000", 2024-09-14T09:06:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8700000000000004e-07 2024-09-14T09:06:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863110+0000", 2024-09-14T09:06:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4283e-05 2024-09-14T09:06:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863157+0000", 2024-09-14T09:06:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6029999999999998e-05 2024-09-14T09:06:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863177+0000", 2024-09-14T09:06:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0662999999999999e-05 2024-09-14T09:06:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863324+0000", 2024-09-14T09:06:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014713899999999999 2024-09-14T09:06:02.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863383+0000", 2024-09-14T09:06:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8218999999999999e-05 2024-09-14T09:06:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.870083+0000", 2024-09-14T09:06:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067001459999999997 2024-09-14T09:06:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.870186+0000", 2024-09-14T09:06:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103776 2024-09-14T09:06:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.942 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:758 3.14 3:2d2c8a97:::rbd_data.115e724e1d99.0000000000000007:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:06:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.888858+0000", 2024-09-14T09:06:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.049780407, 2024-09-14T09:06:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060111740000000002, 2024-09-14T09:06:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.943 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 758 2024-09-14T09:06:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888858+0000", 2024-09-14T09:06:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888858+0000", 2024-09-14T09:06:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:02.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888860+0000", 2024-09-14T09:06:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4529999999999999e-06 2024-09-14T09:06:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888867+0000", 2024-09-14T09:06:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7200000000000006e-06 2024-09-14T09:06:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888892+0000", 2024-09-14T09:06:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4729999999999999e-05 2024-09-14T09:06:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888927+0000", 2024-09-14T09:06:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4752999999999997e-05 2024-09-14T09:06:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.894354+0000", 2024-09-14T09:06:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054273209999999997 2024-09-14T09:06:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.894584+0000", 2024-09-14T09:06:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000229602 2024-09-14T09:06:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.894869+0000", 2024-09-14T09:06:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028559499999999998 2024-09-14T09:06:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:829 3.13 e29/23 3:cc36a41f:::rbd_data.115e1e95bf99.0000000000000000:head v 29'1, mlcod=0'0)", 2024-09-14T09:06:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.901655+0000", 2024-09-14T09:06:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.036983521, 2024-09-14T09:06:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021644360000000001, 2024-09-14T09:06:02.947 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901655+0000", 2024-09-14T09:06:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901655+0000", 2024-09-14T09:06:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901657+0000", 2024-09-14T09:06:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.153e-06 2024-09-14T09:06:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901897+0000", 2024-09-14T09:06:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000239461 2024-09-14T09:06:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901927+0000", 2024-09-14T09:06:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.029e-05 2024-09-14T09:06:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901946+0000", 2024-09-14T09:06:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9170000000000001e-05 2024-09-14T09:06:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902104+0000", 2024-09-14T09:06:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015812199999999999 2024-09-14T09:06:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902157+0000", 2024-09-14T09:06:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2905999999999998e-05 2024-09-14T09:06:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923226+0000", 2024-09-14T09:06:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021068470999999998 2024-09-14T09:06:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923300+0000", 2024-09-14T09:06:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3787000000000004e-05 2024-09-14T09:06:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:845 3.9 3:928dad78:::rbd_data.115e1e95bf99.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13369344~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:06:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.922336+0000", 2024-09-14T09:06:02.951 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.016302604, 2024-09-14T09:06:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010311885999999999, 2024-09-14T09:06:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 845 2024-09-14T09:06:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922336+0000", 2024-09-14T09:06:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922336+0000", 2024-09-14T09:06:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922339+0000", 2024-09-14T09:06:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6879999999999999e-06 2024-09-14T09:06:02.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922350+0000", 2024-09-14T09:06:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1457999999999999e-05 2024-09-14T09:06:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922376+0000", 2024-09-14T09:06:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5301999999999999e-05 2024-09-14T09:06:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922390+0000", 2024-09-14T09:06:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4404e-05 2024-09-14T09:06:02.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922506+0000", 2024-09-14T09:06:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011567599999999999 2024-09-14T09:06:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922741+0000", 2024-09-14T09:06:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00023530600000000001 2024-09-14T09:06:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-09-14T09:06:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923197+0000", 2024-09-14T09:06:02.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00045574899999999998 2024-09-14T09:06:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926027+0000", 2024-09-14T09:06:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002830819 2024-09-14T09:06:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932530+0000", 2024-09-14T09:06:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065027139999999997 2024-09-14T09:06:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:02.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932583+0000", 2024-09-14T09:06:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2342000000000001e-05 2024-09-14T09:06:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932648+0000", 2024-09-14T09:06:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5427999999999998e-05 2024-09-14T09:06:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:842 3.b e29/28 3:d50f9298:::rbd_data.115e1e95bf99.0000000000000004:head v 29'5, mlcod=26'3)", 2024-09-14T09:06:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.922344+0000", 2024-09-14T09:06:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.016294854000002, 2024-09-14T09:06:02.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062661460000000002, 2024-09-14T09:06:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922344+0000", 2024-09-14T09:06:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922344+0000", 2024-09-14T09:06:02.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922345+0000", 2024-09-14T09:06:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.288e-06 2024-09-14T09:06:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922356+0000", 2024-09-14T09:06:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1048e-05 2024-09-14T09:06:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:02.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922371+0000", 2024-09-14T09:06:02.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4338000000000001e-05 2024-09-14T09:06:02.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922382+0000", 2024-09-14T09:06:02.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1e-05 2024-09-14T09:06:02.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:02.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922467+0000", 2024-09-14T09:06:02.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5721000000000001e-05 2024-09-14T09:06:02.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:02.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922489+0000", 2024-09-14T09:06:02.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2181999999999999e-05 2024-09-14T09:06:02.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:02.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.928551+0000", 2024-09-14T09:06:02.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060616100000000003 2024-09-14T09:06:02.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:02.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.928610+0000", 2024-09-14T09:06:02.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8959000000000003e-05 2024-09-14T09:06:02.961 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.961 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:02.962 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.962 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:848 3.1a 3:5ade8e73:::rbd_data.115e1e95bf99.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13369344~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:06:02.962 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.922902+0000", 2024-09-14T09:06:02.962 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.015737192, 2024-09-14T09:06:02.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096031020000000005, 2024-09-14T09:06:02.962 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:02.962 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:02.962 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:02.962 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:02.962 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:02.962 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.962 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 848 2024-09-14T09:06:02.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.963 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:02.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:02.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922902+0000", 2024-09-14T09:06:02.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:02.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922902+0000", 2024-09-14T09:06:02.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:02.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922903+0000", 2024-09-14T09:06:02.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6419999999999999e-06 2024-09-14T09:06:02.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:02.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923127+0000", 2024-09-14T09:06:02.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000223997 2024-09-14T09:06:02.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:02.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923151+0000", 2024-09-14T09:06:02.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3802e-05 2024-09-14T09:06:02.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923169+0000", 2024-09-14T09:06:02.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7858000000000001e-05 2024-09-14T09:06:02.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:02.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923307+0000", 2024-09-14T09:06:02.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013802399999999999 2024-09-14T09:06:02.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:02.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923560+0000", 2024-09-14T09:06:02.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00025296500000000001 2024-09-14T09:06:02.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:02.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923925+0000", 2024-09-14T09:06:02.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036534700000000002 2024-09-14T09:06:02.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:02.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.927231+0000", 2024-09-14T09:06:02.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033055820000000001 2024-09-14T09:06:02.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:02.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932287+0000", 2024-09-14T09:06:02.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0050562810000000001 2024-09-14T09:06:02.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:02.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932395+0000", 2024-09-14T09:06:02.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000108001 2024-09-14T09:06:02.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:02.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932505+0000", 2024-09-14T09:06:02.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109603 2024-09-14T09:06:02.967 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.967 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:02.968 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.968 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:847 3.13 e29/23 3:cc047b4a:::rbd_data.115e1e95bf99.0000000000000009:head v 29'2, mlcod=0'0)", 2024-09-14T09:06:02.968 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.924087+0000", 2024-09-14T09:06:02.968 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.014551430000001, 2024-09-14T09:06:02.968 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0085507929999999992, 2024-09-14T09:06:02.968 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:02.968 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:02.968 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:02.968 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:02.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.968 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:02.969 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924087+0000", 2024-09-14T09:06:02.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.969 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:02.969 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924087+0000", 2024-09-14T09:06:02.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.969 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:02.969 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924089+0000", 2024-09-14T09:06:02.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8050000000000001e-06 2024-09-14T09:06:02.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:02.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924157+0000", 2024-09-14T09:06:02.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7999999999999999e-05 2024-09-14T09:06:02.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:02.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924176+0000", 2024-09-14T09:06:02.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9312000000000001e-05 2024-09-14T09:06:02.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924191+0000", 2024-09-14T09:06:02.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4707e-05 2024-09-14T09:06:02.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:02.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924292+0000", 2024-09-14T09:06:02.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010052000000000001 2024-09-14T09:06:02.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:02.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924328+0000", 2024-09-14T09:06:02.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6211999999999998e-05 2024-09-14T09:06:02.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:02.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932564+0000", 2024-09-14T09:06:02.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0082365960000000005 2024-09-14T09:06:02.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:02.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932638+0000", 2024-09-14T09:06:02.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3640999999999995e-05 2024-09-14T09:06:02.972 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.972 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:02.972 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.972 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:849 3.b e29/28 3:d70d4fef:::rbd_data.115e1e95bf99.000000000000000b:head v 29'6, mlcod=26'3)", 2024-09-14T09:06:02.972 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.924304+0000", 2024-09-14T09:06:02.972 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.014334471000002, 2024-09-14T09:06:02.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083878719999999993, 2024-09-14T09:06:02.972 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:02.972 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:02.973 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:02.973 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:02.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:02.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924304+0000", 2024-09-14T09:06:02.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:02.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924304+0000", 2024-09-14T09:06:02.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:02.974 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924305+0000", 2024-09-14T09:06:02.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.192e-06 2024-09-14T09:06:02.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:02.974 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924676+0000", 2024-09-14T09:06:02.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037050700000000001 2024-09-14T09:06:02.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:02.974 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924698+0000", 2024-09-14T09:06:02.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1977999999999999e-05 2024-09-14T09:06:02.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924712+0000", 2024-09-14T09:06:02.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4477e-05 2024-09-14T09:06:02.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:02.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926069+0000", 2024-09-14T09:06:02.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013562330000000001 2024-09-14T09:06:02.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:02.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926105+0000", 2024-09-14T09:06:02.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6779999999999997e-05 2024-09-14T09:06:02.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:02.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932643+0000", 2024-09-14T09:06:02.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065378290000000002 2024-09-14T09:06:02.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:02.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932692+0000", 2024-09-14T09:06:02.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8875999999999998e-05 2024-09-14T09:06:02.976 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.976 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:02.976 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.977 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:850 3.11 e29/23 3:892642aa:::rbd_data.115e1e95bf99.000000000000000c:head v 29'18, mlcod=29'15)", 2024-09-14T09:06:02.977 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.926814+0000", 2024-09-14T09:06:02.977 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.011824598, 2024-09-14T09:06:02.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0064716569999999996, 2024-09-14T09:06:02.977 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:02.977 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:02.977 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:02.977 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:02.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:02.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926814+0000", 2024-09-14T09:06:02.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:02.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926814+0000", 2024-09-14T09:06:02.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:02.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926816+0000", 2024-09-14T09:06:02.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.491e-06 2024-09-14T09:06:02.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:02.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926828+0000", 2024-09-14T09:06:02.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2288e-05 2024-09-14T09:06:02.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:02.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926851+0000", 2024-09-14T09:06:02.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2674999999999999e-05 2024-09-14T09:06:02.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926865+0000", 2024-09-14T09:06:02.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.449e-05 2024-09-14T09:06:02.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:02.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932217+0000", 2024-09-14T09:06:02.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053523370000000004 2024-09-14T09:06:02.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:02.980 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932241+0000", 2024-09-14T09:06:02.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3210999999999999e-05 2024-09-14T09:06:02.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.980 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:02.980 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.933239+0000", 2024-09-14T09:06:02.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00099850599999999992 2024-09-14T09:06:02.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.980 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:02.980 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.933286+0000", 2024-09-14T09:06:02.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6659000000000002e-05 2024-09-14T09:06:02.980 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.980 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:02.980 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.981 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:9982 3.1a 3:5af3e476:::rbd_data.115eadec874.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:06:02.981 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:02.981 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 25.340669278, 2024-09-14T09:06:02.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042276341000000002, 2024-09-14T09:06:02.981 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:02.981 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:02.981 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:02.981 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:02.981 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:02.981 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.981 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 9982 2024-09-14T09:06:02.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.982 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:02.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:02.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:02.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:02.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:02.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:02.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597970+0000", 2024-09-14T09:06:02.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0599999999999999e-07 2024-09-14T09:06:02.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:02.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597974+0000", 2024-09-14T09:06:02.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2649999999999998e-06 2024-09-14T09:06:02.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:02.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597986+0000", 2024-09-14T09:06:02.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1019000000000001e-05 2024-09-14T09:06:02.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597991+0000", 2024-09-14T09:06:02.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5049999999999998e-06 2024-09-14T09:06:02.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:02.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.598083+0000", 2024-09-14T09:06:02.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2062e-05 2024-09-14T09:06:02.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:02.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.598227+0000", 2024-09-14T09:06:02.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000143609 2024-09-14T09:06:02.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:02.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.603107+0000", 2024-09-14T09:06:02.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048805819999999996 2024-09-14T09:06:02.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:02.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640002+0000", 2024-09-14T09:06:02.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.036894641999999998 2024-09-14T09:06:02.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:02.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640077+0000", 2024-09-14T09:06:02.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4927999999999998e-05 2024-09-14T09:06:02.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:02.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640174+0000", 2024-09-14T09:06:02.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7417999999999995e-05 2024-09-14T09:06:02.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:02.986 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640246+0000", 2024-09-14T09:06:02.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1505000000000001e-05 2024-09-14T09:06:02.986 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.986 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:02.986 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.986 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:9982 3.1a e32/23 ondisk, result = 0)", 2024-09-14T09:06:02.986 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.609682+0000", 2024-09-14T09:06:02.986 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 25.328956669, 2024-09-14T09:06:02.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.030575755, 2024-09-14T09:06:02.986 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:02.987 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:02.987 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:02.987 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:02.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:02.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609682+0000", 2024-09-14T09:06:02.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:02.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609682+0000", 2024-09-14T09:06:02.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:02.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609683+0000", 2024-09-14T09:06:02.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0399999999999995e-07 2024-09-14T09:06:02.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:02.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609687+0000", 2024-09-14T09:06:02.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3460000000000004e-06 2024-09-14T09:06:02.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:02.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609696+0000", 2024-09-14T09:06:02.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0550000000000005e-06 2024-09-14T09:06:02.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609702+0000", 2024-09-14T09:06:02.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.626e-06 2024-09-14T09:06:02.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:02.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640050+0000", 2024-09-14T09:06:02.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.030348671000000001 2024-09-14T09:06:02.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:02.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640066+0000", 2024-09-14T09:06:02.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5617999999999999e-05 2024-09-14T09:06:02.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:02.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640258+0000", 2024-09-14T09:06:02.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019173500000000001 2024-09-14T09:06:02.990 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.990 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:02.990 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.990 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:10031 3.1b 3:df00a663:::rbd_data.115eadec874.0000000000000005:head [stat,write 16619424~53229] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:06:02.990 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.768778+0000", 2024-09-14T09:06:02.990 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 25.169861158, 2024-09-14T09:06:02.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0078922009999999997, 2024-09-14T09:06:02.990 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:02.990 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:02.990 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:02.990 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:02.990 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:02.991 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:02.991 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 10031 2024-09-14T09:06:02.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.991 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:02.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:02.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768778+0000", 2024-09-14T09:06:02.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:02.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768778+0000", 2024-09-14T09:06:02.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:02.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768780+0000", 2024-09-14T09:06:02.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7130000000000001e-06 2024-09-14T09:06:02.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:02.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768824+0000", 2024-09-14T09:06:02.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3989999999999997e-05 2024-09-14T09:06:02.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:02.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768854+0000", 2024-09-14T09:06:02.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9937000000000002e-05 2024-09-14T09:06:02.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768869+0000", 2024-09-14T09:06:02.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5233e-05 2024-09-14T09:06:02.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:02.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768979+0000", 2024-09-14T09:06:02.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109429 2024-09-14T09:06:02.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:02.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.769124+0000", 2024-09-14T09:06:02.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014520200000000001 2024-09-14T09:06:02.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:02.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.769351+0000", 2024-09-14T09:06:02.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000227259 2024-09-14T09:06:02.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:02.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.776133+0000", 2024-09-14T09:06:02.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067818979999999997 2024-09-14T09:06:02.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:02.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.776464+0000", 2024-09-14T09:06:02.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033063400000000001 2024-09-14T09:06:02.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:02.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.776553+0000", 2024-09-14T09:06:02.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8793000000000002e-05 2024-09-14T09:06:02.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:02.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.776670+0000", 2024-09-14T09:06:02.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000117113 2024-09-14T09:06:02.995 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.995 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:02.995 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.996 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:10039 3.f e33/23 3:f6f4e415:::rbd_data.115eadec874.0000000000000000:head v 33'8, mlcod=32'6)", 2024-09-14T09:06:02.996 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.951357+0000", 2024-09-14T09:06:02.996 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.987282016000002, 2024-09-14T09:06:02.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067664509999999997, 2024-09-14T09:06:02.996 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:02.996 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:02.996 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:02.996 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:02.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:02.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951357+0000", 2024-09-14T09:06:02.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:02.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951357+0000", 2024-09-14T09:06:02.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:02.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:02.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951357+0000", 2024-09-14T09:06:02.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4000000000000001e-07 2024-09-14T09:06:02.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:02.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951363+0000", 2024-09-14T09:06:02.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4999999999999999e-06 2024-09-14T09:06:02.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:02.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951376+0000", 2024-09-14T09:06:02.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3315e-05 2024-09-14T09:06:02.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:02.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951384+0000", 2024-09-14T09:06:02.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6590000000000007e-06 2024-09-14T09:06:02.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:02.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951470+0000", 2024-09-14T09:06:02.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6198000000000004e-05 2024-09-14T09:06:02.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:02.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951493+0000", 2024-09-14T09:06:02.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2668999999999999e-05 2024-09-14T09:06:02.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:02.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.958059+0000", 2024-09-14T09:06:02.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065659450000000001 2024-09-14T09:06:02.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:02.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:02.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:02.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.958123+0000", 2024-09-14T09:06:02.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4524999999999996e-05 2024-09-14T09:06:02.999 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:02.999 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:02.999 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:03.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.000 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:10040 3.a e33/23 3:50279329:::rbd_data.115eadec874.0000000000000001:head v 33'16, mlcod=32'14)", 2024-09-14T09:06:03.000 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.951570+0000", 2024-09-14T09:06:03.000 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.987068700999998, 2024-09-14T09:06:03.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0094331320000000003, 2024-09-14T09:06:03.000 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:03.000 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:03.000 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:03.000 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:03.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:03.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951570+0000", 2024-09-14T09:06:03.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:03.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:03.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951570+0000", 2024-09-14T09:06:03.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:03.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:03.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951571+0000", 2024-09-14T09:06:03.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0800000000000004e-07 2024-09-14T09:06:03.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:03.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951578+0000", 2024-09-14T09:06:03.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4919999999999996e-06 2024-09-14T09:06:03.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:03.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951590+0000", 2024-09-14T09:06:03.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1674e-05 2024-09-14T09:06:03.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:03.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951598+0000", 2024-09-14T09:06:03.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4649999999999994e-06 2024-09-14T09:06:03.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:03.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951800+0000", 2024-09-14T09:06:03.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020128600000000001 2024-09-14T09:06:03.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:03.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951832+0000", 2024-09-14T09:06:03.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2512999999999999e-05 2024-09-14T09:06:03.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:03.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.960831+0000", 2024-09-14T09:06:03.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00899872 2024-09-14T09:06:03.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:03.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.961003+0000", 2024-09-14T09:06:03.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017217400000000001 2024-09-14T09:06:03.003 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:03.003 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:03.004 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:03.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.004 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:10121 3.a e33/23 3:50279329:::rbd_data.115eadec874.0000000000000001:head v 33'18, mlcod=33'16)", 2024-09-14T09:06:03.004 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:38.059543+0000", 2024-09-14T09:06:03.004 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.879095586999998, 2024-09-14T09:06:03.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063099250000000001, 2024-09-14T09:06:03.004 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:03.004 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:03.004 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:03.004 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:03.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:03.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059543+0000", 2024-09-14T09:06:03.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:03.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:03.005 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059543+0000", 2024-09-14T09:06:03.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:03.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:03.005 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059545+0000", 2024-09-14T09:06:03.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.863e-06 2024-09-14T09:06:03.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:03.005 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059609+0000", 2024-09-14T09:06:03.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3758999999999997e-05 2024-09-14T09:06:03.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:03.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059633+0000", 2024-09-14T09:06:03.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4000000000000001e-05 2024-09-14T09:06:03.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:03.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059650+0000", 2024-09-14T09:06:03.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7130999999999999e-05 2024-09-14T09:06:03.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:03.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059779+0000", 2024-09-14T09:06:03.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000129152 2024-09-14T09:06:03.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:03.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059840+0000", 2024-09-14T09:06:03.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0930000000000001e-05 2024-09-14T09:06:03.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:03.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.065745+0000", 2024-09-14T09:06:03.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059050309999999998 2024-09-14T09:06:03.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:03.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.065853+0000", 2024-09-14T09:06:03.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000108059 2024-09-14T09:06:03.008 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:03.008 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:03.008 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:03.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.008 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:10182 3.1a 3:5a62ac37:::rbd_data.115eadec874.000000000000000d:head [set-alloc-hint object_size 33554432 write_size 33554432,write 5072829~48248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-14T09:06:03.008 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:38.177016+0000", 2024-09-14T09:06:03.008 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.761622652, 2024-09-14T09:06:03.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006987147, 2024-09-14T09:06:03.008 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:03.008 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:03.008 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:03.009 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:03.009 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:03.009 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:03.009 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 10182 2024-09-14T09:06:03.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.009 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:03.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:03.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177016+0000", 2024-09-14T09:06:03.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:03.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:03.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177016+0000", 2024-09-14T09:06:03.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:03.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:03.010 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177017+0000", 2024-09-14T09:06:03.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0440000000000001e-06 2024-09-14T09:06:03.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:03.010 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177065+0000", 2024-09-14T09:06:03.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7902999999999999e-05 2024-09-14T09:06:03.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:03.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177080+0000", 2024-09-14T09:06:03.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4667999999999999e-05 2024-09-14T09:06:03.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:03.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177087+0000", 2024-09-14T09:06:03.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0430000000000002e-06 2024-09-14T09:06:03.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:03.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177140+0000", 2024-09-14T09:06:03.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2933999999999998e-05 2024-09-14T09:06:03.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:03.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177199+0000", 2024-09-14T09:06:03.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9799999999999997e-05 2024-09-14T09:06:03.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:03.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177309+0000", 2024-09-14T09:06:03.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010948200000000001 2024-09-14T09:06:03.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:03.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.180906+0000", 2024-09-14T09:06:03.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0035966230000000002 2024-09-14T09:06:03.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:03.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.183932+0000", 2024-09-14T09:06:03.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030259760000000001 2024-09-14T09:06:03.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:03.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.183973+0000", 2024-09-14T09:06:03.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1189000000000002e-05 2024-09-14T09:06:03.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:03.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.184003+0000", 2024-09-14T09:06:03.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0485e-05 2024-09-14T09:06:03.014 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:03.014 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:03.014 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:03.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.014 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:20954 3.1d e41/23 3:bd7e9002:::rbd_data.115e889d4809.0000000000000005:head v 41'131, mlcod=41'127)", 2024-09-14T09:06:03.014 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:47.701009+0000", 2024-09-14T09:06:03.014 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.237629292999999, 2024-09-14T09:06:03.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0080632259999999997, 2024-09-14T09:06:03.015 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:03.015 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:03.015 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:03.015 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:03.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:03.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701009+0000", 2024-09-14T09:06:03.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:03.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:03.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701009+0000", 2024-09-14T09:06:03.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:03.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:03.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701011+0000", 2024-09-14T09:06:03.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.703e-06 2024-09-14T09:06:03.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:03.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701234+0000", 2024-09-14T09:06:03.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022278900000000001 2024-09-14T09:06:03.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:03.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701265+0000", 2024-09-14T09:06:03.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1386000000000001e-05 2024-09-14T09:06:03.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:03.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701281+0000", 2024-09-14T09:06:03.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5460000000000001e-05 2024-09-14T09:06:03.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:03.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701391+0000", 2024-09-14T09:06:03.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000110622 2024-09-14T09:06:03.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:03.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701428+0000", 2024-09-14T09:06:03.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6986999999999997e-05 2024-09-14T09:06:03.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:03.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.708992+0000", 2024-09-14T09:06:03.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075640769999999998 2024-09-14T09:06:03.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:03.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.709073+0000", 2024-09-14T09:06:03.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0202000000000002e-05 2024-09-14T09:06:03.019 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:03.019 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:03.019 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:03.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.019 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:21183 3.1d e42/23 3:bba2b35c:::rbd_object_map.115e889d4809:head v 42'156, mlcod=42'149)", 2024-09-14T09:06:03.019 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:48.419596+0000", 2024-09-14T09:06:03.019 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.519043018, 2024-09-14T09:06:03.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063479840000000001, 2024-09-14T09:06:03.019 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:03.019 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:03.020 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:03.020 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:03.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:03.020 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419596+0000", 2024-09-14T09:06:03.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:03.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:03.020 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419596+0000", 2024-09-14T09:06:03.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:03.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:03.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419598+0000", 2024-09-14T09:06:03.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8560000000000001e-06 2024-09-14T09:06:03.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:03.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419610+0000", 2024-09-14T09:06:03.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2577e-05 2024-09-14T09:06:03.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:03.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419645+0000", 2024-09-14T09:06:03.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4873000000000003e-05 2024-09-14T09:06:03.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:03.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419664+0000", 2024-09-14T09:06:03.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9485999999999999e-05 2024-09-14T09:06:03.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:03.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419799+0000", 2024-09-14T09:06:03.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013482300000000001 2024-09-14T09:06:03.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:03.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419853+0000", 2024-09-14T09:06:03.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3439000000000001e-05 2024-09-14T09:06:03.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:03.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.425843+0000", 2024-09-14T09:06:03.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059898809999999999 2024-09-14T09:06:03.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:03.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.425944+0000", 2024-09-14T09:06:03.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101049 2024-09-14T09:06:03.024 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:03.024 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:03.024 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:03.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.024 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(3.1b 53/52 [PushOp(3:df00a663:::rbd_data.115eadec874.0000000000000005:6, version: 35'41, data_included: [1796079~63600], data_size: 63600, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:df00a663:::rbd_data.115eadec874.0000000000000005:6@35'41, size: 1859679, copy_subset: [(0, 1859679)], clone_subset: {}, snapset: 6=[]:{6: [6]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1859679, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T09:06:03.025 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.846807+0000", 2024-09-14T09:06:03.025 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.0918320399999999, 2024-09-14T09:06:03.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0089637359999999999, 2024-09-14T09:06:03.025 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:03.025 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:03.025 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:03.025 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:03.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:03.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.846807+0000", 2024-09-14T09:06:03.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:03.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:03.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.846807+0000", 2024-09-14T09:06:03.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:03.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:03.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.846809+0000", 2024-09-14T09:06:03.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5799999999999999e-06 2024-09-14T09:06:03.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:03.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.846948+0000", 2024-09-14T09:06:03.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013911500000000001 2024-09-14T09:06:03.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:03.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.847007+0000", 2024-09-14T09:06:03.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8437000000000003e-05 2024-09-14T09:06:03.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:03.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.847054+0000", 2024-09-14T09:06:03.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7372e-05 2024-09-14T09:06:03.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:03.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.847295+0000", 2024-09-14T09:06:03.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000241022 2024-09-14T09:06:03.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:03.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.847319+0000", 2024-09-14T09:06:03.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4303000000000001e-05 2024-09-14T09:06:03.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:03.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.855770+0000", 2024-09-14T09:06:03.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084509070000000006 2024-09-14T09:06:03.028 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:03.028 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:03.028 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:03.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.029 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(3.1b 53/52 [PushOp(3:d8e0f8d4:::rbd_data.115eb07a8abd.0000000000000001:head, version: 48'138, data_included: [0~393216,786432~131072], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:d8e0f8d4:::rbd_data.115eb07a8abd.0000000000000001:head@48'138, size: 7208960, copy_subset: [(0, 7208960)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 7208960, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T09:06:03.029 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.943323+0000", 2024-09-14T09:06:03.029 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.99531602500000005, 2024-09-14T09:06:03.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0081121930000000002, 2024-09-14T09:06:03.029 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:03.029 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:03.029 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:03.029 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:03.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:03.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.943323+0000", 2024-09-14T09:06:03.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:03.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:03.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.943323+0000", 2024-09-14T09:06:03.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:03.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:03.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.943324+0000", 2024-09-14T09:06:03.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.206e-06 2024-09-14T09:06:03.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:03.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944255+0000", 2024-09-14T09:06:03.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093096900000000002 2024-09-14T09:06:03.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:03.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944299+0000", 2024-09-14T09:06:03.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4332999999999997e-05 2024-09-14T09:06:03.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:03.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944332+0000", 2024-09-14T09:06:03.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2329999999999997e-05 2024-09-14T09:06:03.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:03.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944590+0000", 2024-09-14T09:06:03.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00025868100000000003 2024-09-14T09:06:03.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:03.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944629+0000", 2024-09-14T09:06:03.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8503e-05 2024-09-14T09:06:03.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:03.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:03.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:03.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.951435+0000", 2024-09-14T09:06:03.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068061709999999997 2024-09-14T09:06:03.032 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:03.033 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:03.033 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:03.033 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:03.033 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:03.033 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:03.034 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:06:03.035 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:03.035+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:03.045 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:06:03.135 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:03.132+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:03.198 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:03.198 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:03.204 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:03.204 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:03.205 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:03.205 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:03.214 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:03.214 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:03.219 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:06:03.230 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:03.230 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:03.248 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:03.249 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:03.264 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:03.267+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:03.266 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:03.266 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:03.284 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:03.285 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:03.304 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:03.304 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:03.324 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:03.325 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:03.337 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:03.332+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:03.341 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:03.341 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:03.355 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:06:03.438 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:03.436+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:03.517 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:03.518 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:03.532 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:06:03.539 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:03.540+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:03.639 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:03.640+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:03.657 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:03.657 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:03.671 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:06:03.740 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:03.743+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:03.796 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:03.796 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:03.810 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:06:03.838 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T09:06:03.842 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:03.836+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:03.941 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:03.943+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:03.975 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:03.975 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:03.989 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:06:04.040 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-14T09:06:04.076 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:04.079+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:04.143 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:04.140+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:04.159 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:04.160 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:04.173 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:06:04.211 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:04.211 INFO:teuthology.orchestra.run.smithi176.stdout:1506 writesame 0xca922ac thru 0xca9fd10 data_size 0x31(0xda65 bytes) 2024-09-14T09:06:04.229 INFO:teuthology.orchestra.run.smithi176.stdout:1508 trunc from 0xe2a97dd to 0xd08290f 2024-09-14T09:06:04.244 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:04.240+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:04.247 INFO:teuthology.orchestra.run.smithi176.stdout:1509 writesame 0x58403ac thru 0x58437d1 data_size 0x32(0x3426 bytes) 2024-09-14T09:06:04.254 INFO:teuthology.orchestra.run.smithi176.stdout:1510 read 0x53256ed thru 0x53276ac (0x1fc0 bytes) 2024-09-14T09:06:04.254 INFO:teuthology.orchestra.run.smithi176.stdout:1511 punch from 0x9ceab03 to 0x9cf14bb, (0x69b8 bytes) 2024-09-14T09:06:04.274 INFO:teuthology.orchestra.run.smithi176.stdout:1513 trunc from 0xd08290f to 0x207da42 2024-09-14T09:06:04.344 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:04.340+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:04.356 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:04.356 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:04.370 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:06:04.376 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:04.404 INFO:teuthology.orchestra.run.smithi176.stdout:1516 write 0x9c18730 thru 0x9c1e259 (0x5b2a bytes) 2024-09-14T09:06:04.409 INFO:teuthology.orchestra.run.smithi176.stdout:1517 writesame 0x77a4dce thru 0x77ac4f3 data_size 0x65(0x7726 bytes) 2024-09-14T09:06:04.435 INFO:teuthology.orchestra.run.smithi176.stdout:1518 write 0xd433c8a thru 0xd443416 (0xf78d bytes) 2024-09-14T09:06:04.441 INFO:teuthology.orchestra.run.smithi176.stdout:1519 writesame 0xbeaaba7 thru 0xbead010 data_size 0x3b(0x246a bytes) 2024-09-14T09:06:04.445 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:04.447+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:04.447 INFO:teuthology.orchestra.run.smithi176.stdout:1520 read 0x8c6aaab thru 0x8c74da3 (0xa2f9 bytes) 2024-09-14T09:06:04.447 INFO:teuthology.orchestra.run.smithi176.stdout:1522 writesame 0x3b5903e thru 0x3b5e149 data_size 0x39(0x510c bytes) 2024-09-14T09:06:04.450 INFO:teuthology.orchestra.run.smithi176.stdout:1524 read 0x6249677 thru 0x62533b9 (0x9d43 bytes) 2024-09-14T09:06:04.450 INFO:teuthology.orchestra.run.smithi176.stdout:1525 write 0x2b8f3f6 thru 0x2b9088a (0x1495 bytes) 2024-09-14T09:06:04.453 INFO:teuthology.orchestra.run.smithi176.stdout:1526 write 0x9c7c054 thru 0x9c88ccb (0xcc78 bytes) 2024-09-14T09:06:04.473 INFO:teuthology.orchestra.run.smithi176.stdout:1527 trunc from 0xd443417 to 0x6b95f02 2024-09-14T09:06:04.493 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:04.493 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:04.496 INFO:teuthology.orchestra.run.smithi176.stdout:1528 read 0x6435af7 thru 0x6436beb (0x10f5 bytes) 2024-09-14T09:06:04.507 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:06:04.511 INFO:teuthology.orchestra.run.smithi176.stdout:1530 trunc from 0x6b95f02 to 0xda14235 2024-09-14T09:06:04.522 INFO:teuthology.orchestra.run.smithi176.stdout:1531 write 0x864ecf8 thru 0x865afb0 (0xc2b9 bytes) 2024-09-14T09:06:04.530 INFO:teuthology.orchestra.run.smithi176.stdout:1532 read 0x45fa68d thru 0x4606481 (0xbdf5 bytes) 2024-09-14T09:06:04.530 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:04.530 INFO:teuthology.orchestra.run.smithi176.stdout:1535 write 0x7a526bc thru 0x7a560b5 (0x39fa bytes) 2024-09-14T09:06:04.536 INFO:teuthology.orchestra.run.smithi176.stdout:1537 write 0x697769 thru 0x699d55 (0x25ed bytes) 2024-09-14T09:06:04.538 INFO:teuthology.orchestra.run.smithi176.stdout:1538 read 0x7690110 thru 0x7695e9e (0x5d8f bytes) 2024-09-14T09:06:04.539 INFO:teuthology.orchestra.run.smithi176.stdout:1539 writesame 0x327968e thru 0x3283e84 data_size 0xb(0xa7f7 bytes) 2024-09-14T09:06:04.546 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:04.544+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:04.547 INFO:teuthology.orchestra.run.smithi176.stdout:1541 read 0x8a1c657 thru 0x8a2b275 (0xec1f bytes) 2024-09-14T09:06:04.547 INFO:teuthology.orchestra.run.smithi176.stdout:1542 read 0x6358297 thru 0x635bc72 (0x39dc bytes) 2024-09-14T09:06:04.548 INFO:teuthology.orchestra.run.smithi176.stdout:1543 write 0x8e3a4fc thru 0x8e3b594 (0x1099 bytes) 2024-09-14T09:06:04.550 INFO:teuthology.orchestra.run.smithi176.stdout:1544 writesame 0x670b8cf thru 0x67180b3 data_size 0x1(0xc7e5 bytes) 2024-09-14T09:06:04.647 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:04.647 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:04.647 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:04.644+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:04.659 INFO:teuthology.orchestra.run.smithi176.stdout:1546 write 0x9838e39 thru 0x9844ae4 (0xbcac bytes) 2024-09-14T09:06:04.663 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:06:04.672 INFO:teuthology.orchestra.run.smithi176.stdout:1547 read 0x4301f85 thru 0x430e71f (0xc79b bytes) 2024-09-14T09:06:04.674 INFO:teuthology.orchestra.run.smithi176.stdout:1548 writesame 0xa23fb30 thru 0xa2475d8 data_size 0x9(0x7aa9 bytes) 2024-09-14T09:06:04.690 INFO:teuthology.orchestra.run.smithi176.stdout:1549 write 0x49e2664 thru 0x49e6685 (0x4022 bytes) 2024-09-14T09:06:04.697 INFO:teuthology.orchestra.run.smithi176.stdout:1551 writesame 0x143f047 thru 0x1448379 data_size 0x35(0x9333 bytes) 2024-09-14T09:06:04.700 INFO:teuthology.orchestra.run.smithi176.stdout:1552 write 0x6454794 thru 0x645cbc5 (0x8432 bytes) 2024-09-14T09:06:04.707 INFO:teuthology.orchestra.run.smithi176.stdout:1553 clone 14 order 25 su 2097152 sc 5 2024-09-14T09:06:04.749 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:04.751+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:04.792 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:04.792 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:04.806 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:06:04.850 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:04.848+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:04.950 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:04.951+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:04.974 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:04.974 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:04.988 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:06:05.052 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:05.048+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:05.071 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is set 2024-09-14T09:06:05.121 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:05.124 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:05.136 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:06:05.180 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone13 from 0xda14235 (overlap 0x207da42) to 0x9bf4ad 2024-09-14T09:06:05.181 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:05.176+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:05.253 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:05.255+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:05.289 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-14T09:06:05.312 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:05.313 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:05.333 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:06:05.343 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:06:05.354 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:05.355+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:05.455 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:05.455+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:05.459 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:06:05.459 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:06:05.459 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:06:05.466 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:05.466 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:05.485 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:06:05.547 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:05.547 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:05.555 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:05.555+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:05.564 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:05.564 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:05.583 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:05.584 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:05.610 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:05.610 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:05.645 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:05.645 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:05.654 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:05.654 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:05.654 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:05.654 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:05.656 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:05.652+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:05.670 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:06:05.680 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:05.681 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:05.716 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:05.716 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:05.727 INFO:teuthology.orchestra.run.smithi176.stdout:1554 trunc from 0xda14235 to 0xf97859 2024-09-14T09:06:05.740 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:05.741 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:05.757 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:05.759+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:05.822 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:05.822 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:05.822 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:05.822 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:05.823 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:05.823 INFO:teuthology.orchestra.run.smithi176.stdout:1556 writesame 0x870d466 thru 0x87133c9 data_size 0x4a(0x5f64 bytes) 2024-09-14T09:06:05.823 INFO:teuthology.orchestra.run.smithi176.stdout:1557 read 0x68a3aca thru 0x68a67bf (0x2cf6 bytes) 2024-09-14T09:06:05.824 INFO:teuthology.orchestra.run.smithi176.stdout:1558 read 0x7677f10 thru 0x7685bf4 (0xdce5 bytes) 2024-09-14T09:06:05.825 INFO:teuthology.orchestra.run.smithi176.stdout:1560 write 0x1a1e2e2 thru 0x1a25200 (0x6f1f bytes) 2024-09-14T09:06:05.834 INFO:teuthology.orchestra.run.smithi176.stdout:1561 trunc from 0x87133ca to 0x5193f83 2024-09-14T09:06:05.841 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:06:05.846 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:06:05.847 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:06:05.858 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:05.859+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:05.865 INFO:teuthology.orchestra.run.smithi176.stdout:1563 writesame 0x879c64c thru 0x87a79ae data_size 0x1(0xb363 bytes) 2024-09-14T09:06:05.867 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:05.867 INFO:teuthology.orchestra.run.smithi176.stdout:1566 write 0x1af214b thru 0x1afd366 (0xb21c bytes) 2024-09-14T09:06:05.869 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-09-14T09:06:05.078308+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":522224977,"num_objects":502,"num_object_clones":149,"num_object_copies":1004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":76,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":502,"num_whiteouts":183,"num_read":51890,"num_read_kb":225453,"num_write":6074,"num_write_kb":99599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":10100679,"num_keys_recovered":20,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4755,"ondisk_log_size":4755,"up":82,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283920,"kb_used_data":17752,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754690800,"statfs":{"total":773094113280,"available":772803379200,"internally_reserved":0,"allocated":18178048,"data_stored":88846272,"data_compressed":3944420,"data_compressed_allocated":8970240,"data_compressed_original":84190480,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":78765689,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":76,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":2033,"num_read_kb":7858,"num_write":587,"num_write_kb":7659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.025896"},"pg_stats":[{"pgid":"3.1f","version":"53'465","reported_seq":2724,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.840323+0000","last_change":"2024-09-14T09:05:33.869426+0000","last_active":"2024-09-14T09:06:01.840323+0000","last_peered":"2024-09-14T09:06:01.840323+0000","last_clean":"2024-09-14T09:06:01.840323+0000","last_became_active":"2024-09-14T09:05:33.854933+0000","last_became_peered":"2024-09-14T09:05:33.854933+0000","last_unstale":"2024-09-14T09:06:01.840323+0000","last_undegraded":"2024-09-14T09:06:01.840323+0000","last_fullsized":"2024-09-14T09:06:01.840323+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:47:16.335423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001387512,"stat_sum":{"num_bytes":1572987,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":4517,"num_read_kb":1956,"num_write":823,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"48'77","reported_seq":1715,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.052953+0000","last_change":"2024-09-14T09:05:29.039503+0000","last_active":"2024-09-14T09:06:01.052953+0000","last_peered":"2024-09-14T09:06:01.052953+0000","last_clean":"2024-09-14T09:06:01.052953+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:06:01.052953+0000","last_undegraded":"2024-09-14T09:06:01.052953+0000","last_fullsized":"2024-09-14T09:06:01.052953+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:34:51.429716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001657176,"stat_sum":{"num_bytes":14608185,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":1584,"num_read_kb":15898,"num_write":103,"num_write_kb":1986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"49'259","reported_seq":1994,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.840016+0000","last_change":"2024-09-14T09:05:28.835298+0000","last_active":"2024-09-14T09:06:01.840016+0000","last_peered":"2024-09-14T09:06:01.840016+0000","last_clean":"2024-09-14T09:06:01.840016+0000","last_became_active":"2024-09-14T09:05:26.835372+0000","last_became_peered":"2024-09-14T09:05:26.835372+0000","last_unstale":"2024-09-14T09:06:01.840016+0000","last_undegraded":"2024-09-14T09:06:01.840016+0000","last_fullsized":"2024-09-14T09:06:01.840016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:47:30.199570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014149329999999999,"stat_sum":{"num_bytes":46441633,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":2406,"num_read_kb":38922,"num_write":409,"num_write_kb":3615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"53'157","reported_seq":1118,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.839999+0000","last_change":"2024-09-14T09:05:39.931845+0000","last_active":"2024-09-14T09:06:01.839999+0000","last_peered":"2024-09-14T09:06:01.839999+0000","last_clean":"2024-09-14T09:06:01.839999+0000","last_became_active":"2024-09-14T09:05:39.918593+0000","last_became_peered":"2024-09-14T09:05:39.918593+0000","last_unstale":"2024-09-14T09:06:01.839999+0000","last_undegraded":"2024-09-14T09:06:01.839999+0000","last_fullsized":"2024-09-14T09:06:01.839999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:52.136931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014661419999999999,"stat_sum":{"num_bytes":13419190,"num_objects":16,"num_object_clones":6,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":1937,"num_read_kb":2742,"num_write":170,"num_write_kb":1435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":17,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"48'138","reported_seq":566,"reported_epoch":52,"state":"remapped+peering","last_fresh":"2024-09-14T09:06:01.054168+0000","last_change":"2024-09-14T09:06:01.054168+0000","last_active":"2024-09-14T09:06:01.054168+0000","last_peered":"2024-09-14T09:05:57.843632+0000","last_clean":"2024-09-14T09:05:57.843632+0000","last_became_active":"2024-09-14T09:05:39.922373+0000","last_became_peered":"2024-09-14T09:05:39.922373+0000","last_unstale":"2024-09-14T09:06:01.054168+0000","last_undegraded":"2024-09-14T09:06:01.054168+0000","last_fullsized":"2024-09-14T09:06:01.054168+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060373399999999998,"stat_sum":{"num_bytes":14703378,"num_objects":18,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":375,"num_read_kb":505,"num_write":142,"num_write_kb":4242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":114614,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"48'119","reported_seq":2122,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.053733+0000","last_change":"2024-09-14T09:05:39.919093+0000","last_active":"2024-09-14T09:06:01.053733+0000","last_peered":"2024-09-14T09:06:01.053733+0000","last_clean":"2024-09-14T09:06:01.053733+0000","last_became_active":"2024-09-14T09:05:39.905126+0000","last_became_peered":"2024-09-14T09:05:39.905126+0000","last_unstale":"2024-09-14T09:06:01.053733+0000","last_undegraded":"2024-09-14T09:06:01.053733+0000","last_fullsized":"2024-09-14T09:06:01.053733+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:12:55.458008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020845550000000001,"stat_sum":{"num_bytes":22979522,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":1964,"num_read_kb":8792,"num_write":136,"num_write_kb":4028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":92669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"49'110","reported_seq":352,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.053577+0000","last_change":"2024-09-14T09:05:28.822565+0000","last_active":"2024-09-14T09:06:01.053577+0000","last_peered":"2024-09-14T09:06:01.053577+0000","last_clean":"2024-09-14T09:06:01.053577+0000","last_became_active":"2024-09-14T09:05:26.816132+0000","last_became_peered":"2024-09-14T09:05:26.816132+0000","last_unstale":"2024-09-14T09:06:01.053577+0000","last_undegraded":"2024-09-14T09:06:01.053577+0000","last_fullsized":"2024-09-14T09:06:01.053577+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:07:23.078036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092420299999999996,"stat_sum":{"num_bytes":8162452,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":206,"num_read_kb":1068,"num_write":149,"num_write_kb":1947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"49'167","reported_seq":1231,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.052542+0000","last_change":"2024-09-14T09:05:33.865470+0000","last_active":"2024-09-14T09:06:01.052542+0000","last_peered":"2024-09-14T09:06:01.052542+0000","last_clean":"2024-09-14T09:06:01.052542+0000","last_became_active":"2024-09-14T09:05:33.846990+0000","last_became_peered":"2024-09-14T09:05:33.846990+0000","last_unstale":"2024-09-14T09:06:01.052542+0000","last_undegraded":"2024-09-14T09:06:01.052542+0000","last_fullsized":"2024-09-14T09:06:01.052542+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:25:26.082467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081579899999999997,"stat_sum":{"num_bytes":32084671,"num_objects":28,"num_object_clones":11,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":10,"num_read":1827,"num_read_kb":3479,"num_write":161,"num_write_kb":928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":70784,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"49'113","reported_seq":1377,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.828645+0000","last_change":"2024-09-14T09:05:28.839928+0000","last_active":"2024-09-14T09:06:01.828645+0000","last_peered":"2024-09-14T09:06:01.828645+0000","last_clean":"2024-09-14T09:06:01.828645+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:06:01.828645+0000","last_undegraded":"2024-09-14T09:06:01.828645+0000","last_fullsized":"2024-09-14T09:06:01.828645+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:37:36.804320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014343089999999999,"stat_sum":{"num_bytes":10702342,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":1633,"num_read_kb":2794,"num_write":108,"num_write_kb":3404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"48'221","reported_seq":821,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.840196+0000","last_change":"2024-09-14T09:05:28.836370+0000","last_active":"2024-09-14T09:06:01.840196+0000","last_peered":"2024-09-14T09:06:01.840196+0000","last_clean":"2024-09-14T09:06:01.840196+0000","last_became_active":"2024-09-14T09:05:26.836744+0000","last_became_peered":"2024-09-14T09:05:26.836744+0000","last_unstale":"2024-09-14T09:06:01.840196+0000","last_undegraded":"2024-09-14T09:06:01.840196+0000","last_fullsized":"2024-09-14T09:06:01.840196+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:16:41.692087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015277190000000001,"stat_sum":{"num_bytes":7898416,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":1784,"num_read_kb":2687,"num_write":301,"num_write_kb":1987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"49'434","reported_seq":1463,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.060200+0000","last_change":"2024-09-14T09:05:28.851587+0000","last_active":"2024-09-14T09:06:01.060200+0000","last_peered":"2024-09-14T09:06:01.060200+0000","last_clean":"2024-09-14T09:06:01.060200+0000","last_became_active":"2024-09-14T09:05:26.840893+0000","last_became_peered":"2024-09-14T09:05:26.840893+0000","last_unstale":"2024-09-14T09:06:01.060200+0000","last_undegraded":"2024-09-14T09:06:01.060200+0000","last_fullsized":"2024-09-14T09:06:01.060200+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:38:09.246686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098594000000000004,"stat_sum":{"num_bytes":2240578,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":3866,"num_read_kb":3815,"num_write":709,"num_write_kb":2450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"49'188","reported_seq":899,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.060043+0000","last_change":"2024-09-14T09:05:42.832730+0000","last_active":"2024-09-14T09:06:01.060043+0000","last_peered":"2024-09-14T09:06:01.060043+0000","last_clean":"2024-09-14T09:06:01.060043+0000","last_became_active":"2024-09-14T09:05:42.829504+0000","last_became_peered":"2024-09-14T09:05:42.829504+0000","last_unstale":"2024-09-14T09:06:01.060043+0000","last_undegraded":"2024-09-14T09:06:01.060043+0000","last_fullsized":"2024-09-14T09:06:01.060043+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:36:41.830116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014079679999999999,"stat_sum":{"num_bytes":17951999,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":1181,"num_read_kb":1832,"num_write":250,"num_write_kb":453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":53852,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":82,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.846104+0000","last_change":"2024-09-14T09:05:19.793674+0000","last_active":"2024-09-14T09:06:01.846104+0000","last_peered":"2024-09-14T09:06:01.846104+0000","last_clean":"2024-09-14T09:06:01.846104+0000","last_became_active":"2024-09-14T09:05:17.284977+0000","last_became_peered":"2024-09-14T09:05:17.284977+0000","last_unstale":"2024-09-14T09:06:01.846104+0000","last_undegraded":"2024-09-14T09:06:01.846104+0000","last_fullsized":"2024-09-14T09:06:01.846104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:30:38.832027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"49'115","reported_seq":2984,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.828168+0000","last_change":"2024-09-14T09:05:28.839614+0000","last_active":"2024-09-14T09:06:01.828168+0000","last_peered":"2024-09-14T09:06:01.828168+0000","last_clean":"2024-09-14T09:06:01.828168+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:06:01.828168+0000","last_undegraded":"2024-09-14T09:06:01.828168+0000","last_fullsized":"2024-09-14T09:06:01.828168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:53:08.193834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014245869999999999,"stat_sum":{"num_bytes":32107323,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":3414,"num_read_kb":29362,"num_write":132,"num_write_kb":1595,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":80,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.053954+0000","last_change":"2024-09-14T09:05:19.314029+0000","last_active":"2024-09-14T09:06:01.053954+0000","last_peered":"2024-09-14T09:06:01.053954+0000","last_clean":"2024-09-14T09:06:01.053954+0000","last_became_active":"2024-09-14T09:05:17.282890+0000","last_became_peered":"2024-09-14T09:05:17.282890+0000","last_unstale":"2024-09-14T09:06:01.053954+0000","last_undegraded":"2024-09-14T09:06:01.053954+0000","last_fullsized":"2024-09-14T09:06:01.053954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:25.304105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"49'283","reported_seq":2136,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.053009+0000","last_change":"2024-09-14T09:05:29.039154+0000","last_active":"2024-09-14T09:06:01.053009+0000","last_peered":"2024-09-14T09:06:01.053009+0000","last_clean":"2024-09-14T09:06:01.053009+0000","last_became_active":"2024-09-14T09:05:26.829855+0000","last_became_peered":"2024-09-14T09:05:26.829855+0000","last_unstale":"2024-09-14T09:06:01.053009+0000","last_undegraded":"2024-09-14T09:06:01.053009+0000","last_fullsized":"2024-09-14T09:06:01.053009+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:19:10.437237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019213749999999999,"stat_sum":{"num_bytes":10037280,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":3148,"num_read_kb":7631,"num_write":365,"num_write_kb":520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":80,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.060845+0000","last_change":"2024-09-14T09:05:19.794165+0000","last_active":"2024-09-14T09:06:01.060845+0000","last_peered":"2024-09-14T09:06:01.060845+0000","last_clean":"2024-09-14T09:06:01.060845+0000","last_became_active":"2024-09-14T09:05:17.295864+0000","last_became_peered":"2024-09-14T09:05:17.295864+0000","last_unstale":"2024-09-14T09:06:01.060845+0000","last_undegraded":"2024-09-14T09:06:01.060845+0000","last_fullsized":"2024-09-14T09:06:01.060845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:53.901865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011399719999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"49'88","reported_seq":466,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.053888+0000","last_change":"2024-09-14T09:05:28.823608+0000","last_active":"2024-09-14T09:06:01.053888+0000","last_peered":"2024-09-14T09:06:01.053888+0000","last_clean":"2024-09-14T09:06:01.053888+0000","last_became_active":"2024-09-14T09:05:26.816468+0000","last_became_peered":"2024-09-14T09:05:26.816468+0000","last_unstale":"2024-09-14T09:06:01.053888+0000","last_undegraded":"2024-09-14T09:06:01.053888+0000","last_fullsized":"2024-09-14T09:06:01.053888+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:37:11.870271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014087959999999999,"stat_sum":{"num_bytes":4316963,"num_objects":13,"num_object_clones":5,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":6,"num_read":336,"num_read_kb":979,"num_write":72,"num_write_kb":2354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":80,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.053941+0000","last_change":"2024-09-14T09:05:19.314129+0000","last_active":"2024-09-14T09:06:01.053941+0000","last_peered":"2024-09-14T09:06:01.053941+0000","last_clean":"2024-09-14T09:06:01.053941+0000","last_became_active":"2024-09-14T09:05:17.295196+0000","last_became_peered":"2024-09-14T09:05:17.295196+0000","last_unstale":"2024-09-14T09:06:01.053941+0000","last_undegraded":"2024-09-14T09:06:01.053941+0000","last_fullsized":"2024-09-14T09:06:01.053941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:35.269115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011613800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"47'77","reported_seq":514,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.840237+0000","last_change":"2024-09-14T09:05:28.835976+0000","last_active":"2024-09-14T09:06:01.840237+0000","last_peered":"2024-09-14T09:06:01.840237+0000","last_clean":"2024-09-14T09:06:01.840237+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:06:01.840237+0000","last_undegraded":"2024-09-14T09:06:01.840237+0000","last_fullsized":"2024-09-14T09:06:01.840237+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:33:32.941970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017480779999999999,"stat_sum":{"num_bytes":29912509,"num_objects":21,"num_object_clones":9,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":394,"num_read_kb":1377,"num_write":73,"num_write_kb":5119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"49'139","reported_seq":1105,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.828718+0000","last_change":"2024-09-14T09:05:39.930107+0000","last_active":"2024-09-14T09:06:01.828718+0000","last_peered":"2024-09-14T09:06:01.828718+0000","last_clean":"2024-09-14T09:06:01.828718+0000","last_became_active":"2024-09-14T09:05:39.914913+0000","last_became_peered":"2024-09-14T09:05:39.914913+0000","last_unstale":"2024-09-14T09:06:01.828718+0000","last_undegraded":"2024-09-14T09:06:01.828718+0000","last_fullsized":"2024-09-14T09:06:01.828718+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:24:00.764912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014128,"stat_sum":{"num_bytes":8477012,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":1150,"num_read_kb":1677,"num_write":183,"num_write_kb":3294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.839810+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:06:01.839810+0000","last_peered":"2024-09-14T09:06:01.839810+0000","last_clean":"2024-09-14T09:06:01.839810+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:06:01.839810+0000","last_undegraded":"2024-09-14T09:06:01.839810+0000","last_fullsized":"2024-09-14T09:06:01.839810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"49'95","reported_seq":631,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.053556+0000","last_change":"2024-09-14T09:05:28.822791+0000","last_active":"2024-09-14T09:06:01.053556+0000","last_peered":"2024-09-14T09:06:01.053556+0000","last_clean":"2024-09-14T09:06:01.053556+0000","last_became_active":"2024-09-14T09:05:26.817109+0000","last_became_peered":"2024-09-14T09:05:26.817109+0000","last_unstale":"2024-09-14T09:06:01.053556+0000","last_undegraded":"2024-09-14T09:06:01.053556+0000","last_fullsized":"2024-09-14T09:06:01.053556+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:35:05.613868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011728439999999999,"stat_sum":{"num_bytes":1167376,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":622,"num_read_kb":1590,"num_write":114,"num_write_kb":582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":82,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.817596+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:06:01.817596+0000","last_peered":"2024-09-14T09:06:01.817596+0000","last_clean":"2024-09-14T09:06:01.817596+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:06:01.817596+0000","last_undegraded":"2024-09-14T09:06:01.817596+0000","last_fullsized":"2024-09-14T09:06:01.817596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"48'87","reported_seq":1060,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.828130+0000","last_change":"2024-09-14T09:05:39.925683+0000","last_active":"2024-09-14T09:06:01.828130+0000","last_peered":"2024-09-14T09:06:01.828130+0000","last_clean":"2024-09-14T09:06:01.828130+0000","last_became_active":"2024-09-14T09:05:39.905252+0000","last_became_peered":"2024-09-14T09:05:39.905252+0000","last_unstale":"2024-09-14T09:06:01.828130+0000","last_undegraded":"2024-09-14T09:06:01.828130+0000","last_fullsized":"2024-09-14T09:06:01.828130+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:15:01.219578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074815999999999999,"stat_sum":{"num_bytes":10378810,"num_objects":10,"num_object_clones":4,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":903,"num_read_kb":3629,"num_write":78,"num_write_kb":4813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2097152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":80,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.059799+0000","last_change":"2024-09-14T09:05:19.793999+0000","last_active":"2024-09-14T09:06:01.059799+0000","last_peered":"2024-09-14T09:06:01.059799+0000","last_clean":"2024-09-14T09:06:01.059799+0000","last_became_active":"2024-09-14T09:05:17.284122+0000","last_became_peered":"2024-09-14T09:05:17.284122+0000","last_unstale":"2024-09-14T09:06:01.059799+0000","last_undegraded":"2024-09-14T09:06:01.059799+0000","last_fullsized":"2024-09-14T09:06:01.059799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:12:01.133215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010234269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"49'123","reported_seq":625,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.052474+0000","last_change":"2024-09-14T09:05:29.038389+0000","last_active":"2024-09-14T09:06:01.052474+0000","last_peered":"2024-09-14T09:06:01.052474+0000","last_clean":"2024-09-14T09:06:01.052474+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:06:01.052474+0000","last_undegraded":"2024-09-14T09:06:01.052474+0000","last_fullsized":"2024-09-14T09:06:01.052474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:25:37.867752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001479076,"stat_sum":{"num_bytes":14892126,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":711,"num_read_kb":2027,"num_write":175,"num_write_kb":1613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.839848+0000","last_change":"2024-09-14T09:05:19.310814+0000","last_active":"2024-09-14T09:06:01.839848+0000","last_peered":"2024-09-14T09:06:01.839848+0000","last_clean":"2024-09-14T09:06:01.839848+0000","last_became_active":"2024-09-14T09:05:17.276973+0000","last_became_peered":"2024-09-14T09:05:17.276973+0000","last_unstale":"2024-09-14T09:06:01.839848+0000","last_undegraded":"2024-09-14T09:06:01.839848+0000","last_fullsized":"2024-09-14T09:06:01.839848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:33:35.187184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086884099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":171,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.059820+0000","last_change":"2024-09-14T09:05:13.239808+0000","last_active":"2024-09-14T09:06:01.059820+0000","last_peered":"2024-09-14T09:06:01.059820+0000","last_clean":"2024-09-14T09:06:01.059820+0000","last_became_active":"2024-09-14T09:05:13.239064+0000","last_became_peered":"2024-09-14T09:05:13.239064+0000","last_unstale":"2024-09-14T09:06:01.059820+0000","last_undegraded":"2024-09-14T09:06:01.059820+0000","last_fullsized":"2024-09-14T09:06:01.059820+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:40:11.559161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"49'46","reported_seq":498,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.052632+0000","last_change":"2024-09-14T09:05:29.037923+0000","last_active":"2024-09-14T09:06:01.052632+0000","last_peered":"2024-09-14T09:06:01.052632+0000","last_clean":"2024-09-14T09:06:01.052632+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:06:01.052632+0000","last_undegraded":"2024-09-14T09:06:01.052632+0000","last_fullsized":"2024-09-14T09:06:01.052632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:51.543836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099199800000000001,"stat_sum":{"num_bytes":7657878,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":385,"num_read_kb":692,"num_write":42,"num_write_kb":306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"49'112","reported_seq":1140,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.817506+0000","last_change":"2024-09-14T09:05:39.933291+0000","last_active":"2024-09-14T09:06:01.817506+0000","last_peered":"2024-09-14T09:06:01.817506+0000","last_clean":"2024-09-14T09:06:01.817506+0000","last_became_active":"2024-09-14T09:05:39.924025+0000","last_became_peered":"2024-09-14T09:05:39.924025+0000","last_unstale":"2024-09-14T09:06:01.817506+0000","last_undegraded":"2024-09-14T09:06:01.817506+0000","last_fullsized":"2024-09-14T09:06:01.817506+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:10:13.812289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001266028,"stat_sum":{"num_bytes":15189928,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":979,"num_read_kb":5390,"num_write":101,"num_write_kb":13509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"53'147","reported_seq":2713,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.857575+0000","last_change":"2024-09-14T09:05:39.975358+0000","last_active":"2024-09-14T09:06:01.857575+0000","last_peered":"2024-09-14T09:06:01.857575+0000","last_clean":"2024-09-14T09:06:01.857575+0000","last_became_active":"2024-09-14T09:05:39.918526+0000","last_became_peered":"2024-09-14T09:05:39.918526+0000","last_unstale":"2024-09-14T09:06:01.857575+0000","last_undegraded":"2024-09-14T09:06:01.857575+0000","last_fullsized":"2024-09-14T09:06:01.857575+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:13:32.027307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185731,"stat_sum":{"num_bytes":44806658,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":10,"num_read":3089,"num_read_kb":24631,"num_write":180,"num_write_kb":11592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":2159421,"num_keys_recovered":3,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"49'131","reported_seq":2999,"reported_epoch":52,"state":"activating+undersized+remapped","last_fresh":"2024-09-14T09:06:01.052641+0000","last_change":"2024-09-14T09:06:01.053342+0000","last_active":"2024-09-14T09:05:59.100591+0000","last_peered":"2024-09-14T09:05:58.188952+0000","last_clean":"2024-09-14T09:05:58.188952+0000","last_became_active":"2024-09-14T09:05:33.851290+0000","last_became_peered":"2024-09-14T09:05:33.851290+0000","last_unstale":"2024-09-14T09:06:01.052641+0000","last_undegraded":"2024-09-14T09:06:01.052641+0000","last_fullsized":"2024-09-14T09:06:01.052641+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001585295,"stat_sum":{"num_bytes":16013105,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":2861,"num_read_kb":28857,"num_write":137,"num_write_kb":3151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":["3","4"],"object_location_counts":[{"shards":"3","objects":22}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"52'83","reported_seq":595,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.839916+0000","last_change":"2024-09-14T09:05:28.834647+0000","last_active":"2024-09-14T09:06:01.839916+0000","last_peered":"2024-09-14T09:06:01.839916+0000","last_clean":"2024-09-14T09:06:01.839916+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:06:01.839916+0000","last_undegraded":"2024-09-14T09:06:01.839916+0000","last_fullsized":"2024-09-14T09:06:01.839916+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:37:43.220355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244290000000001,"stat_sum":{"num_bytes":12248052,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":458,"num_read_kb":1378,"num_write":102,"num_write_kb":2113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"47'61","reported_seq":663,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.060529+0000","last_change":"2024-09-14T09:05:28.851561+0000","last_active":"2024-09-14T09:06:01.060529+0000","last_peered":"2024-09-14T09:06:01.060529+0000","last_clean":"2024-09-14T09:06:01.060529+0000","last_became_active":"2024-09-14T09:05:26.843780+0000","last_became_peered":"2024-09-14T09:05:26.843780+0000","last_unstale":"2024-09-14T09:06:01.060529+0000","last_undegraded":"2024-09-14T09:06:01.060529+0000","last_fullsized":"2024-09-14T09:06:01.060529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:12:50.699873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072987200000000001,"stat_sum":{"num_bytes":12627639,"num_objects":10,"num_object_clones":4,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":5,"num_read":549,"num_read_kb":2894,"num_write":51,"num_write_kb":4271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"49'84","reported_seq":964,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.060695+0000","last_change":"2024-09-14T09:05:40.010535+0000","last_active":"2024-09-14T09:06:01.060695+0000","last_peered":"2024-09-14T09:06:01.060695+0000","last_clean":"2024-09-14T09:06:01.060695+0000","last_became_active":"2024-09-14T09:05:39.921575+0000","last_became_peered":"2024-09-14T09:05:39.921575+0000","last_unstale":"2024-09-14T09:06:01.060695+0000","last_undegraded":"2024-09-14T09:06:01.060695+0000","last_fullsized":"2024-09-14T09:06:01.060695+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:07.961392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057596100000000005,"stat_sum":{"num_bytes":25674437,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":797,"num_read_kb":7806,"num_write":79,"num_write_kb":3903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":111591,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"49'125","reported_seq":620,"reported_epoch":52,"state":"activating+undersized+remapped","last_fresh":"2024-09-14T09:06:01.059815+0000","last_change":"2024-09-14T09:06:01.061273+0000","last_active":"2024-09-14T09:05:59.053936+0000","last_peered":"2024-09-14T09:05:58.512199+0000","last_clean":"2024-09-14T09:05:58.512199+0000","last_became_active":"2024-09-14T09:05:39.925505+0000","last_became_peered":"2024-09-14T09:05:39.925505+0000","last_unstale":"2024-09-14T09:06:01.059815+0000","last_undegraded":"2024-09-14T09:06:01.059815+0000","last_fullsized":"2024-09-14T09:06:01.059815+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000909463,"stat_sum":{"num_bytes":24448703,"num_objects":24,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":588,"num_read_kb":1390,"num_write":103,"num_write_kb":4536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":53097,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7],"avail_no_missing":["7","1"],"object_location_counts":[{"shards":"7","objects":24}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"49'97","reported_seq":971,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.845816+0000","last_change":"2024-09-14T09:06:00.166357+0000","last_active":"2024-09-14T09:06:01.845816+0000","last_peered":"2024-09-14T09:06:01.845816+0000","last_clean":"2024-09-14T09:06:01.845816+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:06:01.845816+0000","last_undegraded":"2024-09-14T09:06:01.845816+0000","last_fullsized":"2024-09-14T09:06:01.845816+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:59:31.600018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011381609999999999,"stat_sum":{"num_bytes":35468563,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":754,"num_read_kb":3346,"num_write":130,"num_write_kb":6565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"47'170","reported_seq":2137,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.060165+0000","last_change":"2024-09-14T09:05:40.038345+0000","last_active":"2024-09-14T09:06:01.060165+0000","last_peered":"2024-09-14T09:06:01.060165+0000","last_clean":"2024-09-14T09:06:01.060165+0000","last_became_active":"2024-09-14T09:05:39.923017+0000","last_became_peered":"2024-09-14T09:05:39.923017+0000","last_unstale":"2024-09-14T09:06:01.060165+0000","last_undegraded":"2024-09-14T09:06:01.060165+0000","last_fullsized":"2024-09-14T09:06:01.060165+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:54:55.370461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092949299999999999,"stat_sum":{"num_bytes":2322505,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":2798,"num_read_kb":2780,"num_write":248,"num_write_kb":253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"49'74","reported_seq":694,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.828044+0000","last_change":"2024-09-14T09:05:28.839774+0000","last_active":"2024-09-14T09:06:01.828044+0000","last_peered":"2024-09-14T09:06:01.828044+0000","last_clean":"2024-09-14T09:06:01.828044+0000","last_became_active":"2024-09-14T09:05:26.826600+0000","last_became_peered":"2024-09-14T09:05:26.826600+0000","last_unstale":"2024-09-14T09:06:01.828044+0000","last_undegraded":"2024-09-14T09:06:01.828044+0000","last_fullsized":"2024-09-14T09:06:01.828044+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:28:45.988927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018357099999999999,"stat_sum":{"num_bytes":14770274,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":6,"num_read":1183,"num_read_kb":895,"num_write":63,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"47'95","reported_seq":3091,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-14T09:06:01.061152+0000","last_change":"2024-09-14T09:05:39.938958+0000","last_active":"2024-09-14T09:06:01.061152+0000","last_peered":"2024-09-14T09:06:01.061152+0000","last_clean":"2024-09-14T09:06:01.061152+0000","last_became_active":"2024-09-14T09:05:39.921023+0000","last_became_peered":"2024-09-14T09:05:39.921023+0000","last_unstale":"2024-09-14T09:06:01.061152+0000","last_undegraded":"2024-09-14T09:06:01.061152+0000","last_fullsized":"2024-09-14T09:06:01.061152+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:34:37.876948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061978900000000004,"stat_sum":{"num_bytes":6352096,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":3425,"num_read_kb":12577,"num_write":82,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":120888,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":521634590,"num_objects":498,"num_object_clones":149,"num_object_copies":996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":76,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":498,"num_whiteouts":183,"num_read":51824,"num_read_kb":225398,"num_write":5971,"num_write_kb":97969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":10100679,"num_keys_recovered":20,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13676544,"data_stored":86236809,"data_compressed":4013610,"data_compressed_allocated":9019392,"data_compressed_original":83551504,"omap_allocated":0,"internal_metadata":0},"log_size":4680,"ondisk_log_size":4680,"up":64,"acting":62,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12424,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34088,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337752,"statfs":{"total":96636764160,"available":96601858048,"internally_reserved":0,"allocated":2961408,"data_stored":18832556,"data_compressed":597639,"data_compressed_allocated":1703936,"data_compressed_original":18160904,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509454,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37312,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334528,"statfs":{"total":96636764160,"available":96598556672,"internally_reserved":0,"allocated":2011136,"data_stored":9513322,"data_compressed":434997,"data_compressed_allocated":925696,"data_compressed_original":8985309,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1589248,"data_stored":5809126,"data_compressed":382053,"data_compressed_allocated":712704,"data_compressed_original":5405251,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33272,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338568,"statfs":{"total":96636764160,"available":96602693632,"internally_reserved":0,"allocated":2129920,"data_stored":12614659,"data_compressed":343397,"data_compressed_allocated":1069056,"data_compressed_original":12060247,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542157,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37880,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333960,"statfs":{"total":96636764160,"available":96597975040,"internally_reserved":0,"allocated":2584576,"data_stored":13427021,"data_compressed":611808,"data_compressed_allocated":1363968,"data_compressed_original":12794934,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542158,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1048576,"data_stored":2232463,"data_compressed":211736,"data_compressed_allocated":282624,"data_compressed_original":1871872,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37536,"kb_used_data":2184,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334304,"statfs":{"total":96636764160,"available":96598327296,"internally_reserved":0,"allocated":2236416,"data_stored":8719104,"data_compressed":399395,"data_compressed_allocated":905216,"data_compressed_original":8044885,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38876,"kb_used_data":3532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332964,"statfs":{"total":96636764160,"available":96596955136,"internally_reserved":0,"allocated":3616768,"data_stored":17698021,"data_compressed":963395,"data_compressed_allocated":2007040,"data_compressed_original":16867078,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1802240,"data_stored":12380843,"data_compressed":418799,"data_compressed_allocated":1155072,"data_compressed_original":11994711,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3035136,"data_stored":17448007,"data_compressed":963395,"data_compressed_allocated":2007040,"data_compressed_original":16867078,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":987136,"data_stored":5547358,"data_compressed":382053,"data_compressed_allocated":712704,"data_compressed_original":5405251,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1658880,"data_stored":8469109,"data_compressed":399395,"data_compressed_allocated":905216,"data_compressed_original":8044885,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":1982468,"data_compressed":211736,"data_compressed_allocated":282624,"data_compressed_original":1871872,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1425408,"data_stored":9257483,"data_compressed":434997,"data_compressed_allocated":925696,"data_compressed_original":8985309,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2359296,"data_stored":18570788,"data_compressed":597639,"data_compressed_allocated":1703936,"data_compressed_original":18160904,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1937408,"data_stored":12580753,"data_compressed":605596,"data_compressed_allocated":1327104,"data_compressed_original":12221494,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:06:05.874 INFO:teuthology.orchestra.run.smithi176.stdout:1567 read 0x4c16a84 thru 0x4c1db16 (0x7093 bytes) 2024-09-14T09:06:05.880 INFO:teuthology.orchestra.run.smithi176.stdout:1568 trunc from 0x87a79af to 0xd091a02 2024-09-14T09:06:05.893 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:05.893 INFO:teuthology.orchestra.run.smithi176.stdout:1571 read 0x642ce26 thru 0x643053f (0x371a bytes) 2024-09-14T09:06:05.894 INFO:teuthology.orchestra.run.smithi176.stdout:1572 read 0x2dd214e thru 0x2dd978f (0x7642 bytes) 2024-09-14T09:06:05.894 INFO:teuthology.orchestra.run.smithi176.stdout:1573 punch from 0xbb0f902 to 0xbb16e59, (0x7557 bytes) 2024-09-14T09:06:05.894 INFO:teuthology.orchestra.run.smithi176.stdout:1574 write 0xb19fbc2 thru 0xb1a03ac (0x7eb bytes) 2024-09-14T09:06:05.898 INFO:teuthology.orchestra.run.smithi176.stdout:1575 writesame 0xa6a5cec thru 0xa6b1024 data_size 0x2b(0xb339 bytes) 2024-09-14T09:06:05.900 INFO:teuthology.orchestra.run.smithi176.stdout:1576 write 0xa980122 thru 0xa982a43 (0x2922 bytes) 2024-09-14T09:06:05.906 INFO:teuthology.orchestra.run.smithi176.stdout:1577 punch from 0xc032ae7 to 0xc03abb0, (0x80c9 bytes) 2024-09-14T09:06:05.906 INFO:teuthology.orchestra.run.smithi176.stdout:1578 writesame 0x6768804 thru 0x677869a data_size 0xa5(0xfe97 bytes) 2024-09-14T09:06:05.910 INFO:teuthology.orchestra.run.smithi176.stdout:1579 read 0x2283a4 thru 0x22d0ca (0x4d27 bytes) 2024-09-14T09:06:05.910 INFO:teuthology.orchestra.run.smithi176.stdout:1580 read 0x3ceaa4e thru 0x3cf97d1 (0xed84 bytes) 2024-09-14T09:06:05.910 INFO:teuthology.orchestra.run.smithi176.stdout:1581 write 0x3c6c931 thru 0x3c77cfe (0xb3ce bytes) 2024-09-14T09:06:05.913 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:05.913 INFO:teuthology.orchestra.run.smithi176.stdout:1584 read 0xcf886b6 thru 0xcf94c00 (0xc54b bytes) 2024-09-14T09:06:05.914 INFO:teuthology.orchestra.run.smithi176.stdout:1585 punch from 0x6b4eb9e to 0x6b5b656, (0xcab8 bytes) 2024-09-14T09:06:05.914 INFO:teuthology.orchestra.run.smithi176.stdout:1586 read 0x56fa301 thru 0x5703a48 (0x9748 bytes) 2024-09-14T09:06:05.914 INFO:teuthology.orchestra.run.smithi176.stdout:1587 writesame 0x7d221db thru 0x7d278fe data_size 0x42(0x5724 bytes) 2024-09-14T09:06:05.915 INFO:teuthology.orchestra.run.smithi176.stdout:1588 read 0x3335b4d thru 0x3342326 (0xc7da bytes) 2024-09-14T09:06:05.916 INFO:teuthology.orchestra.run.smithi176.stdout:1589 writesame 0x54bc5fc thru 0x54c0c49 data_size 0x1(0x464e bytes) 2024-09-14T09:06:05.945 INFO:teuthology.orchestra.run.smithi176.stdout:1591 writesame 0x9ee1e09 thru 0x9ee8678 data_size 0x18(0x6870 bytes) 2024-09-14T09:06:05.951 INFO:teuthology.orchestra.run.smithi176.stdout:1594 trunc from 0xd091a02 to 0xaa768e8 2024-09-14T09:06:05.959 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:05.959+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:05.965 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:05.965 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:05.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:05.966 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:05.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.966 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18383 3.9 3:9252f7c9:::rbd_data.115e3abd3b2c.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:06:05.966 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:05.966 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 22.806676799000002, 2024-09-14T09:06:05.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056433830999999997, 2024-09-14T09:06:05.966 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:05.966 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:05.966 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:05.967 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:05.967 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:05.967 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:05.967 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18383 2024-09-14T09:06:05.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.967 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:05.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:05.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:05.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:05.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:05.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:05.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:05.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:05.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158423+0000", 2024-09-14T09:06:05.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T09:06:05.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:05.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158429+0000", 2024-09-14T09:06:05.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0179999999999998e-06 2024-09-14T09:06:05.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:05.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158440+0000", 2024-09-14T09:06:05.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0769e-05 2024-09-14T09:06:05.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:05.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158445+0000", 2024-09-14T09:06:05.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5840000000000002e-06 2024-09-14T09:06:05.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:05.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158495+0000", 2024-09-14T09:06:05.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9129999999999999e-05 2024-09-14T09:06:05.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:05.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158598+0000", 2024-09-14T09:06:05.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102938 2024-09-14T09:06:05.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:05.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.161048+0000", 2024-09-14T09:06:05.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002450336 2024-09-14T09:06:05.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:05.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214605+0000", 2024-09-14T09:06:05.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053557526000000001 2024-09-14T09:06:05.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:05.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214634+0000", 2024-09-14T09:06:05.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8483000000000002e-05 2024-09-14T09:06:05.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:05.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214731+0000", 2024-09-14T09:06:05.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7026000000000002e-05 2024-09-14T09:06:05.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:05.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214856+0000", 2024-09-14T09:06:05.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012474799999999999 2024-09-14T09:06:05.975 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:05.975 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:05.976 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:05.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.976 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18383 3.9 e38/34 ondisk, result = 0)", 2024-09-14T09:06:05.976 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:05.976 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 22.796360318000001, 2024-09-14T09:06:05.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045880575999999999, 2024-09-14T09:06:05.976 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:05.977 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:05.977 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:05.977 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:05.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:05.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:05.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:05.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:05.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:05.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:05.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:05.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168739+0000", 2024-09-14T09:06:05.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3900000000000004e-07 2024-09-14T09:06:05.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:05.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168744+0000", 2024-09-14T09:06:05.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5929999999999999e-06 2024-09-14T09:06:05.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:05.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168754+0000", 2024-09-14T09:06:05.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0686999999999999e-05 2024-09-14T09:06:05.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:05.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168761+0000", 2024-09-14T09:06:05.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6340000000000003e-06 2024-09-14T09:06:05.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:05.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214578+0000", 2024-09-14T09:06:05.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045817443999999999 2024-09-14T09:06:05.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:05.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214596+0000", 2024-09-14T09:06:05.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7159e-05 2024-09-14T09:06:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214619+0000", 2024-09-14T09:06:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.332e-05 2024-09-14T09:06:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19152 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14159058~8940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.642447862000001, 2024-09-14T09:06:05.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12891994500000001, 2024-09-14T09:06:05.982 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:05.982 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:05.982 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:05.982 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:05.982 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:05.982 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:05.982 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19152 2024-09-14T09:06:05.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.982 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:05.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:05.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:05.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:05.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:05.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:05.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:05.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:05.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322653+0000", 2024-09-14T09:06:05.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3750000000000001e-06 2024-09-14T09:06:05.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:05.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329647+0000", 2024-09-14T09:06:05.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069941960000000003 2024-09-14T09:06:05.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:05.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329673+0000", 2024-09-14T09:06:05.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5760000000000001e-05 2024-09-14T09:06:05.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:05.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329684+0000", 2024-09-14T09:06:05.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0474e-05 2024-09-14T09:06:05.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:05.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329776+0000", 2024-09-14T09:06:05.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2467999999999996e-05 2024-09-14T09:06:05.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:05.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329987+0000", 2024-09-14T09:06:05.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021133 2024-09-14T09:06:05.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:05.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.331303+0000", 2024-09-14T09:06:05.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001315629 2024-09-14T09:06:05.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:05.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.411090+0000", 2024-09-14T09:06:05.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079787477999999995 2024-09-14T09:06:05.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:05.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451463+0000", 2024-09-14T09:06:05.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040372312 2024-09-14T09:06:05.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:05.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451523+0000", 2024-09-14T09:06:05.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0420000000000001e-05 2024-09-14T09:06:05.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:05.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451571+0000", 2024-09-14T09:06:05.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7503000000000002e-05 2024-09-14T09:06:05.988 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:05.988 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:05.988 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:05.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.988 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20925 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:05.988 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:06:05.989 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.293149081999999, 2024-09-14T09:06:05.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017084327999999999, 2024-09-14T09:06:05.989 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:05.989 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:05.989 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:05.989 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:05.989 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:05.989 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:05.990 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20925 2024-09-14T09:06:05.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.990 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:05.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:05.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:06:05.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:05.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:05.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:06:05.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:05.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:05.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:06:05.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3e-07 2024-09-14T09:06:05.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:05.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671953+0000", 2024-09-14T09:06:05.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0299999999999998e-06 2024-09-14T09:06:05.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:05.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671964+0000", 2024-09-14T09:06:05.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1e-05 2024-09-14T09:06:05.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:05.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671971+0000", 2024-09-14T09:06:05.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9929999999999998e-06 2024-09-14T09:06:05.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:05.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.687941+0000", 2024-09-14T09:06:05.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015969357 2024-09-14T09:06:05.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:05.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.688070+0000", 2024-09-14T09:06:05.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000129858 2024-09-14T09:06:05.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:05.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689034+0000", 2024-09-14T09:06:05.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00096336000000000002 2024-09-14T09:06:05.995 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:05.995 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:05.995 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:05.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.996 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20926 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:05.996 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:05.996 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.293115862000001, 2024-09-14T09:06:05.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018572207, 2024-09-14T09:06:05.996 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:05.996 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:05.996 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:05.997 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:05.997 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:05.997 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:05.997 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20926 2024-09-14T09:06:05.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.997 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:05.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:05.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:05.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:05.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:05.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:05.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:05.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:05.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:05.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:06:05.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:05.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:05.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671990+0000", 2024-09-14T09:06:05.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5769999999999999e-06 2024-09-14T09:06:05.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:05.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672001+0000", 2024-09-14T09:06:06.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1293e-05 2024-09-14T09:06:06.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672008+0000", 2024-09-14T09:06:06.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9099999999999999e-06 2024-09-14T09:06:06.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689147+0000", 2024-09-14T09:06:06.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017139060000000001 2024-09-14T09:06:06.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689324+0000", 2024-09-14T09:06:06.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017640200000000001 2024-09-14T09:06:06.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690555+0000", 2024-09-14T09:06:06.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001231212 2024-09-14T09:06:06.002 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.002 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.002 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.002 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20927 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2621440~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:06.002 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:06.002 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.293078896000001, 2024-09-14T09:06:06.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020002227000000001, 2024-09-14T09:06:06.002 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.002 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.002 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.003 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.003 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.003 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.003 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20927 2024-09-14T09:06:06.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.003 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:06.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:06.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:06.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9699999999999995e-07 2024-09-14T09:06:06.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672024+0000", 2024-09-14T09:06:06.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:06:06.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672036+0000", 2024-09-14T09:06:06.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.119e-05 2024-09-14T09:06:06.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672044+0000", 2024-09-14T09:06:06.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8370000000000003e-06 2024-09-14T09:06:06.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690602+0000", 2024-09-14T09:06:06.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018557648 2024-09-14T09:06:06.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690805+0000", 2024-09-14T09:06:06.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020282899999999999 2024-09-14T09:06:06.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692022+0000", 2024-09-14T09:06:06.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012170429999999999 2024-09-14T09:06:06.006 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.006 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.006 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.006 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20928 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 917504~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:06.007 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:06.007 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.293042755999998, 2024-09-14T09:06:06.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021623738, 2024-09-14T09:06:06.007 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.007 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.007 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.007 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.007 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.007 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.007 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20928 2024-09-14T09:06:06.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.007 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:06.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:06.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672057+0000", 2024-09-14T09:06:06.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:06:06.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672059+0000", 2024-09-14T09:06:06.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9270000000000001e-06 2024-09-14T09:06:06.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672070+0000", 2024-09-14T09:06:06.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0773e-05 2024-09-14T09:06:06.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672077+0000", 2024-09-14T09:06:06.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9600000000000003e-06 2024-09-14T09:06:06.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692154+0000", 2024-09-14T09:06:06.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020076482 2024-09-14T09:06:06.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692419+0000", 2024-09-14T09:06:06.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026517400000000002 2024-09-14T09:06:06.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693680+0000", 2024-09-14T09:06:06.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012607390000000001 2024-09-14T09:06:06.010 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.010 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.010 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.011 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20929 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2752512~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:06.011 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:06.011 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.293009714, 2024-09-14T09:06:06.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022898728, 2024-09-14T09:06:06.011 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.011 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.011 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.011 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20929 2024-09-14T09:06:06.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.011 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:06.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:06.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672090+0000", 2024-09-14T09:06:06.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:06:06.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672092+0000", 2024-09-14T09:06:06.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8940000000000002e-06 2024-09-14T09:06:06.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672103+0000", 2024-09-14T09:06:06.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0815999999999999e-05 2024-09-14T09:06:06.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672110+0000", 2024-09-14T09:06:06.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9469999999999997e-06 2024-09-14T09:06:06.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693737+0000", 2024-09-14T09:06:06.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021626467999999999 2024-09-14T09:06:06.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693962+0000", 2024-09-14T09:06:06.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225474 2024-09-14T09:06:06.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.694988+0000", 2024-09-14T09:06:06.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001025469 2024-09-14T09:06:06.015 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.015 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.015 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.015 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20930 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:06.015 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:06.015 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.292976620000001, 2024-09-14T09:06:06.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024303286, 2024-09-14T09:06:06.015 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.015 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.016 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.016 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.016 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.016 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.016 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20930 2024-09-14T09:06:06.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.016 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:06.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:06.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672123+0000", 2024-09-14T09:06:06.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:06:06.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672125+0000", 2024-09-14T09:06:06.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7300000000000001e-06 2024-09-14T09:06:06.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672134+0000", 2024-09-14T09:06:06.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7930000000000003e-06 2024-09-14T09:06:06.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672141+0000", 2024-09-14T09:06:06.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9029999999999998e-06 2024-09-14T09:06:06.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695087+0000", 2024-09-14T09:06:06.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022946013000000001 2024-09-14T09:06:06.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695350+0000", 2024-09-14T09:06:06.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026266000000000001 2024-09-14T09:06:06.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696425+0000", 2024-09-14T09:06:06.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001075527 2024-09-14T09:06:06.019 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.020 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.020 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.020 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20931 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:06.020 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:06.020 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.292943391000001, 2024-09-14T09:06:06.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025331519, 2024-09-14T09:06:06.020 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.020 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.020 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.020 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.020 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.021 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.021 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20931 2024-09-14T09:06:06.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.021 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.021 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.021 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:06.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.021 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.021 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:06.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672156+0000", 2024-09-14T09:06:06.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4300000000000002e-07 2024-09-14T09:06:06.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672159+0000", 2024-09-14T09:06:06.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T09:06:06.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672170+0000", 2024-09-14T09:06:06.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1192999999999999e-05 2024-09-14T09:06:06.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672177+0000", 2024-09-14T09:06:06.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9059999999999996e-06 2024-09-14T09:06:06.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696532+0000", 2024-09-14T09:06:06.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024355291000000001 2024-09-14T09:06:06.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696755+0000", 2024-09-14T09:06:06.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022297699999999999 2024-09-14T09:06:06.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697487+0000", 2024-09-14T09:06:06.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00073171499999999997 2024-09-14T09:06:06.024 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.024 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.025 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.025 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20932 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1179648~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:06.025 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:06.025 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.292909947999998, 2024-09-14T09:06:06.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026104083, 2024-09-14T09:06:06.025 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.025 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.025 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.025 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.025 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.025 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.025 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20932 2024-09-14T09:06:06.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.026 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:06.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:06.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:06.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0699999999999996e-07 2024-09-14T09:06:06.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672192+0000", 2024-09-14T09:06:06.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7700000000000002e-06 2024-09-14T09:06:06.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672203+0000", 2024-09-14T09:06:06.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0956e-05 2024-09-14T09:06:06.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672211+0000", 2024-09-14T09:06:06.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3769999999999996e-06 2024-09-14T09:06:06.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697530+0000", 2024-09-14T09:06:06.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025318411999999998 2024-09-14T09:06:06.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697640+0000", 2024-09-14T09:06:06.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011045299999999999 2024-09-14T09:06:06.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698293+0000", 2024-09-14T09:06:06.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065240800000000005 2024-09-14T09:06:06.029 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.029 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.029 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.029 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20933 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3014656~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:06.029 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:06.029 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.292875745, 2024-09-14T09:06:06.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026558875999999999, 2024-09-14T09:06:06.030 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.030 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.030 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.030 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.030 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.030 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.030 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20933 2024-09-14T09:06:06.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.030 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:06.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:06.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672224+0000", 2024-09-14T09:06:06.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2399999999999997e-07 2024-09-14T09:06:06.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672234+0000", 2024-09-14T09:06:06.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0793e-05 2024-09-14T09:06:06.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672246+0000", 2024-09-14T09:06:06.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659e-05 2024-09-14T09:06:06.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672255+0000", 2024-09-14T09:06:06.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5730000000000008e-06 2024-09-14T09:06:06.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698326+0000", 2024-09-14T09:06:06.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026071414000000001 2024-09-14T09:06:06.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698374+0000", 2024-09-14T09:06:06.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8032999999999999e-05 2024-09-14T09:06:06.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698782+0000", 2024-09-14T09:06:06.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040768 2024-09-14T09:06:06.033 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.033 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.033 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.034 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20934 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1310720~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:06.034 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:06.034 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.292833868999999, 2024-09-14T09:06:06.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027292067999999999, 2024-09-14T09:06:06.034 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.034 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.034 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.034 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.034 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.034 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.034 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20934 2024-09-14T09:06:06.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.035 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:06.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:06.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672266+0000", 2024-09-14T09:06:06.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6299999999999993e-07 2024-09-14T09:06:06.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672269+0000", 2024-09-14T09:06:06.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8569999999999999e-06 2024-09-14T09:06:06.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672280+0000", 2024-09-14T09:06:06.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1236999999999999e-05 2024-09-14T09:06:06.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672287+0000", 2024-09-14T09:06:06.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8260000000000004e-06 2024-09-14T09:06:06.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698820+0000", 2024-09-14T09:06:06.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026533055999999999 2024-09-14T09:06:06.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698933+0000", 2024-09-14T09:06:06.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011291 2024-09-14T09:06:06.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699557+0000", 2024-09-14T09:06:06.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062421899999999997 2024-09-14T09:06:06.039 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.039 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.039 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.039 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20935 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:06.039 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:06.039 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.292800109000002, 2024-09-14T09:06:06.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028108610999999999, 2024-09-14T09:06:06.040 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.040 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.040 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.040 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.040 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.040 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.040 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20935 2024-09-14T09:06:06.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.041 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.041 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.041 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:06.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.041 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:06.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672299+0000", 2024-09-14T09:06:06.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T09:06:06.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672302+0000", 2024-09-14T09:06:06.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7640000000000001e-06 2024-09-14T09:06:06.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672313+0000", 2024-09-14T09:06:06.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1080000000000001e-05 2024-09-14T09:06:06.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672321+0000", 2024-09-14T09:06:06.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6490000000000008e-06 2024-09-14T09:06:06.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699587+0000", 2024-09-14T09:06:06.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027265884000000001 2024-09-14T09:06:06.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699631+0000", 2024-09-14T09:06:06.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4947999999999997e-05 2024-09-14T09:06:06.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700407+0000", 2024-09-14T09:06:06.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00077559299999999999 2024-09-14T09:06:06.044 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.044 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.045 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.045 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20936 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1441792~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:06.045 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:06.045 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.292765932999998, 2024-09-14T09:06:06.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028784750000000001, 2024-09-14T09:06:06.045 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.045 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.045 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.045 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.045 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.045 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.045 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20936 2024-09-14T09:06:06.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.046 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:06.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:06.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:06.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4700000000000001e-07 2024-09-14T09:06:06.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672336+0000", 2024-09-14T09:06:06.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6730000000000001e-06 2024-09-14T09:06:06.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672347+0000", 2024-09-14T09:06:06.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0710000000000001e-05 2024-09-14T09:06:06.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672354+0000", 2024-09-14T09:06:06.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4630000000000004e-06 2024-09-14T09:06:06.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700442+0000", 2024-09-14T09:06:06.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028087951 2024-09-14T09:06:06.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700557+0000", 2024-09-14T09:06:06.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011461800000000001 2024-09-14T09:06:06.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701117+0000", 2024-09-14T09:06:06.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056058799999999999 2024-09-14T09:06:06.049 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.049 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.049 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.049 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20937 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:06.050 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:06.050 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.292732807, 2024-09-14T09:06:06.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029493028000000001, 2024-09-14T09:06:06.050 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.050 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.050 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.050 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.050 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.050 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.050 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20937 2024-09-14T09:06:06.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.050 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:06.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:06.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:06.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T09:06:06.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672384+0000", 2024-09-14T09:06:06.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7127000000000001e-05 2024-09-14T09:06:06.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672395+0000", 2024-09-14T09:06:06.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759999999999999e-05 2024-09-14T09:06:06.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672402+0000", 2024-09-14T09:06:06.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0960000000000003e-06 2024-09-14T09:06:06.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701147+0000", 2024-09-14T09:06:06.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028744241 2024-09-14T09:06:06.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701246+0000", 2024-09-14T09:06:06.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9031999999999996e-05 2024-09-14T09:06:06.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701859+0000", 2024-09-14T09:06:06.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061310499999999997 2024-09-14T09:06:06.054 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.054 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.054 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.054 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20938 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:06.054 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:06.054 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.292685646999999, 2024-09-14T09:06:06.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030109219999999999, 2024-09-14T09:06:06.054 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.054 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.054 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.054 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.055 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.055 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.055 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20938 2024-09-14T09:06:06.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.055 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:06.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:06.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672414+0000", 2024-09-14T09:06:06.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-07 2024-09-14T09:06:06.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672417+0000", 2024-09-14T09:06:06.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:06:06.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672428+0000", 2024-09-14T09:06:06.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0873000000000001e-05 2024-09-14T09:06:06.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672435+0000", 2024-09-14T09:06:06.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:06:06.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701884+0000", 2024-09-14T09:06:06.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029449811999999999 2024-09-14T09:06:06.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701934+0000", 2024-09-14T09:06:06.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9703000000000002e-05 2024-09-14T09:06:06.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702522+0000", 2024-09-14T09:06:06.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00058811200000000001 2024-09-14T09:06:06.058 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.058 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.058 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.059 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20939 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:06.059 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:06.059 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.292653514000001, 2024-09-14T09:06:06.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030779259, 2024-09-14T09:06:06.059 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.059 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.059 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.059 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20939 2024-09-14T09:06:06.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.059 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:06.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:06.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672446+0000", 2024-09-14T09:06:06.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:06:06.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672449+0000", 2024-09-14T09:06:06.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7930000000000002e-06 2024-09-14T09:06:06.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672461+0000", 2024-09-14T09:06:06.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T09:06:06.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672471+0000", 2024-09-14T09:06:06.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0807e-05 2024-09-14T09:06:06.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702551+0000", 2024-09-14T09:06:06.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030079228 2024-09-14T09:06:06.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702664+0000", 2024-09-14T09:06:06.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011351800000000001 2024-09-14T09:06:06.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.703224+0000", 2024-09-14T09:06:06.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056026999999999997 2024-09-14T09:06:06.063 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.063 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.063 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.063 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29321 3.9 3:90303d3a:::rbd_data.115ebc3c67d1.0000000000000014:c [read 770048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:06:06.063 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:06:06.063 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 13.145570189000001, 2024-09-14T09:06:06.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015942787, 2024-09-14T09:06:06.063 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.063 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.063 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.063 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.063 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.064 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.064 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29321 2024-09-14T09:06:06.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.064 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:06:06.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:06:06.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819529+0000", 2024-09-14T09:06:06.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8800000000000003e-07 2024-09-14T09:06:06.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819531+0000", 2024-09-14T09:06:06.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9439999999999999e-06 2024-09-14T09:06:06.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819537+0000", 2024-09-14T09:06:06.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.037e-06 2024-09-14T09:06:06.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819542+0000", 2024-09-14T09:06:06.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7670000000000003e-06 2024-09-14T09:06:06.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835320+0000", 2024-09-14T09:06:06.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015778119 2024-09-14T09:06:06.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835384+0000", 2024-09-14T09:06:06.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3897000000000002e-05 2024-09-14T09:06:06.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835471+0000", 2024-09-14T09:06:06.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7534999999999995e-05 2024-09-14T09:06:06.067 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.067 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.067 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.067 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29322 3.9 3:90303d3a:::rbd_data.115ebc3c67d1.0000000000000014:c [read 780288~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:06:06.067 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:06:06.068 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 13.145550678999999, 2024-09-14T09:06:06.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016069904999999999, 2024-09-14T09:06:06.068 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.068 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.068 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.068 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.068 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.068 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.068 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29322 2024-09-14T09:06:06.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.068 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:06:06.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:06:06.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:06:06.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9499999999999998e-07 2024-09-14T09:06:06.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819550+0000", 2024-09-14T09:06:06.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8309999999999999e-06 2024-09-14T09:06:06.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819556+0000", 2024-09-14T09:06:06.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1299999999999998e-06 2024-09-14T09:06:06.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819561+0000", 2024-09-14T09:06:06.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.707e-06 2024-09-14T09:06:06.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835500+0000", 2024-09-14T09:06:06.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015939456000000001 2024-09-14T09:06:06.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835566+0000", 2024-09-14T09:06:06.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5285999999999998e-05 2024-09-14T09:06:06.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835618+0000", 2024-09-14T09:06:06.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2099999999999999e-05 2024-09-14T09:06:06.072 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.072 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.072 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.072 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.072 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.072 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:06.073 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.073 INFO:teuthology.orchestra.run.smithi176.stdout:1596 punch from 0x3dfe939 to 0x3e0a113, (0xb7da bytes) 2024-09-14T09:06:06.073 INFO:teuthology.orchestra.run.smithi176.stdout:1598 read 0x3202458 thru 0x320d7cb (0xb374 bytes) 2024-09-14T09:06:06.073 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.073 INFO:teuthology.orchestra.run.smithi176.stdout:1602 trunc from 0xaa768e8 to 0x9ee1510 2024-09-14T09:06:06.075 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:06:06.076 INFO:teuthology.orchestra.run.smithi176.stdout:1603 read 0x87b936 thru 0x881d27 (0x63f2 bytes) 2024-09-14T09:06:06.076 INFO:teuthology.orchestra.run.smithi176.stdout:1604 read 0x3acf87d thru 0x3ade945 (0xf0c9 bytes) 2024-09-14T09:06:06.076 INFO:teuthology.orchestra.run.smithi176.stdout:1606 trunc from 0x9ee1510 to 0xa36260a 2024-09-14T09:06:06.076 INFO:teuthology.orchestra.run.smithi176.stdout:1607 write 0x292dea4 thru 0x293403e (0x619b bytes) 2024-09-14T09:06:06.076 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.076 INFO:teuthology.orchestra.run.smithi176.stdout:1609 trunc from 0xa36260a to 0x99ca7f8 2024-09-14T09:06:06.076 INFO:teuthology.orchestra.run.smithi176.stdout:1610 read 0x88b2632 thru 0x88bb45a (0x8e29 bytes) 2024-09-14T09:06:06.076 INFO:teuthology.orchestra.run.smithi176.stdout:1614 writesame 0x1fade17 thru 0x1fbbae3 data_size 0x85(0xdccd bytes) 2024-09-14T09:06:06.076 INFO:teuthology.orchestra.run.smithi176.stdout:1615 write 0x2d65ff2 thru 0x2d747ac (0xe7bb bytes) 2024-09-14T09:06:06.076 INFO:teuthology.orchestra.run.smithi176.stdout:1616 trunc from 0x99ca7f8 to 0x840b04 2024-09-14T09:06:06.077 INFO:teuthology.orchestra.run.smithi176.stdout:1617 read 0x6457fe thru 0x653c9d (0xe4a0 bytes) 2024-09-14T09:06:06.097 INFO:teuthology.orchestra.run.smithi176.stdout:1618 write 0x8201a72 thru 0x8210327 (0xe8b6 bytes) 2024-09-14T09:06:06.097 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:06.092+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:06.100 INFO:teuthology.orchestra.run.smithi176.stdout:1619 punch from 0xc0ef6d to 0xc17d4f, (0x8de2 bytes) 2024-09-14T09:06:06.100 INFO:teuthology.orchestra.run.smithi176.stdout:1620 read 0x12e621c thru 0x12f397b (0xd760 bytes) 2024-09-14T09:06:06.101 INFO:teuthology.orchestra.run.smithi176.stdout:1621 read 0x6a32023 thru 0x6a3d481 (0xb45f bytes) 2024-09-14T09:06:06.101 INFO:teuthology.orchestra.run.smithi176.stdout:1622 read 0x2722e03 thru 0x272f1d0 (0xc3ce bytes) 2024-09-14T09:06:06.106 INFO:teuthology.orchestra.run.smithi176.stdout:1624 trunc from 0x8210328 to 0x5aac470 2024-09-14T09:06:06.138 INFO:teuthology.orchestra.run.smithi176.stdout:1625 writesame 0x5fe7c6a thru 0x5ff6b2f data_size 0xd(0xeec6 bytes) 2024-09-14T09:06:06.140 INFO:teuthology.orchestra.run.smithi176.stdout:1627 read 0x7e1cfb thru 0x7eecce (0xcfd4 bytes) 2024-09-14T09:06:06.140 INFO:teuthology.orchestra.run.smithi176.stdout:1628 write 0x25da155 thru 0x25e4319 (0xa1c5 bytes) 2024-09-14T09:06:06.147 INFO:teuthology.orchestra.run.smithi176.stdout:1629 write 0x1187d94 thru 0x11920ae (0xa31b bytes) 2024-09-14T09:06:06.154 INFO:teuthology.orchestra.run.smithi176.stdout:1630 writesame 0xc2598b thru 0xc3137a data_size 0x88(0xb9f0 bytes) 2024-09-14T09:06:06.158 INFO:teuthology.orchestra.run.smithi176.stdout:1633 writesame 0x333856d thru 0x3345751 data_size 0x1(0xd1e5 bytes) 2024-09-14T09:06:06.178 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:06.176+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:06.229 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:06.229 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:06.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.229 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:06:06.229 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.069897886000000006, 2024-09-14T09:06:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.069961849000000007, 2024-09-14T09:06:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "waiting for sub ops", 2024-09-14T09:06:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:06:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:06:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:06:06.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:06:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:06:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:06:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:06:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:06:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:06:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:06:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:06:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:06:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:06:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:06:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:06:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:06:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:39401 3.b e56/53)", 2024-09-14T09:06:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.044001819999999997, 2024-09-14T09:06:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.044179364999999998, 2024-09-14T09:06:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.234 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186509+0000", 2024-09-14T09:06:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2640000000000001e-06 2024-09-14T09:06:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186516+0000", 2024-09-14T09:06:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1070000000000001e-06 2024-09-14T09:06:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186533+0000", 2024-09-14T09:06:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7818000000000001e-05 2024-09-14T09:06:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186543+0000", 2024-09-14T09:06:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5170000000000005e-06 2024-09-14T09:06:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:06:06.237 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 2 2024-09-14T09:06:06.237 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:06.246 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:06:06.296 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:06.292+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:06.307 INFO:teuthology.orchestra.run.smithi176.stdout:1634 write 0xa3a93b0 thru 0xa3b692d (0xd57e bytes) 2024-09-14T09:06:06.314 INFO:teuthology.orchestra.run.smithi176.stdout:1635 punch from 0x601c157 to 0x602183e, (0x56e7 bytes) 2024-09-14T09:06:06.314 INFO:teuthology.orchestra.run.smithi176.stdout:1636 read 0x3f32b11 thru 0x3f4170c (0xebfc bytes) 2024-09-14T09:06:06.326 INFO:teuthology.orchestra.run.smithi176.stdout:1637 trunc from 0xa3b692e to 0x2104ffe 2024-09-14T09:06:06.350 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.356 INFO:teuthology.orchestra.run.smithi176.stdout:1639 trunc from 0x2104ffe to 0x64353b1 2024-09-14T09:06:06.367 INFO:teuthology.orchestra.run.smithi176.stdout:1640 punch from 0x5fa64f7 to 0x5fb5728, (0xf231 bytes) 2024-09-14T09:06:06.367 INFO:teuthology.orchestra.run.smithi176.stdout:1641 write 0x5a1493c thru 0x5a1a535 (0x5bfa bytes) 2024-09-14T09:06:06.370 INFO:teuthology.orchestra.run.smithi176.stdout:1642 punch from 0x4e61a2c to 0x4e6ab8e, (0x9162 bytes) 2024-09-14T09:06:06.370 INFO:teuthology.orchestra.run.smithi176.stdout:1643 punch from 0x231179d to 0x231722e, (0x5a91 bytes) 2024-09-14T09:06:06.370 INFO:teuthology.orchestra.run.smithi176.stdout:1644 read 0x58bc500 thru 0x58cb5b7 (0xf0b8 bytes) 2024-09-14T09:06:06.381 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:06.376+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:06.391 INFO:teuthology.orchestra.run.smithi176.stdout:1646 writesame 0xa350955 thru 0xa356b33 data_size 0x1(0x61df bytes) 2024-09-14T09:06:06.391 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.391 INFO:teuthology.orchestra.run.smithi176.stdout:1648 punch from 0x74c2ad0 to 0x74c4db6, (0x22e6 bytes) 2024-09-14T09:06:06.392 INFO:teuthology.orchestra.run.smithi176.stdout:1649 writesame 0x3de0ba7 thru 0x3de93e8 data_size 0x2(0x8842 bytes) 2024-09-14T09:06:06.424 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:06.424 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:06.425 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:06.425 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:06.425 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:06.439 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:06:06.442 INFO:teuthology.orchestra.run.smithi176.stdout:1650 write 0xcf246ed thru 0xcf306ac (0xbfc0 bytes) 2024-09-14T09:06:06.466 INFO:teuthology.orchestra.run.smithi176.stdout:1651 writesame 0xe200954 thru 0xe20b514 data_size 0x1(0xabc1 bytes) 2024-09-14T09:06:06.481 INFO:teuthology.orchestra.run.smithi176.stdout:1652 trunc from 0xe20b515 to 0x7e8ac9b 2024-09-14T09:06:06.481 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:06.476+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:06.503 INFO:teuthology.orchestra.run.smithi176.stdout:1653 write 0x71ad838 thru 0x71bc194 (0xe95d bytes) 2024-09-14T09:06:06.526 INFO:teuthology.orchestra.run.smithi176.stdout:1654 trunc from 0x7e8ac9b to 0xe3b43fe 2024-09-14T09:06:06.539 INFO:teuthology.orchestra.run.smithi176.stdout:1657 write 0x6aa829d thru 0x6ab3de5 (0xbb49 bytes) 2024-09-14T09:06:06.547 INFO:teuthology.orchestra.run.smithi176.stdout:1658 read 0x51e885d thru 0x51ebd8b (0x352f bytes) 2024-09-14T09:06:06.548 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.548 INFO:teuthology.orchestra.run.smithi176.stdout:1662 read 0x6ac9b00 thru 0x6ace20d (0x470e bytes) 2024-09-14T09:06:06.549 INFO:teuthology.orchestra.run.smithi176.stdout:1663 read 0xd732007 thru 0xd734e20 (0x2e1a bytes) 2024-09-14T09:06:06.549 INFO:teuthology.orchestra.run.smithi176.stdout:1664 read 0x8207ad7 thru 0x820a130 (0x265a bytes) 2024-09-14T09:06:06.550 INFO:teuthology.orchestra.run.smithi176.stdout:1665 writesame 0x4ab2f0f thru 0x4abb2da data_size 0x46(0x83cc bytes) 2024-09-14T09:06:06.554 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.580 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:06.580 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.492322780999999, 2024-09-14T09:06:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:06:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:06:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:06:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:06:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:06:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:06:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:06:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:06:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:06:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:06:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:06:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:06:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:06:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:06:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:06:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:06:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:06:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:06:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:06:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:06:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.492268123000002, 2024-09-14T09:06:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:06:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:06:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:06:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:06:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:06:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:06:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:06:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:06:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:06:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:06:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:06:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:06:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:06:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:06:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:06:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:06:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:06:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:06:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:06:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.492162749000002, 2024-09-14T09:06:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:06:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:06:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:06:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:06:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:06:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:06:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:06:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:06:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:06:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:06:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:06:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:06:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:06:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:06:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:06:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:06:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:06:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:06:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:06:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:06:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.491805984999999, 2024-09-14T09:06:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:06:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:06:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:06:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:06:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:06:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:06:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:06:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:06:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:06:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:06:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:06:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:06:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:06:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:06:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:06:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:06:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:06:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:06:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:06:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:06:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.491721247000001, 2024-09-14T09:06:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:06:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:06:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:06:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:06:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:06:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:06:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:06:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:06:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:06:06.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:06:06.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:06:06.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:06:06.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:06.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:06:06.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:06:06.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:06:06.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:06:06.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:06:06.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:06:06.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:06:06.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:06:06.609 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.609 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.609 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.609 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:06.609 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:06.609 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.491633561, 2024-09-14T09:06:06.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:06:06.609 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.609 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.609 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.610 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.610 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.610 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.610 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:06:06.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.610 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:06.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:06.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:06:06.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:06:06.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:06:06.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:06:06.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:06:06.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:06:06.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:06:06.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:06:06.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:06:06.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:06:06.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:06.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:06:06.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:06:06.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:06:06.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:06:06.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:06:06.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:06:06.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:06:06.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:06:06.614 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.615 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.615 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.615 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18388 3.18 3:19e2be55:::rbd_data.115e3abd3b2c.0000000000000021:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:06:06.615 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:06.615 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.355720250000001, 2024-09-14T09:06:06.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042293648000000003, 2024-09-14T09:06:06.615 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.615 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.615 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:06.615 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.615 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.615 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.616 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18388 2024-09-14T09:06:06.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.616 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:06.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:06.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220236+0000", 2024-09-14T09:06:06.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1339999999999998e-06 2024-09-14T09:06:06.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220244+0000", 2024-09-14T09:06:06.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1950000000000003e-06 2024-09-14T09:06:06.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220262+0000", 2024-09-14T09:06:06.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8017e-05 2024-09-14T09:06:06.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220270+0000", 2024-09-14T09:06:06.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9480000000000008e-06 2024-09-14T09:06:06.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220357+0000", 2024-09-14T09:06:06.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7185e-05 2024-09-14T09:06:06.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220576+0000", 2024-09-14T09:06:06.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021912200000000001 2024-09-14T09:06:06.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:06.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.222435+0000", 2024-09-14T09:06:06.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018592350000000001 2024-09-14T09:06:06.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:06.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262439+0000", 2024-09-14T09:06:06.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040003368999999997 2024-09-14T09:06:06.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:06.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262452+0000", 2024-09-14T09:06:06.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2941e-05 2024-09-14T09:06:06.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:06.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262490+0000", 2024-09-14T09:06:06.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8478e-05 2024-09-14T09:06:06.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262527+0000", 2024-09-14T09:06:06.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7024000000000003e-05 2024-09-14T09:06:06.621 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.621 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.621 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.621 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18388 3.18 e38/28 ondisk, result = 0)", 2024-09-14T09:06:06.621 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.227795+0000", 2024-09-14T09:06:06.621 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.348158606999998, 2024-09-14T09:06:06.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034648832999999997, 2024-09-14T09:06:06.621 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.621 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.621 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.621 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227795+0000", 2024-09-14T09:06:06.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227795+0000", 2024-09-14T09:06:06.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227796+0000", 2024-09-14T09:06:06.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.879999999999999e-07 2024-09-14T09:06:06.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227801+0000", 2024-09-14T09:06:06.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4909999999999999e-06 2024-09-14T09:06:06.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227815+0000", 2024-09-14T09:06:06.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4905e-05 2024-09-14T09:06:06.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.227824+0000", 2024-09-14T09:06:06.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7770000000000008e-06 2024-09-14T09:06:06.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262425+0000", 2024-09-14T09:06:06.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034601109999999997 2024-09-14T09:06:06.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262434+0000", 2024-09-14T09:06:06.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5250000000000005e-06 2024-09-14T09:06:06.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262444+0000", 2024-09-14T09:06:06.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0037000000000001e-05 2024-09-14T09:06:06.625 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.625 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.625 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.625 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:19543 3.5 e39/23 3:a536ff9b:::rbd_data.115e4c71d6e8.0000000000000002:head v 39'8, mlcod=39'6)", 2024-09-14T09:06:06.625 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:06.625 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 22.05286559, 2024-09-14T09:06:06.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.039280019999999999, 2024-09-14T09:06:06.626 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.626 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.626 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:06.626 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:06.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:06.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523090+0000", 2024-09-14T09:06:06.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.773e-06 2024-09-14T09:06:06.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537290+0000", 2024-09-14T09:06:06.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014200007000000001 2024-09-14T09:06:06.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537317+0000", 2024-09-14T09:06:06.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6704999999999999e-05 2024-09-14T09:06:06.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537328+0000", 2024-09-14T09:06:06.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1299e-05 2024-09-14T09:06:06.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537417+0000", 2024-09-14T09:06:06.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8869999999999994e-05 2024-09-14T09:06:06.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537445+0000", 2024-09-14T09:06:06.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8377999999999998e-05 2024-09-14T09:06:06.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:06.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562324+0000", 2024-09-14T09:06:06.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024879307999999999 2024-09-14T09:06:06.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562368+0000", 2024-09-14T09:06:06.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3680000000000002e-05 2024-09-14T09:06:06.629 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.629 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.630 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.630 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31015 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:06.630 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.062742+0000", 2024-09-14T09:06:06.630 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 13.513211219, 2024-09-14T09:06:06.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027690059999999999, 2024-09-14T09:06:06.630 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.630 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.630 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.630 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.630 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.630 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.631 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31015 2024-09-14T09:06:06.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.631 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062742+0000", 2024-09-14T09:06:06.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062742+0000", 2024-09-14T09:06:06.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062743+0000", 2024-09-14T09:06:06.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.51e-07 2024-09-14T09:06:06.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062745+0000", 2024-09-14T09:06:06.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0480000000000001e-06 2024-09-14T09:06:06.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062751+0000", 2024-09-14T09:06:06.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3929999999999996e-06 2024-09-14T09:06:06.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.062757+0000", 2024-09-14T09:06:06.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5130000000000004e-06 2024-09-14T09:06:06.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.088137+0000", 2024-09-14T09:06:06.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025380494999999999 2024-09-14T09:06:06.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.088178+0000", 2024-09-14T09:06:06.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0963000000000001e-05 2024-09-14T09:06:06.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.090433+0000", 2024-09-14T09:06:06.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0022541969999999999 2024-09-14T09:06:06.634 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.634 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.634 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.635 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:31025 3.1e 3:7cf18efe:::rbd_data.115ebc3c67d1.000000000000000a:c [read 292864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:06.635 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:53.063074+0000", 2024-09-14T09:06:06.635 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 13.512879952, 2024-09-14T09:06:06.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027206443, 2024-09-14T09:06:06.635 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.635 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.635 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.635 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.635 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.635 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.635 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 31025 2024-09-14T09:06:06.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.636 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063074+0000", 2024-09-14T09:06:06.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063074+0000", 2024-09-14T09:06:06.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063074+0000", 2024-09-14T09:06:06.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.68e-07 2024-09-14T09:06:06.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063077+0000", 2024-09-14T09:06:06.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.853e-06 2024-09-14T09:06:06.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063084+0000", 2024-09-14T09:06:06.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0419999999999997e-06 2024-09-14T09:06:06.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.063089+0000", 2024-09-14T09:06:06.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1150000000000002e-06 2024-09-14T09:06:06.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.090144+0000", 2024-09-14T09:06:06.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027054974999999998 2024-09-14T09:06:06.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.090211+0000", 2024-09-14T09:06:06.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6832e-05 2024-09-14T09:06:06.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:53.090280+0000", 2024-09-14T09:06:06.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9058000000000002e-05 2024-09-14T09:06:06.639 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.639 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.639 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.640 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38002 3.1e 3:7df715aa:::rbd_data.115eb07a8abd.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:06.640 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:06.640 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.1862023439999998, 2024-09-14T09:06:06.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.065148304000000004, 2024-09-14T09:06:06.640 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.640 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.640 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:06.640 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.640 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.640 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.640 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38002 2024-09-14T09:06:06.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.640 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:06.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:06.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389752+0000", 2024-09-14T09:06:06.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1829999999999999e-06 2024-09-14T09:06:06.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389758+0000", 2024-09-14T09:06:06.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0919999999999997e-06 2024-09-14T09:06:06.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389769+0000", 2024-09-14T09:06:06.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759e-05 2024-09-14T09:06:06.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389775+0000", 2024-09-14T09:06:06.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8819999999999998e-06 2024-09-14T09:06:06.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389824+0000", 2024-09-14T09:06:06.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8573999999999999e-05 2024-09-14T09:06:06.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389876+0000", 2024-09-14T09:06:06.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2643999999999999e-05 2024-09-14T09:06:06.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:06:06.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.392640+0000", 2024-09-14T09:06:06.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0027636409999999998 2024-09-14T09:06:06.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:06.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454672+0000", 2024-09-14T09:06:06.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062031703000000001 2024-09-14T09:06:06.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:06.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454765+0000", 2024-09-14T09:06:06.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2862000000000006e-05 2024-09-14T09:06:06.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:06.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454841+0000", 2024-09-14T09:06:06.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5857000000000002e-05 2024-09-14T09:06:06.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454900+0000", 2024-09-14T09:06:06.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9107000000000002e-05 2024-09-14T09:06:06.646 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.646 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.646 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.646 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:38002 3.1e e48/23 ondisk, result = 0)", 2024-09-14T09:06:06.646 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:06.646 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.1758857549999995, 2024-09-14T09:06:06.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054841741999999999, 2024-09-14T09:06:06.646 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.646 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.646 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.646 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:06.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:06.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400069+0000", 2024-09-14T09:06:06.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.339e-06 2024-09-14T09:06:06.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400076+0000", 2024-09-14T09:06:06.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7800000000000003e-06 2024-09-14T09:06:06.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400093+0000", 2024-09-14T09:06:06.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6915999999999999e-05 2024-09-14T09:06:06.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400106+0000", 2024-09-14T09:06:06.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2785e-05 2024-09-14T09:06:06.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454713+0000", 2024-09-14T09:06:06.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054607041000000002 2024-09-14T09:06:06.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454724+0000", 2024-09-14T09:06:06.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1202e-05 2024-09-14T09:06:06.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454910+0000", 2024-09-14T09:06:06.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018567900000000001 2024-09-14T09:06:06.650 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.650 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.650 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.650 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38100 3.b 3:d0f67c24:::rbd_data.115eb07a8abd.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:06.650 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:06.650 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.9300320949999996, 2024-09-14T09:06:06.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053701964999999997, 2024-09-14T09:06:06.650 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.650 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.650 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:06.651 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.651 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.651 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.651 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38100 2024-09-14T09:06:06.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.651 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:06.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:06.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645923+0000", 2024-09-14T09:06:06.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:06:06.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645928+0000", 2024-09-14T09:06:06.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0699999999999997e-06 2024-09-14T09:06:06.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645942+0000", 2024-09-14T09:06:06.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4069e-05 2024-09-14T09:06:06.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645951+0000", 2024-09-14T09:06:06.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3230000000000004e-06 2024-09-14T09:06:06.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646008+0000", 2024-09-14T09:06:06.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6583999999999999e-05 2024-09-14T09:06:06.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646089+0000", 2024-09-14T09:06:06.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1557999999999994e-05 2024-09-14T09:06:06.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:06.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.650355+0000", 2024-09-14T09:06:06.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004265525 2024-09-14T09:06:06.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:06.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699439+0000", 2024-09-14T09:06:06.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049083904999999997 2024-09-14T09:06:06.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:06.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699467+0000", 2024-09-14T09:06:06.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8198999999999999e-05 2024-09-14T09:06:06.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:06.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699552+0000", 2024-09-14T09:06:06.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T09:06:06.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699624+0000", 2024-09-14T09:06:06.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1725999999999997e-05 2024-09-14T09:06:06.656 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.656 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.656 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.656 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:38100 3.b e48/28 ondisk, result = 0)", 2024-09-14T09:06:06.656 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.669267+0000", 2024-09-14T09:06:06.656 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.9066864639999999, 2024-09-14T09:06:06.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030186418999999999, 2024-09-14T09:06:06.656 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.656 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.657 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.657 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.669267+0000", 2024-09-14T09:06:06.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.669267+0000", 2024-09-14T09:06:06.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.669270+0000", 2024-09-14T09:06:06.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3269999999999999e-06 2024-09-14T09:06:06.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.669284+0000", 2024-09-14T09:06:06.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4662e-05 2024-09-14T09:06:06.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.669317+0000", 2024-09-14T09:06:06.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3080000000000002e-05 2024-09-14T09:06:06.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.669334+0000", 2024-09-14T09:06:06.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6376999999999999e-05 2024-09-14T09:06:06.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.659 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699411+0000", 2024-09-14T09:06:06.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030077653999999999 2024-09-14T09:06:06.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.659 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699429+0000", 2024-09-14T09:06:06.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7723e-05 2024-09-14T09:06:06.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699454+0000", 2024-09-14T09:06:06.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4596e-05 2024-09-14T09:06:06.660 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.660 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.660 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.660 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39205 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:06:06.660 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:05.751227+0000", 2024-09-14T09:06:06.660 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.82472709, 2024-09-14T09:06:06.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.032126633000000002, 2024-09-14T09:06:06.660 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.660 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.660 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:06.661 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.661 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.661 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.661 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39205 2024-09-14T09:06:06.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.661 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.751227+0000", 2024-09-14T09:06:06.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.751227+0000", 2024-09-14T09:06:06.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.751228+0000", 2024-09-14T09:06:06.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6500000000000001e-06 2024-09-14T09:06:06.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.752953+0000", 2024-09-14T09:06:06.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001724555 2024-09-14T09:06:06.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.752965+0000", 2024-09-14T09:06:06.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2464e-05 2024-09-14T09:06:06.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.752973+0000", 2024-09-14T09:06:06.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9389999999999997e-06 2024-09-14T09:06:06.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.753042+0000", 2024-09-14T09:06:06.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8534999999999993e-05 2024-09-14T09:06:06.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.753192+0000", 2024-09-14T09:06:06.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000150421 2024-09-14T09:06:06.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:06:06.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.753604+0000", 2024-09-14T09:06:06.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00041139300000000002 2024-09-14T09:06:06.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:06.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.782969+0000", 2024-09-14T09:06:06.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029365324000000002 2024-09-14T09:06:06.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:06.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.783228+0000", 2024-09-14T09:06:06.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025916699999999998 2024-09-14T09:06:06.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:06.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.783296+0000", 2024-09-14T09:06:06.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8464e-05 2024-09-14T09:06:06.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.783353+0000", 2024-09-14T09:06:06.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6721000000000001e-05 2024-09-14T09:06:06.666 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.666 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.666 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.666 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39208 3.1e 3:7b420df9:::rbd_data.115ea0ee3dc0.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:06:06.666 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:05.753678+0000", 2024-09-14T09:06:06.666 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.82227602799999999, 2024-09-14T09:06:06.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036224272000000002, 2024-09-14T09:06:06.667 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.667 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.667 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:06.667 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.667 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.667 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.667 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39208 2024-09-14T09:06:06.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.667 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.753678+0000", 2024-09-14T09:06:06.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.753678+0000", 2024-09-14T09:06:06.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.753679+0000", 2024-09-14T09:06:06.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7430000000000001e-06 2024-09-14T09:06:06.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.754838+0000", 2024-09-14T09:06:06.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0011589860000000001 2024-09-14T09:06:06.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.754874+0000", 2024-09-14T09:06:06.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5976000000000002e-05 2024-09-14T09:06:06.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.754890+0000", 2024-09-14T09:06:06.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5906999999999999e-05 2024-09-14T09:06:06.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.755143+0000", 2024-09-14T09:06:06.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000253057 2024-09-14T09:06:06.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.755270+0000", 2024-09-14T09:06:06.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012716999999999999 2024-09-14T09:06:06.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:06:06.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.756130+0000", 2024-09-14T09:06:06.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00085996599999999999 2024-09-14T09:06:06.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:06.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.775332+0000", 2024-09-14T09:06:06.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019201218999999999 2024-09-14T09:06:06.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:06.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.789791+0000", 2024-09-14T09:06:06.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014459394 2024-09-14T09:06:06.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:06.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.789856+0000", 2024-09-14T09:06:06.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4794000000000003e-05 2024-09-14T09:06:06.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:05.789902+0000", 2024-09-14T09:06:06.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6060000000000003e-05 2024-09-14T09:06:06.672 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.672 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.673 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.673 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:06:06.673 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:06.673 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.41534157399999999, 2024-09-14T09:06:06.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:06:06.673 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.673 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.674 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:06.674 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.674 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.674 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.674 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:06:06.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.674 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:06.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:06.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:06:06.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:06:06.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:06:06.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:06:06.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:06:06.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:06:06.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:06:06.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:06:06.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:06:06.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:06:06.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:06:06.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:06:06.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:06:06.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:06:06.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:06:06.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:06.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:06:06.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:06:06.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:06.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:06:06.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:06:06.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:06.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:06:06.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:06:06.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:06:06.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:06:06.679 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.679 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.679 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.680 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:39401 3.b e56/53)", 2024-09-14T09:06:06.680 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:06.680 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.389445508, 2024-09-14T09:06:06.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10443659600000001, 2024-09-14T09:06:06.680 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.680 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.680 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:06.680 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:06.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:06.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186509+0000", 2024-09-14T09:06:06.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2640000000000001e-06 2024-09-14T09:06:06.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186516+0000", 2024-09-14T09:06:06.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1070000000000001e-06 2024-09-14T09:06:06.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186533+0000", 2024-09-14T09:06:06.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7818000000000001e-05 2024-09-14T09:06:06.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186543+0000", 2024-09-14T09:06:06.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5170000000000005e-06 2024-09-14T09:06:06.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290885+0000", 2024-09-14T09:06:06.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.104342221 2024-09-14T09:06:06.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290893+0000", 2024-09-14T09:06:06.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2859999999999999e-06 2024-09-14T09:06:06.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290945+0000", 2024-09-14T09:06:06.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1382999999999999e-05 2024-09-14T09:06:06.683 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.683 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.683 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.683 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39451 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:06:06.683 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.393549+0000", 2024-09-14T09:06:06.684 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.18240458800000001, 2024-09-14T09:06:06.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029697604999999998, 2024-09-14T09:06:06.684 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:06.684 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:06.684 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:06.684 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:06.684 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:06.684 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:06.684 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39451 2024-09-14T09:06:06.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.684 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:06.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:06.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.393549+0000", 2024-09-14T09:06:06.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:06.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.393549+0000", 2024-09-14T09:06:06.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:06.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:06.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.393551+0000", 2024-09-14T09:06:06.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.939e-06 2024-09-14T09:06:06.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:06.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.393558+0000", 2024-09-14T09:06:06.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4329999999999998e-06 2024-09-14T09:06:06.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:06.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.393575+0000", 2024-09-14T09:06:06.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6220999999999999e-05 2024-09-14T09:06:06.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:06.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.393582+0000", 2024-09-14T09:06:06.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2049999999999996e-06 2024-09-14T09:06:06.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:06.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.393687+0000", 2024-09-14T09:06:06.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010504 2024-09-14T09:06:06.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:06.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.393819+0000", 2024-09-14T09:06:06.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013160900000000001 2024-09-14T09:06:06.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:06:06.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.397256+0000", 2024-09-14T09:06:06.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0034377309999999999 2024-09-14T09:06:06.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:06.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.423069+0000", 2024-09-14T09:06:06.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025812261999999999 2024-09-14T09:06:06.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:06.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.423129+0000", 2024-09-14T09:06:06.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0782999999999997e-05 2024-09-14T09:06:06.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:06.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.423195+0000", 2024-09-14T09:06:06.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5772000000000007e-05 2024-09-14T09:06:06.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:06.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:06.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:06.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.423247+0000", 2024-09-14T09:06:06.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1610000000000002e-05 2024-09-14T09:06:06.689 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.689 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.689 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.689 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:06.689 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:06.689 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:06.690 INFO:teuthology.orchestra.run.smithi176.stdout:1667 trunc from 0xe3b43fe to 0x1d2b6e3 2024-09-14T09:06:06.691 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:06:06.693 INFO:teuthology.orchestra.run.smithi176.stdout:1670 writesame 0x910a827 thru 0x9115349 data_size 0x1(0xab23 bytes) 2024-09-14T09:06:06.693 INFO:teuthology.orchestra.run.smithi176.stdout:1671 read 0xd7239e thru 0xd74c27 (0x288a bytes) 2024-09-14T09:06:06.693 INFO:teuthology.orchestra.run.smithi176.stdout:1672 read 0x8f5ebfa thru 0x8f6cbb9 (0xdfc0 bytes) 2024-09-14T09:06:06.693 INFO:teuthology.orchestra.run.smithi176.stdout:1673 punch from 0x7ec6186 to 0x7ec6dd2, (0xc4c bytes) 2024-09-14T09:06:06.693 INFO:teuthology.orchestra.run.smithi176.stdout:1674 read 0x53ac51e thru 0x53b32b1 (0x6d94 bytes) 2024-09-14T09:06:06.693 INFO:teuthology.orchestra.run.smithi176.stdout:1675 punch from 0x6f56f93 to 0x6f62e11, (0xbe7e bytes) 2024-09-14T09:06:06.693 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.693 INFO:teuthology.orchestra.run.smithi176.stdout:1678 write 0x792b3e6 thru 0x7936ea2 (0xbabd bytes) 2024-09-14T09:06:06.693 INFO:teuthology.orchestra.run.smithi176.stdout:1679 write 0xeae2272 thru 0xeae5705 (0x3494 bytes) 2024-09-14T09:06:06.694 INFO:teuthology.orchestra.run.smithi176.stdout:1680 read 0xbce90a thru 0xbdc208 (0xd8ff bytes) 2024-09-14T09:06:06.694 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.694 INFO:teuthology.orchestra.run.smithi176.stdout:1683 write 0x364b138 thru 0x365414f (0x9018 bytes) 2024-09-14T09:06:06.694 INFO:teuthology.orchestra.run.smithi176.stdout:1684 read 0x2b73480 thru 0x2b7bab8 (0x8639 bytes) 2024-09-14T09:06:06.694 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.694 INFO:teuthology.orchestra.run.smithi176.stdout:1686 punch from 0x904dd86 to 0x9058468, (0xa6e2 bytes) 2024-09-14T09:06:06.694 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.694 INFO:teuthology.orchestra.run.smithi176.stdout:1689 punch from 0x99b965e to 0x99c21af, (0x8b51 bytes) 2024-09-14T09:06:06.694 INFO:teuthology.orchestra.run.smithi176.stdout:1690 read 0xe7d6b38 thru 0xe7e11ba (0xa683 bytes) 2024-09-14T09:06:06.694 INFO:teuthology.orchestra.run.smithi176.stdout:1692 write 0x8cff422 thru 0x8d05142 (0x5d21 bytes) 2024-09-14T09:06:06.694 INFO:teuthology.orchestra.run.smithi176.stdout:1693 write 0xb2c24d0 thru 0xb2cf150 (0xcc81 bytes) 2024-09-14T09:06:06.695 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:06.691+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:06.695 INFO:teuthology.orchestra.run.smithi176.stdout:1695 read 0x5ecc7b1 thru 0x5ecf435 (0x2c85 bytes) 2024-09-14T09:06:06.696 INFO:teuthology.orchestra.run.smithi176.stdout:1696 write 0x330fc0e thru 0x331e56b (0xe95e bytes) 2024-09-14T09:06:06.706 INFO:teuthology.orchestra.run.smithi176.stdout:1697 read 0x2fdea3 thru 0x30c2fb (0xe459 bytes) 2024-09-14T09:06:06.708 INFO:teuthology.orchestra.run.smithi176.stdout:1698 read 0x587304c thru 0x5874031 (0xfe6 bytes) 2024-09-14T09:06:06.709 INFO:teuthology.orchestra.run.smithi176.stdout:1699 punch from 0x69f2796 to 0x69f8583, (0x5ded bytes) 2024-09-14T09:06:06.709 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.724 INFO:teuthology.orchestra.run.smithi176.stdout:1702 trunc from 0xeae5706 to 0x94969c6 2024-09-14T09:06:06.768 INFO:teuthology.orchestra.run.smithi176.stdout:1704 write 0xbceb252 thru 0xbcfada5 (0xfb54 bytes) 2024-09-14T09:06:06.775 INFO:teuthology.orchestra.run.smithi176.stdout:1705 write 0x9cf0c8f thru 0x9d00a6c (0xfdde bytes) 2024-09-14T09:06:06.785 INFO:teuthology.orchestra.run.smithi176.stdout:1706 read 0xa7452d8 thru 0xa75285b (0xd584 bytes) 2024-09-14T09:06:06.785 INFO:teuthology.orchestra.run.smithi176.stdout:1708 read 0x86e8dd0 thru 0x86e9b45 (0xd76 bytes) 2024-09-14T09:06:06.787 INFO:teuthology.orchestra.run.smithi176.stdout:1709 trunc from 0xbcfada6 to 0xc46bd31 2024-09-14T09:06:06.791 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:06.788+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:06.800 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.800 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.800 INFO:teuthology.orchestra.run.smithi176.stdout:1712 read 0x4337ac8 thru 0x433c16a (0x46a3 bytes) 2024-09-14T09:06:06.830 INFO:teuthology.orchestra.run.smithi176.stdout:1713 trunc from 0xc46bd31 to 0x1192ebf 2024-09-14T09:06:06.831 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:06.831 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:06.831 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:06.831 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:06.846 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:06:06.855 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.887 INFO:teuthology.orchestra.run.smithi176.stdout:1717 write 0x9d72fcb thru 0x9d81ba8 (0xebde bytes) 2024-09-14T09:06:06.890 INFO:teuthology.orchestra.run.smithi176.stdout:1722 read 0x9a3964b thru 0x9a48852 (0xf208 bytes) 2024-09-14T09:06:06.891 INFO:teuthology.orchestra.run.smithi176.stdout:1723 read 0x98e1f32 thru 0x98ebbc1 (0x9c90 bytes) 2024-09-14T09:06:06.892 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:06.895+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:06.892 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.892 INFO:teuthology.orchestra.run.smithi176.stdout:1725 write 0x4f8e58c thru 0x4f9266a (0x40df bytes) 2024-09-14T09:06:06.896 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.897 INFO:teuthology.orchestra.run.smithi176.stdout:1729 read 0x970bd0b thru 0x9718a68 (0xcd5e bytes) 2024-09-14T09:06:06.897 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.897 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.910 INFO:teuthology.orchestra.run.smithi176.stdout:1733 write 0xaec5c7b thru 0xaed4189 (0xe50f bytes) 2024-09-14T09:06:06.917 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.918 INFO:teuthology.orchestra.run.smithi176.stdout:1735 writesame 0x3ccb25a thru 0x3ccf9b3 data_size 0x1(0x475a bytes) 2024-09-14T09:06:06.950 INFO:teuthology.orchestra.run.smithi176.stdout:1737 punch from 0x163ab71 to 0x164426b, (0x96fa bytes) 2024-09-14T09:06:06.950 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:06.950 INFO:teuthology.orchestra.run.smithi176.stdout:1740 read 0x7f2b531 thru 0x7f30368 (0x4e38 bytes) 2024-09-14T09:06:06.965 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:06.965 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:06.965 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:06.965 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:06.965 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:06.970 INFO:teuthology.orchestra.run.smithi176.stdout:1743 trunc from 0xaed418a to 0x3d99af7 2024-09-14T09:06:06.979 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:06:06.993 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:06.988+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:06.995 INFO:teuthology.orchestra.run.smithi176.stdout:1744 read 0x1f67558 thru 0x1f7654d (0xeff6 bytes) 2024-09-14T09:06:06.996 INFO:teuthology.orchestra.run.smithi176.stdout:1746 write 0x19b9fd1 thru 0x19c378a (0x97ba bytes) 2024-09-14T09:06:07.004 INFO:teuthology.orchestra.run.smithi176.stdout:1750 read 0x25998b9 thru 0x25a4c6c (0xb3b4 bytes) 2024-09-14T09:06:07.005 INFO:teuthology.orchestra.run.smithi176.stdout:1751 punch from 0x1860f74 to 0x1864b2c, (0x3bb8 bytes) 2024-09-14T09:06:07.005 INFO:teuthology.orchestra.run.smithi176.stdout:1752 read 0x35789b9 thru 0x357a3e4 (0x1a2c bytes) 2024-09-14T09:06:07.005 INFO:teuthology.orchestra.run.smithi176.stdout:1754 read 0x23c7e6a thru 0x23c7f72 (0x109 bytes) 2024-09-14T09:06:07.006 INFO:teuthology.orchestra.run.smithi176.stdout:1756 read 0x2570743 thru 0x2572794 (0x2052 bytes) 2024-09-14T09:06:07.007 INFO:teuthology.orchestra.run.smithi176.stdout:1757 read 0x16462f5 thru 0x164e4b9 (0x81c5 bytes) 2024-09-14T09:06:07.048 INFO:teuthology.orchestra.run.smithi176.stdout:1758 writesame 0xed9fd68 thru 0xeda08d5 data_size 0x16(0xb6e bytes) 2024-09-14T09:06:07.048 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:07.048 INFO:teuthology.orchestra.run.smithi176.stdout:1761 punch from 0x15b08be to 0x15bac2e, (0xa370 bytes) 2024-09-14T09:06:07.085 INFO:teuthology.orchestra.run.smithi176.stdout:1762 trunc from 0xeda08d6 to 0x14a5f5 2024-09-14T09:06:07.094 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:07.091+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:07.103 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T09:06:07.152 INFO:teuthology.orchestra.run.smithi176.stdout:1763 write 0xb308426 thru 0xb317518 (0xf0f3 bytes) 2024-09-14T09:06:07.157 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:07.158 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:07.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:07.158 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:07.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.158 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:654 3.11 e29/23 3:8c8e358b:::rbd_object_map.115e1e95bf99:head v 29'3, mlcod=28'1)", 2024-09-14T09:06:07.158 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.863095+0000", 2024-09-14T09:06:07.158 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.295475340000003, 2024-09-14T09:06:07.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0070911430000000003, 2024-09-14T09:06:07.158 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.158 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.158 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:07.158 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863095+0000", 2024-09-14T09:06:07.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863095+0000", 2024-09-14T09:06:07.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863096+0000", 2024-09-14T09:06:07.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8700000000000004e-07 2024-09-14T09:06:07.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863110+0000", 2024-09-14T09:06:07.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4283e-05 2024-09-14T09:06:07.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863157+0000", 2024-09-14T09:06:07.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6029999999999998e-05 2024-09-14T09:06:07.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863177+0000", 2024-09-14T09:06:07.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0662999999999999e-05 2024-09-14T09:06:07.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863324+0000", 2024-09-14T09:06:07.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014713899999999999 2024-09-14T09:06:07.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863383+0000", 2024-09-14T09:06:07.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8218999999999999e-05 2024-09-14T09:06:07.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:07.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.870083+0000", 2024-09-14T09:06:07.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067001459999999997 2024-09-14T09:06:07.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.870186+0000", 2024-09-14T09:06:07.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103776 2024-09-14T09:06:07.162 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.162 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.162 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.163 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:758 3.14 3:2d2c8a97:::rbd_data.115e724e1d99.0000000000000007:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:06:07.163 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.888858+0000", 2024-09-14T09:06:07.163 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.269712359000003, 2024-09-14T09:06:07.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060111740000000002, 2024-09-14T09:06:07.163 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.163 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.163 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.163 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.163 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.163 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.163 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 758 2024-09-14T09:06:07.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.163 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888858+0000", 2024-09-14T09:06:07.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888858+0000", 2024-09-14T09:06:07.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888860+0000", 2024-09-14T09:06:07.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4529999999999999e-06 2024-09-14T09:06:07.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888867+0000", 2024-09-14T09:06:07.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7200000000000006e-06 2024-09-14T09:06:07.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888892+0000", 2024-09-14T09:06:07.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4729999999999999e-05 2024-09-14T09:06:07.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888927+0000", 2024-09-14T09:06:07.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4752999999999997e-05 2024-09-14T09:06:07.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.894354+0000", 2024-09-14T09:06:07.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054273209999999997 2024-09-14T09:06:07.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.894584+0000", 2024-09-14T09:06:07.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000229602 2024-09-14T09:06:07.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.894869+0000", 2024-09-14T09:06:07.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028559499999999998 2024-09-14T09:06:07.167 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.167 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.167 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.167 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:829 3.13 e29/23 3:cc36a41f:::rbd_data.115e1e95bf99.0000000000000000:head v 29'1, mlcod=0'0)", 2024-09-14T09:06:07.167 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.901655+0000", 2024-09-14T09:06:07.167 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.256915472999999, 2024-09-14T09:06:07.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021644360000000001, 2024-09-14T09:06:07.168 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.168 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.168 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:07.168 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901655+0000", 2024-09-14T09:06:07.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901655+0000", 2024-09-14T09:06:07.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901657+0000", 2024-09-14T09:06:07.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.153e-06 2024-09-14T09:06:07.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901897+0000", 2024-09-14T09:06:07.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000239461 2024-09-14T09:06:07.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901927+0000", 2024-09-14T09:06:07.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.029e-05 2024-09-14T09:06:07.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901946+0000", 2024-09-14T09:06:07.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9170000000000001e-05 2024-09-14T09:06:07.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902104+0000", 2024-09-14T09:06:07.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015812199999999999 2024-09-14T09:06:07.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902157+0000", 2024-09-14T09:06:07.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2905999999999998e-05 2024-09-14T09:06:07.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:07.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923226+0000", 2024-09-14T09:06:07.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021068470999999998 2024-09-14T09:06:07.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923300+0000", 2024-09-14T09:06:07.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3787000000000004e-05 2024-09-14T09:06:07.171 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.171 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.172 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.172 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:845 3.9 3:928dad78:::rbd_data.115e1e95bf99.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13369344~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:06:07.172 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.922336+0000", 2024-09-14T09:06:07.172 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.236234555999999, 2024-09-14T09:06:07.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010311885999999999, 2024-09-14T09:06:07.172 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.172 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.172 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:07.172 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.172 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.172 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.173 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 845 2024-09-14T09:06:07.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.173 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922336+0000", 2024-09-14T09:06:07.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922336+0000", 2024-09-14T09:06:07.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922339+0000", 2024-09-14T09:06:07.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6879999999999999e-06 2024-09-14T09:06:07.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922350+0000", 2024-09-14T09:06:07.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1457999999999999e-05 2024-09-14T09:06:07.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922376+0000", 2024-09-14T09:06:07.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5301999999999999e-05 2024-09-14T09:06:07.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922390+0000", 2024-09-14T09:06:07.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4404e-05 2024-09-14T09:06:07.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922506+0000", 2024-09-14T09:06:07.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011567599999999999 2024-09-14T09:06:07.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922741+0000", 2024-09-14T09:06:07.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00023530600000000001 2024-09-14T09:06:07.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-09-14T09:06:07.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923197+0000", 2024-09-14T09:06:07.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00045574899999999998 2024-09-14T09:06:07.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:07.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926027+0000", 2024-09-14T09:06:07.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002830819 2024-09-14T09:06:07.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:07.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932530+0000", 2024-09-14T09:06:07.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065027139999999997 2024-09-14T09:06:07.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:07.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932583+0000", 2024-09-14T09:06:07.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2342000000000001e-05 2024-09-14T09:06:07.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932648+0000", 2024-09-14T09:06:07.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5427999999999998e-05 2024-09-14T09:06:07.178 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.178 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.178 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.178 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:842 3.b e29/28 3:d50f9298:::rbd_data.115e1e95bf99.0000000000000004:head v 29'5, mlcod=26'3)", 2024-09-14T09:06:07.178 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.922344+0000", 2024-09-14T09:06:07.178 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.236226805999998, 2024-09-14T09:06:07.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062661460000000002, 2024-09-14T09:06:07.179 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.179 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.179 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:07.179 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922344+0000", 2024-09-14T09:06:07.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922344+0000", 2024-09-14T09:06:07.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922345+0000", 2024-09-14T09:06:07.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.288e-06 2024-09-14T09:06:07.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922356+0000", 2024-09-14T09:06:07.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1048e-05 2024-09-14T09:06:07.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922371+0000", 2024-09-14T09:06:07.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4338000000000001e-05 2024-09-14T09:06:07.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922382+0000", 2024-09-14T09:06:07.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1e-05 2024-09-14T09:06:07.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922467+0000", 2024-09-14T09:06:07.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5721000000000001e-05 2024-09-14T09:06:07.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922489+0000", 2024-09-14T09:06:07.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2181999999999999e-05 2024-09-14T09:06:07.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:07.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.928551+0000", 2024-09-14T09:06:07.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060616100000000003 2024-09-14T09:06:07.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.928610+0000", 2024-09-14T09:06:07.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8959000000000003e-05 2024-09-14T09:06:07.183 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.183 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.183 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.183 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:848 3.1a 3:5ade8e73:::rbd_data.115e1e95bf99.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13369344~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:06:07.183 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.922902+0000", 2024-09-14T09:06:07.183 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.235669143999999, 2024-09-14T09:06:07.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096031020000000005, 2024-09-14T09:06:07.183 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.183 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.184 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:07.184 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.184 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.184 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.184 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 848 2024-09-14T09:06:07.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.184 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922902+0000", 2024-09-14T09:06:07.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922902+0000", 2024-09-14T09:06:07.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922903+0000", 2024-09-14T09:06:07.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6419999999999999e-06 2024-09-14T09:06:07.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923127+0000", 2024-09-14T09:06:07.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000223997 2024-09-14T09:06:07.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923151+0000", 2024-09-14T09:06:07.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3802e-05 2024-09-14T09:06:07.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923169+0000", 2024-09-14T09:06:07.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7858000000000001e-05 2024-09-14T09:06:07.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923307+0000", 2024-09-14T09:06:07.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013802399999999999 2024-09-14T09:06:07.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923560+0000", 2024-09-14T09:06:07.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00025296500000000001 2024-09-14T09:06:07.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:07.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923925+0000", 2024-09-14T09:06:07.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036534700000000002 2024-09-14T09:06:07.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:07.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.927231+0000", 2024-09-14T09:06:07.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033055820000000001 2024-09-14T09:06:07.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:07.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932287+0000", 2024-09-14T09:06:07.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0050562810000000001 2024-09-14T09:06:07.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:07.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932395+0000", 2024-09-14T09:06:07.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000108001 2024-09-14T09:06:07.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932505+0000", 2024-09-14T09:06:07.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109603 2024-09-14T09:06:07.189 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.189 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.189 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.189 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:847 3.13 e29/23 3:cc047b4a:::rbd_data.115e1e95bf99.0000000000000009:head v 29'2, mlcod=0'0)", 2024-09-14T09:06:07.189 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.924087+0000", 2024-09-14T09:06:07.189 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.234483382000001, 2024-09-14T09:06:07.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0085507929999999992, 2024-09-14T09:06:07.190 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.190 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.190 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:07.190 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924087+0000", 2024-09-14T09:06:07.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924087+0000", 2024-09-14T09:06:07.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924089+0000", 2024-09-14T09:06:07.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8050000000000001e-06 2024-09-14T09:06:07.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924157+0000", 2024-09-14T09:06:07.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7999999999999999e-05 2024-09-14T09:06:07.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924176+0000", 2024-09-14T09:06:07.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9312000000000001e-05 2024-09-14T09:06:07.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924191+0000", 2024-09-14T09:06:07.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4707e-05 2024-09-14T09:06:07.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924292+0000", 2024-09-14T09:06:07.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010052000000000001 2024-09-14T09:06:07.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924328+0000", 2024-09-14T09:06:07.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6211999999999998e-05 2024-09-14T09:06:07.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:07.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932564+0000", 2024-09-14T09:06:07.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0082365960000000005 2024-09-14T09:06:07.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932638+0000", 2024-09-14T09:06:07.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3640999999999995e-05 2024-09-14T09:06:07.193 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.193 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.194 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.194 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:849 3.b e29/28 3:d70d4fef:::rbd_data.115e1e95bf99.000000000000000b:head v 29'6, mlcod=26'3)", 2024-09-14T09:06:07.194 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.924304+0000", 2024-09-14T09:06:07.194 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.234266423000001, 2024-09-14T09:06:07.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083878719999999993, 2024-09-14T09:06:07.194 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.194 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.194 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:07.194 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924304+0000", 2024-09-14T09:06:07.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924304+0000", 2024-09-14T09:06:07.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924305+0000", 2024-09-14T09:06:07.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.192e-06 2024-09-14T09:06:07.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924676+0000", 2024-09-14T09:06:07.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037050700000000001 2024-09-14T09:06:07.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924698+0000", 2024-09-14T09:06:07.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1977999999999999e-05 2024-09-14T09:06:07.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924712+0000", 2024-09-14T09:06:07.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4477e-05 2024-09-14T09:06:07.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926069+0000", 2024-09-14T09:06:07.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013562330000000001 2024-09-14T09:06:07.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926105+0000", 2024-09-14T09:06:07.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6779999999999997e-05 2024-09-14T09:06:07.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:07.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932643+0000", 2024-09-14T09:06:07.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065378290000000002 2024-09-14T09:06:07.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932692+0000", 2024-09-14T09:06:07.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8875999999999998e-05 2024-09-14T09:06:07.198 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.198 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.199 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.199 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:850 3.11 e29/23 3:892642aa:::rbd_data.115e1e95bf99.000000000000000c:head v 29'18, mlcod=29'15)", 2024-09-14T09:06:07.199 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.926814+0000", 2024-09-14T09:06:07.199 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.23175655, 2024-09-14T09:06:07.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0064716569999999996, 2024-09-14T09:06:07.200 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.200 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.200 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:07.200 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926814+0000", 2024-09-14T09:06:07.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926814+0000", 2024-09-14T09:06:07.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926816+0000", 2024-09-14T09:06:07.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.491e-06 2024-09-14T09:06:07.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926828+0000", 2024-09-14T09:06:07.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2288e-05 2024-09-14T09:06:07.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926851+0000", 2024-09-14T09:06:07.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2674999999999999e-05 2024-09-14T09:06:07.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926865+0000", 2024-09-14T09:06:07.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.449e-05 2024-09-14T09:06:07.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932217+0000", 2024-09-14T09:06:07.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053523370000000004 2024-09-14T09:06:07.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932241+0000", 2024-09-14T09:06:07.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3210999999999999e-05 2024-09-14T09:06:07.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:07.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.933239+0000", 2024-09-14T09:06:07.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00099850599999999992 2024-09-14T09:06:07.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.933286+0000", 2024-09-14T09:06:07.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6659000000000002e-05 2024-09-14T09:06:07.207 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.207 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.207 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.207 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:9982 3.1a 3:5af3e476:::rbd_data.115eadec874.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:06:07.207 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:07.208 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.56060123, 2024-09-14T09:06:07.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042276341000000002, 2024-09-14T09:06:07.208 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.208 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.208 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:07.208 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.208 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.208 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.208 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 9982 2024-09-14T09:06:07.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.208 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:07.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:07.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597970+0000", 2024-09-14T09:06:07.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0599999999999999e-07 2024-09-14T09:06:07.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597974+0000", 2024-09-14T09:06:07.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2649999999999998e-06 2024-09-14T09:06:07.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597986+0000", 2024-09-14T09:06:07.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1019000000000001e-05 2024-09-14T09:06:07.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597991+0000", 2024-09-14T09:06:07.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5049999999999998e-06 2024-09-14T09:06:07.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.598083+0000", 2024-09-14T09:06:07.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2062e-05 2024-09-14T09:06:07.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.598227+0000", 2024-09-14T09:06:07.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000143609 2024-09-14T09:06:07.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:07.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.603107+0000", 2024-09-14T09:06:07.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048805819999999996 2024-09-14T09:06:07.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:07.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640002+0000", 2024-09-14T09:06:07.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.036894641999999998 2024-09-14T09:06:07.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:07.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640077+0000", 2024-09-14T09:06:07.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4927999999999998e-05 2024-09-14T09:06:07.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:07.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640174+0000", 2024-09-14T09:06:07.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7417999999999995e-05 2024-09-14T09:06:07.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640246+0000", 2024-09-14T09:06:07.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1505000000000001e-05 2024-09-14T09:06:07.213 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.213 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.213 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.213 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:9982 3.1a e32/23 ondisk, result = 0)", 2024-09-14T09:06:07.214 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.609682+0000", 2024-09-14T09:06:07.214 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.548888621, 2024-09-14T09:06:07.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.030575755, 2024-09-14T09:06:07.214 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.214 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.214 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.214 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609682+0000", 2024-09-14T09:06:07.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609682+0000", 2024-09-14T09:06:07.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609683+0000", 2024-09-14T09:06:07.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0399999999999995e-07 2024-09-14T09:06:07.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609687+0000", 2024-09-14T09:06:07.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3460000000000004e-06 2024-09-14T09:06:07.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609696+0000", 2024-09-14T09:06:07.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0550000000000005e-06 2024-09-14T09:06:07.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609702+0000", 2024-09-14T09:06:07.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.626e-06 2024-09-14T09:06:07.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640050+0000", 2024-09-14T09:06:07.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.030348671000000001 2024-09-14T09:06:07.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640066+0000", 2024-09-14T09:06:07.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5617999999999999e-05 2024-09-14T09:06:07.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640258+0000", 2024-09-14T09:06:07.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019173500000000001 2024-09-14T09:06:07.218 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.218 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.218 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.218 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:10031 3.1b 3:df00a663:::rbd_data.115eadec874.0000000000000005:head [stat,write 16619424~53229] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:06:07.218 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.768778+0000", 2024-09-14T09:06:07.218 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.389793109999999, 2024-09-14T09:06:07.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0078922009999999997, 2024-09-14T09:06:07.218 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.218 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.218 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:07.218 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.218 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.219 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.219 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 10031 2024-09-14T09:06:07.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.219 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768778+0000", 2024-09-14T09:06:07.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768778+0000", 2024-09-14T09:06:07.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768780+0000", 2024-09-14T09:06:07.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7130000000000001e-06 2024-09-14T09:06:07.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768824+0000", 2024-09-14T09:06:07.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3989999999999997e-05 2024-09-14T09:06:07.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768854+0000", 2024-09-14T09:06:07.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9937000000000002e-05 2024-09-14T09:06:07.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768869+0000", 2024-09-14T09:06:07.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5233e-05 2024-09-14T09:06:07.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768979+0000", 2024-09-14T09:06:07.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109429 2024-09-14T09:06:07.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.769124+0000", 2024-09-14T09:06:07.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014520200000000001 2024-09-14T09:06:07.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:07.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.769351+0000", 2024-09-14T09:06:07.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000227259 2024-09-14T09:06:07.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:07.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.776133+0000", 2024-09-14T09:06:07.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067818979999999997 2024-09-14T09:06:07.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:07.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.776464+0000", 2024-09-14T09:06:07.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033063400000000001 2024-09-14T09:06:07.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:07.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.776553+0000", 2024-09-14T09:06:07.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8793000000000002e-05 2024-09-14T09:06:07.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.776670+0000", 2024-09-14T09:06:07.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000117113 2024-09-14T09:06:07.223 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.223 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.224 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.224 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:10039 3.f e33/23 3:f6f4e415:::rbd_data.115eadec874.0000000000000000:head v 33'8, mlcod=32'6)", 2024-09-14T09:06:07.224 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.951357+0000", 2024-09-14T09:06:07.224 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.207213968000001, 2024-09-14T09:06:07.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067664509999999997, 2024-09-14T09:06:07.224 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.224 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.224 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:07.224 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.224 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.224 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951357+0000", 2024-09-14T09:06:07.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951357+0000", 2024-09-14T09:06:07.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951357+0000", 2024-09-14T09:06:07.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4000000000000001e-07 2024-09-14T09:06:07.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951363+0000", 2024-09-14T09:06:07.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4999999999999999e-06 2024-09-14T09:06:07.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951376+0000", 2024-09-14T09:06:07.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3315e-05 2024-09-14T09:06:07.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951384+0000", 2024-09-14T09:06:07.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6590000000000007e-06 2024-09-14T09:06:07.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951470+0000", 2024-09-14T09:06:07.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6198000000000004e-05 2024-09-14T09:06:07.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951493+0000", 2024-09-14T09:06:07.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2668999999999999e-05 2024-09-14T09:06:07.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:07.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.958059+0000", 2024-09-14T09:06:07.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065659450000000001 2024-09-14T09:06:07.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.958123+0000", 2024-09-14T09:06:07.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4524999999999996e-05 2024-09-14T09:06:07.228 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.228 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.229 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.229 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:10040 3.a e33/23 3:50279329:::rbd_data.115eadec874.0000000000000001:head v 33'16, mlcod=32'14)", 2024-09-14T09:06:07.229 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.951570+0000", 2024-09-14T09:06:07.229 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.207000653000001, 2024-09-14T09:06:07.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0094331320000000003, 2024-09-14T09:06:07.229 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.229 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.229 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:07.229 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951570+0000", 2024-09-14T09:06:07.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.230 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.230 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951570+0000", 2024-09-14T09:06:07.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.230 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.230 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951571+0000", 2024-09-14T09:06:07.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0800000000000004e-07 2024-09-14T09:06:07.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.230 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951578+0000", 2024-09-14T09:06:07.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4919999999999996e-06 2024-09-14T09:06:07.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951590+0000", 2024-09-14T09:06:07.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1674e-05 2024-09-14T09:06:07.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951598+0000", 2024-09-14T09:06:07.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4649999999999994e-06 2024-09-14T09:06:07.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951800+0000", 2024-09-14T09:06:07.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020128600000000001 2024-09-14T09:06:07.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951832+0000", 2024-09-14T09:06:07.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2512999999999999e-05 2024-09-14T09:06:07.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:07.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.960831+0000", 2024-09-14T09:06:07.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00899872 2024-09-14T09:06:07.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.961003+0000", 2024-09-14T09:06:07.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017217400000000001 2024-09-14T09:06:07.233 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.233 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.233 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.234 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:10121 3.a e33/23 3:50279329:::rbd_data.115eadec874.0000000000000001:head v 33'18, mlcod=33'16)", 2024-09-14T09:06:07.234 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:38.059543+0000", 2024-09-14T09:06:07.234 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.099027539000001, 2024-09-14T09:06:07.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063099250000000001, 2024-09-14T09:06:07.234 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.234 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.234 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:07.234 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059543+0000", 2024-09-14T09:06:07.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059543+0000", 2024-09-14T09:06:07.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059545+0000", 2024-09-14T09:06:07.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.863e-06 2024-09-14T09:06:07.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059609+0000", 2024-09-14T09:06:07.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3758999999999997e-05 2024-09-14T09:06:07.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059633+0000", 2024-09-14T09:06:07.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4000000000000001e-05 2024-09-14T09:06:07.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059650+0000", 2024-09-14T09:06:07.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7130999999999999e-05 2024-09-14T09:06:07.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059779+0000", 2024-09-14T09:06:07.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000129152 2024-09-14T09:06:07.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059840+0000", 2024-09-14T09:06:07.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0930000000000001e-05 2024-09-14T09:06:07.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:07.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.065745+0000", 2024-09-14T09:06:07.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059050309999999998 2024-09-14T09:06:07.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.065853+0000", 2024-09-14T09:06:07.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000108059 2024-09-14T09:06:07.238 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.238 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.238 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.239 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:10182 3.1a 3:5a62ac37:::rbd_data.115eadec874.000000000000000d:head [set-alloc-hint object_size 33554432 write_size 33554432,write 5072829~48248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-14T09:06:07.239 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:38.177016+0000", 2024-09-14T09:06:07.239 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 28.981554603999999, 2024-09-14T09:06:07.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006987147, 2024-09-14T09:06:07.239 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.239 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.239 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:07.239 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.239 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.239 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.239 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 10182 2024-09-14T09:06:07.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.240 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177016+0000", 2024-09-14T09:06:07.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177016+0000", 2024-09-14T09:06:07.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177017+0000", 2024-09-14T09:06:07.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0440000000000001e-06 2024-09-14T09:06:07.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177065+0000", 2024-09-14T09:06:07.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7902999999999999e-05 2024-09-14T09:06:07.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177080+0000", 2024-09-14T09:06:07.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4667999999999999e-05 2024-09-14T09:06:07.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177087+0000", 2024-09-14T09:06:07.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0430000000000002e-06 2024-09-14T09:06:07.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177140+0000", 2024-09-14T09:06:07.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2933999999999998e-05 2024-09-14T09:06:07.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177199+0000", 2024-09-14T09:06:07.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9799999999999997e-05 2024-09-14T09:06:07.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:07.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177309+0000", 2024-09-14T09:06:07.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010948200000000001 2024-09-14T09:06:07.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:07.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.180906+0000", 2024-09-14T09:06:07.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0035966230000000002 2024-09-14T09:06:07.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:07.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.183932+0000", 2024-09-14T09:06:07.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030259760000000001 2024-09-14T09:06:07.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:07.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.183973+0000", 2024-09-14T09:06:07.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1189000000000002e-05 2024-09-14T09:06:07.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.184003+0000", 2024-09-14T09:06:07.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0485e-05 2024-09-14T09:06:07.244 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.244 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.244 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.245 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:20954 3.1d e41/23 3:bd7e9002:::rbd_data.115e889d4809.0000000000000005:head v 41'131, mlcod=41'127)", 2024-09-14T09:06:07.245 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:47.701009+0000", 2024-09-14T09:06:07.245 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.457561245000001, 2024-09-14T09:06:07.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0080632259999999997, 2024-09-14T09:06:07.245 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.245 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.245 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:07.245 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701009+0000", 2024-09-14T09:06:07.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701009+0000", 2024-09-14T09:06:07.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701011+0000", 2024-09-14T09:06:07.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.703e-06 2024-09-14T09:06:07.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701234+0000", 2024-09-14T09:06:07.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022278900000000001 2024-09-14T09:06:07.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701265+0000", 2024-09-14T09:06:07.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1386000000000001e-05 2024-09-14T09:06:07.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701281+0000", 2024-09-14T09:06:07.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5460000000000001e-05 2024-09-14T09:06:07.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701391+0000", 2024-09-14T09:06:07.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000110622 2024-09-14T09:06:07.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701428+0000", 2024-09-14T09:06:07.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6986999999999997e-05 2024-09-14T09:06:07.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:07.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.708992+0000", 2024-09-14T09:06:07.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075640769999999998 2024-09-14T09:06:07.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.709073+0000", 2024-09-14T09:06:07.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0202000000000002e-05 2024-09-14T09:06:07.249 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.250 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.250 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.250 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:21183 3.1d e42/23 3:bba2b35c:::rbd_object_map.115e889d4809:head v 42'156, mlcod=42'149)", 2024-09-14T09:06:07.250 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:48.419596+0000", 2024-09-14T09:06:07.250 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 18.738974970000001, 2024-09-14T09:06:07.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063479840000000001, 2024-09-14T09:06:07.250 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.250 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.250 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:07.250 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419596+0000", 2024-09-14T09:06:07.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419596+0000", 2024-09-14T09:06:07.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419598+0000", 2024-09-14T09:06:07.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8560000000000001e-06 2024-09-14T09:06:07.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419610+0000", 2024-09-14T09:06:07.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2577e-05 2024-09-14T09:06:07.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419645+0000", 2024-09-14T09:06:07.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4873000000000003e-05 2024-09-14T09:06:07.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419664+0000", 2024-09-14T09:06:07.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9485999999999999e-05 2024-09-14T09:06:07.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419799+0000", 2024-09-14T09:06:07.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013482300000000001 2024-09-14T09:06:07.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419853+0000", 2024-09-14T09:06:07.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3439000000000001e-05 2024-09-14T09:06:07.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:07.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.425843+0000", 2024-09-14T09:06:07.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059898809999999999 2024-09-14T09:06:07.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.425944+0000", 2024-09-14T09:06:07.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101049 2024-09-14T09:06:07.254 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.254 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.254 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.254 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(3.1b 53/52 [PushOp(3:df00a663:::rbd_data.115eadec874.0000000000000005:6, version: 35'41, data_included: [1796079~63600], data_size: 63600, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:df00a663:::rbd_data.115eadec874.0000000000000005:6@35'41, size: 1859679, copy_subset: [(0, 1859679)], clone_subset: {}, snapset: 6=[]:{6: [6]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1859679, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T09:06:07.254 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.846807+0000", 2024-09-14T09:06:07.254 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 5.3117639920000004, 2024-09-14T09:06:07.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0089637359999999999, 2024-09-14T09:06:07.254 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.255 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.255 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.255 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.846807+0000", 2024-09-14T09:06:07.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.846807+0000", 2024-09-14T09:06:07.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.846809+0000", 2024-09-14T09:06:07.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5799999999999999e-06 2024-09-14T09:06:07.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.846948+0000", 2024-09-14T09:06:07.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013911500000000001 2024-09-14T09:06:07.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.847007+0000", 2024-09-14T09:06:07.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8437000000000003e-05 2024-09-14T09:06:07.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.847054+0000", 2024-09-14T09:06:07.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7372e-05 2024-09-14T09:06:07.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.847295+0000", 2024-09-14T09:06:07.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000241022 2024-09-14T09:06:07.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.847319+0000", 2024-09-14T09:06:07.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4303000000000001e-05 2024-09-14T09:06:07.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.855770+0000", 2024-09-14T09:06:07.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084509070000000006 2024-09-14T09:06:07.258 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.258 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.258 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.258 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(3.1b 53/52 [PushOp(3:d8e0f8d4:::rbd_data.115eb07a8abd.0000000000000001:head, version: 48'138, data_included: [0~393216,786432~131072], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:d8e0f8d4:::rbd_data.115eb07a8abd.0000000000000001:head@48'138, size: 7208960, copy_subset: [(0, 7208960)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 7208960, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T09:06:07.258 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.943323+0000", 2024-09-14T09:06:07.259 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 5.2152479769999998, 2024-09-14T09:06:07.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0081121930000000002, 2024-09-14T09:06:07.259 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.259 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.259 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.259 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.943323+0000", 2024-09-14T09:06:07.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.943323+0000", 2024-09-14T09:06:07.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.943324+0000", 2024-09-14T09:06:07.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.206e-06 2024-09-14T09:06:07.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944255+0000", 2024-09-14T09:06:07.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093096900000000002 2024-09-14T09:06:07.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944299+0000", 2024-09-14T09:06:07.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4332999999999997e-05 2024-09-14T09:06:07.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944332+0000", 2024-09-14T09:06:07.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2329999999999997e-05 2024-09-14T09:06:07.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944590+0000", 2024-09-14T09:06:07.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00025868100000000003 2024-09-14T09:06:07.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944629+0000", 2024-09-14T09:06:07.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8503e-05 2024-09-14T09:06:07.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.951435+0000", 2024-09-14T09:06:07.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068061709999999997 2024-09-14T09:06:07.263 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.263 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.263 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.263 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.263 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.263 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:07.266 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:06:07.266 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:07.267+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:07.267 INFO:teuthology.orchestra.run.smithi176.stdout:1764 write 0x6e29509 thru 0x6e36074 (0xcb6c bytes) 2024-09-14T09:06:07.267 INFO:teuthology.orchestra.run.smithi176.stdout:1765 writesame 0xa8ae8ba thru 0xa8ba542 data_size 0x31(0xbc89 bytes) 2024-09-14T09:06:07.267 INFO:teuthology.orchestra.run.smithi176.stdout:1767 write 0x56091bd thru 0x560b5a3 (0x23e7 bytes) 2024-09-14T09:06:07.267 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:07.267 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:07.268 INFO:teuthology.orchestra.run.smithi176.stdout:1771 punch from 0x691a81e to 0x6922086, (0x7868 bytes) 2024-09-14T09:06:07.268 INFO:teuthology.orchestra.run.smithi176.stdout:1772 write 0x605a6ff thru 0x6066e57 (0xc759 bytes) 2024-09-14T09:06:07.268 INFO:teuthology.orchestra.run.smithi176.stdout:1773 punch from 0x2027b6 to 0x2086c1, (0x5f0b bytes) 2024-09-14T09:06:07.268 INFO:teuthology.orchestra.run.smithi176.stdout:1774 read 0x3ba4fee thru 0x3ba8621 (0x3634 bytes) 2024-09-14T09:06:07.268 INFO:teuthology.orchestra.run.smithi176.stdout:1776 write 0xb448e28 thru 0xb452063 (0x923c bytes) 2024-09-14T09:06:07.268 INFO:teuthology.orchestra.run.smithi176.stdout:1777 read 0x1644b36 thru 0x1646d3a (0x2205 bytes) 2024-09-14T09:06:07.268 INFO:teuthology.orchestra.run.smithi176.stdout:1778 read 0x9a984b1 thru 0x9a98903 (0x453 bytes) 2024-09-14T09:06:07.268 INFO:teuthology.orchestra.run.smithi176.stdout:1779 writesame 0x7cbe86b thru 0x7cc100d data_size 0x1(0x27a3 bytes) 2024-09-14T09:06:07.268 INFO:teuthology.orchestra.run.smithi176.stdout:1781 writesame 0x87f8b27 thru 0x8807718 data_size 0x3(0xebf2 bytes) 2024-09-14T09:06:07.268 INFO:teuthology.orchestra.run.smithi176.stdout:1783 write 0x6dfc95f thru 0x6e0bc42 (0xf2e4 bytes) 2024-09-14T09:06:07.268 INFO:teuthology.orchestra.run.smithi176.stdout:1784 write 0xb197d58 thru 0xb19f0fc (0x73a5 bytes) 2024-09-14T09:06:07.268 INFO:teuthology.orchestra.run.smithi176.stdout:1785 write 0x94f444d thru 0x94f739f (0x2f53 bytes) 2024-09-14T09:06:07.268 INFO:teuthology.orchestra.run.smithi176.stdout:1786 read 0x4627a8e thru 0x4631ae3 (0xa056 bytes) 2024-09-14T09:06:07.268 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:07.269 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:07.269 INFO:teuthology.orchestra.run.smithi176.stdout:1791 punch from 0x7332b0f to 0x7336961, (0x3e52 bytes) 2024-09-14T09:06:07.269 INFO:teuthology.orchestra.run.smithi176.stdout:1792 write 0x1afc167 thru 0x1b02c2f (0x6ac9 bytes) 2024-09-14T09:06:07.282 INFO:teuthology.orchestra.run.smithi176.stdout:1793 write 0xeafa354 thru 0xeb07375 (0xd022 bytes) 2024-09-14T09:06:07.286 INFO:teuthology.orchestra.run.smithi176.stdout:1794 punch from 0x143c0a8 to 0x1449d0f, (0xdc67 bytes) 2024-09-14T09:06:07.286 INFO:teuthology.orchestra.run.smithi176.stdout:1795 read 0x4aa51b9 thru 0x4ab4131 (0xef79 bytes) 2024-09-14T09:06:07.287 INFO:teuthology.orchestra.run.smithi176.stdout:1796 punch from 0xa5bf16a to 0xa5c21f1, (0x3087 bytes) 2024-09-14T09:06:07.287 INFO:teuthology.orchestra.run.smithi176.stdout:1797 read 0x7d94c5b thru 0x7d99204 (0x45aa bytes) 2024-09-14T09:06:07.287 INFO:teuthology.orchestra.run.smithi176.stdout:1798 write 0x8176b71 thru 0x818335d (0xc7ed bytes) 2024-09-14T09:06:07.295 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:07.297 INFO:teuthology.orchestra.run.smithi176.stdout:1801 trunc from 0xeb07376 to 0xd4e1414 2024-09-14T09:06:07.311 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:07.311 INFO:teuthology.orchestra.run.smithi176.stdout:1803 writesame 0x2e28216 thru 0x2e37bac data_size 0xd(0xf997 bytes) 2024-09-14T09:06:07.317 INFO:teuthology.orchestra.run.smithi176.stdout:1804 read 0x7d9d470 thru 0x7da825a (0xadeb bytes) 2024-09-14T09:06:07.329 INFO:teuthology.orchestra.run.smithi176.stdout:1805 writesame 0xea571c2 thru 0xea63cee data_size 0x1(0xcb2d bytes) 2024-09-14T09:06:07.343 INFO:teuthology.orchestra.run.smithi176.stdout:1806 trunc from 0xea63cef to 0x570e2c6 2024-09-14T09:06:07.363 INFO:teuthology.orchestra.run.smithi176.stdout:1807 writesame 0x1e9296c thru 0x1e92c4f data_size 0xa(0x2e4 bytes) 2024-09-14T09:06:07.364 INFO:teuthology.orchestra.run.smithi176.stdout:1808 punch from 0x4e5bf23 to 0x4e5e118, (0x21f5 bytes) 2024-09-14T09:06:07.364 INFO:teuthology.orchestra.run.smithi176.stdout:1812 write 0x9234b7 thru 0x92c298 (0x8de2 bytes) 2024-09-14T09:06:07.365 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:07.359+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:07.368 INFO:teuthology.orchestra.run.smithi176.stdout:1813 punch from 0x4355b8c to 0x435988d, (0x3d01 bytes) 2024-09-14T09:06:07.368 INFO:teuthology.orchestra.run.smithi176.stdout:1814 writesame 0x1a02546 thru 0x1a06032 data_size 0x7(0x3aed bytes) 2024-09-14T09:06:07.371 INFO:teuthology.orchestra.run.smithi176.stdout:1815 punch from 0x29cfd49 to 0x29d2d6c, (0x3023 bytes) 2024-09-14T09:06:07.385 INFO:teuthology.orchestra.run.smithi176.stdout:1816 write 0x949a35b thru 0x94a71ef (0xce95 bytes) 2024-09-14T09:06:07.387 INFO:teuthology.orchestra.run.smithi176.stdout:1817 read 0x68dc490 thru 0x68ec34f (0xfec0 bytes) 2024-09-14T09:06:07.388 INFO:teuthology.orchestra.run.smithi176.stdout:1819 writesame 0x4c3ddc1 thru 0x4c3f714 data_size 0x2(0x1954 bytes) 2024-09-14T09:06:07.391 INFO:teuthology.orchestra.run.smithi176.stdout:1821 read 0xa88968 thru 0xa8e7f4 (0x5e8d bytes) 2024-09-14T09:06:07.391 INFO:teuthology.orchestra.run.smithi176.stdout:1822 read 0x2e0d8f7 thru 0x2e19c54 (0xc35e bytes) 2024-09-14T09:06:07.392 INFO:teuthology.orchestra.run.smithi176.stdout:1823 read 0x2f4c532 thru 0x2f4c9cc (0x49b bytes) 2024-09-14T09:06:07.392 INFO:teuthology.orchestra.run.smithi176.stdout:1824 read 0x1bb5229 thru 0x1bb7b08 (0x28e0 bytes) 2024-09-14T09:06:07.404 INFO:teuthology.orchestra.run.smithi176.stdout:1825 write 0xabbde92 thru 0xabc6b9e (0x8d0d bytes) 2024-09-14T09:06:07.409 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:07.410 INFO:teuthology.orchestra.run.smithi176.stdout:1829 read 0x8be0e66 thru 0x8be7889 (0x6a24 bytes) 2024-09-14T09:06:07.411 INFO:teuthology.orchestra.run.smithi176.stdout:1830 punch from 0x7f72ba1 to 0x7f82918, (0xfd77 bytes) 2024-09-14T09:06:07.411 INFO:teuthology.orchestra.run.smithi176.stdout:1831 read 0xcead4f thru 0xcf7871 (0xcb23 bytes) 2024-09-14T09:06:07.411 INFO:teuthology.orchestra.run.smithi176.stdout:1833 write 0x569fb73 thru 0x56a06d7 (0xb65 bytes) 2024-09-14T09:06:07.414 INFO:teuthology.orchestra.run.smithi176.stdout:1834 write 0x1b9bf0a thru 0x1ba28e6 (0x69dd bytes) 2024-09-14T09:06:07.418 INFO:teuthology.orchestra.run.smithi176.stdout:1835 read 0x2f7d838 thru 0x2f8d119 (0xf8e2 bytes) 2024-09-14T09:06:07.419 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:07.419 INFO:teuthology.orchestra.run.smithi176.stdout:1838 read 0x4966988 thru 0x4973f49 (0xd5c2 bytes) 2024-09-14T09:06:07.420 INFO:teuthology.orchestra.run.smithi176.stdout:1839 read 0x24715b3 thru 0x247365f (0x20ad bytes) 2024-09-14T09:06:07.420 INFO:teuthology.orchestra.run.smithi176.stdout:1840 punch from 0x4941770 to 0x494c7dd, (0xb06d bytes) 2024-09-14T09:06:07.421 INFO:teuthology.orchestra.run.smithi176.stdout:1842 clone 15 order 22 su 16384 sc 2 2024-09-14T09:06:07.441 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:07.441 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:07.442 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:07.442 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:07.455 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:06:07.504 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:07.507+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:07.567 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:07.567+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:07.578 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:07.579 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:07.579 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:07.579 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:07.579 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:07.593 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:06:07.680 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:07.683+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:07.761 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:07.761 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:07.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:07.761 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:07.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.761 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:19541 3.10 e39/28 3:0ae7d8d9:::rbd_data.115e4c71d6e8.0000000000000001:head v 39'11, mlcod=39'9)", 2024-09-14T09:06:07.761 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.506445+0000", 2024-09-14T09:06:07.761 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.255430474000001, 2024-09-14T09:06:07.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.050485219999999997, 2024-09-14T09:06:07.761 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.762 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.762 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:07.762 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.506445+0000", 2024-09-14T09:06:07.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.506445+0000", 2024-09-14T09:06:07.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.506449+0000", 2024-09-14T09:06:07.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6200000000000001e-06 2024-09-14T09:06:07.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.514152+0000", 2024-09-14T09:06:07.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077031720000000003 2024-09-14T09:06:07.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.514196+0000", 2024-09-14T09:06:07.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3974000000000003e-05 2024-09-14T09:06:07.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.514217+0000", 2024-09-14T09:06:07.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1229999999999998e-05 2024-09-14T09:06:07.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.514357+0000", 2024-09-14T09:06:07.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013953199999999999 2024-09-14T09:06:07.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.514404+0000", 2024-09-14T09:06:07.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7463999999999998e-05 2024-09-14T09:06:07.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:07.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.556881+0000", 2024-09-14T09:06:07.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042476409999999999 2024-09-14T09:06:07.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.556931+0000", 2024-09-14T09:06:07.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9818000000000003e-05 2024-09-14T09:06:07.766 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.766 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.766 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.766 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19543 3.5 3:a536ff9b:::rbd_data.115e4c71d6e8.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 4194304~8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:07.766 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.514871+0000", 2024-09-14T09:06:07.766 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.247005418000001, 2024-09-14T09:06:07.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.058941764000000001, 2024-09-14T09:06:07.766 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.766 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.766 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:07.766 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.766 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.767 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.767 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19543 2024-09-14T09:06:07.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.767 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.514871+0000", 2024-09-14T09:06:07.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.514871+0000", 2024-09-14T09:06:07.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.514874+0000", 2024-09-14T09:06:07.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.36e-06 2024-09-14T09:06:07.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.523515+0000", 2024-09-14T09:06:07.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086406750000000004 2024-09-14T09:06:07.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.523551+0000", 2024-09-14T09:06:07.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6050999999999998e-05 2024-09-14T09:06:07.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.523570+0000", 2024-09-14T09:06:07.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9675999999999999e-05 2024-09-14T09:06:07.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.523733+0000", 2024-09-14T09:06:07.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162693 2024-09-14T09:06:07.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.524014+0000", 2024-09-14T09:06:07.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000281434 2024-09-14T09:06:07.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-09-14T09:06:07.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.524841+0000", 2024-09-14T09:06:07.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00082681699999999998 2024-09-14T09:06:07.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:07.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.573558+0000", 2024-09-14T09:06:07.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.048716792000000002 2024-09-14T09:06:07.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:07.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.573588+0000", 2024-09-14T09:06:07.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0420999999999999e-05 2024-09-14T09:06:07.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:07.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.573709+0000", 2024-09-14T09:06:07.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012012500000000001 2024-09-14T09:06:07.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.573812+0000", 2024-09-14T09:06:07.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010372 2024-09-14T09:06:07.771 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.772 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.772 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.772 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:31041 3.1d 3:be2f72d8:::rbd_data.115ebc3c67d1.000000000000000b:c [read 223232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:07.772 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.067297+0000", 2024-09-14T09:06:07.772 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.694578482000001, 2024-09-14T09:06:07.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042437080000000002, 2024-09-14T09:06:07.772 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.772 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.772 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.772 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.773 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.773 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.773 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 31041 2024-09-14T09:06:07.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.773 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067297+0000", 2024-09-14T09:06:07.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067297+0000", 2024-09-14T09:06:07.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067298+0000", 2024-09-14T09:06:07.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.879999999999999e-07 2024-09-14T09:06:07.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067302+0000", 2024-09-14T09:06:07.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8299999999999998e-06 2024-09-14T09:06:07.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067315+0000", 2024-09-14T09:06:07.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2713e-05 2024-09-14T09:06:07.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067325+0000", 2024-09-14T09:06:07.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7650000000000005e-06 2024-09-14T09:06:07.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.109579+0000", 2024-09-14T09:06:07.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042253926999999997 2024-09-14T09:06:07.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.109660+0000", 2024-09-14T09:06:07.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1763999999999998e-05 2024-09-14T09:06:07.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.109735+0000", 2024-09-14T09:06:07.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4092999999999997e-05 2024-09-14T09:06:07.776 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.776 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.776 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.776 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:31042 3.1d 3:be2f72d8:::rbd_data.115ebc3c67d1.000000000000000b:c [read 241664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:07.777 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.067337+0000", 2024-09-14T09:06:07.777 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.694538701999999, 2024-09-14T09:06:07.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042596705999999998, 2024-09-14T09:06:07.777 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.777 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.777 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.777 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.777 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.777 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.777 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 31042 2024-09-14T09:06:07.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.777 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067337+0000", 2024-09-14T09:06:07.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067337+0000", 2024-09-14T09:06:07.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067338+0000", 2024-09-14T09:06:07.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8700000000000004e-07 2024-09-14T09:06:07.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067342+0000", 2024-09-14T09:06:07.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9949999999999999e-06 2024-09-14T09:06:07.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067355+0000", 2024-09-14T09:06:07.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2571e-05 2024-09-14T09:06:07.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067365+0000", 2024-09-14T09:06:07.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8069999999999994e-06 2024-09-14T09:06:07.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.109767+0000", 2024-09-14T09:06:07.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042402914 2024-09-14T09:06:07.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.109846+0000", 2024-09-14T09:06:07.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8559000000000005e-05 2024-09-14T09:06:07.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.109934+0000", 2024-09-14T09:06:07.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7873000000000004e-05 2024-09-14T09:06:07.781 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.781 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.781 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.781 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:31043 3.1d 3:be2f72d8:::rbd_data.115ebc3c67d1.000000000000000b:c [read 251904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:07.781 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.067412+0000", 2024-09-14T09:06:07.781 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.694463600000001, 2024-09-14T09:06:07.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042730814999999998, 2024-09-14T09:06:07.781 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.781 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.782 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.782 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.782 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.782 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.782 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 31043 2024-09-14T09:06:07.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.782 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067412+0000", 2024-09-14T09:06:07.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067412+0000", 2024-09-14T09:06:07.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067414+0000", 2024-09-14T09:06:07.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4309999999999999e-06 2024-09-14T09:06:07.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067419+0000", 2024-09-14T09:06:07.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1909999999999999e-06 2024-09-14T09:06:07.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067435+0000", 2024-09-14T09:06:07.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6181999999999999e-05 2024-09-14T09:06:07.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067447+0000", 2024-09-14T09:06:07.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1824999999999999e-05 2024-09-14T09:06:07.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.109969+0000", 2024-09-14T09:06:07.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042521556000000002 2024-09-14T09:06:07.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.110054+0000", 2024-09-14T09:06:07.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5309000000000006e-05 2024-09-14T09:06:07.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.110143+0000", 2024-09-14T09:06:07.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9320999999999994e-05 2024-09-14T09:06:07.785 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.785 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.785 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.785 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:31044 3.1d 3:be2f72d8:::rbd_data.115ebc3c67d1.000000000000000b:c [read 270336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.067464+0000", 2024-09-14T09:06:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.694411571, 2024-09-14T09:06:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042876835000000002, 2024-09-14T09:06:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 31044 2024-09-14T09:06:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067464+0000", 2024-09-14T09:06:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067464+0000", 2024-09-14T09:06:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067465+0000", 2024-09-14T09:06:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9099999999999991e-07 2024-09-14T09:06:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067470+0000", 2024-09-14T09:06:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8210000000000001e-06 2024-09-14T09:06:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067483+0000", 2024-09-14T09:06:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3237999999999999e-05 2024-09-14T09:06:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067494+0000", 2024-09-14T09:06:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0135999999999999e-05 2024-09-14T09:06:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.110168+0000", 2024-09-14T09:06:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042674111000000001 2024-09-14T09:06:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.110250+0000", 2024-09-14T09:06:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.276e-05 2024-09-14T09:06:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.110341+0000", 2024-09-14T09:06:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0778000000000004e-05 2024-09-14T09:06:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:31045 3.1d 3:be2f72d8:::rbd_data.115ebc3c67d1.000000000000000b:c [read 280576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.067506+0000", 2024-09-14T09:06:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.694369801000001, 2024-09-14T09:06:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.043067211000000001, 2024-09-14T09:06:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 31045 2024-09-14T09:06:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067506+0000", 2024-09-14T09:06:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067506+0000", 2024-09-14T09:06:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067507+0000", 2024-09-14T09:06:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0979999999999999e-06 2024-09-14T09:06:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067538+0000", 2024-09-14T09:06:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0289000000000001e-05 2024-09-14T09:06:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067553+0000", 2024-09-14T09:06:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5197e-05 2024-09-14T09:06:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067564+0000", 2024-09-14T09:06:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0824e-05 2024-09-14T09:06:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.110391+0000", 2024-09-14T09:06:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042827582000000003 2024-09-14T09:06:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.110474+0000", 2024-09-14T09:06:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3324000000000004e-05 2024-09-14T09:06:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.110573+0000", 2024-09-14T09:06:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8896999999999999e-05 2024-09-14T09:06:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:31046 3.1d 3:be2f72d8:::rbd_data.115ebc3c67d1.000000000000000b:c [read 299008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.067578+0000", 2024-09-14T09:06:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.694297432999999, 2024-09-14T09:06:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.043178218999999997, 2024-09-14T09:06:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 31046 2024-09-14T09:06:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067578+0000", 2024-09-14T09:06:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067578+0000", 2024-09-14T09:06:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067579+0000", 2024-09-14T09:06:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5999999999999991e-07 2024-09-14T09:06:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067584+0000", 2024-09-14T09:06:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0629999999999999e-06 2024-09-14T09:06:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067596+0000", 2024-09-14T09:06:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2146e-05 2024-09-14T09:06:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.067605+0000", 2024-09-14T09:06:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6800000000000005e-06 2024-09-14T09:06:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.110608+0000", 2024-09-14T09:06:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.043002772000000002 2024-09-14T09:06:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.110684+0000", 2024-09-14T09:06:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6207999999999999e-05 2024-09-14T09:06:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.110757+0000", 2024-09-14T09:06:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2390000000000003e-05 2024-09-14T09:06:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:31292 3.1d 3:be2f72d8:::rbd_data.115ebc3c67d1.000000000000000b:c [read 157696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.077104+0000", 2024-09-14T09:06:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.684771712, 2024-09-14T09:06:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042593514999999998, 2024-09-14T09:06:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 31292 2024-09-14T09:06:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077104+0000", 2024-09-14T09:06:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077104+0000", 2024-09-14T09:06:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077105+0000", 2024-09-14T09:06:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1429999999999999e-06 2024-09-14T09:06:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077110+0000", 2024-09-14T09:06:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3649999999999997e-06 2024-09-14T09:06:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077124+0000", 2024-09-14T09:06:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4691999999999999e-05 2024-09-14T09:06:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077136+0000", 2024-09-14T09:06:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1600000000000001e-05 2024-09-14T09:06:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.119329+0000", 2024-09-14T09:06:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042193346999999999 2024-09-14T09:06:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.119387+0000", 2024-09-14T09:06:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7917000000000001e-05 2024-09-14T09:06:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.119698+0000", 2024-09-14T09:06:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000310451 2024-09-14T09:06:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:31293 3.1d 3:be2f72d8:::rbd_data.115ebc3c67d1.000000000000000b:c [read 176128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.077153+0000", 2024-09-14T09:06:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.684722539999999, 2024-09-14T09:06:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042753818999999998, 2024-09-14T09:06:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 31293 2024-09-14T09:06:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077153+0000", 2024-09-14T09:06:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077153+0000", 2024-09-14T09:06:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077154+0000", 2024-09-14T09:06:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9800000000000002e-07 2024-09-14T09:06:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077159+0000", 2024-09-14T09:06:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9579999999999998e-06 2024-09-14T09:06:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077174+0000", 2024-09-14T09:06:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4829e-05 2024-09-14T09:06:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077186+0000", 2024-09-14T09:06:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1528e-05 2024-09-14T09:06:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.119733+0000", 2024-09-14T09:06:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042547768999999999 2024-09-14T09:06:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.119819+0000", 2024-09-14T09:06:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6098000000000001e-05 2024-09-14T09:06:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.119907+0000", 2024-09-14T09:06:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7738999999999995e-05 2024-09-14T09:06:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:31294 3.1d 3:be2f72d8:::rbd_data.115ebc3c67d1.000000000000000b:c [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.077201+0000", 2024-09-14T09:06:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.684674902999999, 2024-09-14T09:06:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042911632999999998, 2024-09-14T09:06:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 31294 2024-09-14T09:06:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077201+0000", 2024-09-14T09:06:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077201+0000", 2024-09-14T09:06:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077202+0000", 2024-09-14T09:06:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.088e-06 2024-09-14T09:06:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077207+0000", 2024-09-14T09:06:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5390000000000001e-06 2024-09-14T09:06:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077222+0000", 2024-09-14T09:06:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4932999999999999e-05 2024-09-14T09:06:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077233+0000", 2024-09-14T09:06:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1103e-05 2024-09-14T09:06:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.119943+0000", 2024-09-14T09:06:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042710598000000002 2024-09-14T09:06:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.120020+0000", 2024-09-14T09:06:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6345000000000002e-05 2024-09-14T09:06:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.120113+0000", 2024-09-14T09:06:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3027000000000002e-05 2024-09-14T09:06:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:31295 3.1d 3:be2f72d8:::rbd_data.115ebc3c67d1.000000000000000b:c [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.077268+0000", 2024-09-14T09:06:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.684607786000001, 2024-09-14T09:06:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.043033900999999999, 2024-09-14T09:06:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 31295 2024-09-14T09:06:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077268+0000", 2024-09-14T09:06:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077268+0000", 2024-09-14T09:06:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077270+0000", 2024-09-14T09:06:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4160000000000001e-06 2024-09-14T09:06:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077275+0000", 2024-09-14T09:06:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2399999999999998e-06 2024-09-14T09:06:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077291+0000", 2024-09-14T09:06:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6373999999999999e-05 2024-09-14T09:06:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077304+0000", 2024-09-14T09:06:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2636000000000001e-05 2024-09-14T09:06:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.120149+0000", 2024-09-14T09:06:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042845278000000001 2024-09-14T09:06:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.120226+0000", 2024-09-14T09:06:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7315000000000001e-05 2024-09-14T09:06:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.120302+0000", 2024-09-14T09:06:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5642000000000005e-05 2024-09-14T09:06:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:31296 3.1d 3:be2f72d8:::rbd_data.115ebc3c67d1.000000000000000b:c [read 215040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.077323+0000", 2024-09-14T09:06:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.684553266, 2024-09-14T09:06:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.043207991000000001, 2024-09-14T09:06:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 31296 2024-09-14T09:06:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077323+0000", 2024-09-14T09:06:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077323+0000", 2024-09-14T09:06:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077324+0000", 2024-09-14T09:06:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.189e-06 2024-09-14T09:06:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077329+0000", 2024-09-14T09:06:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2510000000000002e-06 2024-09-14T09:06:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077344+0000", 2024-09-14T09:06:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.537e-05 2024-09-14T09:06:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077356+0000", 2024-09-14T09:06:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1469e-05 2024-09-14T09:06:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.120337+0000", 2024-09-14T09:06:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042980641999999999 2024-09-14T09:06:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.120443+0000", 2024-09-14T09:06:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106698 2024-09-14T09:06:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.120531+0000", 2024-09-14T09:06:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7372000000000003e-05 2024-09-14T09:06:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:31297 3.1d 3:be2f72d8:::rbd_data.115ebc3c67d1.000000000000000b:c [read 233472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.077371+0000", 2024-09-14T09:06:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.684504579, 2024-09-14T09:06:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.043414556, 2024-09-14T09:06:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 31297 2024-09-14T09:06:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077371+0000", 2024-09-14T09:06:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077371+0000", 2024-09-14T09:06:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077372+0000", 2024-09-14T09:06:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1119999999999999e-06 2024-09-14T09:06:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077377+0000", 2024-09-14T09:06:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3830000000000002e-06 2024-09-14T09:06:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077391+0000", 2024-09-14T09:06:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3794999999999999e-05 2024-09-14T09:06:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077401+0000", 2024-09-14T09:06:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0725e-05 2024-09-14T09:06:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.120599+0000", 2024-09-14T09:06:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.043197408999999999 2024-09-14T09:06:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.120698+0000", 2024-09-14T09:06:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9728999999999993e-05 2024-09-14T09:06:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.120786+0000", 2024-09-14T09:06:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7403000000000003e-05 2024-09-14T09:06:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:31298 3.1d 3:be2f72d8:::rbd_data.115ebc3c67d1.000000000000000b:c [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.077414+0000", 2024-09-14T09:06:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.68446151, 2024-09-14T09:06:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.043565293999999997, 2024-09-14T09:06:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 31298 2024-09-14T09:06:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077414+0000", 2024-09-14T09:06:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077414+0000", 2024-09-14T09:06:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077415+0000", 2024-09-14T09:06:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2399999999999997e-07 2024-09-14T09:06:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077449+0000", 2024-09-14T09:06:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3528000000000002e-05 2024-09-14T09:06:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077465+0000", 2024-09-14T09:06:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6082999999999998e-05 2024-09-14T09:06:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077477+0000", 2024-09-14T09:06:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2279999999999999e-05 2024-09-14T09:06:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.120820+0000", 2024-09-14T09:06:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.043342564 2024-09-14T09:06:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.120899+0000", 2024-09-14T09:06:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9104999999999996e-05 2024-09-14T09:06:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.120980+0000", 2024-09-14T09:06:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0909999999999996e-05 2024-09-14T09:06:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:31299 3.1d 3:be2f72d8:::rbd_data.115ebc3c67d1.000000000000000b:c [read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.077493+0000", 2024-09-14T09:06:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.684382713, 2024-09-14T09:06:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.043607543999999998, 2024-09-14T09:06:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 31299 2024-09-14T09:06:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077493+0000", 2024-09-14T09:06:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077493+0000", 2024-09-14T09:06:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077494+0000", 2024-09-14T09:06:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0979999999999999e-06 2024-09-14T09:06:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077498+0000", 2024-09-14T09:06:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0720000000000001e-06 2024-09-14T09:06:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077512+0000", 2024-09-14T09:06:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3293e-05 2024-09-14T09:06:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077522+0000", 2024-09-14T09:06:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0565e-05 2024-09-14T09:06:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.121004+0000", 2024-09-14T09:06:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.043482241999999997 2024-09-14T09:06:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.121047+0000", 2024-09-14T09:06:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2975000000000001e-05 2024-09-14T09:06:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.121101+0000", 2024-09-14T09:06:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3298999999999999e-05 2024-09-14T09:06:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:31300 3.1d 3:be2f72d8:::rbd_data.115ebc3c67d1.000000000000000b:c [read 272384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.077535+0000", 2024-09-14T09:06:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.684340615, 2024-09-14T09:06:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.043725643000000002, 2024-09-14T09:06:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 31300 2024-09-14T09:06:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077535+0000", 2024-09-14T09:06:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077535+0000", 2024-09-14T09:06:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077536+0000", 2024-09-14T09:06:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1200000000000001e-07 2024-09-14T09:06:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077540+0000", 2024-09-14T09:06:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8319999999999999e-06 2024-09-14T09:06:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077553+0000", 2024-09-14T09:06:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2883e-05 2024-09-14T09:06:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077563+0000", 2024-09-14T09:06:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0180000000000001e-05 2024-09-14T09:06:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.121125+0000", 2024-09-14T09:06:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.043562020999999999 2024-09-14T09:06:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.121177+0000", 2024-09-14T09:06:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2120999999999998e-05 2024-09-14T09:06:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.121261+0000", 2024-09-14T09:06:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3694000000000002e-05 2024-09-14T09:06:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:31301 3.1d 3:be2f72d8:::rbd_data.115ebc3c67d1.000000000000000b:c [read 290816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.077576+0000", 2024-09-14T09:06:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.684299900999999, 2024-09-14T09:06:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.043886456999999997, 2024-09-14T09:06:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 31301 2024-09-14T09:06:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077576+0000", 2024-09-14T09:06:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077576+0000", 2024-09-14T09:06:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077577+0000", 2024-09-14T09:06:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4600000000000003e-07 2024-09-14T09:06:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077581+0000", 2024-09-14T09:06:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0419999999999996e-06 2024-09-14T09:06:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077593+0000", 2024-09-14T09:06:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2585000000000001e-05 2024-09-14T09:06:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077603+0000", 2024-09-14T09:06:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6469999999999993e-06 2024-09-14T09:06:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.121292+0000", 2024-09-14T09:06:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.043688817999999997 2024-09-14T09:06:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.121374+0000", 2024-09-14T09:06:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2256000000000007e-05 2024-09-14T09:06:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.121462+0000", 2024-09-14T09:06:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8263000000000005e-05 2024-09-14T09:06:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:31302 3.1d 3:be2f72d8:::rbd_data.115ebc3c67d1.000000000000000b:c [read 301056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.077616+0000", 2024-09-14T09:06:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.684260316, 2024-09-14T09:06:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.04405477, 2024-09-14T09:06:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 31302 2024-09-14T09:06:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077616+0000", 2024-09-14T09:06:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077616+0000", 2024-09-14T09:06:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077616+0000", 2024-09-14T09:06:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7300000000000005e-07 2024-09-14T09:06:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077620+0000", 2024-09-14T09:06:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9899999999999999e-06 2024-09-14T09:06:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077634+0000", 2024-09-14T09:06:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3974e-05 2024-09-14T09:06:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.077645+0000", 2024-09-14T09:06:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0368e-05 2024-09-14T09:06:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.121497+0000", 2024-09-14T09:06:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.043852299999999997 2024-09-14T09:06:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.121579+0000", 2024-09-14T09:06:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1542e-05 2024-09-14T09:06:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.121670+0000", 2024-09-14T09:06:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1723000000000003e-05 2024-09-14T09:06:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:31660 3.5 3:a55ddee4:::rbd_data.115e9ba9f186.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 117030~7898] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.094251+0000", 2024-09-14T09:06:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.667624902, 2024-09-14T09:06:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042841972999999998, 2024-09-14T09:06:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 31660 2024-09-14T09:06:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.094251+0000", 2024-09-14T09:06:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.094251+0000", 2024-09-14T09:06:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.094252+0000", 2024-09-14T09:06:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.409e-06 2024-09-14T09:06:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.094689+0000", 2024-09-14T09:06:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000436339 2024-09-14T09:06:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.094707+0000", 2024-09-14T09:06:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8343999999999999e-05 2024-09-14T09:06:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.094715+0000", 2024-09-14T09:06:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2949999999999994e-06 2024-09-14T09:06:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.129643+0000", 2024-09-14T09:06:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034927422999999999 2024-09-14T09:06:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.129823+0000", 2024-09-14T09:06:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000180602 2024-09-14T09:06:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-09-14T09:06:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.130496+0000", 2024-09-14T09:06:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00067226299999999998 2024-09-14T09:06:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.135932+0000", 2024-09-14T09:06:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054367520000000004 2024-09-14T09:06:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.137007+0000", 2024-09-14T09:06:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001074406 2024-09-14T09:06:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.137055+0000", 2024-09-14T09:06:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8180000000000003e-05 2024-09-14T09:06:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.137093+0000", 2024-09-14T09:06:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7960000000000002e-05 2024-09-14T09:06:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.855 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.855 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:07.855 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:07.855 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:07.855 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:06:07.857 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:06:07.857 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:07.859+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:07.957 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:07.959+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:08.010 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:08.010 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:08.027 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:08.027 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:08.027 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:08.027 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:08.028 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:08.028 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:08.042 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:06:08.043 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:08.043 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:08.060 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:08.060 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:08.077 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:08.077 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:08.088 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:08.091+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:08.090 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is unset 2024-09-14T09:06:08.101 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:08.101 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:08.124 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:08.124 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:08.145 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:08.146 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:08.159 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:08.159+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:08.181 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone14 from 0xabc6b9f (overlap 0x14a5f5) to 0xa52fa 2024-09-14T09:06:08.224 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:08.224 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:08.238 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:06:08.260 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:08.255+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:08.319 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-14T09:06:08.361 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:08.355+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:08.382 INFO:teuthology.orchestra.run.smithi176.stdout:1843 read 0x9278e5c thru 0x92840db (0xb280 bytes) 2024-09-14T09:06:08.385 INFO:teuthology.orchestra.run.smithi176.stdout:1844 read 0x482061d thru 0x482985b (0x923f bytes) 2024-09-14T09:06:08.386 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:08.386 INFO:teuthology.orchestra.run.smithi176.stdout:1846 read 0x9f156dd thru 0x9f1b1ec (0x5b10 bytes) 2024-09-14T09:06:08.389 INFO:teuthology.orchestra.run.smithi176.stdout:1848 flatten 2024-09-14T09:06:08.410 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:08.411 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:08.426 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:06:08.472 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:08.475+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:08.564 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:08.559+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:08.574 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:08.574 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:08.589 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:06:08.664 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:08.659+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:08.765 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:08.759+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:08.779 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:08.779 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:08.796 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:06:08.866 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:08.863+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:08.933 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:08.933 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:08.948 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:06:08.948 INFO:teuthology.orchestra.run.smithi176.stdout:1851 read 0x7573847 thru 0x757b40e (0x7bc8 bytes) 2024-09-14T09:06:08.961 INFO:teuthology.orchestra.run.smithi176.stdout:1852 write 0xda27041 thru 0xda276a4 (0x664 bytes) 2024-09-14T09:06:08.966 INFO:teuthology.orchestra.run.smithi176.stdout:1853 read 0x7134340 thru 0x713f06c (0xad2d bytes) 2024-09-14T09:06:08.966 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:08.966 INFO:teuthology.orchestra.run.smithi176.stdout:1855 writesame 0x68079dc thru 0x680fd46 data_size 0x1(0x836b bytes) 2024-09-14T09:06:08.966 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:08.967+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:09.022 INFO:teuthology.orchestra.run.smithi176.stdout:1856 punch from 0x6b5b30f to 0x6b5e4b2, (0x31a3 bytes) 2024-09-14T09:06:09.022 INFO:teuthology.orchestra.run.smithi176.stdout:1857 read 0x5e1441b thru 0x5e1cc58 (0x883e bytes) 2024-09-14T09:06:09.032 INFO:teuthology.orchestra.run.smithi176.stdout:1858 trunc from 0xda276a5 to 0x7779d46 2024-09-14T09:06:09.064 INFO:teuthology.orchestra.run.smithi176.stdout:1859 write 0x9549ee7 thru 0x9553c55 (0x9d6f bytes) 2024-09-14T09:06:09.067 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:09.067+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:09.070 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:09.070 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:09.081 INFO:teuthology.orchestra.run.smithi176.stdout:1861 trunc from 0x9553c56 to 0x435eb2c 2024-09-14T09:06:09.091 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:06:09.103 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:09.103 INFO:teuthology.orchestra.run.smithi176.stdout:1865 read 0x261d071 thru 0x2626395 (0x9325 bytes) 2024-09-14T09:06:09.136 INFO:teuthology.orchestra.run.smithi176.stdout:1866 writesame 0xe6baca4 thru 0xe6bfa0f data_size 0xa(0x4d6c bytes) 2024-09-14T09:06:09.136 INFO:teuthology.orchestra.run.smithi176.stdout:1867 punch from 0x950e0c4 to 0x9511089, (0x2fc5 bytes) 2024-09-14T09:06:09.136 INFO:teuthology.orchestra.run.smithi176.stdout:1868 writesame 0x78b50b6 thru 0x78bc955 data_size 0x50(0x78a0 bytes) 2024-09-14T09:06:09.144 INFO:teuthology.orchestra.run.smithi176.stdout:1869 writesame 0xd45b549 thru 0xd461728 data_size 0x74(0x61e0 bytes) 2024-09-14T09:06:09.153 INFO:teuthology.orchestra.run.smithi176.stdout:1870 punch from 0x8c6c205 to 0x8c7741d, (0xb218 bytes) 2024-09-14T09:06:09.153 INFO:teuthology.orchestra.run.smithi176.stdout:1871 punch from 0xa576ca to 0xa5cce8, (0x561e bytes) 2024-09-14T09:06:09.153 INFO:teuthology.orchestra.run.smithi176.stdout:1872 write 0x60a9b25 thru 0x60b62b4 (0xc790 bytes) 2024-09-14T09:06:09.165 INFO:teuthology.orchestra.run.smithi176.stdout:1873 writesame 0xe2701d thru 0xe2a835 data_size 0x1(0x3819 bytes) 2024-09-14T09:06:09.168 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:09.171+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:09.206 INFO:teuthology.orchestra.run.smithi176.stdout:1875 writesame 0xbf35654 thru 0xbf364f8 data_size 0x1(0xea5 bytes) 2024-09-14T09:06:09.214 INFO:teuthology.orchestra.run.smithi176.stdout:1876 writesame 0x82eb6cd thru 0x82f04a0 data_size 0x22(0x4dd4 bytes) 2024-09-14T09:06:09.222 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:09.222 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:09.222 INFO:teuthology.orchestra.run.smithi176.stdout:1877 write 0x2c83565 thru 0x2c9194b (0xe3e7 bytes) 2024-09-14T09:06:09.227 INFO:teuthology.orchestra.run.smithi176.stdout:1878 read 0xcbb98ed thru 0xcbba406 (0xb1a bytes) 2024-09-14T09:06:09.227 INFO:teuthology.orchestra.run.smithi176.stdout:1879 clone 16 order 23 su 65536 sc 12 2024-09-14T09:06:09.236 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:06:09.269 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:09.271+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:09.370 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:09.363+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:09.393 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:09.393 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:09.407 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:06:09.494 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:09.491+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:09.570 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:09.571 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:09.571 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:09.571+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:09.585 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:06:09.672 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:09.675+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:09.742 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:09.742 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:09.754 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:06:09.798 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:09.795+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:09.861 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:09.861 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:09.874 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:06:09.874 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:09.875+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:09.975 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:09.975+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:10.042 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:10.042 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:10.057 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:06:10.102 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:10.099+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:10.114 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T09:06:10.164 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:06:10.188 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:10.188 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:10.188 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:10.183+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:10.205 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:06:10.277 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:10.279+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:10.327 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:10.327 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:10.359 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:10.359 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:10.377 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:10.371+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:10.380 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:10.380 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:10.382 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:10.382 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:10.394 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:06:10.401 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:10.401 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:10.423 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:10.423 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:10.438 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-14T09:06:10.446 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:10.446 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:10.446 INFO:teuthology.orchestra.run.smithi176.stdout:1880 writesame 0x53241b0 thru 0x5326d1f data_size 0x28(0x2b70 bytes) 2024-09-14T09:06:10.479 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:10.475+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:10.481 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:10.481 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:10.502 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:10.503 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:10.511 INFO:teuthology.orchestra.run.smithi176.stdout:1881 trunc from 0xe6bfa10 to 0x29f085a 2024-09-14T09:06:10.538 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:10.538 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:10.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.538 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49287 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.538 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.519980+0000", 2024-09-14T09:06:10.538 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.014324366999999999, 2024-09-14T09:06:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014342074999999999, 2024-09-14T09:06:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49287 2024-09-14T09:06:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519980+0000", 2024-09-14T09:06:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519980+0000", 2024-09-14T09:06:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519980+0000", 2024-09-14T09:06:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5999999999999999e-07 2024-09-14T09:06:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519982+0000", 2024-09-14T09:06:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.249e-06 2024-09-14T09:06:10.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519989+0000", 2024-09-14T09:06:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1829999999999999e-06 2024-09-14T09:06:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519994+0000", 2024-09-14T09:06:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8980000000000004e-06 2024-09-14T09:06:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49543 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526428+0000", 2024-09-14T09:06:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0078766640000000002, 2024-09-14T09:06:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0079289510000000001, 2024-09-14T09:06:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.542 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49543 2024-09-14T09:06:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526428+0000", 2024-09-14T09:06:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526428+0000", 2024-09-14T09:06:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526428+0000", 2024-09-14T09:06:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7500000000000001e-07 2024-09-14T09:06:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526430+0000", 2024-09-14T09:06:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1330000000000002e-06 2024-09-14T09:06:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526436+0000", 2024-09-14T09:06:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.083e-06 2024-09-14T09:06:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526441+0000", 2024-09-14T09:06:10.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1449999999999999e-06 2024-09-14T09:06:10.546 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.546 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.546 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.546 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49288 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.546 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.520001+0000", 2024-09-14T09:06:10.546 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.014302934, 2024-09-14T09:06:10.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014406251, 2024-09-14T09:06:10.546 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.547 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.547 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.547 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.547 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.547 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.547 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49288 2024-09-14T09:06:10.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.547 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520001+0000", 2024-09-14T09:06:10.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520001+0000", 2024-09-14T09:06:10.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520002+0000", 2024-09-14T09:06:10.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7300000000000002e-07 2024-09-14T09:06:10.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520004+0000", 2024-09-14T09:06:10.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.148e-06 2024-09-14T09:06:10.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520010+0000", 2024-09-14T09:06:10.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8359999999999997e-06 2024-09-14T09:06:10.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520015+0000", 2024-09-14T09:06:10.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0780000000000003e-06 2024-09-14T09:06:10.551 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.551 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.552 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.552 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49544 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.552 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526448+0000", 2024-09-14T09:06:10.552 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.007855796, 2024-09-14T09:06:10.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0080096620000000007, 2024-09-14T09:06:10.552 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.553 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.553 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.553 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.553 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.553 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.553 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49544 2024-09-14T09:06:10.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.553 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526448+0000", 2024-09-14T09:06:10.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526448+0000", 2024-09-14T09:06:10.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526449+0000", 2024-09-14T09:06:10.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7800000000000002e-07 2024-09-14T09:06:10.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526451+0000", 2024-09-14T09:06:10.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9300000000000002e-06 2024-09-14T09:06:10.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526459+0000", 2024-09-14T09:06:10.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3100000000000001e-06 2024-09-14T09:06:10.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526464+0000", 2024-09-14T09:06:10.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0209999999999999e-06 2024-09-14T09:06:10.557 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.557 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.557 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.558 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49364 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.558 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.520900+0000", 2024-09-14T09:06:10.558 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.013404463, 2024-09-14T09:06:10.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013592663, 2024-09-14T09:06:10.558 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.558 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.559 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.559 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.559 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.559 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.559 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49364 2024-09-14T09:06:10.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.559 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520900+0000", 2024-09-14T09:06:10.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520900+0000", 2024-09-14T09:06:10.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520901+0000", 2024-09-14T09:06:10.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7400000000000002e-07 2024-09-14T09:06:10.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520906+0000", 2024-09-14T09:06:10.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9080000000000003e-06 2024-09-14T09:06:10.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520921+0000", 2024-09-14T09:06:10.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5452e-05 2024-09-14T09:06:10.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520929+0000", 2024-09-14T09:06:10.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8639999999999996e-06 2024-09-14T09:06:10.562 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.562 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.562 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.562 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49545 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.563 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526522+0000", 2024-09-14T09:06:10.563 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0077819580000000003, 2024-09-14T09:06:10.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0080085839999999991, 2024-09-14T09:06:10.563 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.563 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.563 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.563 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.563 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.563 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.563 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49545 2024-09-14T09:06:10.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.563 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526522+0000", 2024-09-14T09:06:10.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526522+0000", 2024-09-14T09:06:10.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526523+0000", 2024-09-14T09:06:10.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1799999999999995e-07 2024-09-14T09:06:10.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526526+0000", 2024-09-14T09:06:10.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.982e-06 2024-09-14T09:06:10.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526535+0000", 2024-09-14T09:06:10.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0399999999999998e-06 2024-09-14T09:06:10.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526541+0000", 2024-09-14T09:06:10.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7300000000000002e-06 2024-09-14T09:06:10.566 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.566 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.566 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.566 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49365 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.566 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.520957+0000", 2024-09-14T09:06:10.566 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.013347725, 2024-09-14T09:06:10.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013625757, 2024-09-14T09:06:10.566 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.566 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.566 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.566 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.566 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.567 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.567 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49365 2024-09-14T09:06:10.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.567 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520957+0000", 2024-09-14T09:06:10.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520957+0000", 2024-09-14T09:06:10.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520957+0000", 2024-09-14T09:06:10.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0999999999999998e-07 2024-09-14T09:06:10.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520960+0000", 2024-09-14T09:06:10.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1839999999999999e-06 2024-09-14T09:06:10.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520969+0000", 2024-09-14T09:06:10.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2810000000000008e-06 2024-09-14T09:06:10.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520975+0000", 2024-09-14T09:06:10.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4099999999999996e-06 2024-09-14T09:06:10.569 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.569 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.569 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.569 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49620 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.569 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526550+0000", 2024-09-14T09:06:10.569 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0077544090000000003, 2024-09-14T09:06:10.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0080736899999999997, 2024-09-14T09:06:10.570 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.570 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.570 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.570 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.570 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.570 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.570 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49620 2024-09-14T09:06:10.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.570 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526550+0000", 2024-09-14T09:06:10.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526550+0000", 2024-09-14T09:06:10.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526550+0000", 2024-09-14T09:06:10.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.72e-07 2024-09-14T09:06:10.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526552+0000", 2024-09-14T09:06:10.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1710000000000001e-06 2024-09-14T09:06:10.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526559+0000", 2024-09-14T09:06:10.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4370000000000004e-06 2024-09-14T09:06:10.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526564+0000", 2024-09-14T09:06:10.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9740000000000001e-06 2024-09-14T09:06:10.573 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.573 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.573 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.573 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49366 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.573 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.520985+0000", 2024-09-14T09:06:10.573 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.013319394, 2024-09-14T09:06:10.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013687411999999999, 2024-09-14T09:06:10.573 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.573 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.573 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.573 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.574 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.574 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.574 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49366 2024-09-14T09:06:10.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.574 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520985+0000", 2024-09-14T09:06:10.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520985+0000", 2024-09-14T09:06:10.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520985+0000", 2024-09-14T09:06:10.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7999999999999996e-07 2024-09-14T09:06:10.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520988+0000", 2024-09-14T09:06:10.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5730000000000002e-06 2024-09-14T09:06:10.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.520994+0000", 2024-09-14T09:06:10.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4949999999999996e-06 2024-09-14T09:06:10.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521000+0000", 2024-09-14T09:06:10.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6620000000000002e-06 2024-09-14T09:06:10.576 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.576 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.576 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.576 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49621 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.576 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526571+0000", 2024-09-14T09:06:10.576 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0077334350000000003, 2024-09-14T09:06:10.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0081572659999999998, 2024-09-14T09:06:10.577 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.577 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.577 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.577 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.577 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.577 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.577 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49621 2024-09-14T09:06:10.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.577 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526571+0000", 2024-09-14T09:06:10.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526571+0000", 2024-09-14T09:06:10.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526571+0000", 2024-09-14T09:06:10.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2399999999999999e-07 2024-09-14T09:06:10.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526573+0000", 2024-09-14T09:06:10.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2740000000000002e-06 2024-09-14T09:06:10.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526582+0000", 2024-09-14T09:06:10.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3510000000000002e-06 2024-09-14T09:06:10.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526589+0000", 2024-09-14T09:06:10.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4410000000000004e-06 2024-09-14T09:06:10.579 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.579 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.579 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.580 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49367 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.580 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.521007+0000", 2024-09-14T09:06:10.580 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.013297174, 2024-09-14T09:06:10.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013763164, 2024-09-14T09:06:10.580 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.580 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.580 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.580 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.580 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.580 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.580 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49367 2024-09-14T09:06:10.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.580 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521007+0000", 2024-09-14T09:06:10.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521007+0000", 2024-09-14T09:06:10.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521007+0000", 2024-09-14T09:06:10.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4900000000000001e-07 2024-09-14T09:06:10.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521009+0000", 2024-09-14T09:06:10.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6270000000000001e-06 2024-09-14T09:06:10.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521014+0000", 2024-09-14T09:06:10.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2900000000000002e-06 2024-09-14T09:06:10.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521017+0000", 2024-09-14T09:06:10.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9799999999999998e-06 2024-09-14T09:06:10.582 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.583 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.583 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.583 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49622 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.583 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526596+0000", 2024-09-14T09:06:10.583 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0077080179999999996, 2024-09-14T09:06:10.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0081992750000000007, 2024-09-14T09:06:10.583 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.583 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.583 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.583 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49622 2024-09-14T09:06:10.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.584 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526596+0000", 2024-09-14T09:06:10.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526596+0000", 2024-09-14T09:06:10.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526597+0000", 2024-09-14T09:06:10.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0100000000000001e-07 2024-09-14T09:06:10.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526602+0000", 2024-09-14T09:06:10.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6609999999999997e-06 2024-09-14T09:06:10.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526610+0000", 2024-09-14T09:06:10.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1359999999999997e-06 2024-09-14T09:06:10.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526616+0000", 2024-09-14T09:06:10.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2599999999999996e-06 2024-09-14T09:06:10.586 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.586 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.586 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.586 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49368 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.586 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.521021+0000", 2024-09-14T09:06:10.586 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.013283062999999999, 2024-09-14T09:06:10.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013809475, 2024-09-14T09:06:10.586 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.586 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.586 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.587 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.587 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.587 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.587 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49368 2024-09-14T09:06:10.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.587 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521021+0000", 2024-09-14T09:06:10.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521021+0000", 2024-09-14T09:06:10.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521021+0000", 2024-09-14T09:06:10.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4299999999999999e-07 2024-09-14T09:06:10.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521023+0000", 2024-09-14T09:06:10.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.314e-06 2024-09-14T09:06:10.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521026+0000", 2024-09-14T09:06:10.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0429999999999999e-06 2024-09-14T09:06:10.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521029+0000", 2024-09-14T09:06:10.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9280000000000002e-06 2024-09-14T09:06:10.590 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.590 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.590 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.590 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49623 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.590 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526622+0000", 2024-09-14T09:06:10.590 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0076817769999999999, 2024-09-14T09:06:10.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0082504459999999998, 2024-09-14T09:06:10.590 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.590 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.590 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.591 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.591 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.591 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.591 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49623 2024-09-14T09:06:10.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.591 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526622+0000", 2024-09-14T09:06:10.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526622+0000", 2024-09-14T09:06:10.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526623+0000", 2024-09-14T09:06:10.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.89e-07 2024-09-14T09:06:10.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526625+0000", 2024-09-14T09:06:10.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0880000000000002e-06 2024-09-14T09:06:10.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526631+0000", 2024-09-14T09:06:10.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2729999999999999e-06 2024-09-14T09:06:10.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526636+0000", 2024-09-14T09:06:10.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8919999999999999e-06 2024-09-14T09:06:10.593 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.593 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.593 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.593 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49369 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.593 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.521032+0000", 2024-09-14T09:06:10.593 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.013271855000000001, 2024-09-14T09:06:10.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013866209000000001, 2024-09-14T09:06:10.594 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.594 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.594 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.594 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.594 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.594 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.594 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49369 2024-09-14T09:06:10.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.594 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521032+0000", 2024-09-14T09:06:10.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521032+0000", 2024-09-14T09:06:10.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521033+0000", 2024-09-14T09:06:10.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3200000000000001e-07 2024-09-14T09:06:10.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521034+0000", 2024-09-14T09:06:10.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.243e-06 2024-09-14T09:06:10.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521037+0000", 2024-09-14T09:06:10.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0000000000000001e-06 2024-09-14T09:06:10.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521039+0000", 2024-09-14T09:06:10.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.632e-06 2024-09-14T09:06:10.596 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.596 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.596 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.597 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49624 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.597 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526642+0000", 2024-09-14T09:06:10.597 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0076620660000000004, 2024-09-14T09:06:10.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0082811599999999992, 2024-09-14T09:06:10.597 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.597 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.597 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.597 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.597 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.597 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.597 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49624 2024-09-14T09:06:10.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.598 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526642+0000", 2024-09-14T09:06:10.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526642+0000", 2024-09-14T09:06:10.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526643+0000", 2024-09-14T09:06:10.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2500000000000001e-07 2024-09-14T09:06:10.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526645+0000", 2024-09-14T09:06:10.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1830000000000001e-06 2024-09-14T09:06:10.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526651+0000", 2024-09-14T09:06:10.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1800000000000001e-06 2024-09-14T09:06:10.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526656+0000", 2024-09-14T09:06:10.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7990000000000001e-06 2024-09-14T09:06:10.600 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.600 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.600 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.600 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49113 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 2850816~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.600 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.517921+0000", 2024-09-14T09:06:10.600 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.016383222999999999, 2024-09-14T09:06:10.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017038406999999998, 2024-09-14T09:06:10.600 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.601 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.601 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:10.601 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.601 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.601 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.601 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49113 2024-09-14T09:06:10.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.601 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517921+0000", 2024-09-14T09:06:10.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517921+0000", 2024-09-14T09:06:10.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517921+0000", 2024-09-14T09:06:10.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9799999999999999e-07 2024-09-14T09:06:10.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517923+0000", 2024-09-14T09:06:10.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.877e-06 2024-09-14T09:06:10.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517929+0000", 2024-09-14T09:06:10.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9139999999999996e-06 2024-09-14T09:06:10.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517934+0000", 2024-09-14T09:06:10.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9729999999999997e-06 2024-09-14T09:06:10.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:10.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.534629+0000", 2024-09-14T09:06:10.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016694586000000001 2024-09-14T09:06:10.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:10.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.534706+0000", 2024-09-14T09:06:10.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7115999999999998e-05 2024-09-14T09:06:10.604 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.604 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.604 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.604 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49370 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.604 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.521043+0000", 2024-09-14T09:06:10.604 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.01326081, 2024-09-14T09:06:10.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01395599, 2024-09-14T09:06:10.605 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.605 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.605 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.605 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49370 2024-09-14T09:06:10.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.605 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521043+0000", 2024-09-14T09:06:10.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521043+0000", 2024-09-14T09:06:10.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521044+0000", 2024-09-14T09:06:10.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4799999999999999e-07 2024-09-14T09:06:10.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521046+0000", 2024-09-14T09:06:10.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.731e-06 2024-09-14T09:06:10.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521051+0000", 2024-09-14T09:06:10.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2830000000000001e-06 2024-09-14T09:06:10.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521056+0000", 2024-09-14T09:06:10.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7350000000000004e-06 2024-09-14T09:06:10.607 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.607 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.607 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.608 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49625 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.608 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526664+0000", 2024-09-14T09:06:10.608 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0076401999999999998, 2024-09-14T09:06:10.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.008383738, 2024-09-14T09:06:10.608 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.608 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.608 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.608 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.608 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.608 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.608 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49625 2024-09-14T09:06:10.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.608 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526664+0000", 2024-09-14T09:06:10.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526664+0000", 2024-09-14T09:06:10.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526664+0000", 2024-09-14T09:06:10.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4200000000000002e-07 2024-09-14T09:06:10.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526666+0000", 2024-09-14T09:06:10.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.007e-06 2024-09-14T09:06:10.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526674+0000", 2024-09-14T09:06:10.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8040000000000002e-06 2024-09-14T09:06:10.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526679+0000", 2024-09-14T09:06:10.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9119999999999997e-06 2024-09-14T09:06:10.611 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.611 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.611 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.611 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49114 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.611 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.517940+0000", 2024-09-14T09:06:10.611 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.016364487, 2024-09-14T09:06:10.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017145001999999999, 2024-09-14T09:06:10.611 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.611 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.611 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.611 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.611 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.612 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.612 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49114 2024-09-14T09:06:10.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.612 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517940+0000", 2024-09-14T09:06:10.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517940+0000", 2024-09-14T09:06:10.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517940+0000", 2024-09-14T09:06:10.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2000000000000001e-07 2024-09-14T09:06:10.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517942+0000", 2024-09-14T09:06:10.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.176e-06 2024-09-14T09:06:10.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517948+0000", 2024-09-14T09:06:10.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4970000000000001e-06 2024-09-14T09:06:10.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517952+0000", 2024-09-14T09:06:10.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1370000000000004e-06 2024-09-14T09:06:10.614 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.614 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.614 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.614 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49371 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.614 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.521062+0000", 2024-09-14T09:06:10.614 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.013241842, 2024-09-14T09:06:10.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014049551, 2024-09-14T09:06:10.614 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.615 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.615 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.615 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.615 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.615 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.615 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49371 2024-09-14T09:06:10.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.615 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521062+0000", 2024-09-14T09:06:10.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521062+0000", 2024-09-14T09:06:10.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521063+0000", 2024-09-14T09:06:10.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1600000000000002e-07 2024-09-14T09:06:10.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521065+0000", 2024-09-14T09:06:10.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5550000000000001e-06 2024-09-14T09:06:10.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521073+0000", 2024-09-14T09:06:10.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.712e-06 2024-09-14T09:06:10.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521078+0000", 2024-09-14T09:06:10.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.288e-06 2024-09-14T09:06:10.617 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.617 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.617 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.617 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49626 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.617 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526685+0000", 2024-09-14T09:06:10.618 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0076189300000000003, 2024-09-14T09:06:10.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0084674920000000001, 2024-09-14T09:06:10.618 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.618 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.618 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.618 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.618 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.618 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.618 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49626 2024-09-14T09:06:10.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.618 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526685+0000", 2024-09-14T09:06:10.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526685+0000", 2024-09-14T09:06:10.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526686+0000", 2024-09-14T09:06:10.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.72e-07 2024-09-14T09:06:10.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526688+0000", 2024-09-14T09:06:10.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2110000000000001e-06 2024-09-14T09:06:10.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526695+0000", 2024-09-14T09:06:10.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4309999999999997e-06 2024-09-14T09:06:10.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526700+0000", 2024-09-14T09:06:10.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7809999999999996e-06 2024-09-14T09:06:10.620 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.620 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.620 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.621 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49115 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.621 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.517958+0000", 2024-09-14T09:06:10.621 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.016346537000000001, 2024-09-14T09:06:10.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017245235000000001, 2024-09-14T09:06:10.621 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.621 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.621 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.621 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.621 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.621 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.621 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49115 2024-09-14T09:06:10.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.621 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517958+0000", 2024-09-14T09:06:10.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517958+0000", 2024-09-14T09:06:10.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517958+0000", 2024-09-14T09:06:10.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2899999999999999e-07 2024-09-14T09:06:10.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517960+0000", 2024-09-14T09:06:10.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8369999999999999e-06 2024-09-14T09:06:10.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517965+0000", 2024-09-14T09:06:10.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4739999999999996e-06 2024-09-14T09:06:10.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517970+0000", 2024-09-14T09:06:10.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6500000000000004e-06 2024-09-14T09:06:10.624 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.624 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.624 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.624 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49372 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.624 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.521086+0000", 2024-09-14T09:06:10.624 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.013218229999999999, 2024-09-14T09:06:10.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014155673000000001, 2024-09-14T09:06:10.624 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.624 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.624 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.624 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.624 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.625 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.625 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49372 2024-09-14T09:06:10.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.625 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521086+0000", 2024-09-14T09:06:10.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521086+0000", 2024-09-14T09:06:10.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521086+0000", 2024-09-14T09:06:10.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3000000000000001e-07 2024-09-14T09:06:10.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521089+0000", 2024-09-14T09:06:10.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3590000000000002e-06 2024-09-14T09:06:10.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521096+0000", 2024-09-14T09:06:10.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6299999999999998e-06 2024-09-14T09:06:10.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521102+0000", 2024-09-14T09:06:10.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1590000000000001e-06 2024-09-14T09:06:10.627 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.627 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.627 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.627 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49627 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.627 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526707+0000", 2024-09-14T09:06:10.627 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0075973220000000001, 2024-09-14T09:06:10.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0085856270000000002, 2024-09-14T09:06:10.627 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.627 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.627 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.627 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.627 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.628 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.628 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49627 2024-09-14T09:06:10.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.628 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526707+0000", 2024-09-14T09:06:10.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526707+0000", 2024-09-14T09:06:10.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526707+0000", 2024-09-14T09:06:10.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9000000000000002e-07 2024-09-14T09:06:10.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526710+0000", 2024-09-14T09:06:10.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9019999999999999e-06 2024-09-14T09:06:10.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526718+0000", 2024-09-14T09:06:10.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4909999999999999e-06 2024-09-14T09:06:10.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526723+0000", 2024-09-14T09:06:10.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.95e-06 2024-09-14T09:06:10.630 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.630 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.630 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.630 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49116 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.630 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.517976+0000", 2024-09-14T09:06:10.630 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.01632805, 2024-09-14T09:06:10.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017368280999999999, 2024-09-14T09:06:10.630 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.630 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.630 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.630 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.630 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.631 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.631 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49116 2024-09-14T09:06:10.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.631 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517976+0000", 2024-09-14T09:06:10.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517976+0000", 2024-09-14T09:06:10.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517977+0000", 2024-09-14T09:06:10.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.65e-07 2024-09-14T09:06:10.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517978+0000", 2024-09-14T09:06:10.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8700000000000001e-06 2024-09-14T09:06:10.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517984+0000", 2024-09-14T09:06:10.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9560000000000002e-06 2024-09-14T09:06:10.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517989+0000", 2024-09-14T09:06:10.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6959999999999996e-06 2024-09-14T09:06:10.633 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.633 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.633 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.633 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49373 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.633 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.521109+0000", 2024-09-14T09:06:10.633 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.013195136, 2024-09-14T09:06:10.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014286701000000001, 2024-09-14T09:06:10.634 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.634 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.634 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.634 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.634 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.634 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.634 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49373 2024-09-14T09:06:10.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.634 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521109+0000", 2024-09-14T09:06:10.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521109+0000", 2024-09-14T09:06:10.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521109+0000", 2024-09-14T09:06:10.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9099999999999999e-07 2024-09-14T09:06:10.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521112+0000", 2024-09-14T09:06:10.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2680000000000001e-06 2024-09-14T09:06:10.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521119+0000", 2024-09-14T09:06:10.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5140000000000004e-06 2024-09-14T09:06:10.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521124+0000", 2024-09-14T09:06:10.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1100000000000002e-06 2024-09-14T09:06:10.636 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.636 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.636 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.637 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49628 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.637 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526729+0000", 2024-09-14T09:06:10.637 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0075756319999999997, 2024-09-14T09:06:10.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0087004909999999994, 2024-09-14T09:06:10.637 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.637 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.637 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.637 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.637 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.637 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.637 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49628 2024-09-14T09:06:10.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.637 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526729+0000", 2024-09-14T09:06:10.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526729+0000", 2024-09-14T09:06:10.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526729+0000", 2024-09-14T09:06:10.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2000000000000001e-07 2024-09-14T09:06:10.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526731+0000", 2024-09-14T09:06:10.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2330000000000001e-06 2024-09-14T09:06:10.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526738+0000", 2024-09-14T09:06:10.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.861e-06 2024-09-14T09:06:10.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526742+0000", 2024-09-14T09:06:10.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2459999999999997e-06 2024-09-14T09:06:10.640 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.640 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.640 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.640 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49192 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.640 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.517995+0000", 2024-09-14T09:06:10.640 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.01630912, 2024-09-14T09:06:10.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017459024, 2024-09-14T09:06:10.640 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.640 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.640 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.640 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.640 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.641 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.641 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49192 2024-09-14T09:06:10.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.641 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517995+0000", 2024-09-14T09:06:10.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517995+0000", 2024-09-14T09:06:10.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.517995+0000", 2024-09-14T09:06:10.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5199999999999998e-07 2024-09-14T09:06:10.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518012+0000", 2024-09-14T09:06:10.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6923e-05 2024-09-14T09:06:10.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518019+0000", 2024-09-14T09:06:10.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9210000000000003e-06 2024-09-14T09:06:10.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518024+0000", 2024-09-14T09:06:10.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9250000000000003e-06 2024-09-14T09:06:10.643 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.644 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.644 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.644 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49374 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.644 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.521132+0000", 2024-09-14T09:06:10.644 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.013172418, 2024-09-14T09:06:10.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014346552, 2024-09-14T09:06:10.644 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.644 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.644 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.644 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.644 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.645 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.645 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49374 2024-09-14T09:06:10.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.645 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521132+0000", 2024-09-14T09:06:10.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521132+0000", 2024-09-14T09:06:10.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.646 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521132+0000", 2024-09-14T09:06:10.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9799999999999999e-07 2024-09-14T09:06:10.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.646 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521134+0000", 2024-09-14T09:06:10.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2570000000000002e-06 2024-09-14T09:06:10.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.646 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521142+0000", 2024-09-14T09:06:10.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0480000000000001e-06 2024-09-14T09:06:10.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.521146+0000", 2024-09-14T09:06:10.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8790000000000002e-06 2024-09-14T09:06:10.647 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.647 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.647 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.647 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49629 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.647 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526749+0000", 2024-09-14T09:06:10.647 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0075554960000000001, 2024-09-14T09:06:10.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0087544029999999991, 2024-09-14T09:06:10.647 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.647 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.647 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.648 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.648 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.648 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.648 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49629 2024-09-14T09:06:10.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.648 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526749+0000", 2024-09-14T09:06:10.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526749+0000", 2024-09-14T09:06:10.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526749+0000", 2024-09-14T09:06:10.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4299999999999999e-07 2024-09-14T09:06:10.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526751+0000", 2024-09-14T09:06:10.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.973e-06 2024-09-14T09:06:10.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526756+0000", 2024-09-14T09:06:10.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3020000000000002e-06 2024-09-14T09:06:10.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526761+0000", 2024-09-14T09:06:10.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5920000000000002e-06 2024-09-14T09:06:10.650 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.650 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.650 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.650 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49193 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.650 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.518032+0000", 2024-09-14T09:06:10.650 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.01627224, 2024-09-14T09:06:10.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017505251999999999, 2024-09-14T09:06:10.651 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.651 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.651 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.651 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.651 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.651 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.651 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49193 2024-09-14T09:06:10.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.651 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518032+0000", 2024-09-14T09:06:10.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518032+0000", 2024-09-14T09:06:10.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518032+0000", 2024-09-14T09:06:10.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2500000000000001e-07 2024-09-14T09:06:10.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518034+0000", 2024-09-14T09:06:10.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9980000000000002e-06 2024-09-14T09:06:10.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518040+0000", 2024-09-14T09:06:10.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8109999999999999e-06 2024-09-14T09:06:10.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518045+0000", 2024-09-14T09:06:10.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.651e-06 2024-09-14T09:06:10.654 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.654 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.654 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.654 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49705 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.654 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526767+0000", 2024-09-14T09:06:10.654 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0075375829999999996, 2024-09-14T09:06:10.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0088000260000000007, 2024-09-14T09:06:10.654 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.654 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.654 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.655 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.655 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.655 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.655 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49705 2024-09-14T09:06:10.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.655 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526767+0000", 2024-09-14T09:06:10.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526767+0000", 2024-09-14T09:06:10.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526767+0000", 2024-09-14T09:06:10.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1399999999999998e-07 2024-09-14T09:06:10.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526769+0000", 2024-09-14T09:06:10.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9690000000000001e-06 2024-09-14T09:06:10.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526774+0000", 2024-09-14T09:06:10.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9880000000000004e-06 2024-09-14T09:06:10.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526778+0000", 2024-09-14T09:06:10.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8550000000000004e-06 2024-09-14T09:06:10.657 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.657 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.657 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.657 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49194 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.657 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.518051+0000", 2024-09-14T09:06:10.657 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.016253185, 2024-09-14T09:06:10.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017540555999999999, 2024-09-14T09:06:10.658 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.658 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.658 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.658 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.658 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.658 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.658 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49194 2024-09-14T09:06:10.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.658 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518051+0000", 2024-09-14T09:06:10.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.659 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518051+0000", 2024-09-14T09:06:10.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.659 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518051+0000", 2024-09-14T09:06:10.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9299999999999999e-07 2024-09-14T09:06:10.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.659 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518054+0000", 2024-09-14T09:06:10.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1150000000000001e-06 2024-09-14T09:06:10.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518059+0000", 2024-09-14T09:06:10.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.84e-06 2024-09-14T09:06:10.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518064+0000", 2024-09-14T09:06:10.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6290000000000001e-06 2024-09-14T09:06:10.660 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.660 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.660 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.661 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49450 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.661 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526033+0000", 2024-09-14T09:06:10.661 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0082712889999999994, 2024-09-14T09:06:10.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0096312280000000004, 2024-09-14T09:06:10.661 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.661 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.661 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.661 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.661 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.661 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.661 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49450 2024-09-14T09:06:10.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.661 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526033+0000", 2024-09-14T09:06:10.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526033+0000", 2024-09-14T09:06:10.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526035+0000", 2024-09-14T09:06:10.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8199999999999999e-06 2024-09-14T09:06:10.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526040+0000", 2024-09-14T09:06:10.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1549999999999998e-06 2024-09-14T09:06:10.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526052+0000", 2024-09-14T09:06:10.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2225000000000001e-05 2024-09-14T09:06:10.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526060+0000", 2024-09-14T09:06:10.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9000000000000006e-06 2024-09-14T09:06:10.663 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.663 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.663 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.664 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49706 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.664 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526783+0000", 2024-09-14T09:06:10.664 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0075209989999999996, 2024-09-14T09:06:10.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0089216620000000003, 2024-09-14T09:06:10.664 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.664 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.664 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.664 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.664 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.664 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.664 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49706 2024-09-14T09:06:10.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.664 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526783+0000", 2024-09-14T09:06:10.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526783+0000", 2024-09-14T09:06:10.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526784+0000", 2024-09-14T09:06:10.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4900000000000001e-07 2024-09-14T09:06:10.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526785+0000", 2024-09-14T09:06:10.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7579999999999999e-06 2024-09-14T09:06:10.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526790+0000", 2024-09-14T09:06:10.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8829999999999997e-06 2024-09-14T09:06:10.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526795+0000", 2024-09-14T09:06:10.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5090000000000004e-06 2024-09-14T09:06:10.666 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.667 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.667 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.667 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49195 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.667 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.518070+0000", 2024-09-14T09:06:10.667 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.016234109, 2024-09-14T09:06:10.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017691281, 2024-09-14T09:06:10.667 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.667 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.667 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.667 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.667 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.668 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.668 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49195 2024-09-14T09:06:10.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.668 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518070+0000", 2024-09-14T09:06:10.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518070+0000", 2024-09-14T09:06:10.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518071+0000", 2024-09-14T09:06:10.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.77e-07 2024-09-14T09:06:10.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518072+0000", 2024-09-14T09:06:10.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.871e-06 2024-09-14T09:06:10.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518078+0000", 2024-09-14T09:06:10.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6890000000000001e-06 2024-09-14T09:06:10.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518083+0000", 2024-09-14T09:06:10.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6040000000000003e-06 2024-09-14T09:06:10.670 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.670 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.670 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.670 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49451 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.670 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526074+0000", 2024-09-14T09:06:10.670 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0082298219999999995, 2024-09-14T09:06:10.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0097416510000000005, 2024-09-14T09:06:10.670 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.670 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.671 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.671 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.671 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.671 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.671 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49451 2024-09-14T09:06:10.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.671 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526074+0000", 2024-09-14T09:06:10.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526074+0000", 2024-09-14T09:06:10.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526075+0000", 2024-09-14T09:06:10.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0200000000000002e-07 2024-09-14T09:06:10.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526078+0000", 2024-09-14T09:06:10.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0089999999999999e-06 2024-09-14T09:06:10.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526085+0000", 2024-09-14T09:06:10.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.357e-06 2024-09-14T09:06:10.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526091+0000", 2024-09-14T09:06:10.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3190000000000002e-06 2024-09-14T09:06:10.673 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.673 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.673 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.673 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49707 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.673 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526801+0000", 2024-09-14T09:06:10.674 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0075037389999999997, 2024-09-14T09:06:10.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0090488879999999997, 2024-09-14T09:06:10.674 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.674 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.674 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.674 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.674 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.674 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.674 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49707 2024-09-14T09:06:10.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.674 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526801+0000", 2024-09-14T09:06:10.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526801+0000", 2024-09-14T09:06:10.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526801+0000", 2024-09-14T09:06:10.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4799999999999999e-07 2024-09-14T09:06:10.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526803+0000", 2024-09-14T09:06:10.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0719999999999998e-06 2024-09-14T09:06:10.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526809+0000", 2024-09-14T09:06:10.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8170000000000004e-06 2024-09-14T09:06:10.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526813+0000", 2024-09-14T09:06:10.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9650000000000002e-06 2024-09-14T09:06:10.676 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.677 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.677 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.677 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49196 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.677 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.518089+0000", 2024-09-14T09:06:10.677 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.016215697000000001, 2024-09-14T09:06:10.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017786059, 2024-09-14T09:06:10.677 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.677 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.677 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.677 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.678 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.678 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.678 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49196 2024-09-14T09:06:10.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.678 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518089+0000", 2024-09-14T09:06:10.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518089+0000", 2024-09-14T09:06:10.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518089+0000", 2024-09-14T09:06:10.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.46e-07 2024-09-14T09:06:10.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518091+0000", 2024-09-14T09:06:10.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8649999999999999e-06 2024-09-14T09:06:10.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518096+0000", 2024-09-14T09:06:10.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5439999999999998e-06 2024-09-14T09:06:10.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518099+0000", 2024-09-14T09:06:10.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7250000000000002e-06 2024-09-14T09:06:10.680 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.680 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.680 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.680 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49452 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.680 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526099+0000", 2024-09-14T09:06:10.680 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0082052229999999993, 2024-09-14T09:06:10.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0098308089999999994, 2024-09-14T09:06:10.680 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.680 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.680 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.680 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.681 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.681 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.681 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49452 2024-09-14T09:06:10.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.681 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526099+0000", 2024-09-14T09:06:10.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526099+0000", 2024-09-14T09:06:10.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526099+0000", 2024-09-14T09:06:10.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3000000000000001e-07 2024-09-14T09:06:10.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526102+0000", 2024-09-14T09:06:10.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.413e-06 2024-09-14T09:06:10.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526108+0000", 2024-09-14T09:06:10.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5509999999999996e-06 2024-09-14T09:06:10.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526114+0000", 2024-09-14T09:06:10.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1499999999999998e-06 2024-09-14T09:06:10.683 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.683 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.683 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.683 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49708 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.683 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526878+0000", 2024-09-14T09:06:10.683 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0074260410000000004, 2024-09-14T09:06:10.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0090989750000000005, 2024-09-14T09:06:10.683 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.683 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.684 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.684 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.684 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.684 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.684 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49708 2024-09-14T09:06:10.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.684 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526878+0000", 2024-09-14T09:06:10.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526878+0000", 2024-09-14T09:06:10.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526879+0000", 2024-09-14T09:06:10.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.13e-07 2024-09-14T09:06:10.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526882+0000", 2024-09-14T09:06:10.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7480000000000002e-06 2024-09-14T09:06:10.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526889+0000", 2024-09-14T09:06:10.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4270000000000002e-06 2024-09-14T09:06:10.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526894+0000", 2024-09-14T09:06:10.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9599999999999999e-06 2024-09-14T09:06:10.686 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.686 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.686 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.686 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49197 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.686 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.518103+0000", 2024-09-14T09:06:10.687 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.016201179999999999, 2024-09-14T09:06:10.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017925465000000002, 2024-09-14T09:06:10.687 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.687 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.687 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.687 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.687 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.687 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.687 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49197 2024-09-14T09:06:10.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.687 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518103+0000", 2024-09-14T09:06:10.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518103+0000", 2024-09-14T09:06:10.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518103+0000", 2024-09-14T09:06:10.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7099999999999998e-07 2024-09-14T09:06:10.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518105+0000", 2024-09-14T09:06:10.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2559999999999999e-06 2024-09-14T09:06:10.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518108+0000", 2024-09-14T09:06:10.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2890000000000002e-06 2024-09-14T09:06:10.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518112+0000", 2024-09-14T09:06:10.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8130000000000002e-06 2024-09-14T09:06:10.690 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.690 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.690 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.690 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49453 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.690 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526121+0000", 2024-09-14T09:06:10.690 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0081835390000000001, 2024-09-14T09:06:10.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0099622600000000006, 2024-09-14T09:06:10.691 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.691 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.691 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.691 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.691 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.691 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.691 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49453 2024-09-14T09:06:10.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.691 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526121+0000", 2024-09-14T09:06:10.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526121+0000", 2024-09-14T09:06:10.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526121+0000", 2024-09-14T09:06:10.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3300000000000003e-07 2024-09-14T09:06:10.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526123+0000", 2024-09-14T09:06:10.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1459999999999999e-06 2024-09-14T09:06:10.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526130+0000", 2024-09-14T09:06:10.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2759999999999997e-06 2024-09-14T09:06:10.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526135+0000", 2024-09-14T09:06:10.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9830000000000004e-06 2024-09-14T09:06:10.693 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.693 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.693 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.694 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49709 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.694 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526902+0000", 2024-09-14T09:06:10.694 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0074017830000000003, 2024-09-14T09:06:10.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0092307610000000005, 2024-09-14T09:06:10.694 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.694 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.694 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.694 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.694 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.694 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.694 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49709 2024-09-14T09:06:10.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.694 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526902+0000", 2024-09-14T09:06:10.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526902+0000", 2024-09-14T09:06:10.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526903+0000", 2024-09-14T09:06:10.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3500000000000002e-07 2024-09-14T09:06:10.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526908+0000", 2024-09-14T09:06:10.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4179999999999996e-06 2024-09-14T09:06:10.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526914+0000", 2024-09-14T09:06:10.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.981e-06 2024-09-14T09:06:10.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526918+0000", 2024-09-14T09:06:10.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0770000000000001e-06 2024-09-14T09:06:10.697 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.697 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.697 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.697 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49198 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.697 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.518144+0000", 2024-09-14T09:06:10.697 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.016159985000000002, 2024-09-14T09:06:10.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018042997000000002, 2024-09-14T09:06:10.697 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.697 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.697 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.698 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.698 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.698 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.698 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49198 2024-09-14T09:06:10.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.698 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518144+0000", 2024-09-14T09:06:10.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518144+0000", 2024-09-14T09:06:10.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518145+0000", 2024-09-14T09:06:10.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8999999999999997e-07 2024-09-14T09:06:10.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518147+0000", 2024-09-14T09:06:10.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3499999999999999e-06 2024-09-14T09:06:10.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518153+0000", 2024-09-14T09:06:10.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2840000000000003e-06 2024-09-14T09:06:10.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518159+0000", 2024-09-14T09:06:10.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2630000000000003e-06 2024-09-14T09:06:10.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.700 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.701 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49454 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.701 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526144+0000", 2024-09-14T09:06:10.701 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0081598939999999991, 2024-09-14T09:06:10.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010094262, 2024-09-14T09:06:10.701 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.701 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.701 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.701 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49454 2024-09-14T09:06:10.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.701 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526144+0000", 2024-09-14T09:06:10.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526144+0000", 2024-09-14T09:06:10.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526145+0000", 2024-09-14T09:06:10.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.32e-07 2024-09-14T09:06:10.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526147+0000", 2024-09-14T09:06:10.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.278e-06 2024-09-14T09:06:10.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526153+0000", 2024-09-14T09:06:10.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4219999999999997e-06 2024-09-14T09:06:10.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526159+0000", 2024-09-14T09:06:10.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1229999999999999e-06 2024-09-14T09:06:10.703 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.703 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.704 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.704 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49710 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.704 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526924+0000", 2024-09-14T09:06:10.704 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0073799290000000003, 2024-09-14T09:06:10.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0093686069999999993, 2024-09-14T09:06:10.704 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.704 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.704 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.704 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.704 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.704 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.705 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49710 2024-09-14T09:06:10.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.705 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526924+0000", 2024-09-14T09:06:10.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526924+0000", 2024-09-14T09:06:10.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526925+0000", 2024-09-14T09:06:10.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3599999999999999e-07 2024-09-14T09:06:10.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526926+0000", 2024-09-14T09:06:10.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.795e-06 2024-09-14T09:06:10.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526932+0000", 2024-09-14T09:06:10.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0839999999999999e-06 2024-09-14T09:06:10.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526936+0000", 2024-09-14T09:06:10.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3440000000000003e-06 2024-09-14T09:06:10.707 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.707 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.707 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.707 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49199 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.707 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.518166+0000", 2024-09-14T09:06:10.707 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.016137810999999998, 2024-09-14T09:06:10.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018177368999999999, 2024-09-14T09:06:10.707 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.707 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.707 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.707 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.708 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.708 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.708 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49199 2024-09-14T09:06:10.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.708 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518166+0000", 2024-09-14T09:06:10.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518166+0000", 2024-09-14T09:06:10.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518167+0000", 2024-09-14T09:06:10.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0200000000000003e-07 2024-09-14T09:06:10.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518169+0000", 2024-09-14T09:06:10.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0329999999999998e-06 2024-09-14T09:06:10.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518175+0000", 2024-09-14T09:06:10.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7440000000000004e-06 2024-09-14T09:06:10.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518179+0000", 2024-09-14T09:06:10.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7879999999999997e-06 2024-09-14T09:06:10.710 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.710 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.710 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.710 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49455 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.711 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526166+0000", 2024-09-14T09:06:10.711 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0081383099999999993, 2024-09-14T09:06:10.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010231100999999999, 2024-09-14T09:06:10.711 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.711 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.711 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.711 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.711 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.711 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.711 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49455 2024-09-14T09:06:10.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.711 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526166+0000", 2024-09-14T09:06:10.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526166+0000", 2024-09-14T09:06:10.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526166+0000", 2024-09-14T09:06:10.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9700000000000002e-07 2024-09-14T09:06:10.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526168+0000", 2024-09-14T09:06:10.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1490000000000001e-06 2024-09-14T09:06:10.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526175+0000", 2024-09-14T09:06:10.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.331e-06 2024-09-14T09:06:10.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526180+0000", 2024-09-14T09:06:10.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8330000000000001e-06 2024-09-14T09:06:10.714 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.714 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.714 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.714 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49711 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.714 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:10.714 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0073624099999999998, 2024-09-14T09:06:10.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0095066339999999999, 2024-09-14T09:06:10.714 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.714 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.714 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.714 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.715 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.715 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.715 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49711 2024-09-14T09:06:10.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.715 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:10.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:10.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:10.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1800000000000002e-07 2024-09-14T09:06:10.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526944+0000", 2024-09-14T09:06:10.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.821e-06 2024-09-14T09:06:10.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526950+0000", 2024-09-14T09:06:10.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5600000000000001e-06 2024-09-14T09:06:10.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526954+0000", 2024-09-14T09:06:10.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.437e-06 2024-09-14T09:06:10.717 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.717 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.717 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.717 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49200 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.717 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.518186+0000", 2024-09-14T09:06:10.717 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.016118297, 2024-09-14T09:06:10.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018315325, 2024-09-14T09:06:10.718 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.718 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.718 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.718 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.718 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.718 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.718 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49200 2024-09-14T09:06:10.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.718 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518186+0000", 2024-09-14T09:06:10.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518186+0000", 2024-09-14T09:06:10.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518186+0000", 2024-09-14T09:06:10.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0400000000000002e-07 2024-09-14T09:06:10.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518188+0000", 2024-09-14T09:06:10.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9700000000000002e-06 2024-09-14T09:06:10.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518194+0000", 2024-09-14T09:06:10.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9490000000000001e-06 2024-09-14T09:06:10.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518199+0000", 2024-09-14T09:06:10.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8500000000000002e-06 2024-09-14T09:06:10.720 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.720 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.721 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.721 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49456 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.721 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526186+0000", 2024-09-14T09:06:10.721 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0081179459999999991, 2024-09-14T09:06:10.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01036903, 2024-09-14T09:06:10.721 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.721 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.721 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.721 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.721 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.721 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.722 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49456 2024-09-14T09:06:10.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.722 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526186+0000", 2024-09-14T09:06:10.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526186+0000", 2024-09-14T09:06:10.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526187+0000", 2024-09-14T09:06:10.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2100000000000002e-07 2024-09-14T09:06:10.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526189+0000", 2024-09-14T09:06:10.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.979e-06 2024-09-14T09:06:10.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526194+0000", 2024-09-14T09:06:10.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.716e-06 2024-09-14T09:06:10.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526199+0000", 2024-09-14T09:06:10.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.028e-06 2024-09-14T09:06:10.724 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.724 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.724 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.724 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49712 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.724 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:10.724 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0073442020000000002, 2024-09-14T09:06:10.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0096483130000000004, 2024-09-14T09:06:10.725 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.725 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.725 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.725 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.725 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.725 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.725 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49712 2024-09-14T09:06:10.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.725 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:10.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:10.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:10.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4799999999999999e-07 2024-09-14T09:06:10.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526962+0000", 2024-09-14T09:06:10.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8449999999999999e-06 2024-09-14T09:06:10.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526967+0000", 2024-09-14T09:06:10.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1059999999999999e-06 2024-09-14T09:06:10.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526971+0000", 2024-09-14T09:06:10.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8310000000000003e-06 2024-09-14T09:06:10.727 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.727 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.727 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.728 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49201 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.728 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.518205+0000", 2024-09-14T09:06:10.728 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.016098753, 2024-09-14T09:06:10.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018442521999999999, 2024-09-14T09:06:10.728 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.728 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.728 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.728 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.728 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.728 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.728 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49201 2024-09-14T09:06:10.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.729 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518205+0000", 2024-09-14T09:06:10.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518205+0000", 2024-09-14T09:06:10.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518206+0000", 2024-09-14T09:06:10.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0400000000000002e-07 2024-09-14T09:06:10.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518208+0000", 2024-09-14T09:06:10.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8589999999999999e-06 2024-09-14T09:06:10.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518213+0000", 2024-09-14T09:06:10.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6670000000000001e-06 2024-09-14T09:06:10.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518218+0000", 2024-09-14T09:06:10.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8049999999999997e-06 2024-09-14T09:06:10.731 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.731 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.731 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.731 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49457 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.731 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526206+0000", 2024-09-14T09:06:10.731 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0080979759999999998, 2024-09-14T09:06:10.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010469368999999999, 2024-09-14T09:06:10.731 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.732 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.732 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.732 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.732 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.732 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.732 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49457 2024-09-14T09:06:10.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.732 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526206+0000", 2024-09-14T09:06:10.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526206+0000", 2024-09-14T09:06:10.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526207+0000", 2024-09-14T09:06:10.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9900000000000001e-07 2024-09-14T09:06:10.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526209+0000", 2024-09-14T09:06:10.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.996e-06 2024-09-14T09:06:10.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526215+0000", 2024-09-14T09:06:10.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8599999999999998e-06 2024-09-14T09:06:10.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526219+0000", 2024-09-14T09:06:10.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5260000000000004e-06 2024-09-14T09:06:10.735 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.735 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.735 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.735 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49713 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.735 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:10.735 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0073275299999999996, 2024-09-14T09:06:10.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0097381050000000004, 2024-09-14T09:06:10.735 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.735 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.735 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.735 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.736 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.736 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.736 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49713 2024-09-14T09:06:10.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.736 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:10.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:10.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:10.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1899999999999998e-07 2024-09-14T09:06:10.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526979+0000", 2024-09-14T09:06:10.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8959999999999999e-06 2024-09-14T09:06:10.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526986+0000", 2024-09-14T09:06:10.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1720000000000001e-06 2024-09-14T09:06:10.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526991+0000", 2024-09-14T09:06:10.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.729e-06 2024-09-14T09:06:10.738 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.738 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.738 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.738 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49202 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.738 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.518225+0000", 2024-09-14T09:06:10.738 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.016079545000000001, 2024-09-14T09:06:10.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018515536999999999, 2024-09-14T09:06:10.738 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.739 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.739 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.739 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.739 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.739 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.739 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49202 2024-09-14T09:06:10.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.739 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518225+0000", 2024-09-14T09:06:10.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518225+0000", 2024-09-14T09:06:10.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518225+0000", 2024-09-14T09:06:10.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4900000000000001e-07 2024-09-14T09:06:10.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518227+0000", 2024-09-14T09:06:10.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9970000000000001e-06 2024-09-14T09:06:10.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518246+0000", 2024-09-14T09:06:10.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8916999999999999e-05 2024-09-14T09:06:10.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.518251+0000", 2024-09-14T09:06:10.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2140000000000004e-06 2024-09-14T09:06:10.741 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.741 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.741 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.742 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49458 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.742 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526225+0000", 2024-09-14T09:06:10.742 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0080794319999999992, 2024-09-14T09:06:10.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010542005, 2024-09-14T09:06:10.742 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.742 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.742 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.743 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.743 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.743 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.743 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49458 2024-09-14T09:06:10.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.743 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526225+0000", 2024-09-14T09:06:10.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526225+0000", 2024-09-14T09:06:10.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526225+0000", 2024-09-14T09:06:10.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8000000000000001e-07 2024-09-14T09:06:10.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526227+0000", 2024-09-14T09:06:10.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7320000000000001e-06 2024-09-14T09:06:10.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526232+0000", 2024-09-14T09:06:10.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5650000000000001e-06 2024-09-14T09:06:10.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526237+0000", 2024-09-14T09:06:10.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6079999999999998e-06 2024-09-14T09:06:10.747 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.747 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.747 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.747 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49714 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.747 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526997+0000", 2024-09-14T09:06:10.747 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0073069210000000001, 2024-09-14T09:06:10.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0097937979999999994, 2024-09-14T09:06:10.747 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.747 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.747 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.747 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.748 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.748 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.748 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49714 2024-09-14T09:06:10.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.748 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526997+0000", 2024-09-14T09:06:10.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526997+0000", 2024-09-14T09:06:10.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526998+0000", 2024-09-14T09:06:10.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7800000000000002e-07 2024-09-14T09:06:10.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.527000+0000", 2024-09-14T09:06:10.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9080000000000002e-06 2024-09-14T09:06:10.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.527007+0000", 2024-09-14T09:06:10.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1609999999999997e-06 2024-09-14T09:06:10.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.527011+0000", 2024-09-14T09:06:10.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0929999999999996e-06 2024-09-14T09:06:10.750 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.750 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.750 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.750 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49278 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.750 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.519710+0000", 2024-09-14T09:06:10.751 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.014594117, 2024-09-14T09:06:10.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017108649, 2024-09-14T09:06:10.751 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.751 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.751 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.751 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.751 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.751 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.751 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49278 2024-09-14T09:06:10.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.751 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519710+0000", 2024-09-14T09:06:10.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519710+0000", 2024-09-14T09:06:10.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519711+0000", 2024-09-14T09:06:10.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1519999999999999e-06 2024-09-14T09:06:10.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519717+0000", 2024-09-14T09:06:10.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3129999999999998e-06 2024-09-14T09:06:10.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519736+0000", 2024-09-14T09:06:10.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9028000000000001e-05 2024-09-14T09:06:10.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519746+0000", 2024-09-14T09:06:10.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0205e-05 2024-09-14T09:06:10.754 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.754 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.754 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.754 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49459 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.754 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526243+0000", 2024-09-14T09:06:10.754 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0080611439999999993, 2024-09-14T09:06:10.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010600208999999999, 2024-09-14T09:06:10.754 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.754 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.754 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49459 2024-09-14T09:06:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526243+0000", 2024-09-14T09:06:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526243+0000", 2024-09-14T09:06:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526243+0000", 2024-09-14T09:06:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5900000000000003e-07 2024-09-14T09:06:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526246+0000", 2024-09-14T09:06:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0899999999999999e-06 2024-09-14T09:06:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526251+0000", 2024-09-14T09:06:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5910000000000003e-06 2024-09-14T09:06:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526256+0000", 2024-09-14T09:06:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.69e-06 2024-09-14T09:06:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49279 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.519763+0000", 2024-09-14T09:06:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.014541294, 2024-09-14T09:06:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017106975999999999, 2024-09-14T09:06:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49279 2024-09-14T09:06:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519763+0000", 2024-09-14T09:06:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519763+0000", 2024-09-14T09:06:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519763+0000", 2024-09-14T09:06:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8699999999999995e-07 2024-09-14T09:06:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519767+0000", 2024-09-14T09:06:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.157e-06 2024-09-14T09:06:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519775+0000", 2024-09-14T09:06:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.259e-06 2024-09-14T09:06:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519781+0000", 2024-09-14T09:06:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0769999999999996e-06 2024-09-14T09:06:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49460 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526262+0000", 2024-09-14T09:06:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0080424550000000004, 2024-09-14T09:06:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010632801000000001, 2024-09-14T09:06:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49460 2024-09-14T09:06:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526262+0000", 2024-09-14T09:06:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526262+0000", 2024-09-14T09:06:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526262+0000", 2024-09-14T09:06:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9999999999999998e-07 2024-09-14T09:06:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526264+0000", 2024-09-14T09:06:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8330000000000001e-06 2024-09-14T09:06:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526270+0000", 2024-09-14T09:06:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5890000000000002e-06 2024-09-14T09:06:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526274+0000", 2024-09-14T09:06:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6369999999999998e-06 2024-09-14T09:06:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49280 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.519803+0000", 2024-09-14T09:06:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.014500844000000001, 2024-09-14T09:06:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017141902000000001, 2024-09-14T09:06:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49280 2024-09-14T09:06:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519803+0000", 2024-09-14T09:06:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519803+0000", 2024-09-14T09:06:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519804+0000", 2024-09-14T09:06:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7e-07 2024-09-14T09:06:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519807+0000", 2024-09-14T09:06:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6410000000000002e-06 2024-09-14T09:06:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519814+0000", 2024-09-14T09:06:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2150000000000004e-06 2024-09-14T09:06:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519819+0000", 2024-09-14T09:06:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2660000000000001e-06 2024-09-14T09:06:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49536 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526281+0000", 2024-09-14T09:06:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0080232810000000002, 2024-09-14T09:06:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010716573, 2024-09-14T09:06:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49536 2024-09-14T09:06:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526281+0000", 2024-09-14T09:06:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526281+0000", 2024-09-14T09:06:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526281+0000", 2024-09-14T09:06:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0600000000000001e-07 2024-09-14T09:06:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526287+0000", 2024-09-14T09:06:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6799999999999998e-06 2024-09-14T09:06:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526293+0000", 2024-09-14T09:06:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.314e-06 2024-09-14T09:06:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526298+0000", 2024-09-14T09:06:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9139999999999999e-06 2024-09-14T09:06:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49281 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.519828+0000", 2024-09-14T09:06:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.014476737999999999, 2024-09-14T09:06:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01722342, 2024-09-14T09:06:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49281 2024-09-14T09:06:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519828+0000", 2024-09-14T09:06:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519828+0000", 2024-09-14T09:06:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519828+0000", 2024-09-14T09:06:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1699999999999999e-07 2024-09-14T09:06:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519830+0000", 2024-09-14T09:06:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3460000000000001e-06 2024-09-14T09:06:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519837+0000", 2024-09-14T09:06:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2659999999999998e-06 2024-09-14T09:06:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519842+0000", 2024-09-14T09:06:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1970000000000004e-06 2024-09-14T09:06:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49537 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526306+0000", 2024-09-14T09:06:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0079984849999999996, 2024-09-14T09:06:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010797701, 2024-09-14T09:06:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49537 2024-09-14T09:06:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526306+0000", 2024-09-14T09:06:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526306+0000", 2024-09-14T09:06:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526306+0000", 2024-09-14T09:06:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2300000000000002e-07 2024-09-14T09:06:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526308+0000", 2024-09-14T09:06:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.295e-06 2024-09-14T09:06:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526315+0000", 2024-09-14T09:06:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3010000000000003e-06 2024-09-14T09:06:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526320+0000", 2024-09-14T09:06:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8729999999999998e-06 2024-09-14T09:06:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49282 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.519849+0000", 2024-09-14T09:06:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.014455091, 2024-09-14T09:06:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017368017999999999, 2024-09-14T09:06:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49282 2024-09-14T09:06:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519849+0000", 2024-09-14T09:06:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519849+0000", 2024-09-14T09:06:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519850+0000", 2024-09-14T09:06:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.89e-07 2024-09-14T09:06:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519852+0000", 2024-09-14T09:06:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1509999999999998e-06 2024-09-14T09:06:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519858+0000", 2024-09-14T09:06:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0669999999999997e-06 2024-09-14T09:06:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519863+0000", 2024-09-14T09:06:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9729999999999997e-06 2024-09-14T09:06:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49538 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526326+0000", 2024-09-14T09:06:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0079778439999999996, 2024-09-14T09:06:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010916436, 2024-09-14T09:06:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49538 2024-09-14T09:06:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526326+0000", 2024-09-14T09:06:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526326+0000", 2024-09-14T09:06:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526327+0000", 2024-09-14T09:06:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.01e-07 2024-09-14T09:06:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526329+0000", 2024-09-14T09:06:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0279999999999999e-06 2024-09-14T09:06:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526335+0000", 2024-09-14T09:06:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0780000000000001e-06 2024-09-14T09:06:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526339+0000", 2024-09-14T09:06:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5159999999999996e-06 2024-09-14T09:06:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49283 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.519870+0000", 2024-09-14T09:06:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.014434059000000001, 2024-09-14T09:06:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017409303000000001, 2024-09-14T09:06:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49283 2024-09-14T09:06:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519870+0000", 2024-09-14T09:06:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519870+0000", 2024-09-14T09:06:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519871+0000", 2024-09-14T09:06:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2599999999999998e-07 2024-09-14T09:06:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519873+0000", 2024-09-14T09:06:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3379999999999999e-06 2024-09-14T09:06:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519904+0000", 2024-09-14T09:06:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0784000000000002e-05 2024-09-14T09:06:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519910+0000", 2024-09-14T09:06:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2620000000000004e-06 2024-09-14T09:06:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49539 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526346+0000", 2024-09-14T09:06:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0079578519999999996, 2024-09-14T09:06:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.010970917, 2024-09-14T09:06:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49539 2024-09-14T09:06:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526346+0000", 2024-09-14T09:06:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526346+0000", 2024-09-14T09:06:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526347+0000", 2024-09-14T09:06:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8799999999999998e-07 2024-09-14T09:06:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526349+0000", 2024-09-14T09:06:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1579999999999999e-06 2024-09-14T09:06:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526355+0000", 2024-09-14T09:06:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.28e-06 2024-09-14T09:06:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526360+0000", 2024-09-14T09:06:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8949999999999997e-06 2024-09-14T09:06:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49284 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.519919+0000", 2024-09-14T09:06:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.014385484, 2024-09-14T09:06:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017438175, 2024-09-14T09:06:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49284 2024-09-14T09:06:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519919+0000", 2024-09-14T09:06:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519919+0000", 2024-09-14T09:06:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519919+0000", 2024-09-14T09:06:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4000000000000002e-07 2024-09-14T09:06:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519922+0000", 2024-09-14T09:06:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5019999999999999e-06 2024-09-14T09:06:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519928+0000", 2024-09-14T09:06:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3470000000000004e-06 2024-09-14T09:06:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519933+0000", 2024-09-14T09:06:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8130000000000003e-06 2024-09-14T09:06:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49540 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526367+0000", 2024-09-14T09:06:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0079374670000000001, 2024-09-14T09:06:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011026919, 2024-09-14T09:06:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49540 2024-09-14T09:06:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526367+0000", 2024-09-14T09:06:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526367+0000", 2024-09-14T09:06:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526367+0000", 2024-09-14T09:06:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6399999999999998e-07 2024-09-14T09:06:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526369+0000", 2024-09-14T09:06:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9700000000000002e-06 2024-09-14T09:06:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526375+0000", 2024-09-14T09:06:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2149999999999998e-06 2024-09-14T09:06:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526380+0000", 2024-09-14T09:06:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9099999999999996e-06 2024-09-14T09:06:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49285 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.519940+0000", 2024-09-14T09:06:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.014364170000000001, 2024-09-14T09:06:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017497465, 2024-09-14T09:06:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49285 2024-09-14T09:06:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519940+0000", 2024-09-14T09:06:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519940+0000", 2024-09-14T09:06:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519940+0000", 2024-09-14T09:06:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9299999999999999e-07 2024-09-14T09:06:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519943+0000", 2024-09-14T09:06:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.227e-06 2024-09-14T09:06:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519949+0000", 2024-09-14T09:06:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1240000000000002e-06 2024-09-14T09:06:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519954+0000", 2024-09-14T09:06:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8509999999999998e-06 2024-09-14T09:06:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49541 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526387+0000", 2024-09-14T09:06:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0079169029999999994, 2024-09-14T09:06:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011101313, 2024-09-14T09:06:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49541 2024-09-14T09:06:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526387+0000", 2024-09-14T09:06:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526387+0000", 2024-09-14T09:06:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526388+0000", 2024-09-14T09:06:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9900000000000001e-07 2024-09-14T09:06:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526390+0000", 2024-09-14T09:06:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0770000000000002e-06 2024-09-14T09:06:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526396+0000", 2024-09-14T09:06:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1999999999999999e-06 2024-09-14T09:06:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526401+0000", 2024-09-14T09:06:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6079999999999998e-06 2024-09-14T09:06:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49286 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.519961+0000", 2024-09-14T09:06:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.014343448, 2024-09-14T09:06:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017578133999999999, 2024-09-14T09:06:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49286 2024-09-14T09:06:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519961+0000", 2024-09-14T09:06:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519961+0000", 2024-09-14T09:06:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519961+0000", 2024-09-14T09:06:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8700000000000001e-07 2024-09-14T09:06:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519963+0000", 2024-09-14T09:06:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0940000000000002e-06 2024-09-14T09:06:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519969+0000", 2024-09-14T09:06:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8749999999999997e-06 2024-09-14T09:06:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.519974+0000", 2024-09-14T09:06:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6909999999999996e-06 2024-09-14T09:06:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49542 3.8 3.2f0cebc8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526407+0000", 2024-09-14T09:06:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0078970069999999993, 2024-09-14T09:06:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011185249, 2024-09-14T09:06:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49542 2024-09-14T09:06:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526407+0000", 2024-09-14T09:06:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526407+0000", 2024-09-14T09:06:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526408+0000", 2024-09-14T09:06:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9200000000000002e-07 2024-09-14T09:06:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526410+0000", 2024-09-14T09:06:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0619999999999999e-06 2024-09-14T09:06:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526416+0000", 2024-09-14T09:06:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1380000000000004e-06 2024-09-14T09:06:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526421+0000", 2024-09-14T09:06:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7500000000000003e-06 2024-09-14T09:06:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:06:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 78 2024-09-14T09:06:10.814 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:10.815 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:06:10.816 INFO:teuthology.orchestra.run.smithi176.stdout:1882 write 0x99d56da thru 0x99d8cea (0x3611 bytes) 2024-09-14T09:06:10.816 INFO:teuthology.orchestra.run.smithi176.stdout:1883 writesame 0xd2085bc thru 0xd208dd8 data_size 0x1(0x81d bytes) 2024-09-14T09:06:10.816 INFO:teuthology.orchestra.run.smithi176.stdout:1884 write 0x4889e7 thru 0x4907b3 (0x7dcd bytes) 2024-09-14T09:06:10.816 INFO:teuthology.orchestra.run.smithi176.stdout:1885 trunc from 0xd208dd9 to 0x359f5d4 2024-09-14T09:06:10.816 INFO:teuthology.orchestra.run.smithi176.stdout:1886 trunc from 0x359f5d4 to 0x949aa41 2024-09-14T09:06:10.817 INFO:teuthology.orchestra.run.smithi176.stdout:1887 read 0x159bf41 thru 0x15a1d58 (0x5e18 bytes) 2024-09-14T09:06:10.817 INFO:teuthology.orchestra.run.smithi176.stdout:1888 punch from 0x75b1295 to 0x75b2944, (0x16af bytes) 2024-09-14T09:06:10.817 INFO:teuthology.orchestra.run.smithi176.stdout:1889 write 0x478aab4 thru 0x4793cbe (0x920b bytes) 2024-09-14T09:06:10.817 INFO:teuthology.orchestra.run.smithi176.stdout:1892 clone 17 order 21 su 65536 sc 7 2024-09-14T09:06:10.817 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:10.811+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:10.873 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:06:10.873 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:06:10.873 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-14T09:06:10.873 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T09:06:10.873 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T09:06:10.873 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-14T09:06:10.920 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:10.923+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:10.984 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:10.984 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:10.984 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:10.984 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:10.984 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:10.998 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:06:11.005 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:11.005 INFO:teuthology.orchestra.run.smithi183.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T09:06:11.005 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:11.046 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:11.043+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:11.118 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:11.119+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:11.120 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is unset 2024-09-14T09:06:11.126 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:11.126 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:11.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:11.126 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:11.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.126 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:11.127 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:11.127 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 38.042251196000002, 2024-09-14T09:06:11.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:06:11.127 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.127 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.127 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:11.127 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:11.127 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:11.128 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.128 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:06:11.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.128 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:11.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:11.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:06:11.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:06:11.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:06:11.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:06:11.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:06:11.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:06:11.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:06:11.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:06:11.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:06:11.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:06:11.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:11.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:06:11.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:06:11.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:06:11.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:06:11.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:06:11.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:06:11.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:06:11.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:06:11.132 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.132 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.132 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.132 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:11.132 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:11.133 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 38.042196537999999, 2024-09-14T09:06:11.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:06:11.133 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.133 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.133 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:11.133 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:11.133 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:11.133 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.133 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:06:11.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.133 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:11.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:11.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:11.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:06:11.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:06:11.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:06:11.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:06:11.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:06:11.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:06:11.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:06:11.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:06:11.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:06:11.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:11.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:06:11.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:06:11.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:06:11.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:06:11.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:06:11.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:06:11.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:06:11.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:06:11.138 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.138 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.138 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.138 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:11.138 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:11.138 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 38.042091163999999, 2024-09-14T09:06:11.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:06:11.138 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.138 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.138 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:11.138 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:11.138 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:11.139 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.139 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:06:11.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.139 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:11.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:11.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:06:11.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:06:11.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:06:11.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:06:11.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:06:11.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:06:11.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:06:11.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:06:11.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:06:11.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:06:11.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:11.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:06:11.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:06:11.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:06:11.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:06:11.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:06:11.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:06:11.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:06:11.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:06:11.143 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.143 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.143 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.143 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:11.143 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:11.143 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 38.041734400000003, 2024-09-14T09:06:11.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:06:11.144 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.144 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.144 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:11.144 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:11.144 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:11.144 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.144 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:06:11.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.144 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:11.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:11.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:06:11.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:06:11.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:06:11.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:06:11.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:06:11.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:06:11.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:06:11.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:06:11.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:06:11.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:06:11.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:11.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:06:11.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:06:11.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:06:11.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:06:11.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:06:11.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:06:11.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:06:11.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:06:11.148 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.149 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.149 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.149 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:11.149 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:11.150 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 38.041649661999998, 2024-09-14T09:06:11.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:06:11.150 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.150 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.150 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:11.150 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:11.150 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:11.151 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.151 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:06:11.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.151 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:11.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:11.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:06:11.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:06:11.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:06:11.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:06:11.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:06:11.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:06:11.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:06:11.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:06:11.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:06:11.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:06:11.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:11.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:06:11.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:06:11.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:06:11.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:06:11.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:06:11.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:06:11.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:06:11.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:06:11.155 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.155 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.155 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.155 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:11.155 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:11.155 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 38.041561975999997, 2024-09-14T09:06:11.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:06:11.155 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.155 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.155 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:11.155 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:11.155 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:11.156 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.156 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:06:11.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.156 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:11.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:11.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:06:11.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:06:11.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:06:11.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:06:11.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:06:11.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:06:11.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:06:11.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:06:11.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:06:11.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:06:11.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:11.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:06:11.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:06:11.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:06:11.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:06:11.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:06:11.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:06:11.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:06:11.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:06:11.159 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.160 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.160 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.160 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18388 3.18 3:19e2be55:::rbd_data.115e3abd3b2c.0000000000000021:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:06:11.160 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:11.160 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.905648665000001, 2024-09-14T09:06:11.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042293648000000003, 2024-09-14T09:06:11.160 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.160 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.160 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.160 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:11.160 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:11.161 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.161 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18388 2024-09-14T09:06:11.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.161 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:11.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:11.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220236+0000", 2024-09-14T09:06:11.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1339999999999998e-06 2024-09-14T09:06:11.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220244+0000", 2024-09-14T09:06:11.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1950000000000003e-06 2024-09-14T09:06:11.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220262+0000", 2024-09-14T09:06:11.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8017e-05 2024-09-14T09:06:11.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220270+0000", 2024-09-14T09:06:11.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9480000000000008e-06 2024-09-14T09:06:11.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220357+0000", 2024-09-14T09:06:11.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7185e-05 2024-09-14T09:06:11.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220576+0000", 2024-09-14T09:06:11.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021912200000000001 2024-09-14T09:06:11.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:11.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.222435+0000", 2024-09-14T09:06:11.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018592350000000001 2024-09-14T09:06:11.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:11.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262439+0000", 2024-09-14T09:06:11.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040003368999999997 2024-09-14T09:06:11.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:11.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262452+0000", 2024-09-14T09:06:11.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2941e-05 2024-09-14T09:06:11.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:11.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262490+0000", 2024-09-14T09:06:11.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8478e-05 2024-09-14T09:06:11.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262527+0000", 2024-09-14T09:06:11.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7024000000000003e-05 2024-09-14T09:06:11.165 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.166 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.166 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.166 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:19543 3.5 e39/23 3:a536ff9b:::rbd_data.115e4c71d6e8.0000000000000002:head v 39'8, mlcod=39'6)", 2024-09-14T09:06:11.166 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:11.166 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.602794005, 2024-09-14T09:06:11.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.039280019999999999, 2024-09-14T09:06:11.166 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.166 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.166 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.166 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:11.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:11.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523090+0000", 2024-09-14T09:06:11.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.773e-06 2024-09-14T09:06:11.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537290+0000", 2024-09-14T09:06:11.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014200007000000001 2024-09-14T09:06:11.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537317+0000", 2024-09-14T09:06:11.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6704999999999999e-05 2024-09-14T09:06:11.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537328+0000", 2024-09-14T09:06:11.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1299e-05 2024-09-14T09:06:11.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537417+0000", 2024-09-14T09:06:11.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8869999999999994e-05 2024-09-14T09:06:11.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537445+0000", 2024-09-14T09:06:11.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8377999999999998e-05 2024-09-14T09:06:11.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:11.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562324+0000", 2024-09-14T09:06:11.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024879307999999999 2024-09-14T09:06:11.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562368+0000", 2024-09-14T09:06:11.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3680000000000002e-05 2024-09-14T09:06:11.171 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.171 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.171 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.171 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38002 3.1e 3:7df715aa:::rbd_data.115eb07a8abd.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:11.172 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:11.172 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 13.736130759, 2024-09-14T09:06:11.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.065148304000000004, 2024-09-14T09:06:11.172 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.172 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.172 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.172 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:11.172 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:11.173 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.173 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38002 2024-09-14T09:06:11.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.173 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:11.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:11.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389752+0000", 2024-09-14T09:06:11.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1829999999999999e-06 2024-09-14T09:06:11.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389758+0000", 2024-09-14T09:06:11.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0919999999999997e-06 2024-09-14T09:06:11.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389769+0000", 2024-09-14T09:06:11.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759e-05 2024-09-14T09:06:11.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389775+0000", 2024-09-14T09:06:11.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8819999999999998e-06 2024-09-14T09:06:11.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389824+0000", 2024-09-14T09:06:11.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8573999999999999e-05 2024-09-14T09:06:11.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389876+0000", 2024-09-14T09:06:11.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2643999999999999e-05 2024-09-14T09:06:11.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:06:11.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.392640+0000", 2024-09-14T09:06:11.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0027636409999999998 2024-09-14T09:06:11.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:11.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454672+0000", 2024-09-14T09:06:11.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062031703000000001 2024-09-14T09:06:11.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:11.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454765+0000", 2024-09-14T09:06:11.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2862000000000006e-05 2024-09-14T09:06:11.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:11.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454841+0000", 2024-09-14T09:06:11.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5857000000000002e-05 2024-09-14T09:06:11.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454900+0000", 2024-09-14T09:06:11.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9107000000000002e-05 2024-09-14T09:06:11.181 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.181 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.181 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.181 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:38002 3.1e e48/23 ondisk, result = 0)", 2024-09-14T09:06:11.182 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:11.182 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 13.72581417, 2024-09-14T09:06:11.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054841741999999999, 2024-09-14T09:06:11.182 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.182 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.182 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:11.182 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:11.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:11.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400069+0000", 2024-09-14T09:06:11.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.339e-06 2024-09-14T09:06:11.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400076+0000", 2024-09-14T09:06:11.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7800000000000003e-06 2024-09-14T09:06:11.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400093+0000", 2024-09-14T09:06:11.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6915999999999999e-05 2024-09-14T09:06:11.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400106+0000", 2024-09-14T09:06:11.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2785e-05 2024-09-14T09:06:11.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454713+0000", 2024-09-14T09:06:11.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054607041000000002 2024-09-14T09:06:11.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454724+0000", 2024-09-14T09:06:11.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1202e-05 2024-09-14T09:06:11.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454910+0000", 2024-09-14T09:06:11.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018567900000000001 2024-09-14T09:06:11.186 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.186 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.186 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.187 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38100 3.b 3:d0f67c24:::rbd_data.115eb07a8abd.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:11.187 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:11.187 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 13.47996051, 2024-09-14T09:06:11.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053701964999999997, 2024-09-14T09:06:11.187 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.187 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.187 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.187 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:11.187 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:11.187 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.187 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38100 2024-09-14T09:06:11.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.187 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:11.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:11.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645923+0000", 2024-09-14T09:06:11.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:06:11.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645928+0000", 2024-09-14T09:06:11.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0699999999999997e-06 2024-09-14T09:06:11.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645942+0000", 2024-09-14T09:06:11.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4069e-05 2024-09-14T09:06:11.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645951+0000", 2024-09-14T09:06:11.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3230000000000004e-06 2024-09-14T09:06:11.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646008+0000", 2024-09-14T09:06:11.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6583999999999999e-05 2024-09-14T09:06:11.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646089+0000", 2024-09-14T09:06:11.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1557999999999994e-05 2024-09-14T09:06:11.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:11.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.650355+0000", 2024-09-14T09:06:11.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004265525 2024-09-14T09:06:11.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:11.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699439+0000", 2024-09-14T09:06:11.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049083904999999997 2024-09-14T09:06:11.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:11.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699467+0000", 2024-09-14T09:06:11.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8198999999999999e-05 2024-09-14T09:06:11.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:11.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699552+0000", 2024-09-14T09:06:11.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T09:06:11.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699624+0000", 2024-09-14T09:06:11.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1725999999999997e-05 2024-09-14T09:06:11.192 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.193 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.193 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.193 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:06:11.193 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:11.193 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.9652699890000003, 2024-09-14T09:06:11.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:06:11.193 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.193 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.193 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.193 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:11.193 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:11.193 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.194 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:06:11.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.194 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:11.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:11.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:06:11.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:06:11.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:06:11.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:06:11.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:06:11.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:06:11.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:06:11.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:06:11.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:06:11.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:06:11.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:06:11.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:06:11.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:06:11.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:06:11.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:06:11.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:11.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:06:11.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:06:11.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:11.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:06:11.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:06:11.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:11.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:06:11.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:06:11.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:06:11.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:06:11.198 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.198 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.198 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.199 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:39401 3.b e56/53)", 2024-09-14T09:06:11.199 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:11.199 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.9393739229999998, 2024-09-14T09:06:11.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10443659600000001, 2024-09-14T09:06:11.199 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.199 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.199 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:11.199 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:11.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:11.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186509+0000", 2024-09-14T09:06:11.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2640000000000001e-06 2024-09-14T09:06:11.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186516+0000", 2024-09-14T09:06:11.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1070000000000001e-06 2024-09-14T09:06:11.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186533+0000", 2024-09-14T09:06:11.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7818000000000001e-05 2024-09-14T09:06:11.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186543+0000", 2024-09-14T09:06:11.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5170000000000005e-06 2024-09-14T09:06:11.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290885+0000", 2024-09-14T09:06:11.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.104342221 2024-09-14T09:06:11.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290893+0000", 2024-09-14T09:06:11.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2859999999999999e-06 2024-09-14T09:06:11.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290945+0000", 2024-09-14T09:06:11.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1382999999999999e-05 2024-09-14T09:06:11.202 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.202 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.202 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.203 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49708 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:11.203 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526878+0000", 2024-09-14T09:06:11.203 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.59900387300000002, 2024-09-14T09:06:11.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036535277999999997, 2024-09-14T09:06:11.203 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.203 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.203 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:11.203 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:11.203 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:11.203 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.203 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49708 2024-09-14T09:06:11.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.203 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526878+0000", 2024-09-14T09:06:11.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526878+0000", 2024-09-14T09:06:11.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526879+0000", 2024-09-14T09:06:11.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.13e-07 2024-09-14T09:06:11.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526882+0000", 2024-09-14T09:06:11.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7480000000000002e-06 2024-09-14T09:06:11.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526889+0000", 2024-09-14T09:06:11.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4270000000000002e-06 2024-09-14T09:06:11.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526894+0000", 2024-09-14T09:06:11.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9599999999999999e-06 2024-09-14T09:06:11.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563237+0000", 2024-09-14T09:06:11.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036342646999999999 2024-09-14T09:06:11.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563314+0000", 2024-09-14T09:06:11.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7511999999999999e-05 2024-09-14T09:06:11.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563413+0000", 2024-09-14T09:06:11.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9370999999999994e-05 2024-09-14T09:06:11.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.207 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.207 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49709 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:11.207 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526902+0000", 2024-09-14T09:06:11.207 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.59897961499999997, 2024-09-14T09:06:11.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036648100000000003, 2024-09-14T09:06:11.207 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.207 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.207 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:11.207 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:11.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:11.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.208 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49709 2024-09-14T09:06:11.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.208 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526902+0000", 2024-09-14T09:06:11.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526902+0000", 2024-09-14T09:06:11.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526903+0000", 2024-09-14T09:06:11.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3500000000000002e-07 2024-09-14T09:06:11.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526908+0000", 2024-09-14T09:06:11.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4179999999999996e-06 2024-09-14T09:06:11.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526914+0000", 2024-09-14T09:06:11.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.981e-06 2024-09-14T09:06:11.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526918+0000", 2024-09-14T09:06:11.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0770000000000001e-06 2024-09-14T09:06:11.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563445+0000", 2024-09-14T09:06:11.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036526378999999998 2024-09-14T09:06:11.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563498+0000", 2024-09-14T09:06:11.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3190999999999999e-05 2024-09-14T09:06:11.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563551+0000", 2024-09-14T09:06:11.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2719000000000001e-05 2024-09-14T09:06:11.211 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.211 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.211 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.211 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49710 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:11.211 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526924+0000", 2024-09-14T09:06:11.211 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.59895776099999998, 2024-09-14T09:06:11.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037006095000000003, 2024-09-14T09:06:11.211 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.211 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.211 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:11.211 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:11.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:11.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.212 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49710 2024-09-14T09:06:11.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.212 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526924+0000", 2024-09-14T09:06:11.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526924+0000", 2024-09-14T09:06:11.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526925+0000", 2024-09-14T09:06:11.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3599999999999999e-07 2024-09-14T09:06:11.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526926+0000", 2024-09-14T09:06:11.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.795e-06 2024-09-14T09:06:11.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526932+0000", 2024-09-14T09:06:11.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0839999999999999e-06 2024-09-14T09:06:11.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526936+0000", 2024-09-14T09:06:11.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3440000000000003e-06 2024-09-14T09:06:11.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563593+0000", 2024-09-14T09:06:11.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036657041000000001 2024-09-14T09:06:11.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563648+0000", 2024-09-14T09:06:11.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5152999999999999e-05 2024-09-14T09:06:11.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563930+0000", 2024-09-14T09:06:11.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028234199999999999 2024-09-14T09:06:11.215 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.215 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.215 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.215 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49711 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:11.215 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:11.216 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.59894024199999996, 2024-09-14T09:06:11.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037330945999999997, 2024-09-14T09:06:11.216 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.216 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.216 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:11.216 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:11.216 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:11.216 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.216 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49711 2024-09-14T09:06:11.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.216 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:11.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:11.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:11.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1800000000000002e-07 2024-09-14T09:06:11.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526944+0000", 2024-09-14T09:06:11.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.821e-06 2024-09-14T09:06:11.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526950+0000", 2024-09-14T09:06:11.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5600000000000001e-06 2024-09-14T09:06:11.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526954+0000", 2024-09-14T09:06:11.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.437e-06 2024-09-14T09:06:11.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563965+0000", 2024-09-14T09:06:11.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037010534999999997 2024-09-14T09:06:11.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564021+0000", 2024-09-14T09:06:11.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6728000000000003e-05 2024-09-14T09:06:11.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564273+0000", 2024-09-14T09:06:11.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025154700000000002 2024-09-14T09:06:11.219 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.220 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.220 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.220 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49712 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 3112960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:11.220 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:11.220 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.59892203399999999, 2024-09-14T09:06:11.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037795625999999999, 2024-09-14T09:06:11.220 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.220 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.220 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:11.220 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:11.220 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:11.220 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.221 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49712 2024-09-14T09:06:11.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.221 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:11.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:11.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:11.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4799999999999999e-07 2024-09-14T09:06:11.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526962+0000", 2024-09-14T09:06:11.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8449999999999999e-06 2024-09-14T09:06:11.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526967+0000", 2024-09-14T09:06:11.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1059999999999999e-06 2024-09-14T09:06:11.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526971+0000", 2024-09-14T09:06:11.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8310000000000003e-06 2024-09-14T09:06:11.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564324+0000", 2024-09-14T09:06:11.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037352926000000002 2024-09-14T09:06:11.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564411+0000", 2024-09-14T09:06:11.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7052000000000006e-05 2024-09-14T09:06:11.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564756+0000", 2024-09-14T09:06:11.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034451799999999998 2024-09-14T09:06:11.224 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.224 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.224 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.224 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49713 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 3506176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:11.224 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:11.224 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.598905362, 2024-09-14T09:06:11.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038135466, 2024-09-14T09:06:11.225 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.225 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.225 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:11.225 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:11.225 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:11.225 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.225 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49713 2024-09-14T09:06:11.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.225 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:11.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:11.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:11.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1899999999999998e-07 2024-09-14T09:06:11.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526979+0000", 2024-09-14T09:06:11.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8959999999999999e-06 2024-09-14T09:06:11.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526986+0000", 2024-09-14T09:06:11.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1720000000000001e-06 2024-09-14T09:06:11.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526991+0000", 2024-09-14T09:06:11.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.729e-06 2024-09-14T09:06:11.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564781+0000", 2024-09-14T09:06:11.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037790345000000003 2024-09-14T09:06:11.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564844+0000", 2024-09-14T09:06:11.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2656000000000005e-05 2024-09-14T09:06:11.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565112+0000", 2024-09-14T09:06:11.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026834899999999997 2024-09-14T09:06:11.229 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.229 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.229 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.229 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49714 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:11.229 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526997+0000", 2024-09-14T09:06:11.229 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.59888475299999999, 2024-09-14T09:06:11.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038531469999999998, 2024-09-14T09:06:11.229 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:11.229 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:11.229 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:11.229 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:11.229 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:11.229 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.229 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49714 2024-09-14T09:06:11.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.230 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:11.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:11.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526997+0000", 2024-09-14T09:06:11.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:11.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526997+0000", 2024-09-14T09:06:11.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:11.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:11.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526998+0000", 2024-09-14T09:06:11.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7800000000000002e-07 2024-09-14T09:06:11.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:11.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.527000+0000", 2024-09-14T09:06:11.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9080000000000002e-06 2024-09-14T09:06:11.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:11.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.527007+0000", 2024-09-14T09:06:11.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1609999999999997e-06 2024-09-14T09:06:11.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.527011+0000", 2024-09-14T09:06:11.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0929999999999996e-06 2024-09-14T09:06:11.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:11.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565152+0000", 2024-09-14T09:06:11.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038141520999999998 2024-09-14T09:06:11.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:11.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565236+0000", 2024-09-14T09:06:11.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3498999999999995e-05 2024-09-14T09:06:11.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:11.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:11.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:11.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565529+0000", 2024-09-14T09:06:11.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029291000000000001 2024-09-14T09:06:11.232 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.232 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.232 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.233 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:11.233 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:11.233 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:11.235 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:06:11.235 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone16 from 0x949aa41 (overlap 0x29f085a) to 0x8634de 2024-09-14T09:06:11.236 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:11.235+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:11.335 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:11.335+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:11.339 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-14T09:06:11.368 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:11.369 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:11.369 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:11.369 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:11.383 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:06:11.468 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:11.471+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:11.536 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:11.531+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:11.551 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:11.551 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:11.551 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:11.551 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:11.551 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:11.565 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:06:11.656 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:11.659+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:11.737 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:654 3.11 e29/23 3:8c8e358b:::rbd_object_map.115e1e95bf99:head v 29'3, mlcod=28'1)", 2024-09-14T09:06:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.863095+0000", 2024-09-14T09:06:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.874921350999998, 2024-09-14T09:06:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0070911430000000003, 2024-09-14T09:06:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863095+0000", 2024-09-14T09:06:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863095+0000", 2024-09-14T09:06:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863096+0000", 2024-09-14T09:06:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8700000000000004e-07 2024-09-14T09:06:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863110+0000", 2024-09-14T09:06:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4283e-05 2024-09-14T09:06:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863157+0000", 2024-09-14T09:06:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6029999999999998e-05 2024-09-14T09:06:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863177+0000", 2024-09-14T09:06:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0662999999999999e-05 2024-09-14T09:06:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863324+0000", 2024-09-14T09:06:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014713899999999999 2024-09-14T09:06:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863383+0000", 2024-09-14T09:06:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8218999999999999e-05 2024-09-14T09:06:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.870083+0000", 2024-09-14T09:06:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067001459999999997 2024-09-14T09:06:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.870186+0000", 2024-09-14T09:06:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103776 2024-09-14T09:06:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:758 3.14 3:2d2c8a97:::rbd_data.115e724e1d99.0000000000000007:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:06:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.888858+0000", 2024-09-14T09:06:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.849158369999998, 2024-09-14T09:06:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060111740000000002, 2024-09-14T09:06:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 758 2024-09-14T09:06:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888858+0000", 2024-09-14T09:06:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888858+0000", 2024-09-14T09:06:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888860+0000", 2024-09-14T09:06:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4529999999999999e-06 2024-09-14T09:06:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888867+0000", 2024-09-14T09:06:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7200000000000006e-06 2024-09-14T09:06:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888892+0000", 2024-09-14T09:06:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4729999999999999e-05 2024-09-14T09:06:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888927+0000", 2024-09-14T09:06:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4752999999999997e-05 2024-09-14T09:06:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.894354+0000", 2024-09-14T09:06:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054273209999999997 2024-09-14T09:06:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.894584+0000", 2024-09-14T09:06:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000229602 2024-09-14T09:06:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.894869+0000", 2024-09-14T09:06:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028559499999999998 2024-09-14T09:06:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:829 3.13 e29/23 3:cc36a41f:::rbd_data.115e1e95bf99.0000000000000000:head v 29'1, mlcod=0'0)", 2024-09-14T09:06:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.901655+0000", 2024-09-14T09:06:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.836361484000001, 2024-09-14T09:06:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021644360000000001, 2024-09-14T09:06:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901655+0000", 2024-09-14T09:06:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901655+0000", 2024-09-14T09:06:11.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901657+0000", 2024-09-14T09:06:11.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.153e-06 2024-09-14T09:06:11.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901897+0000", 2024-09-14T09:06:11.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000239461 2024-09-14T09:06:11.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901927+0000", 2024-09-14T09:06:11.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.029e-05 2024-09-14T09:06:11.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901946+0000", 2024-09-14T09:06:11.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9170000000000001e-05 2024-09-14T09:06:11.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902104+0000", 2024-09-14T09:06:11.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015812199999999999 2024-09-14T09:06:11.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902157+0000", 2024-09-14T09:06:11.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2905999999999998e-05 2024-09-14T09:06:11.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:11.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923226+0000", 2024-09-14T09:06:11.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021068470999999998 2024-09-14T09:06:11.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923300+0000", 2024-09-14T09:06:11.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3787000000000004e-05 2024-09-14T09:06:11.751 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.751 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.751 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.751 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:845 3.9 3:928dad78:::rbd_data.115e1e95bf99.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13369344~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:06:11.751 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.922336+0000", 2024-09-14T09:06:11.751 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.815680567000001, 2024-09-14T09:06:11.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010311885999999999, 2024-09-14T09:06:11.751 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.751 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.751 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.751 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:11.752 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:11.752 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.752 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 845 2024-09-14T09:06:11.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.752 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922336+0000", 2024-09-14T09:06:11.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922336+0000", 2024-09-14T09:06:11.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922339+0000", 2024-09-14T09:06:11.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6879999999999999e-06 2024-09-14T09:06:11.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922350+0000", 2024-09-14T09:06:11.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1457999999999999e-05 2024-09-14T09:06:11.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922376+0000", 2024-09-14T09:06:11.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5301999999999999e-05 2024-09-14T09:06:11.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922390+0000", 2024-09-14T09:06:11.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4404e-05 2024-09-14T09:06:11.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922506+0000", 2024-09-14T09:06:11.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011567599999999999 2024-09-14T09:06:11.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922741+0000", 2024-09-14T09:06:11.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00023530600000000001 2024-09-14T09:06:11.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-09-14T09:06:11.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923197+0000", 2024-09-14T09:06:11.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00045574899999999998 2024-09-14T09:06:11.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:11.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926027+0000", 2024-09-14T09:06:11.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002830819 2024-09-14T09:06:11.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:11.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932530+0000", 2024-09-14T09:06:11.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065027139999999997 2024-09-14T09:06:11.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:11.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932583+0000", 2024-09-14T09:06:11.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2342000000000001e-05 2024-09-14T09:06:11.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932648+0000", 2024-09-14T09:06:11.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5427999999999998e-05 2024-09-14T09:06:11.758 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.758 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.758 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.758 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:842 3.b e29/28 3:d50f9298:::rbd_data.115e1e95bf99.0000000000000004:head v 29'5, mlcod=26'3)", 2024-09-14T09:06:11.758 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.922344+0000", 2024-09-14T09:06:11.759 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.815672816999999, 2024-09-14T09:06:11.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062661460000000002, 2024-09-14T09:06:11.759 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.759 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.759 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.759 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922344+0000", 2024-09-14T09:06:11.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922344+0000", 2024-09-14T09:06:11.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922345+0000", 2024-09-14T09:06:11.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.288e-06 2024-09-14T09:06:11.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922356+0000", 2024-09-14T09:06:11.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1048e-05 2024-09-14T09:06:11.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922371+0000", 2024-09-14T09:06:11.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4338000000000001e-05 2024-09-14T09:06:11.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922382+0000", 2024-09-14T09:06:11.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1e-05 2024-09-14T09:06:11.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922467+0000", 2024-09-14T09:06:11.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5721000000000001e-05 2024-09-14T09:06:11.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922489+0000", 2024-09-14T09:06:11.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2181999999999999e-05 2024-09-14T09:06:11.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:11.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.928551+0000", 2024-09-14T09:06:11.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060616100000000003 2024-09-14T09:06:11.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.928610+0000", 2024-09-14T09:06:11.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8959000000000003e-05 2024-09-14T09:06:11.763 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.763 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.763 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.764 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:848 3.1a 3:5ade8e73:::rbd_data.115e1e95bf99.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13369344~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:06:11.764 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.922902+0000", 2024-09-14T09:06:11.764 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.815115155000001, 2024-09-14T09:06:11.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096031020000000005, 2024-09-14T09:06:11.764 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.764 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.764 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.764 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:11.764 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:11.764 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.764 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 848 2024-09-14T09:06:11.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.765 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922902+0000", 2024-09-14T09:06:11.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922902+0000", 2024-09-14T09:06:11.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922903+0000", 2024-09-14T09:06:11.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6419999999999999e-06 2024-09-14T09:06:11.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923127+0000", 2024-09-14T09:06:11.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000223997 2024-09-14T09:06:11.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923151+0000", 2024-09-14T09:06:11.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3802e-05 2024-09-14T09:06:11.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923169+0000", 2024-09-14T09:06:11.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7858000000000001e-05 2024-09-14T09:06:11.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923307+0000", 2024-09-14T09:06:11.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013802399999999999 2024-09-14T09:06:11.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923560+0000", 2024-09-14T09:06:11.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00025296500000000001 2024-09-14T09:06:11.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:11.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923925+0000", 2024-09-14T09:06:11.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036534700000000002 2024-09-14T09:06:11.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:11.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.927231+0000", 2024-09-14T09:06:11.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033055820000000001 2024-09-14T09:06:11.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:11.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932287+0000", 2024-09-14T09:06:11.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0050562810000000001 2024-09-14T09:06:11.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:11.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932395+0000", 2024-09-14T09:06:11.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000108001 2024-09-14T09:06:11.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932505+0000", 2024-09-14T09:06:11.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109603 2024-09-14T09:06:11.769 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.769 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.770 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.770 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:847 3.13 e29/23 3:cc047b4a:::rbd_data.115e1e95bf99.0000000000000009:head v 29'2, mlcod=0'0)", 2024-09-14T09:06:11.770 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.924087+0000", 2024-09-14T09:06:11.770 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.813929393000002, 2024-09-14T09:06:11.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0085507929999999992, 2024-09-14T09:06:11.770 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.770 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.770 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.770 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924087+0000", 2024-09-14T09:06:11.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924087+0000", 2024-09-14T09:06:11.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924089+0000", 2024-09-14T09:06:11.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8050000000000001e-06 2024-09-14T09:06:11.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924157+0000", 2024-09-14T09:06:11.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7999999999999999e-05 2024-09-14T09:06:11.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924176+0000", 2024-09-14T09:06:11.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9312000000000001e-05 2024-09-14T09:06:11.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924191+0000", 2024-09-14T09:06:11.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4707e-05 2024-09-14T09:06:11.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924292+0000", 2024-09-14T09:06:11.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010052000000000001 2024-09-14T09:06:11.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924328+0000", 2024-09-14T09:06:11.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6211999999999998e-05 2024-09-14T09:06:11.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:11.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932564+0000", 2024-09-14T09:06:11.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0082365960000000005 2024-09-14T09:06:11.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932638+0000", 2024-09-14T09:06:11.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3640999999999995e-05 2024-09-14T09:06:11.774 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.774 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.774 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.774 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:849 3.b e29/28 3:d70d4fef:::rbd_data.115e1e95bf99.000000000000000b:head v 29'6, mlcod=26'3)", 2024-09-14T09:06:11.775 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.924304+0000", 2024-09-14T09:06:11.775 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.813712434000003, 2024-09-14T09:06:11.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083878719999999993, 2024-09-14T09:06:11.775 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.775 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.775 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.775 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924304+0000", 2024-09-14T09:06:11.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924304+0000", 2024-09-14T09:06:11.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924305+0000", 2024-09-14T09:06:11.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.192e-06 2024-09-14T09:06:11.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924676+0000", 2024-09-14T09:06:11.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037050700000000001 2024-09-14T09:06:11.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924698+0000", 2024-09-14T09:06:11.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1977999999999999e-05 2024-09-14T09:06:11.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924712+0000", 2024-09-14T09:06:11.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4477e-05 2024-09-14T09:06:11.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926069+0000", 2024-09-14T09:06:11.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013562330000000001 2024-09-14T09:06:11.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926105+0000", 2024-09-14T09:06:11.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6779999999999997e-05 2024-09-14T09:06:11.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:11.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932643+0000", 2024-09-14T09:06:11.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065378290000000002 2024-09-14T09:06:11.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932692+0000", 2024-09-14T09:06:11.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8875999999999998e-05 2024-09-14T09:06:11.778 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.778 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.778 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.779 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:850 3.11 e29/23 3:892642aa:::rbd_data.115e1e95bf99.000000000000000c:head v 29'18, mlcod=29'15)", 2024-09-14T09:06:11.779 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.926814+0000", 2024-09-14T09:06:11.779 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.811202561000002, 2024-09-14T09:06:11.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0064716569999999996, 2024-09-14T09:06:11.779 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.779 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.779 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.779 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926814+0000", 2024-09-14T09:06:11.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926814+0000", 2024-09-14T09:06:11.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926816+0000", 2024-09-14T09:06:11.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.491e-06 2024-09-14T09:06:11.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926828+0000", 2024-09-14T09:06:11.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2288e-05 2024-09-14T09:06:11.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926851+0000", 2024-09-14T09:06:11.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2674999999999999e-05 2024-09-14T09:06:11.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926865+0000", 2024-09-14T09:06:11.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.449e-05 2024-09-14T09:06:11.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932217+0000", 2024-09-14T09:06:11.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053523370000000004 2024-09-14T09:06:11.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932241+0000", 2024-09-14T09:06:11.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3210999999999999e-05 2024-09-14T09:06:11.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:11.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.933239+0000", 2024-09-14T09:06:11.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00099850599999999992 2024-09-14T09:06:11.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.933286+0000", 2024-09-14T09:06:11.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6659000000000002e-05 2024-09-14T09:06:11.783 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.783 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.783 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.783 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:9982 3.1a 3:5af3e476:::rbd_data.115eadec874.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:06:11.783 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:11.783 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.140047240999998, 2024-09-14T09:06:11.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042276341000000002, 2024-09-14T09:06:11.783 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.783 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.783 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.784 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:11.784 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:11.784 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.784 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 9982 2024-09-14T09:06:11.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.784 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:11.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:11.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597970+0000", 2024-09-14T09:06:11.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0599999999999999e-07 2024-09-14T09:06:11.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597974+0000", 2024-09-14T09:06:11.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2649999999999998e-06 2024-09-14T09:06:11.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597986+0000", 2024-09-14T09:06:11.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1019000000000001e-05 2024-09-14T09:06:11.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.786 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.786 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597991+0000", 2024-09-14T09:06:11.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5049999999999998e-06 2024-09-14T09:06:11.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.786 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.786 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.598083+0000", 2024-09-14T09:06:11.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2062e-05 2024-09-14T09:06:11.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.786 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.786 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.598227+0000", 2024-09-14T09:06:11.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000143609 2024-09-14T09:06:11.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:11.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.603107+0000", 2024-09-14T09:06:11.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048805819999999996 2024-09-14T09:06:11.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:11.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640002+0000", 2024-09-14T09:06:11.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.036894641999999998 2024-09-14T09:06:11.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:11.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640077+0000", 2024-09-14T09:06:11.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4927999999999998e-05 2024-09-14T09:06:11.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:11.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640174+0000", 2024-09-14T09:06:11.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7417999999999995e-05 2024-09-14T09:06:11.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640246+0000", 2024-09-14T09:06:11.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1505000000000001e-05 2024-09-14T09:06:11.788 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.788 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.789 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.789 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:9982 3.1a e32/23 ondisk, result = 0)", 2024-09-14T09:06:11.789 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.609682+0000", 2024-09-14T09:06:11.789 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.128334631999998, 2024-09-14T09:06:11.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.030575755, 2024-09-14T09:06:11.789 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.789 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.789 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:11.789 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609682+0000", 2024-09-14T09:06:11.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609682+0000", 2024-09-14T09:06:11.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609683+0000", 2024-09-14T09:06:11.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0399999999999995e-07 2024-09-14T09:06:11.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609687+0000", 2024-09-14T09:06:11.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3460000000000004e-06 2024-09-14T09:06:11.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609696+0000", 2024-09-14T09:06:11.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0550000000000005e-06 2024-09-14T09:06:11.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609702+0000", 2024-09-14T09:06:11.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.626e-06 2024-09-14T09:06:11.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640050+0000", 2024-09-14T09:06:11.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.030348671000000001 2024-09-14T09:06:11.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640066+0000", 2024-09-14T09:06:11.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5617999999999999e-05 2024-09-14T09:06:11.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640258+0000", 2024-09-14T09:06:11.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019173500000000001 2024-09-14T09:06:11.793 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.793 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.793 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.794 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:10031 3.1b 3:df00a663:::rbd_data.115eadec874.0000000000000005:head [stat,write 16619424~53229] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:06:11.794 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.768778+0000", 2024-09-14T09:06:11.794 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.969239121000001, 2024-09-14T09:06:11.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0078922009999999997, 2024-09-14T09:06:11.794 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.794 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.794 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.794 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:11.794 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:11.794 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.794 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 10031 2024-09-14T09:06:11.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.795 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768778+0000", 2024-09-14T09:06:11.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768778+0000", 2024-09-14T09:06:11.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768780+0000", 2024-09-14T09:06:11.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7130000000000001e-06 2024-09-14T09:06:11.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768824+0000", 2024-09-14T09:06:11.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3989999999999997e-05 2024-09-14T09:06:11.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768854+0000", 2024-09-14T09:06:11.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9937000000000002e-05 2024-09-14T09:06:11.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768869+0000", 2024-09-14T09:06:11.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5233e-05 2024-09-14T09:06:11.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768979+0000", 2024-09-14T09:06:11.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109429 2024-09-14T09:06:11.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.769124+0000", 2024-09-14T09:06:11.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014520200000000001 2024-09-14T09:06:11.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:11.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.769351+0000", 2024-09-14T09:06:11.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000227259 2024-09-14T09:06:11.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:11.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.776133+0000", 2024-09-14T09:06:11.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067818979999999997 2024-09-14T09:06:11.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:11.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.776464+0000", 2024-09-14T09:06:11.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033063400000000001 2024-09-14T09:06:11.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:11.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.776553+0000", 2024-09-14T09:06:11.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8793000000000002e-05 2024-09-14T09:06:11.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.776670+0000", 2024-09-14T09:06:11.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000117113 2024-09-14T09:06:11.799 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.799 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.799 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.800 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:10039 3.f e33/23 3:f6f4e415:::rbd_data.115eadec874.0000000000000000:head v 33'8, mlcod=32'6)", 2024-09-14T09:06:11.800 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.951357+0000", 2024-09-14T09:06:11.800 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.786659979, 2024-09-14T09:06:11.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067664509999999997, 2024-09-14T09:06:11.800 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.800 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.800 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.800 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951357+0000", 2024-09-14T09:06:11.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951357+0000", 2024-09-14T09:06:11.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951357+0000", 2024-09-14T09:06:11.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4000000000000001e-07 2024-09-14T09:06:11.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951363+0000", 2024-09-14T09:06:11.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4999999999999999e-06 2024-09-14T09:06:11.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951376+0000", 2024-09-14T09:06:11.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3315e-05 2024-09-14T09:06:11.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951384+0000", 2024-09-14T09:06:11.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6590000000000007e-06 2024-09-14T09:06:11.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951470+0000", 2024-09-14T09:06:11.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6198000000000004e-05 2024-09-14T09:06:11.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951493+0000", 2024-09-14T09:06:11.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2668999999999999e-05 2024-09-14T09:06:11.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:11.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.958059+0000", 2024-09-14T09:06:11.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065659450000000001 2024-09-14T09:06:11.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.958123+0000", 2024-09-14T09:06:11.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4524999999999996e-05 2024-09-14T09:06:11.804 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.804 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.804 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.804 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:10040 3.a e33/23 3:50279329:::rbd_data.115eadec874.0000000000000001:head v 33'16, mlcod=32'14)", 2024-09-14T09:06:11.805 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.951570+0000", 2024-09-14T09:06:11.805 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.786446664000003, 2024-09-14T09:06:11.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0094331320000000003, 2024-09-14T09:06:11.805 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.805 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.805 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.805 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951570+0000", 2024-09-14T09:06:11.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951570+0000", 2024-09-14T09:06:11.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951571+0000", 2024-09-14T09:06:11.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0800000000000004e-07 2024-09-14T09:06:11.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951578+0000", 2024-09-14T09:06:11.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4919999999999996e-06 2024-09-14T09:06:11.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951590+0000", 2024-09-14T09:06:11.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1674e-05 2024-09-14T09:06:11.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951598+0000", 2024-09-14T09:06:11.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4649999999999994e-06 2024-09-14T09:06:11.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951800+0000", 2024-09-14T09:06:11.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020128600000000001 2024-09-14T09:06:11.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951832+0000", 2024-09-14T09:06:11.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2512999999999999e-05 2024-09-14T09:06:11.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:11.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.960831+0000", 2024-09-14T09:06:11.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00899872 2024-09-14T09:06:11.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.961003+0000", 2024-09-14T09:06:11.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017217400000000001 2024-09-14T09:06:11.809 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.809 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.809 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.809 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:10121 3.a e33/23 3:50279329:::rbd_data.115eadec874.0000000000000001:head v 33'18, mlcod=33'16)", 2024-09-14T09:06:11.809 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:38.059543+0000", 2024-09-14T09:06:11.809 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.67847355, 2024-09-14T09:06:11.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063099250000000001, 2024-09-14T09:06:11.810 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.810 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.810 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.810 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059543+0000", 2024-09-14T09:06:11.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059543+0000", 2024-09-14T09:06:11.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059545+0000", 2024-09-14T09:06:11.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.863e-06 2024-09-14T09:06:11.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059609+0000", 2024-09-14T09:06:11.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3758999999999997e-05 2024-09-14T09:06:11.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059633+0000", 2024-09-14T09:06:11.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4000000000000001e-05 2024-09-14T09:06:11.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059650+0000", 2024-09-14T09:06:11.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7130999999999999e-05 2024-09-14T09:06:11.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059779+0000", 2024-09-14T09:06:11.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000129152 2024-09-14T09:06:11.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059840+0000", 2024-09-14T09:06:11.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0930000000000001e-05 2024-09-14T09:06:11.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:11.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.065745+0000", 2024-09-14T09:06:11.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059050309999999998 2024-09-14T09:06:11.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.065853+0000", 2024-09-14T09:06:11.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000108059 2024-09-14T09:06:11.813 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.813 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.813 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.814 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:10182 3.1a 3:5a62ac37:::rbd_data.115eadec874.000000000000000d:head [set-alloc-hint object_size 33554432 write_size 33554432,write 5072829~48248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-14T09:06:11.814 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:38.177016+0000", 2024-09-14T09:06:11.814 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.561000614999998, 2024-09-14T09:06:11.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006987147, 2024-09-14T09:06:11.814 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.814 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.814 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.814 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:11.814 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:11.814 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:11.814 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 10182 2024-09-14T09:06:11.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.815 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177016+0000", 2024-09-14T09:06:11.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177016+0000", 2024-09-14T09:06:11.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177017+0000", 2024-09-14T09:06:11.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0440000000000001e-06 2024-09-14T09:06:11.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177065+0000", 2024-09-14T09:06:11.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7902999999999999e-05 2024-09-14T09:06:11.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177080+0000", 2024-09-14T09:06:11.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4667999999999999e-05 2024-09-14T09:06:11.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177087+0000", 2024-09-14T09:06:11.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0430000000000002e-06 2024-09-14T09:06:11.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177140+0000", 2024-09-14T09:06:11.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2933999999999998e-05 2024-09-14T09:06:11.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177199+0000", 2024-09-14T09:06:11.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9799999999999997e-05 2024-09-14T09:06:11.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:11.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177309+0000", 2024-09-14T09:06:11.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010948200000000001 2024-09-14T09:06:11.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:11.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.180906+0000", 2024-09-14T09:06:11.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0035966230000000002 2024-09-14T09:06:11.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:11.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.183932+0000", 2024-09-14T09:06:11.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030259760000000001 2024-09-14T09:06:11.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:11.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.183973+0000", 2024-09-14T09:06:11.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1189000000000002e-05 2024-09-14T09:06:11.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.184003+0000", 2024-09-14T09:06:11.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0485e-05 2024-09-14T09:06:11.819 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.819 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.819 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.819 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:20954 3.1d e41/23 3:bd7e9002:::rbd_data.115e889d4809.0000000000000005:head v 41'131, mlcod=41'127)", 2024-09-14T09:06:11.819 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:47.701009+0000", 2024-09-14T09:06:11.820 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.037007255999999, 2024-09-14T09:06:11.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0080632259999999997, 2024-09-14T09:06:11.820 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.820 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.820 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.820 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701009+0000", 2024-09-14T09:06:11.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701009+0000", 2024-09-14T09:06:11.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701011+0000", 2024-09-14T09:06:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.703e-06 2024-09-14T09:06:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701234+0000", 2024-09-14T09:06:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022278900000000001 2024-09-14T09:06:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701265+0000", 2024-09-14T09:06:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1386000000000001e-05 2024-09-14T09:06:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701281+0000", 2024-09-14T09:06:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5460000000000001e-05 2024-09-14T09:06:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701391+0000", 2024-09-14T09:06:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000110622 2024-09-14T09:06:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701428+0000", 2024-09-14T09:06:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6986999999999997e-05 2024-09-14T09:06:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.708992+0000", 2024-09-14T09:06:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075640769999999998 2024-09-14T09:06:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.709073+0000", 2024-09-14T09:06:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0202000000000002e-05 2024-09-14T09:06:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:21183 3.1d e42/23 3:bba2b35c:::rbd_object_map.115e889d4809:head v 42'156, mlcod=42'149)", 2024-09-14T09:06:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:48.419596+0000", 2024-09-14T09:06:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.318420980999999, 2024-09-14T09:06:11.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063479840000000001, 2024-09-14T09:06:11.824 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.824 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.824 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:11.824 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419596+0000", 2024-09-14T09:06:11.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419596+0000", 2024-09-14T09:06:11.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419598+0000", 2024-09-14T09:06:11.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8560000000000001e-06 2024-09-14T09:06:11.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419610+0000", 2024-09-14T09:06:11.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2577e-05 2024-09-14T09:06:11.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419645+0000", 2024-09-14T09:06:11.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4873000000000003e-05 2024-09-14T09:06:11.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419664+0000", 2024-09-14T09:06:11.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9485999999999999e-05 2024-09-14T09:06:11.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419799+0000", 2024-09-14T09:06:11.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013482300000000001 2024-09-14T09:06:11.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419853+0000", 2024-09-14T09:06:11.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3439000000000001e-05 2024-09-14T09:06:11.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:11.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.425843+0000", 2024-09-14T09:06:11.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059898809999999999 2024-09-14T09:06:11.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.425944+0000", 2024-09-14T09:06:11.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101049 2024-09-14T09:06:11.827 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.827 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.827 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.827 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(3.1b 53/52 [PushOp(3:df00a663:::rbd_data.115eadec874.0000000000000005:6, version: 35'41, data_included: [1796079~63600], data_size: 63600, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:df00a663:::rbd_data.115eadec874.0000000000000005:6@35'41, size: 1859679, copy_subset: [(0, 1859679)], clone_subset: {}, snapset: 6=[]:{6: [6]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1859679, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T09:06:11.827 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.846807+0000", 2024-09-14T09:06:11.828 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.8912100029999994, 2024-09-14T09:06:11.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0089637359999999999, 2024-09-14T09:06:11.828 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.828 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.828 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:11.828 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.846807+0000", 2024-09-14T09:06:11.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.846807+0000", 2024-09-14T09:06:11.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.846809+0000", 2024-09-14T09:06:11.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5799999999999999e-06 2024-09-14T09:06:11.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.846948+0000", 2024-09-14T09:06:11.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013911500000000001 2024-09-14T09:06:11.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.847007+0000", 2024-09-14T09:06:11.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8437000000000003e-05 2024-09-14T09:06:11.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.847054+0000", 2024-09-14T09:06:11.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7372e-05 2024-09-14T09:06:11.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.847295+0000", 2024-09-14T09:06:11.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000241022 2024-09-14T09:06:11.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.847319+0000", 2024-09-14T09:06:11.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4303000000000001e-05 2024-09-14T09:06:11.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.855770+0000", 2024-09-14T09:06:11.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084509070000000006 2024-09-14T09:06:11.831 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.831 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.831 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.831 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(3.1b 53/52 [PushOp(3:d8e0f8d4:::rbd_data.115eb07a8abd.0000000000000001:head, version: 48'138, data_included: [0~393216,786432~131072], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:d8e0f8d4:::rbd_data.115eb07a8abd.0000000000000001:head@48'138, size: 7208960, copy_subset: [(0, 7208960)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 7208960, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T09:06:11.831 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.943323+0000", 2024-09-14T09:06:11.832 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.7946939880000006, 2024-09-14T09:06:11.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0081121930000000002, 2024-09-14T09:06:11.832 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:11.832 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:11.832 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:11.832 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:11.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:11.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.943323+0000", 2024-09-14T09:06:11.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:11.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.943323+0000", 2024-09-14T09:06:11.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:11.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:11.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.943324+0000", 2024-09-14T09:06:11.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.206e-06 2024-09-14T09:06:11.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:11.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944255+0000", 2024-09-14T09:06:11.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093096900000000002 2024-09-14T09:06:11.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:11.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944299+0000", 2024-09-14T09:06:11.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4332999999999997e-05 2024-09-14T09:06:11.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:11.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944332+0000", 2024-09-14T09:06:11.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2329999999999997e-05 2024-09-14T09:06:11.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:11.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944590+0000", 2024-09-14T09:06:11.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00025868100000000003 2024-09-14T09:06:11.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:11.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944629+0000", 2024-09-14T09:06:11.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8503e-05 2024-09-14T09:06:11.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:11.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:11.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:11.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.951435+0000", 2024-09-14T09:06:11.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068061709999999997 2024-09-14T09:06:11.835 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.835 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.835 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.835 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:11.835 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:11.835 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:11.837 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:06:11.837 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:11.839+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:11.858 INFO:teuthology.orchestra.run.smithi176.stdout:1893 trunc from 0x949aa41 to 0xe137b7f 2024-09-14T09:06:11.874 INFO:teuthology.orchestra.run.smithi176.stdout:1895 write 0xbece181 thru 0xbed216c (0x3fec bytes) 2024-09-14T09:06:11.885 INFO:teuthology.orchestra.run.smithi176.stdout:1896 punch from 0x47bbdc to 0x48b2c8, (0xf6ec bytes) 2024-09-14T09:06:11.886 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:11.886 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:11.886 INFO:teuthology.orchestra.run.smithi176.stdout:1899 write 0x61c968c thru 0x61d46de (0xb053 bytes) 2024-09-14T09:06:11.900 INFO:teuthology.orchestra.run.smithi176.stdout:1900 read 0x778a0e8 thru 0x7791e84 (0x7d9d bytes) 2024-09-14T09:06:11.901 INFO:teuthology.orchestra.run.smithi176.stdout:1901 read 0xdcbd376 thru 0xdcc59ec (0x8677 bytes) 2024-09-14T09:06:11.923 INFO:teuthology.orchestra.run.smithi176.stdout:1904 trunc from 0xe137b7f to 0x2cfceb 2024-09-14T09:06:11.938 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:11.939+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:12.012 INFO:teuthology.orchestra.run.smithi176.stdout:1905 write 0x8048791 thru 0x804fc99 (0x7509 bytes) 2024-09-14T09:06:12.018 INFO:teuthology.orchestra.run.smithi176.stdout:1906 read 0x7e4fcd2 thru 0x7e572b0 (0x75df bytes) 2024-09-14T09:06:12.018 INFO:teuthology.orchestra.run.smithi176.stdout:1907 read 0x73795e6 thru 0x737b665 (0x2080 bytes) 2024-09-14T09:06:12.019 INFO:teuthology.orchestra.run.smithi176.stdout:1909 trunc from 0x804fc9a to 0x8976a67 2024-09-14T09:06:12.022 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:12.022 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:12.023 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:12.023 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:12.028 INFO:teuthology.orchestra.run.smithi176.stdout:1910 read 0x868d69e thru 0x86993a8 (0xbd0b bytes) 2024-09-14T09:06:12.028 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:12.028 INFO:teuthology.orchestra.run.smithi176.stdout:1913 read 0x63c9a73 thru 0x63d081a (0x6da8 bytes) 2024-09-14T09:06:12.032 INFO:teuthology.orchestra.run.smithi176.stdout:1914 trunc from 0x8976a67 to 0xad1bd5a 2024-09-14T09:06:12.037 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:06:12.044 INFO:teuthology.orchestra.run.smithi176.stdout:1915 punch from 0x3aa6339 to 0x3aac6f0, (0x63b7 bytes) 2024-09-14T09:06:12.045 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:12.045 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:12.045 INFO:teuthology.orchestra.run.smithi176.stdout:1918 writesame 0x8a9934 thru 0x8b1fc4 data_size 0x1(0x8691 bytes) 2024-09-14T09:06:12.080 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:12.083+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:12.120 INFO:teuthology.orchestra.run.smithi176.stdout:1920 read 0x9a5d6d9 thru 0x9a61afb (0x4423 bytes) 2024-09-14T09:06:12.121 INFO:teuthology.orchestra.run.smithi176.stdout:1921 write 0x749c434 thru 0x74a69c0 (0xa58d bytes) 2024-09-14T09:06:12.129 INFO:teuthology.orchestra.run.smithi176.stdout:1923 punch from 0x46c273e to 0x46cd38a, (0xac4c bytes) 2024-09-14T09:06:12.130 INFO:teuthology.orchestra.run.smithi176.stdout:1924 read 0x13a3aa4 thru 0x13af57a (0xbad7 bytes) 2024-09-14T09:06:12.130 INFO:teuthology.orchestra.run.smithi176.stdout:1926 read 0x9e941c7 thru 0x9e9552b (0x1365 bytes) 2024-09-14T09:06:12.139 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:12.135+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:12.145 INFO:teuthology.orchestra.run.smithi176.stdout:1927 writesame 0xc2efbce thru 0xc2f4658 data_size 0x1(0x4a8b bytes) 2024-09-14T09:06:12.146 INFO:teuthology.orchestra.run.smithi176.stdout:1928 punch from 0x669e8de to 0x66a5ea3, (0x75c5 bytes) 2024-09-14T09:06:12.146 INFO:teuthology.orchestra.run.smithi176.stdout:1929 write 0xbd57bd7 thru 0xbd60818 (0x8c42 bytes) 2024-09-14T09:06:12.155 INFO:teuthology.orchestra.run.smithi176.stdout:1930 writesame 0x3955bac thru 0x395a695 data_size 0x2b(0x4aea bytes) 2024-09-14T09:06:12.160 INFO:teuthology.orchestra.run.smithi176.stdout:1933 read 0x736a856 thru 0x7378245 (0xd9f0 bytes) 2024-09-14T09:06:12.161 INFO:teuthology.orchestra.run.smithi176.stdout:1934 read 0x966f3d2 thru 0x9670eb9 (0x1ae8 bytes) 2024-09-14T09:06:12.161 INFO:teuthology.orchestra.run.smithi176.stdout:1935 writesame 0x93a76cd thru 0x93b021c data_size 0x18(0x8b50 bytes) 2024-09-14T09:06:12.168 INFO:teuthology.orchestra.run.smithi176.stdout:1936 writesame 0x6163d71 thru 0x616e340 data_size 0x38(0xa5d0 bytes) 2024-09-14T09:06:12.174 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:12.174 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:12.174 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:12.174 INFO:teuthology.orchestra.run.smithi176.stdout:1942 write 0x5708492 thru 0x5717300 (0xee6f bytes) 2024-09-14T09:06:12.185 INFO:teuthology.orchestra.run.smithi176.stdout:1943 punch from 0x5ee0356 to 0x5ee97bf, (0x9469 bytes) 2024-09-14T09:06:12.185 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:12.185 INFO:teuthology.orchestra.run.smithi176.stdout:1946 write 0x1d82f31 thru 0x1d86757 (0x3827 bytes) 2024-09-14T09:06:12.191 INFO:teuthology.orchestra.run.smithi176.stdout:1947 writesame 0xbd17369 thru 0xbd26f5e data_size 0x1(0xfbf6 bytes) 2024-09-14T09:06:12.217 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:12.217 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:12.217 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:12.217 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:12.217 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:12.233 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:06:12.280 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:12.283+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:12.306 INFO:teuthology.orchestra.run.smithi176.stdout:1948 write 0x509fcce thru 0x50a20c4 (0x23f7 bytes) 2024-09-14T09:06:12.314 INFO:teuthology.orchestra.run.smithi176.stdout:1950 writesame 0x2191ac9 thru 0x2194451 data_size 0x1f(0x2989 bytes) 2024-09-14T09:06:12.319 INFO:teuthology.orchestra.run.smithi176.stdout:1951 writesame 0xaba713b thru 0xaba8450 data_size 0x7(0x1316 bytes) 2024-09-14T09:06:12.324 INFO:teuthology.orchestra.run.smithi176.stdout:1952 read 0x7655b1a thru 0x765a9ff (0x4ee6 bytes) 2024-09-14T09:06:12.325 INFO:teuthology.orchestra.run.smithi176.stdout:1954 read 0xe6218e thru 0xe6bf94 (0x9e07 bytes) 2024-09-14T09:06:12.325 INFO:teuthology.orchestra.run.smithi176.stdout:1955 clone 18 order 19 su 16384 sc 10 2024-09-14T09:06:12.341 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:12.343+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:12.359 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:12.359 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:12.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:12.359 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:12.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.360 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49333 3.1d 3:be596f9e:::rbd_data.115e51d92ed2.0000000000000007:12 [read 950272~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:12.360 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:12.360 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.815925215, 2024-09-14T09:06:12.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072172603000000002, 2024-09-14T09:06:12.360 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.360 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.360 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:12.360 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.360 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.361 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.361 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49333 2024-09-14T09:06:12.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.361 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:12.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:12.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:12.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6000000000000003e-07 2024-09-14T09:06:12.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546216+0000", 2024-09-14T09:06:12.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5339999999999998e-06 2024-09-14T09:06:12.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546224+0000", 2024-09-14T09:06:12.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7770000000000002e-06 2024-09-14T09:06:12.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546230+0000", 2024-09-14T09:06:12.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3400000000000003e-06 2024-09-14T09:06:12.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.617881+0000", 2024-09-14T09:06:12.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071650488999999998 2024-09-14T09:06:12.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:12.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.617958+0000", 2024-09-14T09:06:12.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7018e-05 2024-09-14T09:06:12.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618385+0000", 2024-09-14T09:06:12.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00042768500000000002 2024-09-14T09:06:12.365 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.365 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.365 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.365 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49334 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:12.365 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:12.365 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.815898333, 2024-09-14T09:06:12.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072227886000000005, 2024-09-14T09:06:12.365 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.366 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.366 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:12.366 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.366 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.366 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.366 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49334 2024-09-14T09:06:12.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.366 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:12.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:12.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:12.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8899999999999999e-07 2024-09-14T09:06:12.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546243+0000", 2024-09-14T09:06:12.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.503e-06 2024-09-14T09:06:12.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546253+0000", 2024-09-14T09:06:12.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0210999999999999e-05 2024-09-14T09:06:12.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546267+0000", 2024-09-14T09:06:12.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4183e-05 2024-09-14T09:06:12.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618449+0000", 2024-09-14T09:06:12.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072181828000000003 2024-09-14T09:06:12.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618467+0000", 2024-09-14T09:06:12.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8471999999999999e-05 2024-09-14T09:06:12.369 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.369 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.369 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.369 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49335 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:12.369 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:12.369 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.815862182, 2024-09-14T09:06:12.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072255640999999995, 2024-09-14T09:06:12.369 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.369 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.369 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:12.370 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.370 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.370 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.370 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49335 2024-09-14T09:06:12.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.370 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:12.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:12.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:12.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9999999999999997e-07 2024-09-14T09:06:12.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546279+0000", 2024-09-14T09:06:12.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.695e-06 2024-09-14T09:06:12.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546288+0000", 2024-09-14T09:06:12.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7110000000000001e-06 2024-09-14T09:06:12.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546294+0000", 2024-09-14T09:06:12.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9830000000000001e-06 2024-09-14T09:06:12.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618521+0000", 2024-09-14T09:06:12.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072226948999999999 2024-09-14T09:06:12.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618531+0000", 2024-09-14T09:06:12.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0603e-05 2024-09-14T09:06:12.373 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.373 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.373 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.373 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49336 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:12.373 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:12.373 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.815836279, 2024-09-14T09:06:12.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072292223000000003, 2024-09-14T09:06:12.373 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.373 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.374 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:12.374 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.374 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.374 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.374 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49336 2024-09-14T09:06:12.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.374 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:12.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:12.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:12.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4000000000000001e-07 2024-09-14T09:06:12.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546305+0000", 2024-09-14T09:06:12.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3539999999999998e-06 2024-09-14T09:06:12.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546315+0000", 2024-09-14T09:06:12.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0431999999999999e-05 2024-09-14T09:06:12.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546321+0000", 2024-09-14T09:06:12.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7620000000000001e-06 2024-09-14T09:06:12.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618583+0000", 2024-09-14T09:06:12.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072262046999999996 2024-09-14T09:06:12.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618594+0000", 2024-09-14T09:06:12.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0988000000000001e-05 2024-09-14T09:06:12.380 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.380 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.380 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.381 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49337 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:12.381 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:12.381 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.8157886649999999, 2024-09-14T09:06:12.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072294774000000006, 2024-09-14T09:06:12.382 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.382 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.382 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:12.382 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.382 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.382 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.383 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49337 2024-09-14T09:06:12.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.383 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:12.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:12.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546350+0000", 2024-09-14T09:06:12.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4600000000000004e-07 2024-09-14T09:06:12.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546381+0000", 2024-09-14T09:06:12.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0902000000000003e-05 2024-09-14T09:06:12.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546412+0000", 2024-09-14T09:06:12.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0825999999999999e-05 2024-09-14T09:06:12.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546438+0000", 2024-09-14T09:06:12.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6410999999999998e-05 2024-09-14T09:06:12.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618637+0000", 2024-09-14T09:06:12.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072198907000000007 2024-09-14T09:06:12.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618644+0000", 2024-09-14T09:06:12.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9820000000000002e-06 2024-09-14T09:06:12.390 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.390 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.390 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.391 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49338 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:12.391 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:12.391 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.815674638, 2024-09-14T09:06:12.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072233250999999998, 2024-09-14T09:06:12.391 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.391 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.392 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:12.392 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.392 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.392 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.392 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49338 2024-09-14T09:06:12.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.393 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:12.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:12.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546465+0000", 2024-09-14T09:06:12.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.455e-06 2024-09-14T09:06:12.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546481+0000", 2024-09-14T09:06:12.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5834000000000001e-05 2024-09-14T09:06:12.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546498+0000", 2024-09-14T09:06:12.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6937000000000001e-05 2024-09-14T09:06:12.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546508+0000", 2024-09-14T09:06:12.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0446e-05 2024-09-14T09:06:12.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618685+0000", 2024-09-14T09:06:12.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072177280999999996 2024-09-14T09:06:12.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618697+0000", 2024-09-14T09:06:12.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1297999999999999e-05 2024-09-14T09:06:12.400 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.400 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.400 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.400 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49339 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:12.400 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546535+0000", 2024-09-14T09:06:12.401 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.815602621, 2024-09-14T09:06:12.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072197266999999996, 2024-09-14T09:06:12.401 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.401 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.401 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:12.401 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.402 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.402 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.402 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49339 2024-09-14T09:06:12.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.402 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546535+0000", 2024-09-14T09:06:12.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546535+0000", 2024-09-14T09:06:12.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546536+0000", 2024-09-14T09:06:12.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1399999999999996e-07 2024-09-14T09:06:12.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546539+0000", 2024-09-14T09:06:12.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.089e-06 2024-09-14T09:06:12.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546560+0000", 2024-09-14T09:06:12.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1217e-05 2024-09-14T09:06:12.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546567+0000", 2024-09-14T09:06:12.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9e-06 2024-09-14T09:06:12.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618726+0000", 2024-09-14T09:06:12.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.07215887 2024-09-14T09:06:12.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618733+0000", 2024-09-14T09:06:12.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3770000000000001e-06 2024-09-14T09:06:12.405 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.405 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.405 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.405 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49340 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:12.406 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546578+0000", 2024-09-14T09:06:12.406 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.8155597299999999, 2024-09-14T09:06:12.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072205187000000004, 2024-09-14T09:06:12.406 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.406 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.406 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:12.406 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.406 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.406 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.406 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49340 2024-09-14T09:06:12.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.406 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546578+0000", 2024-09-14T09:06:12.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546578+0000", 2024-09-14T09:06:12.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546579+0000", 2024-09-14T09:06:12.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1999999999999999e-07 2024-09-14T09:06:12.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546582+0000", 2024-09-14T09:06:12.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4460000000000001e-06 2024-09-14T09:06:12.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546603+0000", 2024-09-14T09:06:12.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0502000000000001e-05 2024-09-14T09:06:12.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546610+0000", 2024-09-14T09:06:12.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7150000000000001e-06 2024-09-14T09:06:12.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618770+0000", 2024-09-14T09:06:12.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072160600000000005 2024-09-14T09:06:12.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618783+0000", 2024-09-14T09:06:12.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:06:12.410 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.410 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.410 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.410 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49341 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:12.410 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546667+0000", 2024-09-14T09:06:12.410 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.8154706219999999, 2024-09-14T09:06:12.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072181163000000007, 2024-09-14T09:06:12.410 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.410 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.410 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:12.411 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.411 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.411 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.411 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49341 2024-09-14T09:06:12.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.411 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546667+0000", 2024-09-14T09:06:12.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546667+0000", 2024-09-14T09:06:12.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546669+0000", 2024-09-14T09:06:12.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.486e-06 2024-09-14T09:06:12.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546674+0000", 2024-09-14T09:06:12.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5110000000000003e-06 2024-09-14T09:06:12.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546691+0000", 2024-09-14T09:06:12.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7051e-05 2024-09-14T09:06:12.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546703+0000", 2024-09-14T09:06:12.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1155000000000001e-05 2024-09-14T09:06:12.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618836+0000", 2024-09-14T09:06:12.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072133616999999997 2024-09-14T09:06:12.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618848+0000", 2024-09-14T09:06:12.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2343e-05 2024-09-14T09:06:12.414 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.414 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.414 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.414 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49386 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:12.414 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546806+0000", 2024-09-14T09:06:12.414 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.8153314630000001, 2024-09-14T09:06:12.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072019206000000002, 2024-09-14T09:06:12.414 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.414 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.414 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:12.415 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.415 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.415 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.415 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49386 2024-09-14T09:06:12.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.415 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546806+0000", 2024-09-14T09:06:12.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546806+0000", 2024-09-14T09:06:12.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546808+0000", 2024-09-14T09:06:12.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.471e-06 2024-09-14T09:06:12.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546813+0000", 2024-09-14T09:06:12.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1800000000000004e-06 2024-09-14T09:06:12.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546830+0000", 2024-09-14T09:06:12.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.658e-05 2024-09-14T09:06:12.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546842+0000", 2024-09-14T09:06:12.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2442999999999999e-05 2024-09-14T09:06:12.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618816+0000", 2024-09-14T09:06:12.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071973920999999996 2024-09-14T09:06:12.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618826+0000", 2024-09-14T09:06:12.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.611e-06 2024-09-14T09:06:12.418 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.418 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.418 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.418 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49387 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:12.418 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546861+0000", 2024-09-14T09:06:12.419 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.8152768180000001, 2024-09-14T09:06:12.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071996665000000001, 2024-09-14T09:06:12.419 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.419 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.419 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:12.419 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.419 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.419 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.419 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49387 2024-09-14T09:06:12.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.419 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546861+0000", 2024-09-14T09:06:12.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546861+0000", 2024-09-14T09:06:12.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546862+0000", 2024-09-14T09:06:12.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.186e-06 2024-09-14T09:06:12.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546867+0000", 2024-09-14T09:06:12.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9049999999999996e-06 2024-09-14T09:06:12.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546880+0000", 2024-09-14T09:06:12.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2815e-05 2024-09-14T09:06:12.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546889+0000", 2024-09-14T09:06:12.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5420000000000005e-06 2024-09-14T09:06:12.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618850+0000", 2024-09-14T09:06:12.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071961065000000005 2024-09-14T09:06:12.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618858+0000", 2024-09-14T09:06:12.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1519999999999992e-06 2024-09-14T09:06:12.422 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.422 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.422 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.423 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49388 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:12.423 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:12.423 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.8152391329999999, 2024-09-14T09:06:12.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072001645000000003, 2024-09-14T09:06:12.423 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.423 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.423 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:12.423 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.423 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.423 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.423 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49388 2024-09-14T09:06:12.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.424 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:12.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:12.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:12.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:06:12.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546902+0000", 2024-09-14T09:06:12.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.734e-06 2024-09-14T09:06:12.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546910+0000", 2024-09-14T09:06:12.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9629999999999998e-06 2024-09-14T09:06:12.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546916+0000", 2024-09-14T09:06:12.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.28e-06 2024-09-14T09:06:12.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618889+0000", 2024-09-14T09:06:12.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071972555999999993 2024-09-14T09:06:12.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618900+0000", 2024-09-14T09:06:12.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1413999999999999e-05 2024-09-14T09:06:12.427 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.427 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.427 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.427 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49389 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:12.427 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546925+0000", 2024-09-14T09:06:12.427 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.815212973, 2024-09-14T09:06:12.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072026707999999995, 2024-09-14T09:06:12.427 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.427 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.427 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:12.427 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.427 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.427 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.428 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49389 2024-09-14T09:06:12.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.428 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546925+0000", 2024-09-14T09:06:12.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546925+0000", 2024-09-14T09:06:12.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546926+0000", 2024-09-14T09:06:12.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8599999999999998e-07 2024-09-14T09:06:12.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546928+0000", 2024-09-14T09:06:12.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4200000000000001e-06 2024-09-14T09:06:12.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546938+0000", 2024-09-14T09:06:12.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5859999999999994e-06 2024-09-14T09:06:12.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546945+0000", 2024-09-14T09:06:12.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1770000000000001e-06 2024-09-14T09:06:12.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618940+0000", 2024-09-14T09:06:12.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071995491999999994 2024-09-14T09:06:12.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618952+0000", 2024-09-14T09:06:12.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1347e-05 2024-09-14T09:06:12.431 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.431 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.431 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.431 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49390 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:12.431 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547051+0000", 2024-09-14T09:06:12.431 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.8150873830000001, 2024-09-14T09:06:12.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071963947, 2024-09-14T09:06:12.431 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.432 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.432 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:12.432 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.432 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.432 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.432 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49390 2024-09-14T09:06:12.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.432 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547051+0000", 2024-09-14T09:06:12.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547051+0000", 2024-09-14T09:06:12.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547053+0000", 2024-09-14T09:06:12.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.007e-06 2024-09-14T09:06:12.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547064+0000", 2024-09-14T09:06:12.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1146e-05 2024-09-14T09:06:12.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547089+0000", 2024-09-14T09:06:12.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.563e-05 2024-09-14T09:06:12.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547127+0000", 2024-09-14T09:06:12.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8040000000000002e-05 2024-09-14T09:06:12.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619003+0000", 2024-09-14T09:06:12.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071875724000000002 2024-09-14T09:06:12.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619014+0000", 2024-09-14T09:06:12.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1399999999999999e-05 2024-09-14T09:06:12.436 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.436 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.436 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.436 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49391 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:12.436 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547162+0000", 2024-09-14T09:06:12.436 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.8149762200000001, 2024-09-14T09:06:12.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071911484999999997, 2024-09-14T09:06:12.436 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.437 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.437 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:12.437 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.437 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.437 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.437 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49391 2024-09-14T09:06:12.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.437 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547162+0000", 2024-09-14T09:06:12.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547162+0000", 2024-09-14T09:06:12.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547163+0000", 2024-09-14T09:06:12.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.73e-06 2024-09-14T09:06:12.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.439 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547170+0000", 2024-09-14T09:06:12.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0400000000000004e-06 2024-09-14T09:06:12.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.439 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547190+0000", 2024-09-14T09:06:12.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9423e-05 2024-09-14T09:06:12.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547204+0000", 2024-09-14T09:06:12.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4175999999999999e-05 2024-09-14T09:06:12.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619060+0000", 2024-09-14T09:06:12.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071856347000000001 2024-09-14T09:06:12.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619073+0000", 2024-09-14T09:06:12.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2768999999999999e-05 2024-09-14T09:06:12.440 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.441 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.441 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.441 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49392 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:12.441 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547226+0000", 2024-09-14T09:06:12.441 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.814912241, 2024-09-14T09:06:12.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071890509000000005, 2024-09-14T09:06:12.441 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.441 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.441 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:12.441 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.442 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.442 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.442 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49392 2024-09-14T09:06:12.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.442 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547226+0000", 2024-09-14T09:06:12.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547226+0000", 2024-09-14T09:06:12.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547227+0000", 2024-09-14T09:06:12.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3969999999999999e-06 2024-09-14T09:06:12.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547233+0000", 2024-09-14T09:06:12.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4530000000000001e-06 2024-09-14T09:06:12.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547244+0000", 2024-09-14T09:06:12.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1327e-05 2024-09-14T09:06:12.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547252+0000", 2024-09-14T09:06:12.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0269999999999995e-06 2024-09-14T09:06:12.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619110+0000", 2024-09-14T09:06:12.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071857997000000007 2024-09-14T09:06:12.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619116+0000", 2024-09-14T09:06:12.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3080000000000004e-06 2024-09-14T09:06:12.445 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.445 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.445 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.445 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49393 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:12.445 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547263+0000", 2024-09-14T09:06:12.445 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.8148745979999998, 2024-09-14T09:06:12.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071900037999999999, 2024-09-14T09:06:12.445 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.445 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.445 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:12.446 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.446 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.446 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.446 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49393 2024-09-14T09:06:12.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.446 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547263+0000", 2024-09-14T09:06:12.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547263+0000", 2024-09-14T09:06:12.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547264+0000", 2024-09-14T09:06:12.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3399999999999998e-07 2024-09-14T09:06:12.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547299+0000", 2024-09-14T09:06:12.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5252000000000001e-05 2024-09-14T09:06:12.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547317+0000", 2024-09-14T09:06:12.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.736e-05 2024-09-14T09:06:12.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547332+0000", 2024-09-14T09:06:12.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.487e-05 2024-09-14T09:06:12.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619152+0000", 2024-09-14T09:06:12.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071820590000000004 2024-09-14T09:06:12.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.449 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619163+0000", 2024-09-14T09:06:12.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1131999999999999e-05 2024-09-14T09:06:12.449 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.449 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.449 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.449 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49394 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:12.449 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547353+0000", 2024-09-14T09:06:12.449 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.814785348, 2024-09-14T09:06:12.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071828462999999995, 2024-09-14T09:06:12.449 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.450 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.450 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:12.450 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.450 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.450 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.450 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49394 2024-09-14T09:06:12.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.450 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547353+0000", 2024-09-14T09:06:12.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547353+0000", 2024-09-14T09:06:12.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547354+0000", 2024-09-14T09:06:12.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6130000000000001e-06 2024-09-14T09:06:12.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547360+0000", 2024-09-14T09:06:12.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6869999999999999e-06 2024-09-14T09:06:12.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547378+0000", 2024-09-14T09:06:12.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7805999999999999e-05 2024-09-14T09:06:12.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547391+0000", 2024-09-14T09:06:12.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3617e-05 2024-09-14T09:06:12.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619176+0000", 2024-09-14T09:06:12.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071784608999999999 2024-09-14T09:06:12.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619181+0000", 2024-09-14T09:06:12.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1309999999999997e-06 2024-09-14T09:06:12.453 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.453 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.453 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.454 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49395 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:12.454 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547469+0000", 2024-09-14T09:06:12.454 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.814669109, 2024-09-14T09:06:12.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071780231, 2024-09-14T09:06:12.454 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.454 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.454 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:12.454 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.454 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.454 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.454 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49395 2024-09-14T09:06:12.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.454 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547469+0000", 2024-09-14T09:06:12.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547469+0000", 2024-09-14T09:06:12.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547470+0000", 2024-09-14T09:06:12.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-07 2024-09-14T09:06:12.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.456 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.456 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547473+0000", 2024-09-14T09:06:12.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7330000000000001e-06 2024-09-14T09:06:12.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.456 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.456 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547487+0000", 2024-09-14T09:06:12.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3162999999999999e-05 2024-09-14T09:06:12.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.456 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.456 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547495+0000", 2024-09-14T09:06:12.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8100000000000004e-06 2024-09-14T09:06:12.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619243+0000", 2024-09-14T09:06:12.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071747900000000003 2024-09-14T09:06:12.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619249+0000", 2024-09-14T09:06:12.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7350000000000001e-06 2024-09-14T09:06:12.457 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.457 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.458 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.458 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:50478 3.1d 3:b8a2eaf4:::rbd_data.115e5dc498ac.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T09:06:12.458 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:12.458 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.31249811300000002, 2024-09-14T09:06:12.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.075023792000000006, 2024-09-14T09:06:12.458 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:12.458 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:12.458 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:12.459 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:12.459 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:12.459 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:12.459 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 50478 2024-09-14T09:06:12.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.459 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:12.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:12.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:12.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:12.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:12.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:12.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:12.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049641+0000", 2024-09-14T09:06:12.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6270000000000001e-06 2024-09-14T09:06:12.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:12.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049649+0000", 2024-09-14T09:06:12.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6279999999999996e-06 2024-09-14T09:06:12.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:12.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049667+0000", 2024-09-14T09:06:12.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-09-14T09:06:12.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:12.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049677+0000", 2024-09-14T09:06:12.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0509000000000001e-05 2024-09-14T09:06:12.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:12.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049751+0000", 2024-09-14T09:06:12.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3322e-05 2024-09-14T09:06:12.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:12.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049830+0000", 2024-09-14T09:06:12.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9932000000000007e-05 2024-09-14T09:06:12.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:12.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.053203+0000", 2024-09-14T09:06:12.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003372398 2024-09-14T09:06:12.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:12.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124542+0000", 2024-09-14T09:06:12.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071339105999999999 2024-09-14T09:06:12.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:12.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124562+0000", 2024-09-14T09:06:12.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9649000000000001e-05 2024-09-14T09:06:12.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:12.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124616+0000", 2024-09-14T09:06:12.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4642999999999998e-05 2024-09-14T09:06:12.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:12.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:12.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:12.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124664+0000", 2024-09-14T09:06:12.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7342000000000002e-05 2024-09-14T09:06:12.464 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.464 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.464 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.464 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:12.464 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:12.464 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:12.466 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:06:12.466 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:12.467+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:12.533 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:06:12.578 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:12.575+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:12.633 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:12.633 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:12.633 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:12.633 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:12.647 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:06:12.668 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:12.663+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:12.687 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:12.688 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:12.704 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:12.704 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:12.722 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:12.722 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:12.739 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:12.739 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:12.759 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:12.759 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:12.769 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:12.763+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:12.771 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:12.771 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:12.771 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:12.771 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:12.771 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:12.780 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:12.780 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:12.786 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:06:12.799 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:12.799 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:12.819 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:12.819 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:12.870 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:12.871+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:12.913 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:12.913 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:12.928 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:06:12.971 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:12.967+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:13.072 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:13.075+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:13.097 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:13.097 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:13.111 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:06:13.174 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:13.167+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:13.235 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T09:06:13.259 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone17 from 0xc2f4659 (overlap 0x2cfceb) to 0xd7f13 2024-09-14T09:06:13.275 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:13.275+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:13.294 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:13.294 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:13.310 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:06:13.376 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:13.371+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:13.383 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-14T09:06:13.402 INFO:teuthology.orchestra.run.smithi176.stdout:1957 read 0x6a2da17 thru 0x6a31103 (0x36ed bytes) 2024-09-14T09:06:13.403 INFO:teuthology.orchestra.run.smithi176.stdout:1959 write 0x723c848 thru 0x724c187 (0xf940 bytes) 2024-09-14T09:06:13.417 INFO:teuthology.orchestra.run.smithi176.stdout:1960 read 0x89eade3 thru 0x89fa946 (0xfb64 bytes) 2024-09-14T09:06:13.425 INFO:teuthology.orchestra.run.smithi176.stdout:1961 write 0xca4fbe8 thru 0xca58370 (0x8789 bytes) 2024-09-14T09:06:13.434 INFO:teuthology.orchestra.run.smithi176.stdout:1962 read 0x597f203 thru 0x5981f77 (0x2d75 bytes) 2024-09-14T09:06:13.435 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:13.435 INFO:teuthology.orchestra.run.smithi176.stdout:1967 read 0xa77bd6b thru 0xa788b99 (0xce2f bytes) 2024-09-14T09:06:13.435 INFO:teuthology.orchestra.run.smithi176.stdout:1968 read 0xae787f4 thru 0xae80eb2 (0x86bf bytes) 2024-09-14T09:06:13.442 INFO:teuthology.orchestra.run.smithi176.stdout:1970 writesame 0xd685b8b thru 0xd689908 data_size 0x11(0x3d7e bytes) 2024-09-14T09:06:13.442 INFO:teuthology.orchestra.run.smithi176.stdout:1971 punch from 0xb74929c to 0xb7557af, (0xc513 bytes) 2024-09-14T09:06:13.442 INFO:teuthology.orchestra.run.smithi176.stdout:1972 write 0x77e9535 thru 0x77f82cf (0xed9b bytes) 2024-09-14T09:06:13.469 INFO:teuthology.orchestra.run.smithi176.stdout:1973 trunc from 0xd689909 to 0x36bbf74 2024-09-14T09:06:13.477 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:13.479+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:13.490 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:13.490 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:13.506 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:06:13.527 INFO:teuthology.orchestra.run.smithi176.stdout:1974 write 0xddd7018 thru 0xdde0e9c (0x9e85 bytes) 2024-09-14T09:06:13.540 INFO:teuthology.orchestra.run.smithi176.stdout:1975 read 0x672e8b9 thru 0x67305b6 (0x1cfe bytes) 2024-09-14T09:06:13.540 INFO:teuthology.orchestra.run.smithi176.stdout:1977 read 0x4912d5b thru 0x4916a5a (0x3d00 bytes) 2024-09-14T09:06:13.540 INFO:teuthology.orchestra.run.smithi176.stdout:1978 writesame 0x418f1c3 thru 0x4193ccd data_size 0x1(0x4b0b bytes) 2024-09-14T09:06:13.577 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:13.571+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:13.580 INFO:teuthology.orchestra.run.smithi176.stdout:1979 write 0x4317964 thru 0x43256ad (0xdd4a bytes) 2024-09-14T09:06:13.601 INFO:teuthology.orchestra.run.smithi176.stdout:1980 write 0xed47040 thru 0xed4b54d (0x450e bytes) 2024-09-14T09:06:13.612 INFO:teuthology.orchestra.run.smithi176.stdout:1981 clone 19 order 19 su 131072 sc 15 2024-09-14T09:06:13.634 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:13.634 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:13.648 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:06:13.679 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:13.679+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:13.780 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:13.775+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:13.810 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:13.810 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:13.824 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:06:13.914 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:13.915+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:13.981 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:13.983+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:13.990 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:13.990 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:14.004 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:06:14.082 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:14.083+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:14.131 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is set 2024-09-14T09:06:14.182 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:14.182 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:14.182 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:14.183+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:14.197 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:06:14.264 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone18 from 0xed4b54e (overlap 0x36bbf74) to 0x1b5dfba 2024-09-14T09:06:14.283 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:14.283+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:14.384 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:14.379+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:14.398 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:14.398 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:14.415 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:06:14.485 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:14.487+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:14.515 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-14T09:06:14.523 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:14.523 INFO:teuthology.orchestra.run.smithi176.stdout:1985 write 0xe894d44 thru 0xe896b6d (0x1e2a bytes) 2024-09-14T09:06:14.537 INFO:teuthology.orchestra.run.smithi176.stdout:1986 punch from 0x351e366 to 0x35207c6, (0x2460 bytes) 2024-09-14T09:06:14.537 INFO:teuthology.orchestra.run.smithi176.stdout:1987 read 0x8e9f421 thru 0x8e9fa67 (0x647 bytes) 2024-09-14T09:06:14.540 INFO:teuthology.orchestra.run.smithi176.stdout:1988 trunc from 0xed4b54e to 0xd5b1ab0 2024-09-14T09:06:14.583 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:14.583 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:14.584 INFO:teuthology.orchestra.run.smithi176.stdout:1991 punch from 0xc3654d5 to 0xc37052d, (0xb058 bytes) 2024-09-14T09:06:14.585 INFO:teuthology.orchestra.run.smithi176.stdout:1993 write 0xbcb5fe5 thru 0xbcb96f5 (0x3711 bytes) 2024-09-14T09:06:14.585 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:14.587+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:14.601 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:06:14.601 INFO:teuthology.orchestra.run.smithi176.stdout:1994 trunc from 0xd5b1ab0 to 0x8c2054a 2024-09-14T09:06:14.650 INFO:teuthology.orchestra.run.smithi176.stdout:1995 writesame 0x3844bc9 thru 0x3848e27 data_size 0x1(0x425f bytes) 2024-09-14T09:06:14.686 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:14.687+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:14.688 INFO:teuthology.orchestra.run.smithi176.stdout:1996 read 0x6ad21a thru 0x6b6fed (0x9dd4 bytes) 2024-09-14T09:06:14.689 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:14.689 INFO:teuthology.orchestra.run.smithi176.stdout:1999 read 0xaed048 thru 0xaf6544 (0x94fd bytes) 2024-09-14T09:06:14.690 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:14.707 INFO:teuthology.orchestra.run.smithi176.stdout:2001 write 0xb9a85c0 thru 0xb9ad59d (0x4fde bytes) 2024-09-14T09:06:14.711 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:14.711 INFO:teuthology.orchestra.run.smithi176.stdout:2005 write 0x4f980ef thru 0x4f9cd25 (0x4c37 bytes) 2024-09-14T09:06:14.716 INFO:teuthology.orchestra.run.smithi176.stdout:2006 read 0x6441e4c thru 0x64470f1 (0x52a6 bytes) 2024-09-14T09:06:14.718 INFO:teuthology.orchestra.run.smithi176.stdout:2007 trunc from 0xb9ad59e to 0xd0c8106 2024-09-14T09:06:14.738 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:14.738 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:14.742 INFO:teuthology.orchestra.run.smithi176.stdout:2008 write 0xe7acb0f thru 0xe7b3667 (0x6b59 bytes) 2024-09-14T09:06:14.748 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:14.749 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:14.749 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:14.749 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:14.749 INFO:teuthology.orchestra.run.smithi176.stdout:2013 punch from 0xa0180c to 0xa0b2b2, (0x9aa6 bytes) 2024-09-14T09:06:14.749 INFO:teuthology.orchestra.run.smithi176.stdout:2017 punch from 0xdb90ecd to 0xdb9511c, (0x424f bytes) 2024-09-14T09:06:14.749 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:14.754 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:06:14.772 INFO:teuthology.orchestra.run.smithi176.stdout:2019 trunc from 0xe7b3668 to 0x3ff2b82 2024-09-14T09:06:14.787 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:14.783+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:14.822 INFO:teuthology.orchestra.run.smithi176.stdout:2020 read 0x13ca426 thru 0x13d8dea (0xe9c5 bytes) 2024-09-14T09:06:14.836 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:06:14.851 INFO:teuthology.orchestra.run.smithi176.stdout:2021 write 0xc3260ee thru 0xc32cd78 (0x6c8b bytes) 2024-09-14T09:06:14.856 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:14.871 INFO:teuthology.orchestra.run.smithi176.stdout:2023 write 0xcf8fc53 thru 0xcf9d4c7 (0xd875 bytes) 2024-09-14T09:06:14.886 INFO:teuthology.orchestra.run.smithi176.stdout:2024 trunc from 0xcf9d4c8 to 0x76dadb0 2024-09-14T09:06:14.888 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:14.891+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:14.894 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:14.894 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:14.905 INFO:teuthology.orchestra.run.smithi176.stdout:2025 punch from 0x3c3bc1b to 0x3c47f88, (0xc36d bytes) 2024-09-14T09:06:14.909 INFO:teuthology.orchestra.run.smithi176.stdout:2026 trunc from 0x76dadb0 to 0x503a5fb 2024-09-14T09:06:14.911 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:06:14.920 INFO:teuthology.orchestra.run.smithi176.stdout:2027 read 0x2d02311 thru 0x2d03e82 (0x1b72 bytes) 2024-09-14T09:06:14.921 INFO:teuthology.orchestra.run.smithi176.stdout:2030 write 0x2a2fff0 thru 0x2a3c9c8 (0xc9d9 bytes) 2024-09-14T09:06:14.935 INFO:teuthology.orchestra.run.smithi176.stdout:2031 read 0x28127e6 thru 0x281fb89 (0xd3a4 bytes) 2024-09-14T09:06:14.935 INFO:teuthology.orchestra.run.smithi176.stdout:2033 write 0x17ac0aa thru 0x17b6d76 (0xaccd bytes) 2024-09-14T09:06:14.947 INFO:teuthology.orchestra.run.smithi176.stdout:2035 read 0x943c39 thru 0x9444d5 (0x89d bytes) 2024-09-14T09:06:14.955 INFO:teuthology.orchestra.run.smithi176.stdout:2036 trunc from 0x503a5fb to 0x9e990df 2024-09-14T09:06:14.966 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:14.970 INFO:teuthology.orchestra.run.smithi176.stdout:2038 trunc from 0x9e990df to 0xcaea0ca 2024-09-14T09:06:14.982 INFO:teuthology.orchestra.run.smithi176.stdout:2039 read 0xc5b8d0 thru 0xc67728 (0xbe59 bytes) 2024-09-14T09:06:14.989 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:14.991+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:14.991 INFO:teuthology.orchestra.run.smithi176.stdout:2040 trunc from 0xcaea0ca to 0x6e44db0 2024-09-14T09:06:14.992 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:14.992 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:15.010 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:15.010 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:15.017 INFO:teuthology.orchestra.run.smithi176.stdout:2041 write 0x9c80cfe thru 0x9c87a87 (0x6d8a bytes) 2024-09-14T09:06:15.022 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:15.022 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:15.023 INFO:teuthology.orchestra.run.smithi176.stdout:2045 read 0x285ce4e thru 0x2866d6a (0x9f1d bytes) 2024-09-14T09:06:15.023 INFO:teuthology.orchestra.run.smithi176.stdout:2046 read 0x414854d thru 0x41532d3 (0xad87 bytes) 2024-09-14T09:06:15.024 INFO:teuthology.orchestra.run.smithi176.stdout:2048 read 0x37c48ae thru 0x37d10ea (0xc83d bytes) 2024-09-14T09:06:15.024 INFO:teuthology.orchestra.run.smithi176.stdout:2049 read 0x81eecdd thru 0x81f924f (0xa573 bytes) 2024-09-14T09:06:15.024 INFO:teuthology.orchestra.run.smithi176.stdout:2050 write 0x879211b thru 0x87967d5 (0x46bb bytes) 2024-09-14T09:06:15.028 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:15.028 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:15.045 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:15.045 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:15.047 INFO:teuthology.orchestra.run.smithi176.stdout:2052 writesame 0xc85a38f thru 0xc861630 data_size 0x6(0x72a2 bytes) 2024-09-14T09:06:15.047 INFO:teuthology.orchestra.run.smithi176.stdout:2053 read 0x5901200 thru 0x590a8d8 (0x96d9 bytes) 2024-09-14T09:06:15.047 INFO:teuthology.orchestra.run.smithi176.stdout:2054 read 0x522084f thru 0x5229d4b (0x94fd bytes) 2024-09-14T09:06:15.047 INFO:teuthology.orchestra.run.smithi176.stdout:2055 read 0x61731fc thru 0x617f2f8 (0xc0fd bytes) 2024-09-14T09:06:15.047 INFO:teuthology.orchestra.run.smithi176.stdout:2056 writesame 0xb02fd31 thru 0xb03d709 data_size 0x1f(0xd9d9 bytes) 2024-09-14T09:06:15.052 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:15.052 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:15.052 INFO:teuthology.orchestra.run.smithi176.stdout:2062 read 0xa7b2bf4 thru 0xa7b33f2 (0x7ff bytes) 2024-09-14T09:06:15.052 INFO:teuthology.orchestra.run.smithi176.stdout:2063 read 0x287041e thru 0x28729a2 (0x2585 bytes) 2024-09-14T09:06:15.052 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:15.057 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:15.057 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18383 3.9 3:9252f7c9:::rbd_data.115e3abd3b2c.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:06:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 31.899057459000002, 2024-09-14T09:06:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056433830999999997, 2024-09-14T09:06:15.058 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18383 2024-09-14T09:06:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158423+0000", 2024-09-14T09:06:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T09:06:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158429+0000", 2024-09-14T09:06:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0179999999999998e-06 2024-09-14T09:06:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158440+0000", 2024-09-14T09:06:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0769e-05 2024-09-14T09:06:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158445+0000", 2024-09-14T09:06:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5840000000000002e-06 2024-09-14T09:06:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158495+0000", 2024-09-14T09:06:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9129999999999999e-05 2024-09-14T09:06:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158598+0000", 2024-09-14T09:06:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102938 2024-09-14T09:06:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.161048+0000", 2024-09-14T09:06:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002450336 2024-09-14T09:06:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214605+0000", 2024-09-14T09:06:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053557526000000001 2024-09-14T09:06:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214634+0000", 2024-09-14T09:06:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8483000000000002e-05 2024-09-14T09:06:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214731+0000", 2024-09-14T09:06:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7026000000000002e-05 2024-09-14T09:06:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214856+0000", 2024-09-14T09:06:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012474799999999999 2024-09-14T09:06:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.063 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18383 3.9 e38/34 ondisk, result = 0)", 2024-09-14T09:06:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 31.888740978000001, 2024-09-14T09:06:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045880575999999999, 2024-09-14T09:06:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168739+0000", 2024-09-14T09:06:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3900000000000004e-07 2024-09-14T09:06:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168744+0000", 2024-09-14T09:06:15.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5929999999999999e-06 2024-09-14T09:06:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168754+0000", 2024-09-14T09:06:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0686999999999999e-05 2024-09-14T09:06:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168761+0000", 2024-09-14T09:06:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6340000000000003e-06 2024-09-14T09:06:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214578+0000", 2024-09-14T09:06:15.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045817443999999999 2024-09-14T09:06:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214596+0000", 2024-09-14T09:06:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7159e-05 2024-09-14T09:06:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214619+0000", 2024-09-14T09:06:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.332e-05 2024-09-14T09:06:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19152 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14159058~8940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:15.068 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.734828522000001, 2024-09-14T09:06:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12891994500000001, 2024-09-14T09:06:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19152 2024-09-14T09:06:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322653+0000", 2024-09-14T09:06:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3750000000000001e-06 2024-09-14T09:06:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329647+0000", 2024-09-14T09:06:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069941960000000003 2024-09-14T09:06:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329673+0000", 2024-09-14T09:06:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5760000000000001e-05 2024-09-14T09:06:15.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329684+0000", 2024-09-14T09:06:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0474e-05 2024-09-14T09:06:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329776+0000", 2024-09-14T09:06:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2467999999999996e-05 2024-09-14T09:06:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329987+0000", 2024-09-14T09:06:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021133 2024-09-14T09:06:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.331303+0000", 2024-09-14T09:06:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001315629 2024-09-14T09:06:15.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.411090+0000", 2024-09-14T09:06:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079787477999999995 2024-09-14T09:06:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451463+0000", 2024-09-14T09:06:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040372312 2024-09-14T09:06:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451523+0000", 2024-09-14T09:06:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0420000000000001e-05 2024-09-14T09:06:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451571+0000", 2024-09-14T09:06:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7503000000000002e-05 2024-09-14T09:06:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20925 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:06:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.385529741999999, 2024-09-14T09:06:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017084327999999999, 2024-09-14T09:06:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20925 2024-09-14T09:06:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:06:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:06:15.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:06:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3e-07 2024-09-14T09:06:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671953+0000", 2024-09-14T09:06:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0299999999999998e-06 2024-09-14T09:06:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671964+0000", 2024-09-14T09:06:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1e-05 2024-09-14T09:06:15.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671971+0000", 2024-09-14T09:06:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9929999999999998e-06 2024-09-14T09:06:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.687941+0000", 2024-09-14T09:06:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015969357 2024-09-14T09:06:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.688070+0000", 2024-09-14T09:06:15.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000129858 2024-09-14T09:06:15.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689034+0000", 2024-09-14T09:06:15.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00096336000000000002 2024-09-14T09:06:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20926 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:15.082 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.385496522, 2024-09-14T09:06:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018572207, 2024-09-14T09:06:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20926 2024-09-14T09:06:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:06:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671990+0000", 2024-09-14T09:06:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5769999999999999e-06 2024-09-14T09:06:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672001+0000", 2024-09-14T09:06:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1293e-05 2024-09-14T09:06:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672008+0000", 2024-09-14T09:06:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9099999999999999e-06 2024-09-14T09:06:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689147+0000", 2024-09-14T09:06:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017139060000000001 2024-09-14T09:06:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689324+0000", 2024-09-14T09:06:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017640200000000001 2024-09-14T09:06:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690555+0000", 2024-09-14T09:06:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001231212 2024-09-14T09:06:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20927 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2621440~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.385459556000001, 2024-09-14T09:06:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020002227000000001, 2024-09-14T09:06:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20927 2024-09-14T09:06:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9699999999999995e-07 2024-09-14T09:06:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672024+0000", 2024-09-14T09:06:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:06:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672036+0000", 2024-09-14T09:06:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.119e-05 2024-09-14T09:06:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672044+0000", 2024-09-14T09:06:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8370000000000003e-06 2024-09-14T09:06:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690602+0000", 2024-09-14T09:06:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018557648 2024-09-14T09:06:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690805+0000", 2024-09-14T09:06:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020282899999999999 2024-09-14T09:06:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692022+0000", 2024-09-14T09:06:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012170429999999999 2024-09-14T09:06:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20928 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 917504~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.385423415999998, 2024-09-14T09:06:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021623738, 2024-09-14T09:06:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20928 2024-09-14T09:06:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672057+0000", 2024-09-14T09:06:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:06:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672059+0000", 2024-09-14T09:06:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9270000000000001e-06 2024-09-14T09:06:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672070+0000", 2024-09-14T09:06:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0773e-05 2024-09-14T09:06:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672077+0000", 2024-09-14T09:06:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9600000000000003e-06 2024-09-14T09:06:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692154+0000", 2024-09-14T09:06:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020076482 2024-09-14T09:06:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692419+0000", 2024-09-14T09:06:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026517400000000002 2024-09-14T09:06:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693680+0000", 2024-09-14T09:06:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012607390000000001 2024-09-14T09:06:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20929 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2752512~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.385390374, 2024-09-14T09:06:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022898728, 2024-09-14T09:06:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20929 2024-09-14T09:06:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672090+0000", 2024-09-14T09:06:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:06:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672092+0000", 2024-09-14T09:06:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8940000000000002e-06 2024-09-14T09:06:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672103+0000", 2024-09-14T09:06:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0815999999999999e-05 2024-09-14T09:06:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672110+0000", 2024-09-14T09:06:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9469999999999997e-06 2024-09-14T09:06:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693737+0000", 2024-09-14T09:06:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021626467999999999 2024-09-14T09:06:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693962+0000", 2024-09-14T09:06:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225474 2024-09-14T09:06:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.694988+0000", 2024-09-14T09:06:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001025469 2024-09-14T09:06:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20930 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.385357280000001, 2024-09-14T09:06:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024303286, 2024-09-14T09:06:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20930 2024-09-14T09:06:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672123+0000", 2024-09-14T09:06:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:06:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672125+0000", 2024-09-14T09:06:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7300000000000001e-06 2024-09-14T09:06:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672134+0000", 2024-09-14T09:06:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7930000000000003e-06 2024-09-14T09:06:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672141+0000", 2024-09-14T09:06:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9029999999999998e-06 2024-09-14T09:06:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695087+0000", 2024-09-14T09:06:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022946013000000001 2024-09-14T09:06:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695350+0000", 2024-09-14T09:06:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026266000000000001 2024-09-14T09:06:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696425+0000", 2024-09-14T09:06:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001075527 2024-09-14T09:06:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20931 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.385324051000001, 2024-09-14T09:06:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025331519, 2024-09-14T09:06:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20931 2024-09-14T09:06:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672156+0000", 2024-09-14T09:06:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4300000000000002e-07 2024-09-14T09:06:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672159+0000", 2024-09-14T09:06:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T09:06:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672170+0000", 2024-09-14T09:06:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1192999999999999e-05 2024-09-14T09:06:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672177+0000", 2024-09-14T09:06:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9059999999999996e-06 2024-09-14T09:06:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696532+0000", 2024-09-14T09:06:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024355291000000001 2024-09-14T09:06:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696755+0000", 2024-09-14T09:06:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022297699999999999 2024-09-14T09:06:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697487+0000", 2024-09-14T09:06:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00073171499999999997 2024-09-14T09:06:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20932 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1179648~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.385290607999998, 2024-09-14T09:06:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026104083, 2024-09-14T09:06:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20932 2024-09-14T09:06:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0699999999999996e-07 2024-09-14T09:06:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672192+0000", 2024-09-14T09:06:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7700000000000002e-06 2024-09-14T09:06:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672203+0000", 2024-09-14T09:06:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0956e-05 2024-09-14T09:06:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672211+0000", 2024-09-14T09:06:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3769999999999996e-06 2024-09-14T09:06:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697530+0000", 2024-09-14T09:06:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025318411999999998 2024-09-14T09:06:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697640+0000", 2024-09-14T09:06:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011045299999999999 2024-09-14T09:06:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698293+0000", 2024-09-14T09:06:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065240800000000005 2024-09-14T09:06:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20933 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3014656~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.385256405, 2024-09-14T09:06:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026558875999999999, 2024-09-14T09:06:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20933 2024-09-14T09:06:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672224+0000", 2024-09-14T09:06:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2399999999999997e-07 2024-09-14T09:06:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672234+0000", 2024-09-14T09:06:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0793e-05 2024-09-14T09:06:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672246+0000", 2024-09-14T09:06:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659e-05 2024-09-14T09:06:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672255+0000", 2024-09-14T09:06:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5730000000000008e-06 2024-09-14T09:06:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698326+0000", 2024-09-14T09:06:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026071414000000001 2024-09-14T09:06:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698374+0000", 2024-09-14T09:06:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8032999999999999e-05 2024-09-14T09:06:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698782+0000", 2024-09-14T09:06:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040768 2024-09-14T09:06:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20934 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1310720~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.385214528999999, 2024-09-14T09:06:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027292067999999999, 2024-09-14T09:06:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20934 2024-09-14T09:06:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672266+0000", 2024-09-14T09:06:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6299999999999993e-07 2024-09-14T09:06:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672269+0000", 2024-09-14T09:06:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8569999999999999e-06 2024-09-14T09:06:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672280+0000", 2024-09-14T09:06:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1236999999999999e-05 2024-09-14T09:06:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672287+0000", 2024-09-14T09:06:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8260000000000004e-06 2024-09-14T09:06:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698820+0000", 2024-09-14T09:06:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026533055999999999 2024-09-14T09:06:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698933+0000", 2024-09-14T09:06:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011291 2024-09-14T09:06:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699557+0000", 2024-09-14T09:06:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062421899999999997 2024-09-14T09:06:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20935 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.385180769000002, 2024-09-14T09:06:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028108610999999999, 2024-09-14T09:06:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20935 2024-09-14T09:06:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672299+0000", 2024-09-14T09:06:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T09:06:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672302+0000", 2024-09-14T09:06:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7640000000000001e-06 2024-09-14T09:06:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672313+0000", 2024-09-14T09:06:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1080000000000001e-05 2024-09-14T09:06:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672321+0000", 2024-09-14T09:06:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6490000000000008e-06 2024-09-14T09:06:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699587+0000", 2024-09-14T09:06:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027265884000000001 2024-09-14T09:06:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699631+0000", 2024-09-14T09:06:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4947999999999997e-05 2024-09-14T09:06:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700407+0000", 2024-09-14T09:06:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00077559299999999999 2024-09-14T09:06:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20936 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1441792~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.385146593000002, 2024-09-14T09:06:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028784750000000001, 2024-09-14T09:06:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20936 2024-09-14T09:06:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4700000000000001e-07 2024-09-14T09:06:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672336+0000", 2024-09-14T09:06:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6730000000000001e-06 2024-09-14T09:06:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672347+0000", 2024-09-14T09:06:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0710000000000001e-05 2024-09-14T09:06:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672354+0000", 2024-09-14T09:06:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4630000000000004e-06 2024-09-14T09:06:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700442+0000", 2024-09-14T09:06:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028087951 2024-09-14T09:06:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700557+0000", 2024-09-14T09:06:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011461800000000001 2024-09-14T09:06:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701117+0000", 2024-09-14T09:06:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056058799999999999 2024-09-14T09:06:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20937 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.385113467, 2024-09-14T09:06:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029493028000000001, 2024-09-14T09:06:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20937 2024-09-14T09:06:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T09:06:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672384+0000", 2024-09-14T09:06:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7127000000000001e-05 2024-09-14T09:06:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672395+0000", 2024-09-14T09:06:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759999999999999e-05 2024-09-14T09:06:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672402+0000", 2024-09-14T09:06:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0960000000000003e-06 2024-09-14T09:06:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701147+0000", 2024-09-14T09:06:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028744241 2024-09-14T09:06:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701246+0000", 2024-09-14T09:06:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9031999999999996e-05 2024-09-14T09:06:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701859+0000", 2024-09-14T09:06:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061310499999999997 2024-09-14T09:06:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20938 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.385066306999999, 2024-09-14T09:06:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030109219999999999, 2024-09-14T09:06:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20938 2024-09-14T09:06:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672414+0000", 2024-09-14T09:06:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-07 2024-09-14T09:06:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672417+0000", 2024-09-14T09:06:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:06:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672428+0000", 2024-09-14T09:06:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0873000000000001e-05 2024-09-14T09:06:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672435+0000", 2024-09-14T09:06:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:06:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701884+0000", 2024-09-14T09:06:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029449811999999999 2024-09-14T09:06:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701934+0000", 2024-09-14T09:06:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9703000000000002e-05 2024-09-14T09:06:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702522+0000", 2024-09-14T09:06:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00058811200000000001 2024-09-14T09:06:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20939 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.385034174000001, 2024-09-14T09:06:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030779259, 2024-09-14T09:06:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20939 2024-09-14T09:06:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672446+0000", 2024-09-14T09:06:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:06:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672449+0000", 2024-09-14T09:06:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7930000000000002e-06 2024-09-14T09:06:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672461+0000", 2024-09-14T09:06:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T09:06:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672471+0000", 2024-09-14T09:06:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0807e-05 2024-09-14T09:06:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702551+0000", 2024-09-14T09:06:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030079228 2024-09-14T09:06:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702664+0000", 2024-09-14T09:06:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011351800000000001 2024-09-14T09:06:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.703224+0000", 2024-09-14T09:06:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056026999999999997 2024-09-14T09:06:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29321 3.9 3:90303d3a:::rbd_data.115ebc3c67d1.0000000000000014:c [read 770048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:06:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:06:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 22.237950849000001, 2024-09-14T09:06:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015942787, 2024-09-14T09:06:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29321 2024-09-14T09:06:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:06:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:06:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819529+0000", 2024-09-14T09:06:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8800000000000003e-07 2024-09-14T09:06:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819531+0000", 2024-09-14T09:06:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9439999999999999e-06 2024-09-14T09:06:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819537+0000", 2024-09-14T09:06:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.037e-06 2024-09-14T09:06:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819542+0000", 2024-09-14T09:06:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7670000000000003e-06 2024-09-14T09:06:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835320+0000", 2024-09-14T09:06:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015778119 2024-09-14T09:06:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835384+0000", 2024-09-14T09:06:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3897000000000002e-05 2024-09-14T09:06:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835471+0000", 2024-09-14T09:06:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7534999999999995e-05 2024-09-14T09:06:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29322 3.9 3:90303d3a:::rbd_data.115ebc3c67d1.0000000000000014:c [read 780288~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:06:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:06:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 22.237931338999999, 2024-09-14T09:06:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016069904999999999, 2024-09-14T09:06:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29322 2024-09-14T09:06:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:06:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:06:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:06:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9499999999999998e-07 2024-09-14T09:06:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819550+0000", 2024-09-14T09:06:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8309999999999999e-06 2024-09-14T09:06:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819556+0000", 2024-09-14T09:06:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1299999999999998e-06 2024-09-14T09:06:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819561+0000", 2024-09-14T09:06:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.707e-06 2024-09-14T09:06:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835500+0000", 2024-09-14T09:06:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015939456000000001 2024-09-14T09:06:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835566+0000", 2024-09-14T09:06:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5285999999999998e-05 2024-09-14T09:06:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835618+0000", 2024-09-14T09:06:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2099999999999999e-05 2024-09-14T09:06:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.162 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:15.163 INFO:teuthology.orchestra.run.smithi176.stdout:2065 trunc from 0xc861631 to 0x7656801 2024-09-14T09:06:15.163 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:06:15.164 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:15.164 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:15.165 INFO:teuthology.orchestra.run.smithi176.stdout:2066 read 0x5f32570 thru 0x5f32cc8 (0x759 bytes) 2024-09-14T09:06:15.165 INFO:teuthology.orchestra.run.smithi176.stdout:2067 punch from 0x2853c17 to 0x2863158, (0xf541 bytes) 2024-09-14T09:06:15.165 INFO:teuthology.orchestra.run.smithi176.stdout:2068 read 0x3bb7f82 thru 0x3bbfc37 (0x7cb6 bytes) 2024-09-14T09:06:15.165 INFO:teuthology.orchestra.run.smithi176.stdout:2069 writesame 0xb7a858e thru 0xb7abc87 data_size 0x1f(0x36fa bytes) 2024-09-14T09:06:15.165 INFO:teuthology.orchestra.run.smithi176.stdout:2070 writesame 0x1494249 thru 0x149b968 data_size 0x10(0x7720 bytes) 2024-09-14T09:06:15.165 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:15.165 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:15.166 INFO:teuthology.orchestra.run.smithi176.stdout:2071 read 0x89cf92f thru 0x89d58d9 (0x5fab bytes) 2024-09-14T09:06:15.166 INFO:teuthology.orchestra.run.smithi176.stdout:2073 trunc from 0xb7abc88 to 0xe109cef 2024-09-14T09:06:15.166 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:15.166 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:15.167 INFO:teuthology.orchestra.run.smithi176.stdout:2074 writesame 0x2644f6b thru 0x2647a3a data_size 0x14(0x2ad0 bytes) 2024-09-14T09:06:15.167 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:15.167 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:15.167 INFO:teuthology.orchestra.run.smithi176.stdout:2080 write 0x1d1f85 thru 0x1db17c (0x91f8 bytes) 2024-09-14T09:06:15.167 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:15.168 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:15.168 INFO:teuthology.orchestra.run.smithi176.stdout:2081 read 0xa4899dc thru 0xa49646c (0xca91 bytes) 2024-09-14T09:06:15.168 INFO:teuthology.orchestra.run.smithi176.stdout:2082 trunc from 0xe109cef to 0x6f0db14 2024-09-14T09:06:15.169 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:15.163+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:15.185 INFO:teuthology.orchestra.run.smithi176.stdout:2084 trunc from 0x6f0db14 to 0xd8802cc 2024-09-14T09:06:15.197 INFO:teuthology.orchestra.run.smithi176.stdout:2085 punch from 0x9481c66 to 0x948229b, (0x635 bytes) 2024-09-14T09:06:15.208 INFO:teuthology.orchestra.run.smithi176.stdout:2086 trunc from 0xd8802cc to 0x6149822 2024-09-14T09:06:15.244 INFO:teuthology.orchestra.run.smithi176.stdout:2087 write 0xc2590d0 thru 0xc263e67 (0xad98 bytes) 2024-09-14T09:06:15.253 INFO:teuthology.orchestra.run.smithi176.stdout:2088 write 0xc1bf9e1 thru 0xc1cbfa6 (0xc5c6 bytes) 2024-09-14T09:06:15.264 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:15.259+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:15.266 INFO:teuthology.orchestra.run.smithi176.stdout:2090 punch from 0x19d166a to 0x19dcca1, (0xb637 bytes) 2024-09-14T09:06:15.266 INFO:teuthology.orchestra.run.smithi176.stdout:2091 write 0x8cb22 thru 0x9c2f5 (0xf7d4 bytes) 2024-09-14T09:06:15.284 INFO:teuthology.orchestra.run.smithi176.stdout:2094 punch from 0xaee88f2 to 0xaee9b6b, (0x1279 bytes) 2024-09-14T09:06:15.301 INFO:teuthology.orchestra.run.smithi176.stdout:2095 write 0xd6ec506 thru 0xd6eebb1 (0x26ac bytes) 2024-09-14T09:06:15.308 INFO:teuthology.orchestra.run.smithi176.stdout:2096 write 0xb0aeed9 thru 0xb0b2c41 (0x3d69 bytes) 2024-09-14T09:06:15.311 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:15.311 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:15.311 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:15.311 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:15.317 INFO:teuthology.orchestra.run.smithi176.stdout:2097 writesame 0x8b612 thru 0x8bd36 data_size 0x1(0x725 bytes) 2024-09-14T09:06:15.326 INFO:teuthology.orchestra.run.smithi176.stdout:2098 trunc from 0xd6eebb2 to 0xac6b674 2024-09-14T09:06:15.328 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:06:15.349 INFO:teuthology.orchestra.run.smithi176.stdout:2099 writesame 0x9890029 thru 0x989a08c data_size 0xa(0xa064 bytes) 2024-09-14T09:06:15.365 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:15.367+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:15.367 INFO:teuthology.orchestra.run.smithi176.stdout:2100 trunc from 0xac6b674 to 0x5a77265 2024-09-14T09:06:15.387 INFO:teuthology.orchestra.run.smithi176.stdout:2101 read 0x3926d6c thru 0x392960b (0x28a0 bytes) 2024-09-14T09:06:15.387 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:15.409 INFO:teuthology.orchestra.run.smithi176.stdout:2103 write 0xad583db thru 0xad67f5f (0xfb85 bytes) 2024-09-14T09:06:15.419 INFO:teuthology.orchestra.run.smithi176.stdout:2104 read 0x21ce596 thru 0x21dccdb (0xe746 bytes) 2024-09-14T09:06:15.419 INFO:teuthology.orchestra.run.smithi176.stdout:2105 punch from 0x24b56bc to 0x24c4a99, (0xf3dd bytes) 2024-09-14T09:06:15.419 INFO:teuthology.orchestra.run.smithi176.stdout:2106 read 0x5dcc039 thru 0x5dd7285 (0xb24d bytes) 2024-09-14T09:06:15.419 INFO:teuthology.orchestra.run.smithi176.stdout:2107 punch from 0x827ce08 to 0x8282a54, (0x5c4c bytes) 2024-09-14T09:06:15.419 INFO:teuthology.orchestra.run.smithi176.stdout:2108 read 0x5f4bf4 thru 0x60027c (0xb689 bytes) 2024-09-14T09:06:15.420 INFO:teuthology.orchestra.run.smithi176.stdout:2110 punch from 0x2949bce to 0x294a1d8, (0x60a bytes) 2024-09-14T09:06:15.420 INFO:teuthology.orchestra.run.smithi176.stdout:2111 writesame 0x806e535 thru 0x8078d18 data_size 0x1c(0xa7e4 bytes) 2024-09-14T09:06:15.436 INFO:teuthology.orchestra.run.smithi176.stdout:2113 trunc from 0xad67f60 to 0x950035e 2024-09-14T09:06:15.459 INFO:teuthology.orchestra.run.smithi176.stdout:2115 clone 20 order 18 su 2048 sc 12 2024-09-14T09:06:15.466 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:15.467+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:15.484 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:15.485 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:15.485 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:15.485 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:15.485 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:15.499 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:06:15.590 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:15.591+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:15.668 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:15.671+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:15.670 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:15.670 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:15.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:15.670 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:15.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.671 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:15.671 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:15.671 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.585135880999999, 2024-09-14T09:06:15.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:06:15.671 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.671 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.671 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.671 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.671 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.671 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.671 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:06:15.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.671 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:15.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:15.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:06:15.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:06:15.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:06:15.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:06:15.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:06:15.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:06:15.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:06:15.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:06:15.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:06:15.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:06:15.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:15.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:06:15.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:06:15.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:06:15.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:06:15.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:06:15.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:06:15.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:06:15.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:06:15.675 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.676 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.676 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.676 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:15.676 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:15.676 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.585081223000003, 2024-09-14T09:06:15.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:06:15.676 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.676 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.676 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.676 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.676 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.676 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.676 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:06:15.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.677 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:15.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:15.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:15.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:06:15.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:06:15.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:06:15.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:06:15.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:06:15.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:06:15.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:06:15.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:06:15.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:06:15.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:15.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:06:15.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:06:15.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:06:15.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:06:15.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:06:15.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:06:15.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:06:15.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:06:15.680 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.680 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.680 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.680 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:15.681 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:15.681 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.584975849000003, 2024-09-14T09:06:15.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:06:15.681 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.681 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.681 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.681 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.681 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.681 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.681 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:06:15.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.681 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:15.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:15.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:06:15.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:06:15.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:06:15.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:06:15.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:06:15.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:06:15.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:06:15.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:06:15.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:06:15.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:06:15.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:15.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:06:15.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:06:15.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:06:15.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:06:15.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:06:15.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:06:15.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:06:15.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:06:15.685 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.685 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.685 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.685 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:15.685 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:15.685 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.584619085, 2024-09-14T09:06:15.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:06:15.686 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.686 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.686 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.686 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.686 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.686 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.686 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:06:15.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.686 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:15.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:15.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:06:15.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:06:15.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:06:15.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:06:15.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:06:15.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:06:15.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:06:15.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:06:15.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:06:15.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:06:15.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:15.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:06:15.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:06:15.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:06:15.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:06:15.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:06:15.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:06:15.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:06:15.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:06:15.690 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.690 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.690 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.691 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:15.691 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:15.691 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.584534347000002, 2024-09-14T09:06:15.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:06:15.691 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.691 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.691 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.691 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.691 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.691 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.691 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:06:15.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.691 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:15.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:15.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:06:15.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:06:15.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:06:15.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:06:15.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:06:15.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:06:15.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:06:15.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:06:15.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:06:15.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:06:15.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:15.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:06:15.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:06:15.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:06:15.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:06:15.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:06:15.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:06:15.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:06:15.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:06:15.695 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.695 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.695 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.695 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:15.695 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:15.695 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.584446661000001, 2024-09-14T09:06:15.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:06:15.696 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.696 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.696 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.696 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.696 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.696 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.696 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:06:15.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.696 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:15.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:15.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:06:15.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:06:15.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:06:15.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:06:15.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:06:15.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:06:15.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:06:15.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:06:15.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:06:15.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:06:15.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:15.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:06:15.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:06:15.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:06:15.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:06:15.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:06:15.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:06:15.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:06:15.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:06:15.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.700 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.700 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18388 3.18 3:19e2be55:::rbd_data.115e3abd3b2c.0000000000000021:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:06:15.700 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:15.700 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.448533349999998, 2024-09-14T09:06:15.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042293648000000003, 2024-09-14T09:06:15.701 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.701 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.701 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:15.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.701 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18388 2024-09-14T09:06:15.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.702 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:15.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:15.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220236+0000", 2024-09-14T09:06:15.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1339999999999998e-06 2024-09-14T09:06:15.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220244+0000", 2024-09-14T09:06:15.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1950000000000003e-06 2024-09-14T09:06:15.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220262+0000", 2024-09-14T09:06:15.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8017e-05 2024-09-14T09:06:15.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220270+0000", 2024-09-14T09:06:15.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9480000000000008e-06 2024-09-14T09:06:15.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220357+0000", 2024-09-14T09:06:15.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7185e-05 2024-09-14T09:06:15.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220576+0000", 2024-09-14T09:06:15.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021912200000000001 2024-09-14T09:06:15.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:15.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.222435+0000", 2024-09-14T09:06:15.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018592350000000001 2024-09-14T09:06:15.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:15.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262439+0000", 2024-09-14T09:06:15.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040003368999999997 2024-09-14T09:06:15.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:15.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262452+0000", 2024-09-14T09:06:15.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2941e-05 2024-09-14T09:06:15.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:15.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262490+0000", 2024-09-14T09:06:15.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8478e-05 2024-09-14T09:06:15.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262527+0000", 2024-09-14T09:06:15.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7024000000000003e-05 2024-09-14T09:06:15.707 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.707 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.707 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.707 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:19543 3.5 e39/23 3:a536ff9b:::rbd_data.115e4c71d6e8.0000000000000002:head v 39'8, mlcod=39'6)", 2024-09-14T09:06:15.707 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:15.707 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 31.14567869, 2024-09-14T09:06:15.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.039280019999999999, 2024-09-14T09:06:15.707 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.707 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.707 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:15.707 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:15.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:15.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523090+0000", 2024-09-14T09:06:15.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.773e-06 2024-09-14T09:06:15.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537290+0000", 2024-09-14T09:06:15.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014200007000000001 2024-09-14T09:06:15.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537317+0000", 2024-09-14T09:06:15.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6704999999999999e-05 2024-09-14T09:06:15.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537328+0000", 2024-09-14T09:06:15.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1299e-05 2024-09-14T09:06:15.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537417+0000", 2024-09-14T09:06:15.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8869999999999994e-05 2024-09-14T09:06:15.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537445+0000", 2024-09-14T09:06:15.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8377999999999998e-05 2024-09-14T09:06:15.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:15.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562324+0000", 2024-09-14T09:06:15.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024879307999999999 2024-09-14T09:06:15.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562368+0000", 2024-09-14T09:06:15.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3680000000000002e-05 2024-09-14T09:06:15.711 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.711 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.711 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.711 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38002 3.1e 3:7df715aa:::rbd_data.115eb07a8abd.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:15.711 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:15.711 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.279015443999999, 2024-09-14T09:06:15.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.065148304000000004, 2024-09-14T09:06:15.711 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.711 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.712 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:15.712 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.712 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.712 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.712 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38002 2024-09-14T09:06:15.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.712 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:15.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:15.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389752+0000", 2024-09-14T09:06:15.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1829999999999999e-06 2024-09-14T09:06:15.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389758+0000", 2024-09-14T09:06:15.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0919999999999997e-06 2024-09-14T09:06:15.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389769+0000", 2024-09-14T09:06:15.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759e-05 2024-09-14T09:06:15.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389775+0000", 2024-09-14T09:06:15.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8819999999999998e-06 2024-09-14T09:06:15.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389824+0000", 2024-09-14T09:06:15.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8573999999999999e-05 2024-09-14T09:06:15.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389876+0000", 2024-09-14T09:06:15.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2643999999999999e-05 2024-09-14T09:06:15.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:06:15.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.392640+0000", 2024-09-14T09:06:15.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0027636409999999998 2024-09-14T09:06:15.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:15.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454672+0000", 2024-09-14T09:06:15.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062031703000000001 2024-09-14T09:06:15.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:15.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454765+0000", 2024-09-14T09:06:15.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2862000000000006e-05 2024-09-14T09:06:15.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:15.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454841+0000", 2024-09-14T09:06:15.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5857000000000002e-05 2024-09-14T09:06:15.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454900+0000", 2024-09-14T09:06:15.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9107000000000002e-05 2024-09-14T09:06:15.716 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.716 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.716 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.717 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:38002 3.1e e48/23 ondisk, result = 0)", 2024-09-14T09:06:15.717 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:15.717 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.268698855, 2024-09-14T09:06:15.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054841741999999999, 2024-09-14T09:06:15.717 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.717 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.717 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.717 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:15.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:15.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400069+0000", 2024-09-14T09:06:15.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.339e-06 2024-09-14T09:06:15.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400076+0000", 2024-09-14T09:06:15.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7800000000000003e-06 2024-09-14T09:06:15.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400093+0000", 2024-09-14T09:06:15.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6915999999999999e-05 2024-09-14T09:06:15.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400106+0000", 2024-09-14T09:06:15.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2785e-05 2024-09-14T09:06:15.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454713+0000", 2024-09-14T09:06:15.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054607041000000002 2024-09-14T09:06:15.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454724+0000", 2024-09-14T09:06:15.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1202e-05 2024-09-14T09:06:15.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454910+0000", 2024-09-14T09:06:15.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018567900000000001 2024-09-14T09:06:15.721 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.721 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.721 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.721 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38100 3.b 3:d0f67c24:::rbd_data.115eb07a8abd.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:15.721 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:15.721 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.022845194999999, 2024-09-14T09:06:15.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053701964999999997, 2024-09-14T09:06:15.721 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.722 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.722 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:15.722 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.722 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.722 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.722 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38100 2024-09-14T09:06:15.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.722 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:15.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:15.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645923+0000", 2024-09-14T09:06:15.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:06:15.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645928+0000", 2024-09-14T09:06:15.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0699999999999997e-06 2024-09-14T09:06:15.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645942+0000", 2024-09-14T09:06:15.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4069e-05 2024-09-14T09:06:15.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645951+0000", 2024-09-14T09:06:15.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3230000000000004e-06 2024-09-14T09:06:15.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646008+0000", 2024-09-14T09:06:15.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6583999999999999e-05 2024-09-14T09:06:15.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646089+0000", 2024-09-14T09:06:15.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1557999999999994e-05 2024-09-14T09:06:15.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:15.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.650355+0000", 2024-09-14T09:06:15.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004265525 2024-09-14T09:06:15.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:15.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699439+0000", 2024-09-14T09:06:15.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049083904999999997 2024-09-14T09:06:15.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:15.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699467+0000", 2024-09-14T09:06:15.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8198999999999999e-05 2024-09-14T09:06:15.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:15.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699552+0000", 2024-09-14T09:06:15.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T09:06:15.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699624+0000", 2024-09-14T09:06:15.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1725999999999997e-05 2024-09-14T09:06:15.727 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.727 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.727 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.727 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:06:15.727 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:15.727 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.508154674, 2024-09-14T09:06:15.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:06:15.727 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.727 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.727 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:15.727 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.727 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.727 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.728 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:06:15.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.728 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:15.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:15.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:06:15.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:06:15.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:06:15.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:06:15.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:06:15.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:06:15.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:06:15.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:06:15.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:06:15.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:06:15.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:06:15.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:06:15.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:06:15.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:06:15.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:06:15.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:15.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:06:15.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:06:15.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:15.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:06:15.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:06:15.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:15.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:06:15.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:06:15.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:06:15.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:06:15.732 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.732 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.732 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.732 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:39401 3.b e56/53)", 2024-09-14T09:06:15.733 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:15.733 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.4822586080000004, 2024-09-14T09:06:15.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10443659600000001, 2024-09-14T09:06:15.733 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.733 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.733 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.733 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:15.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:15.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186509+0000", 2024-09-14T09:06:15.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2640000000000001e-06 2024-09-14T09:06:15.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186516+0000", 2024-09-14T09:06:15.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1070000000000001e-06 2024-09-14T09:06:15.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186533+0000", 2024-09-14T09:06:15.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7818000000000001e-05 2024-09-14T09:06:15.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186543+0000", 2024-09-14T09:06:15.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5170000000000005e-06 2024-09-14T09:06:15.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290885+0000", 2024-09-14T09:06:15.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.104342221 2024-09-14T09:06:15.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290893+0000", 2024-09-14T09:06:15.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2859999999999999e-06 2024-09-14T09:06:15.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290945+0000", 2024-09-14T09:06:15.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1382999999999999e-05 2024-09-14T09:06:15.736 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.736 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.736 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.736 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49708 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:15.736 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526878+0000", 2024-09-14T09:06:15.736 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.1418885579999998, 2024-09-14T09:06:15.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036535277999999997, 2024-09-14T09:06:15.737 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.737 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.737 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.737 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.737 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.737 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.737 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49708 2024-09-14T09:06:15.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.737 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526878+0000", 2024-09-14T09:06:15.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526878+0000", 2024-09-14T09:06:15.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526879+0000", 2024-09-14T09:06:15.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.13e-07 2024-09-14T09:06:15.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526882+0000", 2024-09-14T09:06:15.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7480000000000002e-06 2024-09-14T09:06:15.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526889+0000", 2024-09-14T09:06:15.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4270000000000002e-06 2024-09-14T09:06:15.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526894+0000", 2024-09-14T09:06:15.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9599999999999999e-06 2024-09-14T09:06:15.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563237+0000", 2024-09-14T09:06:15.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036342646999999999 2024-09-14T09:06:15.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563314+0000", 2024-09-14T09:06:15.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7511999999999999e-05 2024-09-14T09:06:15.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563413+0000", 2024-09-14T09:06:15.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9370999999999994e-05 2024-09-14T09:06:15.740 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.740 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.740 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.740 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49709 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:15.740 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526902+0000", 2024-09-14T09:06:15.740 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.1418642999999999, 2024-09-14T09:06:15.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036648100000000003, 2024-09-14T09:06:15.741 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.741 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.741 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.741 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.741 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.741 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.741 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49709 2024-09-14T09:06:15.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.741 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526902+0000", 2024-09-14T09:06:15.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526902+0000", 2024-09-14T09:06:15.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526903+0000", 2024-09-14T09:06:15.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3500000000000002e-07 2024-09-14T09:06:15.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526908+0000", 2024-09-14T09:06:15.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4179999999999996e-06 2024-09-14T09:06:15.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526914+0000", 2024-09-14T09:06:15.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.981e-06 2024-09-14T09:06:15.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526918+0000", 2024-09-14T09:06:15.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0770000000000001e-06 2024-09-14T09:06:15.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563445+0000", 2024-09-14T09:06:15.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036526378999999998 2024-09-14T09:06:15.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563498+0000", 2024-09-14T09:06:15.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3190999999999999e-05 2024-09-14T09:06:15.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563551+0000", 2024-09-14T09:06:15.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2719000000000001e-05 2024-09-14T09:06:15.744 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.745 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.745 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.745 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49710 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:15.745 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526924+0000", 2024-09-14T09:06:15.745 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.1418424460000001, 2024-09-14T09:06:15.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037006095000000003, 2024-09-14T09:06:15.745 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.745 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.745 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.745 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.746 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.746 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.746 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49710 2024-09-14T09:06:15.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.746 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526924+0000", 2024-09-14T09:06:15.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526924+0000", 2024-09-14T09:06:15.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526925+0000", 2024-09-14T09:06:15.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3599999999999999e-07 2024-09-14T09:06:15.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526926+0000", 2024-09-14T09:06:15.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.795e-06 2024-09-14T09:06:15.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526932+0000", 2024-09-14T09:06:15.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0839999999999999e-06 2024-09-14T09:06:15.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526936+0000", 2024-09-14T09:06:15.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3440000000000003e-06 2024-09-14T09:06:15.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563593+0000", 2024-09-14T09:06:15.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036657041000000001 2024-09-14T09:06:15.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563648+0000", 2024-09-14T09:06:15.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5152999999999999e-05 2024-09-14T09:06:15.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563930+0000", 2024-09-14T09:06:15.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028234199999999999 2024-09-14T09:06:15.749 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.749 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.750 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.750 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49711 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:15.750 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:15.750 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.141824927, 2024-09-14T09:06:15.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037330945999999997, 2024-09-14T09:06:15.750 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.750 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.750 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.750 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.750 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.750 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.751 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49711 2024-09-14T09:06:15.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.751 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:15.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:15.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:15.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1800000000000002e-07 2024-09-14T09:06:15.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526944+0000", 2024-09-14T09:06:15.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.821e-06 2024-09-14T09:06:15.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526950+0000", 2024-09-14T09:06:15.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5600000000000001e-06 2024-09-14T09:06:15.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526954+0000", 2024-09-14T09:06:15.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.437e-06 2024-09-14T09:06:15.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563965+0000", 2024-09-14T09:06:15.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037010534999999997 2024-09-14T09:06:15.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564021+0000", 2024-09-14T09:06:15.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6728000000000003e-05 2024-09-14T09:06:15.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564273+0000", 2024-09-14T09:06:15.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025154700000000002 2024-09-14T09:06:15.754 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.754 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.754 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.754 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49712 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 3112960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:15.755 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:15.755 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.1418067189999999, 2024-09-14T09:06:15.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037795625999999999, 2024-09-14T09:06:15.755 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.756 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.756 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.756 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.756 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.756 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.756 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49712 2024-09-14T09:06:15.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.756 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:15.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:15.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:15.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4799999999999999e-07 2024-09-14T09:06:15.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526962+0000", 2024-09-14T09:06:15.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8449999999999999e-06 2024-09-14T09:06:15.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526967+0000", 2024-09-14T09:06:15.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1059999999999999e-06 2024-09-14T09:06:15.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526971+0000", 2024-09-14T09:06:15.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8310000000000003e-06 2024-09-14T09:06:15.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564324+0000", 2024-09-14T09:06:15.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037352926000000002 2024-09-14T09:06:15.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564411+0000", 2024-09-14T09:06:15.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7052000000000006e-05 2024-09-14T09:06:15.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564756+0000", 2024-09-14T09:06:15.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034451799999999998 2024-09-14T09:06:15.759 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.759 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.759 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.759 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49713 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 3506176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:15.759 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:15.760 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.1417900469999998, 2024-09-14T09:06:15.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038135466, 2024-09-14T09:06:15.760 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.760 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.760 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.760 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.760 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.760 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.760 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49713 2024-09-14T09:06:15.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.760 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:15.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:15.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:15.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1899999999999998e-07 2024-09-14T09:06:15.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526979+0000", 2024-09-14T09:06:15.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8959999999999999e-06 2024-09-14T09:06:15.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526986+0000", 2024-09-14T09:06:15.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1720000000000001e-06 2024-09-14T09:06:15.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526991+0000", 2024-09-14T09:06:15.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.729e-06 2024-09-14T09:06:15.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564781+0000", 2024-09-14T09:06:15.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037790345000000003 2024-09-14T09:06:15.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564844+0000", 2024-09-14T09:06:15.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2656000000000005e-05 2024-09-14T09:06:15.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565112+0000", 2024-09-14T09:06:15.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026834899999999997 2024-09-14T09:06:15.763 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.763 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.763 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.764 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49714 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:15.764 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526997+0000", 2024-09-14T09:06:15.764 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.1417694379999999, 2024-09-14T09:06:15.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038531469999999998, 2024-09-14T09:06:15.764 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:15.764 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:15.764 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:15.764 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:15.764 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:15.764 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:15.764 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49714 2024-09-14T09:06:15.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.765 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:15.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:15.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526997+0000", 2024-09-14T09:06:15.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:15.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526997+0000", 2024-09-14T09:06:15.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:15.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:15.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526998+0000", 2024-09-14T09:06:15.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7800000000000002e-07 2024-09-14T09:06:15.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:15.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.527000+0000", 2024-09-14T09:06:15.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9080000000000002e-06 2024-09-14T09:06:15.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:15.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.527007+0000", 2024-09-14T09:06:15.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1609999999999997e-06 2024-09-14T09:06:15.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:15.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.527011+0000", 2024-09-14T09:06:15.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0929999999999996e-06 2024-09-14T09:06:15.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:15.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565152+0000", 2024-09-14T09:06:15.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038141520999999998 2024-09-14T09:06:15.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:15.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565236+0000", 2024-09-14T09:06:15.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3498999999999995e-05 2024-09-14T09:06:15.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:15.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:15.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:15.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565529+0000", 2024-09-14T09:06:15.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029291000000000001 2024-09-14T09:06:15.767 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.768 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.768 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.768 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:15.768 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:15.768 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:15.769 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:06:15.770 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:15.771+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:15.870 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:15.871+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:15.901 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:15.901 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:15.901 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:15.902 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:15.915 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:06:15.972 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:15.971+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:16.020 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:06:16.020 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:06:16.020 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-14T09:06:16.020 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-14T09:06:16.073 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:16.071+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:16.089 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:16.089 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:16.089 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:16.089 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:16.089 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:16.103 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:06:16.156 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T09:06:16.202 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-14T09:06:16.202 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:16.199+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:16.230 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:654 3.11 e29/23 3:8c8e358b:::rbd_object_map.115e1e95bf99:head v 29'3, mlcod=28'1)", 2024-09-14T09:06:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.863095+0000", 2024-09-14T09:06:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 42.367032211000001, 2024-09-14T09:06:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0070911430000000003, 2024-09-14T09:06:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863095+0000", 2024-09-14T09:06:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863095+0000", 2024-09-14T09:06:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863096+0000", 2024-09-14T09:06:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8700000000000004e-07 2024-09-14T09:06:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863110+0000", 2024-09-14T09:06:16.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4283e-05 2024-09-14T09:06:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863157+0000", 2024-09-14T09:06:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6029999999999998e-05 2024-09-14T09:06:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863177+0000", 2024-09-14T09:06:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0662999999999999e-05 2024-09-14T09:06:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863324+0000", 2024-09-14T09:06:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014713899999999999 2024-09-14T09:06:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.863383+0000", 2024-09-14T09:06:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8218999999999999e-05 2024-09-14T09:06:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.870083+0000", 2024-09-14T09:06:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067001459999999997 2024-09-14T09:06:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.870186+0000", 2024-09-14T09:06:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103776 2024-09-14T09:06:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:758 3.14 3:2d2c8a97:::rbd_data.115e724e1d99.0000000000000007:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:06:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.888858+0000", 2024-09-14T09:06:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 42.341269230000002, 2024-09-14T09:06:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060111740000000002, 2024-09-14T09:06:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.235 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 758 2024-09-14T09:06:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888858+0000", 2024-09-14T09:06:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888858+0000", 2024-09-14T09:06:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888860+0000", 2024-09-14T09:06:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4529999999999999e-06 2024-09-14T09:06:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888867+0000", 2024-09-14T09:06:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7200000000000006e-06 2024-09-14T09:06:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888892+0000", 2024-09-14T09:06:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4729999999999999e-05 2024-09-14T09:06:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.888927+0000", 2024-09-14T09:06:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4752999999999997e-05 2024-09-14T09:06:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.894354+0000", 2024-09-14T09:06:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054273209999999997 2024-09-14T09:06:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.894584+0000", 2024-09-14T09:06:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000229602 2024-09-14T09:06:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.894869+0000", 2024-09-14T09:06:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028559499999999998 2024-09-14T09:06:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:829 3.13 e29/23 3:cc36a41f:::rbd_data.115e1e95bf99.0000000000000000:head v 29'1, mlcod=0'0)", 2024-09-14T09:06:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.901655+0000", 2024-09-14T09:06:16.239 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 42.328472343999998, 2024-09-14T09:06:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021644360000000001, 2024-09-14T09:06:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901655+0000", 2024-09-14T09:06:16.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901655+0000", 2024-09-14T09:06:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901657+0000", 2024-09-14T09:06:16.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.153e-06 2024-09-14T09:06:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901897+0000", 2024-09-14T09:06:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000239461 2024-09-14T09:06:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901927+0000", 2024-09-14T09:06:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.029e-05 2024-09-14T09:06:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.901946+0000", 2024-09-14T09:06:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9170000000000001e-05 2024-09-14T09:06:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902104+0000", 2024-09-14T09:06:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015812199999999999 2024-09-14T09:06:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.902157+0000", 2024-09-14T09:06:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2905999999999998e-05 2024-09-14T09:06:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923226+0000", 2024-09-14T09:06:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.021068470999999998 2024-09-14T09:06:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923300+0000", 2024-09-14T09:06:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3787000000000004e-05 2024-09-14T09:06:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.245 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:845 3.9 3:928dad78:::rbd_data.115e1e95bf99.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13369344~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:06:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.922336+0000", 2024-09-14T09:06:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 42.307791426999998, 2024-09-14T09:06:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010311885999999999, 2024-09-14T09:06:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.246 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 845 2024-09-14T09:06:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922336+0000", 2024-09-14T09:06:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922336+0000", 2024-09-14T09:06:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922339+0000", 2024-09-14T09:06:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6879999999999999e-06 2024-09-14T09:06:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922350+0000", 2024-09-14T09:06:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1457999999999999e-05 2024-09-14T09:06:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922376+0000", 2024-09-14T09:06:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5301999999999999e-05 2024-09-14T09:06:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922390+0000", 2024-09-14T09:06:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4404e-05 2024-09-14T09:06:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922506+0000", 2024-09-14T09:06:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011567599999999999 2024-09-14T09:06:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922741+0000", 2024-09-14T09:06:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00023530600000000001 2024-09-14T09:06:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-09-14T09:06:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923197+0000", 2024-09-14T09:06:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00045574899999999998 2024-09-14T09:06:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926027+0000", 2024-09-14T09:06:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002830819 2024-09-14T09:06:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932530+0000", 2024-09-14T09:06:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065027139999999997 2024-09-14T09:06:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932583+0000", 2024-09-14T09:06:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2342000000000001e-05 2024-09-14T09:06:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932648+0000", 2024-09-14T09:06:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5427999999999998e-05 2024-09-14T09:06:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:842 3.b e29/28 3:d50f9298:::rbd_data.115e1e95bf99.0000000000000004:head v 29'5, mlcod=26'3)", 2024-09-14T09:06:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.922344+0000", 2024-09-14T09:06:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 42.307783677000003, 2024-09-14T09:06:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062661460000000002, 2024-09-14T09:06:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922344+0000", 2024-09-14T09:06:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922344+0000", 2024-09-14T09:06:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922345+0000", 2024-09-14T09:06:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.288e-06 2024-09-14T09:06:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922356+0000", 2024-09-14T09:06:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1048e-05 2024-09-14T09:06:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922371+0000", 2024-09-14T09:06:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4338000000000001e-05 2024-09-14T09:06:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922382+0000", 2024-09-14T09:06:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1e-05 2024-09-14T09:06:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922467+0000", 2024-09-14T09:06:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5721000000000001e-05 2024-09-14T09:06:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922489+0000", 2024-09-14T09:06:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2181999999999999e-05 2024-09-14T09:06:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.928551+0000", 2024-09-14T09:06:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060616100000000003 2024-09-14T09:06:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.928610+0000", 2024-09-14T09:06:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8959000000000003e-05 2024-09-14T09:06:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:848 3.1a 3:5ade8e73:::rbd_data.115e1e95bf99.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13369344~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T09:06:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.922902+0000", 2024-09-14T09:06:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 42.307226014999998, 2024-09-14T09:06:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096031020000000005, 2024-09-14T09:06:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 848 2024-09-14T09:06:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922902+0000", 2024-09-14T09:06:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922902+0000", 2024-09-14T09:06:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.922903+0000", 2024-09-14T09:06:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6419999999999999e-06 2024-09-14T09:06:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923127+0000", 2024-09-14T09:06:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000223997 2024-09-14T09:06:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923151+0000", 2024-09-14T09:06:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3802e-05 2024-09-14T09:06:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923169+0000", 2024-09-14T09:06:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7858000000000001e-05 2024-09-14T09:06:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923307+0000", 2024-09-14T09:06:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013802399999999999 2024-09-14T09:06:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923560+0000", 2024-09-14T09:06:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00025296500000000001 2024-09-14T09:06:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.923925+0000", 2024-09-14T09:06:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036534700000000002 2024-09-14T09:06:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.927231+0000", 2024-09-14T09:06:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033055820000000001 2024-09-14T09:06:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932287+0000", 2024-09-14T09:06:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0050562810000000001 2024-09-14T09:06:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932395+0000", 2024-09-14T09:06:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000108001 2024-09-14T09:06:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932505+0000", 2024-09-14T09:06:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109603 2024-09-14T09:06:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:847 3.13 e29/23 3:cc047b4a:::rbd_data.115e1e95bf99.0000000000000009:head v 29'2, mlcod=0'0)", 2024-09-14T09:06:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.924087+0000", 2024-09-14T09:06:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 42.306040252999999, 2024-09-14T09:06:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0085507929999999992, 2024-09-14T09:06:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924087+0000", 2024-09-14T09:06:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924087+0000", 2024-09-14T09:06:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924089+0000", 2024-09-14T09:06:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8050000000000001e-06 2024-09-14T09:06:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924157+0000", 2024-09-14T09:06:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7999999999999999e-05 2024-09-14T09:06:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924176+0000", 2024-09-14T09:06:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9312000000000001e-05 2024-09-14T09:06:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924191+0000", 2024-09-14T09:06:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4707e-05 2024-09-14T09:06:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924292+0000", 2024-09-14T09:06:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010052000000000001 2024-09-14T09:06:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924328+0000", 2024-09-14T09:06:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6211999999999998e-05 2024-09-14T09:06:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932564+0000", 2024-09-14T09:06:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0082365960000000005 2024-09-14T09:06:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932638+0000", 2024-09-14T09:06:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3640999999999995e-05 2024-09-14T09:06:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:849 3.b e29/28 3:d70d4fef:::rbd_data.115e1e95bf99.000000000000000b:head v 29'6, mlcod=26'3)", 2024-09-14T09:06:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.924304+0000", 2024-09-14T09:06:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 42.305823294, 2024-09-14T09:06:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083878719999999993, 2024-09-14T09:06:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924304+0000", 2024-09-14T09:06:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924304+0000", 2024-09-14T09:06:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924305+0000", 2024-09-14T09:06:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.192e-06 2024-09-14T09:06:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924676+0000", 2024-09-14T09:06:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037050700000000001 2024-09-14T09:06:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924698+0000", 2024-09-14T09:06:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1977999999999999e-05 2024-09-14T09:06:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.924712+0000", 2024-09-14T09:06:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4477e-05 2024-09-14T09:06:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926069+0000", 2024-09-14T09:06:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013562330000000001 2024-09-14T09:06:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926105+0000", 2024-09-14T09:06:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6779999999999997e-05 2024-09-14T09:06:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932643+0000", 2024-09-14T09:06:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065378290000000002 2024-09-14T09:06:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932692+0000", 2024-09-14T09:06:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8875999999999998e-05 2024-09-14T09:06:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:850 3.11 e29/23 3:892642aa:::rbd_data.115e1e95bf99.000000000000000c:head v 29'18, mlcod=29'15)", 2024-09-14T09:06:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:33.926814+0000", 2024-09-14T09:06:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 42.303313420999999, 2024-09-14T09:06:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0064716569999999996, 2024-09-14T09:06:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926814+0000", 2024-09-14T09:06:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926814+0000", 2024-09-14T09:06:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926816+0000", 2024-09-14T09:06:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.491e-06 2024-09-14T09:06:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926828+0000", 2024-09-14T09:06:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2288e-05 2024-09-14T09:06:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926851+0000", 2024-09-14T09:06:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2674999999999999e-05 2024-09-14T09:06:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.926865+0000", 2024-09-14T09:06:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.449e-05 2024-09-14T09:06:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932217+0000", 2024-09-14T09:06:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053523370000000004 2024-09-14T09:06:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.932241+0000", 2024-09-14T09:06:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3210999999999999e-05 2024-09-14T09:06:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.933239+0000", 2024-09-14T09:06:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00099850599999999992 2024-09-14T09:06:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:33.933286+0000", 2024-09-14T09:06:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6659000000000002e-05 2024-09-14T09:06:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:9982 3.1a 3:5af3e476:::rbd_data.115eadec874.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:06:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.632158101000002, 2024-09-14T09:06:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042276341000000002, 2024-09-14T09:06:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 9982 2024-09-14T09:06:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597970+0000", 2024-09-14T09:06:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0599999999999999e-07 2024-09-14T09:06:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597974+0000", 2024-09-14T09:06:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2649999999999998e-06 2024-09-14T09:06:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597986+0000", 2024-09-14T09:06:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1019000000000001e-05 2024-09-14T09:06:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597991+0000", 2024-09-14T09:06:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5049999999999998e-06 2024-09-14T09:06:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.598083+0000", 2024-09-14T09:06:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2062e-05 2024-09-14T09:06:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.598227+0000", 2024-09-14T09:06:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000143609 2024-09-14T09:06:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.603107+0000", 2024-09-14T09:06:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048805819999999996 2024-09-14T09:06:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640002+0000", 2024-09-14T09:06:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.036894641999999998 2024-09-14T09:06:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640077+0000", 2024-09-14T09:06:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4927999999999998e-05 2024-09-14T09:06:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640174+0000", 2024-09-14T09:06:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7417999999999995e-05 2024-09-14T09:06:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640246+0000", 2024-09-14T09:06:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1505000000000001e-05 2024-09-14T09:06:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:9982 3.1a e32/23 ondisk, result = 0)", 2024-09-14T09:06:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.609682+0000", 2024-09-14T09:06:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.620445492000002, 2024-09-14T09:06:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.030575755, 2024-09-14T09:06:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609682+0000", 2024-09-14T09:06:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609682+0000", 2024-09-14T09:06:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609683+0000", 2024-09-14T09:06:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0399999999999995e-07 2024-09-14T09:06:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609687+0000", 2024-09-14T09:06:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3460000000000004e-06 2024-09-14T09:06:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609696+0000", 2024-09-14T09:06:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0550000000000005e-06 2024-09-14T09:06:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.609702+0000", 2024-09-14T09:06:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.626e-06 2024-09-14T09:06:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640050+0000", 2024-09-14T09:06:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.030348671000000001 2024-09-14T09:06:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640066+0000", 2024-09-14T09:06:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5617999999999999e-05 2024-09-14T09:06:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640258+0000", 2024-09-14T09:06:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019173500000000001 2024-09-14T09:06:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:10031 3.1b 3:df00a663:::rbd_data.115eadec874.0000000000000005:head [stat,write 16619424~53229] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:06:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.768778+0000", 2024-09-14T09:06:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.461349980999998, 2024-09-14T09:06:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0078922009999999997, 2024-09-14T09:06:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 10031 2024-09-14T09:06:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768778+0000", 2024-09-14T09:06:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768778+0000", 2024-09-14T09:06:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768780+0000", 2024-09-14T09:06:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7130000000000001e-06 2024-09-14T09:06:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768824+0000", 2024-09-14T09:06:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3989999999999997e-05 2024-09-14T09:06:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768854+0000", 2024-09-14T09:06:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9937000000000002e-05 2024-09-14T09:06:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768869+0000", 2024-09-14T09:06:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5233e-05 2024-09-14T09:06:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.768979+0000", 2024-09-14T09:06:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109429 2024-09-14T09:06:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.769124+0000", 2024-09-14T09:06:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014520200000000001 2024-09-14T09:06:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.769351+0000", 2024-09-14T09:06:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000227259 2024-09-14T09:06:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.776133+0000", 2024-09-14T09:06:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067818979999999997 2024-09-14T09:06:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.776464+0000", 2024-09-14T09:06:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033063400000000001 2024-09-14T09:06:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.776553+0000", 2024-09-14T09:06:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8793000000000002e-05 2024-09-14T09:06:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.776670+0000", 2024-09-14T09:06:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000117113 2024-09-14T09:06:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:10039 3.f e33/23 3:f6f4e415:::rbd_data.115eadec874.0000000000000000:head v 33'8, mlcod=32'6)", 2024-09-14T09:06:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.951357+0000", 2024-09-14T09:06:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.278770839000003, 2024-09-14T09:06:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067664509999999997, 2024-09-14T09:06:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951357+0000", 2024-09-14T09:06:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951357+0000", 2024-09-14T09:06:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951357+0000", 2024-09-14T09:06:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4000000000000001e-07 2024-09-14T09:06:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951363+0000", 2024-09-14T09:06:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4999999999999999e-06 2024-09-14T09:06:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951376+0000", 2024-09-14T09:06:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3315e-05 2024-09-14T09:06:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951384+0000", 2024-09-14T09:06:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6590000000000007e-06 2024-09-14T09:06:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951470+0000", 2024-09-14T09:06:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6198000000000004e-05 2024-09-14T09:06:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951493+0000", 2024-09-14T09:06:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2668999999999999e-05 2024-09-14T09:06:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.958059+0000", 2024-09-14T09:06:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065659450000000001 2024-09-14T09:06:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.958123+0000", 2024-09-14T09:06:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4524999999999996e-05 2024-09-14T09:06:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:10040 3.a e33/23 3:50279329:::rbd_data.115eadec874.0000000000000001:head v 33'16, mlcod=32'14)", 2024-09-14T09:06:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.951570+0000", 2024-09-14T09:06:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.278557524, 2024-09-14T09:06:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0094331320000000003, 2024-09-14T09:06:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951570+0000", 2024-09-14T09:06:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951570+0000", 2024-09-14T09:06:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951571+0000", 2024-09-14T09:06:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0800000000000004e-07 2024-09-14T09:06:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951578+0000", 2024-09-14T09:06:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4919999999999996e-06 2024-09-14T09:06:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951590+0000", 2024-09-14T09:06:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1674e-05 2024-09-14T09:06:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951598+0000", 2024-09-14T09:06:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4649999999999994e-06 2024-09-14T09:06:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951800+0000", 2024-09-14T09:06:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020128600000000001 2024-09-14T09:06:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.951832+0000", 2024-09-14T09:06:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2512999999999999e-05 2024-09-14T09:06:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.960831+0000", 2024-09-14T09:06:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00899872 2024-09-14T09:06:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.961003+0000", 2024-09-14T09:06:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017217400000000001 2024-09-14T09:06:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:10121 3.a e33/23 3:50279329:::rbd_data.115eadec874.0000000000000001:head v 33'18, mlcod=33'16)", 2024-09-14T09:06:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:38.059543+0000", 2024-09-14T09:06:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.170584409999996, 2024-09-14T09:06:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063099250000000001, 2024-09-14T09:06:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059543+0000", 2024-09-14T09:06:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059543+0000", 2024-09-14T09:06:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059545+0000", 2024-09-14T09:06:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.863e-06 2024-09-14T09:06:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059609+0000", 2024-09-14T09:06:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3758999999999997e-05 2024-09-14T09:06:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059633+0000", 2024-09-14T09:06:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4000000000000001e-05 2024-09-14T09:06:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059650+0000", 2024-09-14T09:06:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7130999999999999e-05 2024-09-14T09:06:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059779+0000", 2024-09-14T09:06:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000129152 2024-09-14T09:06:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.059840+0000", 2024-09-14T09:06:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0930000000000001e-05 2024-09-14T09:06:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.065745+0000", 2024-09-14T09:06:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059050309999999998 2024-09-14T09:06:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.065853+0000", 2024-09-14T09:06:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000108059 2024-09-14T09:06:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:10182 3.1a 3:5a62ac37:::rbd_data.115eadec874.000000000000000d:head [set-alloc-hint object_size 33554432 write_size 33554432,write 5072829~48248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-14T09:06:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:38.177016+0000", 2024-09-14T09:06:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.053111475000001, 2024-09-14T09:06:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006987147, 2024-09-14T09:06:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 10182 2024-09-14T09:06:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177016+0000", 2024-09-14T09:06:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177016+0000", 2024-09-14T09:06:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177017+0000", 2024-09-14T09:06:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0440000000000001e-06 2024-09-14T09:06:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177065+0000", 2024-09-14T09:06:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7902999999999999e-05 2024-09-14T09:06:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177080+0000", 2024-09-14T09:06:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4667999999999999e-05 2024-09-14T09:06:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177087+0000", 2024-09-14T09:06:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0430000000000002e-06 2024-09-14T09:06:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177140+0000", 2024-09-14T09:06:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2933999999999998e-05 2024-09-14T09:06:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177199+0000", 2024-09-14T09:06:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9799999999999997e-05 2024-09-14T09:06:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.177309+0000", 2024-09-14T09:06:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010948200000000001 2024-09-14T09:06:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.180906+0000", 2024-09-14T09:06:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0035966230000000002 2024-09-14T09:06:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.183932+0000", 2024-09-14T09:06:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030259760000000001 2024-09-14T09:06:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.183973+0000", 2024-09-14T09:06:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1189000000000002e-05 2024-09-14T09:06:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:38.184003+0000", 2024-09-14T09:06:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0485e-05 2024-09-14T09:06:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:20954 3.1d e41/23 3:bd7e9002:::rbd_data.115e889d4809.0000000000000005:head v 41'131, mlcod=41'127)", 2024-09-14T09:06:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:47.701009+0000", 2024-09-14T09:06:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 28.529118115999999, 2024-09-14T09:06:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0080632259999999997, 2024-09-14T09:06:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701009+0000", 2024-09-14T09:06:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701009+0000", 2024-09-14T09:06:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701011+0000", 2024-09-14T09:06:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.703e-06 2024-09-14T09:06:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701234+0000", 2024-09-14T09:06:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022278900000000001 2024-09-14T09:06:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701265+0000", 2024-09-14T09:06:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1386000000000001e-05 2024-09-14T09:06:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701281+0000", 2024-09-14T09:06:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5460000000000001e-05 2024-09-14T09:06:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701391+0000", 2024-09-14T09:06:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000110622 2024-09-14T09:06:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.701428+0000", 2024-09-14T09:06:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6986999999999997e-05 2024-09-14T09:06:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.708992+0000", 2024-09-14T09:06:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075640769999999998 2024-09-14T09:06:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:47.709073+0000", 2024-09-14T09:06:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0202000000000002e-05 2024-09-14T09:06:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:21183 3.1d e42/23 3:bba2b35c:::rbd_object_map.115e889d4809:head v 42'156, mlcod=42'149)", 2024-09-14T09:06:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:48.419596+0000", 2024-09-14T09:06:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 27.810531841, 2024-09-14T09:06:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063479840000000001, 2024-09-14T09:06:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419596+0000", 2024-09-14T09:06:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419596+0000", 2024-09-14T09:06:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419598+0000", 2024-09-14T09:06:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8560000000000001e-06 2024-09-14T09:06:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419610+0000", 2024-09-14T09:06:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2577e-05 2024-09-14T09:06:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419645+0000", 2024-09-14T09:06:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4873000000000003e-05 2024-09-14T09:06:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419664+0000", 2024-09-14T09:06:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9485999999999999e-05 2024-09-14T09:06:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419799+0000", 2024-09-14T09:06:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013482300000000001 2024-09-14T09:06:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.419853+0000", 2024-09-14T09:06:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3439000000000001e-05 2024-09-14T09:06:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.425843+0000", 2024-09-14T09:06:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059898809999999999 2024-09-14T09:06:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:48.425944+0000", 2024-09-14T09:06:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101049 2024-09-14T09:06:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(3.1b 53/52 [PushOp(3:df00a663:::rbd_data.115eadec874.0000000000000005:6, version: 35'41, data_included: [1796079~63600], data_size: 63600, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:df00a663:::rbd_data.115eadec874.0000000000000005:6@35'41, size: 1859679, copy_subset: [(0, 1859679)], clone_subset: {}, snapset: 6=[]:{6: [6]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1859679, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T09:06:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.846807+0000", 2024-09-14T09:06:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.383320863, 2024-09-14T09:06:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0089637359999999999, 2024-09-14T09:06:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.846807+0000", 2024-09-14T09:06:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.846807+0000", 2024-09-14T09:06:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.846809+0000", 2024-09-14T09:06:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5799999999999999e-06 2024-09-14T09:06:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.846948+0000", 2024-09-14T09:06:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013911500000000001 2024-09-14T09:06:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.847007+0000", 2024-09-14T09:06:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8437000000000003e-05 2024-09-14T09:06:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.847054+0000", 2024-09-14T09:06:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7372e-05 2024-09-14T09:06:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.847295+0000", 2024-09-14T09:06:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000241022 2024-09-14T09:06:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.847319+0000", 2024-09-14T09:06:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4303000000000001e-05 2024-09-14T09:06:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.855770+0000", 2024-09-14T09:06:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084509070000000006 2024-09-14T09:06:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(3.1b 53/52 [PushOp(3:d8e0f8d4:::rbd_data.115eb07a8abd.0000000000000001:head, version: 48'138, data_included: [0~393216,786432~131072], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:d8e0f8d4:::rbd_data.115eb07a8abd.0000000000000001:head@48'138, size: 7208960, copy_subset: [(0, 7208960)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 7208960, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T09:06:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.943323+0000", 2024-09-14T09:06:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.286804847999999, 2024-09-14T09:06:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0081121930000000002, 2024-09-14T09:06:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.943323+0000", 2024-09-14T09:06:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.943323+0000", 2024-09-14T09:06:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.943324+0000", 2024-09-14T09:06:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.206e-06 2024-09-14T09:06:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944255+0000", 2024-09-14T09:06:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093096900000000002 2024-09-14T09:06:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944299+0000", 2024-09-14T09:06:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4332999999999997e-05 2024-09-14T09:06:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944332+0000", 2024-09-14T09:06:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2329999999999997e-05 2024-09-14T09:06:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944590+0000", 2024-09-14T09:06:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00025868100000000003 2024-09-14T09:06:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.944629+0000", 2024-09-14T09:06:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8503e-05 2024-09-14T09:06:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.951435+0000", 2024-09-14T09:06:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068061709999999997 2024-09-14T09:06:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.336 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.336 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.336 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.336 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:16.337 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone19 from 0x950035e (overlap 0x3ff2b82) to 0x2cc380e 2024-09-14T09:06:16.338 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:06:16.338 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:16.339+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:16.438 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:16.435+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:16.478 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-14T09:06:16.524 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:16.524 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:16.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.524 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53308 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.525 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.505238+0000", 2024-09-14T09:06:16.525 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.014813593999999999, 2024-09-14T09:06:16.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014861565, 2024-09-14T09:06:16.525 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.525 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.525 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.525 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.525 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.525 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.526 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53308 2024-09-14T09:06:16.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.526 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505238+0000", 2024-09-14T09:06:16.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505238+0000", 2024-09-14T09:06:16.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505239+0000", 2024-09-14T09:06:16.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.514e-06 2024-09-14T09:06:16.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505245+0000", 2024-09-14T09:06:16.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4199999999999998e-06 2024-09-14T09:06:16.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505262+0000", 2024-09-14T09:06:16.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7689999999999998e-05 2024-09-14T09:06:16.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505277+0000", 2024-09-14T09:06:16.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4293e-05 2024-09-14T09:06:16.528 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.528 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.528 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.528 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53535 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.528 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509100+0000", 2024-09-14T09:06:16.528 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.010951442, 2024-09-14T09:06:16.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011160655, 2024-09-14T09:06:16.528 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.529 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.529 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.529 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.529 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.529 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.529 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53535 2024-09-14T09:06:16.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.529 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509100+0000", 2024-09-14T09:06:16.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509100+0000", 2024-09-14T09:06:16.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509101+0000", 2024-09-14T09:06:16.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7e-06 2024-09-14T09:06:16.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509109+0000", 2024-09-14T09:06:16.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0299999999999996e-06 2024-09-14T09:06:16.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509131+0000", 2024-09-14T09:06:16.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2935999999999999e-05 2024-09-14T09:06:16.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509147+0000", 2024-09-14T09:06:16.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6047000000000001e-05 2024-09-14T09:06:16.531 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.531 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.531 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.531 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53767 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.532 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.518751+0000", 2024-09-14T09:06:16.532 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.00130042, 2024-09-14T09:06:16.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016395229999999999, 2024-09-14T09:06:16.532 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.532 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.532 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.532 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.532 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.532 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.532 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53767 2024-09-14T09:06:16.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.532 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518751+0000", 2024-09-14T09:06:16.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518751+0000", 2024-09-14T09:06:16.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518752+0000", 2024-09-14T09:06:16.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3060000000000001e-06 2024-09-14T09:06:16.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518759+0000", 2024-09-14T09:06:16.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.404e-06 2024-09-14T09:06:16.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518777+0000", 2024-09-14T09:06:16.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.857e-05 2024-09-14T09:06:16.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518791+0000", 2024-09-14T09:06:16.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4100000000000001e-05 2024-09-14T09:06:16.535 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.535 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.535 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.535 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53309 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.535 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.505295+0000", 2024-09-14T09:06:16.535 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.014756174, 2024-09-14T09:06:16.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015222036, 2024-09-14T09:06:16.535 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.535 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.535 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.535 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.535 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.536 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.536 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53309 2024-09-14T09:06:16.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.536 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505295+0000", 2024-09-14T09:06:16.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505295+0000", 2024-09-14T09:06:16.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505296+0000", 2024-09-14T09:06:16.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.207e-06 2024-09-14T09:06:16.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505301+0000", 2024-09-14T09:06:16.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1259999999999997e-06 2024-09-14T09:06:16.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505319+0000", 2024-09-14T09:06:16.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7493e-05 2024-09-14T09:06:16.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505332+0000", 2024-09-14T09:06:16.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3447e-05 2024-09-14T09:06:16.538 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.538 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.538 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.538 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53537 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.538 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509169+0000", 2024-09-14T09:06:16.538 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.010882182000000001, 2024-09-14T09:06:16.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011475077, 2024-09-14T09:06:16.539 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.539 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.539 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.539 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.539 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.539 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.539 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53537 2024-09-14T09:06:16.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.539 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509169+0000", 2024-09-14T09:06:16.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509169+0000", 2024-09-14T09:06:16.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509170+0000", 2024-09-14T09:06:16.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4559999999999999e-06 2024-09-14T09:06:16.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509176+0000", 2024-09-14T09:06:16.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3299999999999998e-06 2024-09-14T09:06:16.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509195+0000", 2024-09-14T09:06:16.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9077e-05 2024-09-14T09:06:16.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509210+0000", 2024-09-14T09:06:16.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4769999999999999e-05 2024-09-14T09:06:16.541 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.541 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.542 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.542 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53768 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.542 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.518809+0000", 2024-09-14T09:06:16.542 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.00124233, 2024-09-14T09:06:16.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019639729999999999, 2024-09-14T09:06:16.542 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.542 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.542 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.542 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.542 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.542 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.543 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53768 2024-09-14T09:06:16.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.543 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518809+0000", 2024-09-14T09:06:16.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518809+0000", 2024-09-14T09:06:16.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518810+0000", 2024-09-14T09:06:16.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1629999999999999e-06 2024-09-14T09:06:16.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518815+0000", 2024-09-14T09:06:16.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3059999999999997e-06 2024-09-14T09:06:16.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518838+0000", 2024-09-14T09:06:16.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3073000000000001e-05 2024-09-14T09:06:16.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518853+0000", 2024-09-14T09:06:16.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4090000000000001e-05 2024-09-14T09:06:16.545 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.545 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.545 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.545 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53310 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.546 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.505350+0000", 2024-09-14T09:06:16.546 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.014700848000000001, 2024-09-14T09:06:16.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015549678000000001, 2024-09-14T09:06:16.546 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.546 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.546 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.546 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.546 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.546 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.546 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53310 2024-09-14T09:06:16.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.546 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505350+0000", 2024-09-14T09:06:16.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505350+0000", 2024-09-14T09:06:16.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505352+0000", 2024-09-14T09:06:16.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3629999999999999e-06 2024-09-14T09:06:16.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505357+0000", 2024-09-14T09:06:16.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2839999999999997e-06 2024-09-14T09:06:16.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505375+0000", 2024-09-14T09:06:16.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7649999999999999e-05 2024-09-14T09:06:16.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505388+0000", 2024-09-14T09:06:16.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3223e-05 2024-09-14T09:06:16.549 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.549 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.549 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.549 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53539 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.549 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509229+0000", 2024-09-14T09:06:16.549 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.01082208, 2024-09-14T09:06:16.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011794495, 2024-09-14T09:06:16.549 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.549 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.549 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.549 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.550 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.550 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.550 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53539 2024-09-14T09:06:16.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.550 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509229+0000", 2024-09-14T09:06:16.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509229+0000", 2024-09-14T09:06:16.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509230+0000", 2024-09-14T09:06:16.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3030000000000001e-06 2024-09-14T09:06:16.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509236+0000", 2024-09-14T09:06:16.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1800000000000004e-06 2024-09-14T09:06:16.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509254+0000", 2024-09-14T09:06:16.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8057e-05 2024-09-14T09:06:16.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509268+0000", 2024-09-14T09:06:16.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3869999999999999e-05 2024-09-14T09:06:16.552 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.552 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.552 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.552 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53769 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.553 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.518871+0000", 2024-09-14T09:06:16.553 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.001179968, 2024-09-14T09:06:16.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022764619999999999, 2024-09-14T09:06:16.553 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.553 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.553 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.553 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53769 2024-09-14T09:06:16.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.553 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518871+0000", 2024-09-14T09:06:16.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518871+0000", 2024-09-14T09:06:16.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518873+0000", 2024-09-14T09:06:16.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7129999999999999e-06 2024-09-14T09:06:16.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518878+0000", 2024-09-14T09:06:16.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0000000000000004e-06 2024-09-14T09:06:16.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518899+0000", 2024-09-14T09:06:16.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0984000000000001e-05 2024-09-14T09:06:16.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518912+0000", 2024-09-14T09:06:16.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3482999999999999e-05 2024-09-14T09:06:16.555 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.555 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.555 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.556 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53312 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.556 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.505406+0000", 2024-09-14T09:06:16.556 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.014645468, 2024-09-14T09:06:16.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015872377999999999, 2024-09-14T09:06:16.556 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.556 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.556 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.556 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.556 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.556 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.556 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53312 2024-09-14T09:06:16.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.556 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505406+0000", 2024-09-14T09:06:16.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505406+0000", 2024-09-14T09:06:16.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505407+0000", 2024-09-14T09:06:16.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.24e-06 2024-09-14T09:06:16.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505412+0000", 2024-09-14T09:06:16.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.977e-06 2024-09-14T09:06:16.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505429+0000", 2024-09-14T09:06:16.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7509000000000001e-05 2024-09-14T09:06:16.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505443+0000", 2024-09-14T09:06:16.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.367e-05 2024-09-14T09:06:16.559 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.559 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.559 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.559 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53572 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.559 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509286+0000", 2024-09-14T09:06:16.559 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.01076479, 2024-09-14T09:06:16.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.012120636000000001, 2024-09-14T09:06:16.559 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.559 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.559 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.559 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.559 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.559 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.560 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53572 2024-09-14T09:06:16.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.560 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509286+0000", 2024-09-14T09:06:16.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509286+0000", 2024-09-14T09:06:16.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509288+0000", 2024-09-14T09:06:16.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2270000000000001e-06 2024-09-14T09:06:16.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509293+0000", 2024-09-14T09:06:16.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8500000000000002e-06 2024-09-14T09:06:16.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509310+0000", 2024-09-14T09:06:16.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7825999999999998e-05 2024-09-14T09:06:16.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509325+0000", 2024-09-14T09:06:16.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4623e-05 2024-09-14T09:06:16.562 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.562 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.562 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.562 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53798 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.563 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.518932+0000", 2024-09-14T09:06:16.563 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.0011196579999999999, 2024-09-14T09:06:16.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026038139999999999, 2024-09-14T09:06:16.563 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.563 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.563 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.563 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.563 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.563 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.563 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53798 2024-09-14T09:06:16.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.563 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518932+0000", 2024-09-14T09:06:16.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518932+0000", 2024-09-14T09:06:16.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518933+0000", 2024-09-14T09:06:16.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.13e-06 2024-09-14T09:06:16.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518938+0000", 2024-09-14T09:06:16.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8199999999999996e-06 2024-09-14T09:06:16.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518959+0000", 2024-09-14T09:06:16.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1373e-05 2024-09-14T09:06:16.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518973+0000", 2024-09-14T09:06:16.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3916e-05 2024-09-14T09:06:16.566 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.566 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.566 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.566 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53313 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.566 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.505461+0000", 2024-09-14T09:06:16.566 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.014590422, 2024-09-14T09:06:16.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016199963000000001, 2024-09-14T09:06:16.566 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.566 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.566 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.566 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.567 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.567 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.567 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53313 2024-09-14T09:06:16.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.567 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505461+0000", 2024-09-14T09:06:16.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505461+0000", 2024-09-14T09:06:16.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505462+0000", 2024-09-14T09:06:16.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.176e-06 2024-09-14T09:06:16.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505467+0000", 2024-09-14T09:06:16.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4700000000000001e-06 2024-09-14T09:06:16.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505485+0000", 2024-09-14T09:06:16.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8050000000000002e-05 2024-09-14T09:06:16.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505500+0000", 2024-09-14T09:06:16.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4253999999999999e-05 2024-09-14T09:06:16.569 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.569 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.569 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.570 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53575 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.570 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509344+0000", 2024-09-14T09:06:16.570 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.010706824, 2024-09-14T09:06:16.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.01244251, 2024-09-14T09:06:16.570 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.570 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.570 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.570 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.570 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.570 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.570 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53575 2024-09-14T09:06:16.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.570 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509344+0000", 2024-09-14T09:06:16.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509344+0000", 2024-09-14T09:06:16.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509346+0000", 2024-09-14T09:06:16.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3030000000000001e-06 2024-09-14T09:06:16.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509351+0000", 2024-09-14T09:06:16.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2839999999999997e-06 2024-09-14T09:06:16.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509368+0000", 2024-09-14T09:06:16.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7212999999999999e-05 2024-09-14T09:06:16.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509382+0000", 2024-09-14T09:06:16.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4059999999999999e-05 2024-09-14T09:06:16.573 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.573 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.573 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.573 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53799 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.573 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.518990+0000", 2024-09-14T09:06:16.573 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.0010608659999999999, 2024-09-14T09:06:16.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002920354, 2024-09-14T09:06:16.574 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.574 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.574 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.574 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.574 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.574 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.574 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53799 2024-09-14T09:06:16.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.574 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518990+0000", 2024-09-14T09:06:16.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518990+0000", 2024-09-14T09:06:16.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518992+0000", 2024-09-14T09:06:16.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2640000000000001e-06 2024-09-14T09:06:16.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518997+0000", 2024-09-14T09:06:16.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.57e-06 2024-09-14T09:06:16.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519015+0000", 2024-09-14T09:06:16.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7779999999999999e-05 2024-09-14T09:06:16.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519029+0000", 2024-09-14T09:06:16.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4e-05 2024-09-14T09:06:16.576 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.576 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.576 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.577 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53353 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.577 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.505518+0000", 2024-09-14T09:06:16.577 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.014533406, 2024-09-14T09:06:16.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016517124000000001, 2024-09-14T09:06:16.577 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.577 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.577 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.577 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.577 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.577 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.578 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53353 2024-09-14T09:06:16.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.578 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505518+0000", 2024-09-14T09:06:16.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505518+0000", 2024-09-14T09:06:16.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505519+0000", 2024-09-14T09:06:16.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1599999999999999e-06 2024-09-14T09:06:16.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505559+0000", 2024-09-14T09:06:16.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0380000000000003e-05 2024-09-14T09:06:16.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505581+0000", 2024-09-14T09:06:16.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2118999999999999e-05 2024-09-14T09:06:16.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505596+0000", 2024-09-14T09:06:16.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5007e-05 2024-09-14T09:06:16.580 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.580 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.580 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.580 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53576 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.580 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509402+0000", 2024-09-14T09:06:16.580 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.010649708000000001, 2024-09-14T09:06:16.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.012759691, 2024-09-14T09:06:16.580 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.581 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.581 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.581 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.581 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.581 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.581 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53576 2024-09-14T09:06:16.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.581 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509402+0000", 2024-09-14T09:06:16.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509402+0000", 2024-09-14T09:06:16.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509403+0000", 2024-09-14T09:06:16.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.33e-06 2024-09-14T09:06:16.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509408+0000", 2024-09-14T09:06:16.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0769999999999998e-06 2024-09-14T09:06:16.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509426+0000", 2024-09-14T09:06:16.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7723e-05 2024-09-14T09:06:16.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509440+0000", 2024-09-14T09:06:16.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4167e-05 2024-09-14T09:06:16.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.583 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.584 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53800 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.584 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.519051+0000", 2024-09-14T09:06:16.584 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.0010002190000000001, 2024-09-14T09:06:16.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003235412, 2024-09-14T09:06:16.584 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.584 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.584 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.584 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53800 2024-09-14T09:06:16.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.585 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519051+0000", 2024-09-14T09:06:16.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519051+0000", 2024-09-14T09:06:16.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519052+0000", 2024-09-14T09:06:16.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2899999999999999e-06 2024-09-14T09:06:16.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519058+0000", 2024-09-14T09:06:16.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8130000000000001e-06 2024-09-14T09:06:16.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519079+0000", 2024-09-14T09:06:16.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0463e-05 2024-09-14T09:06:16.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519093+0000", 2024-09-14T09:06:16.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4100000000000001e-05 2024-09-14T09:06:16.587 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.587 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.587 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.587 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53355 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.587 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.505617+0000", 2024-09-14T09:06:16.587 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.01443405, 2024-09-14T09:06:16.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016801844999999999, 2024-09-14T09:06:16.587 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.587 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.587 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.587 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.588 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.588 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.588 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53355 2024-09-14T09:06:16.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.588 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505617+0000", 2024-09-14T09:06:16.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505617+0000", 2024-09-14T09:06:16.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505618+0000", 2024-09-14T09:06:16.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.31e-06 2024-09-14T09:06:16.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505624+0000", 2024-09-14T09:06:16.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5570000000000003e-06 2024-09-14T09:06:16.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505642+0000", 2024-09-14T09:06:16.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8165999999999999e-05 2024-09-14T09:06:16.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505657+0000", 2024-09-14T09:06:16.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.436e-05 2024-09-14T09:06:16.590 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.590 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.590 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.591 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53577 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.591 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509459+0000", 2024-09-14T09:06:16.591 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.010592585, 2024-09-14T09:06:16.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013088024, 2024-09-14T09:06:16.591 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.591 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.591 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.591 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.591 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.591 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.591 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53577 2024-09-14T09:06:16.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.591 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509459+0000", 2024-09-14T09:06:16.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509459+0000", 2024-09-14T09:06:16.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509460+0000", 2024-09-14T09:06:16.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.283e-06 2024-09-14T09:06:16.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509465+0000", 2024-09-14T09:06:16.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2000000000000002e-06 2024-09-14T09:06:16.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509483+0000", 2024-09-14T09:06:16.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7669999999999999e-05 2024-09-14T09:06:16.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509501+0000", 2024-09-14T09:06:16.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8669999999999999e-05 2024-09-14T09:06:16.594 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.594 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.594 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.594 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53801 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.594 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.519111+0000", 2024-09-14T09:06:16.594 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.00094019000000000004, 2024-09-14T09:06:16.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0035608710000000002, 2024-09-14T09:06:16.594 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.594 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.594 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.594 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.595 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.595 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.595 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53801 2024-09-14T09:06:16.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.595 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519111+0000", 2024-09-14T09:06:16.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519111+0000", 2024-09-14T09:06:16.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519112+0000", 2024-09-14T09:06:16.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2270000000000001e-06 2024-09-14T09:06:16.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519117+0000", 2024-09-14T09:06:16.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0830000000000003e-06 2024-09-14T09:06:16.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519139+0000", 2024-09-14T09:06:16.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1177000000000001e-05 2024-09-14T09:06:16.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519152+0000", 2024-09-14T09:06:16.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3653e-05 2024-09-14T09:06:16.597 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.597 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.597 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.597 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53356 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.597 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.505674+0000", 2024-09-14T09:06:16.597 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.014376923999999999, 2024-09-14T09:06:16.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017107088999999999, 2024-09-14T09:06:16.598 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.598 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.598 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.598 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.598 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.598 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.598 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53356 2024-09-14T09:06:16.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.598 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505674+0000", 2024-09-14T09:06:16.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505674+0000", 2024-09-14T09:06:16.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505676+0000", 2024-09-14T09:06:16.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1999999999999999e-06 2024-09-14T09:06:16.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505680+0000", 2024-09-14T09:06:16.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7829999999999998e-06 2024-09-14T09:06:16.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505698+0000", 2024-09-14T09:06:16.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7960000000000001e-05 2024-09-14T09:06:16.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505712+0000", 2024-09-14T09:06:16.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3869999999999999e-05 2024-09-14T09:06:16.600 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.600 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.600 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.601 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53579 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.601 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509523+0000", 2024-09-14T09:06:16.601 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.010527732, 2024-09-14T09:06:16.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.01331336, 2024-09-14T09:06:16.601 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.601 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.601 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.601 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.601 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.601 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.601 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53579 2024-09-14T09:06:16.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.601 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509523+0000", 2024-09-14T09:06:16.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509523+0000", 2024-09-14T09:06:16.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509525+0000", 2024-09-14T09:06:16.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2699999999999999e-06 2024-09-14T09:06:16.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509530+0000", 2024-09-14T09:06:16.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2399999999999998e-06 2024-09-14T09:06:16.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509548+0000", 2024-09-14T09:06:16.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7986999999999999e-05 2024-09-14T09:06:16.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509562+0000", 2024-09-14T09:06:16.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4126e-05 2024-09-14T09:06:16.604 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.604 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.604 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.604 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53827 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.604 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.519170+0000", 2024-09-14T09:06:16.604 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.00088126999999999997, 2024-09-14T09:06:16.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037175720000000002, 2024-09-14T09:06:16.604 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.604 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.604 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.605 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.605 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.605 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.605 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53827 2024-09-14T09:06:16.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.605 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519170+0000", 2024-09-14T09:06:16.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519170+0000", 2024-09-14T09:06:16.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519171+0000", 2024-09-14T09:06:16.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1599999999999999e-06 2024-09-14T09:06:16.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519177+0000", 2024-09-14T09:06:16.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4500000000000003e-06 2024-09-14T09:06:16.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519198+0000", 2024-09-14T09:06:16.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0939999999999999e-05 2024-09-14T09:06:16.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519212+0000", 2024-09-14T09:06:16.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4572999999999999e-05 2024-09-14T09:06:16.607 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.607 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.607 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.607 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53358 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.607 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.505731+0000", 2024-09-14T09:06:16.608 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.014320134, 2024-09-14T09:06:16.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017206533999999999, 2024-09-14T09:06:16.608 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.608 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.608 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.608 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.608 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.608 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.608 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53358 2024-09-14T09:06:16.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.608 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505731+0000", 2024-09-14T09:06:16.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505731+0000", 2024-09-14T09:06:16.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505732+0000", 2024-09-14T09:06:16.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.373e-06 2024-09-14T09:06:16.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505738+0000", 2024-09-14T09:06:16.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1270000000000002e-06 2024-09-14T09:06:16.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505756+0000", 2024-09-14T09:06:16.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8635999999999999e-05 2024-09-14T09:06:16.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505769+0000", 2024-09-14T09:06:16.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3179999999999999e-05 2024-09-14T09:06:16.611 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.611 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.611 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.611 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53590 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.611 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509581+0000", 2024-09-14T09:06:16.611 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.010469979000000001, 2024-09-14T09:06:16.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013406932, 2024-09-14T09:06:16.611 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.611 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.612 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.612 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.612 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.612 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.612 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53590 2024-09-14T09:06:16.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.612 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509581+0000", 2024-09-14T09:06:16.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509581+0000", 2024-09-14T09:06:16.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509583+0000", 2024-09-14T09:06:16.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.336e-06 2024-09-14T09:06:16.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509588+0000", 2024-09-14T09:06:16.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2669999999999997e-06 2024-09-14T09:06:16.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509606+0000", 2024-09-14T09:06:16.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7716000000000001e-05 2024-09-14T09:06:16.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509620+0000", 2024-09-14T09:06:16.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4354e-05 2024-09-14T09:06:16.614 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.614 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.614 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.615 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53828 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.615 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.519230+0000", 2024-09-14T09:06:16.615 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.00082079099999999997, 2024-09-14T09:06:16.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038079960000000001, 2024-09-14T09:06:16.615 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.615 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.615 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.615 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.615 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.615 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.615 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53828 2024-09-14T09:06:16.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.615 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519230+0000", 2024-09-14T09:06:16.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519230+0000", 2024-09-14T09:06:16.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519232+0000", 2024-09-14T09:06:16.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1999999999999999e-06 2024-09-14T09:06:16.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519237+0000", 2024-09-14T09:06:16.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9899999999999997e-06 2024-09-14T09:06:16.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519258+0000", 2024-09-14T09:06:16.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1056999999999998e-05 2024-09-14T09:06:16.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519271+0000", 2024-09-14T09:06:16.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.366e-05 2024-09-14T09:06:16.617 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.618 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.618 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.618 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53359 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.618 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.505789+0000", 2024-09-14T09:06:16.618 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.014262151000000001, 2024-09-14T09:06:16.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017299181, 2024-09-14T09:06:16.618 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.618 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.618 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.618 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.618 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.618 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.618 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53359 2024-09-14T09:06:16.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.619 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505789+0000", 2024-09-14T09:06:16.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505789+0000", 2024-09-14T09:06:16.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505790+0000", 2024-09-14T09:06:16.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3659999999999999e-06 2024-09-14T09:06:16.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505796+0000", 2024-09-14T09:06:16.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1399999999999999e-06 2024-09-14T09:06:16.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505814+0000", 2024-09-14T09:06:16.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8053000000000001e-05 2024-09-14T09:06:16.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505829+0000", 2024-09-14T09:06:16.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5047e-05 2024-09-14T09:06:16.621 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.621 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.621 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.621 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53592 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.621 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509650+0000", 2024-09-14T09:06:16.621 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.01040143, 2024-09-14T09:06:16.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013488737000000001, 2024-09-14T09:06:16.621 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.621 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.621 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.621 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.621 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.622 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.622 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53592 2024-09-14T09:06:16.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.622 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509650+0000", 2024-09-14T09:06:16.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509650+0000", 2024-09-14T09:06:16.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509651+0000", 2024-09-14T09:06:16.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4670000000000001e-06 2024-09-14T09:06:16.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509657+0000", 2024-09-14T09:06:16.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6869999999999999e-06 2024-09-14T09:06:16.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509676+0000", 2024-09-14T09:06:16.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8700000000000001e-05 2024-09-14T09:06:16.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509689+0000", 2024-09-14T09:06:16.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3003000000000001e-05 2024-09-14T09:06:16.624 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.624 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.624 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.624 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53829 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.624 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.519290+0000", 2024-09-14T09:06:16.624 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.00076143100000000002, 2024-09-14T09:06:16.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038986160000000001, 2024-09-14T09:06:16.625 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.625 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.625 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.625 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.625 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.625 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.625 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53829 2024-09-14T09:06:16.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.625 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519290+0000", 2024-09-14T09:06:16.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519290+0000", 2024-09-14T09:06:16.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519291+0000", 2024-09-14T09:06:16.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.237e-06 2024-09-14T09:06:16.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519305+0000", 2024-09-14T09:06:16.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4326e-05 2024-09-14T09:06:16.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519326+0000", 2024-09-14T09:06:16.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.073e-05 2024-09-14T09:06:16.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519340+0000", 2024-09-14T09:06:16.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3559e-05 2024-09-14T09:06:16.627 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.627 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.628 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.628 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53360 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.628 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.505849+0000", 2024-09-14T09:06:16.628 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.014202319, 2024-09-14T09:06:16.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017389743999999999, 2024-09-14T09:06:16.628 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.628 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.628 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.628 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.628 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.628 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.628 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53360 2024-09-14T09:06:16.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.629 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505849+0000", 2024-09-14T09:06:16.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505849+0000", 2024-09-14T09:06:16.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505850+0000", 2024-09-14T09:06:16.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3370000000000001e-06 2024-09-14T09:06:16.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505856+0000", 2024-09-14T09:06:16.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4330000000000003e-06 2024-09-14T09:06:16.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505874+0000", 2024-09-14T09:06:16.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8014000000000001e-05 2024-09-14T09:06:16.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505887+0000", 2024-09-14T09:06:16.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3563e-05 2024-09-14T09:06:16.631 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.631 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.631 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.631 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53594 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.631 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509709+0000", 2024-09-14T09:06:16.631 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.010342387, 2024-09-14T09:06:16.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013579628999999999, 2024-09-14T09:06:16.631 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.631 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.631 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.631 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.632 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.632 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.632 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53594 2024-09-14T09:06:16.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.632 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509709+0000", 2024-09-14T09:06:16.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509709+0000", 2024-09-14T09:06:16.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509710+0000", 2024-09-14T09:06:16.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.243e-06 2024-09-14T09:06:16.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509715+0000", 2024-09-14T09:06:16.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6539999999999998e-06 2024-09-14T09:06:16.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509733+0000", 2024-09-14T09:06:16.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7886000000000001e-05 2024-09-14T09:06:16.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509747+0000", 2024-09-14T09:06:16.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4333e-05 2024-09-14T09:06:16.634 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.634 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.634 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.634 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53830 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.634 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.519358+0000", 2024-09-14T09:06:16.635 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.00069275200000000004, 2024-09-14T09:06:16.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039801250000000002, 2024-09-14T09:06:16.635 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.635 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.635 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.635 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.635 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.635 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.635 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53830 2024-09-14T09:06:16.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.635 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519358+0000", 2024-09-14T09:06:16.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519358+0000", 2024-09-14T09:06:16.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519360+0000", 2024-09-14T09:06:16.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.367e-06 2024-09-14T09:06:16.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519365+0000", 2024-09-14T09:06:16.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1460000000000004e-06 2024-09-14T09:06:16.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519387+0000", 2024-09-14T09:06:16.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1637e-05 2024-09-14T09:06:16.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519401+0000", 2024-09-14T09:06:16.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3937000000000001e-05 2024-09-14T09:06:16.638 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.638 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.638 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.638 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53372 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.638 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.505906+0000", 2024-09-14T09:06:16.638 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.014145479000000001, 2024-09-14T09:06:16.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.01748268, 2024-09-14T09:06:16.638 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.638 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.638 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.638 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.638 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.638 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.639 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53372 2024-09-14T09:06:16.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.639 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505906+0000", 2024-09-14T09:06:16.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505906+0000", 2024-09-14T09:06:16.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505907+0000", 2024-09-14T09:06:16.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1170000000000001e-06 2024-09-14T09:06:16.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505912+0000", 2024-09-14T09:06:16.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8899999999999998e-06 2024-09-14T09:06:16.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505930+0000", 2024-09-14T09:06:16.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7912999999999999e-05 2024-09-14T09:06:16.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505943+0000", 2024-09-14T09:06:16.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3512999999999999e-05 2024-09-14T09:06:16.641 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.641 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.641 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.641 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53630 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.641 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509765+0000", 2024-09-14T09:06:16.641 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.010285984, 2024-09-14T09:06:16.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013672527, 2024-09-14T09:06:16.641 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.641 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.642 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.642 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.642 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.642 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.642 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53630 2024-09-14T09:06:16.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.642 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509765+0000", 2024-09-14T09:06:16.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509765+0000", 2024-09-14T09:06:16.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509767+0000", 2024-09-14T09:06:16.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.33e-06 2024-09-14T09:06:16.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509772+0000", 2024-09-14T09:06:16.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.967e-06 2024-09-14T09:06:16.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509793+0000", 2024-09-14T09:06:16.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1880000000000001e-05 2024-09-14T09:06:16.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509807+0000", 2024-09-14T09:06:16.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4036000000000001e-05 2024-09-14T09:06:16.644 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.644 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.644 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.645 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53831 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.645 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.519418+0000", 2024-09-14T09:06:16.645 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.00063312899999999996, 2024-09-14T09:06:16.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041170620000000003, 2024-09-14T09:06:16.645 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.645 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.645 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.645 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.645 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.645 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.645 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53831 2024-09-14T09:06:16.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.645 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519418+0000", 2024-09-14T09:06:16.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519418+0000", 2024-09-14T09:06:16.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519419+0000", 2024-09-14T09:06:16.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1939999999999999e-06 2024-09-14T09:06:16.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519424+0000", 2024-09-14T09:06:16.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9529999999999998e-06 2024-09-14T09:06:16.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519445+0000", 2024-09-14T09:06:16.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0733e-05 2024-09-14T09:06:16.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519459+0000", 2024-09-14T09:06:16.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3616e-05 2024-09-14T09:06:16.647 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.647 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.648 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.648 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53376 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.648 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.505961+0000", 2024-09-14T09:06:16.648 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.014090306, 2024-09-14T09:06:16.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017625499999999999, 2024-09-14T09:06:16.648 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.648 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.648 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.648 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.648 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.648 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.648 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53376 2024-09-14T09:06:16.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.649 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505961+0000", 2024-09-14T09:06:16.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505961+0000", 2024-09-14T09:06:16.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505962+0000", 2024-09-14T09:06:16.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.15e-06 2024-09-14T09:06:16.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505967+0000", 2024-09-14T09:06:16.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2129999999999999e-06 2024-09-14T09:06:16.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505985+0000", 2024-09-14T09:06:16.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7572999999999999e-05 2024-09-14T09:06:16.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.505998+0000", 2024-09-14T09:06:16.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3604000000000001e-05 2024-09-14T09:06:16.651 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.651 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.651 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.651 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53632 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.651 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509827+0000", 2024-09-14T09:06:16.651 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.010224087999999999, 2024-09-14T09:06:16.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013816307999999999, 2024-09-14T09:06:16.651 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.651 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.651 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.651 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.651 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.652 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.652 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53632 2024-09-14T09:06:16.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.652 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509827+0000", 2024-09-14T09:06:16.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509827+0000", 2024-09-14T09:06:16.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509828+0000", 2024-09-14T09:06:16.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.333e-06 2024-09-14T09:06:16.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509834+0000", 2024-09-14T09:06:16.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3639999999999998e-06 2024-09-14T09:06:16.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509856+0000", 2024-09-14T09:06:16.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2226e-05 2024-09-14T09:06:16.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509870+0000", 2024-09-14T09:06:16.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4070000000000001e-05 2024-09-14T09:06:16.654 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.654 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.654 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.654 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53872 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.654 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.519617+0000", 2024-09-14T09:06:16.654 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.00043390099999999999, 2024-09-14T09:06:16.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040763099999999997, 2024-09-14T09:06:16.654 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.655 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.655 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.655 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.655 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.655 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.655 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53872 2024-09-14T09:06:16.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.655 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519617+0000", 2024-09-14T09:06:16.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519617+0000", 2024-09-14T09:06:16.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519619+0000", 2024-09-14T09:06:16.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5540000000000001e-06 2024-09-14T09:06:16.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519627+0000", 2024-09-14T09:06:16.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8399999999999995e-06 2024-09-14T09:06:16.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519654+0000", 2024-09-14T09:06:16.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6886e-05 2024-09-14T09:06:16.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.519670+0000", 2024-09-14T09:06:16.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6393e-05 2024-09-14T09:06:16.657 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.657 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.657 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.657 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53380 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.658 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.506033+0000", 2024-09-14T09:06:16.658 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.014017987000000001, 2024-09-14T09:06:16.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017715284000000001, 2024-09-14T09:06:16.658 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.658 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.658 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.658 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.658 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.658 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.658 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53380 2024-09-14T09:06:16.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.658 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506033+0000", 2024-09-14T09:06:16.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506033+0000", 2024-09-14T09:06:16.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506034+0000", 2024-09-14T09:06:16.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2440000000000001e-06 2024-09-14T09:06:16.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506040+0000", 2024-09-14T09:06:16.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5929999999999996e-06 2024-09-14T09:06:16.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506059+0000", 2024-09-14T09:06:16.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8703e-05 2024-09-14T09:06:16.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506072+0000", 2024-09-14T09:06:16.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3443e-05 2024-09-14T09:06:16.660 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.661 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.661 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.661 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53634 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.661 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509889+0000", 2024-09-14T09:06:16.661 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.010162008, 2024-09-14T09:06:16.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013910089, 2024-09-14T09:06:16.661 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.661 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.661 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.661 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.661 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.661 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.661 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53634 2024-09-14T09:06:16.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.662 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509889+0000", 2024-09-14T09:06:16.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509889+0000", 2024-09-14T09:06:16.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509890+0000", 2024-09-14T09:06:16.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.263e-06 2024-09-14T09:06:16.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509895+0000", 2024-09-14T09:06:16.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9629999999999997e-06 2024-09-14T09:06:16.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509916+0000", 2024-09-14T09:06:16.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0903999999999998e-05 2024-09-14T09:06:16.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509930+0000", 2024-09-14T09:06:16.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4005999999999999e-05 2024-09-14T09:06:16.664 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.664 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.664 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.664 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53873 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.664 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.520013+0000", 2024-09-14T09:06:16.664 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 3.8668999999999999e-05, 2024-09-14T09:06:16.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003840319, 2024-09-14T09:06:16.665 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.665 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.665 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.665 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.665 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.665 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.665 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53873 2024-09-14T09:06:16.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.665 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520013+0000", 2024-09-14T09:06:16.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520013+0000", 2024-09-14T09:06:16.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520014+0000", 2024-09-14T09:06:16.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7000000000000003e-07 2024-09-14T09:06:16.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520018+0000", 2024-09-14T09:06:16.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5050000000000001e-06 2024-09-14T09:06:16.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520028+0000", 2024-09-14T09:06:16.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9380000000000004e-06 2024-09-14T09:06:16.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520034+0000", 2024-09-14T09:06:16.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8109999999999999e-06 2024-09-14T09:06:16.667 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.667 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.667 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.668 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53382 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.668 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.506091+0000", 2024-09-14T09:06:16.668 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.013959964, 2024-09-14T09:06:16.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017811653, 2024-09-14T09:06:16.668 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.668 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.668 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.668 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.668 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.668 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.668 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53382 2024-09-14T09:06:16.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.669 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506091+0000", 2024-09-14T09:06:16.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506091+0000", 2024-09-14T09:06:16.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506092+0000", 2024-09-14T09:06:16.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.223e-06 2024-09-14T09:06:16.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506098+0000", 2024-09-14T09:06:16.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1039999999999998e-06 2024-09-14T09:06:16.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506116+0000", 2024-09-14T09:06:16.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8272999999999999e-05 2024-09-14T09:06:16.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506129+0000", 2024-09-14T09:06:16.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3339999999999999e-05 2024-09-14T09:06:16.671 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.671 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.671 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.671 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53636 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.671 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509948+0000", 2024-09-14T09:06:16.671 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.010103005999999999, 2024-09-14T09:06:16.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014008909999999999, 2024-09-14T09:06:16.671 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.671 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.672 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.672 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.672 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.672 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.672 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53636 2024-09-14T09:06:16.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.672 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509948+0000", 2024-09-14T09:06:16.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509948+0000", 2024-09-14T09:06:16.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509950+0000", 2024-09-14T09:06:16.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.347e-06 2024-09-14T09:06:16.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509954+0000", 2024-09-14T09:06:16.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7999999999999998e-06 2024-09-14T09:06:16.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509973+0000", 2024-09-14T09:06:16.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8396000000000001e-05 2024-09-14T09:06:16.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509986+0000", 2024-09-14T09:06:16.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3604000000000001e-05 2024-09-14T09:06:16.674 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.674 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.675 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.675 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53874 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.675 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.520045+0000", 2024-09-14T09:06:16.675 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.6919999999999996e-06, 2024-09-14T09:06:16.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039648469999999996, 2024-09-14T09:06:16.675 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.675 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.675 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.675 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.675 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.675 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.676 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53874 2024-09-14T09:06:16.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.676 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520045+0000", 2024-09-14T09:06:16.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520045+0000", 2024-09-14T09:06:16.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520045+0000", 2024-09-14T09:06:16.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6899999999999998e-07 2024-09-14T09:06:16.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520047+0000", 2024-09-14T09:06:16.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7710000000000001e-06 2024-09-14T09:06:16.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520051+0000", 2024-09-14T09:06:16.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0230000000000003e-06 2024-09-14T09:06:16.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520054+0000", 2024-09-14T09:06:16.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.134e-06 2024-09-14T09:06:16.678 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.678 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.678 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.678 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53404 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.678 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.506148+0000", 2024-09-14T09:06:16.678 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.013903670999999999, 2024-09-14T09:06:16.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017914672, 2024-09-14T09:06:16.678 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.679 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.679 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.679 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.679 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.679 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.679 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53404 2024-09-14T09:06:16.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.679 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506148+0000", 2024-09-14T09:06:16.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506148+0000", 2024-09-14T09:06:16.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506149+0000", 2024-09-14T09:06:16.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1799999999999999e-06 2024-09-14T09:06:16.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506155+0000", 2024-09-14T09:06:16.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9669999999999998e-06 2024-09-14T09:06:16.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506173+0000", 2024-09-14T09:06:16.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7822999999999999e-05 2024-09-14T09:06:16.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506188+0000", 2024-09-14T09:06:16.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5936000000000001e-05 2024-09-14T09:06:16.681 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.681 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.681 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.682 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53643 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.682 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.510005+0000", 2024-09-14T09:06:16.682 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.010046586, 2024-09-14T09:06:16.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014126174999999999, 2024-09-14T09:06:16.682 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.682 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.682 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.682 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.682 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.682 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.682 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53643 2024-09-14T09:06:16.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.682 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510005+0000", 2024-09-14T09:06:16.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510005+0000", 2024-09-14T09:06:16.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510006+0000", 2024-09-14T09:06:16.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2729999999999999e-06 2024-09-14T09:06:16.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510011+0000", 2024-09-14T09:06:16.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1039999999999998e-06 2024-09-14T09:06:16.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510029+0000", 2024-09-14T09:06:16.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7866000000000001e-05 2024-09-14T09:06:16.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510043+0000", 2024-09-14T09:06:16.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4066999999999999e-05 2024-09-14T09:06:16.684 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.684 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.685 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.685 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53875 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.685 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.520058+0000", 2024-09-14T09:06:16.685 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4294967295.9999933, 2024-09-14T09:06:16.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041300219999999997, 2024-09-14T09:06:16.685 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.685 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.685 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.685 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.685 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.685 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.685 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53875 2024-09-14T09:06:16.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.686 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520058+0000", 2024-09-14T09:06:16.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520058+0000", 2024-09-14T09:06:16.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520058+0000", 2024-09-14T09:06:16.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.65e-07 2024-09-14T09:06:16.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520059+0000", 2024-09-14T09:06:16.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2300000000000001e-06 2024-09-14T09:06:16.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520063+0000", 2024-09-14T09:06:16.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2459999999999999e-06 2024-09-14T09:06:16.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520066+0000", 2024-09-14T09:06:16.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8480000000000001e-06 2024-09-14T09:06:16.688 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.688 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.688 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.688 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53405 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.688 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.506207+0000", 2024-09-14T09:06:16.688 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.013844042000000001, 2024-09-14T09:06:16.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018031100000000001, 2024-09-14T09:06:16.688 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.688 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.688 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.689 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.689 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.689 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.689 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53405 2024-09-14T09:06:16.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.689 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506207+0000", 2024-09-14T09:06:16.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506207+0000", 2024-09-14T09:06:16.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506209+0000", 2024-09-14T09:06:16.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3340000000000001e-06 2024-09-14T09:06:16.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506214+0000", 2024-09-14T09:06:16.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1460000000000004e-06 2024-09-14T09:06:16.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506232+0000", 2024-09-14T09:06:16.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.808e-05 2024-09-14T09:06:16.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506245+0000", 2024-09-14T09:06:16.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3647e-05 2024-09-14T09:06:16.691 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.691 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.691 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.692 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53644 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.692 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.510063+0000", 2024-09-14T09:06:16.692 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.0099885600000000005, 2024-09-14T09:06:16.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014225362, 2024-09-14T09:06:16.692 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.692 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.692 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.692 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.692 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.692 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.692 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53644 2024-09-14T09:06:16.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.692 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510063+0000", 2024-09-14T09:06:16.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510063+0000", 2024-09-14T09:06:16.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510064+0000", 2024-09-14T09:06:16.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.344e-06 2024-09-14T09:06:16.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510070+0000", 2024-09-14T09:06:16.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5860000000000004e-06 2024-09-14T09:06:16.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510087+0000", 2024-09-14T09:06:16.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7776000000000001e-05 2024-09-14T09:06:16.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510105+0000", 2024-09-14T09:06:16.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7220000000000001e-05 2024-09-14T09:06:16.695 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.695 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.695 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.695 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53883 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.695 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.520070+0000", 2024-09-14T09:06:16.695 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4294967295.9999809, 2024-09-14T09:06:16.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042721699999999996, 2024-09-14T09:06:16.695 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.695 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.695 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.695 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.695 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.696 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.696 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53883 2024-09-14T09:06:16.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.696 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520070+0000", 2024-09-14T09:06:16.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520070+0000", 2024-09-14T09:06:16.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520071+0000", 2024-09-14T09:06:16.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7700000000000001e-07 2024-09-14T09:06:16.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520072+0000", 2024-09-14T09:06:16.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1370000000000001e-06 2024-09-14T09:06:16.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520075+0000", 2024-09-14T09:06:16.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3160000000000001e-06 2024-09-14T09:06:16.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520078+0000", 2024-09-14T09:06:16.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0730000000000001e-06 2024-09-14T09:06:16.698 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.698 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.698 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.698 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53406 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.698 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.506263+0000", 2024-09-14T09:06:16.698 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.013788012000000001, 2024-09-14T09:06:16.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018129781000000001, 2024-09-14T09:06:16.698 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.698 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.698 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.699 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.699 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.699 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.699 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53406 2024-09-14T09:06:16.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.699 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506263+0000", 2024-09-14T09:06:16.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506263+0000", 2024-09-14T09:06:16.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506265+0000", 2024-09-14T09:06:16.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.297e-06 2024-09-14T09:06:16.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506279+0000", 2024-09-14T09:06:16.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4939999999999999e-05 2024-09-14T09:06:16.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506297+0000", 2024-09-14T09:06:16.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7926000000000001e-05 2024-09-14T09:06:16.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506311+0000", 2024-09-14T09:06:16.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.382e-05 2024-09-14T09:06:16.701 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.701 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.701 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.701 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53645 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.701 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.510123+0000", 2024-09-14T09:06:16.701 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.0099278969999999998, 2024-09-14T09:06:16.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014323664999999999, 2024-09-14T09:06:16.702 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.702 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.702 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.702 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.702 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.702 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.702 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53645 2024-09-14T09:06:16.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.702 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510123+0000", 2024-09-14T09:06:16.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510123+0000", 2024-09-14T09:06:16.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510125+0000", 2024-09-14T09:06:16.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3170000000000001e-06 2024-09-14T09:06:16.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510130+0000", 2024-09-14T09:06:16.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2630000000000003e-06 2024-09-14T09:06:16.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510148+0000", 2024-09-14T09:06:16.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7793000000000001e-05 2024-09-14T09:06:16.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510161+0000", 2024-09-14T09:06:16.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3627e-05 2024-09-14T09:06:16.704 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.704 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.704 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.705 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53884 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.705 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.520082+0000", 2024-09-14T09:06:16.705 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4294967295.9999695, 2024-09-14T09:06:16.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044160040000000003, 2024-09-14T09:06:16.705 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.705 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.705 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.705 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.705 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.705 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.706 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53884 2024-09-14T09:06:16.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.706 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520082+0000", 2024-09-14T09:06:16.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520082+0000", 2024-09-14T09:06:16.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520082+0000", 2024-09-14T09:06:16.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.67e-07 2024-09-14T09:06:16.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520093+0000", 2024-09-14T09:06:16.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0761000000000001e-05 2024-09-14T09:06:16.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520097+0000", 2024-09-14T09:06:16.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8380000000000001e-06 2024-09-14T09:06:16.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520102+0000", 2024-09-14T09:06:16.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9679999999999997e-06 2024-09-14T09:06:16.708 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.708 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.708 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.708 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53407 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.708 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.506332+0000", 2024-09-14T09:06:16.708 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.013719573000000001, 2024-09-14T09:06:16.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018220785, 2024-09-14T09:06:16.709 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.709 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.709 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.709 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.709 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.709 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.709 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53407 2024-09-14T09:06:16.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.709 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506332+0000", 2024-09-14T09:06:16.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506332+0000", 2024-09-14T09:06:16.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506333+0000", 2024-09-14T09:06:16.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.514e-06 2024-09-14T09:06:16.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506338+0000", 2024-09-14T09:06:16.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2900000000000002e-06 2024-09-14T09:06:16.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506356+0000", 2024-09-14T09:06:16.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7929999999999999e-05 2024-09-14T09:06:16.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506370+0000", 2024-09-14T09:06:16.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3823e-05 2024-09-14T09:06:16.711 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.711 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.711 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.712 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53672 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.712 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.510197+0000", 2024-09-14T09:06:16.712 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.0098539839999999997, 2024-09-14T09:06:16.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014405896, 2024-09-14T09:06:16.712 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.712 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.712 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.712 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.712 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.712 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.712 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53672 2024-09-14T09:06:16.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.712 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510197+0000", 2024-09-14T09:06:16.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510197+0000", 2024-09-14T09:06:16.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510199+0000", 2024-09-14T09:06:16.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3859999999999999e-06 2024-09-14T09:06:16.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510204+0000", 2024-09-14T09:06:16.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.344e-06 2024-09-14T09:06:16.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510873+0000", 2024-09-14T09:06:16.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000669124 2024-09-14T09:06:16.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510923+0000", 2024-09-14T09:06:16.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0115999999999999e-05 2024-09-14T09:06:16.715 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.715 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.715 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.715 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53885 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.715 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.520118+0000", 2024-09-14T09:06:16.715 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4294967295.9999332, 2024-09-14T09:06:16.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045384989999999997, 2024-09-14T09:06:16.715 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.715 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.715 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.715 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.716 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.716 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.716 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53885 2024-09-14T09:06:16.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.716 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520118+0000", 2024-09-14T09:06:16.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520118+0000", 2024-09-14T09:06:16.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520119+0000", 2024-09-14T09:06:16.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4899999999999995e-07 2024-09-14T09:06:16.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520121+0000", 2024-09-14T09:06:16.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3520000000000001e-06 2024-09-14T09:06:16.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520128+0000", 2024-09-14T09:06:16.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9609999999999999e-06 2024-09-14T09:06:16.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520131+0000", 2024-09-14T09:06:16.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1970000000000001e-06 2024-09-14T09:06:16.718 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.718 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.718 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.718 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53455 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.718 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.506388+0000", 2024-09-14T09:06:16.718 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.013662986, 2024-09-14T09:06:16.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018318648, 2024-09-14T09:06:16.719 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.719 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.719 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.719 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53455 2024-09-14T09:06:16.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.719 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506388+0000", 2024-09-14T09:06:16.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506388+0000", 2024-09-14T09:06:16.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506390+0000", 2024-09-14T09:06:16.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3659999999999999e-06 2024-09-14T09:06:16.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506395+0000", 2024-09-14T09:06:16.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2939999999999996e-06 2024-09-14T09:06:16.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506415+0000", 2024-09-14T09:06:16.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9876000000000001e-05 2024-09-14T09:06:16.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506428+0000", 2024-09-14T09:06:16.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3423e-05 2024-09-14T09:06:16.721 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.721 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.722 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.722 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53673 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.722 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.510953+0000", 2024-09-14T09:06:16.722 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.0090984870000000006, 2024-09-14T09:06:16.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013807786000000001, 2024-09-14T09:06:16.722 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.722 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.722 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.722 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.722 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.722 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.722 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53673 2024-09-14T09:06:16.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.723 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510953+0000", 2024-09-14T09:06:16.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510953+0000", 2024-09-14T09:06:16.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510954+0000", 2024-09-14T09:06:16.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6130000000000001e-06 2024-09-14T09:06:16.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510963+0000", 2024-09-14T09:06:16.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2800000000000003e-06 2024-09-14T09:06:16.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510984+0000", 2024-09-14T09:06:16.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0979999999999999e-05 2024-09-14T09:06:16.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.510998+0000", 2024-09-14T09:06:16.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4863e-05 2024-09-14T09:06:16.725 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.725 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.725 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.725 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53886 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.725 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.520136+0000", 2024-09-14T09:06:16.725 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4294967295.9999156, 2024-09-14T09:06:16.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046757839999999997, 2024-09-14T09:06:16.725 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.725 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.726 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.726 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.726 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.726 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.726 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53886 2024-09-14T09:06:16.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.726 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520136+0000", 2024-09-14T09:06:16.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520136+0000", 2024-09-14T09:06:16.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520136+0000", 2024-09-14T09:06:16.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0400000000000002e-07 2024-09-14T09:06:16.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520137+0000", 2024-09-14T09:06:16.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4139999999999999e-06 2024-09-14T09:06:16.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520143+0000", 2024-09-14T09:06:16.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8170000000000004e-06 2024-09-14T09:06:16.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520148+0000", 2024-09-14T09:06:16.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7759999999999997e-06 2024-09-14T09:06:16.728 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.728 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.728 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.729 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53456 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.729 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.506448+0000", 2024-09-14T09:06:16.729 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.013603637, 2024-09-14T09:06:16.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018417493, 2024-09-14T09:06:16.729 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.729 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.729 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.729 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.729 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.729 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.729 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53456 2024-09-14T09:06:16.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.730 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506448+0000", 2024-09-14T09:06:16.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506448+0000", 2024-09-14T09:06:16.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506449+0000", 2024-09-14T09:06:16.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0130000000000001e-06 2024-09-14T09:06:16.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506456+0000", 2024-09-14T09:06:16.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5900000000000002e-06 2024-09-14T09:06:16.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506467+0000", 2024-09-14T09:06:16.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0484e-05 2024-09-14T09:06:16.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506474+0000", 2024-09-14T09:06:16.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5700000000000004e-06 2024-09-14T09:06:16.732 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.732 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.732 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.732 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53674 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.732 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.511020+0000", 2024-09-14T09:06:16.732 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.0090313240000000003, 2024-09-14T09:06:16.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013895579999999999, 2024-09-14T09:06:16.732 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.732 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.733 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.733 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.733 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.733 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.733 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53674 2024-09-14T09:06:16.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.733 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.511020+0000", 2024-09-14T09:06:16.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.511020+0000", 2024-09-14T09:06:16.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.511021+0000", 2024-09-14T09:06:16.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.39e-06 2024-09-14T09:06:16.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.511042+0000", 2024-09-14T09:06:16.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1095999999999999e-05 2024-09-14T09:06:16.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.511062+0000", 2024-09-14T09:06:16.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9375999999999999e-05 2024-09-14T09:06:16.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.511690+0000", 2024-09-14T09:06:16.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00062854199999999995 2024-09-14T09:06:16.735 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.735 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.735 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.735 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53891 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.736 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.520154+0000", 2024-09-14T09:06:16.736 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4294967295.999897, 2024-09-14T09:06:16.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004814657, 2024-09-14T09:06:16.736 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.736 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.736 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.736 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.736 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.736 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.736 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53891 2024-09-14T09:06:16.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.736 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520154+0000", 2024-09-14T09:06:16.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520154+0000", 2024-09-14T09:06:16.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520155+0000", 2024-09-14T09:06:16.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6899999999999998e-07 2024-09-14T09:06:16.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520161+0000", 2024-09-14T09:06:16.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6559999999999997e-06 2024-09-14T09:06:16.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520167+0000", 2024-09-14T09:06:16.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7039999999999997e-06 2024-09-14T09:06:16.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520172+0000", 2024-09-14T09:06:16.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.87e-06 2024-09-14T09:06:16.739 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.739 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.739 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.739 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53458 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.739 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.506485+0000", 2024-09-14T09:06:16.739 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.013565756999999999, 2024-09-14T09:06:16.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018533556, 2024-09-14T09:06:16.739 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.739 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.739 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.739 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.739 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.739 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.740 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53458 2024-09-14T09:06:16.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.740 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506485+0000", 2024-09-14T09:06:16.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506485+0000", 2024-09-14T09:06:16.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506487+0000", 2024-09-14T09:06:16.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1400000000000001e-06 2024-09-14T09:06:16.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506491+0000", 2024-09-14T09:06:16.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4299999999999999e-06 2024-09-14T09:06:16.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506507+0000", 2024-09-14T09:06:16.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6359999999999999e-05 2024-09-14T09:06:16.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506521+0000", 2024-09-14T09:06:16.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.364e-05 2024-09-14T09:06:16.742 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.742 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.742 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.742 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53702 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.742 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.518017+0000", 2024-09-14T09:06:16.742 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.002034344, 2024-09-14T09:06:16.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0070519040000000003, 2024-09-14T09:06:16.742 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.743 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.743 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.743 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.743 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.743 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.743 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53702 2024-09-14T09:06:16.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.743 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518017+0000", 2024-09-14T09:06:16.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518017+0000", 2024-09-14T09:06:16.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518019+0000", 2024-09-14T09:06:16.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4399999999999999e-06 2024-09-14T09:06:16.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518032+0000", 2024-09-14T09:06:16.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2763e-05 2024-09-14T09:06:16.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518065+0000", 2024-09-14T09:06:16.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3413e-05 2024-09-14T09:06:16.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518088+0000", 2024-09-14T09:06:16.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2152999999999999e-05 2024-09-14T09:06:16.745 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.745 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.745 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.745 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53892 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.746 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.520179+0000", 2024-09-14T09:06:16.746 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4294967295.9998722, 2024-09-14T09:06:16.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0049437689999999998, 2024-09-14T09:06:16.746 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.746 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.746 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.746 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.746 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.746 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.746 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53892 2024-09-14T09:06:16.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.746 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520179+0000", 2024-09-14T09:06:16.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520179+0000", 2024-09-14T09:06:16.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520179+0000", 2024-09-14T09:06:16.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3999999999999997e-07 2024-09-14T09:06:16.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520181+0000", 2024-09-14T09:06:16.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4339999999999999e-06 2024-09-14T09:06:16.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520184+0000", 2024-09-14T09:06:16.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-09-14T09:06:16.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.520187+0000", 2024-09-14T09:06:16.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6989999999999999e-06 2024-09-14T09:06:16.748 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.749 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.749 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.749 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53459 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.749 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.506539+0000", 2024-09-14T09:06:16.749 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.013512361000000001, 2024-09-14T09:06:16.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018633160999999999, 2024-09-14T09:06:16.749 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.749 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.749 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.749 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.749 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.749 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.749 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53459 2024-09-14T09:06:16.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.750 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506539+0000", 2024-09-14T09:06:16.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506539+0000", 2024-09-14T09:06:16.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506541+0000", 2024-09-14T09:06:16.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6869999999999999e-06 2024-09-14T09:06:16.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506546+0000", 2024-09-14T09:06:16.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0300000000000001e-06 2024-09-14T09:06:16.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506562+0000", 2024-09-14T09:06:16.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6786000000000002e-05 2024-09-14T09:06:16.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506623+0000", 2024-09-14T09:06:16.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1060000000000002e-05 2024-09-14T09:06:16.752 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.752 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.752 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.752 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53703 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.752 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.518125+0000", 2024-09-14T09:06:16.752 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.0019262820000000001, 2024-09-14T09:06:16.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071030429999999999, 2024-09-14T09:06:16.752 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.752 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.752 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.753 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.753 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.753 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.753 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53703 2024-09-14T09:06:16.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.753 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518125+0000", 2024-09-14T09:06:16.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518125+0000", 2024-09-14T09:06:16.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518127+0000", 2024-09-14T09:06:16.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5799999999999999e-06 2024-09-14T09:06:16.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518134+0000", 2024-09-14T09:06:16.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7800000000000001e-06 2024-09-14T09:06:16.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518154+0000", 2024-09-14T09:06:16.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-05 2024-09-14T09:06:16.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518171+0000", 2024-09-14T09:06:16.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6597000000000001e-05 2024-09-14T09:06:16.755 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.755 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.755 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.755 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53467 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.755 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.506647+0000", 2024-09-14T09:06:16.756 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.013404671999999999, 2024-09-14T09:06:16.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018631846000000001, 2024-09-14T09:06:16.756 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.756 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.756 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.756 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.756 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.756 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.756 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53467 2024-09-14T09:06:16.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.756 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.759 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-14T09:06:16.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506647+0000", 2024-09-14T09:06:16.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506647+0000", 2024-09-14T09:06:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506648+0000", 2024-09-14T09:06:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5200000000000001e-06 2024-09-14T09:06:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506654+0000", 2024-09-14T09:06:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9900000000000002e-06 2024-09-14T09:06:16.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506673+0000", 2024-09-14T09:06:16.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9236e-05 2024-09-14T09:06:16.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506688+0000", 2024-09-14T09:06:16.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4324e-05 2024-09-14T09:06:16.761 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.761 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.761 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.762 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53704 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.762 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.518195+0000", 2024-09-14T09:06:16.762 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.0018565389999999999, 2024-09-14T09:06:16.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072495709999999998, 2024-09-14T09:06:16.762 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.762 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.762 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.762 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.762 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.762 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.762 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53704 2024-09-14T09:06:16.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.762 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518195+0000", 2024-09-14T09:06:16.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518195+0000", 2024-09-14T09:06:16.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518196+0000", 2024-09-14T09:06:16.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3e-06 2024-09-14T09:06:16.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518202+0000", 2024-09-14T09:06:16.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0499999999999997e-06 2024-09-14T09:06:16.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518221+0000", 2024-09-14T09:06:16.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8620000000000001e-05 2024-09-14T09:06:16.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518236+0000", 2024-09-14T09:06:16.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.522e-05 2024-09-14T09:06:16.765 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.765 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.765 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.765 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53468 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.765 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.506707+0000", 2024-09-14T09:06:16.765 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.013343842, 2024-09-14T09:06:16.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018791596000000001, 2024-09-14T09:06:16.765 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.765 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.765 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.765 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.765 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.766 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.766 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53468 2024-09-14T09:06:16.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.766 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506707+0000", 2024-09-14T09:06:16.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506707+0000", 2024-09-14T09:06:16.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506709+0000", 2024-09-14T09:06:16.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4699999999999999e-06 2024-09-14T09:06:16.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506715+0000", 2024-09-14T09:06:16.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8599999999999998e-06 2024-09-14T09:06:16.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506733+0000", 2024-09-14T09:06:16.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7912999999999999e-05 2024-09-14T09:06:16.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506746+0000", 2024-09-14T09:06:16.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3743e-05 2024-09-14T09:06:16.768 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.768 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.768 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.768 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53705 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.768 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.518255+0000", 2024-09-14T09:06:16.768 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.0017967199999999999, 2024-09-14T09:06:16.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007299603, 2024-09-14T09:06:16.769 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.769 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.769 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.769 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.769 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.769 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.769 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53705 2024-09-14T09:06:16.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.769 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518255+0000", 2024-09-14T09:06:16.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518255+0000", 2024-09-14T09:06:16.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518256+0000", 2024-09-14T09:06:16.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2470000000000001e-06 2024-09-14T09:06:16.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518261+0000", 2024-09-14T09:06:16.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1529999999999996e-06 2024-09-14T09:06:16.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518280+0000", 2024-09-14T09:06:16.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8737e-05 2024-09-14T09:06:16.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518294+0000", 2024-09-14T09:06:16.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4663e-05 2024-09-14T09:06:16.771 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.771 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.771 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.772 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53469 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.772 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.506766+0000", 2024-09-14T09:06:16.772 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.013285478999999999, 2024-09-14T09:06:16.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018843959, 2024-09-14T09:06:16.772 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.772 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.772 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.772 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.772 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.772 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.772 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53469 2024-09-14T09:06:16.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.772 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506766+0000", 2024-09-14T09:06:16.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506766+0000", 2024-09-14T09:06:16.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506767+0000", 2024-09-14T09:06:16.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2899999999999999e-06 2024-09-14T09:06:16.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506772+0000", 2024-09-14T09:06:16.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1059999999999999e-06 2024-09-14T09:06:16.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506790+0000", 2024-09-14T09:06:16.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7883000000000001e-05 2024-09-14T09:06:16.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506803+0000", 2024-09-14T09:06:16.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.34e-05 2024-09-14T09:06:16.775 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.775 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.775 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.775 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53706 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.775 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.518314+0000", 2024-09-14T09:06:16.776 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.0017371260000000001, 2024-09-14T09:06:16.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073492160000000004, 2024-09-14T09:06:16.776 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.776 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.776 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.776 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.776 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.776 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.776 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53706 2024-09-14T09:06:16.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.776 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518314+0000", 2024-09-14T09:06:16.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518314+0000", 2024-09-14T09:06:16.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518315+0000", 2024-09-14T09:06:16.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3030000000000001e-06 2024-09-14T09:06:16.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518321+0000", 2024-09-14T09:06:16.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1529999999999996e-06 2024-09-14T09:06:16.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518338+0000", 2024-09-14T09:06:16.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.753e-05 2024-09-14T09:06:16.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518352+0000", 2024-09-14T09:06:16.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4270000000000001e-05 2024-09-14T09:06:16.778 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.778 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.779 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.779 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53471 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.779 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.506858+0000", 2024-09-14T09:06:16.779 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.013192802999999999, 2024-09-14T09:06:16.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018855736000000001, 2024-09-14T09:06:16.779 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.779 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.779 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.779 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.779 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.779 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.779 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53471 2024-09-14T09:06:16.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.780 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506858+0000", 2024-09-14T09:06:16.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506858+0000", 2024-09-14T09:06:16.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506860+0000", 2024-09-14T09:06:16.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.423e-06 2024-09-14T09:06:16.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506866+0000", 2024-09-14T09:06:16.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2299999999999996e-06 2024-09-14T09:06:16.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506887+0000", 2024-09-14T09:06:16.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1386000000000002e-05 2024-09-14T09:06:16.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506903+0000", 2024-09-14T09:06:16.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5359999999999999e-05 2024-09-14T09:06:16.782 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.782 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.782 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.782 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53734 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.782 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.518372+0000", 2024-09-14T09:06:16.782 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.0016792770000000001, 2024-09-14T09:06:16.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073959550000000001, 2024-09-14T09:06:16.782 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.783 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.783 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.783 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.783 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.783 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.783 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53734 2024-09-14T09:06:16.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.783 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518372+0000", 2024-09-14T09:06:16.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518372+0000", 2024-09-14T09:06:16.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518373+0000", 2024-09-14T09:06:16.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2100000000000001e-06 2024-09-14T09:06:16.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518379+0000", 2024-09-14T09:06:16.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8900000000000004e-06 2024-09-14T09:06:16.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518397+0000", 2024-09-14T09:06:16.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8306000000000001e-05 2024-09-14T09:06:16.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518412+0000", 2024-09-14T09:06:16.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.473e-05 2024-09-14T09:06:16.785 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.785 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.786 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.786 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53510 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.786 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.506924+0000", 2024-09-14T09:06:16.786 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.013127554, 2024-09-14T09:06:16.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018894832, 2024-09-14T09:06:16.786 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.786 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.786 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.786 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.786 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.786 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.787 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53510 2024-09-14T09:06:16.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.787 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506924+0000", 2024-09-14T09:06:16.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506924+0000", 2024-09-14T09:06:16.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506925+0000", 2024-09-14T09:06:16.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3799999999999999e-06 2024-09-14T09:06:16.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506930+0000", 2024-09-14T09:06:16.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3530000000000002e-06 2024-09-14T09:06:16.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506949+0000", 2024-09-14T09:06:16.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8576999999999998e-05 2024-09-14T09:06:16.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506963+0000", 2024-09-14T09:06:16.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.395e-05 2024-09-14T09:06:16.789 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.789 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.789 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.789 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53735 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.789 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.518430+0000", 2024-09-14T09:06:16.789 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.0016212310000000001, 2024-09-14T09:06:16.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0074412239999999998, 2024-09-14T09:06:16.790 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.790 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.790 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.790 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.790 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.790 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.790 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53735 2024-09-14T09:06:16.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.790 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518430+0000", 2024-09-14T09:06:16.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518430+0000", 2024-09-14T09:06:16.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518431+0000", 2024-09-14T09:06:16.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1939999999999999e-06 2024-09-14T09:06:16.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518437+0000", 2024-09-14T09:06:16.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8159999999999999e-06 2024-09-14T09:06:16.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518455+0000", 2024-09-14T09:06:16.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8383e-05 2024-09-14T09:06:16.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518469+0000", 2024-09-14T09:06:16.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3553000000000001e-05 2024-09-14T09:06:16.792 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.792 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.792 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.793 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53512 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.793 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.506982+0000", 2024-09-14T09:06:16.793 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.013068903999999999, 2024-09-14T09:06:16.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018939040000000001, 2024-09-14T09:06:16.793 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.793 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.793 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.793 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.793 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.793 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.793 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53512 2024-09-14T09:06:16.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.794 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506982+0000", 2024-09-14T09:06:16.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506982+0000", 2024-09-14T09:06:16.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506984+0000", 2024-09-14T09:06:16.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4729999999999999e-06 2024-09-14T09:06:16.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.506989+0000", 2024-09-14T09:06:16.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9869999999999999e-06 2024-09-14T09:06:16.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.507007+0000", 2024-09-14T09:06:16.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8096000000000001e-05 2024-09-14T09:06:16.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.507022+0000", 2024-09-14T09:06:16.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4776e-05 2024-09-14T09:06:16.796 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.796 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.796 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.796 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53736 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.796 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.518488+0000", 2024-09-14T09:06:16.796 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.0015633279999999999, 2024-09-14T09:06:16.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0074874249999999998, 2024-09-14T09:06:16.796 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.796 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.796 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.796 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.797 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.797 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.797 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53736 2024-09-14T09:06:16.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.797 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518488+0000", 2024-09-14T09:06:16.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518488+0000", 2024-09-14T09:06:16.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518489+0000", 2024-09-14T09:06:16.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2699999999999999e-06 2024-09-14T09:06:16.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518494+0000", 2024-09-14T09:06:16.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.006e-06 2024-09-14T09:06:16.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518512+0000", 2024-09-14T09:06:16.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7629999999999999e-05 2024-09-14T09:06:16.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518525+0000", 2024-09-14T09:06:16.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3114e-05 2024-09-14T09:06:16.799 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.799 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.799 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.799 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53514 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.799 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.507042+0000", 2024-09-14T09:06:16.800 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.013009425, 2024-09-14T09:06:16.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.018984207999999999, 2024-09-14T09:06:16.800 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.800 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.800 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.800 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.800 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.800 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.800 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53514 2024-09-14T09:06:16.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.800 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.507042+0000", 2024-09-14T09:06:16.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.507042+0000", 2024-09-14T09:06:16.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.507043+0000", 2024-09-14T09:06:16.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5659999999999999e-06 2024-09-14T09:06:16.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.507061+0000", 2024-09-14T09:06:16.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7224e-05 2024-09-14T09:06:16.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.507079+0000", 2024-09-14T09:06:16.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8423e-05 2024-09-14T09:06:16.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.507093+0000", 2024-09-14T09:06:16.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3487e-05 2024-09-14T09:06:16.803 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.803 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.803 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.803 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53737 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.803 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.518547+0000", 2024-09-14T09:06:16.803 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.001504348, 2024-09-14T09:06:16.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007528542, 2024-09-14T09:06:16.803 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.803 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.803 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.803 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.803 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.804 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.804 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53737 2024-09-14T09:06:16.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.804 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518547+0000", 2024-09-14T09:06:16.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518547+0000", 2024-09-14T09:06:16.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518548+0000", 2024-09-14T09:06:16.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2160000000000001e-06 2024-09-14T09:06:16.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518577+0000", 2024-09-14T09:06:16.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9349999999999999e-05 2024-09-14T09:06:16.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518598+0000", 2024-09-14T09:06:16.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0752999999999999e-05 2024-09-14T09:06:16.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518613+0000", 2024-09-14T09:06:16.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4555999999999999e-05 2024-09-14T09:06:16.806 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.806 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.806 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.806 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53516 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.806 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.507112+0000", 2024-09-14T09:06:16.807 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.012939009, 2024-09-14T09:06:16.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.019017487, 2024-09-14T09:06:16.807 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.807 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.807 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.807 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.807 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.807 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.807 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53516 2024-09-14T09:06:16.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.807 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.507112+0000", 2024-09-14T09:06:16.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.507112+0000", 2024-09-14T09:06:16.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.507114+0000", 2024-09-14T09:06:16.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3370000000000001e-06 2024-09-14T09:06:16.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.507119+0000", 2024-09-14T09:06:16.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4600000000000002e-06 2024-09-14T09:06:16.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.507137+0000", 2024-09-14T09:06:16.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7935999999999999e-05 2024-09-14T09:06:16.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.507150+0000", 2024-09-14T09:06:16.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3509999999999999e-05 2024-09-14T09:06:16.809 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.809 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.810 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.810 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53738 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.810 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.518633+0000", 2024-09-14T09:06:16.810 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.001418309, 2024-09-14T09:06:16.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075463420000000002, 2024-09-14T09:06:16.810 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.810 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.810 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.810 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.810 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.810 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.810 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53738 2024-09-14T09:06:16.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.811 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518633+0000", 2024-09-14T09:06:16.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518633+0000", 2024-09-14T09:06:16.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518634+0000", 2024-09-14T09:06:16.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3629999999999999e-06 2024-09-14T09:06:16.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518640+0000", 2024-09-14T09:06:16.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4369999999999998e-06 2024-09-14T09:06:16.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518657+0000", 2024-09-14T09:06:16.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7399999999999999e-05 2024-09-14T09:06:16.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518671+0000", 2024-09-14T09:06:16.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.377e-05 2024-09-14T09:06:16.813 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.813 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.813 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.813 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53533 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.813 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508963+0000", 2024-09-14T09:06:16.813 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.011088117, 2024-09-14T09:06:16.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017268015000000001, 2024-09-14T09:06:16.813 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.813 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.814 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.814 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.814 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.814 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.814 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53533 2024-09-14T09:06:16.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.814 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508963+0000", 2024-09-14T09:06:16.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508963+0000", 2024-09-14T09:06:16.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508965+0000", 2024-09-14T09:06:16.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6029999999999999e-06 2024-09-14T09:06:16.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508973+0000", 2024-09-14T09:06:16.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6500000000000002e-06 2024-09-14T09:06:16.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509003+0000", 2024-09-14T09:06:16.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9853e-05 2024-09-14T09:06:16.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509023+0000", 2024-09-14T09:06:16.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9856000000000001e-05 2024-09-14T09:06:16.816 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.816 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.816 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.817 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53766 3.1d 3.2f57451d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:16.817 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.518690+0000", 2024-09-14T09:06:16.817 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.0013615229999999999, 2024-09-14T09:06:16.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075915210000000004, 2024-09-14T09:06:16.817 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:16.817 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:16.817 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:16.817 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:16.817 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:16.817 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:16.817 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53766 2024-09-14T09:06:16.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.817 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:16.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:16.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518690+0000", 2024-09-14T09:06:16.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:16.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518690+0000", 2024-09-14T09:06:16.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:16.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:16.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518691+0000", 2024-09-14T09:06:16.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1739999999999999e-06 2024-09-14T09:06:16.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:16.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518696+0000", 2024-09-14T09:06:16.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9429999999999999e-06 2024-09-14T09:06:16.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:16.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518714+0000", 2024-09-14T09:06:16.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8638999999999999e-05 2024-09-14T09:06:16.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:16.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:16.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:16.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518733+0000", 2024-09-14T09:06:16.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8700000000000001e-05 2024-09-14T09:06:16.820 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.820 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:16.820 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.820 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:16.820 INFO:teuthology.orchestra.run.smithi183.stdout: ], 2024-09-14T09:06:16.820 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 86 2024-09-14T09:06:16.820 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:16.821 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:06:16.821 INFO:teuthology.orchestra.run.smithi176.stdout:2116 read 0x7ed70a5 thru 0x7ee0072 (0x8fce bytes) 2024-09-14T09:06:16.821 INFO:teuthology.orchestra.run.smithi176.stdout:2117 punch from 0x2e7bcba to 0x2e84266, (0x85ac bytes) 2024-09-14T09:06:16.821 INFO:teuthology.orchestra.run.smithi176.stdout:2119 writesame 0xd38cfa3 thru 0xd398e99 data_size 0x13(0xbef7 bytes) 2024-09-14T09:06:16.822 INFO:teuthology.orchestra.run.smithi176.stdout:2121 read 0x2d5e829 thru 0x2d6cb13 (0xe2eb bytes) 2024-09-14T09:06:16.822 INFO:teuthology.orchestra.run.smithi176.stdout:2122 writesame 0xa4cfc0 thru 0xa538aa data_size 0x7(0x68eb bytes) 2024-09-14T09:06:16.822 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:16.822 INFO:teuthology.orchestra.run.smithi176.stdout:2125 punch from 0x1cb4503 to 0x1cc2126, (0xdc23 bytes) 2024-09-14T09:06:16.822 INFO:teuthology.orchestra.run.smithi176.stdout:2126 trunc from 0xd398e9a to 0x78dd9b 2024-09-14T09:06:16.822 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:16.755+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:16.823 INFO:teuthology.orchestra.run.smithi176.stdout:2128 write 0xc220cd1 thru 0xc22c7b5 (0xbae5 bytes) 2024-09-14T09:06:16.823 INFO:teuthology.orchestra.run.smithi176.stdout:2129 read 0x7588524 thru 0x75918f7 (0x93d4 bytes) 2024-09-14T09:06:16.823 INFO:teuthology.orchestra.run.smithi176.stdout:2130 read 0xba639e8 thru 0xba6ab73 (0x718c bytes) 2024-09-14T09:06:16.824 INFO:teuthology.orchestra.run.smithi176.stdout:2131 write 0x383fc4d thru 0x3848acd (0x8e81 bytes) 2024-09-14T09:06:16.837 INFO:teuthology.orchestra.run.smithi176.stdout:2132 read 0x1926153 thru 0x193121f (0xb0cd bytes) 2024-09-14T09:06:16.837 INFO:teuthology.orchestra.run.smithi176.stdout:2134 writesame 0xa696e38 thru 0xa6a6a92 data_size 0xb(0xfc5b bytes) 2024-09-14T09:06:16.854 INFO:teuthology.orchestra.run.smithi176.stdout:2135 punch from 0x8b511d8 to 0x8b53dd9, (0x2c01 bytes) 2024-09-14T09:06:16.854 INFO:teuthology.orchestra.run.smithi176.stdout:2137 write 0xa664907 thru 0xa66f2b6 (0xa9b0 bytes) 2024-09-14T09:06:16.858 INFO:teuthology.orchestra.run.smithi176.stdout:2139 read 0x64dd44e thru 0x64e3203 (0x5db6 bytes) 2024-09-14T09:06:16.866 INFO:teuthology.orchestra.run.smithi176.stdout:2142 trunc from 0xc22c7b6 to 0x62e7239 2024-09-14T09:06:16.867 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:16.871+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:16.881 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:16.881 INFO:teuthology.orchestra.run.smithi176.stdout:2145 write 0x5e59e28 thru 0x5e5ca77 (0x2c50 bytes) 2024-09-14T09:06:16.889 INFO:teuthology.orchestra.run.smithi176.stdout:2147 punch from 0x33c38a6 to 0x33c4461, (0xbbb bytes) 2024-09-14T09:06:16.890 INFO:teuthology.orchestra.run.smithi176.stdout:2148 write 0x17ba593 thru 0x17c4e35 (0xa8a3 bytes) 2024-09-14T09:06:16.909 INFO:teuthology.orchestra.run.smithi176.stdout:2149 trunc from 0x62e7239 to 0x9a6d818 2024-09-14T09:06:16.914 INFO:teuthology.orchestra.run.smithi176.stdout:2150 read 0x97c39f9 thru 0x97cbb55 (0x815d bytes) 2024-09-14T09:06:16.915 INFO:teuthology.orchestra.run.smithi176.stdout:2151 punch from 0x497c66f to 0x497def7, (0x1888 bytes) 2024-09-14T09:06:16.915 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:16.915 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:16.915 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:16.921 INFO:teuthology.orchestra.run.smithi176.stdout:2155 write 0x9fe3f63 thru 0x9febc2b (0x7cc9 bytes) 2024-09-14T09:06:16.936 INFO:teuthology.orchestra.run.smithi176.stdout:2156 punch from 0x3cf2a07 to 0x3cf609b, (0x3694 bytes) 2024-09-14T09:06:16.942 INFO:teuthology.orchestra.run.smithi176.stdout:2157 trunc from 0x9febc2c to 0x5783262 2024-09-14T09:06:16.959 INFO:teuthology.orchestra.run.smithi176.stdout:2158 write 0x19877c0 thru 0x198aa64 (0x32a5 bytes) 2024-09-14T09:06:16.959 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:16.963+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:16.963 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:16.963 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:16.963 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:16.963 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:16.963 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:16.968 INFO:teuthology.orchestra.run.smithi176.stdout:2159 punch from 0x1cebba4 to 0x1cf4bfc, (0x9058 bytes) 2024-09-14T09:06:16.976 INFO:teuthology.orchestra.run.smithi176.stdout:2160 trunc from 0x5783262 to 0xbe1cbd0 2024-09-14T09:06:16.979 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:06:16.984 INFO:teuthology.orchestra.run.smithi176.stdout:2161 read 0x1f9bdfa thru 0x1fa7952 (0xbb59 bytes) 2024-09-14T09:06:16.989 INFO:teuthology.orchestra.run.smithi176.stdout:2163 trunc from 0xbe1cbd0 to 0x65af67e 2024-09-14T09:06:16.996 INFO:teuthology.orchestra.run.smithi176.stdout:2164 read 0x306c608 thru 0x3077d89 (0xb782 bytes) 2024-09-14T09:06:16.996 INFO:teuthology.orchestra.run.smithi176.stdout:2165 read 0x2519d80 thru 0x2526265 (0xc4e6 bytes) 2024-09-14T09:06:16.996 INFO:teuthology.orchestra.run.smithi176.stdout:2166 read 0x19af950 thru 0x19b5d12 (0x63c3 bytes) 2024-09-14T09:06:16.997 INFO:teuthology.orchestra.run.smithi176.stdout:2167 read 0x1d9831d thru 0x1da30b8 (0xad9c bytes) 2024-09-14T09:06:16.998 INFO:teuthology.orchestra.run.smithi176.stdout:2168 trunc from 0x65af67e to 0x5e9e469 2024-09-14T09:06:17.004 INFO:teuthology.orchestra.run.smithi176.stdout:2169 write 0xe58261 thru 0xe5d2bf (0x505f bytes) 2024-09-14T09:06:17.015 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-14T09:06:17.022 INFO:teuthology.orchestra.run.smithi176.stdout:2170 trunc from 0x5e9e469 to 0x9352247 2024-09-14T09:06:17.029 INFO:teuthology.orchestra.run.smithi176.stdout:2171 writesame 0x6f0f75e thru 0x6f14aa6 data_size 0x45(0x5349 bytes) 2024-09-14T09:06:17.057 INFO:teuthology.orchestra.run.smithi176.stdout:2172 writesame 0xb79d569 thru 0xb7ac5ad data_size 0x65(0xf045 bytes) 2024-09-14T09:06:17.057 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:17.057 INFO:teuthology.orchestra.run.smithi176.stdout:2176 read 0x81a45bd thru 0x81a4703 (0x147 bytes) 2024-09-14T09:06:17.061 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:17.059+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:17.061 INFO:teuthology.orchestra.run.smithi176.stdout:2178 trunc from 0xb7ac5ae to 0xeae9cba 2024-09-14T09:06:17.067 INFO:teuthology.orchestra.run.smithi176.stdout:2179 punch from 0x5b50831 to 0x5b50db0, (0x57f bytes) 2024-09-14T09:06:17.077 INFO:teuthology.orchestra.run.smithi176.stdout:2181 trunc from 0xeae9cba to 0x6b49218 2024-09-14T09:06:17.097 INFO:teuthology.orchestra.run.smithi176.stdout:2182 writesame 0x83244c3 thru 0x8331cdc data_size 0x2(0xd81a bytes) 2024-09-14T09:06:17.097 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:17.097 INFO:teuthology.orchestra.run.smithi176.stdout:2184 write 0x371b66d thru 0x372152f (0x5ec3 bytes) 2024-09-14T09:06:17.113 INFO:teuthology.orchestra.run.smithi176.stdout:2185 write 0xbafb796 thru 0xbb0b4c3 (0xfd2e bytes) 2024-09-14T09:06:17.127 INFO:teuthology.orchestra.run.smithi176.stdout:2186 write 0x1f4f6ff thru 0x1f5c952 (0xd254 bytes) 2024-09-14T09:06:17.144 INFO:teuthology.orchestra.run.smithi176.stdout:2188 write 0x16aebe1 thru 0x16b29d5 (0x3df5 bytes) 2024-09-14T09:06:17.149 INFO:teuthology.orchestra.run.smithi176.stdout:2189 punch from 0x3948ff3 to 0x3950f05, (0x7f12 bytes) 2024-09-14T09:06:17.149 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is set 2024-09-14T09:06:17.150 INFO:teuthology.orchestra.run.smithi176.stdout:2190 punch from 0x67ef7ee to 0x67efd2f, (0x541 bytes) 2024-09-14T09:06:17.161 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:17.161 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:17.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:17.161 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:17.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.162 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49333 3.1d 3:be596f9e:::rbd_data.115e51d92ed2.0000000000000007:12 [read 950272~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:17.162 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:17.162 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.6169356720000003, 2024-09-14T09:06:17.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072172603000000002, 2024-09-14T09:06:17.162 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.162 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.162 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:17.162 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.162 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.162 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.162 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49333 2024-09-14T09:06:17.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.162 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:17.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:17.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:17.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6000000000000003e-07 2024-09-14T09:06:17.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546216+0000", 2024-09-14T09:06:17.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5339999999999998e-06 2024-09-14T09:06:17.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546224+0000", 2024-09-14T09:06:17.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7770000000000002e-06 2024-09-14T09:06:17.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546230+0000", 2024-09-14T09:06:17.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3400000000000003e-06 2024-09-14T09:06:17.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.617881+0000", 2024-09-14T09:06:17.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071650488999999998 2024-09-14T09:06:17.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:17.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.617958+0000", 2024-09-14T09:06:17.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7018e-05 2024-09-14T09:06:17.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618385+0000", 2024-09-14T09:06:17.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00042768500000000002 2024-09-14T09:06:17.166 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.166 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.166 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.166 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49334 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:17.166 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:17.166 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.6169087900000001, 2024-09-14T09:06:17.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072227886000000005, 2024-09-14T09:06:17.166 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.166 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.167 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:17.167 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.167 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.167 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.167 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49334 2024-09-14T09:06:17.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.167 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:17.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:17.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:17.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8899999999999999e-07 2024-09-14T09:06:17.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546243+0000", 2024-09-14T09:06:17.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.503e-06 2024-09-14T09:06:17.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546253+0000", 2024-09-14T09:06:17.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0210999999999999e-05 2024-09-14T09:06:17.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546267+0000", 2024-09-14T09:06:17.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4183e-05 2024-09-14T09:06:17.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618449+0000", 2024-09-14T09:06:17.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072181828000000003 2024-09-14T09:06:17.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618467+0000", 2024-09-14T09:06:17.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8471999999999999e-05 2024-09-14T09:06:17.170 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.170 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.170 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.170 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49335 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:17.170 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:17.170 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.6168726390000003, 2024-09-14T09:06:17.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072255640999999995, 2024-09-14T09:06:17.170 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.171 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.171 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:17.171 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.171 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.171 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.171 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49335 2024-09-14T09:06:17.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.171 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:17.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:17.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:17.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9999999999999997e-07 2024-09-14T09:06:17.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546279+0000", 2024-09-14T09:06:17.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.695e-06 2024-09-14T09:06:17.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546288+0000", 2024-09-14T09:06:17.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7110000000000001e-06 2024-09-14T09:06:17.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546294+0000", 2024-09-14T09:06:17.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9830000000000001e-06 2024-09-14T09:06:17.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618521+0000", 2024-09-14T09:06:17.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072226948999999999 2024-09-14T09:06:17.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618531+0000", 2024-09-14T09:06:17.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0603e-05 2024-09-14T09:06:17.174 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.174 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.174 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49336 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.6168467360000003, 2024-09-14T09:06:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072292223000000003, 2024-09-14T09:06:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49336 2024-09-14T09:06:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.175 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:17.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:17.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:17.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4000000000000001e-07 2024-09-14T09:06:17.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546305+0000", 2024-09-14T09:06:17.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3539999999999998e-06 2024-09-14T09:06:17.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546315+0000", 2024-09-14T09:06:17.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0431999999999999e-05 2024-09-14T09:06:17.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546321+0000", 2024-09-14T09:06:17.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7620000000000001e-06 2024-09-14T09:06:17.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618583+0000", 2024-09-14T09:06:17.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072262046999999996 2024-09-14T09:06:17.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618594+0000", 2024-09-14T09:06:17.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0988000000000001e-05 2024-09-14T09:06:17.178 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.179 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.179 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.179 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49337 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:17.179 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:17.179 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.6167991219999998, 2024-09-14T09:06:17.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072294774000000006, 2024-09-14T09:06:17.179 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.179 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.179 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:17.179 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.179 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.179 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.180 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49337 2024-09-14T09:06:17.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.180 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:17.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:17.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546350+0000", 2024-09-14T09:06:17.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4600000000000004e-07 2024-09-14T09:06:17.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546381+0000", 2024-09-14T09:06:17.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0902000000000003e-05 2024-09-14T09:06:17.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546412+0000", 2024-09-14T09:06:17.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0825999999999999e-05 2024-09-14T09:06:17.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546438+0000", 2024-09-14T09:06:17.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6410999999999998e-05 2024-09-14T09:06:17.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618637+0000", 2024-09-14T09:06:17.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072198907000000007 2024-09-14T09:06:17.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618644+0000", 2024-09-14T09:06:17.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9820000000000002e-06 2024-09-14T09:06:17.183 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.183 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.183 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.183 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49338 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:17.183 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:17.183 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.6166850950000002, 2024-09-14T09:06:17.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072233250999999998, 2024-09-14T09:06:17.183 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.183 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.183 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:17.183 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.183 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.184 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.184 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49338 2024-09-14T09:06:17.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.184 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:17.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:17.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546465+0000", 2024-09-14T09:06:17.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.455e-06 2024-09-14T09:06:17.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546481+0000", 2024-09-14T09:06:17.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5834000000000001e-05 2024-09-14T09:06:17.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546498+0000", 2024-09-14T09:06:17.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6937000000000001e-05 2024-09-14T09:06:17.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546508+0000", 2024-09-14T09:06:17.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0446e-05 2024-09-14T09:06:17.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618685+0000", 2024-09-14T09:06:17.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072177280999999996 2024-09-14T09:06:17.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618697+0000", 2024-09-14T09:06:17.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1297999999999999e-05 2024-09-14T09:06:17.187 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.187 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.187 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.187 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49339 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:17.187 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546535+0000", 2024-09-14T09:06:17.187 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.6166130780000003, 2024-09-14T09:06:17.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072197266999999996, 2024-09-14T09:06:17.187 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.187 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.187 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:17.187 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.188 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.188 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.188 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49339 2024-09-14T09:06:17.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.188 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546535+0000", 2024-09-14T09:06:17.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546535+0000", 2024-09-14T09:06:17.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546536+0000", 2024-09-14T09:06:17.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1399999999999996e-07 2024-09-14T09:06:17.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546539+0000", 2024-09-14T09:06:17.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.089e-06 2024-09-14T09:06:17.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546560+0000", 2024-09-14T09:06:17.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1217e-05 2024-09-14T09:06:17.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546567+0000", 2024-09-14T09:06:17.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9e-06 2024-09-14T09:06:17.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618726+0000", 2024-09-14T09:06:17.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.07215887 2024-09-14T09:06:17.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618733+0000", 2024-09-14T09:06:17.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3770000000000001e-06 2024-09-14T09:06:17.191 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.191 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.191 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.191 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49340 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:17.191 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546578+0000", 2024-09-14T09:06:17.191 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.6165701869999998, 2024-09-14T09:06:17.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072205187000000004, 2024-09-14T09:06:17.192 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.192 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.192 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:17.192 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.192 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.192 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.192 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49340 2024-09-14T09:06:17.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.192 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546578+0000", 2024-09-14T09:06:17.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546578+0000", 2024-09-14T09:06:17.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546579+0000", 2024-09-14T09:06:17.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1999999999999999e-07 2024-09-14T09:06:17.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546582+0000", 2024-09-14T09:06:17.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4460000000000001e-06 2024-09-14T09:06:17.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546603+0000", 2024-09-14T09:06:17.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0502000000000001e-05 2024-09-14T09:06:17.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546610+0000", 2024-09-14T09:06:17.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7150000000000001e-06 2024-09-14T09:06:17.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618770+0000", 2024-09-14T09:06:17.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072160600000000005 2024-09-14T09:06:17.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618783+0000", 2024-09-14T09:06:17.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:06:17.195 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.195 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.195 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.195 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49341 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:17.196 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546667+0000", 2024-09-14T09:06:17.196 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.6164810789999997, 2024-09-14T09:06:17.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072181163000000007, 2024-09-14T09:06:17.196 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.196 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.196 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:17.196 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.196 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.196 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.196 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49341 2024-09-14T09:06:17.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.196 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546667+0000", 2024-09-14T09:06:17.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546667+0000", 2024-09-14T09:06:17.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546669+0000", 2024-09-14T09:06:17.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.486e-06 2024-09-14T09:06:17.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546674+0000", 2024-09-14T09:06:17.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5110000000000003e-06 2024-09-14T09:06:17.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546691+0000", 2024-09-14T09:06:17.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7051e-05 2024-09-14T09:06:17.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546703+0000", 2024-09-14T09:06:17.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1155000000000001e-05 2024-09-14T09:06:17.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618836+0000", 2024-09-14T09:06:17.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072133616999999997 2024-09-14T09:06:17.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618848+0000", 2024-09-14T09:06:17.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2343e-05 2024-09-14T09:06:17.199 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.199 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.199 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.199 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49386 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:17.200 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546806+0000", 2024-09-14T09:06:17.200 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.61634192, 2024-09-14T09:06:17.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072019206000000002, 2024-09-14T09:06:17.200 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.200 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.200 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:17.200 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.200 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.200 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.200 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49386 2024-09-14T09:06:17.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.200 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546806+0000", 2024-09-14T09:06:17.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546806+0000", 2024-09-14T09:06:17.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546808+0000", 2024-09-14T09:06:17.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.471e-06 2024-09-14T09:06:17.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546813+0000", 2024-09-14T09:06:17.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1800000000000004e-06 2024-09-14T09:06:17.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546830+0000", 2024-09-14T09:06:17.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.658e-05 2024-09-14T09:06:17.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546842+0000", 2024-09-14T09:06:17.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2442999999999999e-05 2024-09-14T09:06:17.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618816+0000", 2024-09-14T09:06:17.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071973920999999996 2024-09-14T09:06:17.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618826+0000", 2024-09-14T09:06:17.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.611e-06 2024-09-14T09:06:17.203 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.203 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.203 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.203 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49387 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:17.204 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546861+0000", 2024-09-14T09:06:17.204 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.6162872750000004, 2024-09-14T09:06:17.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071996665000000001, 2024-09-14T09:06:17.204 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.204 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.204 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:17.204 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.204 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.204 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.204 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49387 2024-09-14T09:06:17.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.204 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546861+0000", 2024-09-14T09:06:17.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546861+0000", 2024-09-14T09:06:17.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546862+0000", 2024-09-14T09:06:17.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.186e-06 2024-09-14T09:06:17.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546867+0000", 2024-09-14T09:06:17.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9049999999999996e-06 2024-09-14T09:06:17.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546880+0000", 2024-09-14T09:06:17.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2815e-05 2024-09-14T09:06:17.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546889+0000", 2024-09-14T09:06:17.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5420000000000005e-06 2024-09-14T09:06:17.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618850+0000", 2024-09-14T09:06:17.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071961065000000005 2024-09-14T09:06:17.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618858+0000", 2024-09-14T09:06:17.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1519999999999992e-06 2024-09-14T09:06:17.207 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.207 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.207 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.208 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49388 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:17.208 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:17.208 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.6162495899999998, 2024-09-14T09:06:17.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072001645000000003, 2024-09-14T09:06:17.208 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.208 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.208 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:17.208 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.208 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.208 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.208 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49388 2024-09-14T09:06:17.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.208 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:17.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:17.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:17.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:06:17.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546902+0000", 2024-09-14T09:06:17.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.734e-06 2024-09-14T09:06:17.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546910+0000", 2024-09-14T09:06:17.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9629999999999998e-06 2024-09-14T09:06:17.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546916+0000", 2024-09-14T09:06:17.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.28e-06 2024-09-14T09:06:17.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618889+0000", 2024-09-14T09:06:17.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071972555999999993 2024-09-14T09:06:17.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618900+0000", 2024-09-14T09:06:17.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1413999999999999e-05 2024-09-14T09:06:17.211 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.211 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.211 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.212 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49389 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:17.212 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546925+0000", 2024-09-14T09:06:17.212 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.6162234299999998, 2024-09-14T09:06:17.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072026707999999995, 2024-09-14T09:06:17.212 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.212 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.212 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:17.212 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.212 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.212 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.212 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49389 2024-09-14T09:06:17.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.213 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546925+0000", 2024-09-14T09:06:17.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546925+0000", 2024-09-14T09:06:17.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546926+0000", 2024-09-14T09:06:17.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8599999999999998e-07 2024-09-14T09:06:17.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546928+0000", 2024-09-14T09:06:17.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4200000000000001e-06 2024-09-14T09:06:17.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546938+0000", 2024-09-14T09:06:17.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5859999999999994e-06 2024-09-14T09:06:17.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546945+0000", 2024-09-14T09:06:17.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1770000000000001e-06 2024-09-14T09:06:17.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618940+0000", 2024-09-14T09:06:17.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071995491999999994 2024-09-14T09:06:17.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618952+0000", 2024-09-14T09:06:17.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1347e-05 2024-09-14T09:06:17.216 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.216 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.216 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.216 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49390 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:17.216 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547051+0000", 2024-09-14T09:06:17.216 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.6160978400000001, 2024-09-14T09:06:17.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071963947, 2024-09-14T09:06:17.216 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.216 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.216 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:17.217 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.217 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.217 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.217 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49390 2024-09-14T09:06:17.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.217 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547051+0000", 2024-09-14T09:06:17.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547051+0000", 2024-09-14T09:06:17.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547053+0000", 2024-09-14T09:06:17.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.007e-06 2024-09-14T09:06:17.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547064+0000", 2024-09-14T09:06:17.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1146e-05 2024-09-14T09:06:17.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547089+0000", 2024-09-14T09:06:17.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.563e-05 2024-09-14T09:06:17.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547127+0000", 2024-09-14T09:06:17.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8040000000000002e-05 2024-09-14T09:06:17.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619003+0000", 2024-09-14T09:06:17.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071875724000000002 2024-09-14T09:06:17.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619014+0000", 2024-09-14T09:06:17.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1399999999999999e-05 2024-09-14T09:06:17.220 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.220 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.220 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.220 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49391 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:17.220 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547162+0000", 2024-09-14T09:06:17.220 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.6159866770000004, 2024-09-14T09:06:17.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071911484999999997, 2024-09-14T09:06:17.220 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.220 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.221 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:17.221 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.221 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.221 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.221 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49391 2024-09-14T09:06:17.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.221 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547162+0000", 2024-09-14T09:06:17.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547162+0000", 2024-09-14T09:06:17.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547163+0000", 2024-09-14T09:06:17.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.73e-06 2024-09-14T09:06:17.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547170+0000", 2024-09-14T09:06:17.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0400000000000004e-06 2024-09-14T09:06:17.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547190+0000", 2024-09-14T09:06:17.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9423e-05 2024-09-14T09:06:17.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547204+0000", 2024-09-14T09:06:17.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4175999999999999e-05 2024-09-14T09:06:17.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.224 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619060+0000", 2024-09-14T09:06:17.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071856347000000001 2024-09-14T09:06:17.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.224 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.224 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619073+0000", 2024-09-14T09:06:17.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2768999999999999e-05 2024-09-14T09:06:17.224 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.224 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.224 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.224 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49392 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:17.225 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547226+0000", 2024-09-14T09:06:17.225 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.6159226980000003, 2024-09-14T09:06:17.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071890509000000005, 2024-09-14T09:06:17.225 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.225 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.225 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:17.225 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.225 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.225 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.225 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49392 2024-09-14T09:06:17.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.225 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547226+0000", 2024-09-14T09:06:17.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547226+0000", 2024-09-14T09:06:17.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547227+0000", 2024-09-14T09:06:17.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3969999999999999e-06 2024-09-14T09:06:17.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547233+0000", 2024-09-14T09:06:17.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4530000000000001e-06 2024-09-14T09:06:17.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547244+0000", 2024-09-14T09:06:17.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1327e-05 2024-09-14T09:06:17.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547252+0000", 2024-09-14T09:06:17.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0269999999999995e-06 2024-09-14T09:06:17.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619110+0000", 2024-09-14T09:06:17.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071857997000000007 2024-09-14T09:06:17.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619116+0000", 2024-09-14T09:06:17.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3080000000000004e-06 2024-09-14T09:06:17.228 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.228 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.228 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.229 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49393 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:17.229 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547263+0000", 2024-09-14T09:06:17.229 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.6158850549999997, 2024-09-14T09:06:17.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071900037999999999, 2024-09-14T09:06:17.229 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.229 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.229 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:17.229 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.229 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.229 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.229 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49393 2024-09-14T09:06:17.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.230 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.230 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.230 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547263+0000", 2024-09-14T09:06:17.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.230 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.230 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547263+0000", 2024-09-14T09:06:17.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.230 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.230 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547264+0000", 2024-09-14T09:06:17.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3399999999999998e-07 2024-09-14T09:06:17.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547299+0000", 2024-09-14T09:06:17.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5252000000000001e-05 2024-09-14T09:06:17.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547317+0000", 2024-09-14T09:06:17.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.736e-05 2024-09-14T09:06:17.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547332+0000", 2024-09-14T09:06:17.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.487e-05 2024-09-14T09:06:17.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619152+0000", 2024-09-14T09:06:17.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071820590000000004 2024-09-14T09:06:17.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619163+0000", 2024-09-14T09:06:17.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1131999999999999e-05 2024-09-14T09:06:17.232 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.232 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.232 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.233 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49394 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:17.233 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547353+0000", 2024-09-14T09:06:17.233 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.6157958050000003, 2024-09-14T09:06:17.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071828462999999995, 2024-09-14T09:06:17.233 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.233 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.233 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:17.233 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.233 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.233 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.233 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49394 2024-09-14T09:06:17.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.233 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547353+0000", 2024-09-14T09:06:17.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547353+0000", 2024-09-14T09:06:17.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547354+0000", 2024-09-14T09:06:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6130000000000001e-06 2024-09-14T09:06:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547360+0000", 2024-09-14T09:06:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6869999999999999e-06 2024-09-14T09:06:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547378+0000", 2024-09-14T09:06:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7805999999999999e-05 2024-09-14T09:06:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547391+0000", 2024-09-14T09:06:17.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3617e-05 2024-09-14T09:06:17.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619176+0000", 2024-09-14T09:06:17.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071784608999999999 2024-09-14T09:06:17.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619181+0000", 2024-09-14T09:06:17.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1309999999999997e-06 2024-09-14T09:06:17.236 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.236 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.237 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.237 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49395 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:17.237 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547469+0000", 2024-09-14T09:06:17.237 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.6156795659999998, 2024-09-14T09:06:17.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071780231, 2024-09-14T09:06:17.237 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.237 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.237 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:17.237 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.237 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.237 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.238 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49395 2024-09-14T09:06:17.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.238 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547469+0000", 2024-09-14T09:06:17.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547469+0000", 2024-09-14T09:06:17.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547470+0000", 2024-09-14T09:06:17.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-07 2024-09-14T09:06:17.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547473+0000", 2024-09-14T09:06:17.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7330000000000001e-06 2024-09-14T09:06:17.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547487+0000", 2024-09-14T09:06:17.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3162999999999999e-05 2024-09-14T09:06:17.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547495+0000", 2024-09-14T09:06:17.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8100000000000004e-06 2024-09-14T09:06:17.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619243+0000", 2024-09-14T09:06:17.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071747900000000003 2024-09-14T09:06:17.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619249+0000", 2024-09-14T09:06:17.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7350000000000001e-06 2024-09-14T09:06:17.240 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.240 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.240 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.241 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:50478 3.1d 3:b8a2eaf4:::rbd_data.115e5dc498ac.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T09:06:17.241 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:17.241 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 5.1135085699999996, 2024-09-14T09:06:17.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.075023792000000006, 2024-09-14T09:06:17.241 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:17.241 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:17.241 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:17.241 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:17.241 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:17.241 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:17.241 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 50478 2024-09-14T09:06:17.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.242 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:17.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:17.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:17.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:17.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:17.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:17.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:17.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049641+0000", 2024-09-14T09:06:17.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6270000000000001e-06 2024-09-14T09:06:17.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:17.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049649+0000", 2024-09-14T09:06:17.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6279999999999996e-06 2024-09-14T09:06:17.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:17.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049667+0000", 2024-09-14T09:06:17.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-09-14T09:06:17.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:17.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049677+0000", 2024-09-14T09:06:17.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0509000000000001e-05 2024-09-14T09:06:17.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:17.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049751+0000", 2024-09-14T09:06:17.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3322e-05 2024-09-14T09:06:17.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:17.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049830+0000", 2024-09-14T09:06:17.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9932000000000007e-05 2024-09-14T09:06:17.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:17.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.053203+0000", 2024-09-14T09:06:17.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003372398 2024-09-14T09:06:17.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:17.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124542+0000", 2024-09-14T09:06:17.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071339105999999999 2024-09-14T09:06:17.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:17.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124562+0000", 2024-09-14T09:06:17.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9649000000000001e-05 2024-09-14T09:06:17.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:17.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124616+0000", 2024-09-14T09:06:17.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4642999999999998e-05 2024-09-14T09:06:17.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:17.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:17.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:17.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124664+0000", 2024-09-14T09:06:17.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7342000000000002e-05 2024-09-14T09:06:17.246 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.246 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.246 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.246 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:17.246 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:17.246 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:17.247 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:06:17.247 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:17.159+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:17.248 INFO:teuthology.orchestra.run.smithi176.stdout:2191 trunc from 0xbb0b4c4 to 0x1291ff6 2024-09-14T09:06:17.249 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:06:17.250 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:17.250 INFO:teuthology.orchestra.run.smithi176.stdout:2193 read 0xc31867 thru 0xc3c54a (0xace4 bytes) 2024-09-14T09:06:17.250 INFO:teuthology.orchestra.run.smithi176.stdout:2194 trunc from 0x1291ff6 to 0x484dd95 2024-09-14T09:06:17.251 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-14T09:06:17.251 INFO:teuthology.orchestra.run.smithi176.stdout:2196 writesame 0x98bf9e1 thru 0x98c7e09 data_size 0x1(0x8429 bytes) 2024-09-14T09:06:17.251 INFO:teuthology.orchestra.run.smithi176.stdout:2197 write 0xe521ffb thru 0xe531dfa (0xfe00 bytes) 2024-09-14T09:06:17.251 INFO:teuthology.orchestra.run.smithi176.stdout:2198 punch from 0xa0058ad to 0xa0091ba, (0x390d bytes) 2024-09-14T09:06:17.251 INFO:teuthology.orchestra.run.smithi176.stdout:2200 writesame 0x70b1997 thru 0x70c0478 data_size 0x23(0xeae2 bytes) 2024-09-14T09:06:17.252 INFO:teuthology.orchestra.run.smithi176.stdout:2203 punch from 0x2855851 to 0x28645e8, (0xed97 bytes) 2024-09-14T09:06:17.252 INFO:teuthology.orchestra.run.smithi176.stdout:2204 write 0xda68774 thru 0xda6e769 (0x5ff6 bytes) 2024-09-14T09:06:17.270 INFO:teuthology.orchestra.run.smithi176.stdout:2205 write 0x9152f49 thru 0x915f103 (0xc1bb bytes) 2024-09-14T09:06:17.289 INFO:teuthology.orchestra.run.smithi176.stdout:2206 punch from 0xdc4c0bc to 0xdc591cf, (0xd113 bytes) 2024-09-14T09:06:17.289 INFO:teuthology.orchestra.run.smithi176.stdout:2207 write 0x52da51f thru 0x52e02af (0x5d91 bytes) 2024-09-14T09:06:17.295 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:17.299+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:17.305 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:17.305 INFO:teuthology.orchestra.run.smithi176.stdout:2209 punch from 0x992d65d to 0x993c43a, (0xeddd bytes) 2024-09-14T09:06:17.306 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:17.306 INFO:teuthology.orchestra.run.smithi176.stdout:2211 read 0xd24a85b thru 0xd251c1b (0x73c1 bytes) 2024-09-14T09:06:17.306 INFO:teuthology.orchestra.run.smithi176.stdout:2212 read 0xb771c8f thru 0xb77769e (0x5a10 bytes) 2024-09-14T09:06:17.306 INFO:teuthology.orchestra.run.smithi176.stdout:2214 write 0x38eb14b thru 0x38f3110 (0x7fc6 bytes) 2024-09-14T09:06:17.324 INFO:teuthology.orchestra.run.smithi176.stdout:2215 write 0x3d83e18 thru 0x3d8f1a9 (0xb392 bytes) 2024-09-14T09:06:17.344 INFO:teuthology.orchestra.run.smithi176.stdout:2216 punch from 0x9e49a9e to 0x9e4f7d1, (0x5d33 bytes) 2024-09-14T09:06:17.344 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:17.344 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:17.344 INFO:teuthology.orchestra.run.smithi176.stdout:2224 writesame 0x45791a8 thru 0x4579d2a data_size 0x1(0xb83 bytes) 2024-09-14T09:06:17.350 INFO:teuthology.orchestra.run.smithi176.stdout:2225 write 0xd33f8c8 thru 0xd3496dd (0x9e16 bytes) 2024-09-14T09:06:17.362 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:17.363+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:17.366 INFO:teuthology.orchestra.run.smithi176.stdout:2226 clone 21 order 21 su 131072 sc 3 2024-09-14T09:06:17.380 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:17.381 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:17.381 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:17.381 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:17.395 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:06:17.409 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:17.409 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:17.425 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:17.425 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:17.428 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-14T09:06:17.441 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:17.441 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:17.457 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:17.457 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:17.463 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:17.463+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:17.473 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:17.473 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:17.492 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:17.492 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:17.508 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:17.509 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:17.526 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:17.527 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:17.564 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:17.564 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:17.564 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:17.567+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:17.577 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:06:17.586 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-14T09:06:17.664 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:17.663+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:17.729 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-14T09:06:17.750 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:17.750 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:17.764 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:06:17.770 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:17.771+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:17.866 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:17.867+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:17.882 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:17.882 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:17.897 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:06:17.988 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:17.991+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:18.065 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:18.065 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:18.068 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:18.071+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:18.079 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:06:18.168 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:18.171+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:18.256 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:18.256 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:18.270 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:18.271+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:18.270 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:06:18.356 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone20 from 0xe531dfb (overlap 0x78dd9b) to 0x3058a4 2024-09-14T09:06:18.372 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:18.371+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:18.427 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:18.427 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:18.445 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:06:18.490 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:18.491+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:18.572 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:18.575+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:18.608 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:18.608 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:18.624 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:06:18.645 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-14T09:06:18.673 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:18.675+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:18.764 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:18.764 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:18.773 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:18.771+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:18.778 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:06:18.833 INFO:teuthology.orchestra.run.smithi176.stdout:2227 punch from 0x2703335 to 0x27048e5, (0x15b0 bytes) 2024-09-14T09:06:18.841 INFO:teuthology.orchestra.run.smithi176.stdout:2228 writesame 0xd799dfc thru 0xd79f196 data_size 0x1(0x539b bytes) 2024-09-14T09:06:18.875 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:18.875+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:18.946 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:18.947 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:18.971 INFO:teuthology.orchestra.run.smithi176.stdout:2229 trunc from 0xe531dfb to 0x205b484 2024-09-14T09:06:18.976 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:18.975+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:18.979 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:06:19.017 INFO:teuthology.orchestra.run.smithi176.stdout:2233 trunc from 0x205b484 to 0xa72fabd 2024-09-14T09:06:19.043 INFO:teuthology.orchestra.run.smithi176.stdout:2235 write 0xb6fb4db thru 0xb700962 (0x5488 bytes) 2024-09-14T09:06:19.052 INFO:teuthology.orchestra.run.smithi176.stdout:2237 punch from 0x275c17 to 0x27d13e, (0x7527 bytes) 2024-09-14T09:06:19.052 INFO:teuthology.orchestra.run.smithi176.stdout:2239 read 0x323b5ee thru 0x32473c8 (0xbddb bytes) 2024-09-14T09:06:19.052 INFO:teuthology.orchestra.run.smithi176.stdout:2240 punch from 0x30d1173 to 0x30d2fe7, (0x1e74 bytes) 2024-09-14T09:06:19.065 INFO:teuthology.orchestra.run.smithi176.stdout:2241 trunc from 0xb700963 to 0x31e6366 2024-09-14T09:06:19.077 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:19.079+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:19.083 INFO:teuthology.orchestra.run.smithi176.stdout:2243 punch from 0x14e0ce1 to 0x14ee34d, (0xd66c bytes) 2024-09-14T09:06:19.084 INFO:teuthology.orchestra.run.smithi176.stdout:2245 trunc from 0x31e6366 to 0x3c4715d 2024-09-14T09:06:19.093 INFO:teuthology.orchestra.run.smithi176.stdout:2248 read 0x1fd3b71 thru 0x1fe16ef (0xdb7f bytes) 2024-09-14T09:06:19.094 INFO:teuthology.orchestra.run.smithi176.stdout:2249 punch from 0x317454a to 0x317a8dd, (0x6393 bytes) 2024-09-14T09:06:19.094 INFO:teuthology.orchestra.run.smithi176.stdout:2252 read 0x3572736 thru 0x35767a7 (0x4072 bytes) 2024-09-14T09:06:19.120 INFO:teuthology.orchestra.run.smithi176.stdout:2255 writesame 0xab07900 thru 0xab0b2e2 data_size 0x7(0x39e3 bytes) 2024-09-14T09:06:19.120 INFO:teuthology.orchestra.run.smithi176.stdout:2256 punch from 0x3834fdb to 0x3838e63, (0x3e88 bytes) 2024-09-14T09:06:19.120 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:19.120 INFO:teuthology.orchestra.run.smithi176.stdout:2258 read 0x2be94d5 thru 0x2bf49f2 (0xb51e bytes) 2024-09-14T09:06:19.120 INFO:teuthology.orchestra.run.smithi176.stdout:2259 punch from 0xcfacd5 to 0xd01379, (0x66a4 bytes) 2024-09-14T09:06:19.120 INFO:teuthology.orchestra.run.smithi176.stdout:2260 read 0x7c81d7f thru 0x7c904a5 (0xe727 bytes) 2024-09-14T09:06:19.135 INFO:teuthology.orchestra.run.smithi176.stdout:2261 write 0xc1d8c44 thru 0xc1df1ce (0x658b bytes) 2024-09-14T09:06:19.142 INFO:teuthology.orchestra.run.smithi176.stdout:2262 write 0x91588e9 thru 0x915fb9c (0x72b4 bytes) 2024-09-14T09:06:19.148 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:19.148 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:19.167 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:06:19.170 INFO:teuthology.orchestra.run.smithi176.stdout:2263 writesame 0xecdbe06 thru 0xece4aae data_size 0x3(0x8ca9 bytes) 2024-09-14T09:06:19.170 INFO:teuthology.orchestra.run.smithi176.stdout:2264 write 0x328f16c thru 0x329ae3a (0xbccf bytes) 2024-09-14T09:06:19.177 INFO:teuthology.orchestra.run.smithi176.stdout:2267 read 0xb8b8118 thru 0xb8bca4b (0x4934 bytes) 2024-09-14T09:06:19.177 INFO:teuthology.orchestra.run.smithi176.stdout:2268 writesame 0x4a89d5d thru 0x4a97541 data_size 0x45(0xd7e5 bytes) 2024-09-14T09:06:19.178 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:19.179+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:19.185 INFO:teuthology.orchestra.run.smithi176.stdout:2269 read 0x686df1d thru 0x6879a76 (0xbb5a bytes) 2024-09-14T09:06:19.185 INFO:teuthology.orchestra.run.smithi176.stdout:2270 punch from 0x9ae7903 to 0x9aea30c, (0x2a09 bytes) 2024-09-14T09:06:19.209 INFO:teuthology.orchestra.run.smithi176.stdout:2271 trunc from 0xece4aaf to 0x307987 2024-09-14T09:06:19.251 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T09:06:19.298 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:19.299+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:19.360 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:19.361 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:19.379 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:19.379+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:19.383 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:06:19.407 INFO:teuthology.orchestra.run.smithi176.stdout:2272 writesame 0x43265f5 thru 0x4332162 data_size 0x21(0xbb6e bytes) 2024-09-14T09:06:19.408 INFO:teuthology.orchestra.run.smithi176.stdout:2273 read 0x4319147 thru 0x431e729 (0x55e3 bytes) 2024-09-14T09:06:19.419 INFO:teuthology.orchestra.run.smithi176.stdout:2274 write 0x829c784 thru 0x829fcc2 (0x353f bytes) 2024-09-14T09:06:19.423 INFO:teuthology.orchestra.run.smithi176.stdout:2276 write 0x173ab7d thru 0x1748db8 (0xe23c bytes) 2024-09-14T09:06:19.435 INFO:teuthology.orchestra.run.smithi176.stdout:2278 trunc from 0x829fcc3 to 0xb5eb122 2024-09-14T09:06:19.441 INFO:teuthology.orchestra.run.smithi176.stdout:2279 punch from 0x83465b to 0x835b89, (0x152e bytes) 2024-09-14T09:06:19.451 INFO:teuthology.orchestra.run.smithi176.stdout:2280 writesame 0xd58cb0c thru 0xd594da1 data_size 0x5(0x8296 bytes) 2024-09-14T09:06:19.452 INFO:teuthology.orchestra.run.smithi176.stdout:2281 write 0xc11547c thru 0xc121933 (0xc4b8 bytes) 2024-09-14T09:06:19.459 INFO:teuthology.orchestra.run.smithi176.stdout:2282 read 0x54c810f thru 0x54cd397 (0x5289 bytes) 2024-09-14T09:06:19.459 INFO:teuthology.orchestra.run.smithi176.stdout:2283 read 0x1f58bbb thru 0x1f65f07 (0xd34d bytes) 2024-09-14T09:06:19.459 INFO:teuthology.orchestra.run.smithi176.stdout:2286 write 0x1a2c1cc thru 0x1a2e157 (0x1f8c bytes) 2024-09-14T09:06:19.462 INFO:teuthology.orchestra.run.smithi176.stdout:2288 write 0x119acfe thru 0x11a3461 (0x8764 bytes) 2024-09-14T09:06:19.471 INFO:teuthology.orchestra.run.smithi176.stdout:2290 write 0xcf88177 thru 0xcf8c67b (0x4505 bytes) 2024-09-14T09:06:19.480 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:19.483+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:19.493 INFO:teuthology.orchestra.run.smithi176.stdout:2293 trunc from 0xd594da2 to 0x23d5b30 2024-09-14T09:06:19.519 INFO:teuthology.orchestra.run.smithi176.stdout:2294 trunc from 0x23d5b30 to 0x548eb84 2024-09-14T09:06:19.532 INFO:teuthology.orchestra.run.smithi176.stdout:2295 read 0x2c0a53a thru 0x2c1261d (0x80e4 bytes) 2024-09-14T09:06:19.533 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:19.533 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:19.543 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:06:19.551 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:06:19.553 INFO:teuthology.orchestra.run.smithi176.stdout:2296 write 0xa04f118 thru 0xa057313 (0x81fc bytes) 2024-09-14T09:06:19.574 INFO:teuthology.orchestra.run.smithi176.stdout:2297 write 0xe44daf1 thru 0xe45167f (0x3b8f bytes) 2024-09-14T09:06:19.580 INFO:teuthology.orchestra.run.smithi176.stdout:2298 write 0x9e00ead thru 0x9e045ba (0x370e bytes) 2024-09-14T09:06:19.580 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:19.583+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:19.581 INFO:teuthology.orchestra.run.smithi176.stdout:2299 write 0xd805865 thru 0xd80ac73 (0x540f bytes) 2024-09-14T09:06:19.589 INFO:teuthology.orchestra.run.smithi176.stdout:2300 write 0x4e2cdc0 thru 0x4e3ad1f (0xdf60 bytes) 2024-09-14T09:06:19.602 INFO:teuthology.orchestra.run.smithi176.stdout:2301 read 0x6f4c3a3 thru 0x6f59d8a (0xd9e8 bytes) 2024-09-14T09:06:19.602 INFO:teuthology.orchestra.run.smithi176.stdout:2302 read 0x9f0c850 thru 0x9f0d979 (0x112a bytes) 2024-09-14T09:06:19.603 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:19.603 INFO:teuthology.orchestra.run.smithi176.stdout:2304 read 0x92ea210 thru 0x92f8b77 (0xe968 bytes) 2024-09-14T09:06:19.603 INFO:teuthology.orchestra.run.smithi176.stdout:2305 punch from 0x61dc10 to 0x61eb83, (0xf73 bytes) 2024-09-14T09:06:19.603 INFO:teuthology.orchestra.run.smithi176.stdout:2306 read 0xb32a45e thru 0xb33681b (0xc3be bytes) 2024-09-14T09:06:19.603 INFO:teuthology.orchestra.run.smithi176.stdout:2308 punch from 0xd7c3e9a to 0xd7ceee0, (0xb046 bytes) 2024-09-14T09:06:19.603 INFO:teuthology.orchestra.run.smithi176.stdout:2313 read 0xb1ac177 thru 0xb1b1acb (0x5955 bytes) 2024-09-14T09:06:19.603 INFO:teuthology.orchestra.run.smithi176.stdout:2315 read 0x9efa93c thru 0x9f05dec (0xb4b1 bytes) 2024-09-14T09:06:19.603 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:19.603 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:19.618 INFO:teuthology.orchestra.run.smithi176.stdout:2318 trunc from 0xe451680 to 0x2f52e88 2024-09-14T09:06:19.647 INFO:teuthology.orchestra.run.smithi176.stdout:2319 write 0x3738c73 thru 0x373e52f (0x58bd bytes) 2024-09-14T09:06:19.656 INFO:teuthology.orchestra.run.smithi176.stdout:2320 trunc from 0x373e530 to 0x70eed8 2024-09-14T09:06:19.674 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:19.674 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:19.676 INFO:teuthology.orchestra.run.smithi176.stdout:2321 read 0x28ab81 thru 0x2939b3 (0x8e33 bytes) 2024-09-14T09:06:19.678 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:19.679 INFO:teuthology.orchestra.run.smithi176.stdout:2325 writesame 0x45459b thru 0x45b5d1 data_size 0x1(0x7037 bytes) 2024-09-14T09:06:19.681 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:19.683+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:19.693 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:06:19.699 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:19.699 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:19.715 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:19.716 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:19.732 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:19.733 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:19.753 INFO:teuthology.orchestra.run.smithi176.stdout:2326 trunc from 0x70eed8 to 0xc1b8648 2024-09-14T09:06:19.757 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:19.757 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:19.775 INFO:teuthology.orchestra.run.smithi176.stdout:2327 trunc from 0xc1b8648 to 0x3748269 2024-09-14T09:06:19.777 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:19.778 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:19.782 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:19.783+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:19.798 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:19.798 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:19.808 INFO:teuthology.orchestra.run.smithi176.stdout:2328 write 0xac2753b thru 0xac29e8b (0x2951 bytes) 2024-09-14T09:06:19.815 INFO:teuthology.orchestra.run.smithi176.stdout:2330 read 0x3e25f3a thru 0x3e33ed0 (0xdf97 bytes) 2024-09-14T09:06:19.815 INFO:teuthology.orchestra.run.smithi176.stdout:2331 write 0x358ad77 thru 0x3592ad3 (0x7d5d bytes) 2024-09-14T09:06:19.820 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:19.820 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:19.821 INFO:teuthology.orchestra.run.smithi176.stdout:2332 read 0x1f0cb0 thru 0x1fdde0 (0xd131 bytes) 2024-09-14T09:06:19.821 INFO:teuthology.orchestra.run.smithi176.stdout:2333 punch from 0x1e7b953 to 0x1e82ad1, (0x717e bytes) 2024-09-14T09:06:19.822 INFO:teuthology.orchestra.run.smithi176.stdout:2334 read 0x26ed314 thru 0x26f7864 (0xa551 bytes) 2024-09-14T09:06:19.831 INFO:teuthology.orchestra.run.smithi176.stdout:2335 trunc from 0xac29e8c to 0x3c88837 2024-09-14T09:06:19.837 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:19.837 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:19.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:19.837 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:19.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.837 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18383 3.9 3:9252f7c9:::rbd_data.115e3abd3b2c.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:06:19.837 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:19.838 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.679055427000002, 2024-09-14T09:06:19.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056433830999999997, 2024-09-14T09:06:19.838 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.838 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.838 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:19.838 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:19.838 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:19.838 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:19.838 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18383 2024-09-14T09:06:19.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.838 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:19.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:19.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158423+0000", 2024-09-14T09:06:19.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T09:06:19.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158429+0000", 2024-09-14T09:06:19.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0179999999999998e-06 2024-09-14T09:06:19.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158440+0000", 2024-09-14T09:06:19.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0769e-05 2024-09-14T09:06:19.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158445+0000", 2024-09-14T09:06:19.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5840000000000002e-06 2024-09-14T09:06:19.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158495+0000", 2024-09-14T09:06:19.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9129999999999999e-05 2024-09-14T09:06:19.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158598+0000", 2024-09-14T09:06:19.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102938 2024-09-14T09:06:19.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:19.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.161048+0000", 2024-09-14T09:06:19.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002450336 2024-09-14T09:06:19.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:19.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214605+0000", 2024-09-14T09:06:19.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053557526000000001 2024-09-14T09:06:19.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:19.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214634+0000", 2024-09-14T09:06:19.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8483000000000002e-05 2024-09-14T09:06:19.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:19.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214731+0000", 2024-09-14T09:06:19.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7026000000000002e-05 2024-09-14T09:06:19.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214856+0000", 2024-09-14T09:06:19.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012474799999999999 2024-09-14T09:06:19.843 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.843 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.843 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.843 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18383 3.9 e38/34 ondisk, result = 0)", 2024-09-14T09:06:19.843 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:19.843 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.668738945999998, 2024-09-14T09:06:19.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045880575999999999, 2024-09-14T09:06:19.843 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.844 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.844 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:19.844 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:19.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:19.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168739+0000", 2024-09-14T09:06:19.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3900000000000004e-07 2024-09-14T09:06:19.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168744+0000", 2024-09-14T09:06:19.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5929999999999999e-06 2024-09-14T09:06:19.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168754+0000", 2024-09-14T09:06:19.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0686999999999999e-05 2024-09-14T09:06:19.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168761+0000", 2024-09-14T09:06:19.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6340000000000003e-06 2024-09-14T09:06:19.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214578+0000", 2024-09-14T09:06:19.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045817443999999999 2024-09-14T09:06:19.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214596+0000", 2024-09-14T09:06:19.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7159e-05 2024-09-14T09:06:19.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214619+0000", 2024-09-14T09:06:19.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.332e-05 2024-09-14T09:06:19.847 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.847 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.847 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.847 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19152 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14159058~8940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:19.847 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:19.848 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 35.514826489999997, 2024-09-14T09:06:19.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12891994500000001, 2024-09-14T09:06:19.848 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.848 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.848 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:19.848 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:19.848 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:19.848 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:19.848 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19152 2024-09-14T09:06:19.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.848 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:19.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:19.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322653+0000", 2024-09-14T09:06:19.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3750000000000001e-06 2024-09-14T09:06:19.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329647+0000", 2024-09-14T09:06:19.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069941960000000003 2024-09-14T09:06:19.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329673+0000", 2024-09-14T09:06:19.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5760000000000001e-05 2024-09-14T09:06:19.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329684+0000", 2024-09-14T09:06:19.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0474e-05 2024-09-14T09:06:19.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329776+0000", 2024-09-14T09:06:19.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2467999999999996e-05 2024-09-14T09:06:19.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329987+0000", 2024-09-14T09:06:19.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021133 2024-09-14T09:06:19.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:19.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.331303+0000", 2024-09-14T09:06:19.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001315629 2024-09-14T09:06:19.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:19.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.411090+0000", 2024-09-14T09:06:19.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079787477999999995 2024-09-14T09:06:19.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:19.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451463+0000", 2024-09-14T09:06:19.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040372312 2024-09-14T09:06:19.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:19.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451523+0000", 2024-09-14T09:06:19.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0420000000000001e-05 2024-09-14T09:06:19.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451571+0000", 2024-09-14T09:06:19.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7503000000000002e-05 2024-09-14T09:06:19.853 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.853 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.853 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.853 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20925 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2490368~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:19.853 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:06:19.853 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.165527709999999, 2024-09-14T09:06:19.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017084327999999999, 2024-09-14T09:06:19.853 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.853 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.854 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:19.854 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:19.854 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:19.854 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:19.854 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20925 2024-09-14T09:06:19.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.854 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:06:19.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:06:19.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671950+0000", 2024-09-14T09:06:19.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3e-07 2024-09-14T09:06:19.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671953+0000", 2024-09-14T09:06:19.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0299999999999998e-06 2024-09-14T09:06:19.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671964+0000", 2024-09-14T09:06:19.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1e-05 2024-09-14T09:06:19.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671971+0000", 2024-09-14T09:06:19.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9929999999999998e-06 2024-09-14T09:06:19.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.687941+0000", 2024-09-14T09:06:19.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015969357 2024-09-14T09:06:19.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.688070+0000", 2024-09-14T09:06:19.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000129858 2024-09-14T09:06:19.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689034+0000", 2024-09-14T09:06:19.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00096336000000000002 2024-09-14T09:06:19.857 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.857 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.858 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.858 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20926 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:19.858 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:19.858 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.16549449, 2024-09-14T09:06:19.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018572207, 2024-09-14T09:06:19.858 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.858 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.858 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:19.858 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:19.858 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:19.858 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:19.859 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20926 2024-09-14T09:06:19.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.859 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:19.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:19.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:19.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:06:19.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671990+0000", 2024-09-14T09:06:19.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5769999999999999e-06 2024-09-14T09:06:19.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672001+0000", 2024-09-14T09:06:19.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1293e-05 2024-09-14T09:06:19.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672008+0000", 2024-09-14T09:06:19.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9099999999999999e-06 2024-09-14T09:06:19.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689147+0000", 2024-09-14T09:06:19.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017139060000000001 2024-09-14T09:06:19.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689324+0000", 2024-09-14T09:06:19.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017640200000000001 2024-09-14T09:06:19.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690555+0000", 2024-09-14T09:06:19.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001231212 2024-09-14T09:06:19.862 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.862 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.862 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.862 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20927 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2621440~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:19.862 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:19.862 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.165457523999997, 2024-09-14T09:06:19.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020002227000000001, 2024-09-14T09:06:19.862 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.862 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.863 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:19.863 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:19.863 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:19.863 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:19.863 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20927 2024-09-14T09:06:19.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.863 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:19.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:19.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:19.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9699999999999995e-07 2024-09-14T09:06:19.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672024+0000", 2024-09-14T09:06:19.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:06:19.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672036+0000", 2024-09-14T09:06:19.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.119e-05 2024-09-14T09:06:19.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672044+0000", 2024-09-14T09:06:19.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8370000000000003e-06 2024-09-14T09:06:19.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690602+0000", 2024-09-14T09:06:19.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018557648 2024-09-14T09:06:19.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690805+0000", 2024-09-14T09:06:19.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020282899999999999 2024-09-14T09:06:19.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692022+0000", 2024-09-14T09:06:19.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012170429999999999 2024-09-14T09:06:19.866 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.866 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.866 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.866 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20928 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 917504~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:19.866 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:19.866 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.165421383999998, 2024-09-14T09:06:19.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021623738, 2024-09-14T09:06:19.867 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.867 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.867 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:19.867 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:19.867 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:19.867 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:19.867 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20928 2024-09-14T09:06:19.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.867 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:19.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:19.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672057+0000", 2024-09-14T09:06:19.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:06:19.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672059+0000", 2024-09-14T09:06:19.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9270000000000001e-06 2024-09-14T09:06:19.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672070+0000", 2024-09-14T09:06:19.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0773e-05 2024-09-14T09:06:19.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672077+0000", 2024-09-14T09:06:19.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9600000000000003e-06 2024-09-14T09:06:19.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692154+0000", 2024-09-14T09:06:19.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020076482 2024-09-14T09:06:19.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692419+0000", 2024-09-14T09:06:19.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026517400000000002 2024-09-14T09:06:19.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693680+0000", 2024-09-14T09:06:19.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012607390000000001 2024-09-14T09:06:19.870 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.870 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.870 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.870 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20929 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2752512~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:19.870 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:19.871 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.165388342, 2024-09-14T09:06:19.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022898728, 2024-09-14T09:06:19.871 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.871 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.871 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:19.871 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:19.871 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:19.871 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:19.871 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20929 2024-09-14T09:06:19.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.871 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:19.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:19.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672090+0000", 2024-09-14T09:06:19.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:06:19.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672092+0000", 2024-09-14T09:06:19.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8940000000000002e-06 2024-09-14T09:06:19.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672103+0000", 2024-09-14T09:06:19.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0815999999999999e-05 2024-09-14T09:06:19.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672110+0000", 2024-09-14T09:06:19.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9469999999999997e-06 2024-09-14T09:06:19.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693737+0000", 2024-09-14T09:06:19.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021626467999999999 2024-09-14T09:06:19.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693962+0000", 2024-09-14T09:06:19.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225474 2024-09-14T09:06:19.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.694988+0000", 2024-09-14T09:06:19.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001025469 2024-09-14T09:06:19.874 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.874 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.874 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.875 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20930 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:19.875 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:19.875 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.165355247999997, 2024-09-14T09:06:19.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024303286, 2024-09-14T09:06:19.875 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.875 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.875 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:19.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:19.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:19.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:19.875 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20930 2024-09-14T09:06:19.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.875 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:19.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:19.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672123+0000", 2024-09-14T09:06:19.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:06:19.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672125+0000", 2024-09-14T09:06:19.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7300000000000001e-06 2024-09-14T09:06:19.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672134+0000", 2024-09-14T09:06:19.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7930000000000003e-06 2024-09-14T09:06:19.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672141+0000", 2024-09-14T09:06:19.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9029999999999998e-06 2024-09-14T09:06:19.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695087+0000", 2024-09-14T09:06:19.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022946013000000001 2024-09-14T09:06:19.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695350+0000", 2024-09-14T09:06:19.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026266000000000001 2024-09-14T09:06:19.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696425+0000", 2024-09-14T09:06:19.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001075527 2024-09-14T09:06:19.878 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.878 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.878 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.879 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20931 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:19.879 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:19.879 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.165322019000001, 2024-09-14T09:06:19.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025331519, 2024-09-14T09:06:19.879 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.879 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.879 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:19.879 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:19.879 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:19.879 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:19.879 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20931 2024-09-14T09:06:19.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.880 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:19.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:19.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672156+0000", 2024-09-14T09:06:19.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4300000000000002e-07 2024-09-14T09:06:19.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672159+0000", 2024-09-14T09:06:19.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T09:06:19.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672170+0000", 2024-09-14T09:06:19.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1192999999999999e-05 2024-09-14T09:06:19.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672177+0000", 2024-09-14T09:06:19.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9059999999999996e-06 2024-09-14T09:06:19.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696532+0000", 2024-09-14T09:06:19.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024355291000000001 2024-09-14T09:06:19.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696755+0000", 2024-09-14T09:06:19.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022297699999999999 2024-09-14T09:06:19.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697487+0000", 2024-09-14T09:06:19.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00073171499999999997 2024-09-14T09:06:19.883 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.883 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.883 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.883 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20932 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1179648~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:19.883 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:19.883 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.165288576000002, 2024-09-14T09:06:19.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026104083, 2024-09-14T09:06:19.883 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.884 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.884 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:19.884 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:19.884 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:19.884 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:19.884 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20932 2024-09-14T09:06:19.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.884 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:19.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:19.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:19.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0699999999999996e-07 2024-09-14T09:06:19.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672192+0000", 2024-09-14T09:06:19.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7700000000000002e-06 2024-09-14T09:06:19.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672203+0000", 2024-09-14T09:06:19.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0956e-05 2024-09-14T09:06:19.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672211+0000", 2024-09-14T09:06:19.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3769999999999996e-06 2024-09-14T09:06:19.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697530+0000", 2024-09-14T09:06:19.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025318411999999998 2024-09-14T09:06:19.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697640+0000", 2024-09-14T09:06:19.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011045299999999999 2024-09-14T09:06:19.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698293+0000", 2024-09-14T09:06:19.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065240800000000005 2024-09-14T09:06:19.887 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.887 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.888 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.888 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20933 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3014656~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:19.888 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:19.888 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.165254373000003, 2024-09-14T09:06:19.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026558875999999999, 2024-09-14T09:06:19.888 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.888 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.888 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:19.888 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:19.888 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:19.888 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:19.888 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20933 2024-09-14T09:06:19.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.889 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:19.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:19.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672224+0000", 2024-09-14T09:06:19.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2399999999999997e-07 2024-09-14T09:06:19.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672234+0000", 2024-09-14T09:06:19.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0793e-05 2024-09-14T09:06:19.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672246+0000", 2024-09-14T09:06:19.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659e-05 2024-09-14T09:06:19.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672255+0000", 2024-09-14T09:06:19.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5730000000000008e-06 2024-09-14T09:06:19.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698326+0000", 2024-09-14T09:06:19.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026071414000000001 2024-09-14T09:06:19.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698374+0000", 2024-09-14T09:06:19.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8032999999999999e-05 2024-09-14T09:06:19.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698782+0000", 2024-09-14T09:06:19.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040768 2024-09-14T09:06:19.892 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.892 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.892 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.892 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20934 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1310720~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:19.892 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:19.892 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.165212496999999, 2024-09-14T09:06:19.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027292067999999999, 2024-09-14T09:06:19.892 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.892 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.892 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:19.892 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:19.892 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:19.892 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:19.893 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20934 2024-09-14T09:06:19.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.893 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:19.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:19.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672266+0000", 2024-09-14T09:06:19.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6299999999999993e-07 2024-09-14T09:06:19.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672269+0000", 2024-09-14T09:06:19.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8569999999999999e-06 2024-09-14T09:06:19.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672280+0000", 2024-09-14T09:06:19.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1236999999999999e-05 2024-09-14T09:06:19.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672287+0000", 2024-09-14T09:06:19.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8260000000000004e-06 2024-09-14T09:06:19.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698820+0000", 2024-09-14T09:06:19.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026533055999999999 2024-09-14T09:06:19.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698933+0000", 2024-09-14T09:06:19.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011291 2024-09-14T09:06:19.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699557+0000", 2024-09-14T09:06:19.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062421899999999997 2024-09-14T09:06:19.896 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.896 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.896 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.896 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20935 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:19.896 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:19.896 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.165178736999998, 2024-09-14T09:06:19.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028108610999999999, 2024-09-14T09:06:19.896 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.896 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.896 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:19.896 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:19.897 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:19.897 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:19.897 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20935 2024-09-14T09:06:19.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.897 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:19.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:19.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672299+0000", 2024-09-14T09:06:19.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T09:06:19.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672302+0000", 2024-09-14T09:06:19.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7640000000000001e-06 2024-09-14T09:06:19.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672313+0000", 2024-09-14T09:06:19.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1080000000000001e-05 2024-09-14T09:06:19.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672321+0000", 2024-09-14T09:06:19.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6490000000000008e-06 2024-09-14T09:06:19.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699587+0000", 2024-09-14T09:06:19.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027265884000000001 2024-09-14T09:06:19.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699631+0000", 2024-09-14T09:06:19.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4947999999999997e-05 2024-09-14T09:06:19.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700407+0000", 2024-09-14T09:06:19.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00077559299999999999 2024-09-14T09:06:19.900 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.900 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.900 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.900 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20936 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1441792~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:19.900 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:19.900 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.165144560999998, 2024-09-14T09:06:19.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028784750000000001, 2024-09-14T09:06:19.900 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.900 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.901 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:19.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:19.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:19.901 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:19.901 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20936 2024-09-14T09:06:19.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.901 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:19.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:19.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:19.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4700000000000001e-07 2024-09-14T09:06:19.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672336+0000", 2024-09-14T09:06:19.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6730000000000001e-06 2024-09-14T09:06:19.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672347+0000", 2024-09-14T09:06:19.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0710000000000001e-05 2024-09-14T09:06:19.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672354+0000", 2024-09-14T09:06:19.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4630000000000004e-06 2024-09-14T09:06:19.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700442+0000", 2024-09-14T09:06:19.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028087951 2024-09-14T09:06:19.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700557+0000", 2024-09-14T09:06:19.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011461800000000001 2024-09-14T09:06:19.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701117+0000", 2024-09-14T09:06:19.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056058799999999999 2024-09-14T09:06:19.904 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.904 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.904 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.904 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20937 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:19.904 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:19.904 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.165111435, 2024-09-14T09:06:19.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029493028000000001, 2024-09-14T09:06:19.904 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.905 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.905 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:19.905 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:19.905 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:19.905 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:19.905 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20937 2024-09-14T09:06:19.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.905 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:19.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:19.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:19.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T09:06:19.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672384+0000", 2024-09-14T09:06:19.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7127000000000001e-05 2024-09-14T09:06:19.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672395+0000", 2024-09-14T09:06:19.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759999999999999e-05 2024-09-14T09:06:19.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672402+0000", 2024-09-14T09:06:19.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0960000000000003e-06 2024-09-14T09:06:19.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701147+0000", 2024-09-14T09:06:19.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028744241 2024-09-14T09:06:19.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701246+0000", 2024-09-14T09:06:19.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9031999999999996e-05 2024-09-14T09:06:19.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701859+0000", 2024-09-14T09:06:19.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061310499999999997 2024-09-14T09:06:19.908 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.908 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.908 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.909 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20938 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:19.909 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:19.909 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.165064274999999, 2024-09-14T09:06:19.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030109219999999999, 2024-09-14T09:06:19.909 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.909 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.909 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:19.909 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:19.909 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:19.909 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:19.909 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20938 2024-09-14T09:06:19.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.909 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:19.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:19.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672414+0000", 2024-09-14T09:06:19.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-07 2024-09-14T09:06:19.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672417+0000", 2024-09-14T09:06:19.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:06:19.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672428+0000", 2024-09-14T09:06:19.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0873000000000001e-05 2024-09-14T09:06:19.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672435+0000", 2024-09-14T09:06:19.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:06:19.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701884+0000", 2024-09-14T09:06:19.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029449811999999999 2024-09-14T09:06:19.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701934+0000", 2024-09-14T09:06:19.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9703000000000002e-05 2024-09-14T09:06:19.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702522+0000", 2024-09-14T09:06:19.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00058811200000000001 2024-09-14T09:06:19.913 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.913 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.913 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.913 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20939 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:19.913 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:19.913 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.165032142000001, 2024-09-14T09:06:19.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030779259, 2024-09-14T09:06:19.913 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.913 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.913 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:19.914 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:19.914 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:19.914 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:19.914 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20939 2024-09-14T09:06:19.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.914 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:19.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:19.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672446+0000", 2024-09-14T09:06:19.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:06:19.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672449+0000", 2024-09-14T09:06:19.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7930000000000002e-06 2024-09-14T09:06:19.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672461+0000", 2024-09-14T09:06:19.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T09:06:19.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672471+0000", 2024-09-14T09:06:19.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0807e-05 2024-09-14T09:06:19.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702551+0000", 2024-09-14T09:06:19.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030079228 2024-09-14T09:06:19.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702664+0000", 2024-09-14T09:06:19.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011351800000000001 2024-09-14T09:06:19.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.703224+0000", 2024-09-14T09:06:19.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056026999999999997 2024-09-14T09:06:19.917 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.917 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.917 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.917 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29321 3.9 3:90303d3a:::rbd_data.115ebc3c67d1.0000000000000014:c [read 770048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:06:19.917 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:06:19.917 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.017948817000001, 2024-09-14T09:06:19.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015942787, 2024-09-14T09:06:19.918 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.918 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.918 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:19.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:19.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:19.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:19.918 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29321 2024-09-14T09:06:19.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.918 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:06:19.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819528+0000", 2024-09-14T09:06:19.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819529+0000", 2024-09-14T09:06:19.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8800000000000003e-07 2024-09-14T09:06:19.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819531+0000", 2024-09-14T09:06:19.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9439999999999999e-06 2024-09-14T09:06:19.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819537+0000", 2024-09-14T09:06:19.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.037e-06 2024-09-14T09:06:19.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819542+0000", 2024-09-14T09:06:19.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7670000000000003e-06 2024-09-14T09:06:19.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835320+0000", 2024-09-14T09:06:19.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015778119 2024-09-14T09:06:19.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835384+0000", 2024-09-14T09:06:19.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3897000000000002e-05 2024-09-14T09:06:19.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835471+0000", 2024-09-14T09:06:19.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7534999999999995e-05 2024-09-14T09:06:19.921 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.921 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.921 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.921 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:29322 3.9 3:90303d3a:::rbd_data.115ebc3c67d1.0000000000000014:c [read 780288~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-14T09:06:19.922 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:06:19.922 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.017929306999999, 2024-09-14T09:06:19.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016069904999999999, 2024-09-14T09:06:19.922 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:19.922 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:19.922 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:19.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:19.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:19.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:19.922 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 29322 2024-09-14T09:06:19.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.922 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:19.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:19.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:06:19.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:19.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:06:19.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:19.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:19.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819548+0000", 2024-09-14T09:06:19.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9499999999999998e-07 2024-09-14T09:06:19.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:19.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819550+0000", 2024-09-14T09:06:19.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8309999999999999e-06 2024-09-14T09:06:19.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:19.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819556+0000", 2024-09-14T09:06:19.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1299999999999998e-06 2024-09-14T09:06:19.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:19.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.819561+0000", 2024-09-14T09:06:19.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.707e-06 2024-09-14T09:06:19.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:19.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835500+0000", 2024-09-14T09:06:19.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015939456000000001 2024-09-14T09:06:19.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:19.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835566+0000", 2024-09-14T09:06:19.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5285999999999998e-05 2024-09-14T09:06:19.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:19.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:19.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:19.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:52.835618+0000", 2024-09-14T09:06:19.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2099999999999999e-05 2024-09-14T09:06:19.925 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.925 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.926 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.926 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:19.926 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:19.926 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:19.926 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:19.927 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:19.927 INFO:teuthology.orchestra.run.smithi176.stdout:2336 read 0x1ae116f thru 0x1aef174 (0xe006 bytes) 2024-09-14T09:06:19.927 INFO:teuthology.orchestra.run.smithi176.stdout:2337 read 0x2852a38 thru 0x285be81 (0x944a bytes) 2024-09-14T09:06:19.928 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:06:19.929 INFO:teuthology.orchestra.run.smithi176.stdout:2338 write 0xc900a73 thru 0xc9099c5 (0x8f53 bytes) 2024-09-14T09:06:19.930 INFO:teuthology.orchestra.run.smithi176.stdout:2339 punch from 0x253d3e0 to 0x254178e, (0x43ae bytes) 2024-09-14T09:06:19.930 INFO:teuthology.orchestra.run.smithi176.stdout:2340 trunc from 0xc9099c6 to 0x952dbf9 2024-09-14T09:06:19.930 INFO:teuthology.orchestra.run.smithi176.stdout:2341 read 0x33eeb85 thru 0x33ef0bd (0x539 bytes) 2024-09-14T09:06:19.930 INFO:teuthology.orchestra.run.smithi176.stdout:2342 read 0x7d3bd92 thru 0x7d3dc45 (0x1eb4 bytes) 2024-09-14T09:06:19.930 INFO:teuthology.orchestra.run.smithi176.stdout:2344 write 0x3327d4f thru 0x3336149 (0xe3fb bytes) 2024-09-14T09:06:19.930 INFO:teuthology.orchestra.run.smithi176.stdout:2345 read 0x59ea3bc thru 0x59f0f31 (0x6b76 bytes) 2024-09-14T09:06:19.930 INFO:teuthology.orchestra.run.smithi176.stdout:2350 read 0x78d4275 thru 0x78e402b (0xfdb7 bytes) 2024-09-14T09:06:19.930 INFO:teuthology.orchestra.run.smithi176.stdout:2351 read 0x5bd096c thru 0x5bd1e95 (0x152a bytes) 2024-09-14T09:06:19.930 INFO:teuthology.orchestra.run.smithi176.stdout:2353 trunc from 0x952dbf9 to 0x6b5185f 2024-09-14T09:06:19.930 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:19.927+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:19.940 INFO:teuthology.orchestra.run.smithi176.stdout:2354 writesame 0x9f2130b thru 0x9f2a822 data_size 0x34(0x9518 bytes) 2024-09-14T09:06:19.944 INFO:teuthology.orchestra.run.smithi176.stdout:2355 trunc from 0x9f2a823 to 0xd839c2e 2024-09-14T09:06:19.954 INFO:teuthology.orchestra.run.smithi176.stdout:2356 write 0xc72b624 thru 0xc72fa61 (0x443e bytes) 2024-09-14T09:06:19.960 INFO:teuthology.orchestra.run.smithi176.stdout:2357 write 0xb03c4ec thru 0xb03e406 (0x1f1b bytes) 2024-09-14T09:06:19.967 INFO:teuthology.orchestra.run.smithi176.stdout:2358 read 0xcf8ddc2 thru 0xcf9cc69 (0xeea8 bytes) 2024-09-14T09:06:19.967 INFO:teuthology.orchestra.run.smithi176.stdout:2359 write 0x9c3f291 thru 0x9c4a492 (0xb202 bytes) 2024-09-14T09:06:19.979 INFO:teuthology.orchestra.run.smithi176.stdout:2361 read 0x3927df1 thru 0x39348d4 (0xcae4 bytes) 2024-09-14T09:06:19.979 INFO:teuthology.orchestra.run.smithi176.stdout:2362 read 0x61e3821 thru 0x61f0bc9 (0xd3a9 bytes) 2024-09-14T09:06:19.979 INFO:teuthology.orchestra.run.smithi176.stdout:2363 write 0xc7425f6 thru 0xc743da8 (0x17b3 bytes) 2024-09-14T09:06:19.985 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:19.985 INFO:teuthology.orchestra.run.smithi176.stdout:2365 punch from 0x8616f8b to 0x8619e32, (0x2ea7 bytes) 2024-09-14T09:06:19.985 INFO:teuthology.orchestra.run.smithi176.stdout:2366 write 0x4be3cfd thru 0x4be8188 (0x448c bytes) 2024-09-14T09:06:19.992 INFO:teuthology.orchestra.run.smithi176.stdout:2367 read 0x969fca7 thru 0x96a539f (0x56f9 bytes) 2024-09-14T09:06:20.006 INFO:teuthology.orchestra.run.smithi176.stdout:2368 trunc from 0xd839c2e to 0x5a27de5 2024-09-14T09:06:20.028 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:20.027+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:20.035 INFO:teuthology.orchestra.run.smithi176.stdout:2370 trunc from 0x5a27de5 to 0xe4d421 2024-09-14T09:06:20.059 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:20.059 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:20.059 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:20.059 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:20.072 INFO:teuthology.orchestra.run.smithi176.stdout:2371 writesame 0x53df700 thru 0x53ef626 data_size 0x1(0xff27 bytes) 2024-09-14T09:06:20.075 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:06:20.079 INFO:teuthology.orchestra.run.smithi176.stdout:2372 trunc from 0x53ef627 to 0x89ebaf0 2024-09-14T09:06:20.090 INFO:teuthology.orchestra.run.smithi176.stdout:2374 write 0xc09ec thru 0xce10f (0xd724 bytes) 2024-09-14T09:06:20.117 INFO:teuthology.orchestra.run.smithi176.stdout:2376 write 0xca399f1 thru 0xca4037f (0x698f bytes) 2024-09-14T09:06:20.124 INFO:teuthology.orchestra.run.smithi176.stdout:2377 write 0x2b012a8 thru 0x2b0f65f (0xe3b8 bytes) 2024-09-14T09:06:20.129 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:20.127+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:20.136 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:20.136 INFO:teuthology.orchestra.run.smithi176.stdout:2379 read 0x40d3394 thru 0x40d7bf9 (0x4866 bytes) 2024-09-14T09:06:20.136 INFO:teuthology.orchestra.run.smithi176.stdout:2380 read 0x554319b thru 0x554dea6 (0xad0c bytes) 2024-09-14T09:06:20.136 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:20.137 INFO:teuthology.orchestra.run.smithi176.stdout:2382 writesame 0xc9547f1 thru 0xc95bbf7 data_size 0x1(0x7407 bytes) 2024-09-14T09:06:20.168 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is unset 2024-09-14T09:06:20.196 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:20.196 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:20.196 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:20.196 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:20.196 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:20.196 INFO:teuthology.orchestra.run.smithi176.stdout:2383 write 0x3748336 thru 0x374902b (0xcf6 bytes) 2024-09-14T09:06:20.207 INFO:teuthology.orchestra.run.smithi176.stdout:2384 write 0x19c759b thru 0x19d18d2 (0xa338 bytes) 2024-09-14T09:06:20.210 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:06:20.214 INFO:teuthology.orchestra.run.smithi176.stdout:2385 read 0x63e39f7 thru 0x63e8dd2 (0x53dc bytes) 2024-09-14T09:06:20.214 INFO:teuthology.orchestra.run.smithi176.stdout:2386 read 0x82c9155 thru 0x82d87e6 (0xf692 bytes) 2024-09-14T09:06:20.226 INFO:teuthology.orchestra.run.smithi176.stdout:2387 trunc from 0xca40380 to 0x25c9031 2024-09-14T09:06:20.229 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:20.227+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:20.247 INFO:teuthology.orchestra.run.smithi176.stdout:2388 read 0xad3d88 thru 0xadbfae (0x8227 bytes) 2024-09-14T09:06:20.247 INFO:teuthology.orchestra.run.smithi176.stdout:2389 punch from 0x63bc87 to 0x63f249, (0x35c2 bytes) 2024-09-14T09:06:20.270 INFO:teuthology.orchestra.run.smithi176.stdout:2391 write 0xd6ab06b thru 0xd6b5f92 (0xaf28 bytes) 2024-09-14T09:06:20.277 INFO:teuthology.orchestra.run.smithi176.stdout:2393 write 0xa2fdee9 thru 0xa306b55 (0x8c6d bytes) 2024-09-14T09:06:20.286 INFO:teuthology.orchestra.run.smithi176.stdout:2394 punch from 0x54e2a68 to 0x54eacdc, (0x8274 bytes) 2024-09-14T09:06:20.308 INFO:teuthology.orchestra.run.smithi176.stdout:2395 trunc from 0xd6b5f93 to 0xacc8bc 2024-09-14T09:06:20.331 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:20.331+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:20.331 INFO:teuthology.orchestra.run.smithi176.stdout:2397 read 0x317f96 thru 0x31c1ae (0x4219 bytes) 2024-09-14T09:06:20.332 INFO:teuthology.orchestra.run.smithi176.stdout:2398 read 0x73755e thru 0x73a402 (0x2ea5 bytes) 2024-09-14T09:06:20.342 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:20.342 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.257602716999997, 2024-09-14T09:06:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:06:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:20.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:06:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:06:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:06:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:06:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:06:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:06:20.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:06:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:06:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:06:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:06:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:06:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:06:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:06:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:06:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:06:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:06:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:06:20.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:06:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:06:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.257548059000001, 2024-09-14T09:06:20.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:06:20.349 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.349 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.349 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:20.349 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:20.349 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:20.349 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.349 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:06:20.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.349 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:20.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:20.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:20.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:06:20.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:06:20.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:06:20.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:06:20.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:06:20.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:06:20.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:06:20.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:06:20.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:06:20.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:20.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:06:20.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:06:20.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:06:20.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:06:20.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:06:20.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:06:20.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:06:20.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:06:20.353 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.353 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.353 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.353 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:20.353 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:20.354 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.257442685000001, 2024-09-14T09:06:20.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:06:20.354 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.354 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.354 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:20.354 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:20.354 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:20.354 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.354 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:06:20.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.354 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:20.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:20.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:06:20.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:06:20.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:06:20.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:06:20.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:06:20.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:06:20.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:06:20.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:06:20.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:06:20.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:06:20.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:20.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:06:20.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:06:20.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:06:20.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:06:20.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:06:20.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:06:20.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:06:20.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:06:20.358 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.358 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.358 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.359 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:20.359 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:20.359 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.257085920999998, 2024-09-14T09:06:20.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:06:20.359 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.359 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.359 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:20.359 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:20.359 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:20.359 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.359 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:06:20.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.359 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:20.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:20.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:06:20.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:06:20.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:06:20.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:06:20.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:06:20.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:06:20.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:06:20.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:06:20.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:06:20.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:06:20.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:20.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:06:20.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:06:20.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:06:20.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:06:20.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:06:20.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:06:20.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:06:20.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:06:20.363 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.364 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.364 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.364 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:20.364 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:20.364 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.257001183, 2024-09-14T09:06:20.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:06:20.364 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.364 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.364 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:20.364 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:20.364 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:20.364 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.365 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:06:20.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.365 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:20.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:20.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:06:20.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:06:20.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:06:20.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:06:20.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:06:20.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:06:20.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:06:20.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:06:20.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:06:20.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:06:20.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:20.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:06:20.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:06:20.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:06:20.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:06:20.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:06:20.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:06:20.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:06:20.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:06:20.369 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.369 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.369 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.369 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:20.369 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:20.369 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.256913496999999, 2024-09-14T09:06:20.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:06:20.369 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.370 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.370 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:20.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:20.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:20.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.370 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:06:20.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.370 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:20.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:20.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:06:20.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:06:20.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:06:20.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:06:20.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:06:20.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:06:20.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:06:20.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:06:20.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:06:20.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:06:20.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:20.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:06:20.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:06:20.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:06:20.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:06:20.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:06:20.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:06:20.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:06:20.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:06:20.374 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.374 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.374 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.374 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18388 3.18 3:19e2be55:::rbd_data.115e3abd3b2c.0000000000000021:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:06:20.374 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:20.374 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.121000186000003, 2024-09-14T09:06:20.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042293648000000003, 2024-09-14T09:06:20.374 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.374 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.374 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:20.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:20.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:20.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.375 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18388 2024-09-14T09:06:20.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.375 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:20.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:20.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220236+0000", 2024-09-14T09:06:20.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1339999999999998e-06 2024-09-14T09:06:20.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220244+0000", 2024-09-14T09:06:20.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1950000000000003e-06 2024-09-14T09:06:20.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220262+0000", 2024-09-14T09:06:20.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8017e-05 2024-09-14T09:06:20.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220270+0000", 2024-09-14T09:06:20.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9480000000000008e-06 2024-09-14T09:06:20.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220357+0000", 2024-09-14T09:06:20.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7185e-05 2024-09-14T09:06:20.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220576+0000", 2024-09-14T09:06:20.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021912200000000001 2024-09-14T09:06:20.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:20.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.222435+0000", 2024-09-14T09:06:20.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018592350000000001 2024-09-14T09:06:20.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:20.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262439+0000", 2024-09-14T09:06:20.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040003368999999997 2024-09-14T09:06:20.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:20.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262452+0000", 2024-09-14T09:06:20.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2941e-05 2024-09-14T09:06:20.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:20.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262490+0000", 2024-09-14T09:06:20.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8478e-05 2024-09-14T09:06:20.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262527+0000", 2024-09-14T09:06:20.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7024000000000003e-05 2024-09-14T09:06:20.379 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.379 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.380 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.380 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:19543 3.5 e39/23 3:a536ff9b:::rbd_data.115e4c71d6e8.0000000000000002:head v 39'8, mlcod=39'6)", 2024-09-14T09:06:20.380 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:20.380 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 35.818145526000002, 2024-09-14T09:06:20.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.039280019999999999, 2024-09-14T09:06:20.380 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.380 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.380 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:20.380 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:20.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:20.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523090+0000", 2024-09-14T09:06:20.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.773e-06 2024-09-14T09:06:20.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537290+0000", 2024-09-14T09:06:20.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014200007000000001 2024-09-14T09:06:20.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537317+0000", 2024-09-14T09:06:20.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6704999999999999e-05 2024-09-14T09:06:20.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537328+0000", 2024-09-14T09:06:20.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1299e-05 2024-09-14T09:06:20.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537417+0000", 2024-09-14T09:06:20.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8869999999999994e-05 2024-09-14T09:06:20.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537445+0000", 2024-09-14T09:06:20.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8377999999999998e-05 2024-09-14T09:06:20.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:20.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562324+0000", 2024-09-14T09:06:20.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024879307999999999 2024-09-14T09:06:20.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562368+0000", 2024-09-14T09:06:20.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3680000000000002e-05 2024-09-14T09:06:20.384 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.384 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.384 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.384 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38002 3.1e 3:7df715aa:::rbd_data.115eb07a8abd.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:20.384 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:20.385 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 22.95148228, 2024-09-14T09:06:20.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.065148304000000004, 2024-09-14T09:06:20.385 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.385 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.385 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:20.385 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:20.385 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:20.385 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.386 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38002 2024-09-14T09:06:20.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.386 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:20.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:20.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389752+0000", 2024-09-14T09:06:20.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1829999999999999e-06 2024-09-14T09:06:20.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389758+0000", 2024-09-14T09:06:20.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0919999999999997e-06 2024-09-14T09:06:20.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389769+0000", 2024-09-14T09:06:20.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759e-05 2024-09-14T09:06:20.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389775+0000", 2024-09-14T09:06:20.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8819999999999998e-06 2024-09-14T09:06:20.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389824+0000", 2024-09-14T09:06:20.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8573999999999999e-05 2024-09-14T09:06:20.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389876+0000", 2024-09-14T09:06:20.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2643999999999999e-05 2024-09-14T09:06:20.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:06:20.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.392640+0000", 2024-09-14T09:06:20.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0027636409999999998 2024-09-14T09:06:20.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:20.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454672+0000", 2024-09-14T09:06:20.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062031703000000001 2024-09-14T09:06:20.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:20.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454765+0000", 2024-09-14T09:06:20.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2862000000000006e-05 2024-09-14T09:06:20.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:20.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454841+0000", 2024-09-14T09:06:20.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5857000000000002e-05 2024-09-14T09:06:20.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454900+0000", 2024-09-14T09:06:20.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9107000000000002e-05 2024-09-14T09:06:20.391 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.391 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.392 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.392 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:38002 3.1e e48/23 ondisk, result = 0)", 2024-09-14T09:06:20.392 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:20.392 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 22.941165690999998, 2024-09-14T09:06:20.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054841741999999999, 2024-09-14T09:06:20.392 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.392 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.392 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:20.392 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:20.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:20.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400069+0000", 2024-09-14T09:06:20.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.339e-06 2024-09-14T09:06:20.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400076+0000", 2024-09-14T09:06:20.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7800000000000003e-06 2024-09-14T09:06:20.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400093+0000", 2024-09-14T09:06:20.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6915999999999999e-05 2024-09-14T09:06:20.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400106+0000", 2024-09-14T09:06:20.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2785e-05 2024-09-14T09:06:20.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454713+0000", 2024-09-14T09:06:20.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054607041000000002 2024-09-14T09:06:20.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454724+0000", 2024-09-14T09:06:20.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1202e-05 2024-09-14T09:06:20.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454910+0000", 2024-09-14T09:06:20.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018567900000000001 2024-09-14T09:06:20.396 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.396 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.396 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.396 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38100 3.b 3:d0f67c24:::rbd_data.115eb07a8abd.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:20.396 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:20.396 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 22.695312031, 2024-09-14T09:06:20.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053701964999999997, 2024-09-14T09:06:20.396 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.396 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.396 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:20.397 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:20.397 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:20.397 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.397 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38100 2024-09-14T09:06:20.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.397 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:20.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:20.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645923+0000", 2024-09-14T09:06:20.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:06:20.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645928+0000", 2024-09-14T09:06:20.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0699999999999997e-06 2024-09-14T09:06:20.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645942+0000", 2024-09-14T09:06:20.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4069e-05 2024-09-14T09:06:20.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645951+0000", 2024-09-14T09:06:20.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3230000000000004e-06 2024-09-14T09:06:20.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646008+0000", 2024-09-14T09:06:20.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6583999999999999e-05 2024-09-14T09:06:20.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646089+0000", 2024-09-14T09:06:20.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1557999999999994e-05 2024-09-14T09:06:20.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:20.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.650355+0000", 2024-09-14T09:06:20.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004265525 2024-09-14T09:06:20.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:20.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699439+0000", 2024-09-14T09:06:20.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049083904999999997 2024-09-14T09:06:20.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:20.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699467+0000", 2024-09-14T09:06:20.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8198999999999999e-05 2024-09-14T09:06:20.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:20.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699552+0000", 2024-09-14T09:06:20.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T09:06:20.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699624+0000", 2024-09-14T09:06:20.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1725999999999997e-05 2024-09-14T09:06:20.401 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.401 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.401 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.402 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:06:20.402 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:20.402 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.18062151, 2024-09-14T09:06:20.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:06:20.402 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.402 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.402 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:20.402 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:20.403 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:20.403 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.403 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:06:20.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.403 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:20.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:20.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:06:20.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:06:20.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:06:20.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:06:20.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:06:20.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:06:20.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:06:20.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:06:20.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:06:20.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:06:20.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:06:20.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:06:20.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:06:20.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:06:20.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:06:20.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:20.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:06:20.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:06:20.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:20.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:06:20.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:06:20.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:20.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:06:20.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:06:20.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:06:20.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:06:20.407 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.407 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.408 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.408 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:39401 3.b e56/53)", 2024-09-14T09:06:20.408 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:20.408 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.154725444, 2024-09-14T09:06:20.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10443659600000001, 2024-09-14T09:06:20.408 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.408 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.408 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:20.408 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:20.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:20.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186509+0000", 2024-09-14T09:06:20.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2640000000000001e-06 2024-09-14T09:06:20.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186516+0000", 2024-09-14T09:06:20.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1070000000000001e-06 2024-09-14T09:06:20.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186533+0000", 2024-09-14T09:06:20.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7818000000000001e-05 2024-09-14T09:06:20.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186543+0000", 2024-09-14T09:06:20.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5170000000000005e-06 2024-09-14T09:06:20.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290885+0000", 2024-09-14T09:06:20.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.104342221 2024-09-14T09:06:20.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290893+0000", 2024-09-14T09:06:20.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2859999999999999e-06 2024-09-14T09:06:20.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290945+0000", 2024-09-14T09:06:20.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1382999999999999e-05 2024-09-14T09:06:20.411 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.411 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.411 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.412 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49708 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:20.412 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526878+0000", 2024-09-14T09:06:20.412 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8143553939999997, 2024-09-14T09:06:20.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036535277999999997, 2024-09-14T09:06:20.412 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.412 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.412 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:20.412 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:20.412 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:20.412 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.412 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49708 2024-09-14T09:06:20.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.413 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526878+0000", 2024-09-14T09:06:20.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526878+0000", 2024-09-14T09:06:20.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526879+0000", 2024-09-14T09:06:20.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.13e-07 2024-09-14T09:06:20.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526882+0000", 2024-09-14T09:06:20.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7480000000000002e-06 2024-09-14T09:06:20.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526889+0000", 2024-09-14T09:06:20.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4270000000000002e-06 2024-09-14T09:06:20.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526894+0000", 2024-09-14T09:06:20.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9599999999999999e-06 2024-09-14T09:06:20.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563237+0000", 2024-09-14T09:06:20.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036342646999999999 2024-09-14T09:06:20.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563314+0000", 2024-09-14T09:06:20.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7511999999999999e-05 2024-09-14T09:06:20.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563413+0000", 2024-09-14T09:06:20.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9370999999999994e-05 2024-09-14T09:06:20.416 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.416 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.416 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.416 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49709 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:20.416 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526902+0000", 2024-09-14T09:06:20.417 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8143311359999998, 2024-09-14T09:06:20.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036648100000000003, 2024-09-14T09:06:20.417 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.417 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.417 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:20.417 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:20.417 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:20.417 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.417 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49709 2024-09-14T09:06:20.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.417 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526902+0000", 2024-09-14T09:06:20.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526902+0000", 2024-09-14T09:06:20.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526903+0000", 2024-09-14T09:06:20.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3500000000000002e-07 2024-09-14T09:06:20.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526908+0000", 2024-09-14T09:06:20.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4179999999999996e-06 2024-09-14T09:06:20.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526914+0000", 2024-09-14T09:06:20.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.981e-06 2024-09-14T09:06:20.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526918+0000", 2024-09-14T09:06:20.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0770000000000001e-06 2024-09-14T09:06:20.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563445+0000", 2024-09-14T09:06:20.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036526378999999998 2024-09-14T09:06:20.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563498+0000", 2024-09-14T09:06:20.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3190999999999999e-05 2024-09-14T09:06:20.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563551+0000", 2024-09-14T09:06:20.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2719000000000001e-05 2024-09-14T09:06:20.421 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.421 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.421 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.421 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49710 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:20.421 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526924+0000", 2024-09-14T09:06:20.421 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.814309282, 2024-09-14T09:06:20.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037006095000000003, 2024-09-14T09:06:20.421 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.421 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.421 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:20.421 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:20.422 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:20.422 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.422 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49710 2024-09-14T09:06:20.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.422 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526924+0000", 2024-09-14T09:06:20.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526924+0000", 2024-09-14T09:06:20.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526925+0000", 2024-09-14T09:06:20.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3599999999999999e-07 2024-09-14T09:06:20.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526926+0000", 2024-09-14T09:06:20.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.795e-06 2024-09-14T09:06:20.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526932+0000", 2024-09-14T09:06:20.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0839999999999999e-06 2024-09-14T09:06:20.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526936+0000", 2024-09-14T09:06:20.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3440000000000003e-06 2024-09-14T09:06:20.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563593+0000", 2024-09-14T09:06:20.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036657041000000001 2024-09-14T09:06:20.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563648+0000", 2024-09-14T09:06:20.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5152999999999999e-05 2024-09-14T09:06:20.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563930+0000", 2024-09-14T09:06:20.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028234199999999999 2024-09-14T09:06:20.425 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.426 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.426 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.426 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49711 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:20.426 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:20.426 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8142917629999999, 2024-09-14T09:06:20.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037330945999999997, 2024-09-14T09:06:20.426 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.426 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.426 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:20.426 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:20.426 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:20.426 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.427 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49711 2024-09-14T09:06:20.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.427 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:20.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:20.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:20.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1800000000000002e-07 2024-09-14T09:06:20.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526944+0000", 2024-09-14T09:06:20.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.821e-06 2024-09-14T09:06:20.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526950+0000", 2024-09-14T09:06:20.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5600000000000001e-06 2024-09-14T09:06:20.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526954+0000", 2024-09-14T09:06:20.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.437e-06 2024-09-14T09:06:20.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563965+0000", 2024-09-14T09:06:20.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037010534999999997 2024-09-14T09:06:20.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564021+0000", 2024-09-14T09:06:20.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6728000000000003e-05 2024-09-14T09:06:20.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564273+0000", 2024-09-14T09:06:20.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025154700000000002 2024-09-14T09:06:20.430 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.430 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.430 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.430 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49712 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 3112960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:20.430 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:20.430 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8142735549999998, 2024-09-14T09:06:20.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037795625999999999, 2024-09-14T09:06:20.430 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.431 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.431 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:20.431 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:20.431 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:20.431 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.431 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49712 2024-09-14T09:06:20.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.431 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:20.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:20.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:20.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4799999999999999e-07 2024-09-14T09:06:20.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526962+0000", 2024-09-14T09:06:20.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8449999999999999e-06 2024-09-14T09:06:20.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526967+0000", 2024-09-14T09:06:20.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1059999999999999e-06 2024-09-14T09:06:20.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526971+0000", 2024-09-14T09:06:20.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8310000000000003e-06 2024-09-14T09:06:20.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564324+0000", 2024-09-14T09:06:20.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037352926000000002 2024-09-14T09:06:20.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564411+0000", 2024-09-14T09:06:20.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7052000000000006e-05 2024-09-14T09:06:20.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564756+0000", 2024-09-14T09:06:20.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034451799999999998 2024-09-14T09:06:20.434 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.435 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.435 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.435 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49713 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 3506176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:20.435 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:20.435 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8142568830000005, 2024-09-14T09:06:20.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038135466, 2024-09-14T09:06:20.435 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.435 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.435 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:20.435 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:20.435 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:20.435 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.435 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49713 2024-09-14T09:06:20.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.436 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:20.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:20.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:20.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1899999999999998e-07 2024-09-14T09:06:20.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526979+0000", 2024-09-14T09:06:20.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8959999999999999e-06 2024-09-14T09:06:20.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526986+0000", 2024-09-14T09:06:20.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1720000000000001e-06 2024-09-14T09:06:20.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526991+0000", 2024-09-14T09:06:20.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.729e-06 2024-09-14T09:06:20.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564781+0000", 2024-09-14T09:06:20.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037790345000000003 2024-09-14T09:06:20.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564844+0000", 2024-09-14T09:06:20.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2656000000000005e-05 2024-09-14T09:06:20.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565112+0000", 2024-09-14T09:06:20.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026834899999999997 2024-09-14T09:06:20.439 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.439 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.439 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.440 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49714 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:20.440 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526997+0000", 2024-09-14T09:06:20.440 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8142362740000006, 2024-09-14T09:06:20.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038531469999999998, 2024-09-14T09:06:20.440 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:20.440 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:20.440 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:20.440 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:20.440 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:20.440 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.441 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49714 2024-09-14T09:06:20.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.441 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:20.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:20.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526997+0000", 2024-09-14T09:06:20.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:20.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526997+0000", 2024-09-14T09:06:20.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:20.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:20.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526998+0000", 2024-09-14T09:06:20.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7800000000000002e-07 2024-09-14T09:06:20.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:20.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.527000+0000", 2024-09-14T09:06:20.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9080000000000002e-06 2024-09-14T09:06:20.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:20.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.527007+0000", 2024-09-14T09:06:20.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1609999999999997e-06 2024-09-14T09:06:20.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.527011+0000", 2024-09-14T09:06:20.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0929999999999996e-06 2024-09-14T09:06:20.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:20.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565152+0000", 2024-09-14T09:06:20.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038141520999999998 2024-09-14T09:06:20.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:20.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565236+0000", 2024-09-14T09:06:20.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3498999999999995e-05 2024-09-14T09:06:20.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:20.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:20.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:20.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565529+0000", 2024-09-14T09:06:20.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029291000000000001 2024-09-14T09:06:20.444 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.444 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.444 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.444 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:20.444 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:20.444 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:20.445 INFO:teuthology.orchestra.run.smithi176.stdout:2399 trunc from 0xacc8bc to 0xb0b2bed 2024-09-14T09:06:20.446 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:06:20.448 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:20.448 INFO:teuthology.orchestra.run.smithi176.stdout:2401 write 0x639fc68 thru 0x63aa4a6 (0xa83f bytes) 2024-09-14T09:06:20.448 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:20.448 INFO:teuthology.orchestra.run.smithi176.stdout:2403 read 0x5bbacc8 thru 0x5bca3f1 (0xf72a bytes) 2024-09-14T09:06:20.448 INFO:teuthology.orchestra.run.smithi176.stdout:2404 read 0xa3e1690 thru 0xa3ea3a1 (0x8d12 bytes) 2024-09-14T09:06:20.448 INFO:teuthology.orchestra.run.smithi176.stdout:2405 write 0x369abce thru 0x369efcc (0x43ff bytes) 2024-09-14T09:06:20.448 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:20.448 INFO:teuthology.orchestra.run.smithi176.stdout:2408 trunc from 0xb0b2bed to 0x35eca9b 2024-09-14T09:06:20.448 INFO:teuthology.orchestra.run.smithi176.stdout:2409 punch from 0x2338bbd to 0x23402c4, (0x7707 bytes) 2024-09-14T09:06:20.448 INFO:teuthology.orchestra.run.smithi176.stdout:2411 read 0x521346 thru 0x530b23 (0xf7de bytes) 2024-09-14T09:06:20.448 INFO:teuthology.orchestra.run.smithi176.stdout:2412 read 0x347b101 thru 0x3485017 (0x9f17 bytes) 2024-09-14T09:06:20.449 INFO:teuthology.orchestra.run.smithi176.stdout:2413 read 0x2d94634 thru 0x2da01a3 (0xbb70 bytes) 2024-09-14T09:06:20.449 INFO:teuthology.orchestra.run.smithi176.stdout:2416 read 0x146ef5b thru 0x1471a0b (0x2ab1 bytes) 2024-09-14T09:06:20.449 INFO:teuthology.orchestra.run.smithi176.stdout:2417 write 0x7160240 thru 0x716abd5 (0xa996 bytes) 2024-09-14T09:06:20.449 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:20.447+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:20.465 INFO:teuthology.orchestra.run.smithi176.stdout:2419 write 0xa1a4fb5 thru 0xa1b4f49 (0xff95 bytes) 2024-09-14T09:06:20.473 INFO:teuthology.orchestra.run.smithi176.stdout:2420 read 0x74f4fc3 thru 0x74fefbf (0x9ffd bytes) 2024-09-14T09:06:20.473 INFO:teuthology.orchestra.run.smithi176.stdout:2421 read 0x2bab6c5 thru 0x2bb0efc (0x5838 bytes) 2024-09-14T09:06:20.491 INFO:teuthology.orchestra.run.smithi176.stdout:2423 write 0xcf07851 thru 0xcf13fe1 (0xc791 bytes) 2024-09-14T09:06:20.498 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:20.498 INFO:teuthology.orchestra.run.smithi176.stdout:2425 write 0x2dbe4b9 thru 0x2dbe82d (0x375 bytes) 2024-09-14T09:06:20.504 INFO:teuthology.orchestra.run.smithi176.stdout:2426 write 0x9a36205 thru 0x9a394f5 (0x32f1 bytes) 2024-09-14T09:06:20.510 INFO:teuthology.orchestra.run.smithi176.stdout:2427 write 0xb2d62e1 thru 0xb2e04c8 (0xa1e8 bytes) 2024-09-14T09:06:20.523 INFO:teuthology.orchestra.run.smithi176.stdout:2428 write 0x513490e thru 0x513f999 (0xb08c bytes) 2024-09-14T09:06:20.535 INFO:teuthology.orchestra.run.smithi176.stdout:2429 read 0x1814c9 thru 0x19085e (0xf396 bytes) 2024-09-14T09:06:20.537 INFO:teuthology.orchestra.run.smithi176.stdout:2432 write 0x173d755 thru 0x1744210 (0x6abc bytes) 2024-09-14T09:06:20.547 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:20.547+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:20.547 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:20.547 INFO:teuthology.orchestra.run.smithi176.stdout:2437 read 0x2cfcc45 thru 0x2d04f3d (0x82f9 bytes) 2024-09-14T09:06:20.548 INFO:teuthology.orchestra.run.smithi176.stdout:2438 write 0x80f624a thru 0x80fb988 (0x573f bytes) 2024-09-14T09:06:20.556 INFO:teuthology.orchestra.run.smithi176.stdout:2440 punch from 0x27ee39a to 0x27f0b5a, (0x27c0 bytes) 2024-09-14T09:06:20.556 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:20.557 INFO:teuthology.orchestra.run.smithi176.stdout:2442 write 0x835dd45 thru 0x8366391 (0x864d bytes) 2024-09-14T09:06:20.566 INFO:teuthology.orchestra.run.smithi176.stdout:2443 punch from 0x5bdf259 to 0x5be449b, (0x5242 bytes) 2024-09-14T09:06:20.583 INFO:teuthology.orchestra.run.smithi176.stdout:2445 write 0xde1887b thru 0xde26ce8 (0xe46e bytes) 2024-09-14T09:06:20.590 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:20.591 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:20.591 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:20.591 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:20.592 INFO:teuthology.orchestra.run.smithi176.stdout:2446 write 0x1fa962f thru 0x1fa9d27 (0x6f9 bytes) 2024-09-14T09:06:20.598 INFO:teuthology.orchestra.run.smithi176.stdout:2449 punch from 0xbc3b329 to 0xbc44995, (0x966c bytes) 2024-09-14T09:06:20.598 INFO:teuthology.orchestra.run.smithi176.stdout:2450 write 0x67d7ec2 thru 0x67e64ac (0xe5eb bytes) 2024-09-14T09:06:20.607 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:06:20.609 INFO:teuthology.orchestra.run.smithi176.stdout:2451 read 0x6b77a54 thru 0x6b81570 (0x9b1d bytes) 2024-09-14T09:06:20.610 INFO:teuthology.orchestra.run.smithi176.stdout:2454 write 0x68d752c thru 0x68e5793 (0xe268 bytes) 2024-09-14T09:06:20.617 INFO:teuthology.orchestra.run.smithi176.stdout:2455 read 0x2410a67 thru 0x2419f85 (0x951f bytes) 2024-09-14T09:06:20.617 INFO:teuthology.orchestra.run.smithi176.stdout:2456 read 0x8c4d2e0 thru 0x8c58e58 (0xbb79 bytes) 2024-09-14T09:06:20.617 INFO:teuthology.orchestra.run.smithi176.stdout:2457 read 0xb88492b thru 0xb88622e (0x1904 bytes) 2024-09-14T09:06:20.617 INFO:teuthology.orchestra.run.smithi176.stdout:2458 writesame 0x3277966 thru 0x32795e0 data_size 0x1(0x1c7b bytes) 2024-09-14T09:06:20.648 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:20.647+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:20.650 INFO:teuthology.orchestra.run.smithi176.stdout:2459 trunc from 0xde26ce9 to 0x4e1fcb7 2024-09-14T09:06:20.715 INFO:teuthology.orchestra.run.smithi176.stdout:2461 write 0xd75ca6c thru 0xd75fd97 (0x332c bytes) 2024-09-14T09:06:20.733 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:20.733 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:20.733 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:20.733 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:20.733 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:20.738 INFO:teuthology.orchestra.run.smithi176.stdout:2462 writesame 0xed84e86 thru 0xed86e27 data_size 0x1(0x1fa2 bytes) 2024-09-14T09:06:20.738 INFO:teuthology.orchestra.run.smithi176.stdout:2463 read 0x9e46e57 thru 0x9e48f8a (0x2134 bytes) 2024-09-14T09:06:20.738 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:20.738 INFO:teuthology.orchestra.run.smithi176.stdout:2469 writesame 0xb6b9583 thru 0xb6bb224 data_size 0x5(0x1ca2 bytes) 2024-09-14T09:06:20.745 INFO:teuthology.orchestra.run.smithi176.stdout:2471 read 0x9b5959f thru 0x9b5b10e (0x1b70 bytes) 2024-09-14T09:06:20.745 INFO:teuthology.orchestra.run.smithi176.stdout:2473 read 0xd32792 thru 0xd37b08 (0x5377 bytes) 2024-09-14T09:06:20.746 INFO:teuthology.orchestra.run.smithi176.stdout:2474 read 0xb9c550d thru 0xb9c680f (0x1303 bytes) 2024-09-14T09:06:20.746 INFO:teuthology.orchestra.run.smithi176.stdout:2475 writesame 0x9278880 thru 0x9285db7 data_size 0x4(0xd538 bytes) 2024-09-14T09:06:20.748 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:20.747+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:20.749 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:06:20.821 INFO:teuthology.orchestra.run.smithi176.stdout:2476 trunc from 0xed86e28 to 0x26d290 2024-09-14T09:06:20.849 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:20.849 INFO:teuthology.orchestra.run.smithi176.stdout:2480 read 0x1d144 thru 0x212e2 (0x419f bytes) 2024-09-14T09:06:20.850 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:20.851+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:20.851 INFO:teuthology.orchestra.run.smithi176.stdout:2481 read 0x269a28 thru 0x26d28f (0x3868 bytes) 2024-09-14T09:06:20.872 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:20.873 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:20.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:20.873 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:20.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.873 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:9982 3.1a 3:5af3e476:::rbd_data.115eadec874.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:06:20.873 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:20.873 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 43.275715001999998, 2024-09-14T09:06:20.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042276341000000002, 2024-09-14T09:06:20.873 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.873 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.873 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:20.873 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.874 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.874 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.874 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 9982 2024-09-14T09:06:20.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.874 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:20.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:20.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597970+0000", 2024-09-14T09:06:20.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0599999999999999e-07 2024-09-14T09:06:20.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597974+0000", 2024-09-14T09:06:20.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2649999999999998e-06 2024-09-14T09:06:20.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597986+0000", 2024-09-14T09:06:20.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1019000000000001e-05 2024-09-14T09:06:20.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597991+0000", 2024-09-14T09:06:20.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5049999999999998e-06 2024-09-14T09:06:20.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.598083+0000", 2024-09-14T09:06:20.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2062e-05 2024-09-14T09:06:20.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.598227+0000", 2024-09-14T09:06:20.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000143609 2024-09-14T09:06:20.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:20.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.603107+0000", 2024-09-14T09:06:20.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048805819999999996 2024-09-14T09:06:20.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:20.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640002+0000", 2024-09-14T09:06:20.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.036894641999999998 2024-09-14T09:06:20.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:20.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640077+0000", 2024-09-14T09:06:20.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4927999999999998e-05 2024-09-14T09:06:20.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:20.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640174+0000", 2024-09-14T09:06:20.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7417999999999995e-05 2024-09-14T09:06:20.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640246+0000", 2024-09-14T09:06:20.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1505000000000001e-05 2024-09-14T09:06:20.878 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.878 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.878 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.879 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53587 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1212416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:20.879 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508126+0000", 2024-09-14T09:06:20.879 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.365558848, 2024-09-14T09:06:20.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033430271999999997, 2024-09-14T09:06:20.879 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.879 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.879 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:20.879 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.879 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.879 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.880 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53587 2024-09-14T09:06:20.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.880 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508126+0000", 2024-09-14T09:06:20.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508126+0000", 2024-09-14T09:06:20.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508127+0000", 2024-09-14T09:06:20.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.11e-06 2024-09-14T09:06:20.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508132+0000", 2024-09-14T09:06:20.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4770000000000002e-06 2024-09-14T09:06:20.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508151+0000", 2024-09-14T09:06:20.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9032999999999999e-05 2024-09-14T09:06:20.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508164+0000", 2024-09-14T09:06:20.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2673e-05 2024-09-14T09:06:20.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541369+0000", 2024-09-14T09:06:20.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033205367999999999 2024-09-14T09:06:20.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541455+0000", 2024-09-14T09:06:20.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5618000000000006e-05 2024-09-14T09:06:20.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541556+0000", 2024-09-14T09:06:20.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100993 2024-09-14T09:06:20.883 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.883 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.883 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.884 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53596 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:20.884 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508184+0000", 2024-09-14T09:06:20.884 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.365500215, 2024-09-14T09:06:20.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033494411000000002, 2024-09-14T09:06:20.884 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.884 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.884 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:20.884 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.884 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.884 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.885 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53596 2024-09-14T09:06:20.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.885 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508184+0000", 2024-09-14T09:06:20.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508184+0000", 2024-09-14T09:06:20.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508185+0000", 2024-09-14T09:06:20.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.217e-06 2024-09-14T09:06:20.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508191+0000", 2024-09-14T09:06:20.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6330000000000001e-06 2024-09-14T09:06:20.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508209+0000", 2024-09-14T09:06:20.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7804000000000001e-05 2024-09-14T09:06:20.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508223+0000", 2024-09-14T09:06:20.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3783e-05 2024-09-14T09:06:20.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541582+0000", 2024-09-14T09:06:20.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033359403000000003 2024-09-14T09:06:20.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541622+0000", 2024-09-14T09:06:20.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9937000000000001e-05 2024-09-14T09:06:20.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541679+0000", 2024-09-14T09:06:20.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6634e-05 2024-09-14T09:06:20.889 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.889 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.889 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.889 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53599 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:20.889 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508241+0000", 2024-09-14T09:06:20.890 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.3654433419999998, 2024-09-14T09:06:20.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033611720999999997, 2024-09-14T09:06:20.890 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.890 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.890 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:20.890 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.890 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.890 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.890 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53599 2024-09-14T09:06:20.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.891 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508241+0000", 2024-09-14T09:06:20.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508241+0000", 2024-09-14T09:06:20.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508242+0000", 2024-09-14T09:06:20.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1230000000000001e-06 2024-09-14T09:06:20.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508247+0000", 2024-09-14T09:06:20.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8439999999999997e-06 2024-09-14T09:06:20.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508276+0000", 2024-09-14T09:06:20.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9363e-05 2024-09-14T09:06:20.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508290+0000", 2024-09-14T09:06:20.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.364e-05 2024-09-14T09:06:20.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541712+0000", 2024-09-14T09:06:20.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033421713999999998 2024-09-14T09:06:20.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541785+0000", 2024-09-14T09:06:20.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3696999999999996e-05 2024-09-14T09:06:20.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541853+0000", 2024-09-14T09:06:20.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7340000000000002e-05 2024-09-14T09:06:20.894 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.894 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.894 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.894 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53601 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:20.894 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508310+0000", 2024-09-14T09:06:20.894 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.3653747259999998, 2024-09-14T09:06:20.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033872092999999999, 2024-09-14T09:06:20.895 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.895 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.895 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:20.895 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.895 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.895 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.895 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53601 2024-09-14T09:06:20.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.895 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508310+0000", 2024-09-14T09:06:20.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508310+0000", 2024-09-14T09:06:20.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508311+0000", 2024-09-14T09:06:20.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.11e-06 2024-09-14T09:06:20.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508317+0000", 2024-09-14T09:06:20.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.127e-06 2024-09-14T09:06:20.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508335+0000", 2024-09-14T09:06:20.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8097e-05 2024-09-14T09:06:20.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508363+0000", 2024-09-14T09:06:20.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8089000000000002e-05 2024-09-14T09:06:20.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541929+0000", 2024-09-14T09:06:20.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033566046000000002 2024-09-14T09:06:20.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542062+0000", 2024-09-14T09:06:20.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013268000000000001 2024-09-14T09:06:20.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542182+0000", 2024-09-14T09:06:20.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119944 2024-09-14T09:06:20.898 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.898 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.898 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.898 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53603 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508409+0000", 2024-09-14T09:06:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.3652749630000001, 2024-09-14T09:06:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033982513999999998, 2024-09-14T09:06:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53603 2024-09-14T09:06:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508409+0000", 2024-09-14T09:06:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508409+0000", 2024-09-14T09:06:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508411+0000", 2024-09-14T09:06:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3859999999999999e-06 2024-09-14T09:06:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508417+0000", 2024-09-14T09:06:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0639999999999999e-06 2024-09-14T09:06:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508437+0000", 2024-09-14T09:06:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0055999999999999e-05 2024-09-14T09:06:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508451+0000", 2024-09-14T09:06:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4144e-05 2024-09-14T09:06:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542235+0000", 2024-09-14T09:06:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033783531999999998 2024-09-14T09:06:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542316+0000", 2024-09-14T09:06:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1704000000000002e-05 2024-09-14T09:06:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542392+0000", 2024-09-14T09:06:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5628000000000002e-05 2024-09-14T09:06:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53638 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508472+0000", 2024-09-14T09:06:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.3652124140000002, 2024-09-14T09:06:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034203029000000003, 2024-09-14T09:06:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53638 2024-09-14T09:06:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508472+0000", 2024-09-14T09:06:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508472+0000", 2024-09-14T09:06:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508473+0000", 2024-09-14T09:06:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3769999999999999e-06 2024-09-14T09:06:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508492+0000", 2024-09-14T09:06:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8643000000000001e-05 2024-09-14T09:06:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508512+0000", 2024-09-14T09:06:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9516000000000001e-05 2024-09-14T09:06:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508525+0000", 2024-09-14T09:06:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3457e-05 2024-09-14T09:06:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542480+0000", 2024-09-14T09:06:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033954766999999997 2024-09-14T09:06:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542579+0000", 2024-09-14T09:06:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9497e-05 2024-09-14T09:06:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542675+0000", 2024-09-14T09:06:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5772000000000004e-05 2024-09-14T09:06:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53646 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508545+0000", 2024-09-14T09:06:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.3651391909999999, 2024-09-14T09:06:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034341172000000003, 2024-09-14T09:06:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53646 2024-09-14T09:06:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508545+0000", 2024-09-14T09:06:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508545+0000", 2024-09-14T09:06:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508547+0000", 2024-09-14T09:06:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7269999999999999e-06 2024-09-14T09:06:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508553+0000", 2024-09-14T09:06:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6930000000000004e-06 2024-09-14T09:06:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508571+0000", 2024-09-14T09:06:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.844e-05 2024-09-14T09:06:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508584+0000", 2024-09-14T09:06:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3157e-05 2024-09-14T09:06:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542730+0000", 2024-09-14T09:06:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034145967999999999 2024-09-14T09:06:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542812+0000", 2024-09-14T09:06:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1761000000000005e-05 2024-09-14T09:06:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542886+0000", 2024-09-14T09:06:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4425999999999995e-05 2024-09-14T09:06:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53647 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 999424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508603+0000", 2024-09-14T09:06:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.3650815180000002, 2024-09-14T09:06:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034662680000000001, 2024-09-14T09:06:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53647 2024-09-14T09:06:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508603+0000", 2024-09-14T09:06:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508603+0000", 2024-09-14T09:06:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508604+0000", 2024-09-14T09:06:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.173e-06 2024-09-14T09:06:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508622+0000", 2024-09-14T09:06:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8326e-05 2024-09-14T09:06:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508641+0000", 2024-09-14T09:06:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8213999999999999e-05 2024-09-14T09:06:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508656+0000", 2024-09-14T09:06:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5336e-05 2024-09-14T09:06:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542946+0000", 2024-09-14T09:06:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034290270999999997 2024-09-14T09:06:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543089+0000", 2024-09-14T09:06:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014298300000000001 2024-09-14T09:06:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543266+0000", 2024-09-14T09:06:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000176377 2024-09-14T09:06:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53648 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508674+0000", 2024-09-14T09:06:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.3650104619999999, 2024-09-14T09:06:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034778557000000002, 2024-09-14T09:06:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53648 2024-09-14T09:06:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508674+0000", 2024-09-14T09:06:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508674+0000", 2024-09-14T09:06:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508675+0000", 2024-09-14T09:06:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.223e-06 2024-09-14T09:06:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508680+0000", 2024-09-14T09:06:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:06:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508694+0000", 2024-09-14T09:06:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4399999999999999e-05 2024-09-14T09:06:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508707+0000", 2024-09-14T09:06:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2723e-05 2024-09-14T09:06:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543316+0000", 2024-09-14T09:06:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034609648999999999 2024-09-14T09:06:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543386+0000", 2024-09-14T09:06:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9548999999999994e-05 2024-09-14T09:06:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543453+0000", 2024-09-14T09:06:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6595999999999997e-05 2024-09-14T09:06:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53649 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1097728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508725+0000", 2024-09-14T09:06:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.3649596590000002, 2024-09-14T09:06:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035096176, 2024-09-14T09:06:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53649 2024-09-14T09:06:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508725+0000", 2024-09-14T09:06:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508725+0000", 2024-09-14T09:06:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508726+0000", 2024-09-14T09:06:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1629999999999999e-06 2024-09-14T09:06:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508731+0000", 2024-09-14T09:06:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.04e-06 2024-09-14T09:06:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508749+0000", 2024-09-14T09:06:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7693000000000002e-05 2024-09-14T09:06:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508762+0000", 2024-09-14T09:06:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3190000000000001e-05 2024-09-14T09:06:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543515+0000", 2024-09-14T09:06:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034753066999999999 2024-09-14T09:06:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543650+0000", 2024-09-14T09:06:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013507100000000001 2024-09-14T09:06:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543821+0000", 2024-09-14T09:06:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017095200000000001 2024-09-14T09:06:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53650 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508780+0000", 2024-09-14T09:06:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.3649040130000003, 2024-09-14T09:06:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035246841000000001, 2024-09-14T09:06:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53650 2024-09-14T09:06:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508780+0000", 2024-09-14T09:06:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508780+0000", 2024-09-14T09:06:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508782+0000", 2024-09-14T09:06:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1799999999999999e-06 2024-09-14T09:06:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508787+0000", 2024-09-14T09:06:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9570000000000001e-06 2024-09-14T09:06:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508804+0000", 2024-09-14T09:06:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7523000000000001e-05 2024-09-14T09:06:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508817+0000", 2024-09-14T09:06:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.259e-05 2024-09-14T09:06:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543866+0000", 2024-09-14T09:06:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035049640999999999 2024-09-14T09:06:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543951+0000", 2024-09-14T09:06:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4897999999999999e-05 2024-09-14T09:06:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544027+0000", 2024-09-14T09:06:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6051999999999996e-05 2024-09-14T09:06:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53651 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1327104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508868+0000", 2024-09-14T09:06:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.3648163740000001, 2024-09-14T09:06:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035314533000000002, 2024-09-14T09:06:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53651 2024-09-14T09:06:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508868+0000", 2024-09-14T09:06:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508868+0000", 2024-09-14T09:06:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508869+0000", 2024-09-14T09:06:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.393e-06 2024-09-14T09:06:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508876+0000", 2024-09-14T09:06:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1539999999999999e-06 2024-09-14T09:06:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508913+0000", 2024-09-14T09:06:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6980000000000002e-05 2024-09-14T09:06:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508928+0000", 2024-09-14T09:06:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5543e-05 2024-09-14T09:06:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544061+0000", 2024-09-14T09:06:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035132656999999998 2024-09-14T09:06:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544127+0000", 2024-09-14T09:06:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6036000000000003e-05 2024-09-14T09:06:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544183+0000", 2024-09-14T09:06:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5770000000000003e-05 2024-09-14T09:06:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53652 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508952+0000", 2024-09-14T09:06:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.364732311, 2024-09-14T09:06:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035409204, 2024-09-14T09:06:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53652 2024-09-14T09:06:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508952+0000", 2024-09-14T09:06:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508952+0000", 2024-09-14T09:06:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508953+0000", 2024-09-14T09:06:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3599999999999999e-06 2024-09-14T09:06:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508959+0000", 2024-09-14T09:06:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9270000000000002e-06 2024-09-14T09:06:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509001+0000", 2024-09-14T09:06:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1896e-05 2024-09-14T09:06:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509022+0000", 2024-09-14T09:06:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.035e-05 2024-09-14T09:06:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544219+0000", 2024-09-14T09:06:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035197581999999998 2024-09-14T09:06:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544295+0000", 2024-09-14T09:06:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5483999999999998e-05 2024-09-14T09:06:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544361+0000", 2024-09-14T09:06:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6605000000000004e-05 2024-09-14T09:06:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53653 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509038+0000", 2024-09-14T09:06:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.3646461570000001, 2024-09-14T09:06:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035495397999999997, 2024-09-14T09:06:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53653 2024-09-14T09:06:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509038+0000", 2024-09-14T09:06:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509038+0000", 2024-09-14T09:06:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509039+0000", 2024-09-14T09:06:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9500000000000001e-07 2024-09-14T09:06:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509044+0000", 2024-09-14T09:06:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4179999999999999e-06 2024-09-14T09:06:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509054+0000", 2024-09-14T09:06:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0891e-05 2024-09-14T09:06:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509061+0000", 2024-09-14T09:06:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3260000000000001e-06 2024-09-14T09:06:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544388+0000", 2024-09-14T09:06:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035327648000000003 2024-09-14T09:06:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544461+0000", 2024-09-14T09:06:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2669999999999994e-05 2024-09-14T09:06:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544534+0000", 2024-09-14T09:06:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2550000000000002e-05 2024-09-14T09:06:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53675 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509071+0000", 2024-09-14T09:06:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.3646137700000001, 2024-09-14T09:06:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035630230999999998, 2024-09-14T09:06:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53675 2024-09-14T09:06:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509071+0000", 2024-09-14T09:06:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509071+0000", 2024-09-14T09:06:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509071+0000", 2024-09-14T09:06:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6899999999999997e-07 2024-09-14T09:06:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509074+0000", 2024-09-14T09:06:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5979999999999999e-06 2024-09-14T09:06:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509081+0000", 2024-09-14T09:06:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0790000000000003e-06 2024-09-14T09:06:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509086+0000", 2024-09-14T09:06:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4280000000000004e-06 2024-09-14T09:06:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544561+0000", 2024-09-14T09:06:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035474736 2024-09-14T09:06:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544634+0000", 2024-09-14T09:06:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3258999999999998e-05 2024-09-14T09:06:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544701+0000", 2024-09-14T09:06:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6562000000000004e-05 2024-09-14T09:06:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53676 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509095+0000", 2024-09-14T09:06:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.364589552, 2024-09-14T09:06:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035734397000000001, 2024-09-14T09:06:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53676 2024-09-14T09:06:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509095+0000", 2024-09-14T09:06:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509095+0000", 2024-09-14T09:06:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509095+0000", 2024-09-14T09:06:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5700000000000002e-07 2024-09-14T09:06:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509098+0000", 2024-09-14T09:06:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.553e-06 2024-09-14T09:06:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509105+0000", 2024-09-14T09:06:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1350000000000003e-06 2024-09-14T09:06:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509110+0000", 2024-09-14T09:06:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1370000000000001e-06 2024-09-14T09:06:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544734+0000", 2024-09-14T09:06:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035624054000000002 2024-09-14T09:06:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544782+0000", 2024-09-14T09:06:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7956e-05 2024-09-14T09:06:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544829+0000", 2024-09-14T09:06:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7005000000000001e-05 2024-09-14T09:06:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53677 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509120+0000", 2024-09-14T09:06:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.3645640239999999, 2024-09-14T09:06:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035825663000000001, 2024-09-14T09:06:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53677 2024-09-14T09:06:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509120+0000", 2024-09-14T09:06:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509120+0000", 2024-09-14T09:06:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509121+0000", 2024-09-14T09:06:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2e-07 2024-09-14T09:06:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509136+0000", 2024-09-14T09:06:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.517e-05 2024-09-14T09:06:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509145+0000", 2024-09-14T09:06:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4880000000000007e-06 2024-09-14T09:06:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509150+0000", 2024-09-14T09:06:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9100000000000002e-06 2024-09-14T09:06:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544860+0000", 2024-09-14T09:06:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035709739999999997 2024-09-14T09:06:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544904+0000", 2024-09-14T09:06:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3791999999999997e-05 2024-09-14T09:06:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544946+0000", 2024-09-14T09:06:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2042999999999997e-05 2024-09-14T09:06:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53678 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509160+0000", 2024-09-14T09:06:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.3645240559999996, 2024-09-14T09:06:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035923798999999999, 2024-09-14T09:06:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53678 2024-09-14T09:06:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509160+0000", 2024-09-14T09:06:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509160+0000", 2024-09-14T09:06:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509161+0000", 2024-09-14T09:06:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9599999999999999e-07 2024-09-14T09:06:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509163+0000", 2024-09-14T09:06:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3410000000000001e-06 2024-09-14T09:06:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509171+0000", 2024-09-14T09:06:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5519999999999999e-06 2024-09-14T09:06:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509176+0000", 2024-09-14T09:06:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4750000000000001e-06 2024-09-14T09:06:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544975+0000", 2024-09-14T09:06:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035798862000000001 2024-09-14T09:06:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.545033+0000", 2024-09-14T09:06:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8148e-05 2024-09-14T09:06:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.545084+0000", 2024-09-14T09:06:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0825000000000002e-05 2024-09-14T09:06:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53838 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 573440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.518551+0000", 2024-09-14T09:06:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.3551335069999997, 2024-09-14T09:06:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033585690000000001, 2024-09-14T09:06:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53838 2024-09-14T09:06:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518551+0000", 2024-09-14T09:06:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518551+0000", 2024-09-14T09:06:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518551+0000", 2024-09-14T09:06:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2499999999999995e-07 2024-09-14T09:06:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518554+0000", 2024-09-14T09:06:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9450000000000002e-06 2024-09-14T09:06:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518561+0000", 2024-09-14T09:06:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4849999999999997e-06 2024-09-14T09:06:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518572+0000", 2024-09-14T09:06:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0818999999999999e-05 2024-09-14T09:06:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.551823+0000", 2024-09-14T09:06:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033250864999999998 2024-09-14T09:06:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.551967+0000", 2024-09-14T09:06:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014399700000000001 2024-09-14T09:06:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.552137+0000", 2024-09-14T09:06:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170054 2024-09-14T09:06:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:20.976 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:20.978 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:06:20.978 INFO:teuthology.orchestra.run.smithi176.stdout:2482 writesame 0xb56cece thru 0xb57211a data_size 0x3(0x524d bytes) 2024-09-14T09:06:20.978 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:20.979 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:20.979 INFO:teuthology.orchestra.run.smithi176.stdout:2486 punch from 0x117b85f to 0x11859bf, (0xa160 bytes) 2024-09-14T09:06:20.979 INFO:teuthology.orchestra.run.smithi176.stdout:2487 write 0x1c7a8ac thru 0x1c829bb (0x8110 bytes) 2024-09-14T09:06:20.979 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:20.979 INFO:teuthology.orchestra.run.smithi176.stdout:2489 write 0x5742add thru 0x574395c (0xe80 bytes) 2024-09-14T09:06:20.979 INFO:teuthology.orchestra.run.smithi176.stdout:2491 punch from 0xb211292 to 0xb211a7f, (0x7ed bytes) 2024-09-14T09:06:20.979 INFO:teuthology.orchestra.run.smithi176.stdout:2492 read 0x1af5fb7 thru 0x1afcb13 (0x6b5d bytes) 2024-09-14T09:06:20.979 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:20.979 INFO:teuthology.orchestra.run.smithi176.stdout:2494 trunc from 0xb57211b to 0x64c9529 2024-09-14T09:06:20.979 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:20.979 INFO:teuthology.orchestra.run.smithi176.stdout:2498 read 0x2679de6 thru 0x267b038 (0x1253 bytes) 2024-09-14T09:06:20.979 INFO:teuthology.orchestra.run.smithi176.stdout:2499 read 0x12adc9f thru 0x12add9f (0x101 bytes) 2024-09-14T09:06:20.979 INFO:teuthology.orchestra.run.smithi176.stdout:2500 write 0x68aaa9d thru 0x68afa31 (0x4f95 bytes) 2024-09-14T09:06:20.980 INFO:teuthology.orchestra.run.smithi176.stdout:2501 trunc from 0x68afa32 to 0x7ec3e2 2024-09-14T09:06:20.980 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:20.975+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:21.020 INFO:teuthology.orchestra.run.smithi176.stdout:2502 write 0x84859c1 thru 0x848c740 (0x6d80 bytes) 2024-09-14T09:06:21.027 INFO:teuthology.orchestra.run.smithi176.stdout:2503 write 0x4c280e7 thru 0x4c367d5 (0xe6ef bytes) 2024-09-14T09:06:21.040 INFO:teuthology.orchestra.run.smithi176.stdout:2505 write 0x115b7c1 thru 0x115f7ec (0x402c bytes) 2024-09-14T09:06:21.049 INFO:teuthology.orchestra.run.smithi176.stdout:2506 punch from 0x873018 to 0x87ff20, (0xcf08 bytes) 2024-09-14T09:06:21.049 INFO:teuthology.orchestra.run.smithi176.stdout:2507 writesame 0x58c5933 thru 0x58cfe90 data_size 0x3d(0xa55e bytes) 2024-09-14T09:06:21.057 INFO:teuthology.orchestra.run.smithi176.stdout:2508 writesame 0x6a3e636 thru 0x6a49b8c data_size 0x1(0xb557 bytes) 2024-09-14T09:06:21.078 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:21.075+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:21.158 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:21.158 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:21.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.158 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65035 3.1d 3:bb585d47:::rbd_data.115e21c31fcf.0000000000000034:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:06:21.158 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.068842+0000", 2024-09-14T09:06:21.158 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.093397010000000003, 2024-09-14T09:06:21.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.093432108999999999, 2024-09-14T09:06:21.159 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.159 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.159 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for sub ops", 2024-09-14T09:06:21.159 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.159 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.159 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.159 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65035 2024-09-14T09:06:21.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.159 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068842+0000", 2024-09-14T09:06:21.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068842+0000", 2024-09-14T09:06:21.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068843+0000", 2024-09-14T09:06:21.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.221e-06 2024-09-14T09:06:21.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068849+0000", 2024-09-14T09:06:21.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4659999999999998e-06 2024-09-14T09:06:21.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068861+0000", 2024-09-14T09:06:21.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2204e-05 2024-09-14T09:06:21.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068871+0000", 2024-09-14T09:06:21.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0318e-05 2024-09-14T09:06:21.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068942+0000", 2024-09-14T09:06:21.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0438999999999994e-05 2024-09-14T09:06:21.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:21.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.069005+0000", 2024-09-14T09:06:21.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3076000000000004e-05 2024-09-14T09:06:21.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:21.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.074957+0000", 2024-09-14T09:06:21.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059524549999999997 2024-09-14T09:06:21.163 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.163 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.163 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.163 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:65035 3.1d e65/23)", 2024-09-14T09:06:21.163 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.087665+0000", 2024-09-14T09:06:21.163 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.074574100000000004, 2024-09-14T09:06:21.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.074676985000000001, 2024-09-14T09:06:21.163 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.163 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.164 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:21.164 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087665+0000", 2024-09-14T09:06:21.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087665+0000", 2024-09-14T09:06:21.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087667+0000", 2024-09-14T09:06:21.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.818e-06 2024-09-14T09:06:21.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087673+0000", 2024-09-14T09:06:21.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9259999999999997e-06 2024-09-14T09:06:21.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087692+0000", 2024-09-14T09:06:21.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9715e-05 2024-09-14T09:06:21.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087703+0000", 2024-09-14T09:06:21.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.094e-05 2024-09-14T09:06:21.166 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.166 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.166 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.166 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.166 INFO:teuthology.orchestra.run.smithi183.stdout: ], 2024-09-14T09:06:21.166 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 2 2024-09-14T09:06:21.166 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:21.167 INFO:teuthology.orchestra.run.smithi176.stdout:2509 read 0x11dc44c thru 0x11dc824 (0x3d9 bytes) 2024-09-14T09:06:21.167 INFO:teuthology.orchestra.run.smithi176.stdout:2510 write 0x4da6187 thru 0x4da83e1 (0x225b bytes) 2024-09-14T09:06:21.167 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:21.167 INFO:teuthology.orchestra.run.smithi176.stdout:2513 read 0x6bce7df thru 0x6bda992 (0xc1b4 bytes) 2024-09-14T09:06:21.167 INFO:teuthology.orchestra.run.smithi176.stdout:2514 read 0x5caf983 thru 0x5cb6092 (0x6710 bytes) 2024-09-14T09:06:21.167 INFO:teuthology.orchestra.run.smithi176.stdout:2516 read 0x58eaf0f thru 0x58eb041 (0x133 bytes) 2024-09-14T09:06:21.174 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:06:21.179 INFO:teuthology.orchestra.run.smithi176.stdout:2517 writesame 0x8fa6bdf thru 0x8fb4ad3 data_size 0x1(0xdef5 bytes) 2024-09-14T09:06:21.191 INFO:teuthology.orchestra.run.smithi176.stdout:2519 trunc from 0x8fb4ad4 to 0x47f3434 2024-09-14T09:06:21.211 INFO:teuthology.orchestra.run.smithi176.stdout:2520 read 0x8f01db thru 0x8f7ad8 (0x78fe bytes) 2024-09-14T09:06:21.211 INFO:teuthology.orchestra.run.smithi176.stdout:2523 read 0x13e8d69 thru 0x13ec94b (0x3be3 bytes) 2024-09-14T09:06:21.211 INFO:teuthology.orchestra.run.smithi176.stdout:2524 punch from 0x17b7fc7 to 0x17bd366, (0x539f bytes) 2024-09-14T09:06:21.212 INFO:teuthology.orchestra.run.smithi176.stdout:2525 trunc from 0x47f3434 to 0x41e45fc 2024-09-14T09:06:21.220 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:21.223+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:21.236 INFO:teuthology.orchestra.run.smithi176.stdout:2526 write 0x596d781 thru 0x597be3d (0xe6bd bytes) 2024-09-14T09:06:21.243 INFO:teuthology.orchestra.run.smithi176.stdout:2527 read 0x97cd69 thru 0x9870d1 (0xa369 bytes) 2024-09-14T09:06:21.267 INFO:teuthology.orchestra.run.smithi176.stdout:2528 writesame 0x9714144 thru 0x972261a data_size 0x1(0xe4d7 bytes) 2024-09-14T09:06:21.267 INFO:teuthology.orchestra.run.smithi176.stdout:2533 read 0x539591 thru 0x547a6d (0xe4dd bytes) 2024-09-14T09:06:21.268 INFO:teuthology.orchestra.run.smithi176.stdout:2534 read 0x5b2287 thru 0x5b3f63 (0x1cdd bytes) 2024-09-14T09:06:21.268 INFO:teuthology.orchestra.run.smithi176.stdout:2535 writesame 0x3ec7f7 thru 0x3fb082 data_size 0x2c(0xe88c bytes) 2024-09-14T09:06:21.276 INFO:teuthology.orchestra.run.smithi176.stdout:2538 write 0x1aaecf6 thru 0x1abba2c (0xcd37 bytes) 2024-09-14T09:06:21.279 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:21.279+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:21.288 INFO:teuthology.orchestra.run.smithi176.stdout:2539 read 0x1666473 thru 0x166e1ac (0x7d3a bytes) 2024-09-14T09:06:21.288 INFO:teuthology.orchestra.run.smithi176.stdout:2540 punch from 0xc48880 to 0xc4978a, (0xf0a bytes) 2024-09-14T09:06:21.288 INFO:teuthology.orchestra.run.smithi176.stdout:2541 read 0x92ec2f0 thru 0x92edbc1 (0x18d2 bytes) 2024-09-14T09:06:21.299 INFO:teuthology.orchestra.run.smithi176.stdout:2543 trunc from 0x972261b to 0x5a39743 2024-09-14T09:06:21.301 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:21.301 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:21.301 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:21.301 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:21.301 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:21.312 INFO:teuthology.orchestra.run.smithi176.stdout:2544 writesame 0x5120f12 thru 0x5123e50 data_size 0x29(0x2f3f bytes) 2024-09-14T09:06:21.316 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:06:21.333 INFO:teuthology.orchestra.run.smithi176.stdout:2545 writesame 0x5aee0ed thru 0x5af76bb data_size 0x1(0x95cf bytes) 2024-09-14T09:06:21.355 INFO:teuthology.orchestra.run.smithi176.stdout:2547 writesame 0x8170dc5 thru 0x8176777 data_size 0x1(0x59b3 bytes) 2024-09-14T09:06:21.355 INFO:teuthology.orchestra.run.smithi176.stdout:2549 read 0x35406d0 thru 0x3544a36 (0x4367 bytes) 2024-09-14T09:06:21.358 INFO:teuthology.orchestra.run.smithi176.stdout:2551 trunc from 0x8176778 to 0x9086b19 2024-09-14T09:06:21.371 INFO:teuthology.orchestra.run.smithi176.stdout:2552 writesame 0x1003159 thru 0x100f43d data_size 0x1(0xc2e5 bytes) 2024-09-14T09:06:21.404 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:21.407+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:21.484 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:21.483+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:21.491 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:21.491 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:21.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:21.491 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:21.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.491 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49333 3.1d 3:be596f9e:::rbd_data.115e51d92ed2.0000000000000007:12 [read 950272~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:21.491 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:21.491 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.946346152, 2024-09-14T09:06:21.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072172603000000002, 2024-09-14T09:06:21.492 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.492 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.492 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:21.492 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.492 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.492 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.492 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49333 2024-09-14T09:06:21.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.492 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:21.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:21.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:21.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6000000000000003e-07 2024-09-14T09:06:21.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546216+0000", 2024-09-14T09:06:21.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5339999999999998e-06 2024-09-14T09:06:21.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546224+0000", 2024-09-14T09:06:21.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7770000000000002e-06 2024-09-14T09:06:21.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546230+0000", 2024-09-14T09:06:21.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3400000000000003e-06 2024-09-14T09:06:21.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.617881+0000", 2024-09-14T09:06:21.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071650488999999998 2024-09-14T09:06:21.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:21.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.617958+0000", 2024-09-14T09:06:21.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7018e-05 2024-09-14T09:06:21.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618385+0000", 2024-09-14T09:06:21.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00042768500000000002 2024-09-14T09:06:21.496 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.496 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.496 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.496 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49334 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:21.496 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:21.496 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.94631927, 2024-09-14T09:06:21.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072227886000000005, 2024-09-14T09:06:21.496 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.497 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.497 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:21.497 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.497 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.497 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.497 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49334 2024-09-14T09:06:21.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.497 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:21.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:21.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:21.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8899999999999999e-07 2024-09-14T09:06:21.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546243+0000", 2024-09-14T09:06:21.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.503e-06 2024-09-14T09:06:21.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546253+0000", 2024-09-14T09:06:21.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0210999999999999e-05 2024-09-14T09:06:21.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546267+0000", 2024-09-14T09:06:21.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4183e-05 2024-09-14T09:06:21.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618449+0000", 2024-09-14T09:06:21.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072181828000000003 2024-09-14T09:06:21.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618467+0000", 2024-09-14T09:06:21.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8471999999999999e-05 2024-09-14T09:06:21.501 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.501 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.501 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.501 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49335 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:21.501 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:21.501 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.946283119, 2024-09-14T09:06:21.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072255640999999995, 2024-09-14T09:06:21.502 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.502 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.502 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:21.502 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.502 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.502 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.502 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49335 2024-09-14T09:06:21.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.502 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:21.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:21.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:21.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9999999999999997e-07 2024-09-14T09:06:21.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546279+0000", 2024-09-14T09:06:21.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.695e-06 2024-09-14T09:06:21.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546288+0000", 2024-09-14T09:06:21.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7110000000000001e-06 2024-09-14T09:06:21.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546294+0000", 2024-09-14T09:06:21.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9830000000000001e-06 2024-09-14T09:06:21.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618521+0000", 2024-09-14T09:06:21.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072226948999999999 2024-09-14T09:06:21.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618531+0000", 2024-09-14T09:06:21.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0603e-05 2024-09-14T09:06:21.505 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.505 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.505 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.506 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49336 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:21.506 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:21.506 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.946257215999999, 2024-09-14T09:06:21.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072292223000000003, 2024-09-14T09:06:21.506 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.506 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.506 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:21.506 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.506 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.506 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.506 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49336 2024-09-14T09:06:21.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.507 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:21.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:21.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:21.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4000000000000001e-07 2024-09-14T09:06:21.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546305+0000", 2024-09-14T09:06:21.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3539999999999998e-06 2024-09-14T09:06:21.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546315+0000", 2024-09-14T09:06:21.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0431999999999999e-05 2024-09-14T09:06:21.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546321+0000", 2024-09-14T09:06:21.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7620000000000001e-06 2024-09-14T09:06:21.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618583+0000", 2024-09-14T09:06:21.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072262046999999996 2024-09-14T09:06:21.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618594+0000", 2024-09-14T09:06:21.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0988000000000001e-05 2024-09-14T09:06:21.509 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.510 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.510 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.510 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49337 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:21.510 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:21.510 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.946209602, 2024-09-14T09:06:21.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072294774000000006, 2024-09-14T09:06:21.510 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.510 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.510 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:21.510 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.510 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.510 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.511 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49337 2024-09-14T09:06:21.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.511 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:21.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:21.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546350+0000", 2024-09-14T09:06:21.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4600000000000004e-07 2024-09-14T09:06:21.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546381+0000", 2024-09-14T09:06:21.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0902000000000003e-05 2024-09-14T09:06:21.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546412+0000", 2024-09-14T09:06:21.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0825999999999999e-05 2024-09-14T09:06:21.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546438+0000", 2024-09-14T09:06:21.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6410999999999998e-05 2024-09-14T09:06:21.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618637+0000", 2024-09-14T09:06:21.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072198907000000007 2024-09-14T09:06:21.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618644+0000", 2024-09-14T09:06:21.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9820000000000002e-06 2024-09-14T09:06:21.513 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.513 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.513 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.514 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49338 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:21.514 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:21.514 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.946095574999999, 2024-09-14T09:06:21.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072233250999999998, 2024-09-14T09:06:21.514 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.514 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.514 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:21.514 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.514 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.514 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.514 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49338 2024-09-14T09:06:21.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.514 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:21.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:21.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546465+0000", 2024-09-14T09:06:21.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.455e-06 2024-09-14T09:06:21.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546481+0000", 2024-09-14T09:06:21.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5834000000000001e-05 2024-09-14T09:06:21.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546498+0000", 2024-09-14T09:06:21.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6937000000000001e-05 2024-09-14T09:06:21.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546508+0000", 2024-09-14T09:06:21.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0446e-05 2024-09-14T09:06:21.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618685+0000", 2024-09-14T09:06:21.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072177280999999996 2024-09-14T09:06:21.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618697+0000", 2024-09-14T09:06:21.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1297999999999999e-05 2024-09-14T09:06:21.517 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.517 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.517 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.518 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49339 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:21.518 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546535+0000", 2024-09-14T09:06:21.518 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.946023558, 2024-09-14T09:06:21.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072197266999999996, 2024-09-14T09:06:21.518 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.518 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.518 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:21.518 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.518 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.518 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.518 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49339 2024-09-14T09:06:21.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.518 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546535+0000", 2024-09-14T09:06:21.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546535+0000", 2024-09-14T09:06:21.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546536+0000", 2024-09-14T09:06:21.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1399999999999996e-07 2024-09-14T09:06:21.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546539+0000", 2024-09-14T09:06:21.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.089e-06 2024-09-14T09:06:21.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546560+0000", 2024-09-14T09:06:21.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1217e-05 2024-09-14T09:06:21.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546567+0000", 2024-09-14T09:06:21.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9e-06 2024-09-14T09:06:21.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618726+0000", 2024-09-14T09:06:21.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.07215887 2024-09-14T09:06:21.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618733+0000", 2024-09-14T09:06:21.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3770000000000001e-06 2024-09-14T09:06:21.521 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.521 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.521 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.522 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49340 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:21.522 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546578+0000", 2024-09-14T09:06:21.522 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.945980667000001, 2024-09-14T09:06:21.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072205187000000004, 2024-09-14T09:06:21.522 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.522 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.522 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:21.522 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.522 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.522 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.522 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49340 2024-09-14T09:06:21.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.522 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546578+0000", 2024-09-14T09:06:21.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546578+0000", 2024-09-14T09:06:21.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546579+0000", 2024-09-14T09:06:21.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1999999999999999e-07 2024-09-14T09:06:21.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546582+0000", 2024-09-14T09:06:21.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4460000000000001e-06 2024-09-14T09:06:21.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546603+0000", 2024-09-14T09:06:21.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0502000000000001e-05 2024-09-14T09:06:21.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546610+0000", 2024-09-14T09:06:21.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7150000000000001e-06 2024-09-14T09:06:21.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618770+0000", 2024-09-14T09:06:21.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072160600000000005 2024-09-14T09:06:21.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618783+0000", 2024-09-14T09:06:21.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:06:21.525 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.525 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.525 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.525 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49341 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:21.526 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546667+0000", 2024-09-14T09:06:21.526 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.945891559, 2024-09-14T09:06:21.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072181163000000007, 2024-09-14T09:06:21.526 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.526 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.526 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:21.526 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.526 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.526 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.526 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49341 2024-09-14T09:06:21.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.526 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546667+0000", 2024-09-14T09:06:21.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546667+0000", 2024-09-14T09:06:21.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546669+0000", 2024-09-14T09:06:21.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.486e-06 2024-09-14T09:06:21.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546674+0000", 2024-09-14T09:06:21.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5110000000000003e-06 2024-09-14T09:06:21.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546691+0000", 2024-09-14T09:06:21.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7051e-05 2024-09-14T09:06:21.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546703+0000", 2024-09-14T09:06:21.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1155000000000001e-05 2024-09-14T09:06:21.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618836+0000", 2024-09-14T09:06:21.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072133616999999997 2024-09-14T09:06:21.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618848+0000", 2024-09-14T09:06:21.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2343e-05 2024-09-14T09:06:21.529 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.529 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.529 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.529 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49386 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:21.529 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546806+0000", 2024-09-14T09:06:21.530 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.9457524, 2024-09-14T09:06:21.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072019206000000002, 2024-09-14T09:06:21.530 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.530 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.530 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:21.530 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.530 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.530 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.530 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49386 2024-09-14T09:06:21.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.530 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546806+0000", 2024-09-14T09:06:21.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546806+0000", 2024-09-14T09:06:21.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546808+0000", 2024-09-14T09:06:21.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.471e-06 2024-09-14T09:06:21.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546813+0000", 2024-09-14T09:06:21.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1800000000000004e-06 2024-09-14T09:06:21.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546830+0000", 2024-09-14T09:06:21.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.658e-05 2024-09-14T09:06:21.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546842+0000", 2024-09-14T09:06:21.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2442999999999999e-05 2024-09-14T09:06:21.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618816+0000", 2024-09-14T09:06:21.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071973920999999996 2024-09-14T09:06:21.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618826+0000", 2024-09-14T09:06:21.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.611e-06 2024-09-14T09:06:21.533 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.533 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.533 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.533 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49387 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:21.533 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546861+0000", 2024-09-14T09:06:21.533 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.945697754999999, 2024-09-14T09:06:21.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071996665000000001, 2024-09-14T09:06:21.533 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.533 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.533 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:21.534 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.534 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.534 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.534 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49387 2024-09-14T09:06:21.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.534 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546861+0000", 2024-09-14T09:06:21.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546861+0000", 2024-09-14T09:06:21.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546862+0000", 2024-09-14T09:06:21.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.186e-06 2024-09-14T09:06:21.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546867+0000", 2024-09-14T09:06:21.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9049999999999996e-06 2024-09-14T09:06:21.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546880+0000", 2024-09-14T09:06:21.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2815e-05 2024-09-14T09:06:21.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546889+0000", 2024-09-14T09:06:21.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5420000000000005e-06 2024-09-14T09:06:21.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618850+0000", 2024-09-14T09:06:21.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071961065000000005 2024-09-14T09:06:21.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618858+0000", 2024-09-14T09:06:21.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1519999999999992e-06 2024-09-14T09:06:21.537 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.537 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.537 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.537 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49388 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:21.537 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:21.537 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.945660070000001, 2024-09-14T09:06:21.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072001645000000003, 2024-09-14T09:06:21.537 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.537 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.537 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:21.538 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.538 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.538 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.538 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49388 2024-09-14T09:06:21.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.538 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:21.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:21.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:21.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:06:21.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546902+0000", 2024-09-14T09:06:21.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.734e-06 2024-09-14T09:06:21.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546910+0000", 2024-09-14T09:06:21.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9629999999999998e-06 2024-09-14T09:06:21.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546916+0000", 2024-09-14T09:06:21.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.28e-06 2024-09-14T09:06:21.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618889+0000", 2024-09-14T09:06:21.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071972555999999993 2024-09-14T09:06:21.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618900+0000", 2024-09-14T09:06:21.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1413999999999999e-05 2024-09-14T09:06:21.541 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.541 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.541 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.541 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49389 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:21.541 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546925+0000", 2024-09-14T09:06:21.541 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.94563391, 2024-09-14T09:06:21.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072026707999999995, 2024-09-14T09:06:21.541 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.541 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.541 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:21.541 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.542 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.542 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.542 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49389 2024-09-14T09:06:21.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.542 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546925+0000", 2024-09-14T09:06:21.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546925+0000", 2024-09-14T09:06:21.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546926+0000", 2024-09-14T09:06:21.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8599999999999998e-07 2024-09-14T09:06:21.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546928+0000", 2024-09-14T09:06:21.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4200000000000001e-06 2024-09-14T09:06:21.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546938+0000", 2024-09-14T09:06:21.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5859999999999994e-06 2024-09-14T09:06:21.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546945+0000", 2024-09-14T09:06:21.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1770000000000001e-06 2024-09-14T09:06:21.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618940+0000", 2024-09-14T09:06:21.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071995491999999994 2024-09-14T09:06:21.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618952+0000", 2024-09-14T09:06:21.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1347e-05 2024-09-14T09:06:21.545 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.545 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.545 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.545 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49390 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:21.545 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547051+0000", 2024-09-14T09:06:21.545 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.94550832, 2024-09-14T09:06:21.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071963947, 2024-09-14T09:06:21.545 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.545 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.545 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:21.545 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.545 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.546 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.546 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49390 2024-09-14T09:06:21.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.546 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547051+0000", 2024-09-14T09:06:21.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547051+0000", 2024-09-14T09:06:21.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547053+0000", 2024-09-14T09:06:21.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.007e-06 2024-09-14T09:06:21.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547064+0000", 2024-09-14T09:06:21.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1146e-05 2024-09-14T09:06:21.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547089+0000", 2024-09-14T09:06:21.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.563e-05 2024-09-14T09:06:21.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547127+0000", 2024-09-14T09:06:21.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8040000000000002e-05 2024-09-14T09:06:21.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619003+0000", 2024-09-14T09:06:21.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071875724000000002 2024-09-14T09:06:21.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619014+0000", 2024-09-14T09:06:21.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1399999999999999e-05 2024-09-14T09:06:21.549 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.549 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.549 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.549 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49391 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:21.549 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547162+0000", 2024-09-14T09:06:21.549 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.945397157, 2024-09-14T09:06:21.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071911484999999997, 2024-09-14T09:06:21.549 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.549 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.550 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:21.550 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.550 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.550 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.550 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49391 2024-09-14T09:06:21.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.550 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547162+0000", 2024-09-14T09:06:21.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547162+0000", 2024-09-14T09:06:21.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547163+0000", 2024-09-14T09:06:21.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.73e-06 2024-09-14T09:06:21.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547170+0000", 2024-09-14T09:06:21.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0400000000000004e-06 2024-09-14T09:06:21.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547190+0000", 2024-09-14T09:06:21.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9423e-05 2024-09-14T09:06:21.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547204+0000", 2024-09-14T09:06:21.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4175999999999999e-05 2024-09-14T09:06:21.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619060+0000", 2024-09-14T09:06:21.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071856347000000001 2024-09-14T09:06:21.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619073+0000", 2024-09-14T09:06:21.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2768999999999999e-05 2024-09-14T09:06:21.553 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.553 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.553 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.553 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49392 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:21.553 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547226+0000", 2024-09-14T09:06:21.553 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.945333178, 2024-09-14T09:06:21.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071890509000000005, 2024-09-14T09:06:21.553 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.553 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.553 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:21.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.554 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49392 2024-09-14T09:06:21.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.554 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547226+0000", 2024-09-14T09:06:21.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547226+0000", 2024-09-14T09:06:21.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547227+0000", 2024-09-14T09:06:21.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3969999999999999e-06 2024-09-14T09:06:21.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547233+0000", 2024-09-14T09:06:21.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4530000000000001e-06 2024-09-14T09:06:21.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547244+0000", 2024-09-14T09:06:21.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1327e-05 2024-09-14T09:06:21.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547252+0000", 2024-09-14T09:06:21.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0269999999999995e-06 2024-09-14T09:06:21.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619110+0000", 2024-09-14T09:06:21.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071857997000000007 2024-09-14T09:06:21.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619116+0000", 2024-09-14T09:06:21.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3080000000000004e-06 2024-09-14T09:06:21.557 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.557 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.557 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.557 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49393 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:21.557 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547263+0000", 2024-09-14T09:06:21.557 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.945295535, 2024-09-14T09:06:21.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071900037999999999, 2024-09-14T09:06:21.557 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.557 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.557 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:21.557 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.558 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.558 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.558 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49393 2024-09-14T09:06:21.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.558 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547263+0000", 2024-09-14T09:06:21.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547263+0000", 2024-09-14T09:06:21.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547264+0000", 2024-09-14T09:06:21.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3399999999999998e-07 2024-09-14T09:06:21.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547299+0000", 2024-09-14T09:06:21.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5252000000000001e-05 2024-09-14T09:06:21.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547317+0000", 2024-09-14T09:06:21.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.736e-05 2024-09-14T09:06:21.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547332+0000", 2024-09-14T09:06:21.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.487e-05 2024-09-14T09:06:21.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619152+0000", 2024-09-14T09:06:21.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071820590000000004 2024-09-14T09:06:21.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619163+0000", 2024-09-14T09:06:21.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1131999999999999e-05 2024-09-14T09:06:21.561 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.561 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.561 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.561 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:50478 3.1d 3:b8a2eaf4:::rbd_data.115e5dc498ac.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T09:06:21.561 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:21.561 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4429190500000004, 2024-09-14T09:06:21.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.075023792000000006, 2024-09-14T09:06:21.561 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.561 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.561 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:21.562 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.562 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.562 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.562 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 50478 2024-09-14T09:06:21.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.562 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:21.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:21.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049641+0000", 2024-09-14T09:06:21.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6270000000000001e-06 2024-09-14T09:06:21.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049649+0000", 2024-09-14T09:06:21.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6279999999999996e-06 2024-09-14T09:06:21.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049667+0000", 2024-09-14T09:06:21.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-09-14T09:06:21.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049677+0000", 2024-09-14T09:06:21.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0509000000000001e-05 2024-09-14T09:06:21.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049751+0000", 2024-09-14T09:06:21.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3322e-05 2024-09-14T09:06:21.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:21.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049830+0000", 2024-09-14T09:06:21.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9932000000000007e-05 2024-09-14T09:06:21.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:21.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.053203+0000", 2024-09-14T09:06:21.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003372398 2024-09-14T09:06:21.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:21.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124542+0000", 2024-09-14T09:06:21.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071339105999999999 2024-09-14T09:06:21.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:21.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124562+0000", 2024-09-14T09:06:21.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9649000000000001e-05 2024-09-14T09:06:21.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:21.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124616+0000", 2024-09-14T09:06:21.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4642999999999998e-05 2024-09-14T09:06:21.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124664+0000", 2024-09-14T09:06:21.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7342000000000002e-05 2024-09-14T09:06:21.567 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.567 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.567 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.567 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65035 3.1d 3:bb585d47:::rbd_data.115e21c31fcf.0000000000000034:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:06:21.567 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.068842+0000", 2024-09-14T09:06:21.567 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.423716816, 2024-09-14T09:06:21.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.094968451999999995, 2024-09-14T09:06:21.567 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.567 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.567 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:21.567 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:21.568 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:21.568 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:21.568 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65035 2024-09-14T09:06:21.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.568 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068842+0000", 2024-09-14T09:06:21.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068842+0000", 2024-09-14T09:06:21.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068843+0000", 2024-09-14T09:06:21.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.221e-06 2024-09-14T09:06:21.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068849+0000", 2024-09-14T09:06:21.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4659999999999998e-06 2024-09-14T09:06:21.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068861+0000", 2024-09-14T09:06:21.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2204e-05 2024-09-14T09:06:21.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068871+0000", 2024-09-14T09:06:21.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0318e-05 2024-09-14T09:06:21.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068942+0000", 2024-09-14T09:06:21.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0438999999999994e-05 2024-09-14T09:06:21.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:21.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.069005+0000", 2024-09-14T09:06:21.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3076000000000004e-05 2024-09-14T09:06:21.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:21.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.074957+0000", 2024-09-14T09:06:21.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059524549999999997 2024-09-14T09:06:21.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:21.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163479+0000", 2024-09-14T09:06:21.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.088521676999999993 2024-09-14T09:06:21.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:21.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163633+0000", 2024-09-14T09:06:21.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000154301 2024-09-14T09:06:21.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:21.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163733+0000", 2024-09-14T09:06:21.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100089 2024-09-14T09:06:21.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163811+0000", 2024-09-14T09:06:21.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7206000000000006e-05 2024-09-14T09:06:21.573 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.573 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.573 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.573 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:65035 3.1d e65/23)", 2024-09-14T09:06:21.573 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.087665+0000", 2024-09-14T09:06:21.573 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.40489390600000003, 2024-09-14T09:06:21.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.076160555000000005, 2024-09-14T09:06:21.574 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:21.574 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:21.574 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:21.574 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:21.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:21.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087665+0000", 2024-09-14T09:06:21.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:21.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087665+0000", 2024-09-14T09:06:21.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:21.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:21.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087667+0000", 2024-09-14T09:06:21.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.818e-06 2024-09-14T09:06:21.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:21.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087673+0000", 2024-09-14T09:06:21.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9259999999999997e-06 2024-09-14T09:06:21.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:21.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087692+0000", 2024-09-14T09:06:21.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9715e-05 2024-09-14T09:06:21.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:21.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087703+0000", 2024-09-14T09:06:21.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.094e-05 2024-09-14T09:06:21.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:21.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163595+0000", 2024-09-14T09:06:21.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.075892121000000007 2024-09-14T09:06:21.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:21.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163621+0000", 2024-09-14T09:06:21.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5383000000000001e-05 2024-09-14T09:06:21.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:21.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:21.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:21.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163826+0000", 2024-09-14T09:06:21.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020465199999999999 2024-09-14T09:06:21.579 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.579 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.579 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.580 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:21.580 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:21.580 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:21.583 INFO:teuthology.orchestra.run.smithi176.stdout:2553 write 0x9b44f28 thru 0x9b4fbee (0xacc7 bytes) 2024-09-14T09:06:21.583 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:21.583 INFO:teuthology.orchestra.run.smithi176.stdout:2555 read 0x34c344f thru 0x34c53fb (0x1fad bytes) 2024-09-14T09:06:21.583 INFO:teuthology.orchestra.run.smithi176.stdout:2557 write 0xdc928dd thru 0xdc9c137 (0x985b bytes) 2024-09-14T09:06:21.583 INFO:teuthology.orchestra.run.smithi176.stdout:2558 writesame 0xc79387 thru 0xc7bb9f data_size 0x1(0x2819 bytes) 2024-09-14T09:06:21.584 INFO:teuthology.orchestra.run.smithi176.stdout:2560 read 0xd9222d5 thru 0xd92bab5 (0x97e1 bytes) 2024-09-14T09:06:21.584 INFO:teuthology.orchestra.run.smithi176.stdout:2561 punch from 0x526a131 to 0x52757a2, (0xb671 bytes) 2024-09-14T09:06:21.584 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:06:21.584 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:21.583+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:21.600 INFO:teuthology.orchestra.run.smithi176.stdout:2562 write 0xe5cca7b thru 0xe5dc904 (0xfe8a bytes) 2024-09-14T09:06:21.609 INFO:teuthology.orchestra.run.smithi176.stdout:2563 punch from 0x7ff4a56 to 0x8004711, (0xfcbb bytes) 2024-09-14T09:06:21.609 INFO:teuthology.orchestra.run.smithi176.stdout:2564 writesame 0x711fdcf thru 0x7121f71 data_size 0x1(0x21a3 bytes) 2024-09-14T09:06:21.626 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:21.640 INFO:teuthology.orchestra.run.smithi176.stdout:2566 trunc from 0xe5dc905 to 0x986d2dd 2024-09-14T09:06:21.669 INFO:teuthology.orchestra.run.smithi176.stdout:2569 trunc from 0x986d2dd to 0xaf699bd 2024-09-14T09:06:21.682 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:21.683+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:21.695 INFO:teuthology.orchestra.run.smithi176.stdout:2570 writesame 0xb5f8210 thru 0xb6040a0 data_size 0xb(0xbe91 bytes) 2024-09-14T09:06:21.695 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:21.695 INFO:teuthology.orchestra.run.smithi176.stdout:2573 read 0xa68769c thru 0xa687e69 (0x7ce bytes) 2024-09-14T09:06:21.695 INFO:teuthology.orchestra.run.smithi176.stdout:2575 read 0x8fb8063 thru 0x8fbff4e (0x7eec bytes) 2024-09-14T09:06:21.695 INFO:teuthology.orchestra.run.smithi176.stdout:2576 punch from 0x426b23d to 0x4274321, (0x90e4 bytes) 2024-09-14T09:06:21.695 INFO:teuthology.orchestra.run.smithi176.stdout:2577 read 0x5aec409 thru 0x5af16eb (0x52e3 bytes) 2024-09-14T09:06:21.696 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:21.696 INFO:teuthology.orchestra.run.smithi176.stdout:2580 writesame 0x7d53b3b thru 0x7d5e5cb data_size 0x29(0xaa91 bytes) 2024-09-14T09:06:21.702 INFO:teuthology.orchestra.run.smithi176.stdout:2582 read 0x4aa715e thru 0x4ab1806 (0xa6a9 bytes) 2024-09-14T09:06:21.703 INFO:teuthology.orchestra.run.smithi176.stdout:2584 punch from 0x6bceba to 0x6c7b53, (0xac99 bytes) 2024-09-14T09:06:21.713 INFO:teuthology.orchestra.run.smithi176.stdout:2589 trunc from 0xb6040a1 to 0x7b2a74e 2024-09-14T09:06:21.765 INFO:teuthology.orchestra.run.smithi176.stdout:2593 write 0xd46fc85 thru 0xd47b36b (0xb6e7 bytes) 2024-09-14T09:06:21.767 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:21.767 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:21.767 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:21.767 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:21.772 INFO:teuthology.orchestra.run.smithi176.stdout:2594 punch from 0x2a69c71 to 0x2a73744, (0x9ad3 bytes) 2024-09-14T09:06:21.772 INFO:teuthology.orchestra.run.smithi176.stdout:2596 writesame 0x8ec21d7 thru 0x8ed2076 data_size 0x38(0xfea0 bytes) 2024-09-14T09:06:21.780 INFO:teuthology.orchestra.run.smithi176.stdout:2599 writesame 0x173d531 thru 0x1741212 data_size 0x1(0x3ce2 bytes) 2024-09-14T09:06:21.782 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:21.783+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:21.784 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:06:21.831 INFO:teuthology.orchestra.run.smithi176.stdout:2600 read 0x8ca82ee thru 0x8cb26aa (0xa3bd bytes) 2024-09-14T09:06:21.832 INFO:teuthology.orchestra.run.smithi176.stdout:2602 read 0xd3df7af thru 0xd3e47a6 (0x4ff8 bytes) 2024-09-14T09:06:21.832 INFO:teuthology.orchestra.run.smithi176.stdout:2603 punch from 0x5fc9c32 to 0x5fd9716, (0xfae4 bytes) 2024-09-14T09:06:21.832 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:21.832 INFO:teuthology.orchestra.run.smithi176.stdout:2606 read 0x66db69b thru 0x66e755a (0xbec0 bytes) 2024-09-14T09:06:21.832 INFO:teuthology.orchestra.run.smithi176.stdout:2608 punch from 0x607a42f to 0x608530a, (0xaedb bytes) 2024-09-14T09:06:21.832 INFO:teuthology.orchestra.run.smithi176.stdout:2609 punch from 0xb89f8ad to 0xb8a629c, (0x69ef bytes) 2024-09-14T09:06:21.852 INFO:teuthology.orchestra.run.smithi176.stdout:2611 trunc from 0xd47b36c to 0x59b1d29 2024-09-14T09:06:21.875 INFO:teuthology.orchestra.run.smithi176.stdout:2612 writesame 0xbcbea6 thru 0xbd6fe9 data_size 0xa(0xb144 bytes) 2024-09-14T09:06:21.883 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:21.883+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:21.902 INFO:teuthology.orchestra.run.smithi176.stdout:2613 punch from 0x4c7274d to 0x4c7284e, (0x101 bytes) 2024-09-14T09:06:21.902 INFO:teuthology.orchestra.run.smithi176.stdout:2614 read 0x3e57025 thru 0x3e637f6 (0xc7d2 bytes) 2024-09-14T09:06:21.929 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:06:21.941 INFO:teuthology.orchestra.run.smithi176.stdout:2615 write 0xee69228 thru 0xee6972c (0x505 bytes) 2024-09-14T09:06:21.959 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:21.959 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:21.959 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:21.959 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:21.959 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:21.971 INFO:teuthology.orchestra.run.smithi176.stdout:2617 trunc from 0xee6972d to 0x133f851 2024-09-14T09:06:21.973 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:06:21.985 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:21.983+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:21.988 INFO:teuthology.orchestra.run.smithi176.stdout:2618 read 0x11ee09e thru 0x11f18c8 (0x382b bytes) 2024-09-14T09:06:22.016 INFO:teuthology.orchestra.run.smithi176.stdout:2619 write 0xd833f58 thru 0xd8349f8 (0xaa1 bytes) 2024-09-14T09:06:22.021 INFO:teuthology.orchestra.run.smithi176.stdout:2620 writesame 0x6403219 thru 0x6408195 data_size 0x77(0x4f7d bytes) 2024-09-14T09:06:22.026 INFO:teuthology.orchestra.run.smithi176.stdout:2621 punch from 0x7318e6b to 0x732095d, (0x7af2 bytes) 2024-09-14T09:06:22.040 INFO:teuthology.orchestra.run.smithi176.stdout:2624 trunc from 0xd8349f9 to 0x55353cf 2024-09-14T09:06:22.058 INFO:teuthology.orchestra.run.smithi176.stdout:2625 write 0x363d631 thru 0x3640539 (0x2f09 bytes) 2024-09-14T09:06:22.085 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:22.083+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:22.089 INFO:teuthology.orchestra.run.smithi176.stdout:2626 writesame 0xd4ee733 thru 0xd4fc38e data_size 0x12(0xdc5c bytes) 2024-09-14T09:06:22.090 INFO:teuthology.orchestra.run.smithi176.stdout:2627 read 0xc45afbe thru 0xc462021 (0x7064 bytes) 2024-09-14T09:06:22.090 INFO:teuthology.orchestra.run.smithi176.stdout:2628 write 0xb7d20ca thru 0xb7d27b0 (0x6e7 bytes) 2024-09-14T09:06:22.093 INFO:teuthology.orchestra.run.smithi176.stdout:2629 write 0xb4a25c0 thru 0xb4aa333 (0x7d74 bytes) 2024-09-14T09:06:22.099 INFO:teuthology.orchestra.run.smithi176.stdout:2630 write 0x60fdb99 thru 0x61034e1 (0x5949 bytes) 2024-09-14T09:06:22.101 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:22.102 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:22.106 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:22.106 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:22.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19520 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475212+0000", 2024-09-14T09:06:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.633877400999999, 2024-09-14T09:06:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067435885000000001, 2024-09-14T09:06:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19520 2024-09-14T09:06:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475212+0000", 2024-09-14T09:06:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475212+0000", 2024-09-14T09:06:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475213+0000", 2024-09-14T09:06:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0440000000000001e-06 2024-09-14T09:06:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475218+0000", 2024-09-14T09:06:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1930000000000001e-06 2024-09-14T09:06:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475237+0000", 2024-09-14T09:06:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9003000000000001e-05 2024-09-14T09:06:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475251+0000", 2024-09-14T09:06:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3720000000000001e-05 2024-09-14T09:06:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.541342+0000", 2024-09-14T09:06:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066091227000000002 2024-09-14T09:06:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.541492+0000", 2024-09-14T09:06:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000149221 2024-09-14T09:06:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542648+0000", 2024-09-14T09:06:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011564769999999999 2024-09-14T09:06:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19530 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475326+0000", 2024-09-14T09:06:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.633763217999999, 2024-09-14T09:06:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067030846000000005, 2024-09-14T09:06:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19530 2024-09-14T09:06:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475326+0000", 2024-09-14T09:06:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475326+0000", 2024-09-14T09:06:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475328+0000", 2024-09-14T09:06:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.133e-06 2024-09-14T09:06:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475358+0000", 2024-09-14T09:06:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0253e-05 2024-09-14T09:06:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475378+0000", 2024-09-14T09:06:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.033e-05 2024-09-14T09:06:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475405+0000", 2024-09-14T09:06:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6568999999999999e-05 2024-09-14T09:06:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.541989+0000", 2024-09-14T09:06:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066583956999999999 2024-09-14T09:06:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542215+0000", 2024-09-14T09:06:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022592200000000001 2024-09-14T09:06:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542357+0000", 2024-09-14T09:06:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014268199999999999 2024-09-14T09:06:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19531 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15073280~11919] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475425+0000", 2024-09-14T09:06:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.633664922999998, 2024-09-14T09:06:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067651759000000006, 2024-09-14T09:06:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19531 2024-09-14T09:06:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475425+0000", 2024-09-14T09:06:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475425+0000", 2024-09-14T09:06:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475426+0000", 2024-09-14T09:06:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2440000000000001e-06 2024-09-14T09:06:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475465+0000", 2024-09-14T09:06:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8643000000000003e-05 2024-09-14T09:06:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475485+0000", 2024-09-14T09:06:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0645999999999999e-05 2024-09-14T09:06:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475499+0000", 2024-09-14T09:06:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3312999999999999e-05 2024-09-14T09:06:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542389+0000", 2024-09-14T09:06:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066890885999999997 2024-09-14T09:06:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542481+0000", 2024-09-14T09:06:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1481e-05 2024-09-14T09:06:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543076+0000", 2024-09-14T09:06:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00059554599999999997 2024-09-14T09:06:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19532 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15335424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475521+0000", 2024-09-14T09:06:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.633568277000002, 2024-09-14T09:06:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067895159999999996, 2024-09-14T09:06:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19532 2024-09-14T09:06:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475521+0000", 2024-09-14T09:06:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475521+0000", 2024-09-14T09:06:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475523+0000", 2024-09-14T09:06:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.277e-06 2024-09-14T09:06:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475528+0000", 2024-09-14T09:06:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8629999999999996e-06 2024-09-14T09:06:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475546+0000", 2024-09-14T09:06:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7750000000000001e-05 2024-09-14T09:06:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475560+0000", 2024-09-14T09:06:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3763e-05 2024-09-14T09:06:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543144+0000", 2024-09-14T09:06:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067584489999999997 2024-09-14T09:06:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543261+0000", 2024-09-14T09:06:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011696699999999999 2024-09-14T09:06:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543416+0000", 2024-09-14T09:06:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015505 2024-09-14T09:06:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19533 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475578+0000", 2024-09-14T09:06:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.633511134000003, 2024-09-14T09:06:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068124984999999999, 2024-09-14T09:06:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19533 2024-09-14T09:06:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475578+0000", 2024-09-14T09:06:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475578+0000", 2024-09-14T09:06:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475580+0000", 2024-09-14T09:06:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.08e-06 2024-09-14T09:06:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475585+0000", 2024-09-14T09:06:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4970000000000001e-06 2024-09-14T09:06:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475602+0000", 2024-09-14T09:06:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7459999999999999e-05 2024-09-14T09:06:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475615+0000", 2024-09-14T09:06:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2339e-05 2024-09-14T09:06:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543466+0000", 2024-09-14T09:06:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067850745000000004 2024-09-14T09:06:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543577+0000", 2024-09-14T09:06:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000111908 2024-09-14T09:06:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543703+0000", 2024-09-14T09:06:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012595600000000001 2024-09-14T09:06:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19534 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475634+0000", 2024-09-14T09:06:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.633455357999999, 2024-09-14T09:06:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068358706000000005, 2024-09-14T09:06:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19534 2024-09-14T09:06:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475634+0000", 2024-09-14T09:06:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475634+0000", 2024-09-14T09:06:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475635+0000", 2024-09-14T09:06:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.187e-06 2024-09-14T09:06:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475641+0000", 2024-09-14T09:06:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5629999999999999e-06 2024-09-14T09:06:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475659+0000", 2024-09-14T09:06:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7649999999999999e-05 2024-09-14T09:06:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475671+0000", 2024-09-14T09:06:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2583e-05 2024-09-14T09:06:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543748+0000", 2024-09-14T09:06:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068076740999999996 2024-09-14T09:06:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543855+0000", 2024-09-14T09:06:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010676800000000001 2024-09-14T09:06:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543993+0000", 2024-09-14T09:06:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138214 2024-09-14T09:06:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19535 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475691+0000", 2024-09-14T09:06:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.633398348, 2024-09-14T09:06:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068607583999999999, 2024-09-14T09:06:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19535 2024-09-14T09:06:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475691+0000", 2024-09-14T09:06:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475691+0000", 2024-09-14T09:06:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475693+0000", 2024-09-14T09:06:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6270000000000001e-06 2024-09-14T09:06:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475699+0000", 2024-09-14T09:06:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9059999999999999e-06 2024-09-14T09:06:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475717+0000", 2024-09-14T09:06:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-09-14T09:06:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475730+0000", 2024-09-14T09:06:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.308e-05 2024-09-14T09:06:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544042+0000", 2024-09-14T09:06:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068312791999999997 2024-09-14T09:06:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544168+0000", 2024-09-14T09:06:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012549699999999999 2024-09-14T09:06:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544299+0000", 2024-09-14T09:06:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013094199999999999 2024-09-14T09:06:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.139 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.139 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19536 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:22.139 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475749+0000", 2024-09-14T09:06:22.139 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.633341068999997, 2024-09-14T09:06:22.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068917143, 2024-09-14T09:06:22.139 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.139 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.139 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:22.139 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:22.139 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:22.139 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:22.139 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19536 2024-09-14T09:06:22.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.139 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475749+0000", 2024-09-14T09:06:22.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475749+0000", 2024-09-14T09:06:22.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475750+0000", 2024-09-14T09:06:22.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0330000000000001e-06 2024-09-14T09:06:22.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475755+0000", 2024-09-14T09:06:22.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9740000000000001e-06 2024-09-14T09:06:22.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475772+0000", 2024-09-14T09:06:22.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7750000000000001e-05 2024-09-14T09:06:22.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475802+0000", 2024-09-14T09:06:22.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0190000000000001e-05 2024-09-14T09:06:22.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544345+0000", 2024-09-14T09:06:22.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068542140000000001 2024-09-14T09:06:22.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544454+0000", 2024-09-14T09:06:22.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109447 2024-09-14T09:06:22.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544666+0000", 2024-09-14T09:06:22.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000211609 2024-09-14T09:06:22.143 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.143 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.143 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.143 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19537 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:22.143 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475822+0000", 2024-09-14T09:06:22.143 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.633267111999999, 2024-09-14T09:06:22.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069185398999999995, 2024-09-14T09:06:22.143 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.143 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.143 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:22.143 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:22.143 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:22.143 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:22.144 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19537 2024-09-14T09:06:22.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.144 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475822+0000", 2024-09-14T09:06:22.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475822+0000", 2024-09-14T09:06:22.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475824+0000", 2024-09-14T09:06:22.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2559999999999999e-06 2024-09-14T09:06:22.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475829+0000", 2024-09-14T09:06:22.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6840000000000001e-06 2024-09-14T09:06:22.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475848+0000", 2024-09-14T09:06:22.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8583000000000001e-05 2024-09-14T09:06:22.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475861+0000", 2024-09-14T09:06:22.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2802999999999999e-05 2024-09-14T09:06:22.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544716+0000", 2024-09-14T09:06:22.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068855036999999994 2024-09-14T09:06:22.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544827+0000", 2024-09-14T09:06:22.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011113200000000001 2024-09-14T09:06:22.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545008+0000", 2024-09-14T09:06:22.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018090399999999999 2024-09-14T09:06:22.147 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.147 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.147 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.147 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19538 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:22.147 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475880+0000", 2024-09-14T09:06:22.147 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.633209436999998, 2024-09-14T09:06:22.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069740072, 2024-09-14T09:06:22.147 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.147 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.147 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:22.147 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:22.147 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:22.148 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:22.148 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19538 2024-09-14T09:06:22.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.148 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475880+0000", 2024-09-14T09:06:22.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475880+0000", 2024-09-14T09:06:22.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475881+0000", 2024-09-14T09:06:22.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.124e-06 2024-09-14T09:06:22.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475886+0000", 2024-09-14T09:06:22.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0130000000000001e-06 2024-09-14T09:06:22.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475904+0000", 2024-09-14T09:06:22.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7779999999999999e-05 2024-09-14T09:06:22.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475918+0000", 2024-09-14T09:06:22.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.385e-05 2024-09-14T09:06:22.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545065+0000", 2024-09-14T09:06:22.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069147307000000005 2024-09-14T09:06:22.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545431+0000", 2024-09-14T09:06:22.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036538400000000001 2024-09-14T09:06:22.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545620+0000", 2024-09-14T09:06:22.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000189614 2024-09-14T09:06:22.151 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.151 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.151 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.151 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19539 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:22.151 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475936+0000", 2024-09-14T09:06:22.151 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.633153853000003, 2024-09-14T09:06:22.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069860063999999999, 2024-09-14T09:06:22.152 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.152 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.152 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:22.152 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:22.152 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:22.152 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:22.152 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19539 2024-09-14T09:06:22.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.152 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475936+0000", 2024-09-14T09:06:22.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475936+0000", 2024-09-14T09:06:22.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475937+0000", 2024-09-14T09:06:22.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5960000000000001e-06 2024-09-14T09:06:22.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475943+0000", 2024-09-14T09:06:22.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2340000000000002e-06 2024-09-14T09:06:22.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475960+0000", 2024-09-14T09:06:22.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7592999999999999e-05 2024-09-14T09:06:22.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475986+0000", 2024-09-14T09:06:22.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6319e-05 2024-09-14T09:06:22.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545650+0000", 2024-09-14T09:06:22.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069663613999999999 2024-09-14T09:06:22.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545714+0000", 2024-09-14T09:06:22.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4209999999999997e-05 2024-09-14T09:06:22.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545796+0000", 2024-09-14T09:06:22.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1497999999999998e-05 2024-09-14T09:06:22.155 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.155 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.155 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.155 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19541 3.10 3:0ae7d8d9:::rbd_data.115e4c71d6e8.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 4194304~8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:22.155 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.497517+0000", 2024-09-14T09:06:22.156 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.611572782000003, 2024-09-14T09:06:22.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069874055000000004, 2024-09-14T09:06:22.156 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.156 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.156 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:22.156 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:22.156 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:22.156 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:22.156 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19541 2024-09-14T09:06:22.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.156 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.497517+0000", 2024-09-14T09:06:22.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.497517+0000", 2024-09-14T09:06:22.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.497520+0000", 2024-09-14T09:06:22.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0730000000000001e-06 2024-09-14T09:06:22.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.504875+0000", 2024-09-14T09:06:22.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007355136 2024-09-14T09:06:22.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.504926+0000", 2024-09-14T09:06:22.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1146000000000001e-05 2024-09-14T09:06:22.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.504950+0000", 2024-09-14T09:06:22.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3638999999999998e-05 2024-09-14T09:06:22.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.505139+0000", 2024-09-14T09:06:22.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018904300000000001 2024-09-14T09:06:22.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.505402+0000", 2024-09-14T09:06:22.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026354699999999999 2024-09-14T09:06:22.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:22.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.506208+0000", 2024-09-14T09:06:22.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00080583399999999996 2024-09-14T09:06:22.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:22.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.566904+0000", 2024-09-14T09:06:22.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.060695510000000001 2024-09-14T09:06:22.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:22.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.567219+0000", 2024-09-14T09:06:22.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031509000000000002 2024-09-14T09:06:22.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:22.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.567319+0000", 2024-09-14T09:06:22.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9804999999999997e-05 2024-09-14T09:06:22.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.567391+0000", 2024-09-14T09:06:22.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2231999999999996e-05 2024-09-14T09:06:22.161 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.161 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.161 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.161 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:30890 3.10 3:0ac3ecfb:::rbd_data.115ebc3c67d1.0000000000000004:c [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:22.161 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.069693+0000", 2024-09-14T09:06:22.161 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.039396993, 2024-09-14T09:06:22.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066892069999999998, 2024-09-14T09:06:22.162 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.162 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.162 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:22.162 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:22.162 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:22.162 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:22.162 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 30890 2024-09-14T09:06:22.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.162 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069693+0000", 2024-09-14T09:06:22.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069693+0000", 2024-09-14T09:06:22.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069694+0000", 2024-09-14T09:06:22.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0100000000000003e-07 2024-09-14T09:06:22.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069698+0000", 2024-09-14T09:06:22.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2100000000000003e-06 2024-09-14T09:06:22.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069711+0000", 2024-09-14T09:06:22.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3332e-05 2024-09-14T09:06:22.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069721+0000", 2024-09-14T09:06:22.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0424999999999999e-05 2024-09-14T09:06:22.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136469+0000", 2024-09-14T09:06:22.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066747714999999999 2024-09-14T09:06:22.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136540+0000", 2024-09-14T09:06:22.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1077999999999999e-05 2024-09-14T09:06:22.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136585+0000", 2024-09-14T09:06:22.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4409000000000001e-05 2024-09-14T09:06:22.165 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.166 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.166 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.166 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:30891 3.10 3:0ac3ecfb:::rbd_data.115ebc3c67d1.0000000000000004:c [read 212992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:22.166 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.069740+0000", 2024-09-14T09:06:22.166 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.039349992999998, 2024-09-14T09:06:22.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066940186999999998, 2024-09-14T09:06:22.166 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.166 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.166 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:22.167 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:22.167 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:22.167 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:22.167 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 30891 2024-09-14T09:06:22.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.167 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069740+0000", 2024-09-14T09:06:22.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069740+0000", 2024-09-14T09:06:22.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070603+0000", 2024-09-14T09:06:22.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00086369599999999995 2024-09-14T09:06:22.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070612+0000", 2024-09-14T09:06:22.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3170000000000002e-06 2024-09-14T09:06:22.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070631+0000", 2024-09-14T09:06:22.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8936999999999998e-05 2024-09-14T09:06:22.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070645+0000", 2024-09-14T09:06:22.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4545999999999999e-05 2024-09-14T09:06:22.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136603+0000", 2024-09-14T09:06:22.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.065957551000000003 2024-09-14T09:06:22.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136644+0000", 2024-09-14T09:06:22.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1671000000000001e-05 2024-09-14T09:06:22.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136680+0000", 2024-09-14T09:06:22.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5469000000000002e-05 2024-09-14T09:06:22.170 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.170 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.170 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.170 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:37542 3.10 3:0e658405:::rbd_data.115eb07a8abd.000000000000000d:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:22.170 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.443185+0000", 2024-09-14T09:06:22.170 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 25.665904966999999, 2024-09-14T09:06:22.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.091143627000000005, 2024-09-14T09:06:22.171 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.171 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.171 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:22.171 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:22.171 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:22.171 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:22.171 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37542 2024-09-14T09:06:22.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.171 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443185+0000", 2024-09-14T09:06:22.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443185+0000", 2024-09-14T09:06:22.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443187+0000", 2024-09-14T09:06:22.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.367e-06 2024-09-14T09:06:22.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443199+0000", 2024-09-14T09:06:22.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2153e-05 2024-09-14T09:06:22.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443225+0000", 2024-09-14T09:06:22.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5457e-05 2024-09-14T09:06:22.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443248+0000", 2024-09-14T09:06:22.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2963e-05 2024-09-14T09:06:22.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443400+0000", 2024-09-14T09:06:22.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015241199999999999 2024-09-14T09:06:22.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443542+0000", 2024-09-14T09:06:22.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014224600000000001 2024-09-14T09:06:22.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:22.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.448965+0000", 2024-09-14T09:06:22.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054226839999999997 2024-09-14T09:06:22.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:22.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534136+0000", 2024-09-14T09:06:22.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.085170939000000001 2024-09-14T09:06:22.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:22.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534164+0000", 2024-09-14T09:06:22.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7863000000000001e-05 2024-09-14T09:06:22.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:22.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534250+0000", 2024-09-14T09:06:22.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5876000000000002e-05 2024-09-14T09:06:22.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534328+0000", 2024-09-14T09:06:22.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8666999999999997e-05 2024-09-14T09:06:22.176 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.176 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.176 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.176 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:37542 3.10 e48/28 ondisk, result = 0)", 2024-09-14T09:06:22.176 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.464008+0000", 2024-09-14T09:06:22.176 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 25.645081558000001, 2024-09-14T09:06:22.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.070141355000000002, 2024-09-14T09:06:22.176 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.176 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.176 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:22.177 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464008+0000", 2024-09-14T09:06:22.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464008+0000", 2024-09-14T09:06:22.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464010+0000", 2024-09-14T09:06:22.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.547e-06 2024-09-14T09:06:22.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464017+0000", 2024-09-14T09:06:22.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4930000000000001e-06 2024-09-14T09:06:22.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464047+0000", 2024-09-14T09:06:22.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0090000000000002e-05 2024-09-14T09:06:22.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464061+0000", 2024-09-14T09:06:22.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4003e-05 2024-09-14T09:06:22.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534107+0000", 2024-09-14T09:06:22.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.070045590000000005 2024-09-14T09:06:22.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534126+0000", 2024-09-14T09:06:22.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8893e-05 2024-09-14T09:06:22.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534149+0000", 2024-09-14T09:06:22.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3739000000000001e-05 2024-09-14T09:06:22.180 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.180 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.180 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.180 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38003 3.10 3:0e658405:::rbd_data.115eb07a8abd.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:22.180 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:57.392070+0000", 2024-09-14T09:06:22.180 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.717019784000001, 2024-09-14T09:06:22.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11697451, 2024-09-14T09:06:22.180 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.180 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.180 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:22.180 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:22.181 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:22.181 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:22.181 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38003 2024-09-14T09:06:22.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.181 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392070+0000", 2024-09-14T09:06:22.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392070+0000", 2024-09-14T09:06:22.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392072+0000", 2024-09-14T09:06:22.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6529999999999998e-06 2024-09-14T09:06:22.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392086+0000", 2024-09-14T09:06:22.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.332e-05 2024-09-14T09:06:22.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392139+0000", 2024-09-14T09:06:22.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2815999999999997e-05 2024-09-14T09:06:22.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392155+0000", 2024-09-14T09:06:22.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6796999999999999e-05 2024-09-14T09:06:22.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392296+0000", 2024-09-14T09:06:22.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014107900000000001 2024-09-14T09:06:22.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392443+0000", 2024-09-14T09:06:22.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014654499999999999 2024-09-14T09:06:22.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:22.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.399418+0000", 2024-09-14T09:06:22.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069751049999999997 2024-09-14T09:06:22.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:22.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508607+0000", 2024-09-14T09:06:22.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10918908400000001 2024-09-14T09:06:22.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:22.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508748+0000", 2024-09-14T09:06:22.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014079699999999999 2024-09-14T09:06:22.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:22.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508921+0000", 2024-09-14T09:06:22.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017345899999999999 2024-09-14T09:06:22.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.509044+0000", 2024-09-14T09:06:22.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000122855 2024-09-14T09:06:22.185 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.185 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.186 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.186 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:38003 3.10 e48/28 ondisk, result = 0)", 2024-09-14T09:06:22.186 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:57.413977+0000", 2024-09-14T09:06:22.186 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.695112560999998, 2024-09-14T09:06:22.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.095088000000000006, 2024-09-14T09:06:22.186 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.186 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.186 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:22.186 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413977+0000", 2024-09-14T09:06:22.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413977+0000", 2024-09-14T09:06:22.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413979+0000", 2024-09-14T09:06:22.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.933e-06 2024-09-14T09:06:22.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413988+0000", 2024-09-14T09:06:22.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2769999999999994e-06 2024-09-14T09:06:22.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.414021+0000", 2024-09-14T09:06:22.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2589999999999998e-05 2024-09-14T09:06:22.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.414040+0000", 2024-09-14T09:06:22.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8796000000000001e-05 2024-09-14T09:06:22.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508701+0000", 2024-09-14T09:06:22.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.094661067000000002 2024-09-14T09:06:22.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508729+0000", 2024-09-14T09:06:22.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8677e-05 2024-09-14T09:06:22.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.509065+0000", 2024-09-14T09:06:22.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033565999999999999 2024-09-14T09:06:22.189 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.190 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.190 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.190 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65139 3.a 3:54c90b7d:::rbd_data.115e21c31fcf.0000000000000008:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:06:22.190 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:06:22.190 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.72937685699999999, 2024-09-14T09:06:22.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14429895700000001, 2024-09-14T09:06:22.190 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.190 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.190 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:22.190 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:22.190 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:22.191 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:22.191 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65139 2024-09-14T09:06:22.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.191 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:06:22.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:06:22.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379715+0000", 2024-09-14T09:06:22.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.576e-06 2024-09-14T09:06:22.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379724+0000", 2024-09-14T09:06:22.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1779999999999992e-06 2024-09-14T09:06:22.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379746+0000", 2024-09-14T09:06:22.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1661000000000001e-05 2024-09-14T09:06:22.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379759+0000", 2024-09-14T09:06:22.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.324e-05 2024-09-14T09:06:22.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379882+0000", 2024-09-14T09:06:22.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012239400000000001 2024-09-14T09:06:22.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.380023+0000", 2024-09-14T09:06:22.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000141402 2024-09-14T09:06:22.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:22.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.391553+0000", 2024-09-14T09:06:22.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011530197000000001 2024-09-14T09:06:22.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:22.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523927+0000", 2024-09-14T09:06:22.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.132373935 2024-09-14T09:06:22.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:22.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523941+0000", 2024-09-14T09:06:22.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3326000000000001e-05 2024-09-14T09:06:22.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:22.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523977+0000", 2024-09-14T09:06:22.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6185e-05 2024-09-14T09:06:22.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.524012+0000", 2024-09-14T09:06:22.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4863000000000001e-05 2024-09-14T09:06:22.196 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.196 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.196 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.197 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:65139 3.a e65/34 ondisk, result = 0)", 2024-09-14T09:06:22.197 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.408501+0000", 2024-09-14T09:06:22.197 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.70058830000000005, 2024-09-14T09:06:22.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.115431275, 2024-09-14T09:06:22.197 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:22.197 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:22.197 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:22.197 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:22.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:22.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408501+0000", 2024-09-14T09:06:22.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:22.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408501+0000", 2024-09-14T09:06:22.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:22.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:22.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408504+0000", 2024-09-14T09:06:22.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2819999999999999e-06 2024-09-14T09:06:22.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:22.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408512+0000", 2024-09-14T09:06:22.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7530000000000007e-06 2024-09-14T09:06:22.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:22.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408534+0000", 2024-09-14T09:06:22.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1993000000000001e-05 2024-09-14T09:06:22.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:22.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408550+0000", 2024-09-14T09:06:22.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5988e-05 2024-09-14T09:06:22.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:22.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523914+0000", 2024-09-14T09:06:22.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.115363542 2024-09-14T09:06:22.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:22.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523922+0000", 2024-09-14T09:06:22.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5529999999999993e-06 2024-09-14T09:06:22.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:22.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:22.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:22.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523933+0000", 2024-09-14T09:06:22.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0164e-05 2024-09-14T09:06:22.200 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.201 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.201 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.201 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:22.201 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:22.201 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:22.202 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T09:06:22.202 INFO:teuthology.orchestra.run.smithi176.stdout:2631 writesame 0x7b80237 thru 0x7b851b5 data_size 0x1(0x4f7f bytes) 2024-09-14T09:06:22.203 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:22.203 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:22.204 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:22.204 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:06:22.204 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:22.205 INFO:teuthology.orchestra.run.smithi176.stdout:2633 read 0x27a026f thru 0x27aa8fa (0xa68c bytes) 2024-09-14T09:06:22.205 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:22.205 INFO:teuthology.orchestra.run.smithi176.stdout:2635 writesame 0x5d50266 thru 0x5d550fd data_size 0x14(0x4e98 bytes) 2024-09-14T09:06:22.205 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:22.206 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:22.206 INFO:teuthology.orchestra.run.smithi176.stdout:2636 trunc from 0xd4fc38f to 0x779a9a1 2024-09-14T09:06:22.206 INFO:teuthology.orchestra.run.smithi176.stdout:2637 write 0x61d428f thru 0x61df5e9 (0xb35b bytes) 2024-09-14T09:06:22.206 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:22.207 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:22.207 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:22.203+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:22.207 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:22.208 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:22.208 INFO:teuthology.orchestra.run.smithi176.stdout:2638 write 0xea6fe17 thru 0xea72bb1 (0x2d9b bytes) 2024-09-14T09:06:22.209 INFO:teuthology.orchestra.run.smithi176.stdout:2639 write 0xde88ac2 thru 0xde95433 (0xc972 bytes) 2024-09-14T09:06:22.220 INFO:teuthology.orchestra.run.smithi176.stdout:2640 read 0x3b468d9 thru 0x3b53690 (0xcdb8 bytes) 2024-09-14T09:06:22.220 INFO:teuthology.orchestra.run.smithi176.stdout:2641 write 0x9c9adc9 thru 0x9ca62f3 (0xb52b bytes) 2024-09-14T09:06:22.224 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:22.224 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:22.228 INFO:teuthology.orchestra.run.smithi176.stdout:2643 write 0x49a0ac8 thru 0x49a9708 (0x8c41 bytes) 2024-09-14T09:06:22.234 INFO:teuthology.orchestra.run.smithi176.stdout:2644 punch from 0xd85b024 to 0xd868549, (0xd525 bytes) 2024-09-14T09:06:22.242 INFO:teuthology.orchestra.run.smithi176.stdout:2645 trunc from 0xea72bb2 to 0x92137ea 2024-09-14T09:06:22.246 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:22.246 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:22.256 INFO:teuthology.orchestra.run.smithi176.stdout:2647 punch from 0x76870b7 to 0x7687265, (0x1ae bytes) 2024-09-14T09:06:22.260 INFO:teuthology.orchestra.run.smithi176.stdout:2648 trunc from 0x92137ea to 0x63ca044 2024-09-14T09:06:22.270 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:22.270 INFO:teuthology.orchestra.run.smithi176.stdout:2650 read 0x39689a1 thru 0x397264e (0x9cae bytes) 2024-09-14T09:06:22.270 INFO:teuthology.orchestra.run.smithi176.stdout:2652 read 0x2d5859e thru 0x2d67e2f (0xf892 bytes) 2024-09-14T09:06:22.281 INFO:teuthology.orchestra.run.smithi176.stdout:2654 trunc from 0x63ca044 to 0xe4be79f 2024-09-14T09:06:22.289 INFO:teuthology.orchestra.run.smithi176.stdout:2655 read 0x9725b49 thru 0x97336a7 (0xdb5f bytes) 2024-09-14T09:06:22.303 INFO:teuthology.orchestra.run.smithi176.stdout:2657 trunc from 0xe4be79f to 0x50fdcd6 2024-09-14T09:06:22.303 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:22.303+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:22.321 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:22.321 INFO:teuthology.orchestra.run.smithi176.stdout:2659 clone 22 order 19 su 4096 sc 3 2024-09-14T09:06:22.349 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:22.349 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:22.364 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:06:22.408 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:22.411+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:22.485 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:22.486 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:22.499 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:06:22.544 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:22.547+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:22.606 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:22.607+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:22.668 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:22.669 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:22.684 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:06:22.730 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:22.731+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:22.811 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:22.811+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:22.813 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:22.813 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:22.829 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:06:22.913 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:22.915+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:22.921 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:06:22.922 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:06:22.922 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T09:06:22.922 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T09:06:22.993 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:22.994 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:23.009 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:06:23.013 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:23.015+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:23.114 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:23.115+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:23.128 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:23.128 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:23.142 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:06:23.182 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is unset 2024-09-14T09:06:23.249 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:06:23.249 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:23.249 INFO:teuthology.orchestra.run.smithi176.stdout: "epoch": 66, 2024-09-14T09:06:23.249 INFO:teuthology.orchestra.run.smithi176.stdout: "fsid": "c635667d-994e-454f-bcea-51b45637b2a2", 2024-09-14T09:06:23.249 INFO:teuthology.orchestra.run.smithi176.stdout: "created": "2024-09-14T09:04:57.077293+0000", 2024-09-14T09:06:23.249 INFO:teuthology.orchestra.run.smithi176.stdout: "modified": "2024-09-14T09:06:23.177116+0000", 2024-09-14T09:06:23.249 INFO:teuthology.orchestra.run.smithi176.stdout: "last_up_change": "2024-09-14T09:05:11.210835+0000", 2024-09-14T09:06:23.249 INFO:teuthology.orchestra.run.smithi176.stdout: "last_in_change": "2024-09-14T09:04:59.344709+0000", 2024-09-14T09:06:23.250 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T09:06:23.250 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_num": 5799936, 2024-09-14T09:06:23.250 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_set": [ 2024-09-14T09:06:23.250 INFO:teuthology.orchestra.run.smithi176.stdout: "pglog_hardlimit", 2024-09-14T09:06:23.250 INFO:teuthology.orchestra.run.smithi176.stdout: "purged_snapdirs", 2024-09-14T09:06:23.250 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_deletes", 2024-09-14T09:06:23.250 INFO:teuthology.orchestra.run.smithi176.stdout: "sortbitwise" 2024-09-14T09:06:23.250 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:06:23.250 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_version": 5, 2024-09-14T09:06:23.250 INFO:teuthology.orchestra.run.smithi176.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T09:06:23.250 INFO:teuthology.orchestra.run.smithi176.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T09:06:23.250 INFO:teuthology.orchestra.run.smithi176.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T09:06:23.250 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_snapshot": "", 2024-09-14T09:06:23.251 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_max": 3, 2024-09-14T09:06:23.251 INFO:teuthology.orchestra.run.smithi176.stdout: "max_osd": 8, 2024-09-14T09:06:23.251 INFO:teuthology.orchestra.run.smithi176.stdout: "require_min_compat_client": "luminous", 2024-09-14T09:06:23.251 INFO:teuthology.orchestra.run.smithi176.stdout: "min_compat_client": "jewel", 2024-09-14T09:06:23.251 INFO:teuthology.orchestra.run.smithi176.stdout: "require_osd_release": "squid", 2024-09-14T09:06:23.251 INFO:teuthology.orchestra.run.smithi176.stdout: "allow_crimson": false, 2024-09-14T09:06:23.251 INFO:teuthology.orchestra.run.smithi176.stdout: "pools": [ 2024-09-14T09:06:23.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.251 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 1, 2024-09-14T09:06:23.251 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_name": ".mgr", 2024-09-14T09:06:23.251 INFO:teuthology.orchestra.run.smithi176.stdout: "create_time": "2024-09-14T09:05:11.997673+0000", 2024-09-14T09:06:23.251 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": 1, 2024-09-14T09:06:23.251 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_names": "hashpspool", 2024-09-14T09:06:23.251 INFO:teuthology.orchestra.run.smithi176.stdout: "type": 1, 2024-09-14T09:06:23.252 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 2, 2024-09-14T09:06:23.252 INFO:teuthology.orchestra.run.smithi176.stdout: "min_size": 1, 2024-09-14T09:06:23.252 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_rule": 0, 2024-09-14T09:06:23.252 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_count": 0, 2024-09-14T09:06:23.252 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_target": 0, 2024-09-14T09:06:23.252 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T09:06:23.252 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T09:06:23.252 INFO:teuthology.orchestra.run.smithi176.stdout: "is_stretch_pool": false, 2024-09-14T09:06:23.252 INFO:teuthology.orchestra.run.smithi176.stdout: "object_hash": 2, 2024-09-14T09:06:23.252 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_autoscale_mode": "off", 2024-09-14T09:06:23.252 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num": 1, 2024-09-14T09:06:23.252 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num": 1, 2024-09-14T09:06:23.252 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num_target": 1, 2024-09-14T09:06:23.252 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_target": 1, 2024-09-14T09:06:23.252 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_pending": 1, 2024-09-14T09:06:23.253 INFO:teuthology.orchestra.run.smithi176.stdout: "last_pg_merge_meta": { 2024-09-14T09:06:23.253 INFO:teuthology.orchestra.run.smithi176.stdout: "source_pgid": "0.0", 2024-09-14T09:06:23.253 INFO:teuthology.orchestra.run.smithi176.stdout: "ready_epoch": 0, 2024-09-14T09:06:23.253 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_started": 0, 2024-09-14T09:06:23.253 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_clean": 0, 2024-09-14T09:06:23.253 INFO:teuthology.orchestra.run.smithi176.stdout: "source_version": "0'0", 2024-09-14T09:06:23.253 INFO:teuthology.orchestra.run.smithi176.stdout: "target_version": "0'0" 2024-09-14T09:06:23.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.253 INFO:teuthology.orchestra.run.smithi176.stdout: "last_change": "18", 2024-09-14T09:06:23.253 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend": "0", 2024-09-14T09:06:23.253 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T09:06:23.253 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T09:06:23.253 INFO:teuthology.orchestra.run.smithi176.stdout: "auid": 0, 2024-09-14T09:06:23.253 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_mode": "selfmanaged", 2024-09-14T09:06:23.254 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_seq": 0, 2024-09-14T09:06:23.254 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_epoch": 0, 2024-09-14T09:06:23.254 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_snaps": [], 2024-09-14T09:06:23.254 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps": "[]", 2024-09-14T09:06:23.254 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_bytes": 0, 2024-09-14T09:06:23.254 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_objects": 0, 2024-09-14T09:06:23.254 INFO:teuthology.orchestra.run.smithi176.stdout: "tiers": [], 2024-09-14T09:06:23.254 INFO:teuthology.orchestra.run.smithi176.stdout: "tier_of": -1, 2024-09-14T09:06:23.254 INFO:teuthology.orchestra.run.smithi176.stdout: "read_tier": -1, 2024-09-14T09:06:23.254 INFO:teuthology.orchestra.run.smithi176.stdout: "write_tier": -1, 2024-09-14T09:06:23.254 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_mode": "none", 2024-09-14T09:06:23.254 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_bytes": 0, 2024-09-14T09:06:23.254 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_objects": 0, 2024-09-14T09:06:23.254 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T09:06:23.254 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T09:06:23.255 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T09:06:23.255 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_flush_age": 0, 2024-09-14T09:06:23.255 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_evict_age": 0, 2024-09-14T09:06:23.255 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profile": "", 2024-09-14T09:06:23.255 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_params": { 2024-09-14T09:06:23.255 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "none" 2024-09-14T09:06:23.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.255 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_period": 0, 2024-09-14T09:06:23.255 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_count": 0, 2024-09-14T09:06:23.255 INFO:teuthology.orchestra.run.smithi176.stdout: "use_gmt_hitset": true, 2024-09-14T09:06:23.255 INFO:teuthology.orchestra.run.smithi176.stdout: "min_read_recency_for_promote": 0, 2024-09-14T09:06:23.255 INFO:teuthology.orchestra.run.smithi176.stdout: "min_write_recency_for_promote": 0, 2024-09-14T09:06:23.255 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T09:06:23.255 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_search_last_n": 0, 2024-09-14T09:06:23.255 INFO:teuthology.orchestra.run.smithi176.stdout: "grade_table": [], 2024-09-14T09:06:23.256 INFO:teuthology.orchestra.run.smithi176.stdout: "stripe_width": 0, 2024-09-14T09:06:23.256 INFO:teuthology.orchestra.run.smithi176.stdout: "expected_num_objects": 0, 2024-09-14T09:06:23.256 INFO:teuthology.orchestra.run.smithi176.stdout: "fast_read": false, 2024-09-14T09:06:23.256 INFO:teuthology.orchestra.run.smithi176.stdout: "options": { 2024-09-14T09:06:23.256 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_max": 32, 2024-09-14T09:06:23.256 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_min": 1 2024-09-14T09:06:23.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.256 INFO:teuthology.orchestra.run.smithi176.stdout: "application_metadata": { 2024-09-14T09:06:23.256 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": {} 2024-09-14T09:06:23.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.256 INFO:teuthology.orchestra.run.smithi176.stdout: "read_balance": { 2024-09-14T09:06:23.256 INFO:teuthology.orchestra.run.smithi176.stdout: "score_type": "Fair distribution", 2024-09-14T09:06:23.256 INFO:teuthology.orchestra.run.smithi176.stdout: "score_acting": 8, 2024-09-14T09:06:23.256 INFO:teuthology.orchestra.run.smithi176.stdout: "score_stable": 8, 2024-09-14T09:06:23.257 INFO:teuthology.orchestra.run.smithi176.stdout: "optimal_score": 0.25, 2024-09-14T09:06:23.257 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_acting": 2, 2024-09-14T09:06:23.257 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_stable": 2, 2024-09-14T09:06:23.257 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity_weighted": 1, 2024-09-14T09:06:23.257 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity": 1, 2024-09-14T09:06:23.257 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity_weighted": 1 2024-09-14T09:06:23.257 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.257 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 2, 2024-09-14T09:06:23.257 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_name": "rbd", 2024-09-14T09:06:23.257 INFO:teuthology.orchestra.run.smithi176.stdout: "create_time": "2024-09-14T09:05:15.352551+0000", 2024-09-14T09:06:23.257 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": 8193, 2024-09-14T09:06:23.257 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T09:06:23.258 INFO:teuthology.orchestra.run.smithi176.stdout: "type": 1, 2024-09-14T09:06:23.258 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 2, 2024-09-14T09:06:23.258 INFO:teuthology.orchestra.run.smithi176.stdout: "min_size": 1, 2024-09-14T09:06:23.258 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_rule": 0, 2024-09-14T09:06:23.258 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_count": 0, 2024-09-14T09:06:23.258 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_target": 0, 2024-09-14T09:06:23.258 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T09:06:23.258 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T09:06:23.258 INFO:teuthology.orchestra.run.smithi176.stdout: "is_stretch_pool": false, 2024-09-14T09:06:23.258 INFO:teuthology.orchestra.run.smithi176.stdout: "object_hash": 2, 2024-09-14T09:06:23.258 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_autoscale_mode": "off", 2024-09-14T09:06:23.258 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num": 8, 2024-09-14T09:06:23.258 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num": 8, 2024-09-14T09:06:23.258 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num_target": 8, 2024-09-14T09:06:23.258 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_target": 8, 2024-09-14T09:06:23.259 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_pending": 8, 2024-09-14T09:06:23.259 INFO:teuthology.orchestra.run.smithi176.stdout: "last_pg_merge_meta": { 2024-09-14T09:06:23.259 INFO:teuthology.orchestra.run.smithi176.stdout: "source_pgid": "0.0", 2024-09-14T09:06:23.259 INFO:teuthology.orchestra.run.smithi176.stdout: "ready_epoch": 0, 2024-09-14T09:06:23.259 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_started": 0, 2024-09-14T09:06:23.259 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_clean": 0, 2024-09-14T09:06:23.259 INFO:teuthology.orchestra.run.smithi176.stdout: "source_version": "0'0", 2024-09-14T09:06:23.259 INFO:teuthology.orchestra.run.smithi176.stdout: "target_version": "0'0" 2024-09-14T09:06:23.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.259 INFO:teuthology.orchestra.run.smithi176.stdout: "last_change": "22", 2024-09-14T09:06:23.259 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend": "0", 2024-09-14T09:06:23.259 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T09:06:23.259 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T09:06:23.259 INFO:teuthology.orchestra.run.smithi176.stdout: "auid": 0, 2024-09-14T09:06:23.260 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_mode": "selfmanaged", 2024-09-14T09:06:23.260 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_seq": 2, 2024-09-14T09:06:23.260 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_epoch": 22, 2024-09-14T09:06:23.260 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_snaps": [], 2024-09-14T09:06:23.260 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps": "[]", 2024-09-14T09:06:23.260 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_bytes": 0, 2024-09-14T09:06:23.260 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_objects": 0, 2024-09-14T09:06:23.260 INFO:teuthology.orchestra.run.smithi176.stdout: "tiers": [], 2024-09-14T09:06:23.260 INFO:teuthology.orchestra.run.smithi176.stdout: "tier_of": -1, 2024-09-14T09:06:23.260 INFO:teuthology.orchestra.run.smithi176.stdout: "read_tier": -1, 2024-09-14T09:06:23.260 INFO:teuthology.orchestra.run.smithi176.stdout: "write_tier": -1, 2024-09-14T09:06:23.260 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_mode": "none", 2024-09-14T09:06:23.260 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_bytes": 0, 2024-09-14T09:06:23.260 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_objects": 0, 2024-09-14T09:06:23.260 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T09:06:23.261 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T09:06:23.261 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T09:06:23.261 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_flush_age": 0, 2024-09-14T09:06:23.261 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_evict_age": 0, 2024-09-14T09:06:23.261 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profile": "", 2024-09-14T09:06:23.261 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_params": { 2024-09-14T09:06:23.261 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "none" 2024-09-14T09:06:23.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.261 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_period": 0, 2024-09-14T09:06:23.261 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_count": 0, 2024-09-14T09:06:23.261 INFO:teuthology.orchestra.run.smithi176.stdout: "use_gmt_hitset": true, 2024-09-14T09:06:23.261 INFO:teuthology.orchestra.run.smithi176.stdout: "min_read_recency_for_promote": 0, 2024-09-14T09:06:23.261 INFO:teuthology.orchestra.run.smithi176.stdout: "min_write_recency_for_promote": 0, 2024-09-14T09:06:23.261 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T09:06:23.262 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_search_last_n": 0, 2024-09-14T09:06:23.262 INFO:teuthology.orchestra.run.smithi176.stdout: "grade_table": [], 2024-09-14T09:06:23.262 INFO:teuthology.orchestra.run.smithi176.stdout: "stripe_width": 0, 2024-09-14T09:06:23.262 INFO:teuthology.orchestra.run.smithi176.stdout: "expected_num_objects": 0, 2024-09-14T09:06:23.262 INFO:teuthology.orchestra.run.smithi176.stdout: "fast_read": false, 2024-09-14T09:06:23.262 INFO:teuthology.orchestra.run.smithi176.stdout: "options": {}, 2024-09-14T09:06:23.262 INFO:teuthology.orchestra.run.smithi176.stdout: "application_metadata": { 2024-09-14T09:06:23.262 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd": {} 2024-09-14T09:06:23.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.262 INFO:teuthology.orchestra.run.smithi176.stdout: "read_balance": { 2024-09-14T09:06:23.262 INFO:teuthology.orchestra.run.smithi176.stdout: "score_type": "Fair distribution", 2024-09-14T09:06:23.262 INFO:teuthology.orchestra.run.smithi176.stdout: "score_acting": 1.9900000095367432, 2024-09-14T09:06:23.262 INFO:teuthology.orchestra.run.smithi176.stdout: "score_stable": 1.9900000095367432, 2024-09-14T09:06:23.265 INFO:teuthology.orchestra.run.smithi176.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T09:06:23.265 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_acting": 1.75, 2024-09-14T09:06:23.265 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_stable": 1.75, 2024-09-14T09:06:23.265 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity_weighted": 1, 2024-09-14T09:06:23.265 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity": 1, 2024-09-14T09:06:23.265 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity_weighted": 1 2024-09-14T09:06:23.265 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.265 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 3, 2024-09-14T09:06:23.265 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_name": "pool_client.0", 2024-09-14T09:06:23.266 INFO:teuthology.orchestra.run.smithi176.stdout: "create_time": "2024-09-14T09:05:25.840963+0000", 2024-09-14T09:06:23.266 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": 8193, 2024-09-14T09:06:23.266 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T09:06:23.266 INFO:teuthology.orchestra.run.smithi176.stdout: "type": 1, 2024-09-14T09:06:23.266 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 2, 2024-09-14T09:06:23.266 INFO:teuthology.orchestra.run.smithi176.stdout: "min_size": 1, 2024-09-14T09:06:23.266 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_rule": 0, 2024-09-14T09:06:23.266 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_count": 0, 2024-09-14T09:06:23.266 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_target": 0, 2024-09-14T09:06:23.266 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T09:06:23.266 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T09:06:23.266 INFO:teuthology.orchestra.run.smithi176.stdout: "is_stretch_pool": false, 2024-09-14T09:06:23.266 INFO:teuthology.orchestra.run.smithi176.stdout: "object_hash": 2, 2024-09-14T09:06:23.266 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_autoscale_mode": "off", 2024-09-14T09:06:23.267 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num": 32, 2024-09-14T09:06:23.267 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num": 32, 2024-09-14T09:06:23.267 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num_target": 32, 2024-09-14T09:06:23.267 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_target": 32, 2024-09-14T09:06:23.267 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_pending": 32, 2024-09-14T09:06:23.267 INFO:teuthology.orchestra.run.smithi176.stdout: "last_pg_merge_meta": { 2024-09-14T09:06:23.267 INFO:teuthology.orchestra.run.smithi176.stdout: "source_pgid": "0.0", 2024-09-14T09:06:23.267 INFO:teuthology.orchestra.run.smithi176.stdout: "ready_epoch": 0, 2024-09-14T09:06:23.267 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_started": 0, 2024-09-14T09:06:23.267 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_clean": 0, 2024-09-14T09:06:23.267 INFO:teuthology.orchestra.run.smithi176.stdout: "source_version": "0'0", 2024-09-14T09:06:23.267 INFO:teuthology.orchestra.run.smithi176.stdout: "target_version": "0'0" 2024-09-14T09:06:23.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.267 INFO:teuthology.orchestra.run.smithi176.stdout: "last_change": "66", 2024-09-14T09:06:23.267 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend": "0", 2024-09-14T09:06:23.268 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T09:06:23.268 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T09:06:23.268 INFO:teuthology.orchestra.run.smithi176.stdout: "auid": 0, 2024-09-14T09:06:23.268 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_mode": "selfmanaged", 2024-09-14T09:06:23.268 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_seq": 24, 2024-09-14T09:06:23.268 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_epoch": 66, 2024-09-14T09:06:23.268 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_snaps": [], 2024-09-14T09:06:23.268 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps": "[]", 2024-09-14T09:06:23.268 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_bytes": 0, 2024-09-14T09:06:23.268 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_objects": 0, 2024-09-14T09:06:23.268 INFO:teuthology.orchestra.run.smithi176.stdout: "tiers": [], 2024-09-14T09:06:23.268 INFO:teuthology.orchestra.run.smithi176.stdout: "tier_of": -1, 2024-09-14T09:06:23.268 INFO:teuthology.orchestra.run.smithi176.stdout: "read_tier": -1, 2024-09-14T09:06:23.268 INFO:teuthology.orchestra.run.smithi176.stdout: "write_tier": -1, 2024-09-14T09:06:23.269 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_mode": "none", 2024-09-14T09:06:23.269 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_bytes": 0, 2024-09-14T09:06:23.269 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_objects": 0, 2024-09-14T09:06:23.269 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T09:06:23.269 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T09:06:23.269 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T09:06:23.269 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_flush_age": 0, 2024-09-14T09:06:23.269 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_evict_age": 0, 2024-09-14T09:06:23.269 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profile": "", 2024-09-14T09:06:23.269 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_params": { 2024-09-14T09:06:23.269 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "none" 2024-09-14T09:06:23.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.270 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_period": 0, 2024-09-14T09:06:23.270 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_count": 0, 2024-09-14T09:06:23.270 INFO:teuthology.orchestra.run.smithi176.stdout: "use_gmt_hitset": true, 2024-09-14T09:06:23.270 INFO:teuthology.orchestra.run.smithi176.stdout: "min_read_recency_for_promote": 0, 2024-09-14T09:06:23.270 INFO:teuthology.orchestra.run.smithi176.stdout: "min_write_recency_for_promote": 0, 2024-09-14T09:06:23.270 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T09:06:23.270 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_search_last_n": 0, 2024-09-14T09:06:23.270 INFO:teuthology.orchestra.run.smithi176.stdout: "grade_table": [], 2024-09-14T09:06:23.270 INFO:teuthology.orchestra.run.smithi176.stdout: "stripe_width": 0, 2024-09-14T09:06:23.270 INFO:teuthology.orchestra.run.smithi176.stdout: "expected_num_objects": 0, 2024-09-14T09:06:23.270 INFO:teuthology.orchestra.run.smithi176.stdout: "fast_read": false, 2024-09-14T09:06:23.270 INFO:teuthology.orchestra.run.smithi176.stdout: "options": {}, 2024-09-14T09:06:23.270 INFO:teuthology.orchestra.run.smithi176.stdout: "application_metadata": { 2024-09-14T09:06:23.271 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd": {} 2024-09-14T09:06:23.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.271 INFO:teuthology.orchestra.run.smithi176.stdout: "read_balance": { 2024-09-14T09:06:23.271 INFO:teuthology.orchestra.run.smithi176.stdout: "score_type": "Fair distribution", 2024-09-14T09:06:23.271 INFO:teuthology.orchestra.run.smithi176.stdout: "score_acting": 1.75, 2024-09-14T09:06:23.271 INFO:teuthology.orchestra.run.smithi176.stdout: "score_stable": 1.75, 2024-09-14T09:06:23.271 INFO:teuthology.orchestra.run.smithi176.stdout: "optimal_score": 1, 2024-09-14T09:06:23.271 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_acting": 1.75, 2024-09-14T09:06:23.271 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_stable": 1.75, 2024-09-14T09:06:23.271 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity_weighted": 1, 2024-09-14T09:06:23.271 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity": 1, 2024-09-14T09:06:23.271 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity_weighted": 1 2024-09-14T09:06:23.271 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.271 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.272 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:06:23.272 INFO:teuthology.orchestra.run.smithi176.stdout: "osds": [ 2024-09-14T09:06:23.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.272 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 0, 2024-09-14T09:06:23.272 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "4ceedb4f-6398-467f-9562-ccf27200860b", 2024-09-14T09:06:23.272 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:06:23.272 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:06:23.272 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 0.8736724853515625, 2024-09-14T09:06:23.272 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:06:23.272 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:06:23.272 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:06:23.272 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:06:23.272 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 34, 2024-09-14T09:06:23.272 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:06:23.273 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:06:23.273 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:06:23.273 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.273 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.273 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6824", 2024-09-14T09:06:23.273 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:06:23.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.273 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.273 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6825", 2024-09-14T09:06:23.273 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:06:23.273 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.273 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.274 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:06:23.274 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.274 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.274 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6826", 2024-09-14T09:06:23.274 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:06:23.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.274 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.274 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6827", 2024-09-14T09:06:23.274 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:06:23.274 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.274 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.275 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:06:23.275 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.275 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.275 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6830", 2024-09-14T09:06:23.275 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:06:23.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.275 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.275 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6831", 2024-09-14T09:06:23.275 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:06:23.275 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.275 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.276 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:06:23.276 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.276 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.276 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6828", 2024-09-14T09:06:23.276 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:06:23.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.276 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.276 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6829", 2024-09-14T09:06:23.276 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:06:23.276 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.277 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.277 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6825/3384710819", 2024-09-14T09:06:23.277 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6827/3384710819", 2024-09-14T09:06:23.277 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6831/3384710819", 2024-09-14T09:06:23.277 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6829/3384710819", 2024-09-14T09:06:23.277 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:06:23.277 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:06:23.277 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:06:23.277 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.277 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 1, 2024-09-14T09:06:23.277 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "604e2f62-7c13-46be-a97f-029b28c44a89", 2024-09-14T09:06:23.278 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:06:23.278 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:06:23.278 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:06:23.278 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:06:23.278 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:06:23.278 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:06:23.278 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:06:23.278 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 53, 2024-09-14T09:06:23.278 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:06:23.278 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:06:23.278 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:06:23.278 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.278 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.278 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6800", 2024-09-14T09:06:23.279 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:06:23.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.279 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.279 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6801", 2024-09-14T09:06:23.279 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:06:23.279 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.279 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.279 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:06:23.279 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.279 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.280 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6802", 2024-09-14T09:06:23.280 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:06:23.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.280 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.280 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6803", 2024-09-14T09:06:23.280 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:06:23.280 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.280 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.280 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:06:23.280 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.280 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.281 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6806", 2024-09-14T09:06:23.281 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:06:23.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.281 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.281 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6807", 2024-09-14T09:06:23.281 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:06:23.281 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.281 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.281 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:06:23.281 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.281 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.282 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6804", 2024-09-14T09:06:23.282 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:06:23.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.282 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.282 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6805", 2024-09-14T09:06:23.282 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:06:23.282 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.282 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.282 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6801/612012679", 2024-09-14T09:06:23.282 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6803/612012679", 2024-09-14T09:06:23.282 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6807/612012679", 2024-09-14T09:06:23.282 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6805/612012679", 2024-09-14T09:06:23.283 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:06:23.283 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:06:23.283 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:06:23.283 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.283 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 2, 2024-09-14T09:06:23.283 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "1f65589e-8b0a-4033-94c0-4ab66531db2b", 2024-09-14T09:06:23.283 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:06:23.283 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:06:23.283 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:06:23.283 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:06:23.283 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:06:23.283 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:06:23.284 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:06:23.284 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 34, 2024-09-14T09:06:23.284 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:06:23.284 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:06:23.284 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:06:23.284 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.284 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.284 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6808", 2024-09-14T09:06:23.284 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:06:23.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.284 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.284 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6809", 2024-09-14T09:06:23.284 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:06:23.285 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.285 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.285 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:06:23.285 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.285 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.285 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6810", 2024-09-14T09:06:23.285 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:06:23.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.285 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.285 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6811", 2024-09-14T09:06:23.285 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:06:23.286 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.286 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.286 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:06:23.286 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.286 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.286 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6814", 2024-09-14T09:06:23.286 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:06:23.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.286 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.286 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6815", 2024-09-14T09:06:23.286 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:06:23.286 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.287 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.287 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:06:23.287 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.287 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.287 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6812", 2024-09-14T09:06:23.287 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:06:23.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.287 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.287 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6813", 2024-09-14T09:06:23.287 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:06:23.287 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.287 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.288 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6809/4167379279", 2024-09-14T09:06:23.288 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6811/4167379279", 2024-09-14T09:06:23.288 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6815/4167379279", 2024-09-14T09:06:23.288 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6813/4167379279", 2024-09-14T09:06:23.288 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:06:23.288 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:06:23.288 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:06:23.288 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.288 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 3, 2024-09-14T09:06:23.288 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "209d4f65-79f9-4a7d-9d23-832a4209507f", 2024-09-14T09:06:23.288 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:06:23.288 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:06:23.288 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:06:23.289 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:06:23.289 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:06:23.289 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:06:23.289 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:06:23.289 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 53, 2024-09-14T09:06:23.289 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:06:23.289 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:06:23.289 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:06:23.289 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.289 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.289 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6816", 2024-09-14T09:06:23.289 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:06:23.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.290 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.290 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6817", 2024-09-14T09:06:23.290 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:06:23.290 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.290 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.290 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:06:23.290 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.290 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.290 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6818", 2024-09-14T09:06:23.290 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:06:23.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.290 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.291 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6819", 2024-09-14T09:06:23.291 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:06:23.291 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.291 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.291 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:06:23.291 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.291 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.291 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6822", 2024-09-14T09:06:23.291 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:06:23.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.291 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.291 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6823", 2024-09-14T09:06:23.292 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:06:23.292 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.292 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.292 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:06:23.292 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.292 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.292 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6820", 2024-09-14T09:06:23.292 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:06:23.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.292 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.292 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6821", 2024-09-14T09:06:23.292 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:06:23.293 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.293 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.293 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6817/1658209015", 2024-09-14T09:06:23.293 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6819/1658209015", 2024-09-14T09:06:23.293 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6823/1658209015", 2024-09-14T09:06:23.293 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6821/1658209015", 2024-09-14T09:06:23.293 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:06:23.293 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:06:23.293 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:06:23.293 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.293 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 4, 2024-09-14T09:06:23.294 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "8f9a8c9b-282e-4523-9e62-5824f4aba401", 2024-09-14T09:06:23.294 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:06:23.294 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:06:23.294 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 0.1816558837890625, 2024-09-14T09:06:23.294 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:06:23.294 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:06:23.294 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:06:23.294 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:06:23.294 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 54, 2024-09-14T09:06:23.294 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:06:23.294 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:06:23.294 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:06:23.294 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.294 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.295 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6808", 2024-09-14T09:06:23.295 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:06:23.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.295 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.295 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6809", 2024-09-14T09:06:23.295 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:06:23.295 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.295 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.295 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:06:23.295 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.295 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.296 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6810", 2024-09-14T09:06:23.296 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:06:23.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.296 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.296 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6811", 2024-09-14T09:06:23.296 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:06:23.296 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.296 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.296 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:06:23.296 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.296 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.297 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6814", 2024-09-14T09:06:23.297 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:06:23.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.297 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.297 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6815", 2024-09-14T09:06:23.297 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:06:23.297 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.297 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.297 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:06:23.297 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.297 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.298 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6812", 2024-09-14T09:06:23.298 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:06:23.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.298 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.298 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6813", 2024-09-14T09:06:23.298 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:06:23.298 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.298 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.298 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6809/1576054389", 2024-09-14T09:06:23.298 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6811/1576054389", 2024-09-14T09:06:23.298 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6815/1576054389", 2024-09-14T09:06:23.298 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6813/1576054389", 2024-09-14T09:06:23.299 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:06:23.299 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:06:23.299 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:06:23.299 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.299 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 5, 2024-09-14T09:06:23.299 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80", 2024-09-14T09:06:23.299 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:06:23.299 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:06:23.299 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:06:23.299 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:06:23.299 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:06:23.299 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:06:23.299 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:06:23.300 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 34, 2024-09-14T09:06:23.300 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:06:23.300 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:06:23.300 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:06:23.300 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.300 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.300 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6824", 2024-09-14T09:06:23.300 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:06:23.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.300 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.300 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6825", 2024-09-14T09:06:23.301 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:06:23.301 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.301 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.301 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:06:23.301 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.301 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.301 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6826", 2024-09-14T09:06:23.301 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:06:23.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.301 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.301 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6827", 2024-09-14T09:06:23.302 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:06:23.302 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.302 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.302 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:06:23.302 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.302 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.302 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6830", 2024-09-14T09:06:23.302 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:06:23.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.303 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.303 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6831", 2024-09-14T09:06:23.303 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:06:23.303 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.303 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.303 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:06:23.303 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.303 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.303 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6828", 2024-09-14T09:06:23.303 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:06:23.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.304 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.304 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6829", 2024-09-14T09:06:23.304 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:06:23.304 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.304 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.304 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6825/1711292386", 2024-09-14T09:06:23.304 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6827/1711292386", 2024-09-14T09:06:23.304 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6831/1711292386", 2024-09-14T09:06:23.304 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6829/1711292386", 2024-09-14T09:06:23.304 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:06:23.305 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:06:23.305 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:06:23.305 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.305 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 6, 2024-09-14T09:06:23.305 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "59d9259d-b9dd-4326-8ab9-9d599a2bf57b", 2024-09-14T09:06:23.305 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:06:23.305 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:06:23.305 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:06:23.305 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:06:23.305 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:06:23.305 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:06:23.306 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:06:23.306 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 50, 2024-09-14T09:06:23.306 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:06:23.306 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:06:23.306 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:06:23.306 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.306 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.306 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6816", 2024-09-14T09:06:23.306 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:06:23.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.306 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.306 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6817", 2024-09-14T09:06:23.307 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:06:23.307 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.307 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.307 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:06:23.307 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.307 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.307 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6818", 2024-09-14T09:06:23.307 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:06:23.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.307 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.307 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6819", 2024-09-14T09:06:23.307 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:06:23.308 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.308 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.308 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:06:23.308 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.308 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.308 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6822", 2024-09-14T09:06:23.308 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:06:23.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.308 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.308 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6823", 2024-09-14T09:06:23.308 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:06:23.309 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.309 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.309 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:06:23.309 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.309 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.309 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6820", 2024-09-14T09:06:23.309 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:06:23.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.309 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.309 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6821", 2024-09-14T09:06:23.310 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:06:23.310 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.310 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.310 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6817/2721245607", 2024-09-14T09:06:23.310 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6819/2721245607", 2024-09-14T09:06:23.310 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6823/2721245607", 2024-09-14T09:06:23.310 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6821/2721245607", 2024-09-14T09:06:23.310 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:06:23.310 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:06:23.310 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:06:23.310 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.311 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 7, 2024-09-14T09:06:23.311 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "e2f27d42-786b-449b-a2fb-9dd28f4e9788", 2024-09-14T09:06:23.311 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:06:23.311 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:06:23.311 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:06:23.311 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:06:23.311 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:06:23.311 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:06:23.311 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:06:23.311 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 53, 2024-09-14T09:06:23.311 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:06:23.311 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:06:23.311 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:06:23.311 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.312 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.312 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6800", 2024-09-14T09:06:23.312 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:06:23.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.312 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.312 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6801", 2024-09-14T09:06:23.312 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:06:23.312 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.312 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.312 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:06:23.312 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.313 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.313 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6802", 2024-09-14T09:06:23.313 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:06:23.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.313 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.313 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6803", 2024-09-14T09:06:23.313 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:06:23.313 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.313 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.313 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:06:23.313 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.314 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.314 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6806", 2024-09-14T09:06:23.314 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:06:23.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.314 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.314 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6807", 2024-09-14T09:06:23.314 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:06:23.314 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.314 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.314 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:06:23.314 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:23.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.315 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:23.315 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6804", 2024-09-14T09:06:23.315 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:06:23.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.315 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:23.315 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6805", 2024-09-14T09:06:23.315 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:06:23.315 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.315 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.315 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6801/2122823465", 2024-09-14T09:06:23.315 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6803/2122823465", 2024-09-14T09:06:23.316 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6807/2122823465", 2024-09-14T09:06:23.316 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6805/2122823465", 2024-09-14T09:06:23.316 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:06:23.316 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:06:23.316 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:06:23.316 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:23.316 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.316 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:06:23.316 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_xinfo": [ 2024-09-14T09:06:23.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.316 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 0, 2024-09-14T09:06:23.316 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:06:23.316 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:06:23.316 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:06:23.316 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:06:23.317 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:06:23.317 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.411789+0000", 2024-09-14T09:06:23.317 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:06:23.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.317 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 1, 2024-09-14T09:06:23.317 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:06:23.317 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:06:23.317 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:06:23.317 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:06:23.317 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:06:23.317 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:05.801873+0000", 2024-09-14T09:06:23.317 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:06:23.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.318 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 2, 2024-09-14T09:06:23.318 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:06:23.318 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:06:23.318 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:06:23.318 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:06:23.318 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:06:23.318 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.206725+0000", 2024-09-14T09:06:23.318 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:06:23.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.318 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 3, 2024-09-14T09:06:23.318 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:06:23.318 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:06:23.318 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:06:23.319 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:06:23.319 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:06:23.319 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:05.882180+0000", 2024-09-14T09:06:23.319 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:06:23.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.319 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 4, 2024-09-14T09:06:23.319 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:06:23.319 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:06:23.319 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:06:23.319 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:06:23.319 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:06:23.319 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.013864+0000", 2024-09-14T09:06:23.319 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:06:23.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.320 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 5, 2024-09-14T09:06:23.320 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:06:23.320 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:06:23.320 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:06:23.320 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:06:23.320 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:06:23.320 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.652975+0000", 2024-09-14T09:06:23.320 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:06:23.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.320 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 6, 2024-09-14T09:06:23.320 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:06:23.320 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:06:23.320 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:06:23.320 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:06:23.321 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:06:23.321 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.741359+0000", 2024-09-14T09:06:23.321 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:06:23.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:23.321 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 7, 2024-09-14T09:06:23.321 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:06:23.321 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:06:23.321 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:06:23.321 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:06:23.322 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:06:23.322 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:05.873675+0000", 2024-09-14T09:06:23.322 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:06:23.322 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.322 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:06:23.322 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_upmap": [], 2024-09-14T09:06:23.322 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_upmap_items": [], 2024-09-14T09:06:23.322 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_upmap_primaries": [], 2024-09-14T09:06:23.322 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_temp": [], 2024-09-14T09:06:23.322 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_temp": [], 2024-09-14T09:06:23.322 INFO:teuthology.orchestra.run.smithi176.stdout: "blocklist": {}, 2024-09-14T09:06:23.322 INFO:teuthology.orchestra.run.smithi176.stdout: "range_blocklist": {}, 2024-09-14T09:06:23.322 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profiles": { 2024-09-14T09:06:23.322 INFO:teuthology.orchestra.run.smithi176.stdout: "default": { 2024-09-14T09:06:23.323 INFO:teuthology.orchestra.run.smithi176.stdout: "crush-failure-domain": "osd", 2024-09-14T09:06:23.323 INFO:teuthology.orchestra.run.smithi176.stdout: "k": "2", 2024-09-14T09:06:23.323 INFO:teuthology.orchestra.run.smithi176.stdout: "m": "1", 2024-09-14T09:06:23.323 INFO:teuthology.orchestra.run.smithi176.stdout: "plugin": "jerasure", 2024-09-14T09:06:23.323 INFO:teuthology.orchestra.run.smithi176.stdout: "technique": "reed_sol_van" 2024-09-14T09:06:23.323 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:23.323 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps_queue": [], 2024-09-14T09:06:23.323 INFO:teuthology.orchestra.run.smithi176.stdout: "new_removed_snaps": [], 2024-09-14T09:06:23.323 INFO:teuthology.orchestra.run.smithi176.stdout: "new_purged_snaps": [], 2024-09-14T09:06:23.323 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_node_flags": {}, 2024-09-14T09:06:23.323 INFO:teuthology.orchestra.run.smithi176.stdout: "device_class_flags": {}, 2024-09-14T09:06:23.323 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_mode": { 2024-09-14T09:06:23.323 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_mode_enabled": false, 2024-09-14T09:06:23.323 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_bucket_count": 0, 2024-09-14T09:06:23.324 INFO:teuthology.orchestra.run.smithi176.stdout: "degraded_stretch_mode": 0, 2024-09-14T09:06:23.324 INFO:teuthology.orchestra.run.smithi176.stdout: "recovering_stretch_mode": 0, 2024-09-14T09:06:23.324 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_mode_bucket": 0 2024-09-14T09:06:23.324 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:23.324 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:23.324 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:23.247+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:23.326 INFO:tasks.thrashosds.thrasher:j is {'epoch': 66, 'fsid': 'c635667d-994e-454f-bcea-51b45637b2a2', 'created': '2024-09-14T09:04:57.077293+0000', 'modified': '2024-09-14T09:06:23.177116+0000', 'last_up_change': '2024-09-14T09:05:11.210835+0000', 'last_in_change': '2024-09-14T09:04:59.344709+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T09:05:11.997673+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T09:05:15.352551+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T09:05:25.840963+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 24, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '4ceedb4f-6398-467f-9562-ccf27200860b', 'up': 1, 'in': 1, 'weight': 0.8736724853515625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6824', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6825', 'nonce': 3384710819}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6826', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6827', 'nonce': 3384710819}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6830', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6831', 'nonce': 3384710819}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6828', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6829', 'nonce': 3384710819}]}, 'public_addr': '172.21.15.176:6825/3384710819', 'cluster_addr': '172.21.15.176:6827/3384710819', 'heartbeat_back_addr': '172.21.15.176:6831/3384710819', 'heartbeat_front_addr': '172.21.15.176:6829/3384710819', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '604e2f62-7c13-46be-a97f-029b28c44a89', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6800', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6801', 'nonce': 612012679}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6802', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6803', 'nonce': 612012679}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6806', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6807', 'nonce': 612012679}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6804', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6805', 'nonce': 612012679}]}, 'public_addr': '172.21.15.176:6801/612012679', 'cluster_addr': '172.21.15.176:6803/612012679', 'heartbeat_back_addr': '172.21.15.176:6807/612012679', 'heartbeat_front_addr': '172.21.15.176:6805/612012679', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1f65589e-8b0a-4033-94c0-4ab66531db2b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6808', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6809', 'nonce': 4167379279}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6810', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6811', 'nonce': 4167379279}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6814', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6815', 'nonce': 4167379279}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6812', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6813', 'nonce': 4167379279}]}, 'public_addr': '172.21.15.176:6809/4167379279', 'cluster_addr': '172.21.15.176:6811/4167379279', 'heartbeat_back_addr': '172.21.15.176:6815/4167379279', 'heartbeat_front_addr': '172.21.15.176:6813/4167379279', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '209d4f65-79f9-4a7d-9d23-832a4209507f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6816', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6817', 'nonce': 1658209015}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6818', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6819', 'nonce': 1658209015}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6822', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6823', 'nonce': 1658209015}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6820', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6821', 'nonce': 1658209015}]}, 'public_addr': '172.21.15.176:6817/1658209015', 'cluster_addr': '172.21.15.176:6819/1658209015', 'heartbeat_back_addr': '172.21.15.176:6823/1658209015', 'heartbeat_front_addr': '172.21.15.176:6821/1658209015', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8f9a8c9b-282e-4523-9e62-5824f4aba401', 'up': 1, 'in': 1, 'weight': 0.1816558837890625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6808', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6809', 'nonce': 1576054389}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6810', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6811', 'nonce': 1576054389}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6814', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6815', 'nonce': 1576054389}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6812', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6813', 'nonce': 1576054389}]}, 'public_addr': '172.21.15.183:6809/1576054389', 'cluster_addr': '172.21.15.183:6811/1576054389', 'heartbeat_back_addr': '172.21.15.183:6815/1576054389', 'heartbeat_front_addr': '172.21.15.183:6813/1576054389', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6824', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6825', 'nonce': 1711292386}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6826', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6827', 'nonce': 1711292386}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6830', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6831', 'nonce': 1711292386}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6828', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6829', 'nonce': 1711292386}]}, 'public_addr': '172.21.15.183:6825/1711292386', 'cluster_addr': '172.21.15.183:6827/1711292386', 'heartbeat_back_addr': '172.21.15.183:6831/1711292386', 'heartbeat_front_addr': '172.21.15.183:6829/1711292386', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '59d9259d-b9dd-4326-8ab9-9d599a2bf57b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6816', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6817', 'nonce': 2721245607}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6818', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6819', 'nonce': 2721245607}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6822', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6823', 'nonce': 2721245607}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6820', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6821', 'nonce': 2721245607}]}, 'public_addr': '172.21.15.183:6817/2721245607', 'cluster_addr': '172.21.15.183:6819/2721245607', 'heartbeat_back_addr': '172.21.15.183:6823/2721245607', 'heartbeat_front_addr': '172.21.15.183:6821/2721245607', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'e2f27d42-786b-449b-a2fb-9dd28f4e9788', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6800', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6801', 'nonce': 2122823465}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6802', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6803', 'nonce': 2122823465}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6806', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6807', 'nonce': 2122823465}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6804', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6805', 'nonce': 2122823465}]}, 'public_addr': '172.21.15.183:6801/2122823465', 'cluster_addr': '172.21.15.183:6803/2122823465', 'heartbeat_back_addr': '172.21.15.183:6807/2122823465', 'heartbeat_front_addr': '172.21.15.183:6805/2122823465', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.411789+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:05.801873+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.206725+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:05.882180+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.013864+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.652975+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.741359+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:05.873675+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T09:06:23.326 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:06:23.326 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:23.326 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:23.333 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:06:23.334 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:23.331+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:23.345 INFO:teuthology.orchestra.run.smithi176.stdout:leaving image image_client.0-clone21 intact 2024-09-14T09:06:23.428 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:23.427+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:23.465 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:23.465 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:23.483 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:06:23.530 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:23.531+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:23.566 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-14T09:06:23.631 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:23.631+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:23.662 INFO:teuthology.orchestra.run.smithi176.stdout:2660 writesame 0x80aea2 thru 0x8119c2 data_size 0x5(0x6b21 bytes) 2024-09-14T09:06:23.667 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:23.667 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:23.687 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:06:23.702 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:06:23.702 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:06:23.704 INFO:teuthology.orchestra.run.smithi176.stdout:2662 writesame 0x69d669c thru 0x69dce2f data_size 0xe(0x6794 bytes) 2024-09-14T09:06:23.704 INFO:teuthology.orchestra.run.smithi176.stdout:2663 punch from 0x3ffa21a to 0x4007a53, (0xd839 bytes) 2024-09-14T09:06:23.704 INFO:teuthology.orchestra.run.smithi176.stdout:2664 punch from 0x14ffe5 to 0x155664, (0x567f bytes) 2024-09-14T09:06:23.718 INFO:teuthology.orchestra.run.smithi176.stdout:2665 write 0xce4dd2b thru 0xce58dbb (0xb091 bytes) 2024-09-14T09:06:23.726 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-09-14T09:06:23.188367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":889315639,"num_objects":1093,"num_object_clones":367,"num_object_copies":2186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1093,"num_whiteouts":440,"num_read":86850,"num_read_kb":462346,"num_write":9631,"num_write_kb":180987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":11394597,"num_keys_recovered":32,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7745,"ondisk_log_size":7745,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":314760,"kb_used_data":31932,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754659960,"statfs":{"total":773094113280,"available":772771799040,"internally_reserved":0,"allocated":32698368,"data_stored":236617119,"data_compressed":7156145,"data_compressed_allocated":19054592,"data_compressed_original":229293112,"omap_allocated":12751,"internal_metadata":289394225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26044459,"num_objects":302,"num_object_clones":140,"num_object_copies":604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":162,"num_read":16514,"num_read_kb":26112,"num_write":2008,"num_write_kb":7889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.038170"},"pg_stats":[{"pgid":"3.1f","version":"65'562","reported_seq":2873,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:21.324149+0000","last_change":"2024-09-14T09:05:33.869426+0000","last_active":"2024-09-14T09:06:21.324149+0000","last_peered":"2024-09-14T09:06:21.324149+0000","last_clean":"2024-09-14T09:06:21.324149+0000","last_became_active":"2024-09-14T09:05:33.854933+0000","last_became_peered":"2024-09-14T09:05:33.854933+0000","last_unstale":"2024-09-14T09:06:21.324149+0000","last_undegraded":"2024-09-14T09:06:21.324149+0000","last_fullsized":"2024-09-14T09:06:21.324149+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:19:33.084331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001387512,"stat_sum":{"num_bytes":4258035,"num_objects":36,"num_object_clones":12,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":15,"num_read":4775,"num_read_kb":2244,"num_write":935,"num_write_kb":1294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"65'167","reported_seq":1945,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.852751+0000","last_change":"2024-09-14T09:05:29.039503+0000","last_active":"2024-09-14T09:06:20.852751+0000","last_peered":"2024-09-14T09:06:20.852751+0000","last_clean":"2024-09-14T09:06:20.852751+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:06:20.852751+0000","last_undegraded":"2024-09-14T09:06:20.852751+0000","last_fullsized":"2024-09-14T09:06:20.852751+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:36:59.625742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001657176,"stat_sum":{"num_bytes":28863051,"num_objects":49,"num_object_clones":18,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":20,"num_read":1746,"num_read_kb":19553,"num_write":203,"num_write_kb":8360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"65'319","reported_seq":3411,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:21.621594+0000","last_change":"2024-09-14T09:05:28.835298+0000","last_active":"2024-09-14T09:06:21.621594+0000","last_peered":"2024-09-14T09:06:21.621594+0000","last_clean":"2024-09-14T09:06:21.621594+0000","last_became_active":"2024-09-14T09:05:26.835372+0000","last_became_peered":"2024-09-14T09:05:26.835372+0000","last_unstale":"2024-09-14T09:06:21.621594+0000","last_undegraded":"2024-09-14T09:06:21.621594+0000","last_fullsized":"2024-09-14T09:06:21.621594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:19:29.937332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014149329999999999,"stat_sum":{"num_bytes":54287137,"num_objects":35,"num_object_clones":10,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":10,"num_read":3753,"num_read_kb":52432,"num_write":464,"num_write_kb":8291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"65'452","reported_seq":1800,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:22.111653+0000","last_change":"2024-09-14T09:05:39.931845+0000","last_active":"2024-09-14T09:06:22.111653+0000","last_peered":"2024-09-14T09:06:22.111653+0000","last_clean":"2024-09-14T09:06:22.111653+0000","last_became_active":"2024-09-14T09:05:39.918593+0000","last_became_peered":"2024-09-14T09:05:39.918593+0000","last_unstale":"2024-09-14T09:06:22.111653+0000","last_undegraded":"2024-09-14T09:06:22.111653+0000","last_fullsized":"2024-09-14T09:06:22.111653+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:08:56.531068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014661419999999999,"stat_sum":{"num_bytes":21524434,"num_objects":35,"num_object_clones":12,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":4074,"num_read_kb":6922,"num_write":614,"num_write_kb":4323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":17,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"65'183","reported_seq":1920,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.618218+0000","last_change":"2024-09-14T09:06:03.834450+0000","last_active":"2024-09-14T09:06:20.618218+0000","last_peered":"2024-09-14T09:06:20.618218+0000","last_clean":"2024-09-14T09:06:20.618218+0000","last_became_active":"2024-09-14T09:06:03.833448+0000","last_became_peered":"2024-09-14T09:06:03.833448+0000","last_unstale":"2024-09-14T09:06:20.618218+0000","last_undegraded":"2024-09-14T09:06:20.618218+0000","last_fullsized":"2024-09-14T09:06:20.618218+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:31:22.434786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060373399999999998,"stat_sum":{"num_bytes":21403007,"num_objects":36,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":1661,"num_read_kb":5136,"num_write":184,"num_write_kb":7293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":801985,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"64'440","reported_seq":2929,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.985937+0000","last_change":"2024-09-14T09:05:39.919093+0000","last_active":"2024-09-14T09:06:20.985937+0000","last_peered":"2024-09-14T09:06:20.985937+0000","last_clean":"2024-09-14T09:06:20.985937+0000","last_became_active":"2024-09-14T09:05:39.905126+0000","last_became_peered":"2024-09-14T09:05:39.905126+0000","last_unstale":"2024-09-14T09:06:20.985937+0000","last_undegraded":"2024-09-14T09:06:20.985937+0000","last_fullsized":"2024-09-14T09:06:20.985937+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:58:23.112420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020845550000000001,"stat_sum":{"num_bytes":28349418,"num_objects":30,"num_object_clones":10,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":4101,"num_read_kb":18160,"num_write":723,"num_write_kb":7773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":92669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"65'177","reported_seq":1778,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.520543+0000","last_change":"2024-09-14T09:05:28.822565+0000","last_active":"2024-09-14T09:06:20.520543+0000","last_peered":"2024-09-14T09:06:20.520543+0000","last_clean":"2024-09-14T09:06:20.520543+0000","last_became_active":"2024-09-14T09:05:26.816132+0000","last_became_peered":"2024-09-14T09:05:26.816132+0000","last_unstale":"2024-09-14T09:06:20.520543+0000","last_undegraded":"2024-09-14T09:06:20.520543+0000","last_fullsized":"2024-09-14T09:06:20.520543+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:18:02.774387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092420299999999996,"stat_sum":{"num_bytes":17853620,"num_objects":29,"num_object_clones":11,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":1559,"num_read_kb":5082,"num_write":213,"num_write_kb":3618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"65'212","reported_seq":2842,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.832690+0000","last_change":"2024-09-14T09:05:33.865470+0000","last_active":"2024-09-14T09:06:20.832690+0000","last_peered":"2024-09-14T09:06:20.832690+0000","last_clean":"2024-09-14T09:06:20.832690+0000","last_became_active":"2024-09-14T09:05:33.846990+0000","last_became_peered":"2024-09-14T09:05:33.846990+0000","last_unstale":"2024-09-14T09:06:20.832690+0000","last_undegraded":"2024-09-14T09:06:20.832690+0000","last_fullsized":"2024-09-14T09:06:20.832690+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:43:36.595541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081579899999999997,"stat_sum":{"num_bytes":47178872,"num_objects":42,"num_object_clones":17,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":14,"num_read":3430,"num_read_kb":7476,"num_write":201,"num_write_kb":3490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":70784,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"65'293","reported_seq":4090,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:21.866474+0000","last_change":"2024-09-14T09:05:28.839928+0000","last_active":"2024-09-14T09:06:21.866474+0000","last_peered":"2024-09-14T09:06:21.866474+0000","last_clean":"2024-09-14T09:06:21.866474+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:06:21.866474+0000","last_undegraded":"2024-09-14T09:06:21.866474+0000","last_fullsized":"2024-09-14T09:06:21.866474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:20:33.224557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014343089999999999,"stat_sum":{"num_bytes":28984315,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":10,"num_read":4801,"num_read_kb":21486,"num_write":376,"num_write_kb":4459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"65'352","reported_seq":1333,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:21.953154+0000","last_change":"2024-09-14T09:05:28.836370+0000","last_active":"2024-09-14T09:06:21.953154+0000","last_peered":"2024-09-14T09:06:21.953154+0000","last_clean":"2024-09-14T09:06:21.953154+0000","last_became_active":"2024-09-14T09:05:26.836744+0000","last_became_peered":"2024-09-14T09:05:26.836744+0000","last_unstale":"2024-09-14T09:06:21.953154+0000","last_undegraded":"2024-09-14T09:06:21.953154+0000","last_fullsized":"2024-09-14T09:06:21.953154+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:52:22.240310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015277190000000001,"stat_sum":{"num_bytes":9045751,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":9,"num_read":2959,"num_read_kb":4268,"num_write":455,"num_write_kb":2209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"65'480","reported_seq":1566,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.208922+0000","last_change":"2024-09-14T09:05:28.851587+0000","last_active":"2024-09-14T09:06:20.208922+0000","last_peered":"2024-09-14T09:06:20.208922+0000","last_clean":"2024-09-14T09:06:20.208922+0000","last_became_active":"2024-09-14T09:05:26.840893+0000","last_became_peered":"2024-09-14T09:05:26.840893+0000","last_unstale":"2024-09-14T09:06:20.208922+0000","last_undegraded":"2024-09-14T09:06:20.208922+0000","last_fullsized":"2024-09-14T09:06:20.208922+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:42:49.452730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098594000000000004,"stat_sum":{"num_bytes":8709117,"num_objects":30,"num_object_clones":9,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":13,"num_read":3902,"num_read_kb":3835,"num_write":746,"num_write_kb":2518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"64'279","reported_seq":1387,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.208725+0000","last_change":"2024-09-14T09:05:42.832730+0000","last_active":"2024-09-14T09:06:20.208725+0000","last_peered":"2024-09-14T09:06:20.208725+0000","last_clean":"2024-09-14T09:06:20.208725+0000","last_became_active":"2024-09-14T09:05:42.829504+0000","last_became_peered":"2024-09-14T09:05:42.829504+0000","last_unstale":"2024-09-14T09:06:20.208725+0000","last_undegraded":"2024-09-14T09:06:20.208725+0000","last_fullsized":"2024-09-14T09:06:20.208725+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:24:43.662532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014079679999999999,"stat_sum":{"num_bytes":20244700,"num_objects":37,"num_object_clones":13,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":16,"num_read":2076,"num_read_kb":3679,"num_write":327,"num_write_kb":632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":53852,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.239510+0000","last_change":"2024-09-14T09:05:19.793674+0000","last_active":"2024-09-14T09:06:20.239510+0000","last_peered":"2024-09-14T09:06:20.239510+0000","last_clean":"2024-09-14T09:06:20.239510+0000","last_became_active":"2024-09-14T09:05:17.284977+0000","last_became_peered":"2024-09-14T09:05:17.284977+0000","last_unstale":"2024-09-14T09:06:20.239510+0000","last_undegraded":"2024-09-14T09:06:20.239510+0000","last_fullsized":"2024-09-14T09:06:20.239510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:30:38.832027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"64'185","reported_seq":3390,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.209038+0000","last_change":"2024-09-14T09:05:28.839614+0000","last_active":"2024-09-14T09:06:20.209038+0000","last_peered":"2024-09-14T09:06:20.209038+0000","last_clean":"2024-09-14T09:06:20.209038+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:06:20.209038+0000","last_undegraded":"2024-09-14T09:06:20.209038+0000","last_fullsized":"2024-09-14T09:06:20.209038+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:16:03.645896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014245869999999999,"stat_sum":{"num_bytes":37571662,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":4088,"num_read_kb":37528,"num_write":192,"num_write_kb":4858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.174307+0000","last_change":"2024-09-14T09:05:19.314029+0000","last_active":"2024-09-14T09:06:20.174307+0000","last_peered":"2024-09-14T09:06:20.174307+0000","last_clean":"2024-09-14T09:06:20.174307+0000","last_became_active":"2024-09-14T09:05:17.282890+0000","last_became_peered":"2024-09-14T09:05:17.282890+0000","last_unstale":"2024-09-14T09:06:20.174307+0000","last_undegraded":"2024-09-14T09:06:20.174307+0000","last_fullsized":"2024-09-14T09:06:20.174307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:25.304105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"65'322","reported_seq":2245,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.781319+0000","last_change":"2024-09-14T09:05:29.039154+0000","last_active":"2024-09-14T09:06:20.781319+0000","last_peered":"2024-09-14T09:06:20.781319+0000","last_clean":"2024-09-14T09:06:20.781319+0000","last_became_active":"2024-09-14T09:05:26.829855+0000","last_became_peered":"2024-09-14T09:05:26.829855+0000","last_unstale":"2024-09-14T09:06:20.781319+0000","last_undegraded":"2024-09-14T09:06:20.781319+0000","last_fullsized":"2024-09-14T09:06:20.781319+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:38:56.377911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019213749999999999,"stat_sum":{"num_bytes":37149226,"num_objects":27,"num_object_clones":9,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":9,"num_read":3234,"num_read_kb":12283,"num_write":432,"num_write_kb":3583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.209066+0000","last_change":"2024-09-14T09:05:19.794165+0000","last_active":"2024-09-14T09:06:20.209066+0000","last_peered":"2024-09-14T09:06:20.209066+0000","last_clean":"2024-09-14T09:06:20.209066+0000","last_became_active":"2024-09-14T09:05:17.295864+0000","last_became_peered":"2024-09-14T09:05:17.295864+0000","last_unstale":"2024-09-14T09:06:20.209066+0000","last_undegraded":"2024-09-14T09:06:20.209066+0000","last_fullsized":"2024-09-14T09:06:20.209066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:53.901865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011399719999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"65'165","reported_seq":826,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.548986+0000","last_change":"2024-09-14T09:05:28.823608+0000","last_active":"2024-09-14T09:06:20.548986+0000","last_peered":"2024-09-14T09:06:20.548986+0000","last_clean":"2024-09-14T09:06:20.548986+0000","last_became_active":"2024-09-14T09:05:26.816468+0000","last_became_peered":"2024-09-14T09:05:26.816468+0000","last_unstale":"2024-09-14T09:06:20.548986+0000","last_undegraded":"2024-09-14T09:06:20.548986+0000","last_fullsized":"2024-09-14T09:06:20.548986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:07:55.157021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014087959999999999,"stat_sum":{"num_bytes":7350316,"num_objects":36,"num_object_clones":15,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":18,"num_read":1046,"num_read_kb":2296,"num_write":135,"num_write_kb":2499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.197079+0000","last_change":"2024-09-14T09:05:19.314129+0000","last_active":"2024-09-14T09:06:20.197079+0000","last_peered":"2024-09-14T09:06:20.197079+0000","last_clean":"2024-09-14T09:06:20.197079+0000","last_became_active":"2024-09-14T09:05:17.295196+0000","last_became_peered":"2024-09-14T09:05:17.295196+0000","last_unstale":"2024-09-14T09:06:20.197079+0000","last_undegraded":"2024-09-14T09:06:20.197079+0000","last_fullsized":"2024-09-14T09:06:20.197079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:35.269115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011613800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"65'137","reported_seq":734,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:21.658786+0000","last_change":"2024-09-14T09:05:28.835976+0000","last_active":"2024-09-14T09:06:21.658786+0000","last_peered":"2024-09-14T09:06:21.658786+0000","last_clean":"2024-09-14T09:06:21.658786+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:06:21.658786+0000","last_undegraded":"2024-09-14T09:06:21.658786+0000","last_fullsized":"2024-09-14T09:06:21.658786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:09.155440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017480779999999999,"stat_sum":{"num_bytes":48500879,"num_objects":42,"num_object_clones":16,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":21,"num_read":537,"num_read_kb":8565,"num_write":123,"num_write_kb":8452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"65'181","reported_seq":1262,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.830772+0000","last_change":"2024-09-14T09:05:39.930107+0000","last_active":"2024-09-14T09:06:20.830772+0000","last_peered":"2024-09-14T09:06:20.830772+0000","last_clean":"2024-09-14T09:06:20.830772+0000","last_became_active":"2024-09-14T09:05:39.914913+0000","last_became_peered":"2024-09-14T09:05:39.914913+0000","last_unstale":"2024-09-14T09:06:20.830772+0000","last_undegraded":"2024-09-14T09:06:20.830772+0000","last_fullsized":"2024-09-14T09:06:20.830772+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:33:30.521607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014128,"stat_sum":{"num_bytes":25020674,"num_objects":33,"num_object_clones":12,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":12,"num_read":1264,"num_read_kb":14541,"num_write":228,"num_write_kb":8481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":108,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.177961+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:06:20.177961+0000","last_peered":"2024-09-14T09:06:20.177961+0000","last_clean":"2024-09-14T09:06:20.177961+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:06:20.177961+0000","last_undegraded":"2024-09-14T09:06:20.177961+0000","last_fullsized":"2024-09-14T09:06:20.177961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"64'146","reported_seq":815,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.173998+0000","last_change":"2024-09-14T09:05:28.822791+0000","last_active":"2024-09-14T09:06:20.173998+0000","last_peered":"2024-09-14T09:06:20.173998+0000","last_clean":"2024-09-14T09:06:20.173998+0000","last_became_active":"2024-09-14T09:05:26.817109+0000","last_became_peered":"2024-09-14T09:05:26.817109+0000","last_unstale":"2024-09-14T09:06:20.173998+0000","last_undegraded":"2024-09-14T09:06:20.173998+0000","last_fullsized":"2024-09-14T09:06:20.173998+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:22:51.207908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011728439999999999,"stat_sum":{"num_bytes":12773633,"num_objects":36,"num_object_clones":12,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":16,"num_read":760,"num_read_kb":3080,"num_write":155,"num_write_kb":684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.234858+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:06:20.234858+0000","last_peered":"2024-09-14T09:06:20.234858+0000","last_clean":"2024-09-14T09:06:20.234858+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:06:20.234858+0000","last_undegraded":"2024-09-14T09:06:20.234858+0000","last_fullsized":"2024-09-14T09:06:20.234858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"65'148","reported_seq":1174,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:22.248371+0000","last_change":"2024-09-14T09:05:39.925683+0000","last_active":"2024-09-14T09:06:22.248371+0000","last_peered":"2024-09-14T09:06:22.248371+0000","last_clean":"2024-09-14T09:06:22.248371+0000","last_became_active":"2024-09-14T09:05:39.905252+0000","last_became_peered":"2024-09-14T09:05:39.905252+0000","last_unstale":"2024-09-14T09:06:22.248371+0000","last_undegraded":"2024-09-14T09:06:22.248371+0000","last_fullsized":"2024-09-14T09:06:22.248371+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:04:27.524926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074815999999999999,"stat_sum":{"num_bytes":16869691,"num_objects":28,"num_object_clones":11,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":13,"num_read":941,"num_read_kb":5577,"num_write":132,"num_write_kb":7873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2097152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.208573+0000","last_change":"2024-09-14T09:05:19.793999+0000","last_active":"2024-09-14T09:06:20.208573+0000","last_peered":"2024-09-14T09:06:20.208573+0000","last_clean":"2024-09-14T09:06:20.208573+0000","last_became_active":"2024-09-14T09:05:17.284122+0000","last_became_peered":"2024-09-14T09:05:17.284122+0000","last_unstale":"2024-09-14T09:06:20.208573+0000","last_undegraded":"2024-09-14T09:06:20.208573+0000","last_fullsized":"2024-09-14T09:06:20.208573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:12:01.133215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010234269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"65'171","reported_seq":997,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.830830+0000","last_change":"2024-09-14T09:05:29.038389+0000","last_active":"2024-09-14T09:06:20.830830+0000","last_peered":"2024-09-14T09:06:20.830830+0000","last_clean":"2024-09-14T09:06:20.830830+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:06:20.830830+0000","last_undegraded":"2024-09-14T09:06:20.830830+0000","last_fullsized":"2024-09-14T09:06:20.830830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:38:06.605562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001479076,"stat_sum":{"num_bytes":19832247,"num_objects":35,"num_object_clones":11,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":15,"num_read":1018,"num_read_kb":3518,"num_write":214,"num_write_kb":1897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":107,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.177948+0000","last_change":"2024-09-14T09:05:19.310814+0000","last_active":"2024-09-14T09:06:20.177948+0000","last_peered":"2024-09-14T09:06:20.177948+0000","last_clean":"2024-09-14T09:06:20.177948+0000","last_became_active":"2024-09-14T09:05:17.276973+0000","last_became_peered":"2024-09-14T09:05:17.276973+0000","last_unstale":"2024-09-14T09:06:20.177948+0000","last_undegraded":"2024-09-14T09:06:20.177948+0000","last_fullsized":"2024-09-14T09:06:20.177948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:33:35.187184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086884099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":197,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.208630+0000","last_change":"2024-09-14T09:05:13.239808+0000","last_active":"2024-09-14T09:06:20.208630+0000","last_peered":"2024-09-14T09:06:20.208630+0000","last_clean":"2024-09-14T09:06:20.208630+0000","last_became_active":"2024-09-14T09:05:13.239064+0000","last_became_peered":"2024-09-14T09:05:13.239064+0000","last_unstale":"2024-09-14T09:06:20.208630+0000","last_undegraded":"2024-09-14T09:06:20.208630+0000","last_fullsized":"2024-09-14T09:06:20.208630+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:40:11.559161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"65'180","reported_seq":3164,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.909491+0000","last_change":"2024-09-14T09:05:29.037923+0000","last_active":"2024-09-14T09:06:20.909491+0000","last_peered":"2024-09-14T09:06:20.909491+0000","last_clean":"2024-09-14T09:06:20.909491+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:06:20.909491+0000","last_undegraded":"2024-09-14T09:06:20.909491+0000","last_fullsized":"2024-09-14T09:06:20.909491+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:20:34.792995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099199800000000001,"stat_sum":{"num_bytes":42207305,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":12,"num_read":3447,"num_read_kb":29432,"num_write":198,"num_write_kb":7209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"65'155","reported_seq":1281,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.401218+0000","last_change":"2024-09-14T09:05:39.933291+0000","last_active":"2024-09-14T09:06:20.401218+0000","last_peered":"2024-09-14T09:06:20.401218+0000","last_clean":"2024-09-14T09:06:20.401218+0000","last_became_active":"2024-09-14T09:05:39.924025+0000","last_became_peered":"2024-09-14T09:05:39.924025+0000","last_unstale":"2024-09-14T09:06:20.401218+0000","last_undegraded":"2024-09-14T09:06:20.401218+0000","last_fullsized":"2024-09-14T09:06:20.401218+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:59:16.951348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001266028,"stat_sum":{"num_bytes":24500553,"num_objects":33,"num_object_clones":11,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":14,"num_read":1066,"num_read_kb":5932,"num_write":136,"num_write_kb":13603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"65'199","reported_seq":2835,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:21.993603+0000","last_change":"2024-09-14T09:05:39.975358+0000","last_active":"2024-09-14T09:06:21.993603+0000","last_peered":"2024-09-14T09:06:21.993603+0000","last_clean":"2024-09-14T09:06:21.993603+0000","last_became_active":"2024-09-14T09:05:39.918526+0000","last_became_peered":"2024-09-14T09:05:39.918526+0000","last_unstale":"2024-09-14T09:06:21.993603+0000","last_undegraded":"2024-09-14T09:06:21.993603+0000","last_fullsized":"2024-09-14T09:06:21.993603+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:41:17.505559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185731,"stat_sum":{"num_bytes":47434280,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":3153,"num_read_kb":30438,"num_write":229,"num_write_kb":15420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":2159421,"num_keys_recovered":3,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"64'203","reported_seq":7154,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.173154+0000","last_change":"2024-09-14T09:06:02.837388+0000","last_active":"2024-09-14T09:06:20.173154+0000","last_peered":"2024-09-14T09:06:20.173154+0000","last_clean":"2024-09-14T09:06:20.173154+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:06:20.173154+0000","last_undegraded":"2024-09-14T09:06:20.173154+0000","last_fullsized":"2024-09-14T09:06:20.173154+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:50:25.989390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001585295,"stat_sum":{"num_bytes":37804655,"num_objects":36,"num_object_clones":12,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":7359,"num_read_kb":51524,"num_write":206,"num_write_kb":6602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"65'180","reported_seq":1008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:21.075698+0000","last_change":"2024-09-14T09:05:28.834647+0000","last_active":"2024-09-14T09:06:21.075698+0000","last_peered":"2024-09-14T09:06:21.075698+0000","last_clean":"2024-09-14T09:06:21.075698+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:06:21.075698+0000","last_undegraded":"2024-09-14T09:06:21.075698+0000","last_fullsized":"2024-09-14T09:06:21.075698+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:24:45.185564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244290000000001,"stat_sum":{"num_bytes":24527389,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":1155,"num_read_kb":6632,"num_write":196,"num_write_kb":6729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"65'170","reported_seq":2491,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.527952+0000","last_change":"2024-09-14T09:05:28.851561+0000","last_active":"2024-09-14T09:06:20.527952+0000","last_peered":"2024-09-14T09:06:20.527952+0000","last_clean":"2024-09-14T09:06:20.527952+0000","last_became_active":"2024-09-14T09:05:26.843780+0000","last_became_peered":"2024-09-14T09:05:26.843780+0000","last_unstale":"2024-09-14T09:06:20.527952+0000","last_undegraded":"2024-09-14T09:06:20.527952+0000","last_fullsized":"2024-09-14T09:06:20.527952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:56:37.671792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072987200000000001,"stat_sum":{"num_bytes":22562174,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":12,"num_read":2569,"num_read_kb":18984,"num_write":205,"num_write_kb":10741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"64'137","reported_seq":1120,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.208959+0000","last_change":"2024-09-14T09:05:40.010535+0000","last_active":"2024-09-14T09:06:20.208959+0000","last_peered":"2024-09-14T09:06:20.208959+0000","last_clean":"2024-09-14T09:06:20.208959+0000","last_became_active":"2024-09-14T09:05:39.921575+0000","last_became_peered":"2024-09-14T09:05:39.921575+0000","last_unstale":"2024-09-14T09:06:20.208959+0000","last_undegraded":"2024-09-14T09:06:20.208959+0000","last_fullsized":"2024-09-14T09:06:20.208959+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:17:50.774987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057596100000000005,"stat_sum":{"num_bytes":29799109,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":877,"num_read_kb":8697,"num_write":123,"num_write_kb":4277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":111591,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"65'174","reported_seq":914,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.386557+0000","last_change":"2024-09-14T09:06:02.836145+0000","last_active":"2024-09-14T09:06:20.386557+0000","last_peered":"2024-09-14T09:06:20.386557+0000","last_clean":"2024-09-14T09:06:20.386557+0000","last_became_active":"2024-09-14T09:06:02.834684+0000","last_became_peered":"2024-09-14T09:06:02.834684+0000","last_unstale":"2024-09-14T09:06:20.386557+0000","last_undegraded":"2024-09-14T09:06:20.386557+0000","last_fullsized":"2024-09-14T09:06:20.386557+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:36:22.495359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000909463,"stat_sum":{"num_bytes":24980440,"num_objects":35,"num_object_clones":11,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":14,"num_read":1024,"num_read_kb":1780,"num_write":141,"num_write_kb":4673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":259265,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"65'192","reported_seq":3854,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:21.863763+0000","last_change":"2024-09-14T09:06:00.166357+0000","last_active":"2024-09-14T09:06:21.863763+0000","last_peered":"2024-09-14T09:06:21.863763+0000","last_clean":"2024-09-14T09:06:21.863763+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:06:21.863763+0000","last_undegraded":"2024-09-14T09:06:21.863763+0000","last_fullsized":"2024-09-14T09:06:21.863763+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:05:17.627684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011381609999999999,"stat_sum":{"num_bytes":75643935,"num_objects":36,"num_object_clones":13,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":3553,"num_read_kb":39869,"num_write":214,"num_write_kb":9316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"64'250","reported_seq":2249,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.208697+0000","last_change":"2024-09-14T09:05:40.038345+0000","last_active":"2024-09-14T09:06:20.208697+0000","last_peered":"2024-09-14T09:06:20.208697+0000","last_clean":"2024-09-14T09:06:20.208697+0000","last_became_active":"2024-09-14T09:05:39.923017+0000","last_became_peered":"2024-09-14T09:05:39.923017+0000","last_unstale":"2024-09-14T09:06:20.208697+0000","last_undegraded":"2024-09-14T09:06:20.208697+0000","last_fullsized":"2024-09-14T09:06:20.208697+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:23:17.066991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092949299999999999,"stat_sum":{"num_bytes":3326061,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":13,"num_read":3079,"num_read_kb":2999,"num_write":360,"num_write_kb":393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"65'245","reported_seq":2201,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:20.788648+0000","last_change":"2024-09-14T09:05:28.839774+0000","last_active":"2024-09-14T09:06:20.788648+0000","last_peered":"2024-09-14T09:06:20.788648+0000","last_clean":"2024-09-14T09:06:20.788648+0000","last_became_active":"2024-09-14T09:05:26.826600+0000","last_became_peered":"2024-09-14T09:05:26.826600+0000","last_unstale":"2024-09-14T09:06:20.788648+0000","last_undegraded":"2024-09-14T09:06:20.788648+0000","last_fullsized":"2024-09-14T09:06:20.788648+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:46:23.922529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018357099999999999,"stat_sum":{"num_bytes":39684167,"num_objects":37,"num_object_clones":13,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":17,"num_read":2957,"num_read_kb":2636,"num_write":277,"num_write_kb":971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"65'214","reported_seq":4043,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-14T09:06:21.021878+0000","last_change":"2024-09-14T09:05:39.938958+0000","last_active":"2024-09-14T09:06:21.021878+0000","last_peered":"2024-09-14T09:06:21.021878+0000","last_clean":"2024-09-14T09:06:21.021878+0000","last_became_active":"2024-09-14T09:05:39.921023+0000","last_became_peered":"2024-09-14T09:05:39.921023+0000","last_unstale":"2024-09-14T09:06:21.021878+0000","last_undegraded":"2024-09-14T09:06:21.021878+0000","last_fullsized":"2024-09-14T09:06:21.021878+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:36:31.235887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061978900000000004,"stat_sum":{"num_bytes":20485399,"num_objects":35,"num_object_clones":11,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":4820,"num_read_kb":25707,"num_write":191,"num_write_kb":6836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":120888,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":888725252,"num_objects":1089,"num_object_clones":367,"num_object_copies":2178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1089,"num_whiteouts":440,"num_read":86784,"num_read_kb":462291,"num_write":9528,"num_write_kb":179357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":11394597,"num_keys_recovered":32,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":26587136,"data_stored":232187388,"data_compressed":7129455,"data_compressed_allocated":18927616,"data_compressed_original":227532488,"omap_allocated":0,"internal_metadata":0},"log_size":7670,"ondisk_log_size":7670,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12424,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35452,"kb_used_data":4220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336388,"statfs":{"total":96636764160,"available":96600461312,"internally_reserved":0,"allocated":4321280,"data_stored":31651452,"data_compressed":1079377,"data_compressed_allocated":2650112,"data_compressed_original":30752664,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43716,"kb_used_data":4212,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328124,"statfs":{"total":96636764160,"available":96591998976,"internally_reserved":0,"allocated":4313088,"data_stored":33483904,"data_compressed":849121,"data_compressed_allocated":2600960,"data_compressed_original":32641707,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37988,"kb_used_data":2640,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333852,"statfs":{"total":96636764160,"available":96597864448,"internally_reserved":0,"allocated":2703360,"data_stored":16555145,"data_compressed":502235,"data_compressed_allocated":1449984,"data_compressed_original":15956547,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509461,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39376,"kb_used_data":4028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332464,"statfs":{"total":96636764160,"available":96596443136,"internally_reserved":0,"allocated":4124672,"data_stored":29132552,"data_compressed":941152,"data_compressed_allocated":2293760,"data_compressed_original":28146764,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43896,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327944,"statfs":{"total":96636764160,"available":96591814656,"internally_reserved":0,"allocated":4493312,"data_stored":32913634,"data_compressed":808786,"data_compressed_allocated":2572288,"data_compressed_original":31786879,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1732608,"data_stored":7944069,"data_compressed":354104,"data_compressed_allocated":749568,"data_compressed_original":7466771,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40704,"kb_used_data":5348,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331136,"statfs":{"total":96636764160,"available":96595083264,"internally_reserved":0,"allocated":5476352,"data_stored":45158977,"data_compressed":1188158,"data_compressed_allocated":3268608,"data_compressed_original":43885706,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40748,"kb_used_data":5404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331092,"statfs":{"total":96636764160,"available":96595038208,"internally_reserved":0,"allocated":5533696,"data_stored":39777386,"data_compressed":1433212,"data_compressed_allocated":3469312,"data_compressed_original":38656074,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3309568,"data_stored":28143387,"data_compressed":924004,"data_compressed_allocated":2244608,"data_compressed_original":27503692,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4767744,"data_stored":39187447,"data_compressed":1440932,"data_compressed_allocated":3461120,"data_compressed_original":38406218,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1966080,"data_stored":16230663,"data_compressed":502235,"data_compressed_allocated":1449984,"data_compressed_original":15956547,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4739072,"data_stored":44491747,"data_compressed":1222789,"data_compressed_allocated":3276800,"data_compressed_original":43546394,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":892928,"data_stored":7518171,"data_compressed":280986,"data_compressed_allocated":671744,"data_compressed_original":7380755,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3600384,"data_stored":33146941,"data_compressed":876103,"data_compressed_allocated":2625536,"data_compressed_original":32629419,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3612672,"data_stored":31473794,"data_compressed":1079832,"data_compressed_allocated":2662400,"data_compressed_original":30896024,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3698688,"data_stored":31995238,"data_compressed":802574,"data_compressed_allocated":2535424,"data_compressed_original":31213439,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:06:23.729 INFO:tasks.thrashosds.thrasher:Setting 2.3 to [3, 2] 2024-09-14T09:06:23.729 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.3', '3', '2'] 2024-09-14T09:06:23.730 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.3 3 2 2024-09-14T09:06:23.730 INFO:teuthology.orchestra.run.smithi176.stdout:2666 punch from 0x7d6a3da to 0x7d79866, (0xf48c bytes) 2024-09-14T09:06:23.732 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:23.735+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:23.736 INFO:teuthology.orchestra.run.smithi176.stdout:2668 write 0xe84e08e thru 0xe85bfe5 (0xdf58 bytes) 2024-09-14T09:06:23.747 INFO:teuthology.orchestra.run.smithi176.stdout:2669 write 0xb679162 thru 0xb688447 (0xf2e6 bytes) 2024-09-14T09:06:23.758 INFO:teuthology.orchestra.run.smithi176.stdout:2670 read 0x4cc67c4 thru 0x4ccc8a6 (0x60e3 bytes) 2024-09-14T09:06:23.776 INFO:teuthology.orchestra.run.smithi176.stdout:2673 trunc from 0xe85bfe6 to 0xc100e 2024-09-14T09:06:23.829 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:23.830 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:23.833 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:23.835+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:23.845 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:06:23.932 INFO:teuthology.orchestra.run.smithi176.stdout:2674 read 0xb8e26 thru 0xc100d (0x81e8 bytes) 2024-09-14T09:06:23.934 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:23.935+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:23.956 INFO:teuthology.orchestra.run.smithi176.stdout:2675 writesame 0xa911658 thru 0xa915db9 data_size 0x1(0x4762 bytes) 2024-09-14T09:06:23.957 INFO:teuthology.orchestra.run.smithi176.stdout:2677 writesame 0x8049a1b thru 0x804abde data_size 0x6(0x11c4 bytes) 2024-09-14T09:06:23.965 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:23.966 INFO:teuthology.orchestra.run.smithi176.stdout:2681 write 0x69c90ae thru 0x69d32b6 (0xa209 bytes) 2024-09-14T09:06:23.977 INFO:teuthology.orchestra.run.smithi176.stdout:2682 punch from 0x1c687cb to 0x1c6b5b2, (0x2de7 bytes) 2024-09-14T09:06:23.977 INFO:teuthology.orchestra.run.smithi176.stdout:2683 read 0x723d792 thru 0x724ced6 (0xf745 bytes) 2024-09-14T09:06:23.977 INFO:teuthology.orchestra.run.smithi176.stdout:2685 write 0x7f2d95d thru 0x7f31223 (0x38c7 bytes) 2024-09-14T09:06:23.988 INFO:teuthology.orchestra.run.smithi176.stdout:2686 read 0x81d61c7 thru 0x81db7b2 (0x55ec bytes) 2024-09-14T09:06:23.988 INFO:teuthology.orchestra.run.smithi176.stdout:2687 writesame 0x633a594 thru 0x633ffc3 data_size 0x1a(0x5a30 bytes) 2024-09-14T09:06:24.000 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:24.000 INFO:teuthology.orchestra.run.smithi176.stdout:2689 punch from 0x51cab1b to 0x51cc519, (0x19fe bytes) 2024-09-14T09:06:24.012 INFO:teuthology.orchestra.run.smithi176.stdout:2690 write 0xd2fd895 thru 0xd2fe10e (0x87a bytes) 2024-09-14T09:06:24.014 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:24.014 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:24.020 INFO:teuthology.orchestra.run.smithi176.stdout:2691 read 0x8880894 thru 0x888d221 (0xc98e bytes) 2024-09-14T09:06:24.020 INFO:teuthology.orchestra.run.smithi176.stdout:2694 write 0xabd6a74 thru 0xabde722 (0x7caf bytes) 2024-09-14T09:06:24.029 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:06:24.035 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:24.039+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:24.048 INFO:teuthology.orchestra.run.smithi176.stdout:2695 trunc from 0xd2fe10f to 0x2a67d29 2024-09-14T09:06:24.068 INFO:teuthology.orchestra.run.smithi176.stdout:2696 read 0x118e447 thru 0x119ca72 (0xe62c bytes) 2024-09-14T09:06:24.069 INFO:teuthology.orchestra.run.smithi176.stdout:2698 write 0xc727e7 thru 0xc72b8d (0x3a7 bytes) 2024-09-14T09:06:24.098 INFO:teuthology.orchestra.run.smithi176.stdout:2699 writesame 0xe4f7223 thru 0xe4f78df data_size 0xf(0x6bd bytes) 2024-09-14T09:06:24.098 INFO:teuthology.orchestra.run.smithi176.stdout:2701 write 0xdf7856c thru 0xdf87074 (0xeb09 bytes) 2024-09-14T09:06:24.115 INFO:teuthology.orchestra.run.smithi176.stdout:2702 writesame 0xc69cc0c thru 0xc6a4534 data_size 0x31(0x7929 bytes) 2024-09-14T09:06:24.127 INFO:teuthology.orchestra.run.smithi176.stdout:2704 punch from 0x773fa66 to 0x774e1fc, (0xe796 bytes) 2024-09-14T09:06:24.127 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:24.127 INFO:teuthology.orchestra.run.smithi176.stdout:2706 read 0x3b059d5 thru 0x3b069df (0x100b bytes) 2024-09-14T09:06:24.127 INFO:teuthology.orchestra.run.smithi176.stdout:2708 read 0xa4fbb48 thru 0xa5088d3 (0xcd8c bytes) 2024-09-14T09:06:24.127 INFO:teuthology.orchestra.run.smithi176.stdout:2709 read 0x1ee6ed7 thru 0x1ef24f6 (0xb620 bytes) 2024-09-14T09:06:24.127 INFO:teuthology.orchestra.run.smithi176.stdout:2710 read 0x44047a4 thru 0x4408f99 (0x47f6 bytes) 2024-09-14T09:06:24.127 INFO:teuthology.orchestra.run.smithi176.stdout:2711 writesame 0x15f5c0d thru 0x16002eb data_size 0x1(0xa6df bytes) 2024-09-14T09:06:24.137 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:24.139+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:24.177 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:24.177 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:24.191 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:06:24.195 INFO:teuthology.orchestra.run.smithi176.stderr:set 2.3 pg_upmap mapping to [3,2] 2024-09-14T09:06:24.213 INFO:teuthology.orchestra.run.smithi176.stdout:2712 write 0x36b2437 thru 0x36bb343 (0x8f0d bytes) 2024-09-14T09:06:24.228 INFO:teuthology.orchestra.run.smithi176.stdout:2713 read 0x5bf96e0 thru 0x5c014af (0x7dd0 bytes) 2024-09-14T09:06:24.228 INFO:teuthology.orchestra.run.smithi176.stdout:2714 read 0x5d27c38 thru 0x5d2e002 (0x63cb bytes) 2024-09-14T09:06:24.228 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:24.228 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:24.228 INFO:teuthology.orchestra.run.smithi176.stdout:2721 write 0xba04a37 thru 0xba0f6b1 (0xac7b bytes) 2024-09-14T09:06:24.237 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:24.235+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:24.239 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:24.239 INFO:teuthology.orchestra.run.smithi176.stdout:2724 read 0x5c3c769 thru 0x5c4bfc8 (0xf860 bytes) 2024-09-14T09:06:24.239 INFO:teuthology.orchestra.run.smithi176.stdout:2725 write 0x7c9d30f thru 0x7ca7d94 (0xaa86 bytes) 2024-09-14T09:06:24.251 INFO:teuthology.orchestra.run.smithi176.stdout:2727 writesame 0x8362387 thru 0x836c53a data_size 0x4f(0xa1b4 bytes) 2024-09-14T09:06:24.263 INFO:teuthology.orchestra.run.smithi176.stdout:2728 read 0x6b52cae thru 0x6b5fe12 (0xd165 bytes) 2024-09-14T09:06:24.263 INFO:teuthology.orchestra.run.smithi176.stdout:2729 punch from 0x59826a0 to 0x5985df2, (0x3752 bytes) 2024-09-14T09:06:24.263 INFO:teuthology.orchestra.run.smithi176.stdout:2730 read 0xb5c98db thru 0xb5d378a (0x9eb0 bytes) 2024-09-14T09:06:24.263 INFO:teuthology.orchestra.run.smithi176.stdout:2731 punch from 0x9d6cef0 to 0x9d79604, (0xc714 bytes) 2024-09-14T09:06:24.263 INFO:teuthology.orchestra.run.smithi176.stdout:2733 punch from 0x81cf0ab to 0x81dcddb, (0xdd30 bytes) 2024-09-14T09:06:24.270 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:06:24.278 INFO:teuthology.orchestra.run.smithi176.stdout:2735 trunc from 0xe4f78e0 to 0x24f8a3a 2024-09-14T09:06:24.318 INFO:teuthology.orchestra.run.smithi176.stdout:2736 write 0xaadb29e thru 0xaae626f (0xafd2 bytes) 2024-09-14T09:06:24.327 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:24.328 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:24.328 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:24.328 INFO:teuthology.orchestra.run.smithi176.stdout:2738 read 0x291ed25 thru 0x2927ab6 (0x8d92 bytes) 2024-09-14T09:06:24.329 INFO:teuthology.orchestra.run.smithi176.stdout:2739 writesame 0x34d972 thru 0x3530f8 data_size 0x4d(0x5787 bytes) 2024-09-14T09:06:24.341 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:24.339+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:24.341 INFO:teuthology.orchestra.run.smithi176.stdout:2741 writesame 0x14ef0c thru 0x15d079 data_size 0x3a(0xe16e bytes) 2024-09-14T09:06:24.343 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:06:24.355 INFO:teuthology.orchestra.run.smithi176.stdout:2742 writesame 0xb4e9ca3 thru 0xb4ee751 data_size 0x1(0x4aaf bytes) 2024-09-14T09:06:24.355 INFO:teuthology.orchestra.run.smithi176.stdout:2743 write 0xdc55b8 thru 0xdca1d2 (0x4c1b bytes) 2024-09-14T09:06:24.366 INFO:teuthology.orchestra.run.smithi176.stdout:2744 writesame 0x7bfd245 thru 0x7c0918f data_size 0x1(0xbf4b bytes) 2024-09-14T09:06:24.420 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:24.420 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:24.438 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:24.438 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:24.441 INFO:teuthology.orchestra.run.smithi176.stdout:2745 read 0x56209fb thru 0x562ff12 (0xf518 bytes) 2024-09-14T09:06:24.441 INFO:teuthology.orchestra.run.smithi176.stdout:2747 read 0x2b38b08 thru 0x2b47e94 (0xf38d bytes) 2024-09-14T09:06:24.441 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:24.439+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:24.451 INFO:teuthology.orchestra.run.smithi176.stdout:2749 write 0xe29ed2b thru 0xe2a5d80 (0x7056 bytes) 2024-09-14T09:06:24.456 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:24.456 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:24.474 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:24.474 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:24.477 INFO:teuthology.orchestra.run.smithi176.stdout:2750 trunc from 0xe2a5d81 to 0x542950f 2024-09-14T09:06:24.481 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:24.481 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:24.481 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:24.481 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:24.481 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:24.493 INFO:teuthology.orchestra.run.smithi176.stdout:2751 punch from 0xf950c to 0x108d06, (0xf7fa bytes) 2024-09-14T09:06:24.494 INFO:teuthology.orchestra.run.smithi176.stdout:2752 writesame 0x1f1c29e thru 0x1f28d1f data_size 0x62(0xca82 bytes) 2024-09-14T09:06:24.497 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:06:24.497 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:24.497 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:24.506 INFO:teuthology.orchestra.run.smithi176.stdout:2753 writesame 0xfbb597 thru 0xfc5e8c data_size 0x36(0xa8f6 bytes) 2024-09-14T09:06:24.515 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:24.516 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:24.520 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:24.520 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:24.532 INFO:teuthology.orchestra.run.smithi176.stdout:2757 write 0xaf4f9da thru 0xaf545d3 (0x4bfa bytes) 2024-09-14T09:06:24.545 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:24.543+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:24.545 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:24.546 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:24.546 INFO:teuthology.orchestra.run.smithi176.stdout:2758 write 0x68742a3 thru 0x687fa10 (0xb76e bytes) 2024-09-14T09:06:24.557 INFO:teuthology.orchestra.run.smithi176.stdout:2759 read 0x9f19b6 thru 0x9f49c0 (0x300b bytes) 2024-09-14T09:06:24.557 INFO:teuthology.orchestra.run.smithi176.stdout:2762 writesame 0x1190dde thru 0x1199518 data_size 0x1(0x873b bytes) 2024-09-14T09:06:24.566 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:24.567 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:24.624 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:24.624 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:24.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:24.624 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:24.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.624 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:24.625 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:24.625 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.54054361, 2024-09-14T09:06:24.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:06:24.625 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.625 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.625 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:24.625 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:24.625 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:24.625 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:24.625 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:06:24.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.625 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:24.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:24.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:06:24.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:06:24.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:06:24.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:06:24.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:06:24.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:06:24.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:06:24.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:06:24.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:06:24.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:06:24.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:24.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:06:24.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:06:24.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:06:24.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:06:24.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:06:24.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:06:24.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:06:24.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:06:24.629 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.630 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.630 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.630 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:24.630 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:24.630 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.540488951999997, 2024-09-14T09:06:24.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:06:24.630 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.630 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.630 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:24.630 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:24.630 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:24.630 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:24.631 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:06:24.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.631 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:24.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:24.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:24.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:06:24.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:06:24.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:06:24.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:06:24.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:06:24.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:06:24.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:06:24.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:06:24.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:06:24.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:24.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:06:24.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:06:24.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:06:24.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:06:24.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:06:24.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:06:24.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:06:24.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:06:24.635 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.635 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.635 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.635 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:24.635 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:24.635 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.540383577999997, 2024-09-14T09:06:24.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:06:24.636 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.636 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.636 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:24.636 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:24.636 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:24.636 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:24.636 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:06:24.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.636 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:24.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:24.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:06:24.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:06:24.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:06:24.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:06:24.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:06:24.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:06:24.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:06:24.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:06:24.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:06:24.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:06:24.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:24.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:06:24.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:06:24.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:06:24.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:06:24.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:06:24.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:06:24.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:06:24.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:06:24.640 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.640 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.640 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.640 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:24.640 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:24.641 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.540026814000001, 2024-09-14T09:06:24.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:06:24.641 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.641 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.641 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:24.641 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:24.641 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:24.641 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:24.641 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:06:24.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.641 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:24.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:24.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:06:24.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:06:24.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:06:24.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:06:24.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:06:24.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:06:24.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:06:24.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:06:24.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:06:24.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:06:24.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:24.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:06:24.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:06:24.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:06:24.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:06:24.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:06:24.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:06:24.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:06:24.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:06:24.645 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.645 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.645 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.646 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:24.646 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:24.646 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.539942076000003, 2024-09-14T09:06:24.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:06:24.646 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.646 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.646 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:24.646 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:24.646 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:24.646 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:24.646 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:06:24.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.646 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:24.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:24.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:06:24.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:06:24.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:06:24.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:06:24.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:06:24.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:06:24.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:06:24.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:06:24.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:06:24.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:06:24.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:24.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:06:24.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:06:24.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:06:24.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:06:24.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:06:24.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:06:24.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:06:24.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:06:24.650 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.650 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.650 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.651 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:24.651 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:24.651 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.539854390000002, 2024-09-14T09:06:24.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:06:24.651 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.651 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.651 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:24.651 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:24.651 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:24.651 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:24.651 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:06:24.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.651 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:24.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:24.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:06:24.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:06:24.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:06:24.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:06:24.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:06:24.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:06:24.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:06:24.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:06:24.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:06:24.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:06:24.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:24.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:06:24.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:06:24.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:06:24.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:06:24.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:06:24.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:06:24.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:06:24.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:06:24.655 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.655 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.655 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.656 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18388 3.18 3:19e2be55:::rbd_data.115e3abd3b2c.0000000000000021:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:06:24.656 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:24.656 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.403941078999999, 2024-09-14T09:06:24.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042293648000000003, 2024-09-14T09:06:24.656 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.656 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.656 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:24.656 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:24.656 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:24.656 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:24.656 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18388 2024-09-14T09:06:24.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.656 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:24.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:24.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220236+0000", 2024-09-14T09:06:24.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1339999999999998e-06 2024-09-14T09:06:24.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220244+0000", 2024-09-14T09:06:24.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1950000000000003e-06 2024-09-14T09:06:24.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220262+0000", 2024-09-14T09:06:24.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8017e-05 2024-09-14T09:06:24.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220270+0000", 2024-09-14T09:06:24.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9480000000000008e-06 2024-09-14T09:06:24.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.659 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220357+0000", 2024-09-14T09:06:24.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7185e-05 2024-09-14T09:06:24.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.659 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220576+0000", 2024-09-14T09:06:24.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021912200000000001 2024-09-14T09:06:24.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:24.659 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.222435+0000", 2024-09-14T09:06:24.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018592350000000001 2024-09-14T09:06:24.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:24.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262439+0000", 2024-09-14T09:06:24.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040003368999999997 2024-09-14T09:06:24.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:24.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262452+0000", 2024-09-14T09:06:24.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2941e-05 2024-09-14T09:06:24.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:24.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262490+0000", 2024-09-14T09:06:24.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8478e-05 2024-09-14T09:06:24.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262527+0000", 2024-09-14T09:06:24.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7024000000000003e-05 2024-09-14T09:06:24.661 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.661 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.661 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.662 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:19543 3.5 e39/23 3:a536ff9b:::rbd_data.115e4c71d6e8.0000000000000002:head v 39'8, mlcod=39'6)", 2024-09-14T09:06:24.662 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:24.662 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 40.101086418999998, 2024-09-14T09:06:24.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.039280019999999999, 2024-09-14T09:06:24.662 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.662 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.662 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:24.662 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:24.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:24.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523090+0000", 2024-09-14T09:06:24.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.773e-06 2024-09-14T09:06:24.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537290+0000", 2024-09-14T09:06:24.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014200007000000001 2024-09-14T09:06:24.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537317+0000", 2024-09-14T09:06:24.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6704999999999999e-05 2024-09-14T09:06:24.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537328+0000", 2024-09-14T09:06:24.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1299e-05 2024-09-14T09:06:24.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537417+0000", 2024-09-14T09:06:24.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8869999999999994e-05 2024-09-14T09:06:24.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537445+0000", 2024-09-14T09:06:24.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8377999999999998e-05 2024-09-14T09:06:24.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:24.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562324+0000", 2024-09-14T09:06:24.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024879307999999999 2024-09-14T09:06:24.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562368+0000", 2024-09-14T09:06:24.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3680000000000002e-05 2024-09-14T09:06:24.666 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.666 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.666 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.666 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38002 3.1e 3:7df715aa:::rbd_data.115eb07a8abd.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:24.667 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:24.667 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.234423173, 2024-09-14T09:06:24.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.065148304000000004, 2024-09-14T09:06:24.667 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.667 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.667 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:24.667 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:24.667 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:24.667 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:24.667 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38002 2024-09-14T09:06:24.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.667 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:24.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:24.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389752+0000", 2024-09-14T09:06:24.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1829999999999999e-06 2024-09-14T09:06:24.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389758+0000", 2024-09-14T09:06:24.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0919999999999997e-06 2024-09-14T09:06:24.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389769+0000", 2024-09-14T09:06:24.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759e-05 2024-09-14T09:06:24.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389775+0000", 2024-09-14T09:06:24.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8819999999999998e-06 2024-09-14T09:06:24.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389824+0000", 2024-09-14T09:06:24.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8573999999999999e-05 2024-09-14T09:06:24.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389876+0000", 2024-09-14T09:06:24.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2643999999999999e-05 2024-09-14T09:06:24.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:06:24.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.392640+0000", 2024-09-14T09:06:24.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0027636409999999998 2024-09-14T09:06:24.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:24.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454672+0000", 2024-09-14T09:06:24.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062031703000000001 2024-09-14T09:06:24.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:24.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454765+0000", 2024-09-14T09:06:24.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2862000000000006e-05 2024-09-14T09:06:24.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:24.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454841+0000", 2024-09-14T09:06:24.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5857000000000002e-05 2024-09-14T09:06:24.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454900+0000", 2024-09-14T09:06:24.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9107000000000002e-05 2024-09-14T09:06:24.672 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.672 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.672 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.672 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:38002 3.1e e48/23 ondisk, result = 0)", 2024-09-14T09:06:24.672 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:24.672 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.224106584000001, 2024-09-14T09:06:24.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054841741999999999, 2024-09-14T09:06:24.673 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.673 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.673 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:24.673 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:24.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:24.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400069+0000", 2024-09-14T09:06:24.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.339e-06 2024-09-14T09:06:24.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400076+0000", 2024-09-14T09:06:24.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7800000000000003e-06 2024-09-14T09:06:24.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400093+0000", 2024-09-14T09:06:24.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6915999999999999e-05 2024-09-14T09:06:24.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400106+0000", 2024-09-14T09:06:24.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2785e-05 2024-09-14T09:06:24.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454713+0000", 2024-09-14T09:06:24.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054607041000000002 2024-09-14T09:06:24.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454724+0000", 2024-09-14T09:06:24.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1202e-05 2024-09-14T09:06:24.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454910+0000", 2024-09-14T09:06:24.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018567900000000001 2024-09-14T09:06:24.676 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.676 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.676 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.676 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38100 3.b 3:d0f67c24:::rbd_data.115eb07a8abd.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:24.676 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:24.676 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.978252924, 2024-09-14T09:06:24.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053701964999999997, 2024-09-14T09:06:24.676 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.677 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.677 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:24.677 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:24.677 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:24.677 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:24.677 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38100 2024-09-14T09:06:24.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.677 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:24.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:24.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645923+0000", 2024-09-14T09:06:24.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:06:24.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645928+0000", 2024-09-14T09:06:24.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0699999999999997e-06 2024-09-14T09:06:24.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645942+0000", 2024-09-14T09:06:24.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4069e-05 2024-09-14T09:06:24.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645951+0000", 2024-09-14T09:06:24.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3230000000000004e-06 2024-09-14T09:06:24.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646008+0000", 2024-09-14T09:06:24.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6583999999999999e-05 2024-09-14T09:06:24.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646089+0000", 2024-09-14T09:06:24.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1557999999999994e-05 2024-09-14T09:06:24.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:24.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.650355+0000", 2024-09-14T09:06:24.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004265525 2024-09-14T09:06:24.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:24.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699439+0000", 2024-09-14T09:06:24.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049083904999999997 2024-09-14T09:06:24.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:24.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699467+0000", 2024-09-14T09:06:24.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8198999999999999e-05 2024-09-14T09:06:24.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:24.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699552+0000", 2024-09-14T09:06:24.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T09:06:24.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699624+0000", 2024-09-14T09:06:24.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1725999999999997e-05 2024-09-14T09:06:24.682 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.682 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.682 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.682 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:06:24.682 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:24.682 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.463562403000001, 2024-09-14T09:06:24.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:06:24.682 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.682 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.682 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:24.682 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:24.683 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:24.683 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:24.683 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:06:24.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.683 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:24.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:24.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:06:24.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:06:24.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:06:24.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:06:24.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:06:24.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:06:24.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:06:24.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:06:24.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:06:24.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:06:24.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:06:24.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:06:24.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:06:24.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:06:24.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:06:24.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:24.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:06:24.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:06:24.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:24.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:06:24.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:06:24.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:24.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:06:24.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:06:24.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:06:24.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:06:24.687 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.687 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.688 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.688 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:39401 3.b e56/53)", 2024-09-14T09:06:24.688 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:24.688 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.437666337, 2024-09-14T09:06:24.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10443659600000001, 2024-09-14T09:06:24.688 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.688 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.688 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:24.688 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:24.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:24.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186509+0000", 2024-09-14T09:06:24.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2640000000000001e-06 2024-09-14T09:06:24.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186516+0000", 2024-09-14T09:06:24.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1070000000000001e-06 2024-09-14T09:06:24.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186533+0000", 2024-09-14T09:06:24.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7818000000000001e-05 2024-09-14T09:06:24.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186543+0000", 2024-09-14T09:06:24.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5170000000000005e-06 2024-09-14T09:06:24.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290885+0000", 2024-09-14T09:06:24.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.104342221 2024-09-14T09:06:24.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290893+0000", 2024-09-14T09:06:24.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2859999999999999e-06 2024-09-14T09:06:24.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290945+0000", 2024-09-14T09:06:24.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1382999999999999e-05 2024-09-14T09:06:24.691 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.691 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.692 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.692 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49708 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:24.692 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526878+0000", 2024-09-14T09:06:24.692 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.097296287000001, 2024-09-14T09:06:24.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036535277999999997, 2024-09-14T09:06:24.692 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.692 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.692 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:24.692 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:24.692 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:24.692 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:24.692 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49708 2024-09-14T09:06:24.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.693 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526878+0000", 2024-09-14T09:06:24.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526878+0000", 2024-09-14T09:06:24.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526879+0000", 2024-09-14T09:06:24.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.13e-07 2024-09-14T09:06:24.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526882+0000", 2024-09-14T09:06:24.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7480000000000002e-06 2024-09-14T09:06:24.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526889+0000", 2024-09-14T09:06:24.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4270000000000002e-06 2024-09-14T09:06:24.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526894+0000", 2024-09-14T09:06:24.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9599999999999999e-06 2024-09-14T09:06:24.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563237+0000", 2024-09-14T09:06:24.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036342646999999999 2024-09-14T09:06:24.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563314+0000", 2024-09-14T09:06:24.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7511999999999999e-05 2024-09-14T09:06:24.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563413+0000", 2024-09-14T09:06:24.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9370999999999994e-05 2024-09-14T09:06:24.696 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.696 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.696 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.696 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49709 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:24.696 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526902+0000", 2024-09-14T09:06:24.696 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.097272029000001, 2024-09-14T09:06:24.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036648100000000003, 2024-09-14T09:06:24.697 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.697 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.697 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:24.697 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:24.697 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:24.697 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:24.697 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49709 2024-09-14T09:06:24.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.697 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526902+0000", 2024-09-14T09:06:24.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526902+0000", 2024-09-14T09:06:24.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526903+0000", 2024-09-14T09:06:24.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3500000000000002e-07 2024-09-14T09:06:24.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526908+0000", 2024-09-14T09:06:24.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4179999999999996e-06 2024-09-14T09:06:24.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526914+0000", 2024-09-14T09:06:24.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.981e-06 2024-09-14T09:06:24.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526918+0000", 2024-09-14T09:06:24.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0770000000000001e-06 2024-09-14T09:06:24.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563445+0000", 2024-09-14T09:06:24.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036526378999999998 2024-09-14T09:06:24.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563498+0000", 2024-09-14T09:06:24.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3190999999999999e-05 2024-09-14T09:06:24.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563551+0000", 2024-09-14T09:06:24.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2719000000000001e-05 2024-09-14T09:06:24.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.700 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.701 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49710 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:24.701 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526924+0000", 2024-09-14T09:06:24.701 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.097250174999999, 2024-09-14T09:06:24.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037006095000000003, 2024-09-14T09:06:24.701 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.701 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.701 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:24.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:24.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:24.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:24.701 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49710 2024-09-14T09:06:24.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.702 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526924+0000", 2024-09-14T09:06:24.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526924+0000", 2024-09-14T09:06:24.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526925+0000", 2024-09-14T09:06:24.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3599999999999999e-07 2024-09-14T09:06:24.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526926+0000", 2024-09-14T09:06:24.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.795e-06 2024-09-14T09:06:24.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526932+0000", 2024-09-14T09:06:24.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0839999999999999e-06 2024-09-14T09:06:24.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526936+0000", 2024-09-14T09:06:24.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3440000000000003e-06 2024-09-14T09:06:24.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563593+0000", 2024-09-14T09:06:24.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036657041000000001 2024-09-14T09:06:24.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563648+0000", 2024-09-14T09:06:24.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5152999999999999e-05 2024-09-14T09:06:24.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563930+0000", 2024-09-14T09:06:24.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028234199999999999 2024-09-14T09:06:24.705 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.705 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.705 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.706 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49711 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:24.706 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:24.706 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.097232655999999, 2024-09-14T09:06:24.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037330945999999997, 2024-09-14T09:06:24.706 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.706 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.706 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:24.706 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:24.706 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:24.706 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:24.706 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49711 2024-09-14T09:06:24.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.706 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:24.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:24.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:24.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1800000000000002e-07 2024-09-14T09:06:24.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526944+0000", 2024-09-14T09:06:24.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.821e-06 2024-09-14T09:06:24.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526950+0000", 2024-09-14T09:06:24.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5600000000000001e-06 2024-09-14T09:06:24.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526954+0000", 2024-09-14T09:06:24.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.437e-06 2024-09-14T09:06:24.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563965+0000", 2024-09-14T09:06:24.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037010534999999997 2024-09-14T09:06:24.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564021+0000", 2024-09-14T09:06:24.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6728000000000003e-05 2024-09-14T09:06:24.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564273+0000", 2024-09-14T09:06:24.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025154700000000002 2024-09-14T09:06:24.710 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.710 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.710 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.710 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49712 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 3112960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:24.710 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:24.710 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.097214448000001, 2024-09-14T09:06:24.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037795625999999999, 2024-09-14T09:06:24.710 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.710 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.711 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:24.711 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:24.711 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:24.711 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:24.711 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49712 2024-09-14T09:06:24.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.711 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:24.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:24.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:24.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4799999999999999e-07 2024-09-14T09:06:24.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526962+0000", 2024-09-14T09:06:24.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8449999999999999e-06 2024-09-14T09:06:24.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526967+0000", 2024-09-14T09:06:24.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1059999999999999e-06 2024-09-14T09:06:24.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526971+0000", 2024-09-14T09:06:24.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8310000000000003e-06 2024-09-14T09:06:24.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564324+0000", 2024-09-14T09:06:24.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037352926000000002 2024-09-14T09:06:24.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564411+0000", 2024-09-14T09:06:24.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7052000000000006e-05 2024-09-14T09:06:24.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564756+0000", 2024-09-14T09:06:24.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034451799999999998 2024-09-14T09:06:24.714 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.714 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.715 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.715 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49713 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 3506176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:24.715 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:24.715 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.097197776, 2024-09-14T09:06:24.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038135466, 2024-09-14T09:06:24.715 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.715 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.715 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:24.715 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:24.715 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:24.715 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:24.715 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49713 2024-09-14T09:06:24.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.716 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:24.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:24.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:24.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1899999999999998e-07 2024-09-14T09:06:24.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526979+0000", 2024-09-14T09:06:24.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8959999999999999e-06 2024-09-14T09:06:24.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526986+0000", 2024-09-14T09:06:24.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1720000000000001e-06 2024-09-14T09:06:24.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526991+0000", 2024-09-14T09:06:24.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.729e-06 2024-09-14T09:06:24.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564781+0000", 2024-09-14T09:06:24.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037790345000000003 2024-09-14T09:06:24.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564844+0000", 2024-09-14T09:06:24.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2656000000000005e-05 2024-09-14T09:06:24.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565112+0000", 2024-09-14T09:06:24.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026834899999999997 2024-09-14T09:06:24.719 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.719 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.719 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.719 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49714 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:24.719 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526997+0000", 2024-09-14T09:06:24.719 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.097177167, 2024-09-14T09:06:24.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038531469999999998, 2024-09-14T09:06:24.719 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:24.719 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:24.719 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:24.719 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:24.720 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:24.720 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:24.720 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49714 2024-09-14T09:06:24.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.720 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:24.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:24.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526997+0000", 2024-09-14T09:06:24.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:24.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526997+0000", 2024-09-14T09:06:24.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:24.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:24.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526998+0000", 2024-09-14T09:06:24.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7800000000000002e-07 2024-09-14T09:06:24.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:24.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.527000+0000", 2024-09-14T09:06:24.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9080000000000002e-06 2024-09-14T09:06:24.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:24.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.527007+0000", 2024-09-14T09:06:24.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1609999999999997e-06 2024-09-14T09:06:24.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:24.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.527011+0000", 2024-09-14T09:06:24.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0929999999999996e-06 2024-09-14T09:06:24.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:24.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565152+0000", 2024-09-14T09:06:24.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038141520999999998 2024-09-14T09:06:24.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:24.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565236+0000", 2024-09-14T09:06:24.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3498999999999995e-05 2024-09-14T09:06:24.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:24.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:24.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:24.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565529+0000", 2024-09-14T09:06:24.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029291000000000001 2024-09-14T09:06:24.723 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.723 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.723 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.723 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:24.723 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:24.723 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:24.724 INFO:teuthology.orchestra.run.smithi176.stdout:2763 write 0x40a1199 thru 0x40a47b5 (0x361d bytes) 2024-09-14T09:06:24.724 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:24.724 INFO:teuthology.orchestra.run.smithi176.stdout:2765 writesame 0x842319b thru 0x84254f5 data_size 0x7(0x235b bytes) 2024-09-14T09:06:24.725 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:06:24.727 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:24.727 INFO:teuthology.orchestra.run.smithi176.stdout:2767 punch from 0xa79af44 to 0xa7a9b38, (0xebf4 bytes) 2024-09-14T09:06:24.727 INFO:teuthology.orchestra.run.smithi176.stdout:2769 write 0x313792e thru 0x313d8ae (0x5f81 bytes) 2024-09-14T09:06:24.727 INFO:teuthology.orchestra.run.smithi176.stdout:2770 read 0x80c8dd1 thru 0x80d259d (0x97cd bytes) 2024-09-14T09:06:24.727 INFO:teuthology.orchestra.run.smithi176.stdout:2772 punch from 0x2ee59bd to 0x2eea0aa, (0x46ed bytes) 2024-09-14T09:06:24.728 INFO:teuthology.orchestra.run.smithi176.stdout:2774 write 0x94499b7 thru 0x9458c13 (0xf25d bytes) 2024-09-14T09:06:24.728 INFO:teuthology.orchestra.run.smithi176.stdout:2775 punch from 0x90d332e to 0x90ded3b, (0xba0d bytes) 2024-09-14T09:06:24.728 INFO:teuthology.orchestra.run.smithi176.stdout:2776 read 0x508f8c8 thru 0x509b12b (0xb864 bytes) 2024-09-14T09:06:24.728 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:24.728 INFO:teuthology.orchestra.run.smithi176.stdout:2780 write 0x1249135 thru 0x1249cd3 (0xb9f bytes) 2024-09-14T09:06:24.728 INFO:teuthology.orchestra.run.smithi176.stdout:2781 read 0x89de24a thru 0x89ec560 (0xe317 bytes) 2024-09-14T09:06:24.728 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:24.728 INFO:teuthology.orchestra.run.smithi176.stdout:2784 read 0x25cc469 thru 0x25ce447 (0x1fdf bytes) 2024-09-14T09:06:24.728 INFO:teuthology.orchestra.run.smithi176.stdout:2785 trunc from 0xaf545d4 to 0xeb896e3 2024-09-14T09:06:24.728 INFO:teuthology.orchestra.run.smithi176.stdout:2786 read 0x7bbb346 thru 0x7bcaa2c (0xf6e7 bytes) 2024-09-14T09:06:24.728 INFO:teuthology.orchestra.run.smithi176.stdout:2787 writesame 0xb31f508 thru 0xb3213ba data_size 0x1(0x1eb3 bytes) 2024-09-14T09:06:24.728 INFO:teuthology.orchestra.run.smithi176.stdout:2788 write 0x26dc563 thru 0x26e9211 (0xccaf bytes) 2024-09-14T09:06:24.729 INFO:teuthology.orchestra.run.smithi176.stdout:2789 write 0xda0f466 thru 0xda1bf48 (0xcae3 bytes) 2024-09-14T09:06:24.729 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:24.723+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:24.735 INFO:teuthology.orchestra.run.smithi176.stdout:2790 trunc from 0xeb896e3 to 0x31c82ed 2024-09-14T09:06:24.778 INFO:teuthology.orchestra.run.smithi176.stdout:2791 write 0xbdae509 thru 0xbdbc422 (0xdf1a bytes) 2024-09-14T09:06:24.788 INFO:teuthology.orchestra.run.smithi176.stdout:2792 read 0x228acc0 thru 0x228b22a (0x56b bytes) 2024-09-14T09:06:24.788 INFO:teuthology.orchestra.run.smithi176.stdout:2793 read 0xac9b6de thru 0xaca16fc (0x601f bytes) 2024-09-14T09:06:24.788 INFO:teuthology.orchestra.run.smithi176.stdout:2794 writesame 0xa18faf8 thru 0xa191703 data_size 0x2(0x1c0c bytes) 2024-09-14T09:06:24.798 INFO:teuthology.orchestra.run.smithi176.stdout:2796 writesame 0x75d371a thru 0x75df0fa data_size 0x5(0xb9e1 bytes) 2024-09-14T09:06:24.808 INFO:teuthology.orchestra.run.smithi176.stdout:2797 write 0x1daec31 thru 0x1db6d76 (0x8146 bytes) 2024-09-14T09:06:24.817 INFO:teuthology.orchestra.run.smithi176.stdout:2798 write 0x43e000b thru 0x43e4e4c (0x4e42 bytes) 2024-09-14T09:06:24.826 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:24.827+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:24.828 INFO:teuthology.orchestra.run.smithi176.stdout:2799 read 0xaeb306e thru 0xaeb9ee2 (0x6e75 bytes) 2024-09-14T09:06:24.828 INFO:teuthology.orchestra.run.smithi176.stdout:2804 writesame 0x49e8f1 thru 0x4a2875 data_size 0x17(0x3f85 bytes) 2024-09-14T09:06:24.840 INFO:teuthology.orchestra.run.smithi176.stdout:2806 writesame 0x509b31a thru 0x50a8579 data_size 0x90(0xd260 bytes) 2024-09-14T09:06:24.862 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:24.862 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:24.862 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:24.862 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:24.865 INFO:teuthology.orchestra.run.smithi176.stdout:2807 trunc from 0xbdbc423 to 0x593c49d 2024-09-14T09:06:24.878 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:06:24.889 INFO:teuthology.orchestra.run.smithi176.stdout:2808 punch from 0x1f6466 to 0x1f833c, (0x1ed6 bytes) 2024-09-14T09:06:24.889 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:24.889 INFO:teuthology.orchestra.run.smithi176.stdout:2813 write 0x218157d thru 0x218271b (0x119f bytes) 2024-09-14T09:06:24.900 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:24.900 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:24.930 INFO:teuthology.orchestra.run.smithi176.stdout:2816 write 0xbbc47e9 thru 0xbbd07e0 (0xbff8 bytes) 2024-09-14T09:06:24.939 INFO:teuthology.orchestra.run.smithi176.stdout:2817 write 0xaec1352 thru 0xaec4a92 (0x3741 bytes) 2024-09-14T09:06:24.948 INFO:teuthology.orchestra.run.smithi176.stdout:2818 read 0x39ac2ce thru 0x39bbed2 (0xfc05 bytes) 2024-09-14T09:06:24.949 INFO:teuthology.orchestra.run.smithi176.stdout:2824 write 0x6906697 thru 0x6912d9b (0xc705 bytes) 2024-09-14T09:06:24.959 INFO:teuthology.orchestra.run.smithi176.stdout:2825 read 0x196f190 thru 0x1971643 (0x24b4 bytes) 2024-09-14T09:06:24.959 INFO:teuthology.orchestra.run.smithi176.stdout:2826 write 0x35d8c3b thru 0x35de05a (0x5420 bytes) 2024-09-14T09:06:24.963 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:24.967+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:24.968 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:24.968 INFO:teuthology.orchestra.run.smithi176.stdout:2828 read 0x8765e35 thru 0x877569b (0xf867 bytes) 2024-09-14T09:06:24.969 INFO:teuthology.orchestra.run.smithi176.stdout:2829 read 0x3d83320 thru 0x3d8d341 (0xa022 bytes) 2024-09-14T09:06:24.969 INFO:teuthology.orchestra.run.smithi176.stdout:2830 write 0x48c9f4 thru 0x491db5 (0x53c2 bytes) 2024-09-14T09:06:24.990 INFO:teuthology.orchestra.run.smithi176.stdout:2831 write 0xdd35a4d thru 0xdd3ce18 (0x73cc bytes) 2024-09-14T09:06:25.004 INFO:teuthology.orchestra.run.smithi176.stdout:2835 writesame 0x3a294b7 thru 0x3a2c118 data_size 0x26(0x2c62 bytes) 2024-09-14T09:06:25.014 INFO:teuthology.orchestra.run.smithi176.stdout:2836 write 0x63cd06a thru 0x63d26c7 (0x565e bytes) 2024-09-14T09:06:25.023 INFO:teuthology.orchestra.run.smithi176.stdout:2837 read 0xb4c15cf thru 0xb4cdef6 (0xc928 bytes) 2024-09-14T09:06:25.027 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:25.031+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:25.047 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:25.047 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:25.047 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:25.047 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:25.047 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:25.054 INFO:teuthology.orchestra.run.smithi176.stdout:2838 trunc from 0xdd3ce19 to 0x2471643 2024-09-14T09:06:25.063 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:06:25.108 INFO:teuthology.orchestra.run.smithi176.stdout:2839 write 0x45b4d09 thru 0x45c412a (0xf422 bytes) 2024-09-14T09:06:25.129 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:25.127+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:25.143 INFO:teuthology.orchestra.run.smithi176.stdout:2840 trunc from 0x45c412b to 0xd766808 2024-09-14T09:06:25.155 INFO:teuthology.orchestra.run.smithi176.stdout:2841 read 0x7140d66 thru 0x7143af1 (0x2d8c bytes) 2024-09-14T09:06:25.155 INFO:teuthology.orchestra.run.smithi176.stdout:2842 punch from 0xaaca62 to 0xaaeeb3, (0x2451 bytes) 2024-09-14T09:06:25.178 INFO:teuthology.orchestra.run.smithi176.stdout:2843 trunc from 0xd766808 to 0x451a3fa 2024-09-14T09:06:25.194 INFO:teuthology.orchestra.run.smithi176.stdout:2844 write 0x226e7ce thru 0x2273dfc (0x562f bytes) 2024-09-14T09:06:25.208 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T09:06:25.234 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:25.234 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:25.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:25.234 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:25.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.235 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:9982 3.1a 3:5af3e476:::rbd_data.115eadec874.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:06:25.235 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:25.235 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.637830954999998, 2024-09-14T09:06:25.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042276341000000002, 2024-09-14T09:06:25.235 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.235 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.235 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:25.235 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.235 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.235 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.235 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 9982 2024-09-14T09:06:25.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.235 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:25.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:25.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597970+0000", 2024-09-14T09:06:25.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0599999999999999e-07 2024-09-14T09:06:25.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597974+0000", 2024-09-14T09:06:25.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2649999999999998e-06 2024-09-14T09:06:25.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597986+0000", 2024-09-14T09:06:25.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1019000000000001e-05 2024-09-14T09:06:25.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597991+0000", 2024-09-14T09:06:25.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5049999999999998e-06 2024-09-14T09:06:25.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.598083+0000", 2024-09-14T09:06:25.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2062e-05 2024-09-14T09:06:25.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.598227+0000", 2024-09-14T09:06:25.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000143609 2024-09-14T09:06:25.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:25.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.603107+0000", 2024-09-14T09:06:25.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048805819999999996 2024-09-14T09:06:25.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:25.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640002+0000", 2024-09-14T09:06:25.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.036894641999999998 2024-09-14T09:06:25.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:25.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640077+0000", 2024-09-14T09:06:25.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4927999999999998e-05 2024-09-14T09:06:25.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:25.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640174+0000", 2024-09-14T09:06:25.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7417999999999995e-05 2024-09-14T09:06:25.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640246+0000", 2024-09-14T09:06:25.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1505000000000001e-05 2024-09-14T09:06:25.241 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.241 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.241 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.241 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53587 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1212416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:25.241 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508126+0000", 2024-09-14T09:06:25.241 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.7276748009999991, 2024-09-14T09:06:25.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033430271999999997, 2024-09-14T09:06:25.242 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.242 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.242 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.242 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.242 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.242 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.242 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53587 2024-09-14T09:06:25.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.242 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508126+0000", 2024-09-14T09:06:25.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508126+0000", 2024-09-14T09:06:25.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508127+0000", 2024-09-14T09:06:25.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.11e-06 2024-09-14T09:06:25.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508132+0000", 2024-09-14T09:06:25.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4770000000000002e-06 2024-09-14T09:06:25.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508151+0000", 2024-09-14T09:06:25.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9032999999999999e-05 2024-09-14T09:06:25.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508164+0000", 2024-09-14T09:06:25.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2673e-05 2024-09-14T09:06:25.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541369+0000", 2024-09-14T09:06:25.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033205367999999999 2024-09-14T09:06:25.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541455+0000", 2024-09-14T09:06:25.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5618000000000006e-05 2024-09-14T09:06:25.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541556+0000", 2024-09-14T09:06:25.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100993 2024-09-14T09:06:25.245 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.245 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.245 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.246 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53596 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:25.246 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508184+0000", 2024-09-14T09:06:25.246 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.7276161680000008, 2024-09-14T09:06:25.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033494411000000002, 2024-09-14T09:06:25.246 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.246 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.246 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.246 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.246 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.246 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.247 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53596 2024-09-14T09:06:25.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.247 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508184+0000", 2024-09-14T09:06:25.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508184+0000", 2024-09-14T09:06:25.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508185+0000", 2024-09-14T09:06:25.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.217e-06 2024-09-14T09:06:25.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508191+0000", 2024-09-14T09:06:25.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6330000000000001e-06 2024-09-14T09:06:25.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508209+0000", 2024-09-14T09:06:25.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7804000000000001e-05 2024-09-14T09:06:25.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508223+0000", 2024-09-14T09:06:25.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3783e-05 2024-09-14T09:06:25.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541582+0000", 2024-09-14T09:06:25.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033359403000000003 2024-09-14T09:06:25.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541622+0000", 2024-09-14T09:06:25.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9937000000000001e-05 2024-09-14T09:06:25.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541679+0000", 2024-09-14T09:06:25.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6634e-05 2024-09-14T09:06:25.250 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.250 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.250 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.250 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53599 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:25.250 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508241+0000", 2024-09-14T09:06:25.250 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.7275592950000007, 2024-09-14T09:06:25.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033611720999999997, 2024-09-14T09:06:25.251 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.251 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.251 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.251 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.251 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.251 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.251 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53599 2024-09-14T09:06:25.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.251 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508241+0000", 2024-09-14T09:06:25.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508241+0000", 2024-09-14T09:06:25.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508242+0000", 2024-09-14T09:06:25.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1230000000000001e-06 2024-09-14T09:06:25.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508247+0000", 2024-09-14T09:06:25.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8439999999999997e-06 2024-09-14T09:06:25.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508276+0000", 2024-09-14T09:06:25.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9363e-05 2024-09-14T09:06:25.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508290+0000", 2024-09-14T09:06:25.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.364e-05 2024-09-14T09:06:25.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541712+0000", 2024-09-14T09:06:25.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033421713999999998 2024-09-14T09:06:25.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541785+0000", 2024-09-14T09:06:25.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3696999999999996e-05 2024-09-14T09:06:25.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541853+0000", 2024-09-14T09:06:25.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7340000000000002e-05 2024-09-14T09:06:25.254 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.255 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.255 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.255 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53601 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:25.255 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508310+0000", 2024-09-14T09:06:25.255 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.7274906790000006, 2024-09-14T09:06:25.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033872092999999999, 2024-09-14T09:06:25.255 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.256 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.256 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.256 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.256 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.256 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.256 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53601 2024-09-14T09:06:25.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.256 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508310+0000", 2024-09-14T09:06:25.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508310+0000", 2024-09-14T09:06:25.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508311+0000", 2024-09-14T09:06:25.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.11e-06 2024-09-14T09:06:25.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508317+0000", 2024-09-14T09:06:25.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.127e-06 2024-09-14T09:06:25.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508335+0000", 2024-09-14T09:06:25.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8097e-05 2024-09-14T09:06:25.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508363+0000", 2024-09-14T09:06:25.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8089000000000002e-05 2024-09-14T09:06:25.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541929+0000", 2024-09-14T09:06:25.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033566046000000002 2024-09-14T09:06:25.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542062+0000", 2024-09-14T09:06:25.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013268000000000001 2024-09-14T09:06:25.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542182+0000", 2024-09-14T09:06:25.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119944 2024-09-14T09:06:25.260 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.260 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.260 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53603 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508409+0000", 2024-09-14T09:06:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.7273909160000009, 2024-09-14T09:06:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033982513999999998, 2024-09-14T09:06:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53603 2024-09-14T09:06:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508409+0000", 2024-09-14T09:06:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508409+0000", 2024-09-14T09:06:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508411+0000", 2024-09-14T09:06:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3859999999999999e-06 2024-09-14T09:06:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508417+0000", 2024-09-14T09:06:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0639999999999999e-06 2024-09-14T09:06:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508437+0000", 2024-09-14T09:06:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0055999999999999e-05 2024-09-14T09:06:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508451+0000", 2024-09-14T09:06:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4144e-05 2024-09-14T09:06:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542235+0000", 2024-09-14T09:06:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033783531999999998 2024-09-14T09:06:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542316+0000", 2024-09-14T09:06:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1704000000000002e-05 2024-09-14T09:06:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542392+0000", 2024-09-14T09:06:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5628000000000002e-05 2024-09-14T09:06:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53638 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508472+0000", 2024-09-14T09:06:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.7273283670000001, 2024-09-14T09:06:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034203029000000003, 2024-09-14T09:06:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53638 2024-09-14T09:06:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508472+0000", 2024-09-14T09:06:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508472+0000", 2024-09-14T09:06:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508473+0000", 2024-09-14T09:06:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3769999999999999e-06 2024-09-14T09:06:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508492+0000", 2024-09-14T09:06:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8643000000000001e-05 2024-09-14T09:06:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508512+0000", 2024-09-14T09:06:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9516000000000001e-05 2024-09-14T09:06:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508525+0000", 2024-09-14T09:06:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3457e-05 2024-09-14T09:06:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542480+0000", 2024-09-14T09:06:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033954766999999997 2024-09-14T09:06:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542579+0000", 2024-09-14T09:06:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9497e-05 2024-09-14T09:06:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542675+0000", 2024-09-14T09:06:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5772000000000004e-05 2024-09-14T09:06:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53646 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508545+0000", 2024-09-14T09:06:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.7272551440000008, 2024-09-14T09:06:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034341172000000003, 2024-09-14T09:06:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53646 2024-09-14T09:06:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508545+0000", 2024-09-14T09:06:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508545+0000", 2024-09-14T09:06:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508547+0000", 2024-09-14T09:06:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7269999999999999e-06 2024-09-14T09:06:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508553+0000", 2024-09-14T09:06:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6930000000000004e-06 2024-09-14T09:06:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508571+0000", 2024-09-14T09:06:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.844e-05 2024-09-14T09:06:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508584+0000", 2024-09-14T09:06:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3157e-05 2024-09-14T09:06:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542730+0000", 2024-09-14T09:06:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034145967999999999 2024-09-14T09:06:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542812+0000", 2024-09-14T09:06:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1761000000000005e-05 2024-09-14T09:06:25.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542886+0000", 2024-09-14T09:06:25.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4425999999999995e-05 2024-09-14T09:06:25.274 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.274 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.274 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.274 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53647 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 999424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:25.274 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508603+0000", 2024-09-14T09:06:25.274 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.7271974710000002, 2024-09-14T09:06:25.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034662680000000001, 2024-09-14T09:06:25.275 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.275 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.275 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.275 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.275 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.275 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.275 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53647 2024-09-14T09:06:25.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.275 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508603+0000", 2024-09-14T09:06:25.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508603+0000", 2024-09-14T09:06:25.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508604+0000", 2024-09-14T09:06:25.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.173e-06 2024-09-14T09:06:25.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508622+0000", 2024-09-14T09:06:25.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8326e-05 2024-09-14T09:06:25.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508641+0000", 2024-09-14T09:06:25.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8213999999999999e-05 2024-09-14T09:06:25.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508656+0000", 2024-09-14T09:06:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5336e-05 2024-09-14T09:06:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542946+0000", 2024-09-14T09:06:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034290270999999997 2024-09-14T09:06:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543089+0000", 2024-09-14T09:06:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014298300000000001 2024-09-14T09:06:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543266+0000", 2024-09-14T09:06:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000176377 2024-09-14T09:06:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53648 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508674+0000", 2024-09-14T09:06:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.7271264150000007, 2024-09-14T09:06:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034778557000000002, 2024-09-14T09:06:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53648 2024-09-14T09:06:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508674+0000", 2024-09-14T09:06:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508674+0000", 2024-09-14T09:06:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508675+0000", 2024-09-14T09:06:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.223e-06 2024-09-14T09:06:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508680+0000", 2024-09-14T09:06:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:06:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508694+0000", 2024-09-14T09:06:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4399999999999999e-05 2024-09-14T09:06:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508707+0000", 2024-09-14T09:06:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2723e-05 2024-09-14T09:06:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543316+0000", 2024-09-14T09:06:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034609648999999999 2024-09-14T09:06:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543386+0000", 2024-09-14T09:06:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9548999999999994e-05 2024-09-14T09:06:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543453+0000", 2024-09-14T09:06:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6595999999999997e-05 2024-09-14T09:06:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53649 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1097728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508725+0000", 2024-09-14T09:06:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.7270756120000001, 2024-09-14T09:06:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035096176, 2024-09-14T09:06:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53649 2024-09-14T09:06:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508725+0000", 2024-09-14T09:06:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508725+0000", 2024-09-14T09:06:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508726+0000", 2024-09-14T09:06:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1629999999999999e-06 2024-09-14T09:06:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508731+0000", 2024-09-14T09:06:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.04e-06 2024-09-14T09:06:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508749+0000", 2024-09-14T09:06:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7693000000000002e-05 2024-09-14T09:06:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508762+0000", 2024-09-14T09:06:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3190000000000001e-05 2024-09-14T09:06:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543515+0000", 2024-09-14T09:06:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034753066999999999 2024-09-14T09:06:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543650+0000", 2024-09-14T09:06:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013507100000000001 2024-09-14T09:06:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543821+0000", 2024-09-14T09:06:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017095200000000001 2024-09-14T09:06:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53650 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508780+0000", 2024-09-14T09:06:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.7270199660000003, 2024-09-14T09:06:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035246841000000001, 2024-09-14T09:06:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53650 2024-09-14T09:06:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508780+0000", 2024-09-14T09:06:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508780+0000", 2024-09-14T09:06:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508782+0000", 2024-09-14T09:06:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1799999999999999e-06 2024-09-14T09:06:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508787+0000", 2024-09-14T09:06:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9570000000000001e-06 2024-09-14T09:06:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508804+0000", 2024-09-14T09:06:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7523000000000001e-05 2024-09-14T09:06:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508817+0000", 2024-09-14T09:06:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.259e-05 2024-09-14T09:06:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543866+0000", 2024-09-14T09:06:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035049640999999999 2024-09-14T09:06:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543951+0000", 2024-09-14T09:06:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4897999999999999e-05 2024-09-14T09:06:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544027+0000", 2024-09-14T09:06:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6051999999999996e-05 2024-09-14T09:06:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53651 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1327104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508868+0000", 2024-09-14T09:06:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.7269323270000001, 2024-09-14T09:06:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035314533000000002, 2024-09-14T09:06:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53651 2024-09-14T09:06:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508868+0000", 2024-09-14T09:06:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508868+0000", 2024-09-14T09:06:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508869+0000", 2024-09-14T09:06:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.393e-06 2024-09-14T09:06:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508876+0000", 2024-09-14T09:06:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1539999999999999e-06 2024-09-14T09:06:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508913+0000", 2024-09-14T09:06:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6980000000000002e-05 2024-09-14T09:06:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508928+0000", 2024-09-14T09:06:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5543e-05 2024-09-14T09:06:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544061+0000", 2024-09-14T09:06:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035132656999999998 2024-09-14T09:06:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544127+0000", 2024-09-14T09:06:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6036000000000003e-05 2024-09-14T09:06:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544183+0000", 2024-09-14T09:06:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5770000000000003e-05 2024-09-14T09:06:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53652 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508952+0000", 2024-09-14T09:06:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.7268482640000009, 2024-09-14T09:06:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035409204, 2024-09-14T09:06:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53652 2024-09-14T09:06:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508952+0000", 2024-09-14T09:06:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508952+0000", 2024-09-14T09:06:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508953+0000", 2024-09-14T09:06:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3599999999999999e-06 2024-09-14T09:06:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508959+0000", 2024-09-14T09:06:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9270000000000002e-06 2024-09-14T09:06:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509001+0000", 2024-09-14T09:06:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1896e-05 2024-09-14T09:06:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509022+0000", 2024-09-14T09:06:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.035e-05 2024-09-14T09:06:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544219+0000", 2024-09-14T09:06:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035197581999999998 2024-09-14T09:06:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544295+0000", 2024-09-14T09:06:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5483999999999998e-05 2024-09-14T09:06:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544361+0000", 2024-09-14T09:06:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6605000000000004e-05 2024-09-14T09:06:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53653 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509038+0000", 2024-09-14T09:06:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.7267621099999992, 2024-09-14T09:06:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035495397999999997, 2024-09-14T09:06:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53653 2024-09-14T09:06:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509038+0000", 2024-09-14T09:06:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509038+0000", 2024-09-14T09:06:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509039+0000", 2024-09-14T09:06:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9500000000000001e-07 2024-09-14T09:06:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509044+0000", 2024-09-14T09:06:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4179999999999999e-06 2024-09-14T09:06:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509054+0000", 2024-09-14T09:06:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0891e-05 2024-09-14T09:06:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509061+0000", 2024-09-14T09:06:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3260000000000001e-06 2024-09-14T09:06:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544388+0000", 2024-09-14T09:06:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035327648000000003 2024-09-14T09:06:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544461+0000", 2024-09-14T09:06:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2669999999999994e-05 2024-09-14T09:06:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544534+0000", 2024-09-14T09:06:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2550000000000002e-05 2024-09-14T09:06:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53675 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509071+0000", 2024-09-14T09:06:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.726729723, 2024-09-14T09:06:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035630230999999998, 2024-09-14T09:06:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53675 2024-09-14T09:06:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509071+0000", 2024-09-14T09:06:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509071+0000", 2024-09-14T09:06:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509071+0000", 2024-09-14T09:06:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6899999999999997e-07 2024-09-14T09:06:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509074+0000", 2024-09-14T09:06:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5979999999999999e-06 2024-09-14T09:06:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509081+0000", 2024-09-14T09:06:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0790000000000003e-06 2024-09-14T09:06:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509086+0000", 2024-09-14T09:06:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4280000000000004e-06 2024-09-14T09:06:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544561+0000", 2024-09-14T09:06:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035474736 2024-09-14T09:06:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544634+0000", 2024-09-14T09:06:25.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3258999999999998e-05 2024-09-14T09:06:25.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544701+0000", 2024-09-14T09:06:25.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6562000000000004e-05 2024-09-14T09:06:25.315 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.315 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.315 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.315 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53676 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:25.315 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509095+0000", 2024-09-14T09:06:25.315 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.726705505, 2024-09-14T09:06:25.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035734397000000001, 2024-09-14T09:06:25.316 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.316 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.316 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.316 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.316 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.316 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.316 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53676 2024-09-14T09:06:25.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.316 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509095+0000", 2024-09-14T09:06:25.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509095+0000", 2024-09-14T09:06:25.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509095+0000", 2024-09-14T09:06:25.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5700000000000002e-07 2024-09-14T09:06:25.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509098+0000", 2024-09-14T09:06:25.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.553e-06 2024-09-14T09:06:25.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509105+0000", 2024-09-14T09:06:25.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1350000000000003e-06 2024-09-14T09:06:25.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509110+0000", 2024-09-14T09:06:25.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1370000000000001e-06 2024-09-14T09:06:25.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544734+0000", 2024-09-14T09:06:25.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035624054000000002 2024-09-14T09:06:25.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544782+0000", 2024-09-14T09:06:25.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7956e-05 2024-09-14T09:06:25.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544829+0000", 2024-09-14T09:06:25.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7005000000000001e-05 2024-09-14T09:06:25.319 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.319 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.319 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.320 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53677 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:25.320 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509120+0000", 2024-09-14T09:06:25.320 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.7266799769999999, 2024-09-14T09:06:25.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035825663000000001, 2024-09-14T09:06:25.320 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.320 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.320 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.320 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.320 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.320 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.320 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53677 2024-09-14T09:06:25.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.321 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509120+0000", 2024-09-14T09:06:25.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509120+0000", 2024-09-14T09:06:25.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509121+0000", 2024-09-14T09:06:25.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2e-07 2024-09-14T09:06:25.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509136+0000", 2024-09-14T09:06:25.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.517e-05 2024-09-14T09:06:25.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509145+0000", 2024-09-14T09:06:25.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4880000000000007e-06 2024-09-14T09:06:25.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509150+0000", 2024-09-14T09:06:25.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9100000000000002e-06 2024-09-14T09:06:25.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544860+0000", 2024-09-14T09:06:25.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035709739999999997 2024-09-14T09:06:25.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544904+0000", 2024-09-14T09:06:25.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3791999999999997e-05 2024-09-14T09:06:25.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544946+0000", 2024-09-14T09:06:25.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2042999999999997e-05 2024-09-14T09:06:25.324 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.324 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.324 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.324 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53678 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:25.324 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509160+0000", 2024-09-14T09:06:25.324 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.7266400090000005, 2024-09-14T09:06:25.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035923798999999999, 2024-09-14T09:06:25.324 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.324 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.324 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.325 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.325 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.325 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.325 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53678 2024-09-14T09:06:25.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.325 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509160+0000", 2024-09-14T09:06:25.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509160+0000", 2024-09-14T09:06:25.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509161+0000", 2024-09-14T09:06:25.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9599999999999999e-07 2024-09-14T09:06:25.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509163+0000", 2024-09-14T09:06:25.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3410000000000001e-06 2024-09-14T09:06:25.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509171+0000", 2024-09-14T09:06:25.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5519999999999999e-06 2024-09-14T09:06:25.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509176+0000", 2024-09-14T09:06:25.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4750000000000001e-06 2024-09-14T09:06:25.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544975+0000", 2024-09-14T09:06:25.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035798862000000001 2024-09-14T09:06:25.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.545033+0000", 2024-09-14T09:06:25.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8148e-05 2024-09-14T09:06:25.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.545084+0000", 2024-09-14T09:06:25.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0825000000000002e-05 2024-09-14T09:06:25.328 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.328 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.328 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.328 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53838 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 573440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:25.328 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.518551+0000", 2024-09-14T09:06:25.329 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.7172494599999997, 2024-09-14T09:06:25.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033585690000000001, 2024-09-14T09:06:25.329 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.329 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.329 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.329 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.329 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.329 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.329 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53838 2024-09-14T09:06:25.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.329 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518551+0000", 2024-09-14T09:06:25.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518551+0000", 2024-09-14T09:06:25.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518551+0000", 2024-09-14T09:06:25.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2499999999999995e-07 2024-09-14T09:06:25.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518554+0000", 2024-09-14T09:06:25.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9450000000000002e-06 2024-09-14T09:06:25.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518561+0000", 2024-09-14T09:06:25.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4849999999999997e-06 2024-09-14T09:06:25.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518572+0000", 2024-09-14T09:06:25.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0818999999999999e-05 2024-09-14T09:06:25.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.551823+0000", 2024-09-14T09:06:25.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033250864999999998 2024-09-14T09:06:25.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.551967+0000", 2024-09-14T09:06:25.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014399700000000001 2024-09-14T09:06:25.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.552137+0000", 2024-09-14T09:06:25.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170054 2024-09-14T09:06:25.332 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.332 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.332 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.333 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.333 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.333 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:25.334 INFO:teuthology.orchestra.run.smithi176.stdout:2846 write 0xbe5bea7 thru 0xbe641d2 (0x832c bytes) 2024-09-14T09:06:25.334 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:25.251+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:25.335 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:06:25.335 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:25.335+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:25.336 INFO:teuthology.orchestra.run.smithi176.stdout:2847 write 0x456e9c8 thru 0x457133c (0x2975 bytes) 2024-09-14T09:06:25.336 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:25.336 INFO:teuthology.orchestra.run.smithi176.stdout:2849 read 0x4a14067 thru 0x4a167cd (0x2767 bytes) 2024-09-14T09:06:25.336 INFO:teuthology.orchestra.run.smithi176.stdout:2850 read 0x5d53ea5 thru 0x5d63b8d (0xfce9 bytes) 2024-09-14T09:06:25.337 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:25.337 INFO:teuthology.orchestra.run.smithi176.stdout:2854 write 0x622bacb thru 0x6235ca4 (0xa1da bytes) 2024-09-14T09:06:25.337 INFO:teuthology.orchestra.run.smithi176.stdout:2855 punch from 0x467b401 to 0x46834ca, (0x80c9 bytes) 2024-09-14T09:06:25.337 INFO:teuthology.orchestra.run.smithi176.stdout:2856 write 0x72cc7b6 thru 0x72d2d48 (0x6593 bytes) 2024-09-14T09:06:25.337 INFO:teuthology.orchestra.run.smithi176.stdout:2857 write 0xcff0195 thru 0xcffbfce (0xbe3a bytes) 2024-09-14T09:06:25.337 INFO:teuthology.orchestra.run.smithi176.stdout:2858 trunc from 0xcffbfcf to 0x1d07c83 2024-09-14T09:06:25.376 INFO:teuthology.orchestra.run.smithi176.stdout:2859 write 0xedf6fb2 thru 0xee0569f (0xe6ee bytes) 2024-09-14T09:06:25.395 INFO:teuthology.orchestra.run.smithi176.stdout:2860 trunc from 0xee056a0 to 0x9dbb8e6 2024-09-14T09:06:25.411 INFO:teuthology.orchestra.run.smithi176.stdout:2861 read 0x4cfcb4a thru 0x4cfdad6 (0xf8d bytes) 2024-09-14T09:06:25.411 INFO:teuthology.orchestra.run.smithi176.stdout:2862 punch from 0x4405d2 to 0x4405eb, (0x19 bytes) 2024-09-14T09:06:25.411 INFO:teuthology.orchestra.run.smithi176.stdout:2865 write 0x1363fd1 thru 0x13698e9 (0x5919 bytes) 2024-09-14T09:06:25.432 INFO:teuthology.orchestra.run.smithi176.stdout:2867 write 0xe347631 thru 0xe357450 (0xfe20 bytes) 2024-09-14T09:06:25.434 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:25.435+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:25.440 INFO:teuthology.orchestra.run.smithi176.stdout:2869 write 0x6715d15 thru 0x671c6c3 (0x69af bytes) 2024-09-14T09:06:25.448 INFO:teuthology.orchestra.run.smithi176.stdout:2870 read 0xb1d56e2 thru 0xb1d785d (0x217c bytes) 2024-09-14T09:06:25.448 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:25.449 INFO:teuthology.orchestra.run.smithi176.stdout:2873 writesame 0xcdfe31d thru 0xce05b36 data_size 0x1(0x781a bytes) 2024-09-14T09:06:25.484 INFO:teuthology.orchestra.run.smithi176.stdout:2878 punch from 0x36bfc71 to 0x36c7819, (0x7ba8 bytes) 2024-09-14T09:06:25.484 INFO:teuthology.orchestra.run.smithi176.stdout:2880 write 0x811d405 thru 0x8122150 (0x4d4c bytes) 2024-09-14T09:06:25.492 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:25.492 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:25.498 INFO:teuthology.orchestra.run.smithi176.stdout:2884 trunc from 0xe357451 to 0xac4853b 2024-09-14T09:06:25.515 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:25.515 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:25.515 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:25.515 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:25.517 INFO:teuthology.orchestra.run.smithi176.stdout:2885 writesame 0x7033c35 thru 0x703caf5 data_size 0x1(0x8ec1 bytes) 2024-09-14T09:06:25.530 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:06:25.535 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:25.535+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:25.552 INFO:teuthology.orchestra.run.smithi176.stdout:2887 write 0x73dfdd2 thru 0x73eb227 (0xb456 bytes) 2024-09-14T09:06:25.565 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:25.565 INFO:teuthology.orchestra.run.smithi176.stdout:2889 read 0x1db228 thru 0x1e83f9 (0xd1d2 bytes) 2024-09-14T09:06:25.565 INFO:teuthology.orchestra.run.smithi176.stdout:2890 read 0x2459267 thru 0x245b8cb (0x2665 bytes) 2024-09-14T09:06:25.565 INFO:teuthology.orchestra.run.smithi176.stdout:2891 punch from 0x4c59249 to 0x4c61b8b, (0x8942 bytes) 2024-09-14T09:06:25.565 INFO:teuthology.orchestra.run.smithi176.stdout:2892 punch from 0x7c9435 to 0x7d23e9, (0x8fb4 bytes) 2024-09-14T09:06:25.566 INFO:teuthology.orchestra.run.smithi176.stdout:2894 punch from 0x54d6856 to 0x54e2951, (0xc0fb bytes) 2024-09-14T09:06:25.566 INFO:teuthology.orchestra.run.smithi176.stdout:2895 read 0x665be9f thru 0x665e9a8 (0x2b0a bytes) 2024-09-14T09:06:25.573 INFO:teuthology.orchestra.run.smithi176.stdout:2897 write 0xc6d6228 thru 0xc6e0f37 (0xad10 bytes) 2024-09-14T09:06:25.594 INFO:teuthology.orchestra.run.smithi176.stdout:2898 writesame 0xed46170 thru 0xed4a09a data_size 0x1(0x3f2b bytes) 2024-09-14T09:06:25.594 INFO:teuthology.orchestra.run.smithi176.stdout:2899 punch from 0xc78f1f8 to 0xc79201e, (0x2e26 bytes) 2024-09-14T09:06:25.600 INFO:teuthology.orchestra.run.smithi176.stdout:2901 trunc from 0xed4a09b to 0xac7a8a0 2024-09-14T09:06:25.616 INFO:teuthology.orchestra.run.smithi176.stdout:2902 read 0x9bc78c0 thru 0x9bcb722 (0x3e63 bytes) 2024-09-14T09:06:25.617 INFO:teuthology.orchestra.run.smithi176.stdout:2903 write 0x8fbe32a thru 0x8fc48d9 (0x65b0 bytes) 2024-09-14T09:06:25.625 INFO:teuthology.orchestra.run.smithi176.stdout:2904 writesame 0x65b8169 thru 0x65bf702 data_size 0x1(0x759a bytes) 2024-09-14T09:06:25.637 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:25.635+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:25.657 INFO:teuthology.orchestra.run.smithi176.stdout:2905 punch from 0x6eefc04 to 0x6ef7a4e, (0x7e4a bytes) 2024-09-14T09:06:25.675 INFO:teuthology.orchestra.run.smithi176.stdout:2906 writesame 0xe356b11 thru 0xe362473 data_size 0x1(0xb963 bytes) 2024-09-14T09:06:25.679 INFO:teuthology.orchestra.run.smithi176.stdout:2907 trunc from 0xe362474 to 0xc6e25b2 2024-09-14T09:06:25.697 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:25.697 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:25.697 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:25.697 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:25.697 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:25.701 INFO:teuthology.orchestra.run.smithi176.stdout:2908 trunc from 0xc6e25b2 to 0x7a23822 2024-09-14T09:06:25.713 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:06:25.724 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:25.724 INFO:teuthology.orchestra.run.smithi176.stdout:2911 flatten 2024-09-14T09:06:25.739 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:25.772 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:25.773 INFO:teuthology.orchestra.run.smithi176.stdout:2917 punch from 0x2f7d81f to 0x2f82bba, (0x539b bytes) 2024-09-14T09:06:25.773 INFO:teuthology.orchestra.run.smithi176.stdout:2918 write 0x3f42fa thru 0x3f49d0 (0x6d7 bytes) 2024-09-14T09:06:25.773 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:25.763+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:25.774 INFO:teuthology.orchestra.run.smithi176.stdout:2919 write 0x457f5e3 thru 0x458dfd9 (0xe9f7 bytes) 2024-09-14T09:06:25.774 INFO:teuthology.orchestra.run.smithi176.stdout:2920 trunc from 0x7a23822 to 0x64c0d04 2024-09-14T09:06:25.812 INFO:teuthology.orchestra.run.smithi176.stdout:2922 writesame 0xd2a320f thru 0xd2ac6b1 data_size 0x1(0x94a3 bytes) 2024-09-14T09:06:25.812 INFO:teuthology.orchestra.run.smithi176.stdout:2923 read 0x6706043 thru 0x6714ad2 (0xea90 bytes) 2024-09-14T09:06:25.812 INFO:teuthology.orchestra.run.smithi176.stdout:2924 read 0x51cab62 thru 0x51cbbb1 (0x1050 bytes) 2024-09-14T09:06:25.813 INFO:teuthology.orchestra.run.smithi176.stdout:2925 write 0x94a93b0 thru 0x94b69af (0xd600 bytes) 2024-09-14T09:06:25.839 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:25.839+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:25.842 INFO:teuthology.orchestra.run.smithi176.stdout:2926 write 0xec2b7db thru 0xec3b136 (0xf95c bytes) 2024-09-14T09:06:25.847 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:25.847 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:25.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:25.847 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:25.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.847 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49333 3.1d 3:be596f9e:::rbd_data.115e51d92ed2.0000000000000007:12 [read 950272~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:25.847 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:25.848 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.30111754, 2024-09-14T09:06:25.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072172603000000002, 2024-09-14T09:06:25.848 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.848 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.848 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.848 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.848 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.848 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.848 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49333 2024-09-14T09:06:25.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.848 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:25.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:25.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:25.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6000000000000003e-07 2024-09-14T09:06:25.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546216+0000", 2024-09-14T09:06:25.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5339999999999998e-06 2024-09-14T09:06:25.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546224+0000", 2024-09-14T09:06:25.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7770000000000002e-06 2024-09-14T09:06:25.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546230+0000", 2024-09-14T09:06:25.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3400000000000003e-06 2024-09-14T09:06:25.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.617881+0000", 2024-09-14T09:06:25.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071650488999999998 2024-09-14T09:06:25.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.617958+0000", 2024-09-14T09:06:25.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7018e-05 2024-09-14T09:06:25.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618385+0000", 2024-09-14T09:06:25.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00042768500000000002 2024-09-14T09:06:25.852 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.852 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.852 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.852 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49334 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:25.852 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:25.852 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.301090658, 2024-09-14T09:06:25.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072227886000000005, 2024-09-14T09:06:25.852 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.852 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.852 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:25.852 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.853 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.853 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.853 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49334 2024-09-14T09:06:25.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.853 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:25.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:25.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:25.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8899999999999999e-07 2024-09-14T09:06:25.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546243+0000", 2024-09-14T09:06:25.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.503e-06 2024-09-14T09:06:25.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546253+0000", 2024-09-14T09:06:25.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0210999999999999e-05 2024-09-14T09:06:25.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546267+0000", 2024-09-14T09:06:25.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4183e-05 2024-09-14T09:06:25.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618449+0000", 2024-09-14T09:06:25.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072181828000000003 2024-09-14T09:06:25.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618467+0000", 2024-09-14T09:06:25.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8471999999999999e-05 2024-09-14T09:06:25.856 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.856 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.856 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.856 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49335 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:25.856 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:25.856 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.301054507, 2024-09-14T09:06:25.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072255640999999995, 2024-09-14T09:06:25.856 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.857 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.857 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:25.857 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.857 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.857 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.857 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49335 2024-09-14T09:06:25.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.857 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:25.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:25.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:25.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9999999999999997e-07 2024-09-14T09:06:25.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546279+0000", 2024-09-14T09:06:25.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.695e-06 2024-09-14T09:06:25.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546288+0000", 2024-09-14T09:06:25.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7110000000000001e-06 2024-09-14T09:06:25.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546294+0000", 2024-09-14T09:06:25.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9830000000000001e-06 2024-09-14T09:06:25.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618521+0000", 2024-09-14T09:06:25.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072226948999999999 2024-09-14T09:06:25.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618531+0000", 2024-09-14T09:06:25.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0603e-05 2024-09-14T09:06:25.860 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.860 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.860 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.860 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49336 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:25.860 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:25.860 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.301028604000001, 2024-09-14T09:06:25.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072292223000000003, 2024-09-14T09:06:25.861 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.861 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.861 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:25.861 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.861 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.861 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.861 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49336 2024-09-14T09:06:25.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.861 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:25.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:25.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:25.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4000000000000001e-07 2024-09-14T09:06:25.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546305+0000", 2024-09-14T09:06:25.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3539999999999998e-06 2024-09-14T09:06:25.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546315+0000", 2024-09-14T09:06:25.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0431999999999999e-05 2024-09-14T09:06:25.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546321+0000", 2024-09-14T09:06:25.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7620000000000001e-06 2024-09-14T09:06:25.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618583+0000", 2024-09-14T09:06:25.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072262046999999996 2024-09-14T09:06:25.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618594+0000", 2024-09-14T09:06:25.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0988000000000001e-05 2024-09-14T09:06:25.864 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.864 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.864 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.864 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49337 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:25.865 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:25.865 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.300980989999999, 2024-09-14T09:06:25.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072294774000000006, 2024-09-14T09:06:25.865 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.865 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.865 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:25.865 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.865 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.865 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.865 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49337 2024-09-14T09:06:25.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.865 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:25.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:25.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546350+0000", 2024-09-14T09:06:25.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4600000000000004e-07 2024-09-14T09:06:25.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546381+0000", 2024-09-14T09:06:25.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0902000000000003e-05 2024-09-14T09:06:25.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546412+0000", 2024-09-14T09:06:25.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0825999999999999e-05 2024-09-14T09:06:25.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546438+0000", 2024-09-14T09:06:25.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6410999999999998e-05 2024-09-14T09:06:25.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618637+0000", 2024-09-14T09:06:25.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072198907000000007 2024-09-14T09:06:25.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618644+0000", 2024-09-14T09:06:25.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9820000000000002e-06 2024-09-14T09:06:25.868 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.868 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.868 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.869 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49338 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:25.869 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:25.869 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.300866963000001, 2024-09-14T09:06:25.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072233250999999998, 2024-09-14T09:06:25.869 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.869 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.869 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:25.869 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.869 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.869 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.869 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49338 2024-09-14T09:06:25.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.869 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:25.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:25.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546465+0000", 2024-09-14T09:06:25.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.455e-06 2024-09-14T09:06:25.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546481+0000", 2024-09-14T09:06:25.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5834000000000001e-05 2024-09-14T09:06:25.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546498+0000", 2024-09-14T09:06:25.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6937000000000001e-05 2024-09-14T09:06:25.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546508+0000", 2024-09-14T09:06:25.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0446e-05 2024-09-14T09:06:25.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618685+0000", 2024-09-14T09:06:25.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072177280999999996 2024-09-14T09:06:25.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618697+0000", 2024-09-14T09:06:25.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1297999999999999e-05 2024-09-14T09:06:25.872 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.872 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.872 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.873 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49339 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:25.873 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546535+0000", 2024-09-14T09:06:25.873 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.300794946, 2024-09-14T09:06:25.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072197266999999996, 2024-09-14T09:06:25.873 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.873 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.873 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:25.873 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.873 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.873 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.873 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49339 2024-09-14T09:06:25.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.873 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546535+0000", 2024-09-14T09:06:25.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546535+0000", 2024-09-14T09:06:25.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546536+0000", 2024-09-14T09:06:25.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1399999999999996e-07 2024-09-14T09:06:25.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546539+0000", 2024-09-14T09:06:25.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.089e-06 2024-09-14T09:06:25.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546560+0000", 2024-09-14T09:06:25.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1217e-05 2024-09-14T09:06:25.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546567+0000", 2024-09-14T09:06:25.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9e-06 2024-09-14T09:06:25.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618726+0000", 2024-09-14T09:06:25.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.07215887 2024-09-14T09:06:25.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618733+0000", 2024-09-14T09:06:25.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3770000000000001e-06 2024-09-14T09:06:25.876 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.876 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.876 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.877 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49340 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:25.877 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546578+0000", 2024-09-14T09:06:25.877 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.300752055, 2024-09-14T09:06:25.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072205187000000004, 2024-09-14T09:06:25.877 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.877 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.877 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:25.877 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.877 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.877 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.877 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49340 2024-09-14T09:06:25.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.877 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546578+0000", 2024-09-14T09:06:25.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546578+0000", 2024-09-14T09:06:25.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546579+0000", 2024-09-14T09:06:25.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1999999999999999e-07 2024-09-14T09:06:25.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546582+0000", 2024-09-14T09:06:25.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4460000000000001e-06 2024-09-14T09:06:25.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546603+0000", 2024-09-14T09:06:25.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0502000000000001e-05 2024-09-14T09:06:25.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546610+0000", 2024-09-14T09:06:25.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7150000000000001e-06 2024-09-14T09:06:25.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618770+0000", 2024-09-14T09:06:25.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072160600000000005 2024-09-14T09:06:25.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618783+0000", 2024-09-14T09:06:25.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:06:25.880 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.880 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.881 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.881 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49341 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:25.881 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546667+0000", 2024-09-14T09:06:25.881 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.300662946999999, 2024-09-14T09:06:25.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072181163000000007, 2024-09-14T09:06:25.881 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.881 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.881 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:25.881 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.881 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.881 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.881 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49341 2024-09-14T09:06:25.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.882 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546667+0000", 2024-09-14T09:06:25.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546667+0000", 2024-09-14T09:06:25.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546669+0000", 2024-09-14T09:06:25.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.486e-06 2024-09-14T09:06:25.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546674+0000", 2024-09-14T09:06:25.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5110000000000003e-06 2024-09-14T09:06:25.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546691+0000", 2024-09-14T09:06:25.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7051e-05 2024-09-14T09:06:25.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546703+0000", 2024-09-14T09:06:25.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1155000000000001e-05 2024-09-14T09:06:25.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618836+0000", 2024-09-14T09:06:25.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072133616999999997 2024-09-14T09:06:25.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618848+0000", 2024-09-14T09:06:25.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2343e-05 2024-09-14T09:06:25.885 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.885 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.885 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.885 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49386 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:25.885 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546806+0000", 2024-09-14T09:06:25.885 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.300523788, 2024-09-14T09:06:25.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072019206000000002, 2024-09-14T09:06:25.885 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.885 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.885 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:25.885 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.885 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.886 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.886 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49386 2024-09-14T09:06:25.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.886 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546806+0000", 2024-09-14T09:06:25.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546806+0000", 2024-09-14T09:06:25.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546808+0000", 2024-09-14T09:06:25.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.471e-06 2024-09-14T09:06:25.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546813+0000", 2024-09-14T09:06:25.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1800000000000004e-06 2024-09-14T09:06:25.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546830+0000", 2024-09-14T09:06:25.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.658e-05 2024-09-14T09:06:25.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546842+0000", 2024-09-14T09:06:25.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2442999999999999e-05 2024-09-14T09:06:25.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618816+0000", 2024-09-14T09:06:25.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071973920999999996 2024-09-14T09:06:25.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618826+0000", 2024-09-14T09:06:25.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.611e-06 2024-09-14T09:06:25.889 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.889 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.889 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.889 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49387 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:25.889 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546861+0000", 2024-09-14T09:06:25.889 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.300469143000001, 2024-09-14T09:06:25.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071996665000000001, 2024-09-14T09:06:25.889 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.889 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.889 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:25.889 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.890 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.890 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.890 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49387 2024-09-14T09:06:25.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.890 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546861+0000", 2024-09-14T09:06:25.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546861+0000", 2024-09-14T09:06:25.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546862+0000", 2024-09-14T09:06:25.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.186e-06 2024-09-14T09:06:25.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546867+0000", 2024-09-14T09:06:25.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9049999999999996e-06 2024-09-14T09:06:25.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546880+0000", 2024-09-14T09:06:25.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2815e-05 2024-09-14T09:06:25.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546889+0000", 2024-09-14T09:06:25.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5420000000000005e-06 2024-09-14T09:06:25.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618850+0000", 2024-09-14T09:06:25.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071961065000000005 2024-09-14T09:06:25.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618858+0000", 2024-09-14T09:06:25.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1519999999999992e-06 2024-09-14T09:06:25.893 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.893 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.893 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.893 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49388 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:25.893 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:25.893 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.300431458, 2024-09-14T09:06:25.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072001645000000003, 2024-09-14T09:06:25.893 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.894 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.894 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:25.894 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.894 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.894 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.894 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49388 2024-09-14T09:06:25.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.894 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:25.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:25.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:25.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:06:25.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546902+0000", 2024-09-14T09:06:25.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.734e-06 2024-09-14T09:06:25.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546910+0000", 2024-09-14T09:06:25.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9629999999999998e-06 2024-09-14T09:06:25.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546916+0000", 2024-09-14T09:06:25.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.28e-06 2024-09-14T09:06:25.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618889+0000", 2024-09-14T09:06:25.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071972555999999993 2024-09-14T09:06:25.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618900+0000", 2024-09-14T09:06:25.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1413999999999999e-05 2024-09-14T09:06:25.897 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.897 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.897 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.897 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49389 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:25.897 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546925+0000", 2024-09-14T09:06:25.897 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.300405297999999, 2024-09-14T09:06:25.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072026707999999995, 2024-09-14T09:06:25.898 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.898 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.898 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:25.898 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.898 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.898 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.898 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49389 2024-09-14T09:06:25.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.898 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546925+0000", 2024-09-14T09:06:25.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546925+0000", 2024-09-14T09:06:25.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546926+0000", 2024-09-14T09:06:25.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8599999999999998e-07 2024-09-14T09:06:25.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546928+0000", 2024-09-14T09:06:25.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4200000000000001e-06 2024-09-14T09:06:25.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546938+0000", 2024-09-14T09:06:25.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5859999999999994e-06 2024-09-14T09:06:25.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546945+0000", 2024-09-14T09:06:25.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1770000000000001e-06 2024-09-14T09:06:25.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618940+0000", 2024-09-14T09:06:25.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071995491999999994 2024-09-14T09:06:25.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618952+0000", 2024-09-14T09:06:25.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1347e-05 2024-09-14T09:06:25.901 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.901 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.901 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.901 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49390 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:25.901 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547051+0000", 2024-09-14T09:06:25.901 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.300279708, 2024-09-14T09:06:25.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071963947, 2024-09-14T09:06:25.902 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.902 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.902 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:25.902 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.902 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.902 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.902 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49390 2024-09-14T09:06:25.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.902 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547051+0000", 2024-09-14T09:06:25.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547051+0000", 2024-09-14T09:06:25.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547053+0000", 2024-09-14T09:06:25.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.007e-06 2024-09-14T09:06:25.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547064+0000", 2024-09-14T09:06:25.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1146e-05 2024-09-14T09:06:25.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547089+0000", 2024-09-14T09:06:25.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.563e-05 2024-09-14T09:06:25.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547127+0000", 2024-09-14T09:06:25.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8040000000000002e-05 2024-09-14T09:06:25.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619003+0000", 2024-09-14T09:06:25.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071875724000000002 2024-09-14T09:06:25.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619014+0000", 2024-09-14T09:06:25.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1399999999999999e-05 2024-09-14T09:06:25.905 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.905 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.905 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.905 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49391 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:25.906 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547162+0000", 2024-09-14T09:06:25.906 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.300168545, 2024-09-14T09:06:25.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071911484999999997, 2024-09-14T09:06:25.906 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.906 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.906 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:25.906 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.906 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.906 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.906 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49391 2024-09-14T09:06:25.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.907 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547162+0000", 2024-09-14T09:06:25.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547162+0000", 2024-09-14T09:06:25.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547163+0000", 2024-09-14T09:06:25.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.73e-06 2024-09-14T09:06:25.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547170+0000", 2024-09-14T09:06:25.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0400000000000004e-06 2024-09-14T09:06:25.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547190+0000", 2024-09-14T09:06:25.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9423e-05 2024-09-14T09:06:25.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547204+0000", 2024-09-14T09:06:25.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4175999999999999e-05 2024-09-14T09:06:25.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619060+0000", 2024-09-14T09:06:25.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071856347000000001 2024-09-14T09:06:25.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619073+0000", 2024-09-14T09:06:25.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2768999999999999e-05 2024-09-14T09:06:25.910 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.910 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.910 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.910 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49392 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:25.910 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547226+0000", 2024-09-14T09:06:25.910 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.300104566, 2024-09-14T09:06:25.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071890509000000005, 2024-09-14T09:06:25.910 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.910 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.910 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:25.910 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.910 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.911 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.911 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49392 2024-09-14T09:06:25.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.911 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547226+0000", 2024-09-14T09:06:25.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547226+0000", 2024-09-14T09:06:25.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547227+0000", 2024-09-14T09:06:25.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3969999999999999e-06 2024-09-14T09:06:25.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547233+0000", 2024-09-14T09:06:25.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4530000000000001e-06 2024-09-14T09:06:25.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547244+0000", 2024-09-14T09:06:25.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1327e-05 2024-09-14T09:06:25.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547252+0000", 2024-09-14T09:06:25.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0269999999999995e-06 2024-09-14T09:06:25.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619110+0000", 2024-09-14T09:06:25.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071857997000000007 2024-09-14T09:06:25.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619116+0000", 2024-09-14T09:06:25.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3080000000000004e-06 2024-09-14T09:06:25.914 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.914 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.914 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.914 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49393 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:25.914 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547263+0000", 2024-09-14T09:06:25.914 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.300066922999999, 2024-09-14T09:06:25.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071900037999999999, 2024-09-14T09:06:25.914 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.914 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.914 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:25.915 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.915 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.915 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.915 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49393 2024-09-14T09:06:25.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.915 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547263+0000", 2024-09-14T09:06:25.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547263+0000", 2024-09-14T09:06:25.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547264+0000", 2024-09-14T09:06:25.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3399999999999998e-07 2024-09-14T09:06:25.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547299+0000", 2024-09-14T09:06:25.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5252000000000001e-05 2024-09-14T09:06:25.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547317+0000", 2024-09-14T09:06:25.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.736e-05 2024-09-14T09:06:25.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547332+0000", 2024-09-14T09:06:25.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.487e-05 2024-09-14T09:06:25.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619152+0000", 2024-09-14T09:06:25.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071820590000000004 2024-09-14T09:06:25.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619163+0000", 2024-09-14T09:06:25.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1131999999999999e-05 2024-09-14T09:06:25.918 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.918 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.918 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.918 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:50478 3.1d 3:b8a2eaf4:::rbd_data.115e5dc498ac.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T09:06:25.918 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:25.918 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.797690438, 2024-09-14T09:06:25.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.075023792000000006, 2024-09-14T09:06:25.918 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.918 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.918 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:25.919 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.919 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.919 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.919 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 50478 2024-09-14T09:06:25.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.919 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:25.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:25.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049641+0000", 2024-09-14T09:06:25.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6270000000000001e-06 2024-09-14T09:06:25.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049649+0000", 2024-09-14T09:06:25.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6279999999999996e-06 2024-09-14T09:06:25.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049667+0000", 2024-09-14T09:06:25.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-09-14T09:06:25.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049677+0000", 2024-09-14T09:06:25.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0509000000000001e-05 2024-09-14T09:06:25.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049751+0000", 2024-09-14T09:06:25.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3322e-05 2024-09-14T09:06:25.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049830+0000", 2024-09-14T09:06:25.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9932000000000007e-05 2024-09-14T09:06:25.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:25.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.053203+0000", 2024-09-14T09:06:25.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003372398 2024-09-14T09:06:25.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:25.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124542+0000", 2024-09-14T09:06:25.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071339105999999999 2024-09-14T09:06:25.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:25.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124562+0000", 2024-09-14T09:06:25.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9649000000000001e-05 2024-09-14T09:06:25.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:25.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124616+0000", 2024-09-14T09:06:25.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4642999999999998e-05 2024-09-14T09:06:25.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124664+0000", 2024-09-14T09:06:25.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7342000000000002e-05 2024-09-14T09:06:25.923 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.924 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.924 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.924 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65035 3.1d 3:bb585d47:::rbd_data.115e21c31fcf.0000000000000034:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:06:25.924 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.068842+0000", 2024-09-14T09:06:25.924 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.7784882040000003, 2024-09-14T09:06:25.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.094968451999999995, 2024-09-14T09:06:25.924 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.924 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.924 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:25.924 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:25.924 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:25.925 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:25.925 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65035 2024-09-14T09:06:25.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.925 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068842+0000", 2024-09-14T09:06:25.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068842+0000", 2024-09-14T09:06:25.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068843+0000", 2024-09-14T09:06:25.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.221e-06 2024-09-14T09:06:25.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068849+0000", 2024-09-14T09:06:25.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4659999999999998e-06 2024-09-14T09:06:25.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068861+0000", 2024-09-14T09:06:25.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2204e-05 2024-09-14T09:06:25.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068871+0000", 2024-09-14T09:06:25.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0318e-05 2024-09-14T09:06:25.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068942+0000", 2024-09-14T09:06:25.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0438999999999994e-05 2024-09-14T09:06:25.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.069005+0000", 2024-09-14T09:06:25.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3076000000000004e-05 2024-09-14T09:06:25.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:25.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.074957+0000", 2024-09-14T09:06:25.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059524549999999997 2024-09-14T09:06:25.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:25.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163479+0000", 2024-09-14T09:06:25.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.088521676999999993 2024-09-14T09:06:25.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:25.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163633+0000", 2024-09-14T09:06:25.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000154301 2024-09-14T09:06:25.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:25.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163733+0000", 2024-09-14T09:06:25.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100089 2024-09-14T09:06:25.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163811+0000", 2024-09-14T09:06:25.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7206000000000006e-05 2024-09-14T09:06:25.929 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.929 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.929 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.930 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:65035 3.1d e65/23)", 2024-09-14T09:06:25.930 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.087665+0000", 2024-09-14T09:06:25.930 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.7596652940000004, 2024-09-14T09:06:25.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.076160555000000005, 2024-09-14T09:06:25.930 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:25.930 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:25.930 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:25.930 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:25.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:25.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087665+0000", 2024-09-14T09:06:25.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:25.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087665+0000", 2024-09-14T09:06:25.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:25.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:25.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087667+0000", 2024-09-14T09:06:25.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.818e-06 2024-09-14T09:06:25.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:25.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087673+0000", 2024-09-14T09:06:25.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9259999999999997e-06 2024-09-14T09:06:25.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:25.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087692+0000", 2024-09-14T09:06:25.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9715e-05 2024-09-14T09:06:25.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:25.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087703+0000", 2024-09-14T09:06:25.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.094e-05 2024-09-14T09:06:25.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:25.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163595+0000", 2024-09-14T09:06:25.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.075892121000000007 2024-09-14T09:06:25.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:25.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163621+0000", 2024-09-14T09:06:25.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5383000000000001e-05 2024-09-14T09:06:25.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:25.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:25.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:25.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163826+0000", 2024-09-14T09:06:25.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020465199999999999 2024-09-14T09:06:25.933 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.933 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.933 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.934 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:25.934 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:25.934 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:25.935 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:06:25.936 INFO:teuthology.orchestra.run.smithi176.stdout:2927 punch from 0xeaad229 to 0xeab9b15, (0xc8ec bytes) 2024-09-14T09:06:25.936 INFO:teuthology.orchestra.run.smithi176.stdout:2928 write 0x8fef533 thru 0x8ff48f5 (0x53c3 bytes) 2024-09-14T09:06:25.936 INFO:teuthology.orchestra.run.smithi176.stdout:2929 write 0xcdf9eb9 thru 0xcdfac77 (0xdbf bytes) 2024-09-14T09:06:25.936 INFO:teuthology.orchestra.run.smithi176.stdout:2930 write 0x78e2e96 thru 0x78e3104 (0x26f bytes) 2024-09-14T09:06:25.936 INFO:teuthology.orchestra.run.smithi176.stdout:2931 write 0xbc69741 thru 0xbc70fd8 (0x7898 bytes) 2024-09-14T09:06:25.936 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:25.937 INFO:teuthology.orchestra.run.smithi176.stdout:2935 read 0x86950d thru 0x86c66b (0x315f bytes) 2024-09-14T09:06:25.937 INFO:teuthology.orchestra.run.smithi176.stdout:2936 trunc from 0xec3b137 to 0xa019d3a 2024-09-14T09:06:25.937 INFO:teuthology.orchestra.run.smithi176.stdout:2937 read 0x7414dce thru 0x7417f91 (0x31c4 bytes) 2024-09-14T09:06:25.937 INFO:teuthology.orchestra.run.smithi176.stdout:2938 write 0x8275e61 thru 0x827aad7 (0x4c77 bytes) 2024-09-14T09:06:25.943 INFO:teuthology.orchestra.run.smithi176.stdout:2939 punch from 0x94a2d3 to 0x952752, (0x847f bytes) 2024-09-14T09:06:25.943 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:25.943 INFO:teuthology.orchestra.run.smithi176.stdout:2942 writesame 0x81ca252 thru 0x81d2ad9 data_size 0x44(0x8888 bytes) 2024-09-14T09:06:25.947 INFO:teuthology.orchestra.run.smithi176.stdout:2943 writesame 0x604b9e6 thru 0x60583a9 data_size 0x4a(0xc9c4 bytes) 2024-09-14T09:06:25.965 INFO:teuthology.orchestra.run.smithi176.stdout:2944 trunc from 0xa019d3a to 0x6feac77 2024-09-14T09:06:25.979 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:25.983+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:25.992 INFO:teuthology.orchestra.run.smithi176.stdout:2945 write 0x6c1ed88 thru 0x6c1ff2c (0x11a5 bytes) 2024-09-14T09:06:26.001 INFO:teuthology.orchestra.run.smithi176.stdout:2946 read 0x2367151 thru 0x237236e (0xb21e bytes) 2024-09-14T09:06:26.002 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:26.002 INFO:teuthology.orchestra.run.smithi176.stdout:2948 read 0x575216 thru 0x57fc2c (0xaa17 bytes) 2024-09-14T09:06:26.022 INFO:teuthology.orchestra.run.smithi176.stdout:2951 trunc from 0x6feac77 to 0xe132fa8 2024-09-14T09:06:26.035 INFO:teuthology.orchestra.run.smithi176.stdout:2952 punch from 0x6221229 to 0x622e9e3, (0xd7ba bytes) 2024-09-14T09:06:26.035 INFO:teuthology.orchestra.run.smithi176.stdout:2954 writesame 0xbc320 thru 0xc6ff1 data_size 0xb(0xacd2 bytes) 2024-09-14T09:06:26.041 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:26.039+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:26.043 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:26.043 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:26.043 INFO:teuthology.orchestra.run.smithi176.stdout:2959 write 0x5394c40 thru 0x539797b (0x2d3c bytes) 2024-09-14T09:06:26.053 INFO:teuthology.orchestra.run.smithi176.stdout:2961 writesame 0x44605c7 thru 0x44645db data_size 0x5(0x4015 bytes) 2024-09-14T09:06:26.064 INFO:teuthology.orchestra.run.smithi176.stdout:2962 clone 23 order 22 su 65536 sc 11 2024-09-14T09:06:26.107 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:26.108 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:26.108 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:26.108 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:26.122 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:06:26.142 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:26.143+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:26.223 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is set 2024-09-14T09:06:26.242 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:26.243+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:26.248 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:26.248 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:26.248 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:26.248 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:26.248 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:26.262 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:06:26.352 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:26.355+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:26.429 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:26.429 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:26.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:26.430 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:26.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.430 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19520 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:26.430 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475212+0000", 2024-09-14T09:06:26.430 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.955187711000001, 2024-09-14T09:06:26.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067435885000000001, 2024-09-14T09:06:26.430 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.430 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.430 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:26.430 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:26.430 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:26.430 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:26.430 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19520 2024-09-14T09:06:26.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.431 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475212+0000", 2024-09-14T09:06:26.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475212+0000", 2024-09-14T09:06:26.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475213+0000", 2024-09-14T09:06:26.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0440000000000001e-06 2024-09-14T09:06:26.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475218+0000", 2024-09-14T09:06:26.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1930000000000001e-06 2024-09-14T09:06:26.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475237+0000", 2024-09-14T09:06:26.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9003000000000001e-05 2024-09-14T09:06:26.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475251+0000", 2024-09-14T09:06:26.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3720000000000001e-05 2024-09-14T09:06:26.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.541342+0000", 2024-09-14T09:06:26.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066091227000000002 2024-09-14T09:06:26.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.541492+0000", 2024-09-14T09:06:26.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000149221 2024-09-14T09:06:26.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542648+0000", 2024-09-14T09:06:26.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011564769999999999 2024-09-14T09:06:26.434 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.434 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.434 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.434 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19530 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:26.434 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475326+0000", 2024-09-14T09:06:26.434 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.955073528, 2024-09-14T09:06:26.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067030846000000005, 2024-09-14T09:06:26.435 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.435 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.435 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:26.435 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:26.435 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:26.435 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:26.435 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19530 2024-09-14T09:06:26.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.435 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475326+0000", 2024-09-14T09:06:26.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475326+0000", 2024-09-14T09:06:26.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475328+0000", 2024-09-14T09:06:26.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.133e-06 2024-09-14T09:06:26.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475358+0000", 2024-09-14T09:06:26.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0253e-05 2024-09-14T09:06:26.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475378+0000", 2024-09-14T09:06:26.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.033e-05 2024-09-14T09:06:26.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475405+0000", 2024-09-14T09:06:26.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6568999999999999e-05 2024-09-14T09:06:26.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.541989+0000", 2024-09-14T09:06:26.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066583956999999999 2024-09-14T09:06:26.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542215+0000", 2024-09-14T09:06:26.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022592200000000001 2024-09-14T09:06:26.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542357+0000", 2024-09-14T09:06:26.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014268199999999999 2024-09-14T09:06:26.438 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.438 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.438 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.439 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19531 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15073280~11919] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:26.439 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475425+0000", 2024-09-14T09:06:26.439 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.954975232999999, 2024-09-14T09:06:26.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067651759000000006, 2024-09-14T09:06:26.439 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.439 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.439 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:26.439 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:26.439 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:26.439 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:26.439 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19531 2024-09-14T09:06:26.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.439 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475425+0000", 2024-09-14T09:06:26.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475425+0000", 2024-09-14T09:06:26.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475426+0000", 2024-09-14T09:06:26.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2440000000000001e-06 2024-09-14T09:06:26.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475465+0000", 2024-09-14T09:06:26.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8643000000000003e-05 2024-09-14T09:06:26.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475485+0000", 2024-09-14T09:06:26.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0645999999999999e-05 2024-09-14T09:06:26.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475499+0000", 2024-09-14T09:06:26.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3312999999999999e-05 2024-09-14T09:06:26.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542389+0000", 2024-09-14T09:06:26.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066890885999999997 2024-09-14T09:06:26.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542481+0000", 2024-09-14T09:06:26.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1481e-05 2024-09-14T09:06:26.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543076+0000", 2024-09-14T09:06:26.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00059554599999999997 2024-09-14T09:06:26.443 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.443 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.443 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.443 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19532 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15335424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:26.443 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475521+0000", 2024-09-14T09:06:26.444 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.954878587000003, 2024-09-14T09:06:26.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067895159999999996, 2024-09-14T09:06:26.444 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.444 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.444 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:26.444 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:26.444 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:26.444 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:26.444 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19532 2024-09-14T09:06:26.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.444 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475521+0000", 2024-09-14T09:06:26.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475521+0000", 2024-09-14T09:06:26.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475523+0000", 2024-09-14T09:06:26.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.277e-06 2024-09-14T09:06:26.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475528+0000", 2024-09-14T09:06:26.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8629999999999996e-06 2024-09-14T09:06:26.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475546+0000", 2024-09-14T09:06:26.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7750000000000001e-05 2024-09-14T09:06:26.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475560+0000", 2024-09-14T09:06:26.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3763e-05 2024-09-14T09:06:26.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543144+0000", 2024-09-14T09:06:26.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067584489999999997 2024-09-14T09:06:26.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543261+0000", 2024-09-14T09:06:26.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011696699999999999 2024-09-14T09:06:26.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543416+0000", 2024-09-14T09:06:26.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015505 2024-09-14T09:06:26.447 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.448 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.448 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.448 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19533 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:26.448 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475578+0000", 2024-09-14T09:06:26.448 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.954821443999997, 2024-09-14T09:06:26.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068124984999999999, 2024-09-14T09:06:26.448 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.448 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.448 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:26.448 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:26.448 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:26.448 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:26.449 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19533 2024-09-14T09:06:26.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.449 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.449 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475578+0000", 2024-09-14T09:06:26.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.449 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475578+0000", 2024-09-14T09:06:26.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475580+0000", 2024-09-14T09:06:26.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.08e-06 2024-09-14T09:06:26.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475585+0000", 2024-09-14T09:06:26.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4970000000000001e-06 2024-09-14T09:06:26.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475602+0000", 2024-09-14T09:06:26.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7459999999999999e-05 2024-09-14T09:06:26.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475615+0000", 2024-09-14T09:06:26.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2339e-05 2024-09-14T09:06:26.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543466+0000", 2024-09-14T09:06:26.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067850745000000004 2024-09-14T09:06:26.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543577+0000", 2024-09-14T09:06:26.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000111908 2024-09-14T09:06:26.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543703+0000", 2024-09-14T09:06:26.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012595600000000001 2024-09-14T09:06:26.452 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.452 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.452 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.452 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19534 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:26.452 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475634+0000", 2024-09-14T09:06:26.452 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.954765668, 2024-09-14T09:06:26.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068358706000000005, 2024-09-14T09:06:26.452 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.453 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.453 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:26.453 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:26.453 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:26.453 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:26.453 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19534 2024-09-14T09:06:26.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.453 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475634+0000", 2024-09-14T09:06:26.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475634+0000", 2024-09-14T09:06:26.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475635+0000", 2024-09-14T09:06:26.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.187e-06 2024-09-14T09:06:26.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475641+0000", 2024-09-14T09:06:26.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5629999999999999e-06 2024-09-14T09:06:26.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475659+0000", 2024-09-14T09:06:26.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7649999999999999e-05 2024-09-14T09:06:26.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475671+0000", 2024-09-14T09:06:26.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2583e-05 2024-09-14T09:06:26.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543748+0000", 2024-09-14T09:06:26.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068076740999999996 2024-09-14T09:06:26.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.456 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543855+0000", 2024-09-14T09:06:26.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010676800000000001 2024-09-14T09:06:26.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.456 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.456 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543993+0000", 2024-09-14T09:06:26.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138214 2024-09-14T09:06:26.456 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.456 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.456 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.456 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19535 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:26.456 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475691+0000", 2024-09-14T09:06:26.457 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.954708658000001, 2024-09-14T09:06:26.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068607583999999999, 2024-09-14T09:06:26.457 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.457 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.457 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:26.457 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:26.457 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:26.457 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:26.457 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19535 2024-09-14T09:06:26.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.457 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475691+0000", 2024-09-14T09:06:26.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475691+0000", 2024-09-14T09:06:26.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475693+0000", 2024-09-14T09:06:26.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6270000000000001e-06 2024-09-14T09:06:26.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475699+0000", 2024-09-14T09:06:26.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9059999999999999e-06 2024-09-14T09:06:26.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475717+0000", 2024-09-14T09:06:26.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-09-14T09:06:26.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475730+0000", 2024-09-14T09:06:26.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.308e-05 2024-09-14T09:06:26.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544042+0000", 2024-09-14T09:06:26.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068312791999999997 2024-09-14T09:06:26.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544168+0000", 2024-09-14T09:06:26.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012549699999999999 2024-09-14T09:06:26.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544299+0000", 2024-09-14T09:06:26.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013094199999999999 2024-09-14T09:06:26.460 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.461 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.461 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.461 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19536 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:26.461 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475749+0000", 2024-09-14T09:06:26.461 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.954651378999998, 2024-09-14T09:06:26.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068917143, 2024-09-14T09:06:26.461 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.461 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.461 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:26.461 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:26.461 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:26.461 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:26.462 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19536 2024-09-14T09:06:26.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.462 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475749+0000", 2024-09-14T09:06:26.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475749+0000", 2024-09-14T09:06:26.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475750+0000", 2024-09-14T09:06:26.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0330000000000001e-06 2024-09-14T09:06:26.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475755+0000", 2024-09-14T09:06:26.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9740000000000001e-06 2024-09-14T09:06:26.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475772+0000", 2024-09-14T09:06:26.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7750000000000001e-05 2024-09-14T09:06:26.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475802+0000", 2024-09-14T09:06:26.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0190000000000001e-05 2024-09-14T09:06:26.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544345+0000", 2024-09-14T09:06:26.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068542140000000001 2024-09-14T09:06:26.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544454+0000", 2024-09-14T09:06:26.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109447 2024-09-14T09:06:26.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544666+0000", 2024-09-14T09:06:26.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000211609 2024-09-14T09:06:26.465 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.465 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.465 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.465 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19537 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:26.465 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475822+0000", 2024-09-14T09:06:26.465 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.954577422, 2024-09-14T09:06:26.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069185398999999995, 2024-09-14T09:06:26.465 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.465 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.466 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:26.466 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:26.466 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:26.466 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:26.466 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19537 2024-09-14T09:06:26.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.466 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475822+0000", 2024-09-14T09:06:26.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475822+0000", 2024-09-14T09:06:26.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475824+0000", 2024-09-14T09:06:26.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2559999999999999e-06 2024-09-14T09:06:26.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475829+0000", 2024-09-14T09:06:26.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6840000000000001e-06 2024-09-14T09:06:26.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475848+0000", 2024-09-14T09:06:26.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8583000000000001e-05 2024-09-14T09:06:26.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475861+0000", 2024-09-14T09:06:26.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2802999999999999e-05 2024-09-14T09:06:26.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544716+0000", 2024-09-14T09:06:26.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068855036999999994 2024-09-14T09:06:26.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544827+0000", 2024-09-14T09:06:26.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011113200000000001 2024-09-14T09:06:26.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545008+0000", 2024-09-14T09:06:26.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018090399999999999 2024-09-14T09:06:26.469 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.469 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.469 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.469 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19538 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:26.469 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475880+0000", 2024-09-14T09:06:26.470 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.954519746999999, 2024-09-14T09:06:26.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069740072, 2024-09-14T09:06:26.470 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.470 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.470 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:26.470 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:26.470 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:26.470 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:26.470 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19538 2024-09-14T09:06:26.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.470 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475880+0000", 2024-09-14T09:06:26.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475880+0000", 2024-09-14T09:06:26.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475881+0000", 2024-09-14T09:06:26.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.124e-06 2024-09-14T09:06:26.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475886+0000", 2024-09-14T09:06:26.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0130000000000001e-06 2024-09-14T09:06:26.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475904+0000", 2024-09-14T09:06:26.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7779999999999999e-05 2024-09-14T09:06:26.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475918+0000", 2024-09-14T09:06:26.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.385e-05 2024-09-14T09:06:26.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545065+0000", 2024-09-14T09:06:26.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069147307000000005 2024-09-14T09:06:26.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545431+0000", 2024-09-14T09:06:26.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036538400000000001 2024-09-14T09:06:26.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545620+0000", 2024-09-14T09:06:26.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000189614 2024-09-14T09:06:26.473 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.474 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.474 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.474 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19539 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:26.474 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475936+0000", 2024-09-14T09:06:26.474 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.954464162999997, 2024-09-14T09:06:26.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069860063999999999, 2024-09-14T09:06:26.474 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.474 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.474 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:26.474 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:26.474 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:26.474 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:26.474 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19539 2024-09-14T09:06:26.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.475 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475936+0000", 2024-09-14T09:06:26.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475936+0000", 2024-09-14T09:06:26.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475937+0000", 2024-09-14T09:06:26.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5960000000000001e-06 2024-09-14T09:06:26.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475943+0000", 2024-09-14T09:06:26.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2340000000000002e-06 2024-09-14T09:06:26.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475960+0000", 2024-09-14T09:06:26.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7592999999999999e-05 2024-09-14T09:06:26.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475986+0000", 2024-09-14T09:06:26.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6319e-05 2024-09-14T09:06:26.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545650+0000", 2024-09-14T09:06:26.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069663613999999999 2024-09-14T09:06:26.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545714+0000", 2024-09-14T09:06:26.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4209999999999997e-05 2024-09-14T09:06:26.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545796+0000", 2024-09-14T09:06:26.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1497999999999998e-05 2024-09-14T09:06:26.478 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.478 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.478 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.478 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19541 3.10 3:0ae7d8d9:::rbd_data.115e4c71d6e8.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 4194304~8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:26.478 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.497517+0000", 2024-09-14T09:06:26.478 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.932883091999997, 2024-09-14T09:06:26.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069874055000000004, 2024-09-14T09:06:26.478 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.479 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.479 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:26.479 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:26.479 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:26.479 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:26.479 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19541 2024-09-14T09:06:26.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.479 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.497517+0000", 2024-09-14T09:06:26.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.497517+0000", 2024-09-14T09:06:26.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.497520+0000", 2024-09-14T09:06:26.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0730000000000001e-06 2024-09-14T09:06:26.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.504875+0000", 2024-09-14T09:06:26.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007355136 2024-09-14T09:06:26.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.504926+0000", 2024-09-14T09:06:26.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1146000000000001e-05 2024-09-14T09:06:26.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.504950+0000", 2024-09-14T09:06:26.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3638999999999998e-05 2024-09-14T09:06:26.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.505139+0000", 2024-09-14T09:06:26.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018904300000000001 2024-09-14T09:06:26.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.505402+0000", 2024-09-14T09:06:26.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026354699999999999 2024-09-14T09:06:26.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:26.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.506208+0000", 2024-09-14T09:06:26.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00080583399999999996 2024-09-14T09:06:26.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:26.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.566904+0000", 2024-09-14T09:06:26.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.060695510000000001 2024-09-14T09:06:26.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:26.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.567219+0000", 2024-09-14T09:06:26.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031509000000000002 2024-09-14T09:06:26.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:26.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.567319+0000", 2024-09-14T09:06:26.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9804999999999997e-05 2024-09-14T09:06:26.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.567391+0000", 2024-09-14T09:06:26.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2231999999999996e-05 2024-09-14T09:06:26.483 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.484 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.484 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.484 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:30890 3.10 3:0ac3ecfb:::rbd_data.115ebc3c67d1.0000000000000004:c [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:26.484 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.069693+0000", 2024-09-14T09:06:26.484 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.360707302999998, 2024-09-14T09:06:26.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066892069999999998, 2024-09-14T09:06:26.484 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.484 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.484 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:26.484 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:26.484 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:26.484 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:26.485 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 30890 2024-09-14T09:06:26.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.485 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069693+0000", 2024-09-14T09:06:26.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069693+0000", 2024-09-14T09:06:26.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069694+0000", 2024-09-14T09:06:26.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0100000000000003e-07 2024-09-14T09:06:26.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069698+0000", 2024-09-14T09:06:26.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2100000000000003e-06 2024-09-14T09:06:26.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069711+0000", 2024-09-14T09:06:26.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3332e-05 2024-09-14T09:06:26.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069721+0000", 2024-09-14T09:06:26.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0424999999999999e-05 2024-09-14T09:06:26.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136469+0000", 2024-09-14T09:06:26.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066747714999999999 2024-09-14T09:06:26.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136540+0000", 2024-09-14T09:06:26.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1077999999999999e-05 2024-09-14T09:06:26.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136585+0000", 2024-09-14T09:06:26.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4409000000000001e-05 2024-09-14T09:06:26.488 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.488 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.488 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.488 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:30891 3.10 3:0ac3ecfb:::rbd_data.115ebc3c67d1.0000000000000004:c [read 212992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:26.488 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.069740+0000", 2024-09-14T09:06:26.488 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.360660303000003, 2024-09-14T09:06:26.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066940186999999998, 2024-09-14T09:06:26.489 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.489 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.489 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:26.489 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:26.489 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:26.489 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:26.489 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 30891 2024-09-14T09:06:26.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.489 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069740+0000", 2024-09-14T09:06:26.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069740+0000", 2024-09-14T09:06:26.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070603+0000", 2024-09-14T09:06:26.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00086369599999999995 2024-09-14T09:06:26.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070612+0000", 2024-09-14T09:06:26.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3170000000000002e-06 2024-09-14T09:06:26.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070631+0000", 2024-09-14T09:06:26.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8936999999999998e-05 2024-09-14T09:06:26.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070645+0000", 2024-09-14T09:06:26.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4545999999999999e-05 2024-09-14T09:06:26.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136603+0000", 2024-09-14T09:06:26.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.065957551000000003 2024-09-14T09:06:26.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136644+0000", 2024-09-14T09:06:26.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1671000000000001e-05 2024-09-14T09:06:26.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136680+0000", 2024-09-14T09:06:26.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5469000000000002e-05 2024-09-14T09:06:26.492 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.492 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.492 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.493 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:37542 3.10 3:0e658405:::rbd_data.115eb07a8abd.000000000000000d:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:26.493 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.443185+0000", 2024-09-14T09:06:26.493 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.987215277000001, 2024-09-14T09:06:26.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.091143627000000005, 2024-09-14T09:06:26.493 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.493 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.493 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:26.493 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:26.493 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:26.493 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:26.493 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37542 2024-09-14T09:06:26.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.493 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443185+0000", 2024-09-14T09:06:26.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443185+0000", 2024-09-14T09:06:26.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443187+0000", 2024-09-14T09:06:26.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.367e-06 2024-09-14T09:06:26.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443199+0000", 2024-09-14T09:06:26.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2153e-05 2024-09-14T09:06:26.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443225+0000", 2024-09-14T09:06:26.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5457e-05 2024-09-14T09:06:26.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443248+0000", 2024-09-14T09:06:26.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2963e-05 2024-09-14T09:06:26.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443400+0000", 2024-09-14T09:06:26.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015241199999999999 2024-09-14T09:06:26.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443542+0000", 2024-09-14T09:06:26.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014224600000000001 2024-09-14T09:06:26.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:26.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.448965+0000", 2024-09-14T09:06:26.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054226839999999997 2024-09-14T09:06:26.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:26.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534136+0000", 2024-09-14T09:06:26.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.085170939000000001 2024-09-14T09:06:26.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:26.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534164+0000", 2024-09-14T09:06:26.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7863000000000001e-05 2024-09-14T09:06:26.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:26.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534250+0000", 2024-09-14T09:06:26.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5876000000000002e-05 2024-09-14T09:06:26.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534328+0000", 2024-09-14T09:06:26.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8666999999999997e-05 2024-09-14T09:06:26.498 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.498 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.498 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.498 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:37542 3.10 e48/28 ondisk, result = 0)", 2024-09-14T09:06:26.499 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.464008+0000", 2024-09-14T09:06:26.499 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.966391867999999, 2024-09-14T09:06:26.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.070141355000000002, 2024-09-14T09:06:26.499 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.499 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.499 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:26.499 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464008+0000", 2024-09-14T09:06:26.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464008+0000", 2024-09-14T09:06:26.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464010+0000", 2024-09-14T09:06:26.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.547e-06 2024-09-14T09:06:26.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464017+0000", 2024-09-14T09:06:26.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4930000000000001e-06 2024-09-14T09:06:26.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464047+0000", 2024-09-14T09:06:26.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0090000000000002e-05 2024-09-14T09:06:26.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464061+0000", 2024-09-14T09:06:26.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4003e-05 2024-09-14T09:06:26.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534107+0000", 2024-09-14T09:06:26.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.070045590000000005 2024-09-14T09:06:26.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534126+0000", 2024-09-14T09:06:26.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8893e-05 2024-09-14T09:06:26.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534149+0000", 2024-09-14T09:06:26.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3739000000000001e-05 2024-09-14T09:06:26.502 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.502 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.502 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.502 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38003 3.10 3:0e658405:::rbd_data.115eb07a8abd.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:26.503 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:57.392070+0000", 2024-09-14T09:06:26.503 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.038330093999999, 2024-09-14T09:06:26.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11697451, 2024-09-14T09:06:26.503 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.503 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.503 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:26.503 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:26.503 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:26.503 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:26.503 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38003 2024-09-14T09:06:26.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.503 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392070+0000", 2024-09-14T09:06:26.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392070+0000", 2024-09-14T09:06:26.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392072+0000", 2024-09-14T09:06:26.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6529999999999998e-06 2024-09-14T09:06:26.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392086+0000", 2024-09-14T09:06:26.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.332e-05 2024-09-14T09:06:26.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392139+0000", 2024-09-14T09:06:26.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2815999999999997e-05 2024-09-14T09:06:26.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392155+0000", 2024-09-14T09:06:26.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6796999999999999e-05 2024-09-14T09:06:26.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392296+0000", 2024-09-14T09:06:26.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014107900000000001 2024-09-14T09:06:26.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392443+0000", 2024-09-14T09:06:26.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014654499999999999 2024-09-14T09:06:26.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:26.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.399418+0000", 2024-09-14T09:06:26.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069751049999999997 2024-09-14T09:06:26.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:26.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508607+0000", 2024-09-14T09:06:26.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10918908400000001 2024-09-14T09:06:26.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:26.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508748+0000", 2024-09-14T09:06:26.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014079699999999999 2024-09-14T09:06:26.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:26.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508921+0000", 2024-09-14T09:06:26.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017345899999999999 2024-09-14T09:06:26.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.509044+0000", 2024-09-14T09:06:26.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000122855 2024-09-14T09:06:26.508 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.508 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.508 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.508 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:38003 3.10 e48/28 ondisk, result = 0)", 2024-09-14T09:06:26.509 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:57.413977+0000", 2024-09-14T09:06:26.509 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.016422871, 2024-09-14T09:06:26.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.095088000000000006, 2024-09-14T09:06:26.509 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.509 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.509 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:26.509 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413977+0000", 2024-09-14T09:06:26.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413977+0000", 2024-09-14T09:06:26.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413979+0000", 2024-09-14T09:06:26.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.933e-06 2024-09-14T09:06:26.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413988+0000", 2024-09-14T09:06:26.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2769999999999994e-06 2024-09-14T09:06:26.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.414021+0000", 2024-09-14T09:06:26.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2589999999999998e-05 2024-09-14T09:06:26.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.414040+0000", 2024-09-14T09:06:26.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8796000000000001e-05 2024-09-14T09:06:26.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508701+0000", 2024-09-14T09:06:26.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.094661067000000002 2024-09-14T09:06:26.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508729+0000", 2024-09-14T09:06:26.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8677e-05 2024-09-14T09:06:26.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.509065+0000", 2024-09-14T09:06:26.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033565999999999999 2024-09-14T09:06:26.512 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.512 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.512 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.512 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65139 3.a 3:54c90b7d:::rbd_data.115e21c31fcf.0000000000000008:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:06:26.513 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:06:26.513 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 5.0506871670000004, 2024-09-14T09:06:26.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14429895700000001, 2024-09-14T09:06:26.513 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.513 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.513 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:26.513 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:26.513 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:26.513 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:26.513 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65139 2024-09-14T09:06:26.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.513 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:06:26.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:06:26.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379715+0000", 2024-09-14T09:06:26.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.576e-06 2024-09-14T09:06:26.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379724+0000", 2024-09-14T09:06:26.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1779999999999992e-06 2024-09-14T09:06:26.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379746+0000", 2024-09-14T09:06:26.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1661000000000001e-05 2024-09-14T09:06:26.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379759+0000", 2024-09-14T09:06:26.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.324e-05 2024-09-14T09:06:26.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379882+0000", 2024-09-14T09:06:26.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012239400000000001 2024-09-14T09:06:26.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.380023+0000", 2024-09-14T09:06:26.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000141402 2024-09-14T09:06:26.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:26.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.391553+0000", 2024-09-14T09:06:26.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011530197000000001 2024-09-14T09:06:26.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:26.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523927+0000", 2024-09-14T09:06:26.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.132373935 2024-09-14T09:06:26.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:26.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523941+0000", 2024-09-14T09:06:26.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3326000000000001e-05 2024-09-14T09:06:26.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:26.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523977+0000", 2024-09-14T09:06:26.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6185e-05 2024-09-14T09:06:26.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.524012+0000", 2024-09-14T09:06:26.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4863000000000001e-05 2024-09-14T09:06:26.518 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.518 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.518 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.518 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:65139 3.a e65/34 ondisk, result = 0)", 2024-09-14T09:06:26.518 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.408501+0000", 2024-09-14T09:06:26.518 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 5.02189861, 2024-09-14T09:06:26.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.115431275, 2024-09-14T09:06:26.518 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:26.518 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:26.519 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:26.519 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:26.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:26.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408501+0000", 2024-09-14T09:06:26.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:26.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408501+0000", 2024-09-14T09:06:26.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:26.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:26.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408504+0000", 2024-09-14T09:06:26.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2819999999999999e-06 2024-09-14T09:06:26.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:26.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408512+0000", 2024-09-14T09:06:26.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7530000000000007e-06 2024-09-14T09:06:26.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:26.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408534+0000", 2024-09-14T09:06:26.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1993000000000001e-05 2024-09-14T09:06:26.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408550+0000", 2024-09-14T09:06:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5988e-05 2024-09-14T09:06:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523914+0000", 2024-09-14T09:06:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.115363542 2024-09-14T09:06:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523922+0000", 2024-09-14T09:06:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5529999999999993e-06 2024-09-14T09:06:26.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:26.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:26.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:26.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523933+0000", 2024-09-14T09:06:26.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0164e-05 2024-09-14T09:06:26.522 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.522 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.522 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.522 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:26.522 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:26.522 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:26.524 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:06:26.524 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:26.527+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:26.586 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:06:26.620 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-14T09:06:26.624 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:26.627+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:26.704 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:26.704 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:26.704 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:26.705 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:26.719 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:06:26.746 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:26.746 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:26.763 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:26.763 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:26.764 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:26.767+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:26.780 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:26.780 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:26.798 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:26.798 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:26.810 INFO:teuthology.orchestra.run.smithi176.stdout:2963 read 0x86af428 thru 0x86b036b (0xf44 bytes) 2024-09-14T09:06:26.810 INFO:teuthology.orchestra.run.smithi176.stdout:2964 write 0xda6f2c9 thru 0xda7c59a (0xd2d2 bytes) 2024-09-14T09:06:26.817 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:26.817 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:26.824 INFO:teuthology.orchestra.run.smithi176.stdout:2966 read 0x253844c thru 0x2546c5b (0xe810 bytes) 2024-09-14T09:06:26.824 INFO:teuthology.orchestra.run.smithi176.stdout:2967 read 0xa19e844 thru 0xa1a6ebd (0x867a bytes) 2024-09-14T09:06:26.824 INFO:teuthology.orchestra.run.smithi176.stdout:2969 writesame 0x6293c38 thru 0x629a642 data_size 0x1(0x6a0b bytes) 2024-09-14T09:06:26.826 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:26.827+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:26.839 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:26.839 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:26.858 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:26.859 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:26.878 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:26.878 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:26.892 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:26.892 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:26.894 INFO:teuthology.orchestra.run.smithi176.stdout:2970 read 0x512c1ca thru 0x5137b74 (0xb9ab bytes) 2024-09-14T09:06:26.894 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:26.894 INFO:teuthology.orchestra.run.smithi176.stdout:2972 write 0xc61c1d9 thru 0xc61dabb (0x18e3 bytes) 2024-09-14T09:06:26.899 INFO:teuthology.orchestra.run.smithi176.stdout:2973 write 0xcd31ec8 thru 0xcd3d0e5 (0xb21e bytes) 2024-09-14T09:06:26.908 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:06:26.909 INFO:teuthology.orchestra.run.smithi176.stdout:2974 write 0xccc20e2 thru 0xccd034f (0xe26e bytes) 2024-09-14T09:06:26.918 INFO:teuthology.orchestra.run.smithi176.stdout:2976 trunc from 0xe132fa8 to 0xd34598e 2024-09-14T09:06:26.933 INFO:teuthology.orchestra.run.smithi176.stdout:2977 write 0x2280116 thru 0x2284228 (0x4113 bytes) 2024-09-14T09:06:26.946 INFO:teuthology.orchestra.run.smithi176.stdout:2978 writesame 0xc7f9c10 thru 0xc7fec10 data_size 0x1(0x5001 bytes) 2024-09-14T09:06:26.959 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:26.963+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:26.991 INFO:teuthology.orchestra.run.smithi176.stdout:2979 punch from 0x2cf211d to 0x2cf89d7, (0x68ba bytes) 2024-09-14T09:06:27.010 INFO:teuthology.orchestra.run.smithi176.stdout:2981 trunc from 0xd34598e to 0x1631d74 2024-09-14T09:06:27.034 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:27.027+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:27.051 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:27.051 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:27.074 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:06:27.101 INFO:teuthology.orchestra.run.smithi176.stdout:2982 trunc from 0x1631d74 to 0x1c62007 2024-09-14T09:06:27.122 INFO:teuthology.orchestra.run.smithi176.stdout:2983 write 0x6145b2c thru 0x6151674 (0xbb49 bytes) 2024-09-14T09:06:27.127 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:27.128 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:27.127+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:27.138 INFO:teuthology.orchestra.run.smithi176.stdout:2985 trunc from 0x6151675 to 0xcd3556a 2024-09-14T09:06:27.149 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:27.149 INFO:teuthology.orchestra.run.smithi176.stdout:2992 writesame 0x312e0e1 thru 0x3138f9d data_size 0x5d(0xaebd bytes) 2024-09-14T09:06:27.167 INFO:teuthology.orchestra.run.smithi176.stdout:2993 writesame 0xd6e7a66 thru 0xd6f4bb0 data_size 0x1(0xd14b bytes) 2024-09-14T09:06:27.168 INFO:teuthology.orchestra.run.smithi176.stdout:2994 read 0x419b36b thru 0x419edc8 (0x3a5e bytes) 2024-09-14T09:06:27.168 INFO:teuthology.orchestra.run.smithi176.stdout:2995 write 0xa251d4 thru 0xa2d8e2 (0x870f bytes) 2024-09-14T09:06:27.173 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:27.173 INFO:teuthology.orchestra.run.smithi176.stdout:2999 write 0x7abfc53 thru 0x7acaff3 (0xb3a1 bytes) 2024-09-14T09:06:27.181 INFO:teuthology.orchestra.run.smithi176.stdout:3000 punch from 0xc6520ce to 0xc65ccdb, (0xac0d bytes) 2024-09-14T09:06:27.181 INFO:teuthology.orchestra.run.smithi176.stdout:3001 write 0x50de987 thru 0x50e407f (0x56f9 bytes) 2024-09-14T09:06:27.188 INFO:teuthology.orchestra.run.smithi176.stdout:3002 write 0x744ce64 thru 0x74514c9 (0x4666 bytes) 2024-09-14T09:06:27.204 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:27.204 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:27.218 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:06:27.262 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:27.263+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:27.329 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:27.327+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:27.382 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:27.382 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:27.396 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:06:27.401 INFO:teuthology.orchestra.run.smithi176.stdout:3003 write 0x18f33b2 thru 0x18f3e0b (0xa5a bytes) 2024-09-14T09:06:27.405 INFO:teuthology.orchestra.run.smithi176.stdout:3004 write 0xc88adca thru 0xc890398 (0x55cf bytes) 2024-09-14T09:06:27.414 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:27.414 INFO:teuthology.orchestra.run.smithi176.stdout:3006 read 0xb89b6b7 thru 0xb8a7dbb (0xc705 bytes) 2024-09-14T09:06:27.414 INFO:teuthology.orchestra.run.smithi176.stdout:3008 write 0xcde10f8 thru 0xcde7321 (0x622a bytes) 2024-09-14T09:06:27.422 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:27.422 INFO:teuthology.orchestra.run.smithi176.stdout:3010 read 0xc98c00d thru 0xc991681 (0x5675 bytes) 2024-09-14T09:06:27.422 INFO:teuthology.orchestra.run.smithi176.stdout:3012 write 0x2259089 thru 0x2265cc3 (0xcc3b bytes) 2024-09-14T09:06:27.428 INFO:teuthology.orchestra.run.smithi176.stdout:3013 read 0xbc435ad thru 0xbc48190 (0x4be4 bytes) 2024-09-14T09:06:27.429 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:27.427+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:27.440 INFO:teuthology.orchestra.run.smithi176.stdout:3014 write 0xdd55a54 thru 0xdd6327e (0xd82b bytes) 2024-09-14T09:06:27.446 INFO:teuthology.orchestra.run.smithi176.stdout:3016 write 0x263e828 thru 0x26408bc (0x2095 bytes) 2024-09-14T09:06:27.449 INFO:teuthology.orchestra.run.smithi176.stdout:3017 read 0x50d9d3e thru 0x50e2a0f (0x8cd2 bytes) 2024-09-14T09:06:27.451 INFO:teuthology.orchestra.run.smithi176.stdout:3018 trunc from 0xdd6327f to 0xd79bd0b 2024-09-14T09:06:27.474 INFO:teuthology.orchestra.run.smithi176.stdout:3021 write 0x6ba5821 thru 0x6bb51cf (0xf9af bytes) 2024-09-14T09:06:27.482 INFO:teuthology.orchestra.run.smithi176.stdout:3022 writesame 0x3e27ca8 thru 0x3e30b42 data_size 0x2b(0x8e9b bytes) 2024-09-14T09:06:27.490 INFO:teuthology.orchestra.run.smithi176.stdout:3024 write 0xcd5033a thru 0xcd5805e (0x7d25 bytes) 2024-09-14T09:06:27.496 INFO:teuthology.orchestra.run.smithi176.stdout:3025 punch from 0xc5054f8 to 0xc50974a, (0x4252 bytes) 2024-09-14T09:06:27.497 INFO:teuthology.orchestra.run.smithi176.stdout:3027 write 0x8b22ed5 thru 0x8b24c43 (0x1d6f bytes) 2024-09-14T09:06:27.502 INFO:teuthology.orchestra.run.smithi176.stdout:3028 punch from 0xc723b7d to 0xc72a00e, (0x6491 bytes) 2024-09-14T09:06:27.503 INFO:teuthology.orchestra.run.smithi176.stdout:3031 read 0x308625b thru 0x309175b (0xb501 bytes) 2024-09-14T09:06:27.503 INFO:teuthology.orchestra.run.smithi176.stdout:3032 clone 24 order 24 su 4194304 sc 2 2024-09-14T09:06:27.525 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:27.525 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:27.530 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:27.531+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:27.540 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:06:27.631 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:27.635+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:27.699 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:27.699 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:27.713 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:06:27.732 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:27.735+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:27.830 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:27.831 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:27.833 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:27.831+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:27.845 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:06:27.934 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:27.935+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:28.011 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:28.011 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:28.025 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:06:28.034 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:28.035+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:28.135 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:28.139+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:28.149 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:28.149 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:28.163 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:06:28.236 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:28.235+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:28.252 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T09:06:28.291 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:28.291 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:28.310 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:06:28.337 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:28.339+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:28.337 INFO:teuthology.orchestra.run.smithi176.stdout:leaving image image_client.0-clone23 intact 2024-09-14T09:06:28.438 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:28.435+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:28.475 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:28.475 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:28.491 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:06:28.526 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-14T09:06:28.538 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:28.539+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:28.639 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:28.639+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:28.657 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:28.657 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:28.673 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:06:28.740 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:28.743+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:28.838 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:28.838 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:28.841 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:28.843+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:28.855 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:06:28.896 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:06:28.941 INFO:teuthology.orchestra.run.smithi176.stdout:3034 read 0x25a758a thru 0x25aa488 (0x2eff bytes) 2024-09-14T09:06:28.941 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:28.939+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:28.951 INFO:teuthology.orchestra.run.smithi176.stdout:3035 trunc from 0xd79bd0b to 0x82b65e1 2024-09-14T09:06:28.970 INFO:teuthology.orchestra.run.smithi176.stdout:3036 read 0x4912779 thru 0x491b126 (0x89ae bytes) 2024-09-14T09:06:28.970 INFO:teuthology.orchestra.run.smithi176.stdout:3037 read 0x479943e thru 0x479e10b (0x4cce bytes) 2024-09-14T09:06:28.970 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:28.970 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:28.981 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:28.981 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:28.983 INFO:teuthology.orchestra.run.smithi176.stdout:3041 writesame 0xc035e85 thru 0xc0430ea data_size 0x2f(0xd266 bytes) 2024-09-14T09:06:28.983 INFO:teuthology.orchestra.run.smithi176.stdout:3042 read 0x416a4ad thru 0x4170646 (0x619a bytes) 2024-09-14T09:06:28.983 INFO:teuthology.orchestra.run.smithi176.stdout:3043 writesame 0x15efd22 thru 0x15fa1a5 data_size 0x2(0xa484 bytes) 2024-09-14T09:06:28.999 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:06:29.042 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:29.043+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:29.064 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:29.064 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:29.079 INFO:teuthology.orchestra.run.smithi176.stdout:3044 write 0x9a66066 thru 0x9a6e145 (0x80e0 bytes) 2024-09-14T09:06:29.082 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:29.082 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:29.083 INFO:teuthology.orchestra.run.smithi176.stdout:3045 write 0x75e7249 thru 0x75eb626 (0x43de bytes) 2024-09-14T09:06:29.089 INFO:teuthology.orchestra.run.smithi176.stdout:3046 read 0x3f81e8 thru 0x4012d6 (0x90ef bytes) 2024-09-14T09:06:29.095 INFO:teuthology.orchestra.run.smithi176.stdout:3047 trunc from 0xc0430eb to 0x877582d 2024-09-14T09:06:29.100 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:29.101 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:29.102 INFO:teuthology.orchestra.run.smithi176.stdout:3051 writesame 0x56f71f5 thru 0x570605c data_size 0xc(0xee68 bytes) 2024-09-14T09:06:29.118 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:29.118 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:29.130 INFO:teuthology.orchestra.run.smithi176.stdout:3054 write 0xa06a365 thru 0xa0786f7 (0xe393 bytes) 2024-09-14T09:06:29.140 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:29.140 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:29.141 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:29.141 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:29.141 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:29.141 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:29.143 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:29.147+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:29.149 INFO:teuthology.orchestra.run.smithi176.stdout:3055 trunc from 0xa0786f8 to 0xf26e33 2024-09-14T09:06:29.156 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:06:29.165 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:29.165 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:29.185 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:29.186 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:29.197 INFO:teuthology.orchestra.run.smithi176.stdout:3058 read 0x27e345 thru 0x2810f7 (0x2db3 bytes) 2024-09-14T09:06:29.207 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:29.207 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:29.220 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:06:29.220 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:06:29.220 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T09:06:29.220 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T09:06:29.225 INFO:teuthology.orchestra.run.smithi176.stdout:3059 write 0xeb81713 thru 0xeb8ede0 (0xd6ce bytes) 2024-09-14T09:06:29.231 INFO:teuthology.orchestra.run.smithi176.stdout:3060 read 0x6401dca thru 0x640680d (0x4a44 bytes) 2024-09-14T09:06:29.232 INFO:teuthology.orchestra.run.smithi176.stdout:3061 write 0x2579861 thru 0x2589442 (0xfbe2 bytes) 2024-09-14T09:06:29.236 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is set 2024-09-14T09:06:29.243 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:29.243+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:29.244 INFO:teuthology.orchestra.run.smithi176.stdout:3063 punch from 0x3466e51 to 0x3469df5, (0x2fa4 bytes) 2024-09-14T09:06:29.245 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:29.245 INFO:teuthology.orchestra.run.smithi176.stdout:3065 writesame 0x61297b9 thru 0x6137bb5 data_size 0xf(0xe3fd bytes) 2024-09-14T09:06:29.257 INFO:teuthology.orchestra.run.smithi176.stdout:3068 write 0x44867dd thru 0x4488d8f (0x25b3 bytes) 2024-09-14T09:06:29.263 INFO:teuthology.orchestra.run.smithi176.stdout:3070 trunc from 0xeb8ede1 to 0xd153bd3 2024-09-14T09:06:29.275 INFO:teuthology.orchestra.run.smithi176.stdout:3071 writesame 0x9f1e963 thru 0x9f2e59e data_size 0x6(0xfc3c bytes) 2024-09-14T09:06:29.301 INFO:teuthology.orchestra.run.smithi176.stdout:3072 read 0x7e83ca8 thru 0x7e8d145 (0x949e bytes) 2024-09-14T09:06:29.301 INFO:teuthology.orchestra.run.smithi176.stdout:3074 writesame 0x1d059a7 thru 0x1d14c7f data_size 0x45(0xf2d9 bytes) 2024-09-14T09:06:29.304 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:29.304 INFO:teuthology.orchestra.run.smithi176.stdout:3078 read 0x939edc2 thru 0x93a9420 (0xa65f bytes) 2024-09-14T09:06:29.315 INFO:teuthology.orchestra.run.smithi176.stdout:3079 trunc from 0xd153bd3 to 0x5577c1f 2024-09-14T09:06:29.328 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:29.328 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:29.328 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:29.328 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:29.328 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:29.331 INFO:teuthology.orchestra.run.smithi176.stdout:3080 writesame 0x37ed533 thru 0x37f46f5 data_size 0x1(0x71c3 bytes) 2024-09-14T09:06:29.343 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:06:29.344 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:29.343+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:29.395 INFO:teuthology.orchestra.run.smithi176.stdout:3081 punch from 0x1c11eb1 to 0x1c15e20, (0x3f6f bytes) 2024-09-14T09:06:29.395 INFO:teuthology.orchestra.run.smithi176.stdout:3082 read 0xbc21f thru 0xc80ce (0xbeb0 bytes) 2024-09-14T09:06:29.414 INFO:teuthology.orchestra.run.smithi176.stdout:3084 write 0x8ca65c7 thru 0x8cafdd9 (0x9813 bytes) 2024-09-14T09:06:29.420 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:29.420 INFO:teuthology.orchestra.run.smithi176.stdout:3086 punch from 0x6ecd74d to 0x6ecfb78, (0x242b bytes) 2024-09-14T09:06:29.420 INFO:teuthology.orchestra.run.smithi176.stdout:3087 punch from 0x1c43d8f to 0x1c4f671, (0xb8e2 bytes) 2024-09-14T09:06:29.434 INFO:teuthology.orchestra.run.smithi176.stdout:3089 trunc from 0x8cafdda to 0x1288559 2024-09-14T09:06:29.446 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:29.447+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:29.456 INFO:teuthology.orchestra.run.smithi176.stdout:3090 read 0x1090437 thru 0x10968ea (0x64b4 bytes) 2024-09-14T09:06:29.467 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:29.468 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:29.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:29.468 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:29.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.468 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:9982 3.1a 3:5af3e476:::rbd_data.115eadec874.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:06:29.468 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:29.468 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.871580356999999, 2024-09-14T09:06:29.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042276341000000002, 2024-09-14T09:06:29.468 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.468 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.468 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:29.468 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.468 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.469 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.469 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 9982 2024-09-14T09:06:29.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.469 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:29.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:06:29.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597970+0000", 2024-09-14T09:06:29.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0599999999999999e-07 2024-09-14T09:06:29.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597974+0000", 2024-09-14T09:06:29.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2649999999999998e-06 2024-09-14T09:06:29.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597986+0000", 2024-09-14T09:06:29.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1019000000000001e-05 2024-09-14T09:06:29.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597991+0000", 2024-09-14T09:06:29.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5049999999999998e-06 2024-09-14T09:06:29.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.598083+0000", 2024-09-14T09:06:29.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2062e-05 2024-09-14T09:06:29.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.598227+0000", 2024-09-14T09:06:29.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000143609 2024-09-14T09:06:29.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:29.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.603107+0000", 2024-09-14T09:06:29.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048805819999999996 2024-09-14T09:06:29.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:29.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640002+0000", 2024-09-14T09:06:29.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.036894641999999998 2024-09-14T09:06:29.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:29.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640077+0000", 2024-09-14T09:06:29.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4927999999999998e-05 2024-09-14T09:06:29.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:29.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640174+0000", 2024-09-14T09:06:29.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7417999999999995e-05 2024-09-14T09:06:29.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640246+0000", 2024-09-14T09:06:29.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1505000000000001e-05 2024-09-14T09:06:29.473 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.473 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.473 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.473 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53587 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1212416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:29.474 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508126+0000", 2024-09-14T09:06:29.474 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.961424203, 2024-09-14T09:06:29.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033430271999999997, 2024-09-14T09:06:29.474 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.474 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.474 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:29.474 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.474 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.474 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.474 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53587 2024-09-14T09:06:29.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.474 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508126+0000", 2024-09-14T09:06:29.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508126+0000", 2024-09-14T09:06:29.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508127+0000", 2024-09-14T09:06:29.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.11e-06 2024-09-14T09:06:29.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508132+0000", 2024-09-14T09:06:29.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4770000000000002e-06 2024-09-14T09:06:29.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508151+0000", 2024-09-14T09:06:29.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9032999999999999e-05 2024-09-14T09:06:29.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508164+0000", 2024-09-14T09:06:29.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2673e-05 2024-09-14T09:06:29.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541369+0000", 2024-09-14T09:06:29.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033205367999999999 2024-09-14T09:06:29.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541455+0000", 2024-09-14T09:06:29.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5618000000000006e-05 2024-09-14T09:06:29.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541556+0000", 2024-09-14T09:06:29.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100993 2024-09-14T09:06:29.477 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.477 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.477 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.478 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53596 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:29.478 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508184+0000", 2024-09-14T09:06:29.478 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.96136557, 2024-09-14T09:06:29.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033494411000000002, 2024-09-14T09:06:29.478 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.478 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.478 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:29.478 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.478 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.478 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.478 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53596 2024-09-14T09:06:29.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.478 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508184+0000", 2024-09-14T09:06:29.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508184+0000", 2024-09-14T09:06:29.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508185+0000", 2024-09-14T09:06:29.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.217e-06 2024-09-14T09:06:29.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508191+0000", 2024-09-14T09:06:29.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6330000000000001e-06 2024-09-14T09:06:29.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508209+0000", 2024-09-14T09:06:29.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7804000000000001e-05 2024-09-14T09:06:29.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508223+0000", 2024-09-14T09:06:29.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3783e-05 2024-09-14T09:06:29.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541582+0000", 2024-09-14T09:06:29.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033359403000000003 2024-09-14T09:06:29.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541622+0000", 2024-09-14T09:06:29.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9937000000000001e-05 2024-09-14T09:06:29.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541679+0000", 2024-09-14T09:06:29.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6634e-05 2024-09-14T09:06:29.482 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.482 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.482 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.482 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53599 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:29.482 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508241+0000", 2024-09-14T09:06:29.482 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.961308697, 2024-09-14T09:06:29.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033611720999999997, 2024-09-14T09:06:29.482 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.482 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.482 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:29.482 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.483 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.483 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.483 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53599 2024-09-14T09:06:29.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.483 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508241+0000", 2024-09-14T09:06:29.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508241+0000", 2024-09-14T09:06:29.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508242+0000", 2024-09-14T09:06:29.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1230000000000001e-06 2024-09-14T09:06:29.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508247+0000", 2024-09-14T09:06:29.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8439999999999997e-06 2024-09-14T09:06:29.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508276+0000", 2024-09-14T09:06:29.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9363e-05 2024-09-14T09:06:29.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508290+0000", 2024-09-14T09:06:29.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.364e-05 2024-09-14T09:06:29.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541712+0000", 2024-09-14T09:06:29.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033421713999999998 2024-09-14T09:06:29.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541785+0000", 2024-09-14T09:06:29.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3696999999999996e-05 2024-09-14T09:06:29.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541853+0000", 2024-09-14T09:06:29.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7340000000000002e-05 2024-09-14T09:06:29.486 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.486 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.486 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.486 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53601 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:29.486 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508310+0000", 2024-09-14T09:06:29.487 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.961240081, 2024-09-14T09:06:29.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033872092999999999, 2024-09-14T09:06:29.487 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.487 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.487 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:29.487 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.487 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.487 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.487 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53601 2024-09-14T09:06:29.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.487 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508310+0000", 2024-09-14T09:06:29.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508310+0000", 2024-09-14T09:06:29.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508311+0000", 2024-09-14T09:06:29.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.11e-06 2024-09-14T09:06:29.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508317+0000", 2024-09-14T09:06:29.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.127e-06 2024-09-14T09:06:29.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508335+0000", 2024-09-14T09:06:29.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8097e-05 2024-09-14T09:06:29.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508363+0000", 2024-09-14T09:06:29.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8089000000000002e-05 2024-09-14T09:06:29.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.541929+0000", 2024-09-14T09:06:29.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033566046000000002 2024-09-14T09:06:29.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542062+0000", 2024-09-14T09:06:29.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013268000000000001 2024-09-14T09:06:29.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542182+0000", 2024-09-14T09:06:29.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119944 2024-09-14T09:06:29.490 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.490 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.491 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.491 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53603 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:29.491 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508409+0000", 2024-09-14T09:06:29.491 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.961140318, 2024-09-14T09:06:29.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033982513999999998, 2024-09-14T09:06:29.491 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.491 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.491 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:29.491 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.491 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.491 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.491 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53603 2024-09-14T09:06:29.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.492 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508409+0000", 2024-09-14T09:06:29.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508409+0000", 2024-09-14T09:06:29.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508411+0000", 2024-09-14T09:06:29.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3859999999999999e-06 2024-09-14T09:06:29.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508417+0000", 2024-09-14T09:06:29.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0639999999999999e-06 2024-09-14T09:06:29.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508437+0000", 2024-09-14T09:06:29.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0055999999999999e-05 2024-09-14T09:06:29.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508451+0000", 2024-09-14T09:06:29.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4144e-05 2024-09-14T09:06:29.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542235+0000", 2024-09-14T09:06:29.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033783531999999998 2024-09-14T09:06:29.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542316+0000", 2024-09-14T09:06:29.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1704000000000002e-05 2024-09-14T09:06:29.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542392+0000", 2024-09-14T09:06:29.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5628000000000002e-05 2024-09-14T09:06:29.495 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.495 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.495 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.495 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53638 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:29.495 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508472+0000", 2024-09-14T09:06:29.495 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.961077768999999, 2024-09-14T09:06:29.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034203029000000003, 2024-09-14T09:06:29.495 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.496 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.496 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:29.496 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.496 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.496 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.496 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53638 2024-09-14T09:06:29.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.496 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508472+0000", 2024-09-14T09:06:29.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508472+0000", 2024-09-14T09:06:29.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508473+0000", 2024-09-14T09:06:29.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3769999999999999e-06 2024-09-14T09:06:29.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508492+0000", 2024-09-14T09:06:29.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8643000000000001e-05 2024-09-14T09:06:29.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508512+0000", 2024-09-14T09:06:29.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9516000000000001e-05 2024-09-14T09:06:29.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508525+0000", 2024-09-14T09:06:29.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3457e-05 2024-09-14T09:06:29.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542480+0000", 2024-09-14T09:06:29.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033954766999999997 2024-09-14T09:06:29.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542579+0000", 2024-09-14T09:06:29.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9497e-05 2024-09-14T09:06:29.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542675+0000", 2024-09-14T09:06:29.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5772000000000004e-05 2024-09-14T09:06:29.499 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.499 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.499 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.500 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53646 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:29.500 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508545+0000", 2024-09-14T09:06:29.500 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.961004546, 2024-09-14T09:06:29.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034341172000000003, 2024-09-14T09:06:29.500 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.500 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.500 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:29.500 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.500 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.500 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.500 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53646 2024-09-14T09:06:29.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.500 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508545+0000", 2024-09-14T09:06:29.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508545+0000", 2024-09-14T09:06:29.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508547+0000", 2024-09-14T09:06:29.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7269999999999999e-06 2024-09-14T09:06:29.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508553+0000", 2024-09-14T09:06:29.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6930000000000004e-06 2024-09-14T09:06:29.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508571+0000", 2024-09-14T09:06:29.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.844e-05 2024-09-14T09:06:29.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508584+0000", 2024-09-14T09:06:29.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3157e-05 2024-09-14T09:06:29.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542730+0000", 2024-09-14T09:06:29.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034145967999999999 2024-09-14T09:06:29.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542812+0000", 2024-09-14T09:06:29.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1761000000000005e-05 2024-09-14T09:06:29.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542886+0000", 2024-09-14T09:06:29.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4425999999999995e-05 2024-09-14T09:06:29.504 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.504 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.504 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.504 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53647 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 999424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:29.504 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508603+0000", 2024-09-14T09:06:29.504 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.960946872999999, 2024-09-14T09:06:29.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034662680000000001, 2024-09-14T09:06:29.504 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.504 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.504 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:29.504 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.505 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.505 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.505 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53647 2024-09-14T09:06:29.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.505 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508603+0000", 2024-09-14T09:06:29.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508603+0000", 2024-09-14T09:06:29.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508604+0000", 2024-09-14T09:06:29.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.173e-06 2024-09-14T09:06:29.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508622+0000", 2024-09-14T09:06:29.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8326e-05 2024-09-14T09:06:29.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508641+0000", 2024-09-14T09:06:29.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8213999999999999e-05 2024-09-14T09:06:29.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508656+0000", 2024-09-14T09:06:29.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5336e-05 2024-09-14T09:06:29.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.542946+0000", 2024-09-14T09:06:29.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034290270999999997 2024-09-14T09:06:29.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543089+0000", 2024-09-14T09:06:29.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014298300000000001 2024-09-14T09:06:29.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543266+0000", 2024-09-14T09:06:29.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000176377 2024-09-14T09:06:29.508 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.508 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.508 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.508 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53648 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:29.508 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508674+0000", 2024-09-14T09:06:29.509 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.960875817, 2024-09-14T09:06:29.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034778557000000002, 2024-09-14T09:06:29.509 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.509 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.509 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:29.509 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.509 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.509 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.509 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53648 2024-09-14T09:06:29.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.509 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508674+0000", 2024-09-14T09:06:29.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508674+0000", 2024-09-14T09:06:29.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508675+0000", 2024-09-14T09:06:29.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.223e-06 2024-09-14T09:06:29.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508680+0000", 2024-09-14T09:06:29.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:06:29.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508694+0000", 2024-09-14T09:06:29.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4399999999999999e-05 2024-09-14T09:06:29.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508707+0000", 2024-09-14T09:06:29.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2723e-05 2024-09-14T09:06:29.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543316+0000", 2024-09-14T09:06:29.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034609648999999999 2024-09-14T09:06:29.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543386+0000", 2024-09-14T09:06:29.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9548999999999994e-05 2024-09-14T09:06:29.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543453+0000", 2024-09-14T09:06:29.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6595999999999997e-05 2024-09-14T09:06:29.512 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.512 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.512 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.513 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53649 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1097728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:29.513 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508725+0000", 2024-09-14T09:06:29.513 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.960825013999999, 2024-09-14T09:06:29.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035096176, 2024-09-14T09:06:29.513 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.513 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.513 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:29.513 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.513 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.513 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.513 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53649 2024-09-14T09:06:29.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.514 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508725+0000", 2024-09-14T09:06:29.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508725+0000", 2024-09-14T09:06:29.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508726+0000", 2024-09-14T09:06:29.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1629999999999999e-06 2024-09-14T09:06:29.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508731+0000", 2024-09-14T09:06:29.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.04e-06 2024-09-14T09:06:29.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508749+0000", 2024-09-14T09:06:29.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7693000000000002e-05 2024-09-14T09:06:29.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508762+0000", 2024-09-14T09:06:29.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3190000000000001e-05 2024-09-14T09:06:29.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543515+0000", 2024-09-14T09:06:29.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.034753066999999999 2024-09-14T09:06:29.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543650+0000", 2024-09-14T09:06:29.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013507100000000001 2024-09-14T09:06:29.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543821+0000", 2024-09-14T09:06:29.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017095200000000001 2024-09-14T09:06:29.517 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.517 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.517 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.517 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53650 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1851392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:29.517 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508780+0000", 2024-09-14T09:06:29.517 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.960769367999999, 2024-09-14T09:06:29.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035246841000000001, 2024-09-14T09:06:29.517 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.518 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.518 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:29.518 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.518 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.518 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.518 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53650 2024-09-14T09:06:29.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.518 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508780+0000", 2024-09-14T09:06:29.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508780+0000", 2024-09-14T09:06:29.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508782+0000", 2024-09-14T09:06:29.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1799999999999999e-06 2024-09-14T09:06:29.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508787+0000", 2024-09-14T09:06:29.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9570000000000001e-06 2024-09-14T09:06:29.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508804+0000", 2024-09-14T09:06:29.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7523000000000001e-05 2024-09-14T09:06:29.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508817+0000", 2024-09-14T09:06:29.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.259e-05 2024-09-14T09:06:29.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543866+0000", 2024-09-14T09:06:29.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035049640999999999 2024-09-14T09:06:29.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.543951+0000", 2024-09-14T09:06:29.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4897999999999999e-05 2024-09-14T09:06:29.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544027+0000", 2024-09-14T09:06:29.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6051999999999996e-05 2024-09-14T09:06:29.521 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.521 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.521 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.522 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53651 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1327104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:29.522 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508868+0000", 2024-09-14T09:06:29.522 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.960681729000001, 2024-09-14T09:06:29.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035314533000000002, 2024-09-14T09:06:29.522 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.522 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.522 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:29.522 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.522 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.522 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.522 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53651 2024-09-14T09:06:29.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.523 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508868+0000", 2024-09-14T09:06:29.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508868+0000", 2024-09-14T09:06:29.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508869+0000", 2024-09-14T09:06:29.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.393e-06 2024-09-14T09:06:29.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508876+0000", 2024-09-14T09:06:29.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1539999999999999e-06 2024-09-14T09:06:29.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508913+0000", 2024-09-14T09:06:29.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6980000000000002e-05 2024-09-14T09:06:29.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508928+0000", 2024-09-14T09:06:29.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5543e-05 2024-09-14T09:06:29.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544061+0000", 2024-09-14T09:06:29.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035132656999999998 2024-09-14T09:06:29.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544127+0000", 2024-09-14T09:06:29.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6036000000000003e-05 2024-09-14T09:06:29.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544183+0000", 2024-09-14T09:06:29.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5770000000000003e-05 2024-09-14T09:06:29.526 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.526 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.526 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.526 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53652 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:29.526 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.508952+0000", 2024-09-14T09:06:29.526 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.960597666, 2024-09-14T09:06:29.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035409204, 2024-09-14T09:06:29.527 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.527 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.527 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:29.527 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.527 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.527 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.527 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53652 2024-09-14T09:06:29.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.527 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508952+0000", 2024-09-14T09:06:29.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508952+0000", 2024-09-14T09:06:29.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508953+0000", 2024-09-14T09:06:29.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3599999999999999e-06 2024-09-14T09:06:29.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.508959+0000", 2024-09-14T09:06:29.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9270000000000002e-06 2024-09-14T09:06:29.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509001+0000", 2024-09-14T09:06:29.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1896e-05 2024-09-14T09:06:29.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509022+0000", 2024-09-14T09:06:29.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.035e-05 2024-09-14T09:06:29.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544219+0000", 2024-09-14T09:06:29.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035197581999999998 2024-09-14T09:06:29.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544295+0000", 2024-09-14T09:06:29.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5483999999999998e-05 2024-09-14T09:06:29.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544361+0000", 2024-09-14T09:06:29.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6605000000000004e-05 2024-09-14T09:06:29.530 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.530 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.530 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.531 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53653 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1425408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:29.531 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509038+0000", 2024-09-14T09:06:29.531 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.960511512, 2024-09-14T09:06:29.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035495397999999997, 2024-09-14T09:06:29.531 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.531 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.531 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:29.531 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.531 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.531 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.531 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53653 2024-09-14T09:06:29.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.531 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509038+0000", 2024-09-14T09:06:29.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509038+0000", 2024-09-14T09:06:29.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509039+0000", 2024-09-14T09:06:29.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9500000000000001e-07 2024-09-14T09:06:29.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509044+0000", 2024-09-14T09:06:29.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4179999999999999e-06 2024-09-14T09:06:29.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509054+0000", 2024-09-14T09:06:29.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0891e-05 2024-09-14T09:06:29.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509061+0000", 2024-09-14T09:06:29.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3260000000000001e-06 2024-09-14T09:06:29.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544388+0000", 2024-09-14T09:06:29.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035327648000000003 2024-09-14T09:06:29.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544461+0000", 2024-09-14T09:06:29.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2669999999999994e-05 2024-09-14T09:06:29.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544534+0000", 2024-09-14T09:06:29.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2550000000000002e-05 2024-09-14T09:06:29.535 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.535 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.535 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.535 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53675 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:29.535 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509071+0000", 2024-09-14T09:06:29.535 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.960479124999999, 2024-09-14T09:06:29.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035630230999999998, 2024-09-14T09:06:29.535 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.535 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.535 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:29.536 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.536 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.536 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.536 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53675 2024-09-14T09:06:29.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.536 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509071+0000", 2024-09-14T09:06:29.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509071+0000", 2024-09-14T09:06:29.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509071+0000", 2024-09-14T09:06:29.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6899999999999997e-07 2024-09-14T09:06:29.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509074+0000", 2024-09-14T09:06:29.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5979999999999999e-06 2024-09-14T09:06:29.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509081+0000", 2024-09-14T09:06:29.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0790000000000003e-06 2024-09-14T09:06:29.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509086+0000", 2024-09-14T09:06:29.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4280000000000004e-06 2024-09-14T09:06:29.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544561+0000", 2024-09-14T09:06:29.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035474736 2024-09-14T09:06:29.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544634+0000", 2024-09-14T09:06:29.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3258999999999998e-05 2024-09-14T09:06:29.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544701+0000", 2024-09-14T09:06:29.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6562000000000004e-05 2024-09-14T09:06:29.539 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.539 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.539 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.540 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53676 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:29.540 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509095+0000", 2024-09-14T09:06:29.540 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.960454907000001, 2024-09-14T09:06:29.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035734397000000001, 2024-09-14T09:06:29.540 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.540 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.540 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:29.540 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.540 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.540 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.540 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53676 2024-09-14T09:06:29.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.540 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509095+0000", 2024-09-14T09:06:29.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509095+0000", 2024-09-14T09:06:29.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509095+0000", 2024-09-14T09:06:29.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5700000000000002e-07 2024-09-14T09:06:29.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509098+0000", 2024-09-14T09:06:29.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.553e-06 2024-09-14T09:06:29.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509105+0000", 2024-09-14T09:06:29.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1350000000000003e-06 2024-09-14T09:06:29.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509110+0000", 2024-09-14T09:06:29.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1370000000000001e-06 2024-09-14T09:06:29.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544734+0000", 2024-09-14T09:06:29.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035624054000000002 2024-09-14T09:06:29.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544782+0000", 2024-09-14T09:06:29.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7956e-05 2024-09-14T09:06:29.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544829+0000", 2024-09-14T09:06:29.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7005000000000001e-05 2024-09-14T09:06:29.544 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.544 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.544 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.544 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53677 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:29.544 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509120+0000", 2024-09-14T09:06:29.544 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.960429379000001, 2024-09-14T09:06:29.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035825663000000001, 2024-09-14T09:06:29.544 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.544 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.544 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:29.545 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.545 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.545 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.545 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53677 2024-09-14T09:06:29.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.545 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509120+0000", 2024-09-14T09:06:29.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509120+0000", 2024-09-14T09:06:29.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509121+0000", 2024-09-14T09:06:29.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2e-07 2024-09-14T09:06:29.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509136+0000", 2024-09-14T09:06:29.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.517e-05 2024-09-14T09:06:29.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509145+0000", 2024-09-14T09:06:29.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4880000000000007e-06 2024-09-14T09:06:29.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509150+0000", 2024-09-14T09:06:29.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9100000000000002e-06 2024-09-14T09:06:29.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544860+0000", 2024-09-14T09:06:29.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035709739999999997 2024-09-14T09:06:29.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544904+0000", 2024-09-14T09:06:29.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3791999999999997e-05 2024-09-14T09:06:29.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544946+0000", 2024-09-14T09:06:29.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2042999999999997e-05 2024-09-14T09:06:29.548 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.548 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.548 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.548 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53678 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:29.548 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.509160+0000", 2024-09-14T09:06:29.549 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.960389411, 2024-09-14T09:06:29.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035923798999999999, 2024-09-14T09:06:29.549 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.549 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.549 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:29.549 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.549 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.549 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.549 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53678 2024-09-14T09:06:29.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.549 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509160+0000", 2024-09-14T09:06:29.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509160+0000", 2024-09-14T09:06:29.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509161+0000", 2024-09-14T09:06:29.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9599999999999999e-07 2024-09-14T09:06:29.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509163+0000", 2024-09-14T09:06:29.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3410000000000001e-06 2024-09-14T09:06:29.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509171+0000", 2024-09-14T09:06:29.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5519999999999999e-06 2024-09-14T09:06:29.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.509176+0000", 2024-09-14T09:06:29.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4750000000000001e-06 2024-09-14T09:06:29.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.544975+0000", 2024-09-14T09:06:29.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.035798862000000001 2024-09-14T09:06:29.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.545033+0000", 2024-09-14T09:06:29.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8148e-05 2024-09-14T09:06:29.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.545084+0000", 2024-09-14T09:06:29.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0825000000000002e-05 2024-09-14T09:06:29.552 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.552 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.553 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.553 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:53838 3.1b 3:dc7ab6b8:::rbd_data.115e5dc498ac.0000000000000006:14 [read 573440~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-14T09:06:29.553 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:16.518551+0000", 2024-09-14T09:06:29.553 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.950998862, 2024-09-14T09:06:29.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033585690000000001, 2024-09-14T09:06:29.553 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:29.553 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:29.553 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:29.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:29.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:29.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:29.553 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 53838 2024-09-14T09:06:29.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.554 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:29.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:29.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518551+0000", 2024-09-14T09:06:29.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:29.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518551+0000", 2024-09-14T09:06:29.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:29.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:29.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518551+0000", 2024-09-14T09:06:29.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2499999999999995e-07 2024-09-14T09:06:29.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:29.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518554+0000", 2024-09-14T09:06:29.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9450000000000002e-06 2024-09-14T09:06:29.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:29.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518561+0000", 2024-09-14T09:06:29.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4849999999999997e-06 2024-09-14T09:06:29.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:29.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.518572+0000", 2024-09-14T09:06:29.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0818999999999999e-05 2024-09-14T09:06:29.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:29.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.551823+0000", 2024-09-14T09:06:29.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.033250864999999998 2024-09-14T09:06:29.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:29.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.551967+0000", 2024-09-14T09:06:29.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014399700000000001 2024-09-14T09:06:29.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:29.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:29.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:29.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:16.552137+0000", 2024-09-14T09:06:29.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170054 2024-09-14T09:06:29.557 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.557 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.557 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.557 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:29.557 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:29.557 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:29.559 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:06:29.560 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:29.559+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:29.560 INFO:teuthology.orchestra.run.smithi176.stdout:3091 write 0xb6d4ce0 thru 0xb6e0848 (0xbb69 bytes) 2024-09-14T09:06:29.560 INFO:teuthology.orchestra.run.smithi176.stdout:3093 writesame 0x9256e27 thru 0x925737e data_size 0x18(0x558 bytes) 2024-09-14T09:06:29.560 INFO:teuthology.orchestra.run.smithi176.stdout:3094 punch from 0x412728a to 0x4130532, (0x92a8 bytes) 2024-09-14T09:06:29.560 INFO:teuthology.orchestra.run.smithi176.stdout:3095 punch from 0x2734cd9 to 0x2742ed9, (0xe200 bytes) 2024-09-14T09:06:29.560 INFO:teuthology.orchestra.run.smithi176.stdout:3098 write 0xbe80b6e thru 0xbe8e09e (0xd531 bytes) 2024-09-14T09:06:29.560 INFO:teuthology.orchestra.run.smithi176.stdout:3100 read 0x7fe6be thru 0x80cf9b (0xe8de bytes) 2024-09-14T09:06:29.560 INFO:teuthology.orchestra.run.smithi176.stdout:3101 read 0xbe7879 thru 0xbe9fdb (0x2763 bytes) 2024-09-14T09:06:29.561 INFO:teuthology.orchestra.run.smithi176.stdout:3102 trunc from 0xbe8e09f to 0x95a266b 2024-09-14T09:06:29.561 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:06:29.561 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:29.561 INFO:teuthology.orchestra.run.smithi176.stdout: "epoch": 71, 2024-09-14T09:06:29.561 INFO:teuthology.orchestra.run.smithi176.stdout: "fsid": "c635667d-994e-454f-bcea-51b45637b2a2", 2024-09-14T09:06:29.561 INFO:teuthology.orchestra.run.smithi176.stdout: "created": "2024-09-14T09:04:57.077293+0000", 2024-09-14T09:06:29.561 INFO:teuthology.orchestra.run.smithi176.stdout: "modified": "2024-09-14T09:06:29.226690+0000", 2024-09-14T09:06:29.561 INFO:teuthology.orchestra.run.smithi176.stdout: "last_up_change": "2024-09-14T09:05:11.210835+0000", 2024-09-14T09:06:29.561 INFO:teuthology.orchestra.run.smithi176.stdout: "last_in_change": "2024-09-14T09:04:59.344709+0000", 2024-09-14T09:06:29.562 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T09:06:29.562 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_num": 5806080, 2024-09-14T09:06:29.562 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_set": [ 2024-09-14T09:06:29.562 INFO:teuthology.orchestra.run.smithi176.stdout: "nodeep-scrub", 2024-09-14T09:06:29.562 INFO:teuthology.orchestra.run.smithi176.stdout: "noscrub", 2024-09-14T09:06:29.562 INFO:teuthology.orchestra.run.smithi176.stdout: "pglog_hardlimit", 2024-09-14T09:06:29.562 INFO:teuthology.orchestra.run.smithi176.stdout: "purged_snapdirs", 2024-09-14T09:06:29.562 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_deletes", 2024-09-14T09:06:29.562 INFO:teuthology.orchestra.run.smithi176.stdout: "sortbitwise" 2024-09-14T09:06:29.562 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:06:29.562 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_version": 5, 2024-09-14T09:06:29.562 INFO:teuthology.orchestra.run.smithi176.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T09:06:29.562 INFO:teuthology.orchestra.run.smithi176.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T09:06:29.562 INFO:teuthology.orchestra.run.smithi176.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T09:06:29.562 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_snapshot": "", 2024-09-14T09:06:29.562 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_max": 3, 2024-09-14T09:06:29.563 INFO:teuthology.orchestra.run.smithi176.stdout: "max_osd": 8, 2024-09-14T09:06:29.563 INFO:teuthology.orchestra.run.smithi176.stdout: "require_min_compat_client": "luminous", 2024-09-14T09:06:29.563 INFO:teuthology.orchestra.run.smithi176.stdout: "min_compat_client": "luminous", 2024-09-14T09:06:29.563 INFO:teuthology.orchestra.run.smithi176.stdout: "require_osd_release": "squid", 2024-09-14T09:06:29.563 INFO:teuthology.orchestra.run.smithi176.stdout: "allow_crimson": false, 2024-09-14T09:06:29.563 INFO:teuthology.orchestra.run.smithi176.stdout: "pools": [ 2024-09-14T09:06:29.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.563 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 1, 2024-09-14T09:06:29.563 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_name": ".mgr", 2024-09-14T09:06:29.563 INFO:teuthology.orchestra.run.smithi176.stdout: "create_time": "2024-09-14T09:05:11.997673+0000", 2024-09-14T09:06:29.563 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": 1, 2024-09-14T09:06:29.563 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_names": "hashpspool", 2024-09-14T09:06:29.563 INFO:teuthology.orchestra.run.smithi176.stdout: "type": 1, 2024-09-14T09:06:29.563 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 2, 2024-09-14T09:06:29.564 INFO:teuthology.orchestra.run.smithi176.stdout: "min_size": 1, 2024-09-14T09:06:29.564 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_rule": 0, 2024-09-14T09:06:29.564 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_count": 0, 2024-09-14T09:06:29.564 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_target": 0, 2024-09-14T09:06:29.564 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T09:06:29.564 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T09:06:29.564 INFO:teuthology.orchestra.run.smithi176.stdout: "is_stretch_pool": false, 2024-09-14T09:06:29.564 INFO:teuthology.orchestra.run.smithi176.stdout: "object_hash": 2, 2024-09-14T09:06:29.564 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_autoscale_mode": "off", 2024-09-14T09:06:29.564 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num": 1, 2024-09-14T09:06:29.564 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num": 1, 2024-09-14T09:06:29.564 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num_target": 1, 2024-09-14T09:06:29.564 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_target": 1, 2024-09-14T09:06:29.564 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_pending": 1, 2024-09-14T09:06:29.565 INFO:teuthology.orchestra.run.smithi176.stdout: "last_pg_merge_meta": { 2024-09-14T09:06:29.565 INFO:teuthology.orchestra.run.smithi176.stdout: "source_pgid": "0.0", 2024-09-14T09:06:29.565 INFO:teuthology.orchestra.run.smithi176.stdout: "ready_epoch": 0, 2024-09-14T09:06:29.565 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_started": 0, 2024-09-14T09:06:29.565 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_clean": 0, 2024-09-14T09:06:29.565 INFO:teuthology.orchestra.run.smithi176.stdout: "source_version": "0'0", 2024-09-14T09:06:29.565 INFO:teuthology.orchestra.run.smithi176.stdout: "target_version": "0'0" 2024-09-14T09:06:29.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.565 INFO:teuthology.orchestra.run.smithi176.stdout: "last_change": "18", 2024-09-14T09:06:29.565 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend": "0", 2024-09-14T09:06:29.565 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T09:06:29.565 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T09:06:29.565 INFO:teuthology.orchestra.run.smithi176.stdout: "auid": 0, 2024-09-14T09:06:29.565 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_mode": "selfmanaged", 2024-09-14T09:06:29.566 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_seq": 0, 2024-09-14T09:06:29.566 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_epoch": 0, 2024-09-14T09:06:29.566 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_snaps": [], 2024-09-14T09:06:29.566 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps": "[]", 2024-09-14T09:06:29.566 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_bytes": 0, 2024-09-14T09:06:29.566 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_objects": 0, 2024-09-14T09:06:29.566 INFO:teuthology.orchestra.run.smithi176.stdout: "tiers": [], 2024-09-14T09:06:29.566 INFO:teuthology.orchestra.run.smithi176.stdout: "tier_of": -1, 2024-09-14T09:06:29.566 INFO:teuthology.orchestra.run.smithi176.stdout: "read_tier": -1, 2024-09-14T09:06:29.566 INFO:teuthology.orchestra.run.smithi176.stdout: "write_tier": -1, 2024-09-14T09:06:29.566 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_mode": "none", 2024-09-14T09:06:29.566 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_bytes": 0, 2024-09-14T09:06:29.566 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_objects": 0, 2024-09-14T09:06:29.567 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T09:06:29.567 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T09:06:29.567 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T09:06:29.567 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_flush_age": 0, 2024-09-14T09:06:29.567 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_evict_age": 0, 2024-09-14T09:06:29.567 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profile": "", 2024-09-14T09:06:29.567 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_params": { 2024-09-14T09:06:29.567 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "none" 2024-09-14T09:06:29.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.567 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_period": 0, 2024-09-14T09:06:29.567 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_count": 0, 2024-09-14T09:06:29.567 INFO:teuthology.orchestra.run.smithi176.stdout: "use_gmt_hitset": true, 2024-09-14T09:06:29.567 INFO:teuthology.orchestra.run.smithi176.stdout: "min_read_recency_for_promote": 0, 2024-09-14T09:06:29.567 INFO:teuthology.orchestra.run.smithi176.stdout: "min_write_recency_for_promote": 0, 2024-09-14T09:06:29.567 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T09:06:29.568 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_search_last_n": 0, 2024-09-14T09:06:29.568 INFO:teuthology.orchestra.run.smithi176.stdout: "grade_table": [], 2024-09-14T09:06:29.568 INFO:teuthology.orchestra.run.smithi176.stdout: "stripe_width": 0, 2024-09-14T09:06:29.568 INFO:teuthology.orchestra.run.smithi176.stdout: "expected_num_objects": 0, 2024-09-14T09:06:29.568 INFO:teuthology.orchestra.run.smithi176.stdout: "fast_read": false, 2024-09-14T09:06:29.568 INFO:teuthology.orchestra.run.smithi176.stdout: "options": { 2024-09-14T09:06:29.568 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_max": 32, 2024-09-14T09:06:29.568 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_min": 1 2024-09-14T09:06:29.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.568 INFO:teuthology.orchestra.run.smithi176.stdout: "application_metadata": { 2024-09-14T09:06:29.568 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": {} 2024-09-14T09:06:29.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.568 INFO:teuthology.orchestra.run.smithi176.stdout: "read_balance": { 2024-09-14T09:06:29.568 INFO:teuthology.orchestra.run.smithi176.stdout: "score_type": "Fair distribution", 2024-09-14T09:06:29.569 INFO:teuthology.orchestra.run.smithi176.stdout: "score_acting": 8, 2024-09-14T09:06:29.569 INFO:teuthology.orchestra.run.smithi176.stdout: "score_stable": 8, 2024-09-14T09:06:29.569 INFO:teuthology.orchestra.run.smithi176.stdout: "optimal_score": 0.25, 2024-09-14T09:06:29.569 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_acting": 2, 2024-09-14T09:06:29.569 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_stable": 2, 2024-09-14T09:06:29.569 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity_weighted": 1, 2024-09-14T09:06:29.569 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity": 1, 2024-09-14T09:06:29.569 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity_weighted": 1 2024-09-14T09:06:29.569 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.569 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 2, 2024-09-14T09:06:29.569 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_name": "rbd", 2024-09-14T09:06:29.569 INFO:teuthology.orchestra.run.smithi176.stdout: "create_time": "2024-09-14T09:05:15.352551+0000", 2024-09-14T09:06:29.569 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": 8193, 2024-09-14T09:06:29.570 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T09:06:29.570 INFO:teuthology.orchestra.run.smithi176.stdout: "type": 1, 2024-09-14T09:06:29.570 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 2, 2024-09-14T09:06:29.570 INFO:teuthology.orchestra.run.smithi176.stdout: "min_size": 1, 2024-09-14T09:06:29.570 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_rule": 0, 2024-09-14T09:06:29.570 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_count": 0, 2024-09-14T09:06:29.570 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_target": 0, 2024-09-14T09:06:29.570 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T09:06:29.570 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T09:06:29.570 INFO:teuthology.orchestra.run.smithi176.stdout: "is_stretch_pool": false, 2024-09-14T09:06:29.570 INFO:teuthology.orchestra.run.smithi176.stdout: "object_hash": 2, 2024-09-14T09:06:29.570 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_autoscale_mode": "off", 2024-09-14T09:06:29.570 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num": 8, 2024-09-14T09:06:29.571 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num": 8, 2024-09-14T09:06:29.571 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num_target": 8, 2024-09-14T09:06:29.571 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_target": 8, 2024-09-14T09:06:29.571 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_pending": 8, 2024-09-14T09:06:29.571 INFO:teuthology.orchestra.run.smithi176.stdout: "last_pg_merge_meta": { 2024-09-14T09:06:29.571 INFO:teuthology.orchestra.run.smithi176.stdout: "source_pgid": "0.0", 2024-09-14T09:06:29.571 INFO:teuthology.orchestra.run.smithi176.stdout: "ready_epoch": 0, 2024-09-14T09:06:29.571 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_started": 0, 2024-09-14T09:06:29.571 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_clean": 0, 2024-09-14T09:06:29.571 INFO:teuthology.orchestra.run.smithi176.stdout: "source_version": "0'0", 2024-09-14T09:06:29.571 INFO:teuthology.orchestra.run.smithi176.stdout: "target_version": "0'0" 2024-09-14T09:06:29.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.571 INFO:teuthology.orchestra.run.smithi176.stdout: "last_change": "22", 2024-09-14T09:06:29.571 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend": "0", 2024-09-14T09:06:29.572 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T09:06:29.572 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T09:06:29.572 INFO:teuthology.orchestra.run.smithi176.stdout: "auid": 0, 2024-09-14T09:06:29.572 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_mode": "selfmanaged", 2024-09-14T09:06:29.572 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_seq": 2, 2024-09-14T09:06:29.572 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_epoch": 22, 2024-09-14T09:06:29.572 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_snaps": [], 2024-09-14T09:06:29.572 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps": "[]", 2024-09-14T09:06:29.572 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_bytes": 0, 2024-09-14T09:06:29.572 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_objects": 0, 2024-09-14T09:06:29.572 INFO:teuthology.orchestra.run.smithi176.stdout: "tiers": [], 2024-09-14T09:06:29.572 INFO:teuthology.orchestra.run.smithi176.stdout: "tier_of": -1, 2024-09-14T09:06:29.572 INFO:teuthology.orchestra.run.smithi176.stdout: "read_tier": -1, 2024-09-14T09:06:29.572 INFO:teuthology.orchestra.run.smithi176.stdout: "write_tier": -1, 2024-09-14T09:06:29.572 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_mode": "none", 2024-09-14T09:06:29.572 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_bytes": 0, 2024-09-14T09:06:29.573 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_objects": 0, 2024-09-14T09:06:29.573 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T09:06:29.573 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T09:06:29.573 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T09:06:29.573 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_flush_age": 0, 2024-09-14T09:06:29.573 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_evict_age": 0, 2024-09-14T09:06:29.573 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profile": "", 2024-09-14T09:06:29.573 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_params": { 2024-09-14T09:06:29.573 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "none" 2024-09-14T09:06:29.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.573 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_period": 0, 2024-09-14T09:06:29.573 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_count": 0, 2024-09-14T09:06:29.573 INFO:teuthology.orchestra.run.smithi176.stdout: "use_gmt_hitset": true, 2024-09-14T09:06:29.573 INFO:teuthology.orchestra.run.smithi176.stdout: "min_read_recency_for_promote": 0, 2024-09-14T09:06:29.574 INFO:teuthology.orchestra.run.smithi176.stdout: "min_write_recency_for_promote": 0, 2024-09-14T09:06:29.574 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T09:06:29.574 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_search_last_n": 0, 2024-09-14T09:06:29.574 INFO:teuthology.orchestra.run.smithi176.stdout: "grade_table": [], 2024-09-14T09:06:29.574 INFO:teuthology.orchestra.run.smithi176.stdout: "stripe_width": 0, 2024-09-14T09:06:29.574 INFO:teuthology.orchestra.run.smithi176.stdout: "expected_num_objects": 0, 2024-09-14T09:06:29.574 INFO:teuthology.orchestra.run.smithi176.stdout: "fast_read": false, 2024-09-14T09:06:29.574 INFO:teuthology.orchestra.run.smithi176.stdout: "options": {}, 2024-09-14T09:06:29.574 INFO:teuthology.orchestra.run.smithi176.stdout: "application_metadata": { 2024-09-14T09:06:29.574 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd": {} 2024-09-14T09:06:29.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.574 INFO:teuthology.orchestra.run.smithi176.stdout: "read_balance": { 2024-09-14T09:06:29.574 INFO:teuthology.orchestra.run.smithi176.stdout: "score_type": "Fair distribution", 2024-09-14T09:06:29.574 INFO:teuthology.orchestra.run.smithi176.stdout: "score_acting": 1.9900000095367432, 2024-09-14T09:06:29.574 INFO:teuthology.orchestra.run.smithi176.stdout: "score_stable": 1.9900000095367432, 2024-09-14T09:06:29.575 INFO:teuthology.orchestra.run.smithi176.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T09:06:29.575 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_acting": 1.75, 2024-09-14T09:06:29.575 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_stable": 1.75, 2024-09-14T09:06:29.575 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity_weighted": 1, 2024-09-14T09:06:29.575 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity": 1, 2024-09-14T09:06:29.575 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity_weighted": 1 2024-09-14T09:06:29.575 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.575 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 3, 2024-09-14T09:06:29.575 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_name": "pool_client.0", 2024-09-14T09:06:29.575 INFO:teuthology.orchestra.run.smithi176.stdout: "create_time": "2024-09-14T09:05:25.840963+0000", 2024-09-14T09:06:29.575 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": 8193, 2024-09-14T09:06:29.575 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T09:06:29.575 INFO:teuthology.orchestra.run.smithi176.stdout: "type": 1, 2024-09-14T09:06:29.575 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 2, 2024-09-14T09:06:29.575 INFO:teuthology.orchestra.run.smithi176.stdout: "min_size": 1, 2024-09-14T09:06:29.576 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_rule": 0, 2024-09-14T09:06:29.576 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_count": 0, 2024-09-14T09:06:29.576 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_target": 0, 2024-09-14T09:06:29.576 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T09:06:29.576 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T09:06:29.576 INFO:teuthology.orchestra.run.smithi176.stdout: "is_stretch_pool": false, 2024-09-14T09:06:29.576 INFO:teuthology.orchestra.run.smithi176.stdout: "object_hash": 2, 2024-09-14T09:06:29.576 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_autoscale_mode": "off", 2024-09-14T09:06:29.576 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num": 32, 2024-09-14T09:06:29.576 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num": 32, 2024-09-14T09:06:29.576 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num_target": 32, 2024-09-14T09:06:29.576 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_target": 32, 2024-09-14T09:06:29.576 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_pending": 32, 2024-09-14T09:06:29.576 INFO:teuthology.orchestra.run.smithi176.stdout: "last_pg_merge_meta": { 2024-09-14T09:06:29.577 INFO:teuthology.orchestra.run.smithi176.stdout: "source_pgid": "0.0", 2024-09-14T09:06:29.577 INFO:teuthology.orchestra.run.smithi176.stdout: "ready_epoch": 0, 2024-09-14T09:06:29.577 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_started": 0, 2024-09-14T09:06:29.577 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_clean": 0, 2024-09-14T09:06:29.577 INFO:teuthology.orchestra.run.smithi176.stdout: "source_version": "0'0", 2024-09-14T09:06:29.577 INFO:teuthology.orchestra.run.smithi176.stdout: "target_version": "0'0" 2024-09-14T09:06:29.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.577 INFO:teuthology.orchestra.run.smithi176.stdout: "last_change": "70", 2024-09-14T09:06:29.577 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend": "0", 2024-09-14T09:06:29.577 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T09:06:29.577 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T09:06:29.577 INFO:teuthology.orchestra.run.smithi176.stdout: "auid": 0, 2024-09-14T09:06:29.577 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_mode": "selfmanaged", 2024-09-14T09:06:29.577 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_seq": 26, 2024-09-14T09:06:29.578 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_epoch": 70, 2024-09-14T09:06:29.578 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_snaps": [], 2024-09-14T09:06:29.578 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps": "[]", 2024-09-14T09:06:29.578 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_bytes": 0, 2024-09-14T09:06:29.578 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_objects": 0, 2024-09-14T09:06:29.578 INFO:teuthology.orchestra.run.smithi176.stdout: "tiers": [], 2024-09-14T09:06:29.578 INFO:teuthology.orchestra.run.smithi176.stdout: "tier_of": -1, 2024-09-14T09:06:29.578 INFO:teuthology.orchestra.run.smithi176.stdout: "read_tier": -1, 2024-09-14T09:06:29.578 INFO:teuthology.orchestra.run.smithi176.stdout: "write_tier": -1, 2024-09-14T09:06:29.578 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_mode": "none", 2024-09-14T09:06:29.578 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_bytes": 0, 2024-09-14T09:06:29.578 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_objects": 0, 2024-09-14T09:06:29.578 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T09:06:29.578 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T09:06:29.578 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T09:06:29.578 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_flush_age": 0, 2024-09-14T09:06:29.579 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_evict_age": 0, 2024-09-14T09:06:29.579 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profile": "", 2024-09-14T09:06:29.579 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_params": { 2024-09-14T09:06:29.579 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "none" 2024-09-14T09:06:29.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.579 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_period": 0, 2024-09-14T09:06:29.579 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_count": 0, 2024-09-14T09:06:29.579 INFO:teuthology.orchestra.run.smithi176.stdout: "use_gmt_hitset": true, 2024-09-14T09:06:29.579 INFO:teuthology.orchestra.run.smithi176.stdout: "min_read_recency_for_promote": 0, 2024-09-14T09:06:29.579 INFO:teuthology.orchestra.run.smithi176.stdout: "min_write_recency_for_promote": 0, 2024-09-14T09:06:29.579 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T09:06:29.579 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_search_last_n": 0, 2024-09-14T09:06:29.579 INFO:teuthology.orchestra.run.smithi176.stdout: "grade_table": [], 2024-09-14T09:06:29.580 INFO:teuthology.orchestra.run.smithi176.stdout: "stripe_width": 0, 2024-09-14T09:06:29.580 INFO:teuthology.orchestra.run.smithi176.stdout: "expected_num_objects": 0, 2024-09-14T09:06:29.580 INFO:teuthology.orchestra.run.smithi176.stdout: "fast_read": false, 2024-09-14T09:06:29.580 INFO:teuthology.orchestra.run.smithi176.stdout: "options": {}, 2024-09-14T09:06:29.580 INFO:teuthology.orchestra.run.smithi176.stdout: "application_metadata": { 2024-09-14T09:06:29.580 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd": {} 2024-09-14T09:06:29.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.580 INFO:teuthology.orchestra.run.smithi176.stdout: "read_balance": { 2024-09-14T09:06:29.580 INFO:teuthology.orchestra.run.smithi176.stdout: "score_type": "Fair distribution", 2024-09-14T09:06:29.580 INFO:teuthology.orchestra.run.smithi176.stdout: "score_acting": 1.75, 2024-09-14T09:06:29.580 INFO:teuthology.orchestra.run.smithi176.stdout: "score_stable": 1.75, 2024-09-14T09:06:29.580 INFO:teuthology.orchestra.run.smithi176.stdout: "optimal_score": 1, 2024-09-14T09:06:29.580 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_acting": 1.75, 2024-09-14T09:06:29.580 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_stable": 1.75, 2024-09-14T09:06:29.581 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity_weighted": 1, 2024-09-14T09:06:29.581 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity": 1, 2024-09-14T09:06:29.581 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity_weighted": 1 2024-09-14T09:06:29.581 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.581 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.581 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:06:29.581 INFO:teuthology.orchestra.run.smithi176.stdout: "osds": [ 2024-09-14T09:06:29.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.581 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 0, 2024-09-14T09:06:29.581 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "4ceedb4f-6398-467f-9562-ccf27200860b", 2024-09-14T09:06:29.581 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:06:29.581 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:06:29.581 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 0.8736724853515625, 2024-09-14T09:06:29.581 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:06:29.581 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:06:29.582 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:06:29.582 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:06:29.582 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 34, 2024-09-14T09:06:29.582 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:06:29.582 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:06:29.582 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:06:29.582 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.582 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.582 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6824", 2024-09-14T09:06:29.582 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:06:29.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.583 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.583 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6825", 2024-09-14T09:06:29.583 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:06:29.583 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.583 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.583 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:06:29.583 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.583 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.583 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6826", 2024-09-14T09:06:29.583 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:06:29.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.584 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.584 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6827", 2024-09-14T09:06:29.584 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:06:29.584 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.584 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.584 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:06:29.584 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.584 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.584 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6830", 2024-09-14T09:06:29.584 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:06:29.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.585 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.585 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6831", 2024-09-14T09:06:29.585 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:06:29.585 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.585 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.585 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:06:29.585 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.585 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.585 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6828", 2024-09-14T09:06:29.585 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:06:29.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.586 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.586 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6829", 2024-09-14T09:06:29.586 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:06:29.586 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.586 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.586 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6825/3384710819", 2024-09-14T09:06:29.586 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6827/3384710819", 2024-09-14T09:06:29.586 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6831/3384710819", 2024-09-14T09:06:29.586 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6829/3384710819", 2024-09-14T09:06:29.586 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:06:29.586 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:06:29.587 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:06:29.587 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.587 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 1, 2024-09-14T09:06:29.587 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "604e2f62-7c13-46be-a97f-029b28c44a89", 2024-09-14T09:06:29.587 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:06:29.587 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:06:29.587 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:06:29.587 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:06:29.587 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:06:29.587 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:06:29.587 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:06:29.587 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 53, 2024-09-14T09:06:29.588 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:06:29.588 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:06:29.588 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:06:29.588 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.588 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.588 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6800", 2024-09-14T09:06:29.588 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:06:29.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.588 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.588 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6801", 2024-09-14T09:06:29.588 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:06:29.588 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.589 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.589 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:06:29.589 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.589 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.589 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6802", 2024-09-14T09:06:29.589 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:06:29.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.589 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.589 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6803", 2024-09-14T09:06:29.589 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:06:29.590 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.590 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.590 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:06:29.590 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.590 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.590 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6806", 2024-09-14T09:06:29.590 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:06:29.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.590 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.590 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6807", 2024-09-14T09:06:29.590 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:06:29.590 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.591 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.591 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:06:29.591 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.591 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.591 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6804", 2024-09-14T09:06:29.591 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:06:29.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.591 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.591 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6805", 2024-09-14T09:06:29.591 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:06:29.591 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.591 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.592 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6801/612012679", 2024-09-14T09:06:29.592 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6803/612012679", 2024-09-14T09:06:29.592 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6807/612012679", 2024-09-14T09:06:29.592 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6805/612012679", 2024-09-14T09:06:29.592 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:06:29.592 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:06:29.592 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:06:29.592 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.592 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 2, 2024-09-14T09:06:29.592 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "1f65589e-8b0a-4033-94c0-4ab66531db2b", 2024-09-14T09:06:29.592 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:06:29.593 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:06:29.593 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:06:29.593 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:06:29.593 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:06:29.593 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:06:29.593 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:06:29.593 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 34, 2024-09-14T09:06:29.593 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:06:29.593 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:06:29.593 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:06:29.593 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.593 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.593 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6808", 2024-09-14T09:06:29.594 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:06:29.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.594 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.594 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6809", 2024-09-14T09:06:29.594 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:06:29.594 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.594 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.594 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:06:29.594 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.594 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.594 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6810", 2024-09-14T09:06:29.595 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:06:29.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.595 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.595 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6811", 2024-09-14T09:06:29.595 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:06:29.595 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.595 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.595 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:06:29.595 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.595 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.596 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6814", 2024-09-14T09:06:29.596 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:06:29.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.596 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.596 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6815", 2024-09-14T09:06:29.596 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:06:29.596 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.596 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.596 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:06:29.596 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.596 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.597 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6812", 2024-09-14T09:06:29.597 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:06:29.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.597 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.597 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6813", 2024-09-14T09:06:29.597 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:06:29.597 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.597 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.597 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6809/4167379279", 2024-09-14T09:06:29.597 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6811/4167379279", 2024-09-14T09:06:29.597 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6815/4167379279", 2024-09-14T09:06:29.597 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6813/4167379279", 2024-09-14T09:06:29.598 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:06:29.598 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:06:29.598 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:06:29.598 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.598 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 3, 2024-09-14T09:06:29.598 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "209d4f65-79f9-4a7d-9d23-832a4209507f", 2024-09-14T09:06:29.598 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:06:29.598 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:06:29.598 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:06:29.598 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:06:29.598 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:06:29.598 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:06:29.599 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:06:29.599 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 67, 2024-09-14T09:06:29.599 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:06:29.599 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:06:29.599 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:06:29.599 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.599 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.599 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6816", 2024-09-14T09:06:29.599 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:06:29.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.599 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.599 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6817", 2024-09-14T09:06:29.599 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:06:29.599 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.600 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.600 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:06:29.600 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.600 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.600 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6818", 2024-09-14T09:06:29.600 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:06:29.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.600 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.600 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6819", 2024-09-14T09:06:29.600 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:06:29.601 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.601 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.601 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:06:29.601 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.601 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.601 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6822", 2024-09-14T09:06:29.601 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:06:29.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.601 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.601 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6823", 2024-09-14T09:06:29.601 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:06:29.602 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.602 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.602 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:06:29.602 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.602 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.602 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6820", 2024-09-14T09:06:29.602 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:06:29.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.602 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.602 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6821", 2024-09-14T09:06:29.602 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:06:29.603 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.603 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.603 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6817/1658209015", 2024-09-14T09:06:29.603 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6819/1658209015", 2024-09-14T09:06:29.603 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6823/1658209015", 2024-09-14T09:06:29.603 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6821/1658209015", 2024-09-14T09:06:29.603 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:06:29.603 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:06:29.603 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:06:29.603 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.603 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 4, 2024-09-14T09:06:29.603 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "8f9a8c9b-282e-4523-9e62-5824f4aba401", 2024-09-14T09:06:29.604 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:06:29.604 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:06:29.604 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 0.1816558837890625, 2024-09-14T09:06:29.604 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:06:29.604 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:06:29.604 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:06:29.604 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:06:29.604 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 54, 2024-09-14T09:06:29.604 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:06:29.604 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:06:29.604 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:06:29.604 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.604 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.604 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6808", 2024-09-14T09:06:29.605 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:06:29.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.605 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.605 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6809", 2024-09-14T09:06:29.605 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:06:29.605 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.605 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.605 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:06:29.605 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.605 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.605 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6810", 2024-09-14T09:06:29.606 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:06:29.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.606 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.606 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6811", 2024-09-14T09:06:29.606 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:06:29.606 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.606 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.606 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:06:29.606 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.606 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.606 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6814", 2024-09-14T09:06:29.606 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:06:29.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.607 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.607 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6815", 2024-09-14T09:06:29.607 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:06:29.607 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.607 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.607 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:06:29.607 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.607 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.607 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6812", 2024-09-14T09:06:29.607 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:06:29.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.608 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.608 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6813", 2024-09-14T09:06:29.608 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:06:29.608 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.608 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.608 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6809/1576054389", 2024-09-14T09:06:29.608 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6811/1576054389", 2024-09-14T09:06:29.608 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6815/1576054389", 2024-09-14T09:06:29.608 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6813/1576054389", 2024-09-14T09:06:29.608 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:06:29.608 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:06:29.608 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:06:29.609 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.609 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 5, 2024-09-14T09:06:29.609 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80", 2024-09-14T09:06:29.609 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:06:29.609 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:06:29.609 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:06:29.609 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:06:29.609 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:06:29.609 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:06:29.609 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:06:29.609 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 34, 2024-09-14T09:06:29.610 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:06:29.610 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:06:29.610 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:06:29.610 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.610 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.610 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6824", 2024-09-14T09:06:29.610 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:06:29.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.610 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.610 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6825", 2024-09-14T09:06:29.610 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:06:29.610 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.610 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.611 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:06:29.611 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.611 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.611 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6826", 2024-09-14T09:06:29.611 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:06:29.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.611 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.611 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6827", 2024-09-14T09:06:29.611 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:06:29.611 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.611 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.611 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:06:29.611 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.612 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.612 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6830", 2024-09-14T09:06:29.612 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:06:29.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.612 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.612 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6831", 2024-09-14T09:06:29.612 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:06:29.612 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.612 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.612 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:06:29.612 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.613 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.613 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6828", 2024-09-14T09:06:29.613 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:06:29.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.613 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.613 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6829", 2024-09-14T09:06:29.613 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:06:29.613 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.613 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.613 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6825/1711292386", 2024-09-14T09:06:29.613 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6827/1711292386", 2024-09-14T09:06:29.613 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6831/1711292386", 2024-09-14T09:06:29.613 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6829/1711292386", 2024-09-14T09:06:29.614 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:06:29.614 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:06:29.614 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:06:29.614 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.614 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 6, 2024-09-14T09:06:29.614 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "59d9259d-b9dd-4326-8ab9-9d599a2bf57b", 2024-09-14T09:06:29.614 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:06:29.614 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:06:29.614 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:06:29.614 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:06:29.614 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:06:29.614 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:06:29.615 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:06:29.615 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 50, 2024-09-14T09:06:29.615 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:06:29.615 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:06:29.615 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:06:29.615 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.615 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.615 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6816", 2024-09-14T09:06:29.615 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:06:29.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.615 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.615 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6817", 2024-09-14T09:06:29.616 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:06:29.616 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.616 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.616 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:06:29.616 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.616 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.616 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6818", 2024-09-14T09:06:29.616 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:06:29.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.616 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.616 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6819", 2024-09-14T09:06:29.616 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:06:29.617 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.617 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.617 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:06:29.617 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.617 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.617 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6822", 2024-09-14T09:06:29.617 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:06:29.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.617 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.617 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6823", 2024-09-14T09:06:29.618 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:06:29.618 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.618 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.618 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:06:29.618 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.618 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.618 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6820", 2024-09-14T09:06:29.618 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:06:29.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.618 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.618 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6821", 2024-09-14T09:06:29.619 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:06:29.619 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.619 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.619 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6817/2721245607", 2024-09-14T09:06:29.619 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6819/2721245607", 2024-09-14T09:06:29.619 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6823/2721245607", 2024-09-14T09:06:29.619 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6821/2721245607", 2024-09-14T09:06:29.619 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:06:29.619 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:06:29.619 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:06:29.619 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.620 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 7, 2024-09-14T09:06:29.620 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "e2f27d42-786b-449b-a2fb-9dd28f4e9788", 2024-09-14T09:06:29.620 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:06:29.620 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:06:29.620 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:06:29.620 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:06:29.620 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:06:29.620 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:06:29.620 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:06:29.620 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 53, 2024-09-14T09:06:29.620 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:06:29.620 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:06:29.620 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:06:29.621 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.621 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.621 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6800", 2024-09-14T09:06:29.621 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:06:29.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.621 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.621 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6801", 2024-09-14T09:06:29.621 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:06:29.621 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.621 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.621 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:06:29.622 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.622 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.622 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6802", 2024-09-14T09:06:29.622 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:06:29.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.622 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.622 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6803", 2024-09-14T09:06:29.622 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:06:29.622 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.622 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.622 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:06:29.622 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.623 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.623 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6806", 2024-09-14T09:06:29.623 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:06:29.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.623 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.623 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6807", 2024-09-14T09:06:29.623 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:06:29.623 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.623 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.623 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:06:29.623 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:06:29.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.623 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:06:29.624 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6804", 2024-09-14T09:06:29.624 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:06:29.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.624 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:06:29.624 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6805", 2024-09-14T09:06:29.624 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:06:29.624 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.624 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.624 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6801/2122823465", 2024-09-14T09:06:29.624 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6803/2122823465", 2024-09-14T09:06:29.624 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6807/2122823465", 2024-09-14T09:06:29.625 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6805/2122823465", 2024-09-14T09:06:29.625 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:06:29.625 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:06:29.625 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:06:29.625 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.625 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.625 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:06:29.625 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_xinfo": [ 2024-09-14T09:06:29.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.625 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 0, 2024-09-14T09:06:29.625 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:06:29.625 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:06:29.625 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:06:29.625 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:06:29.626 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:06:29.626 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.411789+0000", 2024-09-14T09:06:29.626 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:06:29.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.626 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 1, 2024-09-14T09:06:29.626 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:06:29.626 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:06:29.626 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:06:29.626 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:06:29.626 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:06:29.626 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:05.801873+0000", 2024-09-14T09:06:29.626 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:06:29.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.627 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 2, 2024-09-14T09:06:29.627 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:06:29.627 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:06:29.627 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:06:29.627 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:06:29.627 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:06:29.627 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.206725+0000", 2024-09-14T09:06:29.627 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:06:29.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.627 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 3, 2024-09-14T09:06:29.627 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:06:29.627 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:06:29.627 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:06:29.628 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:06:29.628 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:06:29.628 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:05.882180+0000", 2024-09-14T09:06:29.628 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:06:29.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.628 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 4, 2024-09-14T09:06:29.628 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:06:29.628 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:06:29.628 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:06:29.628 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:06:29.628 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:06:29.628 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.013864+0000", 2024-09-14T09:06:29.628 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:06:29.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.629 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 5, 2024-09-14T09:06:29.629 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:06:29.629 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:06:29.629 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:06:29.629 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:06:29.629 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:06:29.629 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.652975+0000", 2024-09-14T09:06:29.629 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:06:29.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.629 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 6, 2024-09-14T09:06:29.629 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:06:29.629 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:06:29.630 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:06:29.630 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:06:29.630 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:06:29.630 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.741359+0000", 2024-09-14T09:06:29.630 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:06:29.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.630 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 7, 2024-09-14T09:06:29.630 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:06:29.630 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:06:29.630 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:06:29.630 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:06:29.630 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:06:29.630 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:05.873675+0000", 2024-09-14T09:06:29.631 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:06:29.631 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.631 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:06:29.631 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_upmap": [ 2024-09-14T09:06:29.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:29.631 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "2.3", 2024-09-14T09:06:29.631 INFO:teuthology.orchestra.run.smithi176.stdout: "osds": [ 2024-09-14T09:06:29.631 INFO:teuthology.orchestra.run.smithi176.stdout: 3, 2024-09-14T09:06:29.631 INFO:teuthology.orchestra.run.smithi176.stdout: 2 2024-09-14T09:06:29.631 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:29.631 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.631 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:06:29.631 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_upmap_items": [], 2024-09-14T09:06:29.631 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_upmap_primaries": [], 2024-09-14T09:06:29.631 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_temp": [], 2024-09-14T09:06:29.631 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_temp": [], 2024-09-14T09:06:29.632 INFO:teuthology.orchestra.run.smithi176.stdout: "blocklist": {}, 2024-09-14T09:06:29.632 INFO:teuthology.orchestra.run.smithi176.stdout: "range_blocklist": {}, 2024-09-14T09:06:29.632 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profiles": { 2024-09-14T09:06:29.632 INFO:teuthology.orchestra.run.smithi176.stdout: "default": { 2024-09-14T09:06:29.632 INFO:teuthology.orchestra.run.smithi176.stdout: "crush-failure-domain": "osd", 2024-09-14T09:06:29.632 INFO:teuthology.orchestra.run.smithi176.stdout: "k": "2", 2024-09-14T09:06:29.632 INFO:teuthology.orchestra.run.smithi176.stdout: "m": "1", 2024-09-14T09:06:29.632 INFO:teuthology.orchestra.run.smithi176.stdout: "plugin": "jerasure", 2024-09-14T09:06:29.632 INFO:teuthology.orchestra.run.smithi176.stdout: "technique": "reed_sol_van" 2024-09-14T09:06:29.632 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:29.632 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps_queue": [], 2024-09-14T09:06:29.632 INFO:teuthology.orchestra.run.smithi176.stdout: "new_removed_snaps": [], 2024-09-14T09:06:29.633 INFO:teuthology.orchestra.run.smithi176.stdout: "new_purged_snaps": [], 2024-09-14T09:06:29.633 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_node_flags": {}, 2024-09-14T09:06:29.633 INFO:teuthology.orchestra.run.smithi176.stdout: "device_class_flags": {}, 2024-09-14T09:06:29.633 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_mode": { 2024-09-14T09:06:29.633 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_mode_enabled": false, 2024-09-14T09:06:29.633 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_bucket_count": 0, 2024-09-14T09:06:29.633 INFO:teuthology.orchestra.run.smithi176.stdout: "degraded_stretch_mode": 0, 2024-09-14T09:06:29.633 INFO:teuthology.orchestra.run.smithi176.stdout: "recovering_stretch_mode": 0, 2024-09-14T09:06:29.633 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_mode_bucket": 0 2024-09-14T09:06:29.633 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:29.633 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:29.634 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:29.634 INFO:teuthology.orchestra.run.smithi176.stdout:3105 write 0xaad1d06 thru 0xaad9db8 (0x80b3 bytes) 2024-09-14T09:06:29.634 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:29.634 INFO:teuthology.orchestra.run.smithi176.stdout:3109 write 0x9dd8b58 thru 0x9de2fa8 (0xa451 bytes) 2024-09-14T09:06:29.636 INFO:tasks.thrashosds.thrasher:j is {'epoch': 71, 'fsid': 'c635667d-994e-454f-bcea-51b45637b2a2', 'created': '2024-09-14T09:04:57.077293+0000', 'modified': '2024-09-14T09:06:29.226690+0000', 'last_up_change': '2024-09-14T09:05:11.210835+0000', 'last_in_change': '2024-09-14T09:04:59.344709+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T09:05:11.997673+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T09:05:15.352551+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T09:05:25.840963+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 26, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '4ceedb4f-6398-467f-9562-ccf27200860b', 'up': 1, 'in': 1, 'weight': 0.8736724853515625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6824', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6825', 'nonce': 3384710819}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6826', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6827', 'nonce': 3384710819}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6830', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6831', 'nonce': 3384710819}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6828', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6829', 'nonce': 3384710819}]}, 'public_addr': '172.21.15.176:6825/3384710819', 'cluster_addr': '172.21.15.176:6827/3384710819', 'heartbeat_back_addr': '172.21.15.176:6831/3384710819', 'heartbeat_front_addr': '172.21.15.176:6829/3384710819', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '604e2f62-7c13-46be-a97f-029b28c44a89', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6800', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6801', 'nonce': 612012679}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6802', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6803', 'nonce': 612012679}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6806', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6807', 'nonce': 612012679}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6804', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6805', 'nonce': 612012679}]}, 'public_addr': '172.21.15.176:6801/612012679', 'cluster_addr': '172.21.15.176:6803/612012679', 'heartbeat_back_addr': '172.21.15.176:6807/612012679', 'heartbeat_front_addr': '172.21.15.176:6805/612012679', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1f65589e-8b0a-4033-94c0-4ab66531db2b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6808', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6809', 'nonce': 4167379279}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6810', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6811', 'nonce': 4167379279}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6814', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6815', 'nonce': 4167379279}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6812', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6813', 'nonce': 4167379279}]}, 'public_addr': '172.21.15.176:6809/4167379279', 'cluster_addr': '172.21.15.176:6811/4167379279', 'heartbeat_back_addr': '172.21.15.176:6815/4167379279', 'heartbeat_front_addr': '172.21.15.176:6813/4167379279', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '209d4f65-79f9-4a7d-9d23-832a4209507f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 67, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6816', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6817', 'nonce': 1658209015}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6818', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6819', 'nonce': 1658209015}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6822', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6823', 'nonce': 1658209015}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6820', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6821', 'nonce': 1658209015}]}, 'public_addr': '172.21.15.176:6817/1658209015', 'cluster_addr': '172.21.15.176:6819/1658209015', 'heartbeat_back_addr': '172.21.15.176:6823/1658209015', 'heartbeat_front_addr': '172.21.15.176:6821/1658209015', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8f9a8c9b-282e-4523-9e62-5824f4aba401', 'up': 1, 'in': 1, 'weight': 0.1816558837890625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6808', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6809', 'nonce': 1576054389}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6810', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6811', 'nonce': 1576054389}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6814', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6815', 'nonce': 1576054389}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6812', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6813', 'nonce': 1576054389}]}, 'public_addr': '172.21.15.183:6809/1576054389', 'cluster_addr': '172.21.15.183:6811/1576054389', 'heartbeat_back_addr': '172.21.15.183:6815/1576054389', 'heartbeat_front_addr': '172.21.15.183:6813/1576054389', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6824', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6825', 'nonce': 1711292386}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6826', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6827', 'nonce': 1711292386}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6830', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6831', 'nonce': 1711292386}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6828', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6829', 'nonce': 1711292386}]}, 'public_addr': '172.21.15.183:6825/1711292386', 'cluster_addr': '172.21.15.183:6827/1711292386', 'heartbeat_back_addr': '172.21.15.183:6831/1711292386', 'heartbeat_front_addr': '172.21.15.183:6829/1711292386', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '59d9259d-b9dd-4326-8ab9-9d599a2bf57b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6816', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6817', 'nonce': 2721245607}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6818', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6819', 'nonce': 2721245607}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6822', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6823', 'nonce': 2721245607}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6820', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6821', 'nonce': 2721245607}]}, 'public_addr': '172.21.15.183:6817/2721245607', 'cluster_addr': '172.21.15.183:6819/2721245607', 'heartbeat_back_addr': '172.21.15.183:6823/2721245607', 'heartbeat_front_addr': '172.21.15.183:6821/2721245607', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'e2f27d42-786b-449b-a2fb-9dd28f4e9788', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6800', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6801', 'nonce': 2122823465}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6802', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6803', 'nonce': 2122823465}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6806', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6807', 'nonce': 2122823465}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6804', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6805', 'nonce': 2122823465}]}, 'public_addr': '172.21.15.183:6801/2122823465', 'cluster_addr': '172.21.15.183:6803/2122823465', 'heartbeat_back_addr': '172.21.15.183:6807/2122823465', 'heartbeat_front_addr': '172.21.15.183:6805/2122823465', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.411789+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:05.801873+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.206725+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:05.882180+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.013864+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.652975+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.741359+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:05.873675+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.3', 'osds': [3, 2]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T09:06:29.636 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:06:29.637 INFO:teuthology.orchestra.run.smithi176.stdout:3110 punch from 0x7031bda to 0x7040dca, (0xf1f0 bytes) 2024-09-14T09:06:29.637 INFO:teuthology.orchestra.run.smithi176.stdout:3111 writesame 0x2f5368d thru 0x2f5cdb5 data_size 0x1(0x9729 bytes) 2024-09-14T09:06:29.659 INFO:teuthology.orchestra.run.smithi176.stdout:3112 read 0x8154631 thru 0x8160fcc (0xc99c bytes) 2024-09-14T09:06:29.675 INFO:teuthology.orchestra.run.smithi176.stdout:3113 write 0xb983167 thru 0xb98facc (0xc966 bytes) 2024-09-14T09:06:29.678 INFO:teuthology.orchestra.run.smithi176.stdout:3114 read 0x12352ee thru 0x123588d (0x5a0 bytes) 2024-09-14T09:06:29.679 INFO:teuthology.orchestra.run.smithi176.stdout:3116 write 0x21c00f1 thru 0x21c78b9 (0x77c9 bytes) 2024-09-14T09:06:29.683 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:29.687+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:29.688 INFO:teuthology.orchestra.run.smithi176.stdout:3117 write 0x30eb0b7 thru 0x30fac9d (0xfbe7 bytes) 2024-09-14T09:06:29.697 INFO:teuthology.orchestra.run.smithi176.stdout:3118 read 0x7f3da6d thru 0x7f435d7 (0x5b6b bytes) 2024-09-14T09:06:29.697 INFO:teuthology.orchestra.run.smithi176.stdout:3119 read 0xf2894e thru 0xf31e8d (0x9540 bytes) 2024-09-14T09:06:29.698 INFO:teuthology.orchestra.run.smithi176.stdout:3120 writesame 0x7d0e0cf thru 0x7d0ed6e data_size 0x10(0xca0 bytes) 2024-09-14T09:06:29.704 INFO:teuthology.orchestra.run.smithi176.stdout:3121 writesame 0x9e9f3f7 thru 0x9ea4e90 data_size 0x1(0x5a9a bytes) 2024-09-14T09:06:29.734 INFO:teuthology.orchestra.run.smithi176.stdout:3122 read 0xa2376d thru 0xa24104 (0x998 bytes) 2024-09-14T09:06:29.735 INFO:teuthology.orchestra.run.smithi176.stdout:3124 writesame 0x20d73ac thru 0x20e3cd8 data_size 0x1(0xc92d bytes) 2024-09-14T09:06:29.760 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:29.759+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:29.761 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:29.761 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:29.761 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:29.761 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:29.775 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:06:29.860 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:29.859+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:29.885 INFO:teuthology.orchestra.run.smithi176.stdout:3125 writesame 0x9eb9162 thru 0x9ec7bb1 data_size 0xa3(0xea50 bytes) 2024-09-14T09:06:29.887 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:29.887 INFO:teuthology.orchestra.run.smithi176.stdout:3127 write 0x86df529 thru 0x86eacd4 (0xb7ac bytes) 2024-09-14T09:06:29.904 INFO:teuthology.orchestra.run.smithi176.stdout:3128 write 0xd7be170 thru 0xd7c613c (0x7fcd bytes) 2024-09-14T09:06:29.910 INFO:teuthology.orchestra.run.smithi176.stdout:3129 writesame 0x7b3a50d thru 0x7b4407d data_size 0x1(0x9b71 bytes) 2024-09-14T09:06:29.941 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:29.942 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:29.942 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:29.942 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:29.942 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:29.957 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:06:29.960 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:29.959+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:29.987 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:06:29.988 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:06:30.006 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-14T09:06:29.980062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":917551045,"num_objects":1152,"num_object_clones":367,"num_object_copies":2304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1152,"num_whiteouts":440,"num_read":93598,"num_read_kb":475163,"num_write":11008,"num_write_kb":189991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":11394597,"num_keys_recovered":32,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8731,"ondisk_log_size":8731,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":328616,"kb_used_data":33168,"kb_used_omap":12,"kb_used_meta":295091,"kb_avail":754646104,"statfs":{"total":773094113280,"available":772757610496,"internally_reserved":0,"allocated":33964032,"data_stored":238695438,"data_compressed":7298254,"data_compressed_allocated":19378176,"data_compressed_original":230814184,"omap_allocated":12751,"internal_metadata":302173745},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28235406,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":6748,"num_read_kb":12817,"num_write":1377,"num_write_kb":9004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.775723"},"pg_stats":[{"pgid":"3.1f","version":"69'884","reported_seq":3211,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T09:06:26.474587+0000","last_change":"2024-09-14T09:05:33.869426+0000","last_active":"2024-09-14T09:06:26.474587+0000","last_peered":"2024-09-14T09:06:26.474587+0000","last_clean":"2024-09-14T09:06:26.474587+0000","last_became_active":"2024-09-14T09:05:33.854933+0000","last_became_peered":"2024-09-14T09:05:33.854933+0000","last_unstale":"2024-09-14T09:06:26.474587+0000","last_undegraded":"2024-09-14T09:06:26.474587+0000","last_fullsized":"2024-09-14T09:06:26.474587+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:57:37.732431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001387512,"stat_sum":{"num_bytes":4258183,"num_objects":37,"num_object_clones":12,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":6499,"num_read_kb":3654,"num_write":1520,"num_write_kb":1924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"68'179","reported_seq":2351,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.897805+0000","last_change":"2024-09-14T09:05:29.039503+0000","last_active":"2024-09-14T09:06:25.897805+0000","last_peered":"2024-09-14T09:06:25.897805+0000","last_clean":"2024-09-14T09:06:25.897805+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:06:25.897805+0000","last_undegraded":"2024-09-14T09:06:25.897805+0000","last_fullsized":"2024-09-14T09:06:25.897805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:00:43.704148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001657176,"stat_sum":{"num_bytes":31504424,"num_objects":50,"num_object_clones":18,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":20,"num_read":2134,"num_read_kb":21091,"num_write":215,"num_write_kb":8535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"69'333","reported_seq":3663,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T09:06:26.622643+0000","last_change":"2024-09-14T09:05:28.835298+0000","last_active":"2024-09-14T09:06:26.622643+0000","last_peered":"2024-09-14T09:06:26.622643+0000","last_clean":"2024-09-14T09:06:26.622643+0000","last_became_active":"2024-09-14T09:05:26.835372+0000","last_became_peered":"2024-09-14T09:05:26.835372+0000","last_unstale":"2024-09-14T09:06:26.622643+0000","last_undegraded":"2024-09-14T09:06:26.622643+0000","last_fullsized":"2024-09-14T09:06:26.622643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:15:57.494246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014149329999999999,"stat_sum":{"num_bytes":55397153,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":10,"num_read":3984,"num_read_kb":52885,"num_write":478,"num_write_kb":8421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"69'492","reported_seq":1921,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T09:06:27.021651+0000","last_change":"2024-09-14T09:05:39.931845+0000","last_active":"2024-09-14T09:06:27.021651+0000","last_peered":"2024-09-14T09:06:27.021651+0000","last_clean":"2024-09-14T09:06:27.021651+0000","last_became_active":"2024-09-14T09:05:39.918593+0000","last_became_peered":"2024-09-14T09:05:39.918593+0000","last_unstale":"2024-09-14T09:06:27.021651+0000","last_undegraded":"2024-09-14T09:06:27.021651+0000","last_fullsized":"2024-09-14T09:06:27.021651+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:13:32.252943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014661419999999999,"stat_sum":{"num_bytes":21248736,"num_objects":34,"num_object_clones":12,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":4346,"num_read_kb":7178,"num_write":666,"num_write_kb":4507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":17,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"67'189","reported_seq":2132,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.206458+0000","last_change":"2024-09-14T09:06:03.834450+0000","last_active":"2024-09-14T09:06:25.206458+0000","last_peered":"2024-09-14T09:06:25.206458+0000","last_clean":"2024-09-14T09:06:25.206458+0000","last_became_active":"2024-09-14T09:06:03.833448+0000","last_became_peered":"2024-09-14T09:06:03.833448+0000","last_unstale":"2024-09-14T09:06:25.206458+0000","last_undegraded":"2024-09-14T09:06:25.206458+0000","last_fullsized":"2024-09-14T09:06:25.206458+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:30:30.839885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060373399999999998,"stat_sum":{"num_bytes":21403007,"num_objects":36,"num_object_clones":16,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":1861,"num_read_kb":5534,"num_write":190,"num_write_kb":7318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":801985,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"68'469","reported_seq":3065,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.993608+0000","last_change":"2024-09-14T09:05:39.919093+0000","last_active":"2024-09-14T09:06:25.993608+0000","last_peered":"2024-09-14T09:06:25.993608+0000","last_clean":"2024-09-14T09:06:25.993608+0000","last_became_active":"2024-09-14T09:05:39.905126+0000","last_became_peered":"2024-09-14T09:05:39.905126+0000","last_unstale":"2024-09-14T09:06:25.993608+0000","last_undegraded":"2024-09-14T09:06:25.993608+0000","last_fullsized":"2024-09-14T09:06:25.993608+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:18:50.653602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020845550000000001,"stat_sum":{"num_bytes":28648388,"num_objects":32,"num_object_clones":10,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":12,"num_read":4201,"num_read_kb":18260,"num_write":752,"num_write_kb":7956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":92669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"68'194","reported_seq":1996,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.977686+0000","last_change":"2024-09-14T09:05:28.822565+0000","last_active":"2024-09-14T09:06:25.977686+0000","last_peered":"2024-09-14T09:06:25.977686+0000","last_clean":"2024-09-14T09:06:25.977686+0000","last_became_active":"2024-09-14T09:05:26.816132+0000","last_became_peered":"2024-09-14T09:05:26.816132+0000","last_unstale":"2024-09-14T09:06:25.977686+0000","last_undegraded":"2024-09-14T09:06:25.977686+0000","last_fullsized":"2024-09-14T09:06:25.977686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:15:38.663444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092420299999999996,"stat_sum":{"num_bytes":18328878,"num_objects":30,"num_object_clones":11,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":1755,"num_read_kb":5415,"num_write":231,"num_write_kb":3991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"68'241","reported_seq":3073,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.920052+0000","last_change":"2024-09-14T09:05:33.865470+0000","last_active":"2024-09-14T09:06:25.920052+0000","last_peered":"2024-09-14T09:06:25.920052+0000","last_clean":"2024-09-14T09:06:25.920052+0000","last_became_active":"2024-09-14T09:05:33.846990+0000","last_became_peered":"2024-09-14T09:05:33.846990+0000","last_unstale":"2024-09-14T09:06:25.920052+0000","last_undegraded":"2024-09-14T09:06:25.920052+0000","last_fullsized":"2024-09-14T09:06:25.920052+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:32:27.329301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081579899999999997,"stat_sum":{"num_bytes":47387004,"num_objects":44,"num_object_clones":17,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":14,"num_read":3626,"num_read_kb":7865,"num_write":230,"num_write_kb":3650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":70784,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"69'309","reported_seq":4268,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T09:06:27.051755+0000","last_change":"2024-09-14T09:05:28.839928+0000","last_active":"2024-09-14T09:06:27.051755+0000","last_peered":"2024-09-14T09:06:27.051755+0000","last_clean":"2024-09-14T09:06:27.051755+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:06:27.051755+0000","last_undegraded":"2024-09-14T09:06:27.051755+0000","last_fullsized":"2024-09-14T09:06:27.051755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:54:13.165203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014343089999999999,"stat_sum":{"num_bytes":29478117,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":10,"num_read":4957,"num_read_kb":22082,"num_write":392,"num_write_kb":4554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"69'366","reported_seq":1462,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T09:06:27.083805+0000","last_change":"2024-09-14T09:05:28.836370+0000","last_active":"2024-09-14T09:06:27.083805+0000","last_peered":"2024-09-14T09:06:27.083805+0000","last_clean":"2024-09-14T09:06:27.083805+0000","last_became_active":"2024-09-14T09:05:26.836744+0000","last_became_peered":"2024-09-14T09:05:26.836744+0000","last_unstale":"2024-09-14T09:06:27.083805+0000","last_undegraded":"2024-09-14T09:06:27.083805+0000","last_fullsized":"2024-09-14T09:06:27.083805+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:00:23.823360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015277190000000001,"stat_sum":{"num_bytes":14293559,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":9,"num_read":3070,"num_read_kb":5247,"num_write":496,"num_write_kb":3458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"68'489","reported_seq":1582,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.264293+0000","last_change":"2024-09-14T09:05:28.851587+0000","last_active":"2024-09-14T09:06:25.264293+0000","last_peered":"2024-09-14T09:06:25.264293+0000","last_clean":"2024-09-14T09:06:25.264293+0000","last_became_active":"2024-09-14T09:05:26.840893+0000","last_became_peered":"2024-09-14T09:05:26.840893+0000","last_unstale":"2024-09-14T09:06:25.264293+0000","last_undegraded":"2024-09-14T09:06:25.264293+0000","last_fullsized":"2024-09-14T09:06:25.264293+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:16:08.331581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098594000000000004,"stat_sum":{"num_bytes":8828839,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":3902,"num_read_kb":3835,"num_write":755,"num_write_kb":2577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"68'302","reported_seq":1448,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.610449+0000","last_change":"2024-09-14T09:05:42.832730+0000","last_active":"2024-09-14T09:06:25.610449+0000","last_peered":"2024-09-14T09:06:25.610449+0000","last_clean":"2024-09-14T09:06:25.610449+0000","last_became_active":"2024-09-14T09:05:42.829504+0000","last_became_peered":"2024-09-14T09:05:42.829504+0000","last_unstale":"2024-09-14T09:06:25.610449+0000","last_undegraded":"2024-09-14T09:06:25.610449+0000","last_fullsized":"2024-09-14T09:06:25.610449+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:40:57.100894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014079679999999999,"stat_sum":{"num_bytes":20404444,"num_objects":39,"num_object_clones":13,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":2179,"num_read_kb":3766,"num_write":350,"num_write_kb":779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":53852,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T09:06:26.282949+0000","last_change":"2024-09-14T09:05:19.793674+0000","last_active":"2024-09-14T09:06:26.282949+0000","last_peered":"2024-09-14T09:06:26.282949+0000","last_clean":"2024-09-14T09:06:26.282949+0000","last_became_active":"2024-09-14T09:05:17.284977+0000","last_became_peered":"2024-09-14T09:05:17.284977+0000","last_unstale":"2024-09-14T09:06:26.282949+0000","last_undegraded":"2024-09-14T09:06:26.282949+0000","last_fullsized":"2024-09-14T09:06:26.282949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:30:38.832027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"69'203","reported_seq":3460,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T09:06:27.019316+0000","last_change":"2024-09-14T09:05:28.839614+0000","last_active":"2024-09-14T09:06:27.019316+0000","last_peered":"2024-09-14T09:06:27.019316+0000","last_clean":"2024-09-14T09:06:27.019316+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:06:27.019316+0000","last_undegraded":"2024-09-14T09:06:27.019316+0000","last_fullsized":"2024-09-14T09:06:27.019316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:50:30.650903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014245869999999999,"stat_sum":{"num_bytes":39824462,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":12,"num_read":4231,"num_read_kb":37664,"num_write":220,"num_write_kb":5550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":112,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.252357+0000","last_change":"2024-09-14T09:05:19.314029+0000","last_active":"2024-09-14T09:06:25.252357+0000","last_peered":"2024-09-14T09:06:25.252357+0000","last_clean":"2024-09-14T09:06:25.252357+0000","last_became_active":"2024-09-14T09:05:17.282890+0000","last_became_peered":"2024-09-14T09:05:17.282890+0000","last_unstale":"2024-09-14T09:06:25.252357+0000","last_undegraded":"2024-09-14T09:06:25.252357+0000","last_fullsized":"2024-09-14T09:06:25.252357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:25.304105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"68'335","reported_seq":2269,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.272370+0000","last_change":"2024-09-14T09:05:29.039154+0000","last_active":"2024-09-14T09:06:25.272370+0000","last_peered":"2024-09-14T09:06:25.272370+0000","last_clean":"2024-09-14T09:06:25.272370+0000","last_became_active":"2024-09-14T09:05:26.829855+0000","last_became_peered":"2024-09-14T09:05:26.829855+0000","last_unstale":"2024-09-14T09:06:25.272370+0000","last_undegraded":"2024-09-14T09:06:25.272370+0000","last_fullsized":"2024-09-14T09:06:25.272370+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:53:37.851193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019213749999999999,"stat_sum":{"num_bytes":37534250,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":3241,"num_read_kb":12288,"num_write":445,"num_write_kb":3706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":112,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.264439+0000","last_change":"2024-09-14T09:05:19.794165+0000","last_active":"2024-09-14T09:06:25.264439+0000","last_peered":"2024-09-14T09:06:25.264439+0000","last_clean":"2024-09-14T09:06:25.264439+0000","last_became_active":"2024-09-14T09:05:17.295864+0000","last_became_peered":"2024-09-14T09:05:17.295864+0000","last_unstale":"2024-09-14T09:06:25.264439+0000","last_undegraded":"2024-09-14T09:06:25.264439+0000","last_fullsized":"2024-09-14T09:06:25.264439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:53.901865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011399719999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"68'180","reported_seq":878,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.252201+0000","last_change":"2024-09-14T09:05:28.823608+0000","last_active":"2024-09-14T09:06:25.252201+0000","last_peered":"2024-09-14T09:06:25.252201+0000","last_clean":"2024-09-14T09:06:25.252201+0000","last_became_active":"2024-09-14T09:05:26.816468+0000","last_became_peered":"2024-09-14T09:05:26.816468+0000","last_unstale":"2024-09-14T09:06:25.252201+0000","last_undegraded":"2024-09-14T09:06:25.252201+0000","last_fullsized":"2024-09-14T09:06:25.252201+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:01:06.744652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014087959999999999,"stat_sum":{"num_bytes":8423935,"num_objects":40,"num_object_clones":15,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":1148,"num_read_kb":2383,"num_write":150,"num_write_kb":2589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":112,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.252231+0000","last_change":"2024-09-14T09:05:19.314129+0000","last_active":"2024-09-14T09:06:25.252231+0000","last_peered":"2024-09-14T09:06:25.252231+0000","last_clean":"2024-09-14T09:06:25.252231+0000","last_became_active":"2024-09-14T09:05:17.295196+0000","last_became_peered":"2024-09-14T09:05:17.295196+0000","last_unstale":"2024-09-14T09:06:25.252231+0000","last_undegraded":"2024-09-14T09:06:25.252231+0000","last_fullsized":"2024-09-14T09:06:25.252231+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:35.269115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011613800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"68'153","reported_seq":809,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T09:06:27.067213+0000","last_change":"2024-09-14T09:05:28.835976+0000","last_active":"2024-09-14T09:06:27.067213+0000","last_peered":"2024-09-14T09:06:27.067213+0000","last_clean":"2024-09-14T09:06:27.067213+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:06:27.067213+0000","last_undegraded":"2024-09-14T09:06:27.067213+0000","last_fullsized":"2024-09-14T09:06:27.067213+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:05.604586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017480779999999999,"stat_sum":{"num_bytes":49329019,"num_objects":44,"num_object_clones":16,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":21,"num_read":588,"num_read_kb":9386,"num_write":140,"num_write_kb":8842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"69'241","reported_seq":1337,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T09:06:27.186134+0000","last_change":"2024-09-14T09:05:39.930107+0000","last_active":"2024-09-14T09:06:27.186134+0000","last_peered":"2024-09-14T09:06:27.186134+0000","last_clean":"2024-09-14T09:06:27.186134+0000","last_became_active":"2024-09-14T09:05:39.914913+0000","last_became_peered":"2024-09-14T09:05:39.914913+0000","last_unstale":"2024-09-14T09:06:27.186134+0000","last_undegraded":"2024-09-14T09:06:27.186134+0000","last_fullsized":"2024-09-14T09:06:27.186134+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:49:30.769472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014128,"stat_sum":{"num_bytes":25020722,"num_objects":34,"num_object_clones":12,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":1492,"num_read_kb":14754,"num_write":320,"num_write_kb":8694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":116,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T09:06:26.226747+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:06:26.226747+0000","last_peered":"2024-09-14T09:06:26.226747+0000","last_clean":"2024-09-14T09:06:26.226747+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:06:26.226747+0000","last_undegraded":"2024-09-14T09:06:26.226747+0000","last_fullsized":"2024-09-14T09:06:26.226747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"68'160","reported_seq":838,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.766047+0000","last_change":"2024-09-14T09:05:28.822791+0000","last_active":"2024-09-14T09:06:25.766047+0000","last_peered":"2024-09-14T09:06:25.766047+0000","last_clean":"2024-09-14T09:06:25.766047+0000","last_became_active":"2024-09-14T09:05:26.817109+0000","last_became_peered":"2024-09-14T09:05:26.817109+0000","last_unstale":"2024-09-14T09:06:25.766047+0000","last_undegraded":"2024-09-14T09:06:25.766047+0000","last_fullsized":"2024-09-14T09:06:25.766047+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:22:32.671991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011728439999999999,"stat_sum":{"num_bytes":13646081,"num_objects":38,"num_object_clones":12,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":16,"num_read":762,"num_read_kb":3085,"num_write":169,"num_write_kb":761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T09:06:26.221104+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:06:26.221104+0000","last_peered":"2024-09-14T09:06:26.221104+0000","last_clean":"2024-09-14T09:06:26.221104+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:06:26.221104+0000","last_undegraded":"2024-09-14T09:06:26.221104+0000","last_fullsized":"2024-09-14T09:06:26.221104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"69'163","reported_seq":1199,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T09:06:27.018713+0000","last_change":"2024-09-14T09:05:39.925683+0000","last_active":"2024-09-14T09:06:27.018713+0000","last_peered":"2024-09-14T09:06:27.018713+0000","last_clean":"2024-09-14T09:06:27.018713+0000","last_became_active":"2024-09-14T09:05:39.905252+0000","last_became_peered":"2024-09-14T09:05:39.905252+0000","last_unstale":"2024-09-14T09:06:27.018713+0000","last_undegraded":"2024-09-14T09:06:27.018713+0000","last_fullsized":"2024-09-14T09:06:27.018713+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:31:55.257886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074815999999999999,"stat_sum":{"num_bytes":18966843,"num_objects":29,"num_object_clones":11,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":13,"num_read":944,"num_read_kb":5577,"num_write":165,"num_write_kb":8671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2097152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":112,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.263705+0000","last_change":"2024-09-14T09:05:19.793999+0000","last_active":"2024-09-14T09:06:25.263705+0000","last_peered":"2024-09-14T09:06:25.263705+0000","last_clean":"2024-09-14T09:06:25.263705+0000","last_became_active":"2024-09-14T09:05:17.284122+0000","last_became_peered":"2024-09-14T09:05:17.284122+0000","last_unstale":"2024-09-14T09:06:25.263705+0000","last_undegraded":"2024-09-14T09:06:25.263705+0000","last_fullsized":"2024-09-14T09:06:25.263705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:12:01.133215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010234269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"68'195","reported_seq":1028,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.886505+0000","last_change":"2024-09-14T09:05:29.038389+0000","last_active":"2024-09-14T09:06:25.886505+0000","last_peered":"2024-09-14T09:06:25.886505+0000","last_clean":"2024-09-14T09:06:25.886505+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:06:25.886505+0000","last_undegraded":"2024-09-14T09:06:25.886505+0000","last_fullsized":"2024-09-14T09:06:25.886505+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:42.193695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001479076,"stat_sum":{"num_bytes":19439154,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":1018,"num_read_kb":3518,"num_write":238,"num_write_kb":2073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.228752+0000","last_change":"2024-09-14T09:06:25.227644+0000","last_active":"2024-09-14T09:06:25.228752+0000","last_peered":"2024-09-14T09:06:25.228752+0000","last_clean":"2024-09-14T09:06:25.228752+0000","last_became_active":"2024-09-14T09:06:25.222344+0000","last_became_peered":"2024-09-14T09:06:25.222344+0000","last_unstale":"2024-09-14T09:06:25.228752+0000","last_undegraded":"2024-09-14T09:06:25.228752+0000","last_fullsized":"2024-09-14T09:06:25.228752+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:23:33.414859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":203,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.263523+0000","last_change":"2024-09-14T09:05:13.239808+0000","last_active":"2024-09-14T09:06:25.263523+0000","last_peered":"2024-09-14T09:06:25.263523+0000","last_clean":"2024-09-14T09:06:25.263523+0000","last_became_active":"2024-09-14T09:05:13.239064+0000","last_became_peered":"2024-09-14T09:05:13.239064+0000","last_unstale":"2024-09-14T09:06:25.263523+0000","last_undegraded":"2024-09-14T09:06:25.263523+0000","last_fullsized":"2024-09-14T09:06:25.263523+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:40:11.559161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"68'205","reported_seq":3199,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.979284+0000","last_change":"2024-09-14T09:05:29.037923+0000","last_active":"2024-09-14T09:06:25.979284+0000","last_peered":"2024-09-14T09:06:25.979284+0000","last_clean":"2024-09-14T09:06:25.979284+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:06:25.979284+0000","last_undegraded":"2024-09-14T09:06:25.979284+0000","last_fullsized":"2024-09-14T09:06:25.979284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:45:22.159293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099199800000000001,"stat_sum":{"num_bytes":42354761,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":3453,"num_read_kb":29435,"num_write":223,"num_write_kb":7317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"69'167","reported_seq":1354,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T09:06:27.055720+0000","last_change":"2024-09-14T09:05:39.933291+0000","last_active":"2024-09-14T09:06:27.055720+0000","last_peered":"2024-09-14T09:06:27.055720+0000","last_clean":"2024-09-14T09:06:27.055720+0000","last_became_active":"2024-09-14T09:05:39.924025+0000","last_became_peered":"2024-09-14T09:05:39.924025+0000","last_unstale":"2024-09-14T09:06:27.055720+0000","last_undegraded":"2024-09-14T09:06:27.055720+0000","last_fullsized":"2024-09-14T09:06:27.055720+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:47:41.330286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001266028,"stat_sum":{"num_bytes":27628229,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":1121,"num_read_kb":6589,"num_write":149,"num_write_kb":13929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"69'227","reported_seq":3004,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T09:06:26.669472+0000","last_change":"2024-09-14T09:05:39.975358+0000","last_active":"2024-09-14T09:06:26.669472+0000","last_peered":"2024-09-14T09:06:26.669472+0000","last_clean":"2024-09-14T09:06:26.669472+0000","last_became_active":"2024-09-14T09:05:39.918526+0000","last_became_peered":"2024-09-14T09:05:39.918526+0000","last_unstale":"2024-09-14T09:06:26.669472+0000","last_undegraded":"2024-09-14T09:06:26.669472+0000","last_fullsized":"2024-09-14T09:06:26.669472+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:28:53.175668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185731,"stat_sum":{"num_bytes":47745576,"num_objects":38,"num_object_clones":11,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":15,"num_read":3287,"num_read_kb":31772,"num_write":257,"num_write_kb":15589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":2159421,"num_keys_recovered":3,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"68'269","reported_seq":7706,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:26.067349+0000","last_change":"2024-09-14T09:06:02.837388+0000","last_active":"2024-09-14T09:06:26.067349+0000","last_peered":"2024-09-14T09:06:26.067349+0000","last_clean":"2024-09-14T09:06:26.067349+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:06:26.067349+0000","last_undegraded":"2024-09-14T09:06:26.067349+0000","last_fullsized":"2024-09-14T09:06:26.067349+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:09:23.942445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001585295,"stat_sum":{"num_bytes":37997167,"num_objects":38,"num_object_clones":12,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":14,"num_read":8420,"num_read_kb":52629,"num_write":269,"num_write_kb":6743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"68'194","reported_seq":1125,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T09:06:27.061686+0000","last_change":"2024-09-14T09:05:28.834647+0000","last_active":"2024-09-14T09:06:27.061686+0000","last_peered":"2024-09-14T09:06:27.061686+0000","last_clean":"2024-09-14T09:06:27.061686+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:06:27.061686+0000","last_undegraded":"2024-09-14T09:06:27.061686+0000","last_fullsized":"2024-09-14T09:06:27.061686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:12:11.247515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244290000000001,"stat_sum":{"num_bytes":24928842,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":14,"num_read":1347,"num_read_kb":7145,"num_write":208,"num_write_kb":6829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"68'196","reported_seq":2716,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.897740+0000","last_change":"2024-09-14T09:05:28.851561+0000","last_active":"2024-09-14T09:06:25.897740+0000","last_peered":"2024-09-14T09:06:25.897740+0000","last_clean":"2024-09-14T09:06:25.897740+0000","last_became_active":"2024-09-14T09:05:26.843780+0000","last_became_peered":"2024-09-14T09:05:26.843780+0000","last_unstale":"2024-09-14T09:06:25.897740+0000","last_undegraded":"2024-09-14T09:06:25.897740+0000","last_fullsized":"2024-09-14T09:06:25.897740+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:53:52.806963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072987200000000001,"stat_sum":{"num_bytes":23103013,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":2761,"num_read_kb":19368,"num_write":231,"num_write_kb":10902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"68'146","reported_seq":1136,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.391156+0000","last_change":"2024-09-14T09:05:40.010535+0000","last_active":"2024-09-14T09:06:25.391156+0000","last_peered":"2024-09-14T09:06:25.391156+0000","last_clean":"2024-09-14T09:06:25.391156+0000","last_became_active":"2024-09-14T09:05:39.921575+0000","last_became_peered":"2024-09-14T09:05:39.921575+0000","last_unstale":"2024-09-14T09:06:25.391156+0000","last_undegraded":"2024-09-14T09:06:25.391156+0000","last_fullsized":"2024-09-14T09:06:25.391156+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:10:49.796014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057596100000000005,"stat_sum":{"num_bytes":30069074,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":13,"num_read":877,"num_read_kb":8697,"num_write":132,"num_write_kb":4337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":111591,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"68'188","reported_seq":937,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.774172+0000","last_change":"2024-09-14T09:06:02.836145+0000","last_active":"2024-09-14T09:06:25.774172+0000","last_peered":"2024-09-14T09:06:25.774172+0000","last_clean":"2024-09-14T09:06:25.774172+0000","last_became_active":"2024-09-14T09:06:02.834684+0000","last_became_peered":"2024-09-14T09:06:02.834684+0000","last_unstale":"2024-09-14T09:06:25.774172+0000","last_undegraded":"2024-09-14T09:06:25.774172+0000","last_fullsized":"2024-09-14T09:06:25.774172+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:01:59.355670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000909463,"stat_sum":{"num_bytes":25504728,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":1026,"num_read_kb":1784,"num_write":155,"num_write_kb":4782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":259265,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"69'206","reported_seq":3948,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T09:06:27.098203+0000","last_change":"2024-09-14T09:06:00.166357+0000","last_active":"2024-09-14T09:06:27.098203+0000","last_peered":"2024-09-14T09:06:27.098203+0000","last_clean":"2024-09-14T09:06:27.098203+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:06:27.098203+0000","last_undegraded":"2024-09-14T09:06:27.098203+0000","last_fullsized":"2024-09-14T09:06:27.098203+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:36:30.215848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011381609999999999,"stat_sum":{"num_bytes":79883100,"num_objects":40,"num_object_clones":13,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":3629,"num_read_kb":39980,"num_write":260,"num_write_kb":10794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"68'270","reported_seq":2277,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.564536+0000","last_change":"2024-09-14T09:05:40.038345+0000","last_active":"2024-09-14T09:06:25.564536+0000","last_peered":"2024-09-14T09:06:25.564536+0000","last_clean":"2024-09-14T09:06:25.564536+0000","last_became_active":"2024-09-14T09:05:39.923017+0000","last_became_peered":"2024-09-14T09:05:39.923017+0000","last_unstale":"2024-09-14T09:06:25.564536+0000","last_undegraded":"2024-09-14T09:06:25.564536+0000","last_fullsized":"2024-09-14T09:06:25.564536+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:24:24.766507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092949299999999999,"stat_sum":{"num_bytes":3898608,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":13,"num_read":3081,"num_read_kb":3037,"num_write":380,"num_write_kb":599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"68'252","reported_seq":2410,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T09:06:26.222325+0000","last_change":"2024-09-14T09:05:28.839774+0000","last_active":"2024-09-14T09:06:26.222325+0000","last_peered":"2024-09-14T09:06:26.222325+0000","last_clean":"2024-09-14T09:06:26.222325+0000","last_became_active":"2024-09-14T09:05:26.826600+0000","last_became_peered":"2024-09-14T09:05:26.826600+0000","last_unstale":"2024-09-14T09:06:26.222325+0000","last_undegraded":"2024-09-14T09:06:26.222325+0000","last_fullsized":"2024-09-14T09:06:26.222325+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:20:40.526770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018357099999999999,"stat_sum":{"num_bytes":39996963,"num_objects":38,"num_object_clones":13,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":17,"num_read":3149,"num_read_kb":2956,"num_write":284,"num_write_kb":1009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"68'259","reported_seq":4316,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T09:06:25.861457+0000","last_change":"2024-09-14T09:05:39.938958+0000","last_active":"2024-09-14T09:06:25.861457+0000","last_peered":"2024-09-14T09:06:25.861457+0000","last_clean":"2024-09-14T09:06:25.861457+0000","last_became_active":"2024-09-14T09:05:39.921023+0000","last_became_peered":"2024-09-14T09:05:39.921023+0000","last_unstale":"2024-09-14T09:06:25.861457+0000","last_undegraded":"2024-09-14T09:06:25.861457+0000","last_fullsized":"2024-09-14T09:06:25.861457+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:28:35.420086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061978900000000004,"stat_sum":{"num_bytes":20485399,"num_objects":35,"num_object_clones":11,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":5443,"num_read_kb":26249,"num_write":235,"num_write_kb":6975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":120888,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":916960658,"num_objects":1148,"num_object_clones":367,"num_object_copies":2296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1148,"num_whiteouts":440,"num_read":93532,"num_read_kb":475108,"num_write":10905,"num_write_kb":188361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":11394597,"num_keys_recovered":32,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28475392,"data_stored":241507941,"data_compressed":7466819,"data_compressed_allocated":19898368,"data_compressed_original":236160264,"omap_allocated":0,"internal_metadata":0},"log_size":8656,"ondisk_log_size":8656,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12424,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39804,"kb_used_data":4380,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332036,"statfs":{"total":96636764160,"available":96596004864,"internally_reserved":0,"allocated":4485120,"data_stored":31910236,"data_compressed":1101724,"data_compressed_allocated":2695168,"data_compressed_original":30949272,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43956,"kb_used_data":4448,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327884,"statfs":{"total":96636764160,"available":96591753216,"internally_reserved":0,"allocated":4554752,"data_stored":33898591,"data_compressed":886167,"data_compressed_allocated":2678784,"data_compressed_original":32965291,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38156,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333684,"statfs":{"total":96636764160,"available":96597692416,"internally_reserved":0,"allocated":2875392,"data_stored":16940703,"data_compressed":537593,"data_compressed_allocated":1507328,"data_compressed_original":16272739,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509462,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39480,"kb_used_data":4128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332360,"statfs":{"total":96636764160,"available":96596336640,"internally_reserved":0,"allocated":4227072,"data_stored":29148822,"data_compressed":943881,"data_compressed_allocated":2301952,"data_compressed_original":28113996,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44112,"kb_used_data":4604,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327728,"statfs":{"total":96636764160,"available":96591593472,"internally_reserved":0,"allocated":4714496,"data_stored":33396049,"data_compressed":842211,"data_compressed_allocated":2650112,"data_compressed_original":32171903,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1658880,"data_stored":7801871,"data_compressed":280858,"data_compressed_allocated":663552,"data_compressed_original":7319315,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45056,"kb_used_data":5548,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326784,"statfs":{"total":96636764160,"available":96590626816,"internally_reserved":0,"allocated":5681152,"data_stored":45576305,"data_compressed":1239243,"data_compressed_allocated":3354624,"data_compressed_original":44238618,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45248,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326592,"statfs":{"total":96636764160,"available":96590430208,"internally_reserved":0,"allocated":5767168,"data_stored":40022861,"data_compressed":1466577,"data_compressed_allocated":3526656,"data_compressed_original":38783050,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3870720,"data_stored":31721352,"data_compressed":1031921,"data_compressed_allocated":2564096,"data_compressed_original":30874700,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4939776,"data_stored":39647026,"data_compressed":1477480,"data_compressed_allocated":3534848,"data_compressed_original":38795338,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2076672,"data_stored":16583010,"data_compressed":537593,"data_compressed_allocated":1507328,"data_compressed_original":16272739,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4902912,"data_stored":45254819,"data_compressed":1253186,"data_compressed_allocated":3383296,"data_compressed_original":44287770,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":888832,"data_stored":7469291,"data_compressed":291677,"data_compressed_allocated":675840,"data_compressed_original":7335699,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4034560,"data_stored":35389239,"data_compressed":925123,"data_compressed_allocated":2826240,"data_compressed_original":34693803,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3874816,"data_stored":32988026,"data_compressed":1121893,"data_compressed_allocated":2797568,"data_compressed_original":32305048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3887104,"data_stored":32455178,"data_compressed":827946,"data_compressed_allocated":2609152,"data_compressed_original":31595167,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:06:30.008 INFO:tasks.thrashosds.thrasher:Setting 3.4 to [1, 7] 2024-09-14T09:06:30.008 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.4', '1', '7'] 2024-09-14T09:06:30.008 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.4 1 7 2024-09-14T09:06:30.047 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:30.061 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:30.063+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:30.065 INFO:teuthology.orchestra.run.smithi176.stdout:3131 trunc from 0xd7c613d to 0x1640790 2024-09-14T09:06:30.091 INFO:teuthology.orchestra.run.smithi176.stdout:3132 write 0x4f6bdc5 thru 0x4f77b05 (0xbd41 bytes) 2024-09-14T09:06:30.094 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:30.094 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:30.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:30.095 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:30.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.095 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49333 3.1d 3:be596f9e:::rbd_data.115e51d92ed2.0000000000000007:12 [read 950272~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:30.095 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:30.095 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.551371456999998, 2024-09-14T09:06:30.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072172603000000002, 2024-09-14T09:06:30.095 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.095 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.095 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:30.095 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.095 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.096 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.096 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49333 2024-09-14T09:06:30.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.096 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:30.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:30.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:30.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6000000000000003e-07 2024-09-14T09:06:30.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546216+0000", 2024-09-14T09:06:30.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5339999999999998e-06 2024-09-14T09:06:30.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546224+0000", 2024-09-14T09:06:30.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7770000000000002e-06 2024-09-14T09:06:30.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546230+0000", 2024-09-14T09:06:30.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3400000000000003e-06 2024-09-14T09:06:30.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.617881+0000", 2024-09-14T09:06:30.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071650488999999998 2024-09-14T09:06:30.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.617958+0000", 2024-09-14T09:06:30.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7018e-05 2024-09-14T09:06:30.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618385+0000", 2024-09-14T09:06:30.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00042768500000000002 2024-09-14T09:06:30.100 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.100 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.100 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.100 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49334 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:30.100 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:30.100 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.551344575000002, 2024-09-14T09:06:30.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072227886000000005, 2024-09-14T09:06:30.101 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.101 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.101 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:30.101 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.101 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.101 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.101 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49334 2024-09-14T09:06:30.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.101 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:30.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:30.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:30.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8899999999999999e-07 2024-09-14T09:06:30.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546243+0000", 2024-09-14T09:06:30.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.503e-06 2024-09-14T09:06:30.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546253+0000", 2024-09-14T09:06:30.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0210999999999999e-05 2024-09-14T09:06:30.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546267+0000", 2024-09-14T09:06:30.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4183e-05 2024-09-14T09:06:30.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618449+0000", 2024-09-14T09:06:30.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072181828000000003 2024-09-14T09:06:30.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618467+0000", 2024-09-14T09:06:30.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8471999999999999e-05 2024-09-14T09:06:30.105 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.105 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.105 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.105 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49335 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:30.105 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:30.105 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.551308423999998, 2024-09-14T09:06:30.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072255640999999995, 2024-09-14T09:06:30.105 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.106 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.106 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:30.106 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.106 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.106 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.106 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49335 2024-09-14T09:06:30.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.106 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:30.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:30.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:30.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9999999999999997e-07 2024-09-14T09:06:30.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546279+0000", 2024-09-14T09:06:30.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.695e-06 2024-09-14T09:06:30.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546288+0000", 2024-09-14T09:06:30.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7110000000000001e-06 2024-09-14T09:06:30.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546294+0000", 2024-09-14T09:06:30.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9830000000000001e-06 2024-09-14T09:06:30.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618521+0000", 2024-09-14T09:06:30.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072226948999999999 2024-09-14T09:06:30.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618531+0000", 2024-09-14T09:06:30.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0603e-05 2024-09-14T09:06:30.109 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.109 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.109 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.110 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49336 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:30.110 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:30.110 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.551282521000001, 2024-09-14T09:06:30.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072292223000000003, 2024-09-14T09:06:30.110 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.110 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.110 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:30.110 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.110 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.110 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.110 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49336 2024-09-14T09:06:30.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.111 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:30.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:30.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:30.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4000000000000001e-07 2024-09-14T09:06:30.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546305+0000", 2024-09-14T09:06:30.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3539999999999998e-06 2024-09-14T09:06:30.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546315+0000", 2024-09-14T09:06:30.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0431999999999999e-05 2024-09-14T09:06:30.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546321+0000", 2024-09-14T09:06:30.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7620000000000001e-06 2024-09-14T09:06:30.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618583+0000", 2024-09-14T09:06:30.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072262046999999996 2024-09-14T09:06:30.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618594+0000", 2024-09-14T09:06:30.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0988000000000001e-05 2024-09-14T09:06:30.114 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.114 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.114 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.114 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49337 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:30.114 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:30.115 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.551234907000001, 2024-09-14T09:06:30.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072294774000000006, 2024-09-14T09:06:30.115 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.115 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.115 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:30.115 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.115 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.115 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.115 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49337 2024-09-14T09:06:30.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.115 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:30.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:30.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546350+0000", 2024-09-14T09:06:30.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4600000000000004e-07 2024-09-14T09:06:30.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546381+0000", 2024-09-14T09:06:30.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0902000000000003e-05 2024-09-14T09:06:30.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546412+0000", 2024-09-14T09:06:30.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0825999999999999e-05 2024-09-14T09:06:30.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546438+0000", 2024-09-14T09:06:30.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6410999999999998e-05 2024-09-14T09:06:30.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618637+0000", 2024-09-14T09:06:30.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072198907000000007 2024-09-14T09:06:30.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618644+0000", 2024-09-14T09:06:30.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9820000000000002e-06 2024-09-14T09:06:30.119 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.119 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.119 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.119 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49338 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:30.119 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:30.119 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.551120879999999, 2024-09-14T09:06:30.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072233250999999998, 2024-09-14T09:06:30.120 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.120 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.120 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:30.120 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.120 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.120 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.120 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49338 2024-09-14T09:06:30.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.120 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:30.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:30.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546465+0000", 2024-09-14T09:06:30.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.455e-06 2024-09-14T09:06:30.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546481+0000", 2024-09-14T09:06:30.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5834000000000001e-05 2024-09-14T09:06:30.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546498+0000", 2024-09-14T09:06:30.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6937000000000001e-05 2024-09-14T09:06:30.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546508+0000", 2024-09-14T09:06:30.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0446e-05 2024-09-14T09:06:30.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618685+0000", 2024-09-14T09:06:30.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072177280999999996 2024-09-14T09:06:30.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618697+0000", 2024-09-14T09:06:30.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1297999999999999e-05 2024-09-14T09:06:30.124 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.124 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.124 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.124 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49339 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:30.124 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546535+0000", 2024-09-14T09:06:30.124 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.551048862999998, 2024-09-14T09:06:30.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072197266999999996, 2024-09-14T09:06:30.124 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.124 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.124 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:30.124 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.125 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.125 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.125 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49339 2024-09-14T09:06:30.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.125 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546535+0000", 2024-09-14T09:06:30.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546535+0000", 2024-09-14T09:06:30.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546536+0000", 2024-09-14T09:06:30.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1399999999999996e-07 2024-09-14T09:06:30.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546539+0000", 2024-09-14T09:06:30.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.089e-06 2024-09-14T09:06:30.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546560+0000", 2024-09-14T09:06:30.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1217e-05 2024-09-14T09:06:30.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546567+0000", 2024-09-14T09:06:30.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9e-06 2024-09-14T09:06:30.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618726+0000", 2024-09-14T09:06:30.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.07215887 2024-09-14T09:06:30.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618733+0000", 2024-09-14T09:06:30.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3770000000000001e-06 2024-09-14T09:06:30.128 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.128 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.128 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.128 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49340 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:30.128 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546578+0000", 2024-09-14T09:06:30.128 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.551005971999999, 2024-09-14T09:06:30.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072205187000000004, 2024-09-14T09:06:30.128 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.128 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.128 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:30.128 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.129 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.129 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.129 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49340 2024-09-14T09:06:30.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.129 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546578+0000", 2024-09-14T09:06:30.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546578+0000", 2024-09-14T09:06:30.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546579+0000", 2024-09-14T09:06:30.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1999999999999999e-07 2024-09-14T09:06:30.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546582+0000", 2024-09-14T09:06:30.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4460000000000001e-06 2024-09-14T09:06:30.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546603+0000", 2024-09-14T09:06:30.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0502000000000001e-05 2024-09-14T09:06:30.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546610+0000", 2024-09-14T09:06:30.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7150000000000001e-06 2024-09-14T09:06:30.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618770+0000", 2024-09-14T09:06:30.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072160600000000005 2024-09-14T09:06:30.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618783+0000", 2024-09-14T09:06:30.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:06:30.131 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.132 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.132 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.132 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49341 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:30.132 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546667+0000", 2024-09-14T09:06:30.132 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.550916864000001, 2024-09-14T09:06:30.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072181163000000007, 2024-09-14T09:06:30.132 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.132 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.132 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:30.132 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.132 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.132 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.133 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49341 2024-09-14T09:06:30.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.133 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546667+0000", 2024-09-14T09:06:30.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546667+0000", 2024-09-14T09:06:30.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546669+0000", 2024-09-14T09:06:30.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.486e-06 2024-09-14T09:06:30.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546674+0000", 2024-09-14T09:06:30.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5110000000000003e-06 2024-09-14T09:06:30.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546691+0000", 2024-09-14T09:06:30.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7051e-05 2024-09-14T09:06:30.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546703+0000", 2024-09-14T09:06:30.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1155000000000001e-05 2024-09-14T09:06:30.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618836+0000", 2024-09-14T09:06:30.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072133616999999997 2024-09-14T09:06:30.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618848+0000", 2024-09-14T09:06:30.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2343e-05 2024-09-14T09:06:30.136 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.136 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.136 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.136 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49386 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:30.136 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546806+0000", 2024-09-14T09:06:30.136 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.550777705000002, 2024-09-14T09:06:30.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072019206000000002, 2024-09-14T09:06:30.136 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.136 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.136 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:30.137 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.137 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.137 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.137 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49386 2024-09-14T09:06:30.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.137 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546806+0000", 2024-09-14T09:06:30.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546806+0000", 2024-09-14T09:06:30.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546808+0000", 2024-09-14T09:06:30.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.471e-06 2024-09-14T09:06:30.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546813+0000", 2024-09-14T09:06:30.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1800000000000004e-06 2024-09-14T09:06:30.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546830+0000", 2024-09-14T09:06:30.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.658e-05 2024-09-14T09:06:30.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546842+0000", 2024-09-14T09:06:30.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2442999999999999e-05 2024-09-14T09:06:30.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618816+0000", 2024-09-14T09:06:30.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071973920999999996 2024-09-14T09:06:30.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618826+0000", 2024-09-14T09:06:30.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.611e-06 2024-09-14T09:06:30.140 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.140 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.140 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.140 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49387 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:30.140 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546861+0000", 2024-09-14T09:06:30.140 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.550723059999999, 2024-09-14T09:06:30.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071996665000000001, 2024-09-14T09:06:30.141 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.141 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.141 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:30.141 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.141 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.141 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.141 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49387 2024-09-14T09:06:30.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.141 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546861+0000", 2024-09-14T09:06:30.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546861+0000", 2024-09-14T09:06:30.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546862+0000", 2024-09-14T09:06:30.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.186e-06 2024-09-14T09:06:30.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546867+0000", 2024-09-14T09:06:30.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9049999999999996e-06 2024-09-14T09:06:30.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546880+0000", 2024-09-14T09:06:30.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2815e-05 2024-09-14T09:06:30.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546889+0000", 2024-09-14T09:06:30.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5420000000000005e-06 2024-09-14T09:06:30.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618850+0000", 2024-09-14T09:06:30.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071961065000000005 2024-09-14T09:06:30.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618858+0000", 2024-09-14T09:06:30.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1519999999999992e-06 2024-09-14T09:06:30.144 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.144 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.144 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.144 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49388 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:30.145 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:30.145 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.550685375, 2024-09-14T09:06:30.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072001645000000003, 2024-09-14T09:06:30.145 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.145 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.145 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:30.145 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.145 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.145 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.145 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49388 2024-09-14T09:06:30.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.145 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:30.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:30.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:30.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:06:30.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546902+0000", 2024-09-14T09:06:30.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.734e-06 2024-09-14T09:06:30.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546910+0000", 2024-09-14T09:06:30.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9629999999999998e-06 2024-09-14T09:06:30.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546916+0000", 2024-09-14T09:06:30.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.28e-06 2024-09-14T09:06:30.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618889+0000", 2024-09-14T09:06:30.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071972555999999993 2024-09-14T09:06:30.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618900+0000", 2024-09-14T09:06:30.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1413999999999999e-05 2024-09-14T09:06:30.148 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.148 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.149 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.149 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49389 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:30.149 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546925+0000", 2024-09-14T09:06:30.149 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.550659215, 2024-09-14T09:06:30.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072026707999999995, 2024-09-14T09:06:30.149 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.149 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.149 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:30.149 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.149 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.149 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.150 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49389 2024-09-14T09:06:30.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.150 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546925+0000", 2024-09-14T09:06:30.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546925+0000", 2024-09-14T09:06:30.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546926+0000", 2024-09-14T09:06:30.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8599999999999998e-07 2024-09-14T09:06:30.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546928+0000", 2024-09-14T09:06:30.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4200000000000001e-06 2024-09-14T09:06:30.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546938+0000", 2024-09-14T09:06:30.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5859999999999994e-06 2024-09-14T09:06:30.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546945+0000", 2024-09-14T09:06:30.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1770000000000001e-06 2024-09-14T09:06:30.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618940+0000", 2024-09-14T09:06:30.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071995491999999994 2024-09-14T09:06:30.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618952+0000", 2024-09-14T09:06:30.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1347e-05 2024-09-14T09:06:30.152 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.153 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.153 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.153 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49390 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:30.153 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547051+0000", 2024-09-14T09:06:30.153 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.550533625, 2024-09-14T09:06:30.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071963947, 2024-09-14T09:06:30.153 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.153 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.153 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:30.153 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.153 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.154 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.154 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49390 2024-09-14T09:06:30.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.154 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547051+0000", 2024-09-14T09:06:30.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547051+0000", 2024-09-14T09:06:30.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547053+0000", 2024-09-14T09:06:30.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.007e-06 2024-09-14T09:06:30.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547064+0000", 2024-09-14T09:06:30.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1146e-05 2024-09-14T09:06:30.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547089+0000", 2024-09-14T09:06:30.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.563e-05 2024-09-14T09:06:30.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547127+0000", 2024-09-14T09:06:30.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8040000000000002e-05 2024-09-14T09:06:30.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619003+0000", 2024-09-14T09:06:30.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071875724000000002 2024-09-14T09:06:30.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619014+0000", 2024-09-14T09:06:30.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1399999999999999e-05 2024-09-14T09:06:30.157 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.157 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.157 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.157 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49391 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:30.157 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547162+0000", 2024-09-14T09:06:30.157 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.550422462, 2024-09-14T09:06:30.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071911484999999997, 2024-09-14T09:06:30.157 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.157 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.158 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:30.158 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.158 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.158 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.158 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49391 2024-09-14T09:06:30.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.158 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547162+0000", 2024-09-14T09:06:30.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547162+0000", 2024-09-14T09:06:30.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547163+0000", 2024-09-14T09:06:30.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.73e-06 2024-09-14T09:06:30.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547170+0000", 2024-09-14T09:06:30.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0400000000000004e-06 2024-09-14T09:06:30.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547190+0000", 2024-09-14T09:06:30.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9423e-05 2024-09-14T09:06:30.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547204+0000", 2024-09-14T09:06:30.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4175999999999999e-05 2024-09-14T09:06:30.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619060+0000", 2024-09-14T09:06:30.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071856347000000001 2024-09-14T09:06:30.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619073+0000", 2024-09-14T09:06:30.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2768999999999999e-05 2024-09-14T09:06:30.161 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.161 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.161 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.162 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49393 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:30.162 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547263+0000", 2024-09-14T09:06:30.162 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.550320840000001, 2024-09-14T09:06:30.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071900037999999999, 2024-09-14T09:06:30.162 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.162 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.162 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:30.162 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.162 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.162 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.162 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49393 2024-09-14T09:06:30.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.162 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547263+0000", 2024-09-14T09:06:30.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547263+0000", 2024-09-14T09:06:30.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547264+0000", 2024-09-14T09:06:30.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3399999999999998e-07 2024-09-14T09:06:30.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547299+0000", 2024-09-14T09:06:30.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5252000000000001e-05 2024-09-14T09:06:30.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547317+0000", 2024-09-14T09:06:30.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.736e-05 2024-09-14T09:06:30.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547332+0000", 2024-09-14T09:06:30.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.487e-05 2024-09-14T09:06:30.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619152+0000", 2024-09-14T09:06:30.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071820590000000004 2024-09-14T09:06:30.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619163+0000", 2024-09-14T09:06:30.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1131999999999999e-05 2024-09-14T09:06:30.166 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.166 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.166 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.166 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:50478 3.1d 3:b8a2eaf4:::rbd_data.115e5dc498ac.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T09:06:30.166 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:30.166 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 18.047944354999998, 2024-09-14T09:06:30.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.075023792000000006, 2024-09-14T09:06:30.166 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.166 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.166 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:30.166 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.167 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.167 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.167 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 50478 2024-09-14T09:06:30.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.167 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:30.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:30.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049641+0000", 2024-09-14T09:06:30.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6270000000000001e-06 2024-09-14T09:06:30.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049649+0000", 2024-09-14T09:06:30.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6279999999999996e-06 2024-09-14T09:06:30.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049667+0000", 2024-09-14T09:06:30.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-09-14T09:06:30.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049677+0000", 2024-09-14T09:06:30.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0509000000000001e-05 2024-09-14T09:06:30.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049751+0000", 2024-09-14T09:06:30.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3322e-05 2024-09-14T09:06:30.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049830+0000", 2024-09-14T09:06:30.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9932000000000007e-05 2024-09-14T09:06:30.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:30.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.053203+0000", 2024-09-14T09:06:30.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003372398 2024-09-14T09:06:30.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:30.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124542+0000", 2024-09-14T09:06:30.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071339105999999999 2024-09-14T09:06:30.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:30.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124562+0000", 2024-09-14T09:06:30.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9649000000000001e-05 2024-09-14T09:06:30.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:30.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124616+0000", 2024-09-14T09:06:30.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4642999999999998e-05 2024-09-14T09:06:30.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124664+0000", 2024-09-14T09:06:30.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7342000000000002e-05 2024-09-14T09:06:30.172 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.172 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.173 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.173 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65035 3.1d 3:bb585d47:::rbd_data.115e21c31fcf.0000000000000034:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:06:30.173 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.068842+0000", 2024-09-14T09:06:30.173 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.0287421210000005, 2024-09-14T09:06:30.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.094968451999999995, 2024-09-14T09:06:30.173 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.173 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.173 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:30.173 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.173 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.173 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.174 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65035 2024-09-14T09:06:30.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.174 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068842+0000", 2024-09-14T09:06:30.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068842+0000", 2024-09-14T09:06:30.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068843+0000", 2024-09-14T09:06:30.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.221e-06 2024-09-14T09:06:30.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068849+0000", 2024-09-14T09:06:30.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4659999999999998e-06 2024-09-14T09:06:30.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068861+0000", 2024-09-14T09:06:30.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2204e-05 2024-09-14T09:06:30.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068871+0000", 2024-09-14T09:06:30.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0318e-05 2024-09-14T09:06:30.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068942+0000", 2024-09-14T09:06:30.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0438999999999994e-05 2024-09-14T09:06:30.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.069005+0000", 2024-09-14T09:06:30.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3076000000000004e-05 2024-09-14T09:06:30.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:30.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.074957+0000", 2024-09-14T09:06:30.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059524549999999997 2024-09-14T09:06:30.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:30.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163479+0000", 2024-09-14T09:06:30.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.088521676999999993 2024-09-14T09:06:30.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:30.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163633+0000", 2024-09-14T09:06:30.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000154301 2024-09-14T09:06:30.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:30.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163733+0000", 2024-09-14T09:06:30.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100089 2024-09-14T09:06:30.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163811+0000", 2024-09-14T09:06:30.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7206000000000006e-05 2024-09-14T09:06:30.178 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.179 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.179 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.179 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:65035 3.1d e65/23)", 2024-09-14T09:06:30.179 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.087665+0000", 2024-09-14T09:06:30.179 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.0099192109999997, 2024-09-14T09:06:30.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.076160555000000005, 2024-09-14T09:06:30.179 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.179 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.179 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:30.179 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087665+0000", 2024-09-14T09:06:30.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087665+0000", 2024-09-14T09:06:30.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087667+0000", 2024-09-14T09:06:30.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.818e-06 2024-09-14T09:06:30.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087673+0000", 2024-09-14T09:06:30.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9259999999999997e-06 2024-09-14T09:06:30.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087692+0000", 2024-09-14T09:06:30.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9715e-05 2024-09-14T09:06:30.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087703+0000", 2024-09-14T09:06:30.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.094e-05 2024-09-14T09:06:30.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163595+0000", 2024-09-14T09:06:30.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.075892121000000007 2024-09-14T09:06:30.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163621+0000", 2024-09-14T09:06:30.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5383000000000001e-05 2024-09-14T09:06:30.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163826+0000", 2024-09-14T09:06:30.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020465199999999999 2024-09-14T09:06:30.182 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.183 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.183 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.183 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:71031 3.1f 3:f9e41410:::rbd_data.115ebd4a9f36.0000000000000003:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:06:30.183 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:29.576817+0000", 2024-09-14T09:06:30.183 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.52076673399999995, 2024-09-14T09:06:30.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.087207509000000002, 2024-09-14T09:06:30.183 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.183 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.183 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:30.183 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.183 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.183 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.184 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 71031 2024-09-14T09:06:30.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.184 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.576817+0000", 2024-09-14T09:06:30.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.576817+0000", 2024-09-14T09:06:30.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.576820+0000", 2024-09-14T09:06:30.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.283e-06 2024-09-14T09:06:30.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.576832+0000", 2024-09-14T09:06:30.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1843e-05 2024-09-14T09:06:30.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.576861+0000", 2024-09-14T09:06:30.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9043e-05 2024-09-14T09:06:30.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.576876+0000", 2024-09-14T09:06:30.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5293000000000001e-05 2024-09-14T09:06:30.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.577000+0000", 2024-09-14T09:06:30.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000124473 2024-09-14T09:06:30.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.577126+0000", 2024-09-14T09:06:30.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012597199999999999 2024-09-14T09:06:30.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-09-14T09:06:30.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.585227+0000", 2024-09-14T09:06:30.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0081002810000000008 2024-09-14T09:06:30.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:30.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.663671+0000", 2024-09-14T09:06:30.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.078444609999999998 2024-09-14T09:06:30.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:30.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.663867+0000", 2024-09-14T09:06:30.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019541099999999999 2024-09-14T09:06:30.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:30.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.663960+0000", 2024-09-14T09:06:30.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3191999999999998e-05 2024-09-14T09:06:30.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.664025+0000", 2024-09-14T09:06:30.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5108000000000001e-05 2024-09-14T09:06:30.188 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.188 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.188 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.188 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.189 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.189 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:30.191 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:06:30.191 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:30.191+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:30.192 INFO:teuthology.orchestra.run.smithi176.stdout:3135 read 0x180df24 thru 0x1818060 (0xa13d bytes) 2024-09-14T09:06:30.192 INFO:teuthology.orchestra.run.smithi176.stdout:3137 write 0xd1f99e thru 0xd27832 (0x7e95 bytes) 2024-09-14T09:06:30.192 INFO:teuthology.orchestra.run.smithi176.stdout:3139 trunc from 0x4f77b06 to 0xa6ad119 2024-09-14T09:06:30.192 INFO:teuthology.orchestra.run.smithi176.stdout:3140 write 0xb7152cd thru 0xb715b97 (0x8cb bytes) 2024-09-14T09:06:30.192 INFO:teuthology.orchestra.run.smithi176.stdout:3141 punch from 0x6656392 to 0x665e311, (0x7f7f bytes) 2024-09-14T09:06:30.193 INFO:teuthology.orchestra.run.smithi176.stdout:3142 read 0x34db632 thru 0x34dd7b2 (0x2181 bytes) 2024-09-14T09:06:30.193 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:30.193 INFO:teuthology.orchestra.run.smithi176.stdout:3145 trunc from 0xb715b98 to 0x12c573c 2024-09-14T09:06:30.193 INFO:teuthology.orchestra.run.smithi176.stdout:3146 writesame 0x97a97fd thru 0x97b13a9 data_size 0x1(0x7bad bytes) 2024-09-14T09:06:30.193 INFO:teuthology.orchestra.run.smithi176.stdout:3147 write 0x3c5bbc4 thru 0x3c5d8b4 (0x1cf1 bytes) 2024-09-14T09:06:30.193 INFO:teuthology.orchestra.run.smithi176.stdout:3148 read 0x1a60ce5 thru 0x1a68bc1 (0x7edd bytes) 2024-09-14T09:06:30.193 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:30.193 INFO:teuthology.orchestra.run.smithi176.stdout:3150 trunc from 0x97b13aa to 0x846f9a3 2024-09-14T09:06:30.193 INFO:teuthology.orchestra.run.smithi176.stdout:3151 punch from 0x5be1bd5 to 0x5bee39b, (0xc7c6 bytes) 2024-09-14T09:06:30.193 INFO:teuthology.orchestra.run.smithi176.stdout:3153 trunc from 0x846f9a3 to 0x973744a 2024-09-14T09:06:30.193 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:30.193 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:30.193 INFO:teuthology.orchestra.run.smithi176.stdout:3159 write 0x7a7d84a thru 0x7a88126 (0xa8dd bytes) 2024-09-14T09:06:30.202 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:30.202 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:30.202 INFO:teuthology.orchestra.run.smithi176.stdout:3163 read 0x5b65c3b thru 0x5b6c069 (0x642f bytes) 2024-09-14T09:06:30.202 INFO:teuthology.orchestra.run.smithi176.stdout:3164 read 0xfd09a5 thru 0xfd264d (0x1ca9 bytes) 2024-09-14T09:06:30.210 INFO:teuthology.orchestra.run.smithi176.stdout:3165 writesame 0xa3ad50e thru 0xa3bbbee data_size 0x1(0xe6e1 bytes) 2024-09-14T09:06:30.210 INFO:teuthology.orchestra.run.smithi176.stdout:3166 read 0x1e6db9a thru 0x1e7166c (0x3ad3 bytes) 2024-09-14T09:06:30.210 INFO:teuthology.orchestra.run.smithi176.stdout:3168 write 0xd66f16 thru 0xd67d01 (0xdec bytes) 2024-09-14T09:06:30.211 INFO:teuthology.orchestra.run.smithi176.stdout:3169 write 0x485173c thru 0x485659a (0x4e5f bytes) 2024-09-14T09:06:30.216 INFO:teuthology.orchestra.run.smithi176.stdout:3170 punch from 0x66ead96 to 0x66f0a3c, (0x5ca6 bytes) 2024-09-14T09:06:30.220 INFO:teuthology.orchestra.run.smithi176.stdout:3172 trunc from 0xa3bbbef to 0xcba525f 2024-09-14T09:06:30.240 INFO:teuthology.orchestra.run.smithi176.stdout:3175 trunc from 0xcba525f to 0x3de4a0a 2024-09-14T09:06:30.257 INFO:teuthology.orchestra.run.smithi176.stdout:3176 read 0x35e37c6 thru 0x35e65be (0x2df9 bytes) 2024-09-14T09:06:30.257 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:30.257 INFO:teuthology.orchestra.run.smithi176.stdout:3178 read 0xfd7259 thru 0xfe614b (0xeef3 bytes) 2024-09-14T09:06:30.258 INFO:teuthology.orchestra.run.smithi176.stdout:3179 read 0x2693d9e thru 0x269719b (0x33fe bytes) 2024-09-14T09:06:30.278 INFO:teuthology.orchestra.run.smithi176.stdout:3180 writesame 0xc999c06 thru 0xc9a5550 data_size 0x35(0xb94b bytes) 2024-09-14T09:06:30.278 INFO:teuthology.orchestra.run.smithi176.stdout:3181 read 0x666fe32 thru 0x667ccde (0xcead bytes) 2024-09-14T09:06:30.278 INFO:teuthology.orchestra.run.smithi176.stdout:3182 write 0xabbb418 thru 0xabc38e8 (0x84d1 bytes) 2024-09-14T09:06:30.290 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:30.291+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:30.292 INFO:teuthology.orchestra.run.smithi176.stdout:3183 write 0xdbf362c thru 0xdbf8959 (0x532e bytes) 2024-09-14T09:06:30.297 INFO:teuthology.orchestra.run.smithi176.stdout:3185 punch from 0x16f8cee to 0x16fe2f2, (0x5604 bytes) 2024-09-14T09:06:30.297 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:30.297 INFO:teuthology.orchestra.run.smithi176.stdout:3187 read 0xa7ea0ba thru 0xa7f015a (0x60a1 bytes) 2024-09-14T09:06:30.297 INFO:teuthology.orchestra.run.smithi176.stdout:3189 writesame 0x8fdc25 thru 0x906db4 data_size 0x88(0x9190 bytes) 2024-09-14T09:06:30.300 INFO:teuthology.orchestra.run.smithi176.stdout:3190 read 0x151afba thru 0x15225d9 (0x7620 bytes) 2024-09-14T09:06:30.301 INFO:teuthology.orchestra.run.smithi176.stdout:3191 writesame 0x17a61d thru 0x185367 data_size 0x25(0xad4b bytes) 2024-09-14T09:06:30.305 INFO:teuthology.orchestra.run.smithi176.stdout:3192 write 0x16061ce thru 0x160abc7 (0x49fa bytes) 2024-09-14T09:06:30.307 INFO:teuthology.orchestra.run.smithi176.stdout:3193 punch from 0x335c726 to 0x336ab16, (0xe3f0 bytes) 2024-09-14T09:06:30.308 INFO:teuthology.orchestra.run.smithi176.stdout:3194 flatten 2024-09-14T09:06:30.314 INFO:teuthology.orchestra.run.smithi176.stdout:3195 read 0xb12db69 thru 0xb133ee3 (0x637b bytes) 2024-09-14T09:06:30.315 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:30.315 INFO:teuthology.orchestra.run.smithi176.stdout:3198 punch from 0xb1191c8 to 0xb119877, (0x6af bytes) 2024-09-14T09:06:30.315 INFO:teuthology.orchestra.run.smithi176.stdout:3199 read 0x3a3af18 thru 0x3a3b04f (0x138 bytes) 2024-09-14T09:06:30.315 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:30.315 INFO:teuthology.orchestra.run.smithi176.stdout:3201 punch from 0x684fb86 to 0x6858fcf, (0x9449 bytes) 2024-09-14T09:06:30.315 INFO:teuthology.orchestra.run.smithi176.stdout:3202 write 0x14714cb thru 0x147289d (0x13d3 bytes) 2024-09-14T09:06:30.317 INFO:teuthology.orchestra.run.smithi176.stdout:3203 punch from 0x9358f5b to 0x9368156, (0xf1fb bytes) 2024-09-14T09:06:30.328 INFO:teuthology.orchestra.run.smithi176.stdout:3204 trunc from 0xdbf895a to 0x6a2e98c 2024-09-14T09:06:30.342 INFO:teuthology.orchestra.run.smithi176.stdout:3208 write 0x3124153 thru 0x312b9a9 (0x7857 bytes) 2024-09-14T09:06:30.349 INFO:teuthology.orchestra.run.smithi176.stdout:3209 punch from 0x4505298 to 0x4513115, (0xde7d bytes) 2024-09-14T09:06:30.349 INFO:teuthology.orchestra.run.smithi176.stdout:3210 read 0x21c6bf4 thru 0x21d5c74 (0xf081 bytes) 2024-09-14T09:06:30.349 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:30.352 INFO:teuthology.orchestra.run.smithi176.stdout:3212 trunc from 0x6a2e98c to 0x5180ee9 2024-09-14T09:06:30.362 INFO:teuthology.orchestra.run.smithi176.stdout:3213 read 0x20f8c44 thru 0x2101f4a (0x9307 bytes) 2024-09-14T09:06:30.362 INFO:teuthology.orchestra.run.smithi176.stdout:3214 read 0x298607c thru 0x298b461 (0x53e6 bytes) 2024-09-14T09:06:30.363 INFO:teuthology.orchestra.run.smithi176.stdout:3215 read 0x29a1196 thru 0x29aeeca (0xdd35 bytes) 2024-09-14T09:06:30.367 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:30.367 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:30.367 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:30.367 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:30.379 INFO:teuthology.orchestra.run.smithi176.stdout:3216 trunc from 0x5180ee9 to 0xd9f4e32 2024-09-14T09:06:30.381 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:06:30.400 INFO:teuthology.orchestra.run.smithi176.stdout:3217 writesame 0xe6f6d77 thru 0xe6fb600 data_size 0x6(0x488a bytes) 2024-09-14T09:06:30.400 INFO:teuthology.orchestra.run.smithi176.stdout:3218 write 0x3f30771 thru 0x3f3deed (0xd77d bytes) 2024-09-14T09:06:30.409 INFO:teuthology.orchestra.run.smithi176.stdout:3219 read 0x7fac156 thru 0x7fb1a2d (0x58d8 bytes) 2024-09-14T09:06:30.410 INFO:teuthology.orchestra.run.smithi176.stdout:3220 write 0x18b12c2 thru 0x18b4141 (0x2e80 bytes) 2024-09-14T09:06:30.413 INFO:teuthology.orchestra.run.smithi176.stdout:3221 punch from 0xb6859f2 to 0xb6887e9, (0x2df7 bytes) 2024-09-14T09:06:30.427 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:30.431+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:30.430 INFO:teuthology.orchestra.run.smithi176.stdout:3223 trunc from 0xe6fb601 to 0x34fd019 2024-09-14T09:06:30.455 INFO:teuthology.orchestra.run.smithi176.stdout:3224 write 0x5a4efea thru 0x5a5a54f (0xb566 bytes) 2024-09-14T09:06:30.481 INFO:teuthology.orchestra.run.smithi176.stdout:3226 write 0xc92c5cf thru 0xc92e5c9 (0x1ffb bytes) 2024-09-14T09:06:30.485 INFO:teuthology.orchestra.run.smithi176.stdout:3227 read 0x1f14821 thru 0x1f153d6 (0xbb6 bytes) 2024-09-14T09:06:30.492 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:30.491+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:30.504 INFO:teuthology.orchestra.run.smithi176.stdout:3229 trunc from 0xc92e5ca to 0x30124f6 2024-09-14T09:06:30.524 INFO:teuthology.orchestra.run.smithi176.stdout:3230 trunc from 0x30124f6 to 0x285f904 2024-09-14T09:06:30.538 INFO:teuthology.orchestra.run.smithi176.stdout:3231 read 0x2752859 thru 0x275f4d8 (0xcc80 bytes) 2024-09-14T09:06:30.539 INFO:teuthology.orchestra.run.smithi176.stdout:3232 punch from 0xa5edf1 to 0xa6715a, (0x8369 bytes) 2024-09-14T09:06:30.539 INFO:teuthology.orchestra.run.smithi176.stdout:3233 punch from 0x1f238dd to 0x1f25688, (0x1dab bytes) 2024-09-14T09:06:30.549 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:30.550 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:30.550 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:30.550 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:30.550 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:30.564 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:06:30.570 INFO:teuthology.orchestra.run.smithi176.stdout:3234 write 0xb5ca8dc thru 0xb5d11de (0x6903 bytes) 2024-09-14T09:06:30.576 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:30.594 INFO:teuthology.orchestra.run.smithi176.stdout:3237 write 0xe0fd20f thru 0xe0fe329 (0x111b bytes) 2024-09-14T09:06:30.600 INFO:teuthology.orchestra.run.smithi176.stdout:3238 write 0xd3d71d2 thru 0xd3e0dfd (0x9c2c bytes) 2024-09-14T09:06:30.611 INFO:teuthology.orchestra.run.smithi176.stdout:3240 trunc from 0xe0fe32a to 0xd29c689 2024-09-14T09:06:30.612 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:30.615+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:30.649 INFO:teuthology.orchestra.run.smithi176.stdout:3242 write 0xe0df6c3 thru 0xe0e4290 (0x4bce bytes) 2024-09-14T09:06:30.683 INFO:teuthology.orchestra.run.smithi176.stdout:3245 trunc from 0xe0e4291 to 0x2ffbeb1 2024-09-14T09:06:30.694 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:30.694 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:30.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:30.694 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:30.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19520 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475212+0000", 2024-09-14T09:06:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.220208732000003, 2024-09-14T09:06:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067435885000000001, 2024-09-14T09:06:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19520 2024-09-14T09:06:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475212+0000", 2024-09-14T09:06:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475212+0000", 2024-09-14T09:06:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475213+0000", 2024-09-14T09:06:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0440000000000001e-06 2024-09-14T09:06:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475218+0000", 2024-09-14T09:06:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1930000000000001e-06 2024-09-14T09:06:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475237+0000", 2024-09-14T09:06:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9003000000000001e-05 2024-09-14T09:06:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475251+0000", 2024-09-14T09:06:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3720000000000001e-05 2024-09-14T09:06:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.541342+0000", 2024-09-14T09:06:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066091227000000002 2024-09-14T09:06:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.541492+0000", 2024-09-14T09:06:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000149221 2024-09-14T09:06:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542648+0000", 2024-09-14T09:06:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011564769999999999 2024-09-14T09:06:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19530 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475326+0000", 2024-09-14T09:06:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.220094549000002, 2024-09-14T09:06:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067030846000000005, 2024-09-14T09:06:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19530 2024-09-14T09:06:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475326+0000", 2024-09-14T09:06:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475326+0000", 2024-09-14T09:06:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475328+0000", 2024-09-14T09:06:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.133e-06 2024-09-14T09:06:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475358+0000", 2024-09-14T09:06:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0253e-05 2024-09-14T09:06:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475378+0000", 2024-09-14T09:06:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.033e-05 2024-09-14T09:06:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475405+0000", 2024-09-14T09:06:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6568999999999999e-05 2024-09-14T09:06:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.541989+0000", 2024-09-14T09:06:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066583956999999999 2024-09-14T09:06:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542215+0000", 2024-09-14T09:06:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022592200000000001 2024-09-14T09:06:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542357+0000", 2024-09-14T09:06:30.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014268199999999999 2024-09-14T09:06:30.703 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.703 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.703 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.703 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19531 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15073280~11919] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:30.703 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475425+0000", 2024-09-14T09:06:30.703 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.219996254000002, 2024-09-14T09:06:30.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067651759000000006, 2024-09-14T09:06:30.703 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.703 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.704 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:30.704 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.704 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.704 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.704 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19531 2024-09-14T09:06:30.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.704 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475425+0000", 2024-09-14T09:06:30.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475425+0000", 2024-09-14T09:06:30.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475426+0000", 2024-09-14T09:06:30.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2440000000000001e-06 2024-09-14T09:06:30.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475465+0000", 2024-09-14T09:06:30.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8643000000000003e-05 2024-09-14T09:06:30.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475485+0000", 2024-09-14T09:06:30.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0645999999999999e-05 2024-09-14T09:06:30.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475499+0000", 2024-09-14T09:06:30.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3312999999999999e-05 2024-09-14T09:06:30.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542389+0000", 2024-09-14T09:06:30.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066890885999999997 2024-09-14T09:06:30.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542481+0000", 2024-09-14T09:06:30.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1481e-05 2024-09-14T09:06:30.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543076+0000", 2024-09-14T09:06:30.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00059554599999999997 2024-09-14T09:06:30.707 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.707 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.707 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.707 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19532 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15335424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:30.707 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475521+0000", 2024-09-14T09:06:30.707 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.219899607999999, 2024-09-14T09:06:30.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067895159999999996, 2024-09-14T09:06:30.707 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.707 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.708 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:30.708 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.708 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.708 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.708 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19532 2024-09-14T09:06:30.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.708 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475521+0000", 2024-09-14T09:06:30.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475521+0000", 2024-09-14T09:06:30.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475523+0000", 2024-09-14T09:06:30.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.277e-06 2024-09-14T09:06:30.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475528+0000", 2024-09-14T09:06:30.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8629999999999996e-06 2024-09-14T09:06:30.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475546+0000", 2024-09-14T09:06:30.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7750000000000001e-05 2024-09-14T09:06:30.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475560+0000", 2024-09-14T09:06:30.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3763e-05 2024-09-14T09:06:30.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543144+0000", 2024-09-14T09:06:30.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067584489999999997 2024-09-14T09:06:30.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543261+0000", 2024-09-14T09:06:30.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011696699999999999 2024-09-14T09:06:30.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543416+0000", 2024-09-14T09:06:30.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015505 2024-09-14T09:06:30.711 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.711 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.711 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.711 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19533 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:30.711 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475578+0000", 2024-09-14T09:06:30.711 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.219842464999999, 2024-09-14T09:06:30.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068124984999999999, 2024-09-14T09:06:30.711 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.712 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.712 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:30.712 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.712 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.712 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.712 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19533 2024-09-14T09:06:30.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.712 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475578+0000", 2024-09-14T09:06:30.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475578+0000", 2024-09-14T09:06:30.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475580+0000", 2024-09-14T09:06:30.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.08e-06 2024-09-14T09:06:30.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475585+0000", 2024-09-14T09:06:30.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4970000000000001e-06 2024-09-14T09:06:30.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475602+0000", 2024-09-14T09:06:30.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7459999999999999e-05 2024-09-14T09:06:30.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475615+0000", 2024-09-14T09:06:30.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2339e-05 2024-09-14T09:06:30.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543466+0000", 2024-09-14T09:06:30.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067850745000000004 2024-09-14T09:06:30.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543577+0000", 2024-09-14T09:06:30.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000111908 2024-09-14T09:06:30.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543703+0000", 2024-09-14T09:06:30.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012595600000000001 2024-09-14T09:06:30.715 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.715 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.715 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.716 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19534 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:30.716 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475634+0000", 2024-09-14T09:06:30.716 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.219786689000003, 2024-09-14T09:06:30.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068358706000000005, 2024-09-14T09:06:30.716 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.716 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.716 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:30.716 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.716 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.716 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.716 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19534 2024-09-14T09:06:30.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.716 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475634+0000", 2024-09-14T09:06:30.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475634+0000", 2024-09-14T09:06:30.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475635+0000", 2024-09-14T09:06:30.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.187e-06 2024-09-14T09:06:30.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475641+0000", 2024-09-14T09:06:30.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5629999999999999e-06 2024-09-14T09:06:30.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475659+0000", 2024-09-14T09:06:30.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7649999999999999e-05 2024-09-14T09:06:30.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475671+0000", 2024-09-14T09:06:30.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2583e-05 2024-09-14T09:06:30.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543748+0000", 2024-09-14T09:06:30.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068076740999999996 2024-09-14T09:06:30.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543855+0000", 2024-09-14T09:06:30.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010676800000000001 2024-09-14T09:06:30.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543993+0000", 2024-09-14T09:06:30.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138214 2024-09-14T09:06:30.720 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.720 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.720 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.720 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19535 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:30.720 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475691+0000", 2024-09-14T09:06:30.720 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.219729678999997, 2024-09-14T09:06:30.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068607583999999999, 2024-09-14T09:06:30.720 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.721 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.721 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:30.721 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.721 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.721 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.721 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19535 2024-09-14T09:06:30.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.721 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475691+0000", 2024-09-14T09:06:30.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475691+0000", 2024-09-14T09:06:30.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475693+0000", 2024-09-14T09:06:30.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6270000000000001e-06 2024-09-14T09:06:30.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475699+0000", 2024-09-14T09:06:30.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9059999999999999e-06 2024-09-14T09:06:30.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475717+0000", 2024-09-14T09:06:30.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-09-14T09:06:30.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475730+0000", 2024-09-14T09:06:30.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.308e-05 2024-09-14T09:06:30.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544042+0000", 2024-09-14T09:06:30.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068312791999999997 2024-09-14T09:06:30.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544168+0000", 2024-09-14T09:06:30.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012549699999999999 2024-09-14T09:06:30.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544299+0000", 2024-09-14T09:06:30.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013094199999999999 2024-09-14T09:06:30.725 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.725 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.725 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.725 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19536 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:30.726 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475749+0000", 2024-09-14T09:06:30.726 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.2196724, 2024-09-14T09:06:30.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068917143, 2024-09-14T09:06:30.726 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.726 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.726 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:30.726 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.726 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.726 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.726 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19536 2024-09-14T09:06:30.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.726 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475749+0000", 2024-09-14T09:06:30.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475749+0000", 2024-09-14T09:06:30.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475750+0000", 2024-09-14T09:06:30.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0330000000000001e-06 2024-09-14T09:06:30.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475755+0000", 2024-09-14T09:06:30.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9740000000000001e-06 2024-09-14T09:06:30.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475772+0000", 2024-09-14T09:06:30.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7750000000000001e-05 2024-09-14T09:06:30.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475802+0000", 2024-09-14T09:06:30.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0190000000000001e-05 2024-09-14T09:06:30.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.729 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.729 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544345+0000", 2024-09-14T09:06:30.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068542140000000001 2024-09-14T09:06:30.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.729 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.729 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544454+0000", 2024-09-14T09:06:30.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109447 2024-09-14T09:06:30.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.729 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.729 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544666+0000", 2024-09-14T09:06:30.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000211609 2024-09-14T09:06:30.730 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.730 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.730 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.730 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19537 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:30.730 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475822+0000", 2024-09-14T09:06:30.730 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.219598443000002, 2024-09-14T09:06:30.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069185398999999995, 2024-09-14T09:06:30.730 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.730 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.730 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:30.730 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.730 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.730 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.731 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19537 2024-09-14T09:06:30.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.731 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475822+0000", 2024-09-14T09:06:30.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475822+0000", 2024-09-14T09:06:30.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475824+0000", 2024-09-14T09:06:30.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2559999999999999e-06 2024-09-14T09:06:30.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475829+0000", 2024-09-14T09:06:30.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6840000000000001e-06 2024-09-14T09:06:30.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475848+0000", 2024-09-14T09:06:30.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8583000000000001e-05 2024-09-14T09:06:30.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475861+0000", 2024-09-14T09:06:30.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2802999999999999e-05 2024-09-14T09:06:30.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544716+0000", 2024-09-14T09:06:30.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068855036999999994 2024-09-14T09:06:30.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544827+0000", 2024-09-14T09:06:30.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011113200000000001 2024-09-14T09:06:30.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545008+0000", 2024-09-14T09:06:30.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018090399999999999 2024-09-14T09:06:30.734 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.734 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.734 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.735 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19538 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:30.735 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475880+0000", 2024-09-14T09:06:30.735 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.219540768000002, 2024-09-14T09:06:30.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069740072, 2024-09-14T09:06:30.735 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.735 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.735 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:30.735 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.735 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.735 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.735 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19538 2024-09-14T09:06:30.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.735 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475880+0000", 2024-09-14T09:06:30.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475880+0000", 2024-09-14T09:06:30.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475881+0000", 2024-09-14T09:06:30.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.124e-06 2024-09-14T09:06:30.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475886+0000", 2024-09-14T09:06:30.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0130000000000001e-06 2024-09-14T09:06:30.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475904+0000", 2024-09-14T09:06:30.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7779999999999999e-05 2024-09-14T09:06:30.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475918+0000", 2024-09-14T09:06:30.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.385e-05 2024-09-14T09:06:30.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545065+0000", 2024-09-14T09:06:30.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069147307000000005 2024-09-14T09:06:30.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545431+0000", 2024-09-14T09:06:30.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036538400000000001 2024-09-14T09:06:30.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545620+0000", 2024-09-14T09:06:30.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000189614 2024-09-14T09:06:30.739 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.739 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.739 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.739 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19539 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:30.739 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475936+0000", 2024-09-14T09:06:30.739 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.219485184, 2024-09-14T09:06:30.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069860063999999999, 2024-09-14T09:06:30.739 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.739 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.739 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:30.739 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.739 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.740 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.740 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19539 2024-09-14T09:06:30.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.740 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475936+0000", 2024-09-14T09:06:30.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475936+0000", 2024-09-14T09:06:30.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475937+0000", 2024-09-14T09:06:30.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5960000000000001e-06 2024-09-14T09:06:30.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475943+0000", 2024-09-14T09:06:30.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2340000000000002e-06 2024-09-14T09:06:30.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475960+0000", 2024-09-14T09:06:30.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7592999999999999e-05 2024-09-14T09:06:30.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475986+0000", 2024-09-14T09:06:30.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6319e-05 2024-09-14T09:06:30.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545650+0000", 2024-09-14T09:06:30.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069663613999999999 2024-09-14T09:06:30.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545714+0000", 2024-09-14T09:06:30.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4209999999999997e-05 2024-09-14T09:06:30.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545796+0000", 2024-09-14T09:06:30.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1497999999999998e-05 2024-09-14T09:06:30.743 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.743 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.743 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.743 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19541 3.10 3:0ae7d8d9:::rbd_data.115e4c71d6e8.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 4194304~8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:30.744 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.497517+0000", 2024-09-14T09:06:30.744 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.197904113, 2024-09-14T09:06:30.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069874055000000004, 2024-09-14T09:06:30.744 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.744 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.744 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:30.744 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.744 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.744 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.744 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19541 2024-09-14T09:06:30.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.744 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.497517+0000", 2024-09-14T09:06:30.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.497517+0000", 2024-09-14T09:06:30.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.497520+0000", 2024-09-14T09:06:30.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0730000000000001e-06 2024-09-14T09:06:30.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.504875+0000", 2024-09-14T09:06:30.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007355136 2024-09-14T09:06:30.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.504926+0000", 2024-09-14T09:06:30.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1146000000000001e-05 2024-09-14T09:06:30.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.504950+0000", 2024-09-14T09:06:30.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3638999999999998e-05 2024-09-14T09:06:30.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.505139+0000", 2024-09-14T09:06:30.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018904300000000001 2024-09-14T09:06:30.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.505402+0000", 2024-09-14T09:06:30.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026354699999999999 2024-09-14T09:06:30.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:30.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.506208+0000", 2024-09-14T09:06:30.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00080583399999999996 2024-09-14T09:06:30.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:30.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.566904+0000", 2024-09-14T09:06:30.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.060695510000000001 2024-09-14T09:06:30.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:30.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.567219+0000", 2024-09-14T09:06:30.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031509000000000002 2024-09-14T09:06:30.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:30.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.567319+0000", 2024-09-14T09:06:30.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9804999999999997e-05 2024-09-14T09:06:30.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.567391+0000", 2024-09-14T09:06:30.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2231999999999996e-05 2024-09-14T09:06:30.748 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.748 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.749 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.749 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:30890 3.10 3:0ac3ecfb:::rbd_data.115ebc3c67d1.0000000000000004:c [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:30.749 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.069693+0000", 2024-09-14T09:06:30.749 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.625728324000001, 2024-09-14T09:06:30.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066892069999999998, 2024-09-14T09:06:30.749 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.749 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.749 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:30.749 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.749 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.749 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.749 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 30890 2024-09-14T09:06:30.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.749 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069693+0000", 2024-09-14T09:06:30.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069693+0000", 2024-09-14T09:06:30.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069694+0000", 2024-09-14T09:06:30.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0100000000000003e-07 2024-09-14T09:06:30.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069698+0000", 2024-09-14T09:06:30.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2100000000000003e-06 2024-09-14T09:06:30.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069711+0000", 2024-09-14T09:06:30.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3332e-05 2024-09-14T09:06:30.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069721+0000", 2024-09-14T09:06:30.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0424999999999999e-05 2024-09-14T09:06:30.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136469+0000", 2024-09-14T09:06:30.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066747714999999999 2024-09-14T09:06:30.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136540+0000", 2024-09-14T09:06:30.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1077999999999999e-05 2024-09-14T09:06:30.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136585+0000", 2024-09-14T09:06:30.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4409000000000001e-05 2024-09-14T09:06:30.753 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.753 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.753 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.753 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:30891 3.10 3:0ac3ecfb:::rbd_data.115ebc3c67d1.0000000000000004:c [read 212992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:30.753 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.069740+0000", 2024-09-14T09:06:30.753 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.625681323999999, 2024-09-14T09:06:30.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066940186999999998, 2024-09-14T09:06:30.753 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.753 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.754 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:30.754 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.754 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.754 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.754 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 30891 2024-09-14T09:06:30.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.754 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069740+0000", 2024-09-14T09:06:30.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069740+0000", 2024-09-14T09:06:30.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070603+0000", 2024-09-14T09:06:30.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00086369599999999995 2024-09-14T09:06:30.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070612+0000", 2024-09-14T09:06:30.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3170000000000002e-06 2024-09-14T09:06:30.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070631+0000", 2024-09-14T09:06:30.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8936999999999998e-05 2024-09-14T09:06:30.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070645+0000", 2024-09-14T09:06:30.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4545999999999999e-05 2024-09-14T09:06:30.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136603+0000", 2024-09-14T09:06:30.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.065957551000000003 2024-09-14T09:06:30.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136644+0000", 2024-09-14T09:06:30.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1671000000000001e-05 2024-09-14T09:06:30.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136680+0000", 2024-09-14T09:06:30.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5469000000000002e-05 2024-09-14T09:06:30.757 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.757 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.757 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.757 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:37542 3.10 3:0e658405:::rbd_data.115eb07a8abd.000000000000000d:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:30.757 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.443185+0000", 2024-09-14T09:06:30.758 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.252236298, 2024-09-14T09:06:30.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.091143627000000005, 2024-09-14T09:06:30.758 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.758 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.758 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:30.758 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.758 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.758 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.758 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37542 2024-09-14T09:06:30.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.758 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443185+0000", 2024-09-14T09:06:30.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443185+0000", 2024-09-14T09:06:30.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443187+0000", 2024-09-14T09:06:30.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.367e-06 2024-09-14T09:06:30.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443199+0000", 2024-09-14T09:06:30.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2153e-05 2024-09-14T09:06:30.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443225+0000", 2024-09-14T09:06:30.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5457e-05 2024-09-14T09:06:30.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443248+0000", 2024-09-14T09:06:30.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2963e-05 2024-09-14T09:06:30.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443400+0000", 2024-09-14T09:06:30.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015241199999999999 2024-09-14T09:06:30.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443542+0000", 2024-09-14T09:06:30.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014224600000000001 2024-09-14T09:06:30.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:30.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.448965+0000", 2024-09-14T09:06:30.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054226839999999997 2024-09-14T09:06:30.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:30.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534136+0000", 2024-09-14T09:06:30.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.085170939000000001 2024-09-14T09:06:30.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:30.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534164+0000", 2024-09-14T09:06:30.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7863000000000001e-05 2024-09-14T09:06:30.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:30.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534250+0000", 2024-09-14T09:06:30.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5876000000000002e-05 2024-09-14T09:06:30.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534328+0000", 2024-09-14T09:06:30.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8666999999999997e-05 2024-09-14T09:06:30.763 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.763 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.763 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.763 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:37542 3.10 e48/28 ondisk, result = 0)", 2024-09-14T09:06:30.763 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.464008+0000", 2024-09-14T09:06:30.763 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.231412888999998, 2024-09-14T09:06:30.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.070141355000000002, 2024-09-14T09:06:30.763 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.763 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.763 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:30.763 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464008+0000", 2024-09-14T09:06:30.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464008+0000", 2024-09-14T09:06:30.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464010+0000", 2024-09-14T09:06:30.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.547e-06 2024-09-14T09:06:30.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464017+0000", 2024-09-14T09:06:30.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4930000000000001e-06 2024-09-14T09:06:30.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464047+0000", 2024-09-14T09:06:30.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0090000000000002e-05 2024-09-14T09:06:30.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464061+0000", 2024-09-14T09:06:30.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4003e-05 2024-09-14T09:06:30.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534107+0000", 2024-09-14T09:06:30.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.070045590000000005 2024-09-14T09:06:30.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534126+0000", 2024-09-14T09:06:30.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8893e-05 2024-09-14T09:06:30.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534149+0000", 2024-09-14T09:06:30.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3739000000000001e-05 2024-09-14T09:06:30.766 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.767 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.767 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.767 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38003 3.10 3:0e658405:::rbd_data.115eb07a8abd.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:30.767 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:57.392070+0000", 2024-09-14T09:06:30.767 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.303351114999998, 2024-09-14T09:06:30.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11697451, 2024-09-14T09:06:30.767 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.767 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.767 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:30.767 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.767 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.767 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.767 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38003 2024-09-14T09:06:30.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.768 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392070+0000", 2024-09-14T09:06:30.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392070+0000", 2024-09-14T09:06:30.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392072+0000", 2024-09-14T09:06:30.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6529999999999998e-06 2024-09-14T09:06:30.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392086+0000", 2024-09-14T09:06:30.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.332e-05 2024-09-14T09:06:30.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392139+0000", 2024-09-14T09:06:30.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2815999999999997e-05 2024-09-14T09:06:30.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392155+0000", 2024-09-14T09:06:30.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6796999999999999e-05 2024-09-14T09:06:30.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392296+0000", 2024-09-14T09:06:30.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014107900000000001 2024-09-14T09:06:30.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392443+0000", 2024-09-14T09:06:30.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014654499999999999 2024-09-14T09:06:30.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:30.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.399418+0000", 2024-09-14T09:06:30.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069751049999999997 2024-09-14T09:06:30.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:30.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508607+0000", 2024-09-14T09:06:30.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10918908400000001 2024-09-14T09:06:30.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:30.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508748+0000", 2024-09-14T09:06:30.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014079699999999999 2024-09-14T09:06:30.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:30.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508921+0000", 2024-09-14T09:06:30.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017345899999999999 2024-09-14T09:06:30.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.509044+0000", 2024-09-14T09:06:30.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000122855 2024-09-14T09:06:30.772 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.772 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.772 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.772 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:38003 3.10 e48/28 ondisk, result = 0)", 2024-09-14T09:06:30.772 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:57.413977+0000", 2024-09-14T09:06:30.772 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.281443891999999, 2024-09-14T09:06:30.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.095088000000000006, 2024-09-14T09:06:30.773 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.773 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.773 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:30.773 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413977+0000", 2024-09-14T09:06:30.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413977+0000", 2024-09-14T09:06:30.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413979+0000", 2024-09-14T09:06:30.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.933e-06 2024-09-14T09:06:30.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413988+0000", 2024-09-14T09:06:30.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2769999999999994e-06 2024-09-14T09:06:30.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.414021+0000", 2024-09-14T09:06:30.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2589999999999998e-05 2024-09-14T09:06:30.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.414040+0000", 2024-09-14T09:06:30.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8796000000000001e-05 2024-09-14T09:06:30.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508701+0000", 2024-09-14T09:06:30.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.094661067000000002 2024-09-14T09:06:30.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508729+0000", 2024-09-14T09:06:30.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8677e-05 2024-09-14T09:06:30.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.509065+0000", 2024-09-14T09:06:30.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033565999999999999 2024-09-14T09:06:30.776 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.776 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.776 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.776 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65139 3.a 3:54c90b7d:::rbd_data.115e21c31fcf.0000000000000008:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:06:30.776 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:06:30.776 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.3157081880000003, 2024-09-14T09:06:30.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14429895700000001, 2024-09-14T09:06:30.776 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.776 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.776 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:30.777 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:30.777 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:30.777 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:30.777 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65139 2024-09-14T09:06:30.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.777 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:06:30.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:06:30.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379715+0000", 2024-09-14T09:06:30.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.576e-06 2024-09-14T09:06:30.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379724+0000", 2024-09-14T09:06:30.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1779999999999992e-06 2024-09-14T09:06:30.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379746+0000", 2024-09-14T09:06:30.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1661000000000001e-05 2024-09-14T09:06:30.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379759+0000", 2024-09-14T09:06:30.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.324e-05 2024-09-14T09:06:30.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379882+0000", 2024-09-14T09:06:30.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012239400000000001 2024-09-14T09:06:30.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.380023+0000", 2024-09-14T09:06:30.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000141402 2024-09-14T09:06:30.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:30.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.391553+0000", 2024-09-14T09:06:30.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011530197000000001 2024-09-14T09:06:30.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:30.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523927+0000", 2024-09-14T09:06:30.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.132373935 2024-09-14T09:06:30.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:30.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523941+0000", 2024-09-14T09:06:30.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3326000000000001e-05 2024-09-14T09:06:30.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:30.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523977+0000", 2024-09-14T09:06:30.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6185e-05 2024-09-14T09:06:30.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.524012+0000", 2024-09-14T09:06:30.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4863000000000001e-05 2024-09-14T09:06:30.781 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.781 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.781 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.781 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:65139 3.a e65/34 ondisk, result = 0)", 2024-09-14T09:06:30.781 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.408501+0000", 2024-09-14T09:06:30.781 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.286919631, 2024-09-14T09:06:30.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.115431275, 2024-09-14T09:06:30.782 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:30.782 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:30.782 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:30.782 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:30.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:30.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408501+0000", 2024-09-14T09:06:30.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:30.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408501+0000", 2024-09-14T09:06:30.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:30.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:30.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408504+0000", 2024-09-14T09:06:30.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2819999999999999e-06 2024-09-14T09:06:30.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:30.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408512+0000", 2024-09-14T09:06:30.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7530000000000007e-06 2024-09-14T09:06:30.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:30.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408534+0000", 2024-09-14T09:06:30.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1993000000000001e-05 2024-09-14T09:06:30.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:30.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408550+0000", 2024-09-14T09:06:30.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5988e-05 2024-09-14T09:06:30.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:30.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523914+0000", 2024-09-14T09:06:30.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.115363542 2024-09-14T09:06:30.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:30.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523922+0000", 2024-09-14T09:06:30.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5529999999999993e-06 2024-09-14T09:06:30.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:30.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:30.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:30.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523933+0000", 2024-09-14T09:06:30.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0164e-05 2024-09-14T09:06:30.785 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.785 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.785 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.785 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:30.785 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:30.785 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:30.785 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:30.695+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:30.786 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:06:30.787 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:30.787 INFO:teuthology.orchestra.run.smithi176.stdout:3248 read 0x10c21a5 thru 0x10cd551 (0xb3ad bytes) 2024-09-14T09:06:30.787 INFO:teuthology.orchestra.run.smithi176.stdout:3249 clone 25 order 21 su 16384 sc 5 2024-09-14T09:06:30.831 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:30.835+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:30.895 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:30.895+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:30.958 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:30.959 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:30.959 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:30.959 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:30.973 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:06:30.996 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:30.995+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:31.097 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:31.095+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:31.137 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:31.137 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:31.138 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:31.138 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:31.138 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:31.152 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:06:31.199 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:31.199+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:31.234 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:06:31.253 INFO:teuthology.orchestra.run.smithi176.stderr:set 3.4 pg_upmap mapping to [1,7] 2024-09-14T09:06:31.258 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T09:06:31.282 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:31.282 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:31.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:31.283 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:31.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.283 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:19152 3.9 e39/34 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head v 39'12, mlcod=39'9)", 2024-09-14T09:06:31.283 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.333629+0000", 2024-09-14T09:06:31.283 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.950892107999998, 2024-09-14T09:06:31.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.1194095, 2024-09-14T09:06:31.283 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.283 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.283 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:31.283 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.333629+0000", 2024-09-14T09:06:31.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.333629+0000", 2024-09-14T09:06:31.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.333631+0000", 2024-09-14T09:06:31.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.007e-06 2024-09-14T09:06:31.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.350796+0000", 2024-09-14T09:06:31.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017164690999999999 2024-09-14T09:06:31.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.350865+0000", 2024-09-14T09:06:31.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8989e-05 2024-09-14T09:06:31.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.350884+0000", 2024-09-14T09:06:31.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9323000000000001e-05 2024-09-14T09:06:31.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.351045+0000", 2024-09-14T09:06:31.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016102900000000001 2024-09-14T09:06:31.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:31.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.351087+0000", 2024-09-14T09:06:31.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1430000000000001e-05 2024-09-14T09:06:31.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:31.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.452865+0000", 2024-09-14T09:06:31.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10177839900000001 2024-09-14T09:06:31.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.453039+0000", 2024-09-14T09:06:31.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017363200000000001 2024-09-14T09:06:31.287 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.287 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.287 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.288 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:37543 3.1b 3:d8e0f8d4:::rbd_data.115eb07a8abd.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:31.288 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.535280+0000", 2024-09-14T09:06:31.288 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.749241099999999, 2024-09-14T09:06:31.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.12853271799999999, 2024-09-14T09:06:31.288 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.288 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.288 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:31.288 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:31.288 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:31.288 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:31.289 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37543 2024-09-14T09:06:31.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.289 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535280+0000", 2024-09-14T09:06:31.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535280+0000", 2024-09-14T09:06:31.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535284+0000", 2024-09-14T09:06:31.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6770000000000001e-06 2024-09-14T09:06:31.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535303+0000", 2024-09-14T09:06:31.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9553000000000001e-05 2024-09-14T09:06:31.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535349+0000", 2024-09-14T09:06:31.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5833e-05 2024-09-14T09:06:31.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535368+0000", 2024-09-14T09:06:31.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8542999999999998e-05 2024-09-14T09:06:31.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535548+0000", 2024-09-14T09:06:31.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000180688 2024-09-14T09:06:31.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:31.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535814+0000", 2024-09-14T09:06:31.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026614100000000001 2024-09-14T09:06:31.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:31.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.543779+0000", 2024-09-14T09:06:31.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0079644720000000002 2024-09-14T09:06:31.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:31.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663456+0000", 2024-09-14T09:06:31.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.119676694 2024-09-14T09:06:31.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:31.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663626+0000", 2024-09-14T09:06:31.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170559 2024-09-14T09:06:31.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:31.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663728+0000", 2024-09-14T09:06:31.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101889 2024-09-14T09:06:31.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663813+0000", 2024-09-14T09:06:31.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4668999999999999e-05 2024-09-14T09:06:31.295 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.295 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.295 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.295 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:37543 3.1b e48/34 ondisk, result = 0)", 2024-09-14T09:06:31.295 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.566755+0000", 2024-09-14T09:06:31.295 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.717766562000001, 2024-09-14T09:06:31.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.096718023, 2024-09-14T09:06:31.295 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.295 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.296 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:31.296 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566755+0000", 2024-09-14T09:06:31.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566755+0000", 2024-09-14T09:06:31.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566757+0000", 2024-09-14T09:06:31.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9800000000000001e-06 2024-09-14T09:06:31.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566767+0000", 2024-09-14T09:06:31.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0036e-05 2024-09-14T09:06:31.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566797+0000", 2024-09-14T09:06:31.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.012e-05 2024-09-14T09:06:31.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566813+0000", 2024-09-14T09:06:31.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6575999999999999e-05 2024-09-14T09:06:31.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663408+0000", 2024-09-14T09:06:31.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.096594901999999996 2024-09-14T09:06:31.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:31.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663426+0000", 2024-09-14T09:06:31.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7765999999999999e-05 2024-09-14T09:06:31.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663473+0000", 2024-09-14T09:06:31.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6643e-05 2024-09-14T09:06:31.299 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.299 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.299 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.299 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38593 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [watch reconnect cookie 139718844451680 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:06:31.299 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:06:31.299 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.466103693000001, 2024-09-14T09:06:31.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0218296630000001, 2024-09-14T09:06:31.299 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.299 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.299 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:31.299 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:31.299 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:31.299 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:31.300 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38593 2024-09-14T09:06:31.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.300 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:06:31.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:06:31.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818420+0000", 2024-09-14T09:06:31.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8399999999999999e-06 2024-09-14T09:06:31.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818431+0000", 2024-09-14T09:06:31.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0859999999999999e-05 2024-09-14T09:06:31.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818466+0000", 2024-09-14T09:06:31.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4486999999999997e-05 2024-09-14T09:06:31.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818481+0000", 2024-09-14T09:06:31.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5362999999999998e-05 2024-09-14T09:06:31.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818659+0000", 2024-09-14T09:06:31.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017807500000000001 2024-09-14T09:06:31.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op must wait for map", 2024-09-14T09:06:31.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818681+0000", 2024-09-14T09:06:31.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2096999999999999e-05 2024-09-14T09:06:31.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825472+0000", 2024-09-14T09:06:31.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067905170000000003 2024-09-14T09:06:31.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:06:31.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825491+0000", 2024-09-14T09:06:31.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9372999999999999e-05 2024-09-14T09:06:31.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835402+0000", 2024-09-14T09:06:31.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0099111620000001 2024-09-14T09:06:31.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:31.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835548+0000", 2024-09-14T09:06:31.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014607300000000001 2024-09-14T09:06:31.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.840247+0000", 2024-09-14T09:06:31.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046988159999999998 2024-09-14T09:06:31.304 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.304 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.304 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.305 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38615 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:06:31.305 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:06:31.305 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.417738485000001, 2024-09-14T09:06:31.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.97014128899999996, 2024-09-14T09:06:31.305 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.305 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.305 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:31.305 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:31.305 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:31.305 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:31.305 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38615 2024-09-14T09:06:31.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.305 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:06:31.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:06:31.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866787+0000", 2024-09-14T09:06:31.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.476e-06 2024-09-14T09:06:31.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866806+0000", 2024-09-14T09:06:31.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8606000000000001e-05 2024-09-14T09:06:31.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866839+0000", 2024-09-14T09:06:31.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3510000000000003e-05 2024-09-14T09:06:31.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866867+0000", 2024-09-14T09:06:31.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7610000000000002e-05 2024-09-14T09:06:31.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867031+0000", 2024-09-14T09:06:31.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016414199999999999 2024-09-14T09:06:31.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:06:31.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867054+0000", 2024-09-14T09:06:31.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2693000000000001e-05 2024-09-14T09:06:31.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836462+0000", 2024-09-14T09:06:31.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.96940797700000003 2024-09-14T09:06:31.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:31.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836547+0000", 2024-09-14T09:06:31.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5294e-05 2024-09-14T09:06:31.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836924+0000", 2024-09-14T09:06:31.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037698099999999997 2024-09-14T09:06:31.309 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.309 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.309 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.309 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49230 3.d 3:b4a67b64:::rbd_data.115e51d92ed2.0000000000000003:12 [read 2621440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:31.309 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.523859+0000", 2024-09-14T09:06:31.310 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.760661893999998, 2024-09-14T09:06:31.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067232675000000006, 2024-09-14T09:06:31.310 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.310 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.310 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:31.310 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:31.310 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:31.310 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:31.310 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49230 2024-09-14T09:06:31.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.310 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523859+0000", 2024-09-14T09:06:31.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523859+0000", 2024-09-14T09:06:31.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523860+0000", 2024-09-14T09:06:31.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0130000000000001e-06 2024-09-14T09:06:31.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523865+0000", 2024-09-14T09:06:31.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4000000000000002e-06 2024-09-14T09:06:31.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523880+0000", 2024-09-14T09:06:31.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5287000000000001e-05 2024-09-14T09:06:31.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523892+0000", 2024-09-14T09:06:31.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2006e-05 2024-09-14T09:06:31.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.590817+0000", 2024-09-14T09:06:31.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066924692999999993 2024-09-14T09:06:31.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:31.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.590937+0000", 2024-09-14T09:06:31.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012081700000000001 2024-09-14T09:06:31.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591092+0000", 2024-09-14T09:06:31.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015445899999999999 2024-09-14T09:06:31.313 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.313 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.313 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.314 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49231 3.d 3:b4a67b64:::rbd_data.115e51d92ed2.0000000000000003:12 [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:31.314 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.523908+0000", 2024-09-14T09:06:31.314 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.760613435, 2024-09-14T09:06:31.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067373731000000006, 2024-09-14T09:06:31.314 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.314 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.314 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:31.314 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:31.314 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:31.314 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:31.314 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49231 2024-09-14T09:06:31.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.315 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523908+0000", 2024-09-14T09:06:31.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523908+0000", 2024-09-14T09:06:31.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523909+0000", 2024-09-14T09:06:31.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9699999999999994e-07 2024-09-14T09:06:31.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523913+0000", 2024-09-14T09:06:31.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6070000000000001e-06 2024-09-14T09:06:31.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523929+0000", 2024-09-14T09:06:31.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5415999999999999e-05 2024-09-14T09:06:31.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523940+0000", 2024-09-14T09:06:31.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1707e-05 2024-09-14T09:06:31.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591167+0000", 2024-09-14T09:06:31.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067226576999999996 2024-09-14T09:06:31.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591281+0000", 2024-09-14T09:06:31.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011442699999999999 2024-09-14T09:06:31.317 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.318 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.318 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.318 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49232 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:31.318 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.523956+0000", 2024-09-14T09:06:31.318 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.760565101000001, 2024-09-14T09:06:31.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067398703000000004, 2024-09-14T09:06:31.318 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.318 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.318 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:31.318 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:31.318 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:31.318 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:31.318 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49232 2024-09-14T09:06:31.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.319 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523956+0000", 2024-09-14T09:06:31.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523956+0000", 2024-09-14T09:06:31.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523957+0000", 2024-09-14T09:06:31.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8299999999999995e-07 2024-09-14T09:06:31.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523961+0000", 2024-09-14T09:06:31.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3329999999999998e-06 2024-09-14T09:06:31.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523977+0000", 2024-09-14T09:06:31.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.577e-05 2024-09-14T09:06:31.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523989+0000", 2024-09-14T09:06:31.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2089999999999999e-05 2024-09-14T09:06:31.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591340+0000", 2024-09-14T09:06:31.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067350278999999999 2024-09-14T09:06:31.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591355+0000", 2024-09-14T09:06:31.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5248e-05 2024-09-14T09:06:31.321 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.321 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.322 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.322 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49233 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:31.322 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.524004+0000", 2024-09-14T09:06:31.322 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.760517192000002, 2024-09-14T09:06:31.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067426573000000004, 2024-09-14T09:06:31.322 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.322 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.322 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:31.322 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:31.322 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:31.322 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:31.322 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49233 2024-09-14T09:06:31.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.322 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524004+0000", 2024-09-14T09:06:31.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524004+0000", 2024-09-14T09:06:31.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524005+0000", 2024-09-14T09:06:31.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3699999999999999e-07 2024-09-14T09:06:31.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524009+0000", 2024-09-14T09:06:31.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3359999999999997e-06 2024-09-14T09:06:31.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524026+0000", 2024-09-14T09:06:31.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6753e-05 2024-09-14T09:06:31.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524040+0000", 2024-09-14T09:06:31.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3694e-05 2024-09-14T09:06:31.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591411+0000", 2024-09-14T09:06:31.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067371767999999999 2024-09-14T09:06:31.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591431+0000", 2024-09-14T09:06:31.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9085000000000001e-05 2024-09-14T09:06:31.325 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.325 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.326 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.326 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49257 3.d 3.bb54482d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:31.326 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.524584+0000", 2024-09-14T09:06:31.326 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.75993729, 2024-09-14T09:06:31.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067046887999999999, 2024-09-14T09:06:31.326 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.326 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.326 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:31.326 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:31.326 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:31.326 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:31.326 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49257 2024-09-14T09:06:31.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.327 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524584+0000", 2024-09-14T09:06:31.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524584+0000", 2024-09-14T09:06:31.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524585+0000", 2024-09-14T09:06:31.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1699999999999997e-07 2024-09-14T09:06:31.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524609+0000", 2024-09-14T09:06:31.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4448999999999999e-05 2024-09-14T09:06:31.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524627+0000", 2024-09-14T09:06:31.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7543000000000001e-05 2024-09-14T09:06:31.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524634+0000", 2024-09-14T09:06:31.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7540000000000006e-06 2024-09-14T09:06:31.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591622+0000", 2024-09-14T09:06:31.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066987668 2024-09-14T09:06:31.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591631+0000", 2024-09-14T09:06:31.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7569999999999993e-06 2024-09-14T09:06:31.329 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.330 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.330 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.330 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49310 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:31.330 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525200+0000", 2024-09-14T09:06:31.330 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.759320994999999, 2024-09-14T09:06:31.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067082507, 2024-09-14T09:06:31.330 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.330 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.330 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:31.330 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:31.330 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:31.330 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:31.331 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49310 2024-09-14T09:06:31.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.331 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525200+0000", 2024-09-14T09:06:31.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525200+0000", 2024-09-14T09:06:31.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525201+0000", 2024-09-14T09:06:31.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.023e-06 2024-09-14T09:06:31.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525206+0000", 2024-09-14T09:06:31.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5000000000000001e-06 2024-09-14T09:06:31.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525242+0000", 2024-09-14T09:06:31.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6672999999999999e-05 2024-09-14T09:06:31.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525256+0000", 2024-09-14T09:06:31.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3360000000000001e-05 2024-09-14T09:06:31.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592266+0000", 2024-09-14T09:06:31.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067009868 2024-09-14T09:06:31.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592283+0000", 2024-09-14T09:06:31.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7082999999999999e-05 2024-09-14T09:06:31.333 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.333 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.334 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.334 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49311 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:31.334 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525275+0000", 2024-09-14T09:06:31.334 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.759246219000001, 2024-09-14T09:06:31.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067067112999999998, 2024-09-14T09:06:31.334 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.334 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.334 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:31.334 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:31.334 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:31.334 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:31.334 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49311 2024-09-14T09:06:31.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.335 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525275+0000", 2024-09-14T09:06:31.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525275+0000", 2024-09-14T09:06:31.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525276+0000", 2024-09-14T09:06:31.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.313e-06 2024-09-14T09:06:31.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525282+0000", 2024-09-14T09:06:31.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.127e-06 2024-09-14T09:06:31.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525299+0000", 2024-09-14T09:06:31.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6903000000000001e-05 2024-09-14T09:06:31.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525312+0000", 2024-09-14T09:06:31.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3193999999999999e-05 2024-09-14T09:06:31.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592323+0000", 2024-09-14T09:06:31.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067010504999999998 2024-09-14T09:06:31.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592342+0000", 2024-09-14T09:06:31.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9071000000000001e-05 2024-09-14T09:06:31.337 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.337 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.337 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.338 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49312 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:31.338 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525330+0000", 2024-09-14T09:06:31.338 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.759190826000001, 2024-09-14T09:06:31.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.06706268, 2024-09-14T09:06:31.338 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.338 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.338 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:31.338 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:31.338 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:31.338 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:31.338 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49312 2024-09-14T09:06:31.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.339 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525330+0000", 2024-09-14T09:06:31.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525330+0000", 2024-09-14T09:06:31.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525332+0000", 2024-09-14T09:06:31.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.187e-06 2024-09-14T09:06:31.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525350+0000", 2024-09-14T09:06:31.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8833000000000001e-05 2024-09-14T09:06:31.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525385+0000", 2024-09-14T09:06:31.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4686e-05 2024-09-14T09:06:31.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525399+0000", 2024-09-14T09:06:31.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4239999999999999e-05 2024-09-14T09:06:31.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592377+0000", 2024-09-14T09:06:31.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066977255999999999 2024-09-14T09:06:31.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592393+0000", 2024-09-14T09:06:31.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6478e-05 2024-09-14T09:06:31.341 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.341 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.341 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.342 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49313 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:31.342 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525423+0000", 2024-09-14T09:06:31.342 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.759098427000001, 2024-09-14T09:06:31.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067039319, 2024-09-14T09:06:31.342 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.342 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.342 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:31.342 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:31.342 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:31.342 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:31.342 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49313 2024-09-14T09:06:31.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.342 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525423+0000", 2024-09-14T09:06:31.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525423+0000", 2024-09-14T09:06:31.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525424+0000", 2024-09-14T09:06:31.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.257e-06 2024-09-14T09:06:31.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525430+0000", 2024-09-14T09:06:31.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8699999999999997e-06 2024-09-14T09:06:31.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525447+0000", 2024-09-14T09:06:31.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7560000000000001e-05 2024-09-14T09:06:31.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525460+0000", 2024-09-14T09:06:31.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2243e-05 2024-09-14T09:06:31.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592444+0000", 2024-09-14T09:06:31.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066984118999999995 2024-09-14T09:06:31.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592462+0000", 2024-09-14T09:06:31.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.827e-05 2024-09-14T09:06:31.345 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.345 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.345 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.346 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49314 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:31.346 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525478+0000", 2024-09-14T09:06:31.346 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.759042663999999, 2024-09-14T09:06:31.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.06702756, 2024-09-14T09:06:31.346 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.346 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.346 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:31.346 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:31.346 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:31.346 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:31.346 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49314 2024-09-14T09:06:31.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.346 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525478+0000", 2024-09-14T09:06:31.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525478+0000", 2024-09-14T09:06:31.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525480+0000", 2024-09-14T09:06:31.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.246e-06 2024-09-14T09:06:31.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525485+0000", 2024-09-14T09:06:31.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7440000000000004e-06 2024-09-14T09:06:31.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525504+0000", 2024-09-14T09:06:31.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8709999999999999e-05 2024-09-14T09:06:31.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525512+0000", 2024-09-14T09:06:31.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6260000000000003e-06 2024-09-14T09:06:31.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592491+0000", 2024-09-14T09:06:31.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066979274000000005 2024-09-14T09:06:31.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592506+0000", 2024-09-14T09:06:31.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4960000000000001e-05 2024-09-14T09:06:31.349 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.349 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.349 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.350 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49315 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:31.350 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525523+0000", 2024-09-14T09:06:31.350 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.758997868000002, 2024-09-14T09:06:31.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067036894, 2024-09-14T09:06:31.350 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.350 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.350 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:31.350 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:31.350 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:31.350 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:31.350 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49315 2024-09-14T09:06:31.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.350 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525523+0000", 2024-09-14T09:06:31.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525523+0000", 2024-09-14T09:06:31.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525524+0000", 2024-09-14T09:06:31.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2999999999999999e-07 2024-09-14T09:06:31.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525528+0000", 2024-09-14T09:06:31.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4000000000000001e-06 2024-09-14T09:06:31.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525537+0000", 2024-09-14T09:06:31.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2739999999999996e-06 2024-09-14T09:06:31.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525543+0000", 2024-09-14T09:06:31.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6329999999999999e-06 2024-09-14T09:06:31.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592548+0000", 2024-09-14T09:06:31.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067004806 2024-09-14T09:06:31.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592560+0000", 2024-09-14T09:06:31.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1951e-05 2024-09-14T09:06:31.353 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.353 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.353 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.354 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49316 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:31.354 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525553+0000", 2024-09-14T09:06:31.354 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.758967974000001, 2024-09-14T09:06:31.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067056432999999999, 2024-09-14T09:06:31.354 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.354 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.354 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:31.354 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:31.354 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:31.354 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:31.354 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49316 2024-09-14T09:06:31.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.355 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525553+0000", 2024-09-14T09:06:31.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525553+0000", 2024-09-14T09:06:31.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525554+0000", 2024-09-14T09:06:31.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0299999999999998e-07 2024-09-14T09:06:31.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525557+0000", 2024-09-14T09:06:31.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9129999999999999e-06 2024-09-14T09:06:31.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525566+0000", 2024-09-14T09:06:31.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8000000000000004e-06 2024-09-14T09:06:31.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525572+0000", 2024-09-14T09:06:31.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.72e-06 2024-09-14T09:06:31.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592598+0000", 2024-09-14T09:06:31.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067025682000000003 2024-09-14T09:06:31.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592610+0000", 2024-09-14T09:06:31.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1515000000000001e-05 2024-09-14T09:06:31.357 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.358 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.358 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.358 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49317 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:31.358 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:31.358 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.758939515000002, 2024-09-14T09:06:31.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067082547000000006, 2024-09-14T09:06:31.358 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.358 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.358 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:31.358 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:31.358 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:31.358 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:31.359 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49317 2024-09-14T09:06:31.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.359 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:31.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:31.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:31.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:06:31.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525585+0000", 2024-09-14T09:06:31.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8700000000000001e-06 2024-09-14T09:06:31.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525594+0000", 2024-09-14T09:06:31.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9740000000000008e-06 2024-09-14T09:06:31.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525601+0000", 2024-09-14T09:06:31.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9489999999999999e-06 2024-09-14T09:06:31.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592652+0000", 2024-09-14T09:06:31.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067050803000000006 2024-09-14T09:06:31.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592664+0000", 2024-09-14T09:06:31.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2198000000000001e-05 2024-09-14T09:06:31.361 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.362 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.362 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.362 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49318 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:31.362 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525642+0000", 2024-09-14T09:06:31.362 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.758879015000002, 2024-09-14T09:06:31.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067064055999999997, 2024-09-14T09:06:31.362 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.362 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.362 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:31.362 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:31.362 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:31.362 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:31.362 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49318 2024-09-14T09:06:31.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.363 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525642+0000", 2024-09-14T09:06:31.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525642+0000", 2024-09-14T09:06:31.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525644+0000", 2024-09-14T09:06:31.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0159999999999998e-06 2024-09-14T09:06:31.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525649+0000", 2024-09-14T09:06:31.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2739999999999998e-06 2024-09-14T09:06:31.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525666+0000", 2024-09-14T09:06:31.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6990000000000002e-05 2024-09-14T09:06:31.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525680+0000", 2024-09-14T09:06:31.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.331e-05 2024-09-14T09:06:31.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592697+0000", 2024-09-14T09:06:31.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067017367999999994 2024-09-14T09:06:31.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592706+0000", 2024-09-14T09:06:31.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.098e-06 2024-09-14T09:06:31.366 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.366 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.366 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.366 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49319 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:31.366 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525704+0000", 2024-09-14T09:06:31.366 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.758817609000001, 2024-09-14T09:06:31.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067049867999999999, 2024-09-14T09:06:31.366 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:31.366 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:31.366 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:31.366 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:31.367 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:31.367 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:31.367 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49319 2024-09-14T09:06:31.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.367 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:31.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:31.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525704+0000", 2024-09-14T09:06:31.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:31.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525704+0000", 2024-09-14T09:06:31.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:31.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:31.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525705+0000", 2024-09-14T09:06:31.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.113e-06 2024-09-14T09:06:31.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:31.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525710+0000", 2024-09-14T09:06:31.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.843e-06 2024-09-14T09:06:31.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:31.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525726+0000", 2024-09-14T09:06:31.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.679e-05 2024-09-14T09:06:31.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:31.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525739+0000", 2024-09-14T09:06:31.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2962999999999999e-05 2024-09-14T09:06:31.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:31.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592745+0000", 2024-09-14T09:06:31.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067005341999999996 2024-09-14T09:06:31.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:31.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:31.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:31.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592753+0000", 2024-09-14T09:06:31.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8170000000000005e-06 2024-09-14T09:06:31.369 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.370 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.370 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.370 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:31.370 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:31.370 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:31.371 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:06:31.372 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:31.371+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:31.384 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:31.384 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:31.412 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:31.413 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:31.440 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:31.440 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:31.458 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:31.458 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:31.472 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:31.471+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:31.478 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:31.478 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:31.489 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:31.490 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:31.499 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:31.500 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:31.504 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:06:31.519 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:31.520 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:31.540 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:31.541 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:31.573 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:31.575+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:31.634 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:31.634 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:31.648 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:06:31.674 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:31.675+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:31.769 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:31.769 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:31.774 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:31.775+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:31.783 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:06:31.826 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is unset 2024-09-14T09:06:31.876 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:31.879+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:31.904 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:31.904 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:31.918 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:06:31.977 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:31.975+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:32.040 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:32.040 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:32.054 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:06:32.098 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:32.099+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:32.178 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:32.179+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:32.218 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:32.218 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:32.232 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:06:32.280 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:32.283+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:32.381 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:32.379+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:32.391 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:32.391 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:32.405 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:06:32.482 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:32.483+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:32.572 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:32.572 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:32.582 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:32.583+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:32.586 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:06:32.684 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:32.683+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:32.754 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:32.754 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:32.768 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:06:32.810 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:32.811+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:32.886 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:32.887+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:32.939 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:32.939 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:32.953 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:06:32.987 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:32.991+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:33.075 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:33.075 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:33.089 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:33.091+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:33.090 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:06:33.190 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:33.187+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:33.219 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:33.219 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:33.236 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:06:33.291 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:33.291+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:33.333 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-14T09:06:33.372 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:33.372 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:33.386 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:06:33.392 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:33.391+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:33.492 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:33.491+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:33.552 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:33.553 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:33.559 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:06:33.566 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:06:33.615 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:33.619+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:33.689 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:33.689 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:33.693 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:33.691+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:33.704 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:06:33.717 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:33.718 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:33.735 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:33.735 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:33.752 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:33.752 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:33.768 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:33.768 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:33.786 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:33.786 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:33.795 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:33.795+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:33.795 INFO:teuthology.orchestra.run.smithi176.stdout:3250 read 0xab174a thru 0xabf7bf (0xe076 bytes) 2024-09-14T09:06:33.804 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:33.804 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:33.811 INFO:teuthology.orchestra.run.smithi176.stdout:3252 writesame 0x4b8f866 thru 0x4b90608 data_size 0x1(0xda3 bytes) 2024-09-14T09:06:33.812 INFO:teuthology.orchestra.run.smithi176.stdout:3253 writesame 0x13a5553 thru 0x13abb34 data_size 0x51(0x65e2 bytes) 2024-09-14T09:06:33.826 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:33.826 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:33.847 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T09:06:33.853 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:33.853 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:33.869 INFO:teuthology.orchestra.run.smithi176.stdout:3255 writesame 0xb5a5929 thru 0xb5b0b61 data_size 0x49(0xb239 bytes) 2024-09-14T09:06:33.869 INFO:teuthology.orchestra.run.smithi176.stdout:3256 punch from 0x2217473 to 0x2219ade, (0x266b bytes) 2024-09-14T09:06:33.869 INFO:teuthology.orchestra.run.smithi176.stdout:3257 punch from 0xa8942fc to 0xa895154, (0xe58 bytes) 2024-09-14T09:06:33.869 INFO:teuthology.orchestra.run.smithi176.stdout:3258 read 0x2d13e4a thru 0x2d1d443 (0x95fa bytes) 2024-09-14T09:06:33.877 INFO:teuthology.orchestra.run.smithi176.stdout:3261 write 0xb6c921c thru 0xb6d16ec (0x84d1 bytes) 2024-09-14T09:06:33.879 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:33.879 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:33.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.879 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:71916 3.1f 3:fa4cd46f:::rbd_object_map.115ea8d131ff:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:33.879 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:33.882489+0000", 2024-09-14T09:06:33.879 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.001421481, 2024-09-14T09:06:33.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001444414, 2024-09-14T09:06:33.880 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:33.880 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:33.880 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for sub ops", 2024-09-14T09:06:33.880 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:33.880 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:33.880 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:33.880 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 71916 2024-09-14T09:06:33.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.880 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:33.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:33.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882489+0000", 2024-09-14T09:06:33.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:33.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:33.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882489+0000", 2024-09-14T09:06:33.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:33.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:33.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882491+0000", 2024-09-14T09:06:33.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1339999999999998e-06 2024-09-14T09:06:33.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:33.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882502+0000", 2024-09-14T09:06:33.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.083e-05 2024-09-14T09:06:33.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:33.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882529+0000", 2024-09-14T09:06:33.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7923e-05 2024-09-14T09:06:33.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:33.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882544+0000", 2024-09-14T09:06:33.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4555999999999999e-05 2024-09-14T09:06:33.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:33.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882705+0000", 2024-09-14T09:06:33.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016126500000000001 2024-09-14T09:06:33.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:33.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882897+0000", 2024-09-14T09:06:33.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019215599999999999 2024-09-14T09:06:33.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-09-14T09:06:33.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.883494+0000", 2024-09-14T09:06:33.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00059697500000000002 2024-09-14T09:06:33.883 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:33.883 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:33.883 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:33.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.883 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:71917 3.1f 3.f62b325f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:33.884 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:33.882606+0000", 2024-09-14T09:06:33.884 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.0013035379999999999, 2024-09-14T09:06:33.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00137743, 2024-09-14T09:06:33.884 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:33.884 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:33.884 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:33.884 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:33.884 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:33.884 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:33.884 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 71917 2024-09-14T09:06:33.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.884 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:33.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:33.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882606+0000", 2024-09-14T09:06:33.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:33.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:33.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882606+0000", 2024-09-14T09:06:33.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:33.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:33.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882608+0000", 2024-09-14T09:06:33.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9659999999999999e-06 2024-09-14T09:06:33.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:33.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882617+0000", 2024-09-14T09:06:33.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6670000000000002e-06 2024-09-14T09:06:33.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:33.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882648+0000", 2024-09-14T09:06:33.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0892999999999997e-05 2024-09-14T09:06:33.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:33.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882668+0000", 2024-09-14T09:06:33.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9813000000000002e-05 2024-09-14T09:06:33.887 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:33.887 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:33.887 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:33.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.887 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:71918 3.1f 3.f62b325f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:33.887 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:33.882699+0000", 2024-09-14T09:06:33.887 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.001210809, 2024-09-14T09:06:33.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013184309999999999, 2024-09-14T09:06:33.887 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:33.887 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:33.888 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "queued for pg", 2024-09-14T09:06:33.888 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:33.888 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:33.888 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:33.888 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 71918 2024-09-14T09:06:33.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.888 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:33.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:33.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882699+0000", 2024-09-14T09:06:33.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:33.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:33.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882699+0000", 2024-09-14T09:06:33.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:33.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:33.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882701+0000", 2024-09-14T09:06:33.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4729999999999999e-06 2024-09-14T09:06:33.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:33.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882708+0000", 2024-09-14T09:06:33.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5900000000000002e-06 2024-09-14T09:06:33.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:33.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882747+0000", 2024-09-14T09:06:33.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8776999999999998e-05 2024-09-14T09:06:33.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:33.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:33.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:33.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:33.882764+0000", 2024-09-14T09:06:33.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6993000000000001e-05 2024-09-14T09:06:33.890 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:33.890 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:33.890 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:33.890 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:33.890 INFO:teuthology.orchestra.run.smithi183.stdout: ], 2024-09-14T09:06:33.890 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 3 2024-09-14T09:06:33.891 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:33.891 INFO:teuthology.orchestra.run.smithi176.stdout:3262 read 0x921283f thru 0x9218274 (0x5a36 bytes) 2024-09-14T09:06:33.891 INFO:teuthology.orchestra.run.smithi176.stdout:3266 trunc from 0xb6d16ed to 0x8b5f409 2024-09-14T09:06:33.895 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:06:33.896 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:33.899+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:33.903 INFO:teuthology.orchestra.run.smithi176.stdout:3267 read 0x9246 thru 0x19057 (0xfe12 bytes) 2024-09-14T09:06:33.905 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:33.905 INFO:teuthology.orchestra.run.smithi176.stdout:3269 write 0x813cab9 thru 0x814976a (0xccb2 bytes) 2024-09-14T09:06:33.912 INFO:teuthology.orchestra.run.smithi176.stdout:3271 read 0x66c8b3d thru 0x66d2a8d (0x9f51 bytes) 2024-09-14T09:06:33.912 INFO:teuthology.orchestra.run.smithi176.stdout:3272 punch from 0x73da1c8 to 0x73dab7b, (0x9b3 bytes) 2024-09-14T09:06:33.928 INFO:teuthology.orchestra.run.smithi176.stdout:3274 writesame 0xe0a0f4a thru 0xe0a960e data_size 0x1(0x86c5 bytes) 2024-09-14T09:06:33.928 INFO:teuthology.orchestra.run.smithi176.stdout:3276 punch from 0x817e548 to 0x818531b, (0x6dd3 bytes) 2024-09-14T09:06:33.928 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:33.928 INFO:teuthology.orchestra.run.smithi176.stdout:3278 write 0x82528fc thru 0x8262753 (0xfe58 bytes) 2024-09-14T09:06:33.936 INFO:teuthology.orchestra.run.smithi176.stdout:3280 write 0x7fe611b thru 0x7ff365f (0xd545 bytes) 2024-09-14T09:06:33.940 INFO:teuthology.orchestra.run.smithi176.stdout:3281 write 0x8c763a3 thru 0x8c7a0d3 (0x3d31 bytes) 2024-09-14T09:06:33.948 INFO:teuthology.orchestra.run.smithi176.stdout:3282 write 0x1080772 thru 0x1083206 (0x2a95 bytes) 2024-09-14T09:06:33.981 INFO:teuthology.orchestra.run.smithi176.stdout:3283 read 0xb3e4f10 thru 0xb3efa47 (0xab38 bytes) 2024-09-14T09:06:33.981 INFO:teuthology.orchestra.run.smithi176.stdout:3284 punch from 0x5d5e5cb to 0x5d6afa9, (0xc9de bytes) 2024-09-14T09:06:33.981 INFO:teuthology.orchestra.run.smithi176.stdout:3285 punch from 0x9135727 to 0x9138060, (0x2939 bytes) 2024-09-14T09:06:33.981 INFO:teuthology.orchestra.run.smithi176.stdout:3287 writesame 0xa50a0d3 thru 0xa50c9c1 data_size 0x7(0x28ef bytes) 2024-09-14T09:06:33.986 INFO:teuthology.orchestra.run.smithi176.stdout:3288 write 0x71cec62 thru 0x71d918f (0xa52e bytes) 2024-09-14T09:06:33.996 INFO:teuthology.orchestra.run.smithi176.stdout:3290 write 0x8148f38 thru 0x814f84d (0x6916 bytes) 2024-09-14T09:06:33.996 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:33.995+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:33.999 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:33.999 INFO:teuthology.orchestra.run.smithi176.stdout:3292 read 0x5e6ab1c thru 0x5e77432 (0xc917 bytes) 2024-09-14T09:06:33.999 INFO:teuthology.orchestra.run.smithi176.stdout:3293 read 0xb4c465e thru 0xb4d4157 (0xfafa bytes) 2024-09-14T09:06:33.999 INFO:teuthology.orchestra.run.smithi176.stdout:3296 read 0xb848ec8 thru 0xb8565b6 (0xd6ef bytes) 2024-09-14T09:06:33.999 INFO:teuthology.orchestra.run.smithi176.stdout:3299 read 0x5ff5564 thru 0x5ff6711 (0x11ae bytes) 2024-09-14T09:06:33.999 INFO:teuthology.orchestra.run.smithi176.stdout:3300 write 0xc2254fa thru 0xc22fef6 (0xa9fd bytes) 2024-09-14T09:06:34.005 INFO:teuthology.orchestra.run.smithi176.stdout:3301 writesame 0x92995d3 thru 0x929fcae data_size 0x3a(0x66dc bytes) 2024-09-14T09:06:34.010 INFO:teuthology.orchestra.run.smithi176.stdout:3303 write 0x5357693 thru 0x5365c9a (0xe608 bytes) 2024-09-14T09:06:34.020 INFO:teuthology.orchestra.run.smithi176.stdout:3304 writesame 0x527535 thru 0x53491c data_size 0x4(0xd3e8 bytes) 2024-09-14T09:06:34.077 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:34.078 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:34.078 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:34.078 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:34.078 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:34.098 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:06:34.098 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:34.095+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:34.181 INFO:teuthology.orchestra.run.smithi176.stdout:3305 trunc from 0xe0a960f to 0x789aa84 2024-09-14T09:06:34.198 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:34.199+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:34.205 INFO:teuthology.orchestra.run.smithi176.stdout:3306 read 0x6c17ec0 thru 0x6c25d9c (0xdedd bytes) 2024-09-14T09:06:34.205 INFO:teuthology.orchestra.run.smithi176.stdout:3307 read 0x3676cdd thru 0x367fd79 (0x909d bytes) 2024-09-14T09:06:34.205 INFO:teuthology.orchestra.run.smithi176.stdout:3308 punch from 0x619a43d to 0x61a1431, (0x6ff4 bytes) 2024-09-14T09:06:34.213 INFO:teuthology.orchestra.run.smithi176.stdout:3309 write 0x85265f1 thru 0x8530c00 (0xa610 bytes) 2024-09-14T09:06:34.221 INFO:teuthology.orchestra.run.smithi176.stdout:3311 write 0x2f6a4f9 thru 0x2f7332b (0x8e33 bytes) 2024-09-14T09:06:34.232 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:34.233 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:34.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:34.233 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:34.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.233 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49333 3.1d 3:be596f9e:::rbd_data.115e51d92ed2.0000000000000007:12 [read 950272~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:34.233 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:34.233 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.688052643999999, 2024-09-14T09:06:34.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072172603000000002, 2024-09-14T09:06:34.233 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.233 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.233 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:34.233 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.233 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.234 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.234 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49333 2024-09-14T09:06:34.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.234 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:34.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:34.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546213+0000", 2024-09-14T09:06:34.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6000000000000003e-07 2024-09-14T09:06:34.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546216+0000", 2024-09-14T09:06:34.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5339999999999998e-06 2024-09-14T09:06:34.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546224+0000", 2024-09-14T09:06:34.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7770000000000002e-06 2024-09-14T09:06:34.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546230+0000", 2024-09-14T09:06:34.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3400000000000003e-06 2024-09-14T09:06:34.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.617881+0000", 2024-09-14T09:06:34.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071650488999999998 2024-09-14T09:06:34.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.617958+0000", 2024-09-14T09:06:34.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7018e-05 2024-09-14T09:06:34.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618385+0000", 2024-09-14T09:06:34.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00042768500000000002 2024-09-14T09:06:34.237 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.237 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.237 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.237 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49334 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:34.238 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:34.238 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.688025761999999, 2024-09-14T09:06:34.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072227886000000005, 2024-09-14T09:06:34.238 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.238 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.238 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:34.238 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.238 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.238 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.238 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49334 2024-09-14T09:06:34.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.239 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:34.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:34.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:34.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8899999999999999e-07 2024-09-14T09:06:34.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546243+0000", 2024-09-14T09:06:34.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.503e-06 2024-09-14T09:06:34.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546253+0000", 2024-09-14T09:06:34.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0210999999999999e-05 2024-09-14T09:06:34.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546267+0000", 2024-09-14T09:06:34.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4183e-05 2024-09-14T09:06:34.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618449+0000", 2024-09-14T09:06:34.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072181828000000003 2024-09-14T09:06:34.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618467+0000", 2024-09-14T09:06:34.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8471999999999999e-05 2024-09-14T09:06:34.242 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.242 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.242 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.242 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49335 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:34.243 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:34.243 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.687989610999999, 2024-09-14T09:06:34.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072255640999999995, 2024-09-14T09:06:34.243 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.243 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.243 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:34.243 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.243 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.243 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.243 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49335 2024-09-14T09:06:34.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.243 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:34.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:34.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:34.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9999999999999997e-07 2024-09-14T09:06:34.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546279+0000", 2024-09-14T09:06:34.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.695e-06 2024-09-14T09:06:34.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546288+0000", 2024-09-14T09:06:34.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7110000000000001e-06 2024-09-14T09:06:34.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546294+0000", 2024-09-14T09:06:34.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9830000000000001e-06 2024-09-14T09:06:34.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618521+0000", 2024-09-14T09:06:34.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072226948999999999 2024-09-14T09:06:34.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618531+0000", 2024-09-14T09:06:34.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0603e-05 2024-09-14T09:06:34.247 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.247 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.247 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.247 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49336 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:34.247 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:34.247 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.687963708000002, 2024-09-14T09:06:34.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072292223000000003, 2024-09-14T09:06:34.247 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.247 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.247 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:34.247 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.247 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.248 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.248 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49336 2024-09-14T09:06:34.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.248 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:34.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:34.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:34.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4000000000000001e-07 2024-09-14T09:06:34.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546305+0000", 2024-09-14T09:06:34.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3539999999999998e-06 2024-09-14T09:06:34.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546315+0000", 2024-09-14T09:06:34.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0431999999999999e-05 2024-09-14T09:06:34.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546321+0000", 2024-09-14T09:06:34.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7620000000000001e-06 2024-09-14T09:06:34.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618583+0000", 2024-09-14T09:06:34.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072262046999999996 2024-09-14T09:06:34.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618594+0000", 2024-09-14T09:06:34.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0988000000000001e-05 2024-09-14T09:06:34.251 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.251 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.251 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.251 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49337 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:34.251 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:34.251 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.687916093999998, 2024-09-14T09:06:34.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072294774000000006, 2024-09-14T09:06:34.251 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.252 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.252 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:34.252 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.252 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.252 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.252 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49337 2024-09-14T09:06:34.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.252 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:34.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:34.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546350+0000", 2024-09-14T09:06:34.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4600000000000004e-07 2024-09-14T09:06:34.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546381+0000", 2024-09-14T09:06:34.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0902000000000003e-05 2024-09-14T09:06:34.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546412+0000", 2024-09-14T09:06:34.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0825999999999999e-05 2024-09-14T09:06:34.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546438+0000", 2024-09-14T09:06:34.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6410999999999998e-05 2024-09-14T09:06:34.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618637+0000", 2024-09-14T09:06:34.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072198907000000007 2024-09-14T09:06:34.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618644+0000", 2024-09-14T09:06:34.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9820000000000002e-06 2024-09-14T09:06:34.255 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.255 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.255 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.255 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49338 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:34.255 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:34.255 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.687802067, 2024-09-14T09:06:34.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072233250999999998, 2024-09-14T09:06:34.256 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.256 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.256 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:34.256 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.256 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.256 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.256 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49338 2024-09-14T09:06:34.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.256 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:34.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:34.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546465+0000", 2024-09-14T09:06:34.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.455e-06 2024-09-14T09:06:34.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546481+0000", 2024-09-14T09:06:34.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5834000000000001e-05 2024-09-14T09:06:34.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546498+0000", 2024-09-14T09:06:34.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6937000000000001e-05 2024-09-14T09:06:34.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546508+0000", 2024-09-14T09:06:34.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0446e-05 2024-09-14T09:06:34.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618685+0000", 2024-09-14T09:06:34.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072177280999999996 2024-09-14T09:06:34.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618697+0000", 2024-09-14T09:06:34.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1297999999999999e-05 2024-09-14T09:06:34.259 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.259 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.259 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.259 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49339 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:34.259 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546535+0000", 2024-09-14T09:06:34.259 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.687730049999999, 2024-09-14T09:06:34.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072197266999999996, 2024-09-14T09:06:34.260 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.260 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.260 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:34.260 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.260 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.260 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.260 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49339 2024-09-14T09:06:34.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.260 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546535+0000", 2024-09-14T09:06:34.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546535+0000", 2024-09-14T09:06:34.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546536+0000", 2024-09-14T09:06:34.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1399999999999996e-07 2024-09-14T09:06:34.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546539+0000", 2024-09-14T09:06:34.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.089e-06 2024-09-14T09:06:34.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546560+0000", 2024-09-14T09:06:34.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1217e-05 2024-09-14T09:06:34.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546567+0000", 2024-09-14T09:06:34.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9e-06 2024-09-14T09:06:34.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618726+0000", 2024-09-14T09:06:34.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.07215887 2024-09-14T09:06:34.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618733+0000", 2024-09-14T09:06:34.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3770000000000001e-06 2024-09-14T09:06:34.263 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.263 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.263 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.263 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49340 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:34.263 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546578+0000", 2024-09-14T09:06:34.263 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.687687158999999, 2024-09-14T09:06:34.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072205187000000004, 2024-09-14T09:06:34.263 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.263 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.264 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:34.264 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.264 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.264 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.264 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49340 2024-09-14T09:06:34.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.264 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546578+0000", 2024-09-14T09:06:34.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546578+0000", 2024-09-14T09:06:34.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546579+0000", 2024-09-14T09:06:34.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1999999999999999e-07 2024-09-14T09:06:34.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546582+0000", 2024-09-14T09:06:34.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4460000000000001e-06 2024-09-14T09:06:34.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546603+0000", 2024-09-14T09:06:34.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0502000000000001e-05 2024-09-14T09:06:34.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546610+0000", 2024-09-14T09:06:34.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7150000000000001e-06 2024-09-14T09:06:34.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618770+0000", 2024-09-14T09:06:34.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072160600000000005 2024-09-14T09:06:34.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618783+0000", 2024-09-14T09:06:34.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:06:34.267 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.267 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.267 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.267 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49341 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:34.267 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546667+0000", 2024-09-14T09:06:34.267 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.687598050999998, 2024-09-14T09:06:34.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072181163000000007, 2024-09-14T09:06:34.267 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.267 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.267 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:34.268 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.268 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.268 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.268 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49341 2024-09-14T09:06:34.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.268 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546667+0000", 2024-09-14T09:06:34.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546667+0000", 2024-09-14T09:06:34.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546669+0000", 2024-09-14T09:06:34.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.486e-06 2024-09-14T09:06:34.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546674+0000", 2024-09-14T09:06:34.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5110000000000003e-06 2024-09-14T09:06:34.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546691+0000", 2024-09-14T09:06:34.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7051e-05 2024-09-14T09:06:34.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546703+0000", 2024-09-14T09:06:34.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1155000000000001e-05 2024-09-14T09:06:34.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618836+0000", 2024-09-14T09:06:34.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072133616999999997 2024-09-14T09:06:34.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618848+0000", 2024-09-14T09:06:34.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2343e-05 2024-09-14T09:06:34.271 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.271 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.271 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.271 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49386 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:34.271 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546806+0000", 2024-09-14T09:06:34.271 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.687458891999999, 2024-09-14T09:06:34.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072019206000000002, 2024-09-14T09:06:34.271 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.271 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.272 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:34.272 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.272 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.272 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.272 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49386 2024-09-14T09:06:34.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.272 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546806+0000", 2024-09-14T09:06:34.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546806+0000", 2024-09-14T09:06:34.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546808+0000", 2024-09-14T09:06:34.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.471e-06 2024-09-14T09:06:34.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546813+0000", 2024-09-14T09:06:34.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1800000000000004e-06 2024-09-14T09:06:34.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546830+0000", 2024-09-14T09:06:34.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.658e-05 2024-09-14T09:06:34.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546842+0000", 2024-09-14T09:06:34.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2442999999999999e-05 2024-09-14T09:06:34.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618816+0000", 2024-09-14T09:06:34.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071973920999999996 2024-09-14T09:06:34.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618826+0000", 2024-09-14T09:06:34.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.611e-06 2024-09-14T09:06:34.275 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.275 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.275 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.275 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49387 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:34.275 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546861+0000", 2024-09-14T09:06:34.275 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.687404247, 2024-09-14T09:06:34.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071996665000000001, 2024-09-14T09:06:34.275 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.276 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.276 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:34.276 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.276 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.276 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.276 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49387 2024-09-14T09:06:34.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.276 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546861+0000", 2024-09-14T09:06:34.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546861+0000", 2024-09-14T09:06:34.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546862+0000", 2024-09-14T09:06:34.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.186e-06 2024-09-14T09:06:34.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546867+0000", 2024-09-14T09:06:34.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9049999999999996e-06 2024-09-14T09:06:34.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546880+0000", 2024-09-14T09:06:34.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2815e-05 2024-09-14T09:06:34.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546889+0000", 2024-09-14T09:06:34.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5420000000000005e-06 2024-09-14T09:06:34.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618850+0000", 2024-09-14T09:06:34.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071961065000000005 2024-09-14T09:06:34.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618858+0000", 2024-09-14T09:06:34.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1519999999999992e-06 2024-09-14T09:06:34.279 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.279 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.279 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.279 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49388 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:34.279 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:34.279 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.687366562000001, 2024-09-14T09:06:34.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072001645000000003, 2024-09-14T09:06:34.280 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.280 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.280 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:34.280 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.280 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.280 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.280 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49388 2024-09-14T09:06:34.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.280 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:34.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:34.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546899+0000", 2024-09-14T09:06:34.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:06:34.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546902+0000", 2024-09-14T09:06:34.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.734e-06 2024-09-14T09:06:34.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546910+0000", 2024-09-14T09:06:34.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9629999999999998e-06 2024-09-14T09:06:34.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546916+0000", 2024-09-14T09:06:34.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.28e-06 2024-09-14T09:06:34.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618889+0000", 2024-09-14T09:06:34.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071972555999999993 2024-09-14T09:06:34.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618900+0000", 2024-09-14T09:06:34.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1413999999999999e-05 2024-09-14T09:06:34.283 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.283 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.283 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.283 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49389 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:34.283 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546925+0000", 2024-09-14T09:06:34.283 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.687340402, 2024-09-14T09:06:34.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072026707999999995, 2024-09-14T09:06:34.284 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.284 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.284 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:34.284 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.284 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.284 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.284 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49389 2024-09-14T09:06:34.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.284 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546925+0000", 2024-09-14T09:06:34.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546925+0000", 2024-09-14T09:06:34.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546926+0000", 2024-09-14T09:06:34.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8599999999999998e-07 2024-09-14T09:06:34.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546928+0000", 2024-09-14T09:06:34.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4200000000000001e-06 2024-09-14T09:06:34.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546938+0000", 2024-09-14T09:06:34.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5859999999999994e-06 2024-09-14T09:06:34.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546945+0000", 2024-09-14T09:06:34.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1770000000000001e-06 2024-09-14T09:06:34.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618940+0000", 2024-09-14T09:06:34.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071995491999999994 2024-09-14T09:06:34.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618952+0000", 2024-09-14T09:06:34.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1347e-05 2024-09-14T09:06:34.287 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.287 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.287 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.287 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49390 3.5 3.9f124b25 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:34.288 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.547051+0000", 2024-09-14T09:06:34.288 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.687214812000001, 2024-09-14T09:06:34.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071963947, 2024-09-14T09:06:34.288 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.288 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.288 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:34.288 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.288 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.288 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.288 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49390 2024-09-14T09:06:34.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.288 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547051+0000", 2024-09-14T09:06:34.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547051+0000", 2024-09-14T09:06:34.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547053+0000", 2024-09-14T09:06:34.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.007e-06 2024-09-14T09:06:34.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547064+0000", 2024-09-14T09:06:34.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1146e-05 2024-09-14T09:06:34.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547089+0000", 2024-09-14T09:06:34.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.563e-05 2024-09-14T09:06:34.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.547127+0000", 2024-09-14T09:06:34.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8040000000000002e-05 2024-09-14T09:06:34.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619003+0000", 2024-09-14T09:06:34.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071875724000000002 2024-09-14T09:06:34.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.619014+0000", 2024-09-14T09:06:34.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1399999999999999e-05 2024-09-14T09:06:34.291 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.291 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.291 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.291 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:50478 3.1d 3:b8a2eaf4:::rbd_data.115e5dc498ac.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T09:06:34.291 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:34.292 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 22.184625541999999, 2024-09-14T09:06:34.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.075023792000000006, 2024-09-14T09:06:34.292 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.292 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.292 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:34.292 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.292 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.292 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.292 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 50478 2024-09-14T09:06:34.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.292 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:34.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:34.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049641+0000", 2024-09-14T09:06:34.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6270000000000001e-06 2024-09-14T09:06:34.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049649+0000", 2024-09-14T09:06:34.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6279999999999996e-06 2024-09-14T09:06:34.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049667+0000", 2024-09-14T09:06:34.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-09-14T09:06:34.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049677+0000", 2024-09-14T09:06:34.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0509000000000001e-05 2024-09-14T09:06:34.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049751+0000", 2024-09-14T09:06:34.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3322e-05 2024-09-14T09:06:34.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049830+0000", 2024-09-14T09:06:34.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9932000000000007e-05 2024-09-14T09:06:34.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:34.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.053203+0000", 2024-09-14T09:06:34.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003372398 2024-09-14T09:06:34.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:34.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124542+0000", 2024-09-14T09:06:34.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071339105999999999 2024-09-14T09:06:34.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:34.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124562+0000", 2024-09-14T09:06:34.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9649000000000001e-05 2024-09-14T09:06:34.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:34.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124616+0000", 2024-09-14T09:06:34.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4642999999999998e-05 2024-09-14T09:06:34.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124664+0000", 2024-09-14T09:06:34.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7342000000000002e-05 2024-09-14T09:06:34.297 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.297 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.297 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.297 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65035 3.1d 3:bb585d47:::rbd_data.115e21c31fcf.0000000000000034:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:06:34.297 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.068842+0000", 2024-09-14T09:06:34.297 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.165423307999999, 2024-09-14T09:06:34.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.094968451999999995, 2024-09-14T09:06:34.297 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.297 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.297 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:34.297 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.298 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.298 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.298 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65035 2024-09-14T09:06:34.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.298 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068842+0000", 2024-09-14T09:06:34.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068842+0000", 2024-09-14T09:06:34.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068843+0000", 2024-09-14T09:06:34.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.221e-06 2024-09-14T09:06:34.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068849+0000", 2024-09-14T09:06:34.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4659999999999998e-06 2024-09-14T09:06:34.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068861+0000", 2024-09-14T09:06:34.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2204e-05 2024-09-14T09:06:34.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068871+0000", 2024-09-14T09:06:34.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0318e-05 2024-09-14T09:06:34.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068942+0000", 2024-09-14T09:06:34.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0438999999999994e-05 2024-09-14T09:06:34.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.069005+0000", 2024-09-14T09:06:34.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3076000000000004e-05 2024-09-14T09:06:34.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:34.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.074957+0000", 2024-09-14T09:06:34.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059524549999999997 2024-09-14T09:06:34.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:34.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163479+0000", 2024-09-14T09:06:34.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.088521676999999993 2024-09-14T09:06:34.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:34.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163633+0000", 2024-09-14T09:06:34.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000154301 2024-09-14T09:06:34.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:34.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163733+0000", 2024-09-14T09:06:34.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100089 2024-09-14T09:06:34.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163811+0000", 2024-09-14T09:06:34.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7206000000000006e-05 2024-09-14T09:06:34.302 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.302 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.302 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.302 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:65035 3.1d e65/23)", 2024-09-14T09:06:34.302 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.087665+0000", 2024-09-14T09:06:34.303 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.146600398, 2024-09-14T09:06:34.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.076160555000000005, 2024-09-14T09:06:34.303 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.303 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.303 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:34.303 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087665+0000", 2024-09-14T09:06:34.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087665+0000", 2024-09-14T09:06:34.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087667+0000", 2024-09-14T09:06:34.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.818e-06 2024-09-14T09:06:34.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087673+0000", 2024-09-14T09:06:34.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9259999999999997e-06 2024-09-14T09:06:34.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087692+0000", 2024-09-14T09:06:34.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9715e-05 2024-09-14T09:06:34.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087703+0000", 2024-09-14T09:06:34.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.094e-05 2024-09-14T09:06:34.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163595+0000", 2024-09-14T09:06:34.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.075892121000000007 2024-09-14T09:06:34.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163621+0000", 2024-09-14T09:06:34.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5383000000000001e-05 2024-09-14T09:06:34.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163826+0000", 2024-09-14T09:06:34.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020465199999999999 2024-09-14T09:06:34.306 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.306 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.306 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.306 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:71031 3.1f 3:f9e41410:::rbd_data.115ebd4a9f36.0000000000000003:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:06:34.307 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:29.576817+0000", 2024-09-14T09:06:34.307 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.6574479210000002, 2024-09-14T09:06:34.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.087207509000000002, 2024-09-14T09:06:34.307 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.307 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.307 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:34.307 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.307 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.307 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.307 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 71031 2024-09-14T09:06:34.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.307 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.576817+0000", 2024-09-14T09:06:34.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.576817+0000", 2024-09-14T09:06:34.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.576820+0000", 2024-09-14T09:06:34.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.283e-06 2024-09-14T09:06:34.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.576832+0000", 2024-09-14T09:06:34.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1843e-05 2024-09-14T09:06:34.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.576861+0000", 2024-09-14T09:06:34.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9043e-05 2024-09-14T09:06:34.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.576876+0000", 2024-09-14T09:06:34.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5293000000000001e-05 2024-09-14T09:06:34.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.577000+0000", 2024-09-14T09:06:34.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000124473 2024-09-14T09:06:34.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.577126+0000", 2024-09-14T09:06:34.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012597199999999999 2024-09-14T09:06:34.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-09-14T09:06:34.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.585227+0000", 2024-09-14T09:06:34.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0081002810000000008 2024-09-14T09:06:34.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:34.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.663671+0000", 2024-09-14T09:06:34.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.078444609999999998 2024-09-14T09:06:34.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:34.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.663867+0000", 2024-09-14T09:06:34.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019541099999999999 2024-09-14T09:06:34.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:34.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.663960+0000", 2024-09-14T09:06:34.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3191999999999998e-05 2024-09-14T09:06:34.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.664025+0000", 2024-09-14T09:06:34.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5108000000000001e-05 2024-09-14T09:06:34.312 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.312 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.312 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.312 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:71067 3.1d 3:b9bb9b2f:::rbd_data.115ebd4a9f36.0000000000000006:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:06:34.312 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:29.920442+0000", 2024-09-14T09:06:34.312 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.3138233289999999, 2024-09-14T09:06:34.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13139427400000001, 2024-09-14T09:06:34.313 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.313 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.313 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:34.313 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.313 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.313 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.313 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 71067 2024-09-14T09:06:34.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.313 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920442+0000", 2024-09-14T09:06:34.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920442+0000", 2024-09-14T09:06:34.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920445+0000", 2024-09-14T09:06:34.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7099999999999999e-06 2024-09-14T09:06:34.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920461+0000", 2024-09-14T09:06:34.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6436e-05 2024-09-14T09:06:34.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920492+0000", 2024-09-14T09:06:34.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1069999999999999e-05 2024-09-14T09:06:34.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920508+0000", 2024-09-14T09:06:34.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5463000000000001e-05 2024-09-14T09:06:34.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920625+0000", 2024-09-14T09:06:34.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116932 2024-09-14T09:06:34.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920766+0000", 2024-09-14T09:06:34.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000141092 2024-09-14T09:06:34.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:34.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.929373+0000", 2024-09-14T09:06:34.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086074510000000003 2024-09-14T09:06:34.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:34.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:30.051654+0000", 2024-09-14T09:06:34.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.12228082999999999 2024-09-14T09:06:34.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:34.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:30.051711+0000", 2024-09-14T09:06:34.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7108000000000003e-05 2024-09-14T09:06:34.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:34.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:30.051779+0000", 2024-09-14T09:06:34.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7696999999999999e-05 2024-09-14T09:06:34.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:30.051836+0000", 2024-09-14T09:06:34.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7485000000000003e-05 2024-09-14T09:06:34.318 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.318 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.318 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.319 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:71067 3.1d e71/23 ondisk, result = 0)", 2024-09-14T09:06:34.319 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:29.966416+0000", 2024-09-14T09:06:34.319 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.267849655, 2024-09-14T09:06:34.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.085428365000000006, 2024-09-14T09:06:34.319 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.319 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.319 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:34.319 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.966416+0000", 2024-09-14T09:06:34.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.966416+0000", 2024-09-14T09:06:34.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.966417+0000", 2024-09-14T09:06:34.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.226e-06 2024-09-14T09:06:34.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.966423+0000", 2024-09-14T09:06:34.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7300000000000002e-06 2024-09-14T09:06:34.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.966443+0000", 2024-09-14T09:06:34.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.022e-05 2024-09-14T09:06:34.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.966453+0000", 2024-09-14T09:06:34.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0468e-05 2024-09-14T09:06:34.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:30.051691+0000", 2024-09-14T09:06:34.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.085237948999999993 2024-09-14T09:06:34.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:30.051704+0000", 2024-09-14T09:06:34.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2737e-05 2024-09-14T09:06:34.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:30.051844+0000", 2024-09-14T09:06:34.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000140035 2024-09-14T09:06:34.322 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.322 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.322 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.322 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.323 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.323 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:34.324 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:34.324 INFO:teuthology.orchestra.run.smithi176.stdout:3313 punch from 0x244c476 to 0x2452c8b, (0x6815 bytes) 2024-09-14T09:06:34.324 INFO:teuthology.orchestra.run.smithi176.stdout:3314 write 0x1f98718 thru 0x1fa4d9d (0xc686 bytes) 2024-09-14T09:06:34.325 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:06:34.325 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:34.327+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:34.326 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is unset 2024-09-14T09:06:34.425 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:34.427+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:34.496 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:34.496 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:34.496 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:34.496 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:34.510 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:06:34.552 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:34.555+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:34.627 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:34.627+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:34.670 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:34.670 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:34.670 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:34.670 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:34.670 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:34.685 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:06:34.727 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:34.727+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:34.804 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:34.804 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:34.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:34.805 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:34.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.805 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19520 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:34.805 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475212+0000", 2024-09-14T09:06:34.805 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.331269333999998, 2024-09-14T09:06:34.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067435885000000001, 2024-09-14T09:06:34.805 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.805 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.805 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:34.805 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.805 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.805 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.806 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19520 2024-09-14T09:06:34.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.806 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475212+0000", 2024-09-14T09:06:34.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475212+0000", 2024-09-14T09:06:34.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475213+0000", 2024-09-14T09:06:34.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0440000000000001e-06 2024-09-14T09:06:34.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475218+0000", 2024-09-14T09:06:34.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1930000000000001e-06 2024-09-14T09:06:34.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475237+0000", 2024-09-14T09:06:34.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9003000000000001e-05 2024-09-14T09:06:34.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475251+0000", 2024-09-14T09:06:34.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3720000000000001e-05 2024-09-14T09:06:34.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.541342+0000", 2024-09-14T09:06:34.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066091227000000002 2024-09-14T09:06:34.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.541492+0000", 2024-09-14T09:06:34.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000149221 2024-09-14T09:06:34.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542648+0000", 2024-09-14T09:06:34.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011564769999999999 2024-09-14T09:06:34.809 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.809 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.809 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.809 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19530 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:34.810 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475326+0000", 2024-09-14T09:06:34.810 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.331155150999997, 2024-09-14T09:06:34.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067030846000000005, 2024-09-14T09:06:34.810 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.810 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.810 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:34.810 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.810 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.810 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.810 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19530 2024-09-14T09:06:34.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.810 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475326+0000", 2024-09-14T09:06:34.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475326+0000", 2024-09-14T09:06:34.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475328+0000", 2024-09-14T09:06:34.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.133e-06 2024-09-14T09:06:34.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475358+0000", 2024-09-14T09:06:34.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0253e-05 2024-09-14T09:06:34.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475378+0000", 2024-09-14T09:06:34.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.033e-05 2024-09-14T09:06:34.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475405+0000", 2024-09-14T09:06:34.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6568999999999999e-05 2024-09-14T09:06:34.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.541989+0000", 2024-09-14T09:06:34.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066583956999999999 2024-09-14T09:06:34.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542215+0000", 2024-09-14T09:06:34.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022592200000000001 2024-09-14T09:06:34.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542357+0000", 2024-09-14T09:06:34.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014268199999999999 2024-09-14T09:06:34.814 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.814 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.814 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.814 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19531 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15073280~11919] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:34.814 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475425+0000", 2024-09-14T09:06:34.814 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.331056856000004, 2024-09-14T09:06:34.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067651759000000006, 2024-09-14T09:06:34.814 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.814 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.814 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:34.814 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.814 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.814 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.815 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19531 2024-09-14T09:06:34.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.815 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475425+0000", 2024-09-14T09:06:34.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475425+0000", 2024-09-14T09:06:34.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475426+0000", 2024-09-14T09:06:34.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2440000000000001e-06 2024-09-14T09:06:34.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475465+0000", 2024-09-14T09:06:34.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8643000000000003e-05 2024-09-14T09:06:34.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475485+0000", 2024-09-14T09:06:34.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0645999999999999e-05 2024-09-14T09:06:34.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475499+0000", 2024-09-14T09:06:34.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3312999999999999e-05 2024-09-14T09:06:34.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542389+0000", 2024-09-14T09:06:34.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066890885999999997 2024-09-14T09:06:34.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542481+0000", 2024-09-14T09:06:34.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1481e-05 2024-09-14T09:06:34.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543076+0000", 2024-09-14T09:06:34.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00059554599999999997 2024-09-14T09:06:34.818 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.818 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.818 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.818 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19532 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15335424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:34.818 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475521+0000", 2024-09-14T09:06:34.818 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.330960210000001, 2024-09-14T09:06:34.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067895159999999996, 2024-09-14T09:06:34.818 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.818 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.818 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:34.818 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.819 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.819 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.819 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19532 2024-09-14T09:06:34.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.819 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475521+0000", 2024-09-14T09:06:34.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475521+0000", 2024-09-14T09:06:34.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475523+0000", 2024-09-14T09:06:34.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.277e-06 2024-09-14T09:06:34.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475528+0000", 2024-09-14T09:06:34.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8629999999999996e-06 2024-09-14T09:06:34.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475546+0000", 2024-09-14T09:06:34.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7750000000000001e-05 2024-09-14T09:06:34.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475560+0000", 2024-09-14T09:06:34.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3763e-05 2024-09-14T09:06:34.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543144+0000", 2024-09-14T09:06:34.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067584489999999997 2024-09-14T09:06:34.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543261+0000", 2024-09-14T09:06:34.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011696699999999999 2024-09-14T09:06:34.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543416+0000", 2024-09-14T09:06:34.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015505 2024-09-14T09:06:34.822 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.822 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.822 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.823 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19533 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:34.823 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475578+0000", 2024-09-14T09:06:34.823 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.330903067000001, 2024-09-14T09:06:34.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068124984999999999, 2024-09-14T09:06:34.823 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.823 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.823 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:34.823 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.823 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.823 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.823 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19533 2024-09-14T09:06:34.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.823 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475578+0000", 2024-09-14T09:06:34.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475578+0000", 2024-09-14T09:06:34.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475580+0000", 2024-09-14T09:06:34.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.08e-06 2024-09-14T09:06:34.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475585+0000", 2024-09-14T09:06:34.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4970000000000001e-06 2024-09-14T09:06:34.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475602+0000", 2024-09-14T09:06:34.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7459999999999999e-05 2024-09-14T09:06:34.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475615+0000", 2024-09-14T09:06:34.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2339e-05 2024-09-14T09:06:34.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543466+0000", 2024-09-14T09:06:34.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067850745000000004 2024-09-14T09:06:34.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543577+0000", 2024-09-14T09:06:34.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000111908 2024-09-14T09:06:34.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543703+0000", 2024-09-14T09:06:34.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012595600000000001 2024-09-14T09:06:34.827 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.827 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.827 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.827 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19534 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:34.827 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475634+0000", 2024-09-14T09:06:34.827 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.330847290999998, 2024-09-14T09:06:34.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068358706000000005, 2024-09-14T09:06:34.827 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.827 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.827 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:34.827 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.827 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.828 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.828 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19534 2024-09-14T09:06:34.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.828 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475634+0000", 2024-09-14T09:06:34.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475634+0000", 2024-09-14T09:06:34.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475635+0000", 2024-09-14T09:06:34.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.187e-06 2024-09-14T09:06:34.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475641+0000", 2024-09-14T09:06:34.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5629999999999999e-06 2024-09-14T09:06:34.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475659+0000", 2024-09-14T09:06:34.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7649999999999999e-05 2024-09-14T09:06:34.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475671+0000", 2024-09-14T09:06:34.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2583e-05 2024-09-14T09:06:34.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543748+0000", 2024-09-14T09:06:34.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068076740999999996 2024-09-14T09:06:34.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543855+0000", 2024-09-14T09:06:34.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010676800000000001 2024-09-14T09:06:34.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543993+0000", 2024-09-14T09:06:34.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138214 2024-09-14T09:06:34.831 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.831 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.831 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.831 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19535 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:34.831 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475691+0000", 2024-09-14T09:06:34.831 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.330790280999999, 2024-09-14T09:06:34.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068607583999999999, 2024-09-14T09:06:34.832 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.832 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.832 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:34.832 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.832 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.832 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.832 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19535 2024-09-14T09:06:34.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.832 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475691+0000", 2024-09-14T09:06:34.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475691+0000", 2024-09-14T09:06:34.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475693+0000", 2024-09-14T09:06:34.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6270000000000001e-06 2024-09-14T09:06:34.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475699+0000", 2024-09-14T09:06:34.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9059999999999999e-06 2024-09-14T09:06:34.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475717+0000", 2024-09-14T09:06:34.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-09-14T09:06:34.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475730+0000", 2024-09-14T09:06:34.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.308e-05 2024-09-14T09:06:34.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544042+0000", 2024-09-14T09:06:34.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068312791999999997 2024-09-14T09:06:34.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544168+0000", 2024-09-14T09:06:34.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012549699999999999 2024-09-14T09:06:34.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544299+0000", 2024-09-14T09:06:34.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013094199999999999 2024-09-14T09:06:34.835 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.835 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.836 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.836 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19536 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:34.836 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475749+0000", 2024-09-14T09:06:34.836 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.330733002000002, 2024-09-14T09:06:34.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068917143, 2024-09-14T09:06:34.836 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.836 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.836 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:34.836 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.837 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.837 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.837 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19536 2024-09-14T09:06:34.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.837 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475749+0000", 2024-09-14T09:06:34.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475749+0000", 2024-09-14T09:06:34.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475750+0000", 2024-09-14T09:06:34.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0330000000000001e-06 2024-09-14T09:06:34.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475755+0000", 2024-09-14T09:06:34.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9740000000000001e-06 2024-09-14T09:06:34.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475772+0000", 2024-09-14T09:06:34.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7750000000000001e-05 2024-09-14T09:06:34.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475802+0000", 2024-09-14T09:06:34.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0190000000000001e-05 2024-09-14T09:06:34.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544345+0000", 2024-09-14T09:06:34.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068542140000000001 2024-09-14T09:06:34.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544454+0000", 2024-09-14T09:06:34.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109447 2024-09-14T09:06:34.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544666+0000", 2024-09-14T09:06:34.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000211609 2024-09-14T09:06:34.840 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.840 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.840 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.840 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19537 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:34.841 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475822+0000", 2024-09-14T09:06:34.841 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.330659044999997, 2024-09-14T09:06:34.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069185398999999995, 2024-09-14T09:06:34.841 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.841 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.841 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:34.841 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.841 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.841 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.841 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19537 2024-09-14T09:06:34.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.841 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475822+0000", 2024-09-14T09:06:34.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475822+0000", 2024-09-14T09:06:34.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475824+0000", 2024-09-14T09:06:34.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2559999999999999e-06 2024-09-14T09:06:34.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475829+0000", 2024-09-14T09:06:34.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6840000000000001e-06 2024-09-14T09:06:34.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475848+0000", 2024-09-14T09:06:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8583000000000001e-05 2024-09-14T09:06:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475861+0000", 2024-09-14T09:06:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2802999999999999e-05 2024-09-14T09:06:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544716+0000", 2024-09-14T09:06:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068855036999999994 2024-09-14T09:06:34.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544827+0000", 2024-09-14T09:06:34.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011113200000000001 2024-09-14T09:06:34.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545008+0000", 2024-09-14T09:06:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018090399999999999 2024-09-14T09:06:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19538 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475880+0000", 2024-09-14T09:06:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.330601369999997, 2024-09-14T09:06:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069740072, 2024-09-14T09:06:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19538 2024-09-14T09:06:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475880+0000", 2024-09-14T09:06:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475880+0000", 2024-09-14T09:06:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475881+0000", 2024-09-14T09:06:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.124e-06 2024-09-14T09:06:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475886+0000", 2024-09-14T09:06:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0130000000000001e-06 2024-09-14T09:06:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475904+0000", 2024-09-14T09:06:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7779999999999999e-05 2024-09-14T09:06:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475918+0000", 2024-09-14T09:06:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.385e-05 2024-09-14T09:06:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545065+0000", 2024-09-14T09:06:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069147307000000005 2024-09-14T09:06:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545431+0000", 2024-09-14T09:06:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036538400000000001 2024-09-14T09:06:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545620+0000", 2024-09-14T09:06:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000189614 2024-09-14T09:06:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19539 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475936+0000", 2024-09-14T09:06:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.330545786000002, 2024-09-14T09:06:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069860063999999999, 2024-09-14T09:06:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19539 2024-09-14T09:06:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475936+0000", 2024-09-14T09:06:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475936+0000", 2024-09-14T09:06:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475937+0000", 2024-09-14T09:06:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5960000000000001e-06 2024-09-14T09:06:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475943+0000", 2024-09-14T09:06:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2340000000000002e-06 2024-09-14T09:06:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475960+0000", 2024-09-14T09:06:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7592999999999999e-05 2024-09-14T09:06:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475986+0000", 2024-09-14T09:06:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6319e-05 2024-09-14T09:06:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545650+0000", 2024-09-14T09:06:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069663613999999999 2024-09-14T09:06:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545714+0000", 2024-09-14T09:06:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4209999999999997e-05 2024-09-14T09:06:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545796+0000", 2024-09-14T09:06:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1497999999999998e-05 2024-09-14T09:06:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19541 3.10 3:0ae7d8d9:::rbd_data.115e4c71d6e8.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 4194304~8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.497517+0000", 2024-09-14T09:06:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.308964715000002, 2024-09-14T09:06:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069874055000000004, 2024-09-14T09:06:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19541 2024-09-14T09:06:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.497517+0000", 2024-09-14T09:06:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.497517+0000", 2024-09-14T09:06:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.497520+0000", 2024-09-14T09:06:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0730000000000001e-06 2024-09-14T09:06:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.504875+0000", 2024-09-14T09:06:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007355136 2024-09-14T09:06:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.504926+0000", 2024-09-14T09:06:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1146000000000001e-05 2024-09-14T09:06:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.504950+0000", 2024-09-14T09:06:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3638999999999998e-05 2024-09-14T09:06:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.505139+0000", 2024-09-14T09:06:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018904300000000001 2024-09-14T09:06:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.505402+0000", 2024-09-14T09:06:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026354699999999999 2024-09-14T09:06:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.506208+0000", 2024-09-14T09:06:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00080583399999999996 2024-09-14T09:06:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.566904+0000", 2024-09-14T09:06:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.060695510000000001 2024-09-14T09:06:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.567219+0000", 2024-09-14T09:06:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031509000000000002 2024-09-14T09:06:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.567319+0000", 2024-09-14T09:06:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9804999999999997e-05 2024-09-14T09:06:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.567391+0000", 2024-09-14T09:06:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2231999999999996e-05 2024-09-14T09:06:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:30890 3.10 3:0ac3ecfb:::rbd_data.115ebc3c67d1.0000000000000004:c [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.069693+0000", 2024-09-14T09:06:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.736788926000003, 2024-09-14T09:06:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066892069999999998, 2024-09-14T09:06:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 30890 2024-09-14T09:06:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069693+0000", 2024-09-14T09:06:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069693+0000", 2024-09-14T09:06:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069694+0000", 2024-09-14T09:06:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0100000000000003e-07 2024-09-14T09:06:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069698+0000", 2024-09-14T09:06:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2100000000000003e-06 2024-09-14T09:06:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069711+0000", 2024-09-14T09:06:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3332e-05 2024-09-14T09:06:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069721+0000", 2024-09-14T09:06:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0424999999999999e-05 2024-09-14T09:06:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136469+0000", 2024-09-14T09:06:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066747714999999999 2024-09-14T09:06:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136540+0000", 2024-09-14T09:06:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1077999999999999e-05 2024-09-14T09:06:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136585+0000", 2024-09-14T09:06:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4409000000000001e-05 2024-09-14T09:06:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:30891 3.10 3:0ac3ecfb:::rbd_data.115ebc3c67d1.0000000000000004:c [read 212992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.069740+0000", 2024-09-14T09:06:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.736741926000001, 2024-09-14T09:06:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066940186999999998, 2024-09-14T09:06:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 30891 2024-09-14T09:06:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069740+0000", 2024-09-14T09:06:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069740+0000", 2024-09-14T09:06:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070603+0000", 2024-09-14T09:06:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00086369599999999995 2024-09-14T09:06:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070612+0000", 2024-09-14T09:06:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3170000000000002e-06 2024-09-14T09:06:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070631+0000", 2024-09-14T09:06:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8936999999999998e-05 2024-09-14T09:06:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070645+0000", 2024-09-14T09:06:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4545999999999999e-05 2024-09-14T09:06:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136603+0000", 2024-09-14T09:06:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.065957551000000003 2024-09-14T09:06:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136644+0000", 2024-09-14T09:06:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1671000000000001e-05 2024-09-14T09:06:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136680+0000", 2024-09-14T09:06:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5469000000000002e-05 2024-09-14T09:06:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:37542 3.10 3:0e658405:::rbd_data.115eb07a8abd.000000000000000d:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.443185+0000", 2024-09-14T09:06:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.363296900000002, 2024-09-14T09:06:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.091143627000000005, 2024-09-14T09:06:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37542 2024-09-14T09:06:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443185+0000", 2024-09-14T09:06:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443185+0000", 2024-09-14T09:06:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443187+0000", 2024-09-14T09:06:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.367e-06 2024-09-14T09:06:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443199+0000", 2024-09-14T09:06:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2153e-05 2024-09-14T09:06:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443225+0000", 2024-09-14T09:06:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5457e-05 2024-09-14T09:06:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443248+0000", 2024-09-14T09:06:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2963e-05 2024-09-14T09:06:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443400+0000", 2024-09-14T09:06:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015241199999999999 2024-09-14T09:06:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443542+0000", 2024-09-14T09:06:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014224600000000001 2024-09-14T09:06:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.448965+0000", 2024-09-14T09:06:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054226839999999997 2024-09-14T09:06:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534136+0000", 2024-09-14T09:06:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.085170939000000001 2024-09-14T09:06:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534164+0000", 2024-09-14T09:06:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7863000000000001e-05 2024-09-14T09:06:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534250+0000", 2024-09-14T09:06:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5876000000000002e-05 2024-09-14T09:06:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534328+0000", 2024-09-14T09:06:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8666999999999997e-05 2024-09-14T09:06:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:37542 3.10 e48/28 ondisk, result = 0)", 2024-09-14T09:06:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.464008+0000", 2024-09-14T09:06:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.342473491, 2024-09-14T09:06:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.070141355000000002, 2024-09-14T09:06:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464008+0000", 2024-09-14T09:06:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464008+0000", 2024-09-14T09:06:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464010+0000", 2024-09-14T09:06:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.547e-06 2024-09-14T09:06:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464017+0000", 2024-09-14T09:06:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4930000000000001e-06 2024-09-14T09:06:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464047+0000", 2024-09-14T09:06:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0090000000000002e-05 2024-09-14T09:06:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464061+0000", 2024-09-14T09:06:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4003e-05 2024-09-14T09:06:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534107+0000", 2024-09-14T09:06:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.070045590000000005 2024-09-14T09:06:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534126+0000", 2024-09-14T09:06:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8893e-05 2024-09-14T09:06:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534149+0000", 2024-09-14T09:06:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3739000000000001e-05 2024-09-14T09:06:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38003 3.10 3:0e658405:::rbd_data.115eb07a8abd.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:57.392070+0000", 2024-09-14T09:06:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.414411717, 2024-09-14T09:06:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11697451, 2024-09-14T09:06:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38003 2024-09-14T09:06:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392070+0000", 2024-09-14T09:06:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392070+0000", 2024-09-14T09:06:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392072+0000", 2024-09-14T09:06:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6529999999999998e-06 2024-09-14T09:06:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392086+0000", 2024-09-14T09:06:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.332e-05 2024-09-14T09:06:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392139+0000", 2024-09-14T09:06:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2815999999999997e-05 2024-09-14T09:06:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392155+0000", 2024-09-14T09:06:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6796999999999999e-05 2024-09-14T09:06:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392296+0000", 2024-09-14T09:06:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014107900000000001 2024-09-14T09:06:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392443+0000", 2024-09-14T09:06:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014654499999999999 2024-09-14T09:06:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.399418+0000", 2024-09-14T09:06:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069751049999999997 2024-09-14T09:06:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508607+0000", 2024-09-14T09:06:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10918908400000001 2024-09-14T09:06:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508748+0000", 2024-09-14T09:06:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014079699999999999 2024-09-14T09:06:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508921+0000", 2024-09-14T09:06:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017345899999999999 2024-09-14T09:06:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.509044+0000", 2024-09-14T09:06:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000122855 2024-09-14T09:06:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:38003 3.10 e48/28 ondisk, result = 0)", 2024-09-14T09:06:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:57.413977+0000", 2024-09-14T09:06:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.392504494000001, 2024-09-14T09:06:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.095088000000000006, 2024-09-14T09:06:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413977+0000", 2024-09-14T09:06:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413977+0000", 2024-09-14T09:06:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413979+0000", 2024-09-14T09:06:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.933e-06 2024-09-14T09:06:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413988+0000", 2024-09-14T09:06:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2769999999999994e-06 2024-09-14T09:06:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.414021+0000", 2024-09-14T09:06:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2589999999999998e-05 2024-09-14T09:06:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.414040+0000", 2024-09-14T09:06:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8796000000000001e-05 2024-09-14T09:06:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508701+0000", 2024-09-14T09:06:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.094661067000000002 2024-09-14T09:06:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508729+0000", 2024-09-14T09:06:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8677e-05 2024-09-14T09:06:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.509065+0000", 2024-09-14T09:06:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033565999999999999 2024-09-14T09:06:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65139 3.a 3:54c90b7d:::rbd_data.115e21c31fcf.0000000000000008:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:06:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:06:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.426768790000001, 2024-09-14T09:06:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14429895700000001, 2024-09-14T09:06:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65139 2024-09-14T09:06:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:06:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:06:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379715+0000", 2024-09-14T09:06:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.576e-06 2024-09-14T09:06:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379724+0000", 2024-09-14T09:06:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1779999999999992e-06 2024-09-14T09:06:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379746+0000", 2024-09-14T09:06:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1661000000000001e-05 2024-09-14T09:06:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379759+0000", 2024-09-14T09:06:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.324e-05 2024-09-14T09:06:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379882+0000", 2024-09-14T09:06:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012239400000000001 2024-09-14T09:06:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.380023+0000", 2024-09-14T09:06:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000141402 2024-09-14T09:06:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.391553+0000", 2024-09-14T09:06:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011530197000000001 2024-09-14T09:06:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523927+0000", 2024-09-14T09:06:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.132373935 2024-09-14T09:06:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523941+0000", 2024-09-14T09:06:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3326000000000001e-05 2024-09-14T09:06:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523977+0000", 2024-09-14T09:06:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6185e-05 2024-09-14T09:06:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.524012+0000", 2024-09-14T09:06:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4863000000000001e-05 2024-09-14T09:06:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:65139 3.a e65/34 ondisk, result = 0)", 2024-09-14T09:06:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.408501+0000", 2024-09-14T09:06:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.397980233, 2024-09-14T09:06:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.115431275, 2024-09-14T09:06:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408501+0000", 2024-09-14T09:06:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408501+0000", 2024-09-14T09:06:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408504+0000", 2024-09-14T09:06:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2819999999999999e-06 2024-09-14T09:06:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408512+0000", 2024-09-14T09:06:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7530000000000007e-06 2024-09-14T09:06:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408534+0000", 2024-09-14T09:06:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1993000000000001e-05 2024-09-14T09:06:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408550+0000", 2024-09-14T09:06:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5988e-05 2024-09-14T09:06:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523914+0000", 2024-09-14T09:06:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.115363542 2024-09-14T09:06:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523922+0000", 2024-09-14T09:06:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5529999999999993e-06 2024-09-14T09:06:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523933+0000", 2024-09-14T09:06:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0164e-05 2024-09-14T09:06:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.900 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.900 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:34.900 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:34.900 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:34.901 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:06:34.902 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:34.903+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:35.002 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:35.003+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:35.070 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:35.070 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:35.070 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:35.071 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:35.085 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:06:35.103 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:35.103+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:35.204 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:35.207+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:35.209 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:35.209 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:35.209 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:35.209 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:35.209 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:35.224 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:06:35.308 INFO:teuthology.orchestra.run.smithi176.stdout:3318 trunc from 0x8530c01 to 0x8786ea2 2024-09-14T09:06:35.315 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:35.319+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:35.320 INFO:teuthology.orchestra.run.smithi176.stdout:3321 read 0x1f6bc56 thru 0x1f6decb (0x2276 bytes) 2024-09-14T09:06:35.337 INFO:teuthology.orchestra.run.smithi176.stdout:3323 trunc from 0x8786ea2 to 0xe05cc02 2024-09-14T09:06:35.351 INFO:teuthology.orchestra.run.smithi176.stdout:3324 write 0x7218933 thru 0x7220a1f (0x80ed bytes) 2024-09-14T09:06:35.389 INFO:teuthology.orchestra.run.smithi176.stdout:3325 trunc from 0xe05cc02 to 0x282e993 2024-09-14T09:06:35.403 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:19152 3.9 e39/34 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head v 39'12, mlcod=39'9)", 2024-09-14T09:06:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.333629+0000", 2024-09-14T09:06:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.071497205, 2024-09-14T09:06:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.1194095, 2024-09-14T09:06:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.333629+0000", 2024-09-14T09:06:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.333629+0000", 2024-09-14T09:06:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.333631+0000", 2024-09-14T09:06:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.007e-06 2024-09-14T09:06:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.350796+0000", 2024-09-14T09:06:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017164690999999999 2024-09-14T09:06:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.350865+0000", 2024-09-14T09:06:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8989e-05 2024-09-14T09:06:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.350884+0000", 2024-09-14T09:06:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9323000000000001e-05 2024-09-14T09:06:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.351045+0000", 2024-09-14T09:06:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016102900000000001 2024-09-14T09:06:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.351087+0000", 2024-09-14T09:06:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1430000000000001e-05 2024-09-14T09:06:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.452865+0000", 2024-09-14T09:06:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10177839900000001 2024-09-14T09:06:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.453039+0000", 2024-09-14T09:06:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017363200000000001 2024-09-14T09:06:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:37543 3.1b 3:d8e0f8d4:::rbd_data.115eb07a8abd.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.535280+0000", 2024-09-14T09:06:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.869846197000001, 2024-09-14T09:06:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.12853271799999999, 2024-09-14T09:06:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37543 2024-09-14T09:06:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535280+0000", 2024-09-14T09:06:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535280+0000", 2024-09-14T09:06:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535284+0000", 2024-09-14T09:06:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6770000000000001e-06 2024-09-14T09:06:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535303+0000", 2024-09-14T09:06:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9553000000000001e-05 2024-09-14T09:06:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535349+0000", 2024-09-14T09:06:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5833e-05 2024-09-14T09:06:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535368+0000", 2024-09-14T09:06:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8542999999999998e-05 2024-09-14T09:06:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535548+0000", 2024-09-14T09:06:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000180688 2024-09-14T09:06:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535814+0000", 2024-09-14T09:06:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026614100000000001 2024-09-14T09:06:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.543779+0000", 2024-09-14T09:06:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0079644720000000002 2024-09-14T09:06:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663456+0000", 2024-09-14T09:06:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.119676694 2024-09-14T09:06:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663626+0000", 2024-09-14T09:06:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170559 2024-09-14T09:06:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663728+0000", 2024-09-14T09:06:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101889 2024-09-14T09:06:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663813+0000", 2024-09-14T09:06:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4668999999999999e-05 2024-09-14T09:06:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:37543 3.1b e48/34 ondisk, result = 0)", 2024-09-14T09:06:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.566755+0000", 2024-09-14T09:06:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.838371659000003, 2024-09-14T09:06:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.096718023, 2024-09-14T09:06:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566755+0000", 2024-09-14T09:06:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566755+0000", 2024-09-14T09:06:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566757+0000", 2024-09-14T09:06:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9800000000000001e-06 2024-09-14T09:06:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566767+0000", 2024-09-14T09:06:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0036e-05 2024-09-14T09:06:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566797+0000", 2024-09-14T09:06:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.012e-05 2024-09-14T09:06:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566813+0000", 2024-09-14T09:06:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6575999999999999e-05 2024-09-14T09:06:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663408+0000", 2024-09-14T09:06:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.096594901999999996 2024-09-14T09:06:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663426+0000", 2024-09-14T09:06:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7765999999999999e-05 2024-09-14T09:06:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663473+0000", 2024-09-14T09:06:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6643e-05 2024-09-14T09:06:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38593 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [watch reconnect cookie 139718844451680 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:06:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:06:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.586708790000003, 2024-09-14T09:06:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0218296630000001, 2024-09-14T09:06:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38593 2024-09-14T09:06:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:06:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:06:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818420+0000", 2024-09-14T09:06:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8399999999999999e-06 2024-09-14T09:06:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818431+0000", 2024-09-14T09:06:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0859999999999999e-05 2024-09-14T09:06:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818466+0000", 2024-09-14T09:06:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4486999999999997e-05 2024-09-14T09:06:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818481+0000", 2024-09-14T09:06:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5362999999999998e-05 2024-09-14T09:06:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818659+0000", 2024-09-14T09:06:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017807500000000001 2024-09-14T09:06:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op must wait for map", 2024-09-14T09:06:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818681+0000", 2024-09-14T09:06:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2096999999999999e-05 2024-09-14T09:06:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825472+0000", 2024-09-14T09:06:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067905170000000003 2024-09-14T09:06:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:06:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825491+0000", 2024-09-14T09:06:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9372999999999999e-05 2024-09-14T09:06:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835402+0000", 2024-09-14T09:06:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0099111620000001 2024-09-14T09:06:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835548+0000", 2024-09-14T09:06:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014607300000000001 2024-09-14T09:06:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.840247+0000", 2024-09-14T09:06:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046988159999999998 2024-09-14T09:06:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38615 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:06:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:06:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.538343582000003, 2024-09-14T09:06:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.97014128899999996, 2024-09-14T09:06:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38615 2024-09-14T09:06:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:06:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:06:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866787+0000", 2024-09-14T09:06:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.476e-06 2024-09-14T09:06:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866806+0000", 2024-09-14T09:06:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8606000000000001e-05 2024-09-14T09:06:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866839+0000", 2024-09-14T09:06:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3510000000000003e-05 2024-09-14T09:06:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866867+0000", 2024-09-14T09:06:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7610000000000002e-05 2024-09-14T09:06:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867031+0000", 2024-09-14T09:06:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016414199999999999 2024-09-14T09:06:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:06:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867054+0000", 2024-09-14T09:06:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2693000000000001e-05 2024-09-14T09:06:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836462+0000", 2024-09-14T09:06:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.96940797700000003 2024-09-14T09:06:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836547+0000", 2024-09-14T09:06:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5294e-05 2024-09-14T09:06:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836924+0000", 2024-09-14T09:06:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037698099999999997 2024-09-14T09:06:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49230 3.d 3:b4a67b64:::rbd_data.115e51d92ed2.0000000000000003:12 [read 2621440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.523859+0000", 2024-09-14T09:06:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.881266991, 2024-09-14T09:06:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067232675000000006, 2024-09-14T09:06:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49230 2024-09-14T09:06:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523859+0000", 2024-09-14T09:06:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523859+0000", 2024-09-14T09:06:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523860+0000", 2024-09-14T09:06:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0130000000000001e-06 2024-09-14T09:06:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523865+0000", 2024-09-14T09:06:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4000000000000002e-06 2024-09-14T09:06:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523880+0000", 2024-09-14T09:06:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5287000000000001e-05 2024-09-14T09:06:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523892+0000", 2024-09-14T09:06:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2006e-05 2024-09-14T09:06:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.590817+0000", 2024-09-14T09:06:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066924692999999993 2024-09-14T09:06:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.590937+0000", 2024-09-14T09:06:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012081700000000001 2024-09-14T09:06:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591092+0000", 2024-09-14T09:06:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015445899999999999 2024-09-14T09:06:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49231 3.d 3:b4a67b64:::rbd_data.115e51d92ed2.0000000000000003:12 [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.523908+0000", 2024-09-14T09:06:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.881218531999998, 2024-09-14T09:06:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067373731000000006, 2024-09-14T09:06:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49231 2024-09-14T09:06:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523908+0000", 2024-09-14T09:06:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523908+0000", 2024-09-14T09:06:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523909+0000", 2024-09-14T09:06:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9699999999999994e-07 2024-09-14T09:06:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523913+0000", 2024-09-14T09:06:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6070000000000001e-06 2024-09-14T09:06:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523929+0000", 2024-09-14T09:06:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5415999999999999e-05 2024-09-14T09:06:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523940+0000", 2024-09-14T09:06:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1707e-05 2024-09-14T09:06:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591167+0000", 2024-09-14T09:06:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067226576999999996 2024-09-14T09:06:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591281+0000", 2024-09-14T09:06:35.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011442699999999999 2024-09-14T09:06:35.438 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.438 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.438 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.438 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49232 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:35.438 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.523956+0000", 2024-09-14T09:06:35.438 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.881170198, 2024-09-14T09:06:35.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067398703000000004, 2024-09-14T09:06:35.438 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.439 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.439 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:35.439 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:35.439 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:35.439 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:35.439 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49232 2024-09-14T09:06:35.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.439 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.439 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523956+0000", 2024-09-14T09:06:35.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523956+0000", 2024-09-14T09:06:35.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523957+0000", 2024-09-14T09:06:35.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8299999999999995e-07 2024-09-14T09:06:35.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523961+0000", 2024-09-14T09:06:35.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3329999999999998e-06 2024-09-14T09:06:35.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523977+0000", 2024-09-14T09:06:35.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.577e-05 2024-09-14T09:06:35.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523989+0000", 2024-09-14T09:06:35.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2089999999999999e-05 2024-09-14T09:06:35.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591340+0000", 2024-09-14T09:06:35.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067350278999999999 2024-09-14T09:06:35.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591355+0000", 2024-09-14T09:06:35.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5248e-05 2024-09-14T09:06:35.442 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.442 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.442 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.442 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49233 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:35.442 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.524004+0000", 2024-09-14T09:06:35.443 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.881122289, 2024-09-14T09:06:35.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067426573000000004, 2024-09-14T09:06:35.443 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.443 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.443 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:35.443 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:35.443 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:35.443 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:35.443 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49233 2024-09-14T09:06:35.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.443 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524004+0000", 2024-09-14T09:06:35.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524004+0000", 2024-09-14T09:06:35.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524005+0000", 2024-09-14T09:06:35.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3699999999999999e-07 2024-09-14T09:06:35.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524009+0000", 2024-09-14T09:06:35.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3359999999999997e-06 2024-09-14T09:06:35.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524026+0000", 2024-09-14T09:06:35.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6753e-05 2024-09-14T09:06:35.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524040+0000", 2024-09-14T09:06:35.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3694e-05 2024-09-14T09:06:35.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591411+0000", 2024-09-14T09:06:35.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067371767999999999 2024-09-14T09:06:35.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591431+0000", 2024-09-14T09:06:35.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9085000000000001e-05 2024-09-14T09:06:35.446 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.446 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.447 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.447 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49257 3.d 3.bb54482d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:35.447 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.524584+0000", 2024-09-14T09:06:35.447 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.880542386999998, 2024-09-14T09:06:35.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067046887999999999, 2024-09-14T09:06:35.447 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.447 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.447 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:35.447 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:35.447 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:35.447 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:35.447 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49257 2024-09-14T09:06:35.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.448 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524584+0000", 2024-09-14T09:06:35.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524584+0000", 2024-09-14T09:06:35.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.449 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524585+0000", 2024-09-14T09:06:35.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1699999999999997e-07 2024-09-14T09:06:35.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.449 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524609+0000", 2024-09-14T09:06:35.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4448999999999999e-05 2024-09-14T09:06:35.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.449 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524627+0000", 2024-09-14T09:06:35.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7543000000000001e-05 2024-09-14T09:06:35.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524634+0000", 2024-09-14T09:06:35.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7540000000000006e-06 2024-09-14T09:06:35.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591622+0000", 2024-09-14T09:06:35.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066987668 2024-09-14T09:06:35.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591631+0000", 2024-09-14T09:06:35.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7569999999999993e-06 2024-09-14T09:06:35.451 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.451 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.451 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.451 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49310 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:35.451 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525200+0000", 2024-09-14T09:06:35.451 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.879926092000002, 2024-09-14T09:06:35.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067082507, 2024-09-14T09:06:35.451 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.451 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.452 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:35.452 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:35.452 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:35.452 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:35.452 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49310 2024-09-14T09:06:35.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.452 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525200+0000", 2024-09-14T09:06:35.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525200+0000", 2024-09-14T09:06:35.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525201+0000", 2024-09-14T09:06:35.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.023e-06 2024-09-14T09:06:35.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525206+0000", 2024-09-14T09:06:35.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5000000000000001e-06 2024-09-14T09:06:35.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525242+0000", 2024-09-14T09:06:35.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6672999999999999e-05 2024-09-14T09:06:35.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525256+0000", 2024-09-14T09:06:35.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3360000000000001e-05 2024-09-14T09:06:35.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592266+0000", 2024-09-14T09:06:35.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067009868 2024-09-14T09:06:35.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592283+0000", 2024-09-14T09:06:35.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7082999999999999e-05 2024-09-14T09:06:35.455 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.455 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.455 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.455 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49311 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:35.455 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525275+0000", 2024-09-14T09:06:35.456 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.879851316, 2024-09-14T09:06:35.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067067112999999998, 2024-09-14T09:06:35.456 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.456 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.456 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:35.456 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:35.456 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:35.456 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:35.456 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49311 2024-09-14T09:06:35.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.456 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.456 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525275+0000", 2024-09-14T09:06:35.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525275+0000", 2024-09-14T09:06:35.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525276+0000", 2024-09-14T09:06:35.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.313e-06 2024-09-14T09:06:35.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525282+0000", 2024-09-14T09:06:35.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.127e-06 2024-09-14T09:06:35.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525299+0000", 2024-09-14T09:06:35.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6903000000000001e-05 2024-09-14T09:06:35.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525312+0000", 2024-09-14T09:06:35.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3193999999999999e-05 2024-09-14T09:06:35.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592323+0000", 2024-09-14T09:06:35.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067010504999999998 2024-09-14T09:06:35.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592342+0000", 2024-09-14T09:06:35.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9071000000000001e-05 2024-09-14T09:06:35.459 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.459 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.459 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.459 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49312 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:35.459 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525330+0000", 2024-09-14T09:06:35.459 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.879795923, 2024-09-14T09:06:35.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.06706268, 2024-09-14T09:06:35.460 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.460 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.460 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:35.460 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:35.460 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:35.460 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:35.460 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49312 2024-09-14T09:06:35.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.460 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525330+0000", 2024-09-14T09:06:35.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525330+0000", 2024-09-14T09:06:35.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525332+0000", 2024-09-14T09:06:35.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.187e-06 2024-09-14T09:06:35.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525350+0000", 2024-09-14T09:06:35.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8833000000000001e-05 2024-09-14T09:06:35.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525385+0000", 2024-09-14T09:06:35.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4686e-05 2024-09-14T09:06:35.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525399+0000", 2024-09-14T09:06:35.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4239999999999999e-05 2024-09-14T09:06:35.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592377+0000", 2024-09-14T09:06:35.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066977255999999999 2024-09-14T09:06:35.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592393+0000", 2024-09-14T09:06:35.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6478e-05 2024-09-14T09:06:35.463 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.463 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.463 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.463 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49313 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:35.463 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525423+0000", 2024-09-14T09:06:35.463 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.879703524, 2024-09-14T09:06:35.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067039319, 2024-09-14T09:06:35.464 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.464 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.464 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:35.464 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:35.464 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:35.464 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:35.464 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49313 2024-09-14T09:06:35.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.464 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525423+0000", 2024-09-14T09:06:35.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525423+0000", 2024-09-14T09:06:35.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525424+0000", 2024-09-14T09:06:35.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.257e-06 2024-09-14T09:06:35.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525430+0000", 2024-09-14T09:06:35.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8699999999999997e-06 2024-09-14T09:06:35.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525447+0000", 2024-09-14T09:06:35.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7560000000000001e-05 2024-09-14T09:06:35.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525460+0000", 2024-09-14T09:06:35.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2243e-05 2024-09-14T09:06:35.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592444+0000", 2024-09-14T09:06:35.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066984118999999995 2024-09-14T09:06:35.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592462+0000", 2024-09-14T09:06:35.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.827e-05 2024-09-14T09:06:35.467 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.467 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.467 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.467 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49314 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:35.467 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525478+0000", 2024-09-14T09:06:35.468 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.879647761000001, 2024-09-14T09:06:35.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.06702756, 2024-09-14T09:06:35.468 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.468 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.468 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:35.468 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:35.468 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:35.468 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:35.468 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49314 2024-09-14T09:06:35.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.468 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525478+0000", 2024-09-14T09:06:35.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525478+0000", 2024-09-14T09:06:35.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525480+0000", 2024-09-14T09:06:35.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.246e-06 2024-09-14T09:06:35.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525485+0000", 2024-09-14T09:06:35.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7440000000000004e-06 2024-09-14T09:06:35.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525504+0000", 2024-09-14T09:06:35.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8709999999999999e-05 2024-09-14T09:06:35.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525512+0000", 2024-09-14T09:06:35.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6260000000000003e-06 2024-09-14T09:06:35.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592491+0000", 2024-09-14T09:06:35.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066979274000000005 2024-09-14T09:06:35.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592506+0000", 2024-09-14T09:06:35.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4960000000000001e-05 2024-09-14T09:06:35.471 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.471 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.471 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.471 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49315 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:35.471 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525523+0000", 2024-09-14T09:06:35.471 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.879602965, 2024-09-14T09:06:35.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067036894, 2024-09-14T09:06:35.472 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.472 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.472 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:35.472 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:35.472 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:35.472 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:35.472 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49315 2024-09-14T09:06:35.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.472 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525523+0000", 2024-09-14T09:06:35.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525523+0000", 2024-09-14T09:06:35.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525524+0000", 2024-09-14T09:06:35.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2999999999999999e-07 2024-09-14T09:06:35.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525528+0000", 2024-09-14T09:06:35.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4000000000000001e-06 2024-09-14T09:06:35.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525537+0000", 2024-09-14T09:06:35.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2739999999999996e-06 2024-09-14T09:06:35.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525543+0000", 2024-09-14T09:06:35.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6329999999999999e-06 2024-09-14T09:06:35.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592548+0000", 2024-09-14T09:06:35.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067004806 2024-09-14T09:06:35.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592560+0000", 2024-09-14T09:06:35.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1951e-05 2024-09-14T09:06:35.475 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.475 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.475 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.475 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49316 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:35.475 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525553+0000", 2024-09-14T09:06:35.475 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.879573070999999, 2024-09-14T09:06:35.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067056432999999999, 2024-09-14T09:06:35.476 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.476 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.476 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:35.476 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:35.476 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:35.476 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:35.476 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49316 2024-09-14T09:06:35.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.476 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525553+0000", 2024-09-14T09:06:35.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525553+0000", 2024-09-14T09:06:35.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525554+0000", 2024-09-14T09:06:35.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0299999999999998e-07 2024-09-14T09:06:35.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525557+0000", 2024-09-14T09:06:35.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9129999999999999e-06 2024-09-14T09:06:35.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525566+0000", 2024-09-14T09:06:35.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8000000000000004e-06 2024-09-14T09:06:35.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525572+0000", 2024-09-14T09:06:35.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.72e-06 2024-09-14T09:06:35.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592598+0000", 2024-09-14T09:06:35.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067025682000000003 2024-09-14T09:06:35.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592610+0000", 2024-09-14T09:06:35.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1515000000000001e-05 2024-09-14T09:06:35.479 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.479 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.479 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.479 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49317 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:35.479 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:35.479 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.879544612, 2024-09-14T09:06:35.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067082547000000006, 2024-09-14T09:06:35.480 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.480 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.480 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:35.480 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:35.480 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:35.480 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:35.480 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49317 2024-09-14T09:06:35.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.480 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:35.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:35.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:35.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:06:35.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525585+0000", 2024-09-14T09:06:35.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8700000000000001e-06 2024-09-14T09:06:35.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525594+0000", 2024-09-14T09:06:35.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9740000000000008e-06 2024-09-14T09:06:35.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525601+0000", 2024-09-14T09:06:35.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9489999999999999e-06 2024-09-14T09:06:35.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592652+0000", 2024-09-14T09:06:35.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067050803000000006 2024-09-14T09:06:35.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592664+0000", 2024-09-14T09:06:35.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2198000000000001e-05 2024-09-14T09:06:35.483 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.483 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.483 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.483 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49318 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:35.483 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525642+0000", 2024-09-14T09:06:35.483 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.879484112, 2024-09-14T09:06:35.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067064055999999997, 2024-09-14T09:06:35.483 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.484 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.484 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:35.484 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:35.484 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:35.484 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:35.484 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49318 2024-09-14T09:06:35.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.484 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525642+0000", 2024-09-14T09:06:35.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525642+0000", 2024-09-14T09:06:35.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525644+0000", 2024-09-14T09:06:35.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0159999999999998e-06 2024-09-14T09:06:35.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525649+0000", 2024-09-14T09:06:35.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2739999999999998e-06 2024-09-14T09:06:35.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525666+0000", 2024-09-14T09:06:35.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6990000000000002e-05 2024-09-14T09:06:35.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525680+0000", 2024-09-14T09:06:35.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.331e-05 2024-09-14T09:06:35.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592697+0000", 2024-09-14T09:06:35.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067017367999999994 2024-09-14T09:06:35.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592706+0000", 2024-09-14T09:06:35.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.098e-06 2024-09-14T09:06:35.487 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.487 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.487 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.487 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49319 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:35.487 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525704+0000", 2024-09-14T09:06:35.488 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.879422706, 2024-09-14T09:06:35.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067049867999999999, 2024-09-14T09:06:35.488 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:35.488 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:35.488 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:35.488 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:35.488 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:35.488 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:35.488 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49319 2024-09-14T09:06:35.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.488 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:35.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:35.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525704+0000", 2024-09-14T09:06:35.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:35.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525704+0000", 2024-09-14T09:06:35.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:35.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:35.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525705+0000", 2024-09-14T09:06:35.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.113e-06 2024-09-14T09:06:35.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:35.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525710+0000", 2024-09-14T09:06:35.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.843e-06 2024-09-14T09:06:35.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:35.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525726+0000", 2024-09-14T09:06:35.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.679e-05 2024-09-14T09:06:35.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:35.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525739+0000", 2024-09-14T09:06:35.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2962999999999999e-05 2024-09-14T09:06:35.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:35.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592745+0000", 2024-09-14T09:06:35.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067005341999999996 2024-09-14T09:06:35.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:35.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:35.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:35.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592753+0000", 2024-09-14T09:06:35.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8170000000000005e-06 2024-09-14T09:06:35.491 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.491 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.491 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.491 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:35.491 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:35.491 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:35.492 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:06:35.493 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:35.495+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:35.505 INFO:teuthology.orchestra.run.smithi176.stdout:3326 write 0xaba3d17 thru 0xabaab04 (0x6dee bytes) 2024-09-14T09:06:35.536 INFO:teuthology.orchestra.run.smithi176.stdout:3327 write 0xd97cbbd thru 0xd982229 (0x566d bytes) 2024-09-14T09:06:35.544 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:35.544 INFO:teuthology.orchestra.run.smithi176.stdout:3329 write 0x376823a thru 0x376b092 (0x2e59 bytes) 2024-09-14T09:06:35.551 INFO:teuthology.orchestra.run.smithi176.stdout:3332 write 0x9f4ba26 thru 0x9f5a9a7 (0xef82 bytes) 2024-09-14T09:06:35.580 INFO:teuthology.orchestra.run.smithi176.stdout:3334 trunc from 0xd98222a to 0x759a6ea 2024-09-14T09:06:35.593 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:35.591+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:35.614 INFO:teuthology.orchestra.run.smithi176.stdout:3335 trunc from 0x759a6ea to 0xe66b9c3 2024-09-14T09:06:35.623 INFO:teuthology.orchestra.run.smithi176.stdout:3336 write 0xb840f36 thru 0xb84e464 (0xd52f bytes) 2024-09-14T09:06:35.637 INFO:teuthology.orchestra.run.smithi176.stdout:3337 write 0x108b824 thru 0x109aba2 (0xf37f bytes) 2024-09-14T09:06:35.646 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:35.646 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:35.646 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:35.647 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:35.667 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:06:35.710 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:35.711+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:35.758 INFO:teuthology.orchestra.run.smithi176.stdout:3338 punch from 0xe29aef8 to 0xe2a45e0, (0x96e8 bytes) 2024-09-14T09:06:35.758 INFO:teuthology.orchestra.run.smithi176.stdout:3340 writesame 0xc625ad4 thru 0xc625c7b data_size 0x4(0x1a8 bytes) 2024-09-14T09:06:35.765 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:35.765 INFO:teuthology.orchestra.run.smithi176.stdout:3343 read 0x99c157 thru 0x9a4776 (0x8620 bytes) 2024-09-14T09:06:35.767 INFO:teuthology.orchestra.run.smithi176.stdout:3344 write 0x3a4e106 thru 0x3a5c4aa (0xe3a5 bytes) 2024-09-14T09:06:35.787 INFO:teuthology.orchestra.run.smithi176.stdout:3345 read 0xaf37f2b thru 0xaf39a3c (0x1b12 bytes) 2024-09-14T09:06:35.788 INFO:teuthology.orchestra.run.smithi176.stdout:3348 read 0x3fee46b thru 0x3ff5ba3 (0x7739 bytes) 2024-09-14T09:06:35.788 INFO:teuthology.orchestra.run.smithi176.stdout:3350 writesame 0x880ead1 thru 0x881ccb8 data_size 0x1(0xe1e8 bytes) 2024-09-14T09:06:35.795 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:35.795+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:35.840 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:35.841 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:35.841 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:35.841 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:35.841 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:35.855 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:06:35.878 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:06:35.884 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:35.884 INFO:teuthology.orchestra.run.smithi176.stdout:3352 read 0xe5ed2a5 thru 0xe5fb9bd (0xe719 bytes) 2024-09-14T09:06:35.887 INFO:teuthology.orchestra.run.smithi176.stdout:3353 trunc from 0xe66b9c3 to 0xd52d8b6 2024-09-14T09:06:35.896 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:35.899+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:35.897 INFO:teuthology.orchestra.run.smithi176.stdout:3354 read 0xae83481 thru 0xae84d92 (0x1912 bytes) 2024-09-14T09:06:35.907 INFO:teuthology.orchestra.run.smithi176.stdout:3356 writesame 0xdf52a4e thru 0xdf5e8ef data_size 0xd(0xbea2 bytes) 2024-09-14T09:06:35.912 INFO:teuthology.orchestra.run.smithi176.stdout:3357 trunc from 0xdf5e8f0 to 0xb73e871 2024-09-14T09:06:35.942 INFO:teuthology.orchestra.run.smithi176.stdout:3358 write 0xca13390 thru 0xca2116a (0xdddb bytes) 2024-09-14T09:06:35.971 INFO:teuthology.orchestra.run.smithi176.stdout:3360 trunc from 0xca2116b to 0x510040c 2024-09-14T09:06:35.997 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:35.999+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:36.009 INFO:teuthology.orchestra.run.smithi176.stdout:3363 trunc from 0x510040c to 0xc90b508 2024-09-14T09:06:36.017 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:36.017 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:36.029 INFO:teuthology.orchestra.run.smithi176.stdout:3366 writesame 0xe29be2d thru 0xe2a33cc data_size 0x10(0x75a0 bytes) 2024-09-14T09:06:36.029 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:36.029 INFO:teuthology.orchestra.run.smithi176.stdout:3368 write 0xbe6d98c thru 0xbe7a594 (0xcc09 bytes) 2024-09-14T09:06:36.040 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:36.040 INFO:teuthology.orchestra.run.smithi176.stdout:3371 write 0xa3f7aa0 thru 0xa402d41 (0xb2a2 bytes) 2024-09-14T09:06:36.047 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:36.047 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:36.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:36.047 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:36.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.048 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20858 3.17 3:ed5052d1:::rbd_data.115e889d4809.000000000000015c:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:36.048 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:06:36.048 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.617206443000001, 2024-09-14T09:06:36.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053381794000000003, 2024-09-14T09:06:36.048 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.048 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.048 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:36.048 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:36.048 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:36.048 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:36.048 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20858 2024-09-14T09:06:36.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.049 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:06:36.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:06:36.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429351+0000", 2024-09-14T09:06:36.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1149999999999998e-06 2024-09-14T09:06:36.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429363+0000", 2024-09-14T09:06:36.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1819e-05 2024-09-14T09:06:36.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429389+0000", 2024-09-14T09:06:36.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5605e-05 2024-09-14T09:06:36.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429402+0000", 2024-09-14T09:06:36.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3175e-05 2024-09-14T09:06:36.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429492+0000", 2024-09-14T09:06:36.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.988e-05 2024-09-14T09:06:36.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429597+0000", 2024-09-14T09:06:36.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010564 2024-09-14T09:06:36.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:36.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.433465+0000", 2024-09-14T09:06:36.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0038674249999999999 2024-09-14T09:06:36.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:36.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482483+0000", 2024-09-14T09:06:36.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049017999999999999 2024-09-14T09:06:36.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:36.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482625+0000", 2024-09-14T09:06:36.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000142447 2024-09-14T09:06:36.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:36.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482684+0000", 2024-09-14T09:06:36.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8522e-05 2024-09-14T09:06:36.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482730+0000", 2024-09-14T09:06:36.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6165999999999998e-05 2024-09-14T09:06:36.053 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.053 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.053 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.054 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:20858 3.17 e41/23 ondisk, result = 0)", 2024-09-14T09:06:36.054 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:06:36.054 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.602446999999998, 2024-09-14T09:06:36.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038631402000000002, 2024-09-14T09:06:36.054 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.054 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.054 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:36.054 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:06:36.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:06:36.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444109+0000", 2024-09-14T09:06:36.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.331e-06 2024-09-14T09:06:36.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444114+0000", 2024-09-14T09:06:36.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3709999999999999e-06 2024-09-14T09:06:36.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444130+0000", 2024-09-14T09:06:36.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6079e-05 2024-09-14T09:06:36.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444144+0000", 2024-09-14T09:06:36.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3331e-05 2024-09-14T09:06:36.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482538+0000", 2024-09-14T09:06:36.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038394878 2024-09-14T09:06:36.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482548+0000", 2024-09-14T09:06:36.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0093e-05 2024-09-14T09:06:36.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482739+0000", 2024-09-14T09:06:36.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019031899999999999 2024-09-14T09:06:36.058 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.058 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.058 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.058 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:37800 3.1 3:830c4670:::rbd_data.115eb07a8abd.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:36.058 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:06:36.058 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.015999033, 2024-09-14T09:06:36.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.068040724999999996, 2024-09-14T09:06:36.058 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.058 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.058 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:36.058 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:36.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:36.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:36.059 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 37800 2024-09-14T09:06:36.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.059 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:06:36.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:06:36.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030557+0000", 2024-09-14T09:06:36.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.888e-06 2024-09-14T09:06:36.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030566+0000", 2024-09-14T09:06:36.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8540000000000003e-06 2024-09-14T09:06:36.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030585+0000", 2024-09-14T09:06:36.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8878000000000001e-05 2024-09-14T09:06:36.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030593+0000", 2024-09-14T09:06:36.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5870000000000004e-06 2024-09-14T09:06:36.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030667+0000", 2024-09-14T09:06:36.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4207000000000003e-05 2024-09-14T09:06:36.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030761+0000", 2024-09-14T09:06:36.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4223999999999998e-05 2024-09-14T09:06:36.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:06:36.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.034400+0000", 2024-09-14T09:06:36.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0036387759999999998 2024-09-14T09:06:36.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:36.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098457+0000", 2024-09-14T09:06:36.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.064057501000000003 2024-09-14T09:06:36.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:36.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098518+0000", 2024-09-14T09:06:36.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0730999999999998e-05 2024-09-14T09:06:36.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:36.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098560+0000", 2024-09-14T09:06:36.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2267e-05 2024-09-14T09:06:36.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098596+0000", 2024-09-14T09:06:36.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5812000000000002e-05 2024-09-14T09:06:36.063 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.064 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.064 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.064 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:37800 3.1 e48/34 ondisk, result = 0)", 2024-09-14T09:06:36.064 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:06:36.064 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.005007417000002, 2024-09-14T09:06:36.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057056260999999997, 2024-09-14T09:06:36.064 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.064 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.064 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:36.064 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:06:36.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:06:36.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041548+0000", 2024-09-14T09:06:36.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8100000000000002e-07 2024-09-14T09:06:36.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041552+0000", 2024-09-14T09:06:36.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7419999999999999e-06 2024-09-14T09:06:36.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041563+0000", 2024-09-14T09:06:36.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1926000000000001e-05 2024-09-14T09:06:36.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041572+0000", 2024-09-14T09:06:36.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8610000000000004e-06 2024-09-14T09:06:36.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098503+0000", 2024-09-14T09:06:36.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056930827000000003 2024-09-14T09:06:36.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098512+0000", 2024-09-14T09:06:36.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5019999999999992e-06 2024-09-14T09:06:36.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098603+0000", 2024-09-14T09:06:36.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1621999999999998e-05 2024-09-14T09:06:36.068 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.068 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.068 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.068 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38523 3.3 3:c62e0486:::rbd_data.115e9e133b92.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 4194304~28499] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T09:06:36.068 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:06:36.068 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.592579719, 2024-09-14T09:06:36.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035654406999999999, 2024-09-14T09:06:36.068 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.068 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.068 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:36.068 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:36.069 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:36.069 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:36.069 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38523 2024-09-14T09:06:36.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.069 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:06:36.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:06:36.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453977+0000", 2024-09-14T09:06:36.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6299999999999998e-06 2024-09-14T09:06:36.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455684+0000", 2024-09-14T09:06:36.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001706843 2024-09-14T09:06:36.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455710+0000", 2024-09-14T09:06:36.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6302e-05 2024-09-14T09:06:36.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455722+0000", 2024-09-14T09:06:36.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1798e-05 2024-09-14T09:06:36.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455825+0000", 2024-09-14T09:06:36.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102822 2024-09-14T09:06:36.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.456032+0000", 2024-09-14T09:06:36.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020704400000000001 2024-09-14T09:06:36.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:06:36.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.456759+0000", 2024-09-14T09:06:36.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00072735699999999998 2024-09-14T09:06:36.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:36.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.477856+0000", 2024-09-14T09:06:36.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021096011000000001 2024-09-14T09:06:36.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:36.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489526+0000", 2024-09-14T09:06:36.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011670313 2024-09-14T09:06:36.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:36.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489572+0000", 2024-09-14T09:06:36.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6408e-05 2024-09-14T09:06:36.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489629+0000", 2024-09-14T09:06:36.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6879000000000002e-05 2024-09-14T09:06:36.074 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.074 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.074 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.074 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49715 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:36.074 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.524874+0000", 2024-09-14T09:06:36.074 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.521680777, 2024-09-14T09:06:36.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034330422999999999, 2024-09-14T09:06:36.074 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.074 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.074 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:36.074 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:36.075 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:36.075 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:36.075 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49715 2024-09-14T09:06:36.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.075 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524874+0000", 2024-09-14T09:06:36.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524874+0000", 2024-09-14T09:06:36.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524875+0000", 2024-09-14T09:06:36.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1500000000000003e-07 2024-09-14T09:06:36.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524878+0000", 2024-09-14T09:06:36.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9090000000000002e-06 2024-09-14T09:06:36.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524889+0000", 2024-09-14T09:06:36.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.007e-05 2024-09-14T09:06:36.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524895+0000", 2024-09-14T09:06:36.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.285e-06 2024-09-14T09:06:36.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.558620+0000", 2024-09-14T09:06:36.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.033725009 2024-09-14T09:06:36.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.558681+0000", 2024-09-14T09:06:36.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1051999999999998e-05 2024-09-14T09:06:36.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559204+0000", 2024-09-14T09:06:36.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00052318299999999998 2024-09-14T09:06:36.078 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.078 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.078 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.078 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49716 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:36.079 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525151+0000", 2024-09-14T09:06:36.079 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.521403058000001, 2024-09-14T09:06:36.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034564820000000003, 2024-09-14T09:06:36.079 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.079 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.079 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:36.079 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:36.079 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:36.079 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:36.079 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49716 2024-09-14T09:06:36.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.079 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525151+0000", 2024-09-14T09:06:36.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525151+0000", 2024-09-14T09:06:36.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525152+0000", 2024-09-14T09:06:36.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5799999999999998e-07 2024-09-14T09:06:36.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525157+0000", 2024-09-14T09:06:36.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6380000000000003e-06 2024-09-14T09:06:36.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525167+0000", 2024-09-14T09:06:36.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0457e-05 2024-09-14T09:06:36.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525174+0000", 2024-09-14T09:06:36.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6220000000000003e-06 2024-09-14T09:06:36.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559237+0000", 2024-09-14T09:06:36.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034063361 2024-09-14T09:06:36.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559308+0000", 2024-09-14T09:06:36.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1124999999999994e-05 2024-09-14T09:06:36.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559716+0000", 2024-09-14T09:06:36.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000407759 2024-09-14T09:06:36.083 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.083 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.083 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.083 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49717 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:36.083 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525390+0000", 2024-09-14T09:06:36.083 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.521164260999999, 2024-09-14T09:06:36.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034552639000000003, 2024-09-14T09:06:36.083 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.083 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.083 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:36.084 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:36.084 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:36.084 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:36.084 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49717 2024-09-14T09:06:36.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.084 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525390+0000", 2024-09-14T09:06:36.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525390+0000", 2024-09-14T09:06:36.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525391+0000", 2024-09-14T09:06:36.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3099999999999997e-07 2024-09-14T09:06:36.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525395+0000", 2024-09-14T09:06:36.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.267e-06 2024-09-14T09:06:36.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525406+0000", 2024-09-14T09:06:36.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1224e-05 2024-09-14T09:06:36.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525413+0000", 2024-09-14T09:06:36.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5420000000000002e-06 2024-09-14T09:06:36.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559760+0000", 2024-09-14T09:06:36.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034347177999999999 2024-09-14T09:06:36.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559838+0000", 2024-09-14T09:06:36.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7708999999999997e-05 2024-09-14T09:06:36.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559943+0000", 2024-09-14T09:06:36.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010498800000000001 2024-09-14T09:06:36.087 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.087 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.087 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.088 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49718 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:36.088 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525514+0000", 2024-09-14T09:06:36.088 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.521040456000001, 2024-09-14T09:06:36.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034602766, 2024-09-14T09:06:36.088 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.088 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.088 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:36.088 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:36.088 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:36.088 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:36.088 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49718 2024-09-14T09:06:36.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.088 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.089 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.089 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525514+0000", 2024-09-14T09:06:36.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.089 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.089 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525514+0000", 2024-09-14T09:06:36.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.089 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.089 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525515+0000", 2024-09-14T09:06:36.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-07 2024-09-14T09:06:36.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.090 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525518+0000", 2024-09-14T09:06:36.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7220000000000001e-06 2024-09-14T09:06:36.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.090 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525528+0000", 2024-09-14T09:06:36.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4639999999999995e-06 2024-09-14T09:06:36.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.090 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525534+0000", 2024-09-14T09:06:36.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2330000000000003e-06 2024-09-14T09:06:36.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559974+0000", 2024-09-14T09:06:36.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034439866 2024-09-14T09:06:36.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560041+0000", 2024-09-14T09:06:36.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7343999999999998e-05 2024-09-14T09:06:36.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560117+0000", 2024-09-14T09:06:36.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5415999999999998e-05 2024-09-14T09:06:36.092 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.092 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.092 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.092 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49719 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:36.092 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:06:36.092 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.520921118, 2024-09-14T09:06:36.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034680563999999997, 2024-09-14T09:06:36.092 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.093 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.093 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:36.093 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:36.093 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:36.093 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:36.093 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49719 2024-09-14T09:06:36.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.093 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:06:36.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.094 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:06:36.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.094 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.094 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525634+0000", 2024-09-14T09:06:36.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4000000000000001e-07 2024-09-14T09:06:36.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.094 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.094 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525638+0000", 2024-09-14T09:06:36.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5899999999999999e-06 2024-09-14T09:06:36.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.094 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525647+0000", 2024-09-14T09:06:36.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2029999999999998e-06 2024-09-14T09:06:36.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525653+0000", 2024-09-14T09:06:36.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9850000000000003e-06 2024-09-14T09:06:36.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560150+0000", 2024-09-14T09:06:36.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034496845999999998 2024-09-14T09:06:36.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560221+0000", 2024-09-14T09:06:36.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1400000000000001e-05 2024-09-14T09:06:36.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560314+0000", 2024-09-14T09:06:36.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2800000000000006e-05 2024-09-14T09:06:36.096 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.096 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.096 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.097 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49720 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:36.097 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:06:36.097 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.520669494, 2024-09-14T09:06:36.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034919095999999997, 2024-09-14T09:06:36.097 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.097 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.097 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:36.097 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:36.097 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:36.097 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:36.097 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49720 2024-09-14T09:06:36.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.098 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:06:36.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:06:36.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525886+0000", 2024-09-14T09:06:36.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6000000000000002e-07 2024-09-14T09:06:36.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525890+0000", 2024-09-14T09:06:36.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4519999999999999e-06 2024-09-14T09:06:36.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525899+0000", 2024-09-14T09:06:36.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1090000000000004e-06 2024-09-14T09:06:36.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525906+0000", 2024-09-14T09:06:36.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0219999999999999e-06 2024-09-14T09:06:36.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560344+0000", 2024-09-14T09:06:36.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034437970999999998 2024-09-14T09:06:36.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560422+0000", 2024-09-14T09:06:36.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7857999999999999e-05 2024-09-14T09:06:36.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560804+0000", 2024-09-14T09:06:36.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00038182400000000001 2024-09-14T09:06:36.101 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.101 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.101 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.101 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49721 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:36.101 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:06:36.101 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.520622012, 2024-09-14T09:06:36.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035347380999999997, 2024-09-14T09:06:36.101 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.101 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.101 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:36.102 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:36.102 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:36.102 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:36.102 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49721 2024-09-14T09:06:36.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.102 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:06:36.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:06:36.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525933+0000", 2024-09-14T09:06:36.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T09:06:36.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525936+0000", 2024-09-14T09:06:36.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6759999999999999e-06 2024-09-14T09:06:36.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525943+0000", 2024-09-14T09:06:36.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2459999999999997e-06 2024-09-14T09:06:36.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525948+0000", 2024-09-14T09:06:36.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9429999999999999e-06 2024-09-14T09:06:36.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560845+0000", 2024-09-14T09:06:36.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034897354999999998 2024-09-14T09:06:36.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560925+0000", 2024-09-14T09:06:36.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9804999999999999e-05 2024-09-14T09:06:36.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561280+0000", 2024-09-14T09:06:36.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035480100000000002 2024-09-14T09:06:36.105 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.105 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.105 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.105 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49722 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3112960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:36.106 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:36.106 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.520598366000002, 2024-09-14T09:06:36.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035750491000000002, 2024-09-14T09:06:36.106 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.106 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.106 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:36.106 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:36.106 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:36.106 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:36.106 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49722 2024-09-14T09:06:36.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.106 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:36.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:36.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:36.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.96e-07 2024-09-14T09:06:36.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525958+0000", 2024-09-14T09:06:36.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.024e-06 2024-09-14T09:06:36.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525964+0000", 2024-09-14T09:06:36.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6699999999999999e-06 2024-09-14T09:06:36.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525969+0000", 2024-09-14T09:06:36.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3939999999999998e-06 2024-09-14T09:06:36.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561319+0000", 2024-09-14T09:06:36.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.03535013 2024-09-14T09:06:36.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561390+0000", 2024-09-14T09:06:36.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0920000000000005e-05 2024-09-14T09:06:36.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561707+0000", 2024-09-14T09:06:36.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031695699999999998 2024-09-14T09:06:36.109 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.109 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.110 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.110 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49723 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3506176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:36.110 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:36.110 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.520579383000001, 2024-09-14T09:06:36.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036113064, 2024-09-14T09:06:36.110 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.110 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.110 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:36.110 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:36.110 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:36.110 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:36.110 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49723 2024-09-14T09:06:36.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.111 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:36.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:36.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:36.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1699999999999999e-07 2024-09-14T09:06:36.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525977+0000", 2024-09-14T09:06:36.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9250000000000002e-06 2024-09-14T09:06:36.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525983+0000", 2024-09-14T09:06:36.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4620000000000004e-06 2024-09-14T09:06:36.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525987+0000", 2024-09-14T09:06:36.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.279e-06 2024-09-14T09:06:36.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561749+0000", 2024-09-14T09:06:36.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035762222000000003 2024-09-14T09:06:36.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561796+0000", 2024-09-14T09:06:36.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6959999999999998e-05 2024-09-14T09:06:36.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562088+0000", 2024-09-14T09:06:36.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029179899999999998 2024-09-14T09:06:36.114 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.114 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.114 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.114 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49724 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:36.114 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:06:36.114 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.520561227000002, 2024-09-14T09:06:36.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036449388999999999, 2024-09-14T09:06:36.114 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.114 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.114 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:36.115 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:36.115 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:36.115 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:36.115 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49724 2024-09-14T09:06:36.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.115 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:06:36.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:06:36.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525994+0000", 2024-09-14T09:06:36.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3099999999999998e-07 2024-09-14T09:06:36.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525996+0000", 2024-09-14T09:06:36.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.939e-06 2024-09-14T09:06:36.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526005+0000", 2024-09-14T09:06:36.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8430000000000005e-06 2024-09-14T09:06:36.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526010+0000", 2024-09-14T09:06:36.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9389999999999996e-06 2024-09-14T09:06:36.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562114+0000", 2024-09-14T09:06:36.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036103510999999998 2024-09-14T09:06:36.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562189+0000", 2024-09-14T09:06:36.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5162999999999998e-05 2024-09-14T09:06:36.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562443+0000", 2024-09-14T09:06:36.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000253563 2024-09-14T09:06:36.118 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.118 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.118 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.118 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:64935 3.12 3:4cd6d7cb:::rbd_data.115e21c31fcf.0000000000000049:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:06:36.118 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:06:36.118 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.295195879, 2024-09-14T09:06:36.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037353866999999999, 2024-09-14T09:06:36.119 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.119 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.119 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:36.119 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:36.119 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:36.119 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:36.119 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64935 2024-09-14T09:06:36.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.119 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:06:36.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:06:36.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751362+0000", 2024-09-14T09:06:36.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6899999999999998e-06 2024-09-14T09:06:36.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751378+0000", 2024-09-14T09:06:36.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6147e-05 2024-09-14T09:06:36.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751421+0000", 2024-09-14T09:06:36.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2691999999999997e-05 2024-09-14T09:06:36.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751444+0000", 2024-09-14T09:06:36.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3237000000000001e-05 2024-09-14T09:06:36.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751594+0000", 2024-09-14T09:06:36.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014947800000000001 2024-09-14T09:06:36.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751733+0000", 2024-09-14T09:06:36.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013945200000000001 2024-09-14T09:06:36.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:36.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.753315+0000", 2024-09-14T09:06:36.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0015815969999999999 2024-09-14T09:06:36.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:36.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788546+0000", 2024-09-14T09:06:36.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035231449999999997 2024-09-14T09:06:36.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:36.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788570+0000", 2024-09-14T09:06:36.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3991000000000001e-05 2024-09-14T09:06:36.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:36.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788645+0000", 2024-09-14T09:06:36.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4379000000000001e-05 2024-09-14T09:06:36.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788712+0000", 2024-09-14T09:06:36.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7754000000000002e-05 2024-09-14T09:06:36.124 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.124 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.124 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.124 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:71052 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:06:36.124 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:06:36.124 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.3089866279999995, 2024-09-14T09:06:36.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14854635599999999, 2024-09-14T09:06:36.125 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.125 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.125 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:36.125 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:36.125 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:36.125 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:36.125 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 71052 2024-09-14T09:06:36.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.125 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:06:36.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:06:36.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737570+0000", 2024-09-14T09:06:36.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4770000000000002e-06 2024-09-14T09:06:36.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737584+0000", 2024-09-14T09:06:36.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3457e-05 2024-09-14T09:06:36.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737608+0000", 2024-09-14T09:06:36.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3893999999999999e-05 2024-09-14T09:06:36.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737620+0000", 2024-09-14T09:06:36.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2157000000000001e-05 2024-09-14T09:06:36.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737738+0000", 2024-09-14T09:06:36.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118409 2024-09-14T09:06:36.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737892+0000", 2024-09-14T09:06:36.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154222 2024-09-14T09:06:36.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:06:36.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.749622+0000", 2024-09-14T09:06:36.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011729537 2024-09-14T09:06:36.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:36.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885934+0000", 2024-09-14T09:06:36.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13631222200000001 2024-09-14T09:06:36.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:36.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885996+0000", 2024-09-14T09:06:36.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1555000000000003e-05 2024-09-14T09:06:36.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:36.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886060+0000", 2024-09-14T09:06:36.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3980999999999996e-05 2024-09-14T09:06:36.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886114+0000", 2024-09-14T09:06:36.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4444999999999998e-05 2024-09-14T09:06:36.130 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.130 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.130 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.130 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:71052 3.1 e71/34 ondisk, result = 0)", 2024-09-14T09:06:36.130 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:06:36.130 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.2660832500000003, 2024-09-14T09:06:36.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10565334799999999, 2024-09-14T09:06:36.131 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.131 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.131 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:36.131 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:06:36.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:06:36.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780473+0000", 2024-09-14T09:06:36.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8980000000000001e-06 2024-09-14T09:06:36.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780482+0000", 2024-09-14T09:06:36.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9080000000000001e-06 2024-09-14T09:06:36.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780511+0000", 2024-09-14T09:06:36.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9490999999999999e-05 2024-09-14T09:06:36.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780527+0000", 2024-09-14T09:06:36.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.508e-05 2024-09-14T09:06:36.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885978+0000", 2024-09-14T09:06:36.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.105451328 2024-09-14T09:06:36.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885989+0000", 2024-09-14T09:06:36.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0875e-05 2024-09-14T09:06:36.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886125+0000", 2024-09-14T09:06:36.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013576800000000001 2024-09-14T09:06:36.134 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.134 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.134 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.134 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73871 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:1b [read 8339456~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:36.134 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:06:36.134 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.62059824200000002, 2024-09-14T09:06:36.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034605047, 2024-09-14T09:06:36.135 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.135 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.135 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:36.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:36.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:36.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:36.135 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73871 2024-09-14T09:06:36.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.135 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:06:36.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:06:36.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425957+0000", 2024-09-14T09:06:36.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T09:06:36.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425961+0000", 2024-09-14T09:06:36.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8279999999999996e-06 2024-09-14T09:06:36.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425985+0000", 2024-09-14T09:06:36.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4182e-05 2024-09-14T09:06:36.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425997+0000", 2024-09-14T09:06:36.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.132e-05 2024-09-14T09:06:36.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460070+0000", 2024-09-14T09:06:36.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034072913000000003 2024-09-14T09:06:36.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460298+0000", 2024-09-14T09:06:36.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022819 2024-09-14T09:06:36.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460561+0000", 2024-09-14T09:06:36.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026341400000000001 2024-09-14T09:06:36.138 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.138 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.138 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.139 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:74276 3.12 3:49b683dd:::rbd_data.115ea8d131ff.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1376256~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:36.139 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:06:36.139 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.361451152, 2024-09-14T09:06:36.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036482939999999998, 2024-09-14T09:06:36.139 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:36.139 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:36.139 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:36.139 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:36.139 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:36.139 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:36.139 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74276 2024-09-14T09:06:36.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.139 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:36.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:36.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:06:36.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:36.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:06:36.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:36.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:36.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685105+0000", 2024-09-14T09:06:36.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9520000000000001e-06 2024-09-14T09:06:36.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:36.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686530+0000", 2024-09-14T09:06:36.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001424629 2024-09-14T09:06:36.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:36.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686550+0000", 2024-09-14T09:06:36.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0472e-05 2024-09-14T09:06:36.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:36.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686561+0000", 2024-09-14T09:06:36.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T09:06:36.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:36.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686671+0000", 2024-09-14T09:06:36.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110621 2024-09-14T09:06:36.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:36.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686919+0000", 2024-09-14T09:06:36.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024790199999999999 2024-09-14T09:06:36.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:36.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.687375+0000", 2024-09-14T09:06:36.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00045572100000000002 2024-09-14T09:06:36.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:36.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.710871+0000", 2024-09-14T09:06:36.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023496481999999999 2024-09-14T09:06:36.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:36.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721478+0000", 2024-09-14T09:06:36.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01060709 2024-09-14T09:06:36.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:36.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721534+0000", 2024-09-14T09:06:36.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5912999999999998e-05 2024-09-14T09:06:36.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:36.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:36.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:36.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721586+0000", 2024-09-14T09:06:36.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1792000000000002e-05 2024-09-14T09:06:36.144 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.144 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.144 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.144 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:36.144 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:36.145 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:36.145 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:36.146 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:36.146 INFO:teuthology.orchestra.run.smithi176.stdout:3375 write 0x275e275 thru 0x276670c (0x8498 bytes) 2024-09-14T09:06:36.147 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:06:36.147 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:36.148 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:36.148 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:36.148 INFO:teuthology.orchestra.run.smithi176.stdout:3380 write 0x2e58052 thru 0x2e59391 (0x1340 bytes) 2024-09-14T09:06:36.148 INFO:teuthology.orchestra.run.smithi176.stdout:3381 flatten 2024-09-14T09:06:36.149 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:36.149 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:36.149 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:36.149 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:36.150 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:36.150 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:36.150 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:36.147+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:36.177 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:36.178 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:36.195 INFO:teuthology.orchestra.run.smithi176.stdout:3382 punch from 0x5d24b7c to 0x5d3050d, (0xb991 bytes) 2024-09-14T09:06:36.195 INFO:teuthology.orchestra.run.smithi176.stdout:3384 write 0x805055d thru 0x805a7d9 (0xa27d bytes) 2024-09-14T09:06:36.200 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:36.201 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:36.203 INFO:teuthology.orchestra.run.smithi176.stdout:3385 write 0x218984e thru 0x21902e6 (0x6a99 bytes) 2024-09-14T09:06:36.207 INFO:teuthology.orchestra.run.smithi176.stdout:3387 punch from 0x7693d34 to 0x769a819, (0x6ae5 bytes) 2024-09-14T09:06:36.207 INFO:teuthology.orchestra.run.smithi176.stdout:3389 read 0xb175b2a thru 0xb183c76 (0xe14d bytes) 2024-09-14T09:06:36.207 INFO:teuthology.orchestra.run.smithi176.stdout:3391 write 0xcd806bb thru 0xcd8cc01 (0xc547 bytes) 2024-09-14T09:06:36.218 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:36.219 INFO:teuthology.orchestra.run.smithi176.stdout:3393 read 0xddd22c5 thru 0xdddd00f (0xad4b bytes) 2024-09-14T09:06:36.219 INFO:teuthology.orchestra.run.smithi176.stdout:3394 writesame 0xd73058a thru 0xd7381da data_size 0x43(0x7c51 bytes) 2024-09-14T09:06:36.227 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:36.227 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:36.246 INFO:teuthology.orchestra.run.smithi176.stdout:3395 trunc from 0xe2a33cd to 0x43bff34 2024-09-14T09:06:36.246 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:36.247+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:36.265 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:36.276 INFO:teuthology.orchestra.run.smithi176.stdout:3397 trunc from 0x43bff34 to 0xb40ba1c 2024-09-14T09:06:36.280 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:06:36.280 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T09:06:36.282 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:36.282 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:36.284 INFO:teuthology.orchestra.run.smithi176.stdout:3398 writesame 0x696a1f3 thru 0x696e124 data_size 0x1(0x3f32 bytes) 2024-09-14T09:06:36.301 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:06:36.327 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T09:06:36.330 INFO:teuthology.orchestra.run.smithi176.stdout:3399 write 0xb31163f thru 0xb316181 (0x4b43 bytes) 2024-09-14T09:06:36.338 INFO:teuthology.orchestra.run.smithi176.stdout:3402 read 0x5b9f276 thru 0x5ba0b7e (0x1909 bytes) 2024-09-14T09:06:36.339 INFO:teuthology.orchestra.run.smithi176.stdout:3404 read 0x4a51875 thru 0x4a58a86 (0x7212 bytes) 2024-09-14T09:06:36.347 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:36.351+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:36.351 INFO:teuthology.orchestra.run.smithi176.stdout:3405 trunc from 0xb40ba1c to 0x245bf25 2024-09-14T09:06:36.379 INFO:teuthology.orchestra.run.smithi176.stdout:3406 punch from 0x1e48593 to 0x1e4d316, (0x4d83 bytes) 2024-09-14T09:06:36.390 INFO:teuthology.orchestra.run.smithi176.stdout:3408 writesame 0x431c295 thru 0x431f1ba data_size 0x2(0x2f26 bytes) 2024-09-14T09:06:36.390 INFO:teuthology.orchestra.run.smithi176.stdout:3411 writesame 0x1d2f200 thru 0x1d33510 data_size 0x3b(0x4311 bytes) 2024-09-14T09:06:36.397 INFO:teuthology.orchestra.run.smithi176.stdout:3412 trunc from 0x431f1bb to 0x4e2bcbf 2024-09-14T09:06:36.408 INFO:teuthology.orchestra.run.smithi176.stdout:3413 read 0x2b5d5a9 thru 0x2b5dc68 (0x6c0 bytes) 2024-09-14T09:06:36.408 INFO:teuthology.orchestra.run.smithi176.stdout:3414 read 0x14df6f8 thru 0x14e06af (0xfb8 bytes) 2024-09-14T09:06:36.408 INFO:teuthology.orchestra.run.smithi176.stdout:3417 read 0x556f3a thru 0x559542 (0x2609 bytes) 2024-09-14T09:06:36.419 INFO:teuthology.orchestra.run.smithi176.stdout:3418 write 0x5dc67b3 thru 0x5dcd955 (0x71a3 bytes) 2024-09-14T09:06:36.447 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:36.451+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:36.450 INFO:teuthology.orchestra.run.smithi176.stdout:3419 write 0xd841821 thru 0xd844022 (0x2802 bytes) 2024-09-14T09:06:36.474 INFO:teuthology.orchestra.run.smithi176.stdout:3420 trunc from 0xd844023 to 0x2ab94ac 2024-09-14T09:06:36.479 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:36.479 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:36.505 INFO:teuthology.orchestra.run.smithi176.stdout:3421 writesame 0x44a2bbc thru 0x44af159 data_size 0x5(0xc59e bytes) 2024-09-14T09:06:36.505 INFO:teuthology.orchestra.run.smithi176.stdout:3422 punch from 0x18c9df9 to 0x18d885f, (0xea66 bytes) 2024-09-14T09:06:36.506 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:06:36.506 INFO:teuthology.orchestra.run.smithi176.stdout:3423 write 0x1e0321d thru 0x1e0d9bb (0xa79f bytes) 2024-09-14T09:06:36.509 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:36.527 INFO:teuthology.orchestra.run.smithi176.stdout:3427 write 0xb424b7f thru 0xb42da47 (0x8ec9 bytes) 2024-09-14T09:06:36.548 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:36.547+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:36.549 INFO:teuthology.orchestra.run.smithi176.stdout:3428 writesame 0xcec9444 thru 0xcece1e9 data_size 0x3(0x4da6 bytes) 2024-09-14T09:06:36.549 INFO:teuthology.orchestra.run.smithi176.stdout:3430 write 0x60ca9b5 thru 0x60d08ce (0x5f1a bytes) 2024-09-14T09:06:36.563 INFO:teuthology.orchestra.run.smithi176.stdout:3433 trunc from 0xcece1ea to 0xd7ac3cc 2024-09-14T09:06:36.571 INFO:teuthology.orchestra.run.smithi176.stdout:3434 read 0x344c844 thru 0x344f8c1 (0x307e bytes) 2024-09-14T09:06:36.571 INFO:teuthology.orchestra.run.smithi176.stdout:3435 read 0x179f3d7 thru 0x17a1527 (0x2151 bytes) 2024-09-14T09:06:36.571 INFO:teuthology.orchestra.run.smithi176.stdout:3436 write 0xcef0b79 thru 0xcef59ca (0x4e52 bytes) 2024-09-14T09:06:36.588 INFO:teuthology.orchestra.run.smithi176.stdout:3437 write 0xdf85c68 thru 0xdf9371f (0xdab8 bytes) 2024-09-14T09:06:36.613 INFO:teuthology.orchestra.run.smithi176.stdout:3439 trunc from 0xdf93720 to 0x5f2b704 2024-09-14T09:06:36.637 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:36.637 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:36.649 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:36.651+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:36.650 INFO:teuthology.orchestra.run.smithi176.stdout:3440 trunc from 0x5f2b704 to 0xe9649af 2024-09-14T09:06:36.658 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:36.658 INFO:teuthology.orchestra.run.smithi176.stdout:3442 write 0x6d1fc0a thru 0x6d227e1 (0x2bd8 bytes) 2024-09-14T09:06:36.661 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:06:36.667 INFO:teuthology.orchestra.run.smithi176.stdout:3443 punch from 0x8ffc721 to 0x90064cc, (0x9dab bytes) 2024-09-14T09:06:36.668 INFO:teuthology.orchestra.run.smithi176.stdout:3446 write 0x2f51d94 thru 0x2f54885 (0x2af2 bytes) 2024-09-14T09:06:36.674 INFO:teuthology.orchestra.run.smithi176.stdout:3447 punch from 0x61d7e6e to 0x61e6518, (0xe6aa bytes) 2024-09-14T09:06:36.675 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:36.675 INFO:teuthology.orchestra.run.smithi176.stdout:3450 read 0x9cb4bcb thru 0x9cc3fca (0xf400 bytes) 2024-09-14T09:06:36.675 INFO:teuthology.orchestra.run.smithi176.stdout:3451 read 0x10b721e thru 0x10c5114 (0xdef7 bytes) 2024-09-14T09:06:36.676 INFO:teuthology.orchestra.run.smithi176.stdout:3452 writesame 0xbef5e6e thru 0xbf00882 data_size 0x1(0xaa15 bytes) 2024-09-14T09:06:36.729 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:36.730 INFO:teuthology.orchestra.run.smithi176.stdout:3454 write 0x2833d63 thru 0x2843749 (0xf9e7 bytes) 2024-09-14T09:06:36.749 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:36.751+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:36.756 INFO:teuthology.orchestra.run.smithi176.stdout:3456 trunc from 0xe9649af to 0x329f9b9 2024-09-14T09:06:36.774 INFO:teuthology.orchestra.run.smithi176.stdout:3458 punch from 0xf7df08 to 0xf8560f, (0x7707 bytes) 2024-09-14T09:06:36.778 INFO:teuthology.orchestra.run.smithi176.stdout:3459 trunc from 0x329f9b9 to 0x6b06ee1 2024-09-14T09:06:36.786 INFO:teuthology.orchestra.run.smithi176.stdout:3460 read 0x41da605 thru 0x41dd0bd (0x2ab9 bytes) 2024-09-14T09:06:36.786 INFO:teuthology.orchestra.run.smithi176.stdout:3461 write 0x50762c5 thru 0x5078a81 (0x27bd bytes) 2024-09-14T09:06:36.797 INFO:teuthology.orchestra.run.smithi176.stdout:3463 trunc from 0x6b06ee1 to 0x48ba287 2024-09-14T09:06:36.804 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:36.804 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:36.816 INFO:teuthology.orchestra.run.smithi176.stdout:3464 read 0x3dc5a90 thru 0x3dd2f1a (0xd48b bytes) 2024-09-14T09:06:36.821 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:06:36.828 INFO:teuthology.orchestra.run.smithi176.stdout:3465 write 0xa08f1f8 thru 0xa09a3f5 (0xb1fe bytes) 2024-09-14T09:06:36.846 INFO:teuthology.orchestra.run.smithi176.stdout:3469 writesame 0x852a857 thru 0x852c3a7 data_size 0x1b(0x1b51 bytes) 2024-09-14T09:06:36.850 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:36.851+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:36.854 INFO:teuthology.orchestra.run.smithi176.stdout:3470 read 0x2f03bad thru 0x2f05414 (0x1868 bytes) 2024-09-14T09:06:36.854 INFO:teuthology.orchestra.run.smithi176.stdout:3472 trunc from 0xa09a3f6 to 0xa511476 2024-09-14T09:06:36.862 INFO:teuthology.orchestra.run.smithi176.stdout:3473 read 0x2542edc thru 0x254853d (0x5662 bytes) 2024-09-14T09:06:36.863 INFO:teuthology.orchestra.run.smithi176.stdout:3474 write 0x8f070a thru 0x8f65dc (0x5ed3 bytes) 2024-09-14T09:06:36.869 INFO:teuthology.orchestra.run.smithi176.stdout:3476 read 0x995faed thru 0x9963c2e (0x4142 bytes) 2024-09-14T09:06:36.869 INFO:teuthology.orchestra.run.smithi176.stdout:3479 writesame 0x7e43d53 thru 0x7e5086f data_size 0xb(0xcb1d bytes) 2024-09-14T09:06:36.902 INFO:teuthology.orchestra.run.smithi176.stdout:3482 writesame 0xe6b3da3 thru 0xe6bd5ad data_size 0x1(0x980b bytes) 2024-09-14T09:06:36.902 INFO:teuthology.orchestra.run.smithi176.stdout:3483 punch from 0x7e9b23b to 0x7eaa185, (0xef4a bytes) 2024-09-14T09:06:36.902 INFO:teuthology.orchestra.run.smithi176.stdout:3484 read 0x4444be3 thru 0x444d74f (0x8b6d bytes) 2024-09-14T09:06:36.902 INFO:teuthology.orchestra.run.smithi176.stdout:3486 read 0xd2dd577 thru 0xd2e5e4c (0x88d6 bytes) 2024-09-14T09:06:36.902 INFO:teuthology.orchestra.run.smithi176.stdout:3487 read 0x40c99cc thru 0x40cb0f8 (0x172d bytes) 2024-09-14T09:06:36.903 INFO:teuthology.orchestra.run.smithi176.stdout:3488 punch from 0x2a30ea1 to 0x2a402b4, (0xf413 bytes) 2024-09-14T09:06:36.906 INFO:teuthology.orchestra.run.smithi176.stdout:3489 trunc from 0xe6bd5ae to 0xc8f809f 2024-09-14T09:06:36.919 INFO:teuthology.orchestra.run.smithi176.stdout:3491 read 0x91faaa6 thru 0x91fbdf4 (0x134f bytes) 2024-09-14T09:06:36.919 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:36.919 INFO:teuthology.orchestra.run.smithi176.stdout:3494 punch from 0xae3e07c to 0xae48828, (0xa7ac bytes) 2024-09-14T09:06:36.921 INFO:teuthology.orchestra.run.smithi176.stdout:3495 trunc from 0xc8f809f to 0xe266a38 2024-09-14T09:06:36.932 INFO:teuthology.orchestra.run.smithi176.stdout:3497 trunc from 0xe266a38 to 0xda9ef49 2024-09-14T09:06:36.943 INFO:teuthology.orchestra.run.smithi176.stdout:3500 punch from 0x1c747db to 0x1c790c8, (0x48ed bytes) 2024-09-14T09:06:36.943 INFO:teuthology.orchestra.run.smithi176.stdout:3502 punch from 0x6d2fc49 to 0x6d2fd6f, (0x126 bytes) 2024-09-14T09:06:36.943 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:36.943 INFO:teuthology.orchestra.run.smithi176.stdout:3505 read 0x663748d thru 0x6646833 (0xf3a7 bytes) 2024-09-14T09:06:36.943 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:36.943 INFO:teuthology.orchestra.run.smithi176.stdout:3510 write 0x4ffc1cf thru 0x500324c (0x707e bytes) 2024-09-14T09:06:36.951 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:36.951+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:36.953 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:36.953 INFO:teuthology.orchestra.run.smithi176.stdout:3513 read 0x32b57cb thru 0x32b6991 (0x11c7 bytes) 2024-09-14T09:06:36.953 INFO:teuthology.orchestra.run.smithi176.stdout:3514 write 0x5dc1eb1 thru 0x5dcc9fa (0xab4a bytes) 2024-09-14T09:06:36.954 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:36.954 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:36.969 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:06:36.970 INFO:teuthology.orchestra.run.smithi176.stdout:3515 read 0x87c243 thru 0x88609e (0x9e5c bytes) 2024-09-14T09:06:36.971 INFO:teuthology.orchestra.run.smithi176.stdout:3517 write 0x83646ea thru 0x8368112 (0x3a29 bytes) 2024-09-14T09:06:36.981 INFO:teuthology.orchestra.run.smithi176.stdout:3518 writesame 0xcbb6a2f thru 0xcbb9f8f data_size 0x1(0x3561 bytes) 2024-09-14T09:06:36.992 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:36.992 INFO:teuthology.orchestra.run.smithi176.stdout:3520 read 0xbbc494f thru 0xbbcb9fe (0x70b0 bytes) 2024-09-14T09:06:36.992 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:36.992 INFO:teuthology.orchestra.run.smithi176.stdout:3522 writesame 0x6b6880f thru 0x6b7199b data_size 0x1(0x918d bytes) 2024-09-14T09:06:37.046 INFO:teuthology.orchestra.run.smithi176.stdout:3523 write 0x438504a thru 0x438b279 (0x6230 bytes) 2024-09-14T09:06:37.051 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:37.051+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:37.055 INFO:teuthology.orchestra.run.smithi176.stdout:3524 writesame 0xca93af6 thru 0xcaa1e06 data_size 0x1(0xe311 bytes) 2024-09-14T09:06:37.125 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:37.125 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:37.138 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:06:37.151 INFO:teuthology.orchestra.run.smithi176.stdout:3526 read 0x6d8ddc thru 0x6df7ae (0x69d3 bytes) 2024-09-14T09:06:37.152 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:37.155+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:37.153 INFO:teuthology.orchestra.run.smithi176.stdout:3528 read 0x437d1be thru 0x4383cbb (0x6afe bytes) 2024-09-14T09:06:37.153 INFO:teuthology.orchestra.run.smithi176.stdout:3529 writesame 0x858ea56 thru 0x8591f68 data_size 0x1(0x3513 bytes) 2024-09-14T09:06:37.163 INFO:teuthology.orchestra.run.smithi176.stdout:3530 writesame 0x5c40427 thru 0x5c4a21a data_size 0x16(0x9df4 bytes) 2024-09-14T09:06:37.173 INFO:teuthology.orchestra.run.smithi176.stdout:3533 read 0x3a3704b thru 0x3a45742 (0xe6f8 bytes) 2024-09-14T09:06:37.173 INFO:teuthology.orchestra.run.smithi176.stdout:3534 write 0x53736aa thru 0x5377892 (0x41e9 bytes) 2024-09-14T09:06:37.181 INFO:teuthology.orchestra.run.smithi176.stdout:3535 punch from 0xc640ec0 to 0xc64987f, (0x89bf bytes) 2024-09-14T09:06:37.181 INFO:teuthology.orchestra.run.smithi176.stdout:3536 write 0x7657a7c thru 0x7664c6c (0xd1f1 bytes) 2024-09-14T09:06:37.198 INFO:teuthology.orchestra.run.smithi176.stdout:3539 write 0x8d62e5a thru 0x8d670c9 (0x4270 bytes) 2024-09-14T09:06:37.208 INFO:teuthology.orchestra.run.smithi176.stdout:3540 write 0x31bcb49 thru 0x31ca32f (0xd7e7 bytes) 2024-09-14T09:06:37.212 INFO:teuthology.orchestra.run.smithi176.stdout:3541 punch from 0x67c7768 to 0x67cc866, (0x50fe bytes) 2024-09-14T09:06:37.212 INFO:teuthology.orchestra.run.smithi176.stdout:3542 writesame 0x952da99 thru 0x9536cef data_size 0x1(0x9257 bytes) 2024-09-14T09:06:37.253 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:37.251+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:37.275 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:37.275 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:37.289 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:06:37.291 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is set 2024-09-14T09:06:37.300 INFO:teuthology.orchestra.run.smithi176.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T09:06:37.303 INFO:teuthology.orchestra.run.smithi176.stdout:3543 write 0x38d0d95 thru 0x38dac4f (0x9ebb bytes) 2024-09-14T09:06:37.322 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T09:06:37.370 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:37.371+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:37.409 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:37.409 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:37.423 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:06:37.466 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:37.467+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:37.555 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:37.555+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:37.584 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:37.584 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:37.597 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:06:37.656 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:37.655+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:37.729 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:37.729 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:37.744 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:06:37.757 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:37.755+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:37.858 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:37.855+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:37.864 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:37.864 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:37.878 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:06:37.959 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:37.963+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:37.996 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:37.996 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:38.010 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:06:38.095 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:38.099+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:38.162 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:38.159+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:38.173 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:38.173 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:38.187 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:06:38.262 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:38.259+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:38.263 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:06:38.303 INFO:teuthology.orchestra.run.smithi176.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T09:06:38.334 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T09:06:38.359 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:38.359 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:38.363 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:38.363+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:38.373 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:06:38.421 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:38.421 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:38.438 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:38.438 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:38.455 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:38.455 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:38.463 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:38.463+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:38.472 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:38.472 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:38.490 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:38.490 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:38.510 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:38.510 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:38.530 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:38.530 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:38.541 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:38.541 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:38.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:38.541 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:38.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.541 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49334 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:38.541 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:38.541 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 27.996192213, 2024-09-14T09:06:38.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072227886000000005, 2024-09-14T09:06:38.542 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.542 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.542 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:38.542 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:38.542 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:38.542 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:38.542 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49334 2024-09-14T09:06:38.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.542 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:38.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:38.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546240+0000", 2024-09-14T09:06:38.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8899999999999999e-07 2024-09-14T09:06:38.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546243+0000", 2024-09-14T09:06:38.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.503e-06 2024-09-14T09:06:38.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546253+0000", 2024-09-14T09:06:38.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0210999999999999e-05 2024-09-14T09:06:38.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546267+0000", 2024-09-14T09:06:38.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4183e-05 2024-09-14T09:06:38.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618449+0000", 2024-09-14T09:06:38.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072181828000000003 2024-09-14T09:06:38.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618467+0000", 2024-09-14T09:06:38.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8471999999999999e-05 2024-09-14T09:06:38.545 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.545 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.545 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.545 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49335 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:38.545 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:38.545 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 27.996156062000001, 2024-09-14T09:06:38.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072255640999999995, 2024-09-14T09:06:38.545 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.546 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.546 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:38.546 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:38.546 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:38.546 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:38.546 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49335 2024-09-14T09:06:38.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.546 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:38.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:38.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546276+0000", 2024-09-14T09:06:38.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9999999999999997e-07 2024-09-14T09:06:38.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546279+0000", 2024-09-14T09:06:38.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.695e-06 2024-09-14T09:06:38.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546288+0000", 2024-09-14T09:06:38.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7110000000000001e-06 2024-09-14T09:06:38.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546294+0000", 2024-09-14T09:06:38.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9830000000000001e-06 2024-09-14T09:06:38.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618521+0000", 2024-09-14T09:06:38.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072226948999999999 2024-09-14T09:06:38.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618531+0000", 2024-09-14T09:06:38.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0603e-05 2024-09-14T09:06:38.549 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.549 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.549 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.549 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49336 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:38.549 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:38.549 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 27.996130159, 2024-09-14T09:06:38.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072292223000000003, 2024-09-14T09:06:38.549 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.549 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.549 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:38.549 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:38.549 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:38.550 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:38.550 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49336 2024-09-14T09:06:38.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.550 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:38.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:38.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546302+0000", 2024-09-14T09:06:38.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4000000000000001e-07 2024-09-14T09:06:38.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546305+0000", 2024-09-14T09:06:38.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3539999999999998e-06 2024-09-14T09:06:38.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546315+0000", 2024-09-14T09:06:38.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0431999999999999e-05 2024-09-14T09:06:38.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546321+0000", 2024-09-14T09:06:38.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7620000000000001e-06 2024-09-14T09:06:38.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618583+0000", 2024-09-14T09:06:38.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072262046999999996 2024-09-14T09:06:38.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618594+0000", 2024-09-14T09:06:38.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0988000000000001e-05 2024-09-14T09:06:38.553 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.553 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.553 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.553 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49337 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:38.553 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:38.553 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 27.996082545, 2024-09-14T09:06:38.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072294774000000006, 2024-09-14T09:06:38.553 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.553 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.553 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:38.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:38.554 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:38.554 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:38.554 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49337 2024-09-14T09:06:38.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.554 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:38.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546349+0000", 2024-09-14T09:06:38.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546350+0000", 2024-09-14T09:06:38.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4600000000000004e-07 2024-09-14T09:06:38.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546381+0000", 2024-09-14T09:06:38.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0902000000000003e-05 2024-09-14T09:06:38.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546412+0000", 2024-09-14T09:06:38.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0825999999999999e-05 2024-09-14T09:06:38.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546438+0000", 2024-09-14T09:06:38.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6410999999999998e-05 2024-09-14T09:06:38.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618637+0000", 2024-09-14T09:06:38.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072198907000000007 2024-09-14T09:06:38.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618644+0000", 2024-09-14T09:06:38.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9820000000000002e-06 2024-09-14T09:06:38.557 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.557 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.557 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.557 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49338 3.1d 3.79f69a7d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:38.557 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:38.557 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 27.995968518000002, 2024-09-14T09:06:38.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072233250999999998, 2024-09-14T09:06:38.558 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.558 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.558 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:38.558 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:38.558 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:38.558 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:38.558 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49338 2024-09-14T09:06:38.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.558 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:38.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546463+0000", 2024-09-14T09:06:38.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546465+0000", 2024-09-14T09:06:38.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.455e-06 2024-09-14T09:06:38.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546481+0000", 2024-09-14T09:06:38.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5834000000000001e-05 2024-09-14T09:06:38.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546498+0000", 2024-09-14T09:06:38.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6937000000000001e-05 2024-09-14T09:06:38.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.546508+0000", 2024-09-14T09:06:38.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0446e-05 2024-09-14T09:06:38.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618685+0000", 2024-09-14T09:06:38.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072177280999999996 2024-09-14T09:06:38.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.618697+0000", 2024-09-14T09:06:38.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1297999999999999e-05 2024-09-14T09:06:38.561 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.561 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.561 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.561 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:50478 3.1d 3:b8a2eaf4:::rbd_data.115e5dc498ac.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T09:06:38.561 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:38.562 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 26.492791993000001, 2024-09-14T09:06:38.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.075023792000000006, 2024-09-14T09:06:38.562 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.562 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.562 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:38.562 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:38.562 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:38.562 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:38.562 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 50478 2024-09-14T09:06:38.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.562 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:38.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049640+0000", 2024-09-14T09:06:38.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049641+0000", 2024-09-14T09:06:38.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6270000000000001e-06 2024-09-14T09:06:38.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049649+0000", 2024-09-14T09:06:38.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6279999999999996e-06 2024-09-14T09:06:38.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049667+0000", 2024-09-14T09:06:38.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-09-14T09:06:38.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049677+0000", 2024-09-14T09:06:38.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0509000000000001e-05 2024-09-14T09:06:38.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049751+0000", 2024-09-14T09:06:38.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3322e-05 2024-09-14T09:06:38.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:38.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.049830+0000", 2024-09-14T09:06:38.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9932000000000007e-05 2024-09-14T09:06:38.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:38.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.053203+0000", 2024-09-14T09:06:38.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003372398 2024-09-14T09:06:38.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:38.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124542+0000", 2024-09-14T09:06:38.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.071339105999999999 2024-09-14T09:06:38.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:38.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124562+0000", 2024-09-14T09:06:38.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9649000000000001e-05 2024-09-14T09:06:38.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:38.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124616+0000", 2024-09-14T09:06:38.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4642999999999998e-05 2024-09-14T09:06:38.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:12.124664+0000", 2024-09-14T09:06:38.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7342000000000002e-05 2024-09-14T09:06:38.567 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.567 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.567 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.567 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65035 3.1d 3:bb585d47:::rbd_data.115e21c31fcf.0000000000000034:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:06:38.567 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.068842+0000", 2024-09-14T09:06:38.567 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 17.473589758999999, 2024-09-14T09:06:38.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.094968451999999995, 2024-09-14T09:06:38.567 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.567 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.568 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:38.568 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:38.568 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:38.568 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:38.568 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65035 2024-09-14T09:06:38.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.568 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068842+0000", 2024-09-14T09:06:38.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068842+0000", 2024-09-14T09:06:38.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068843+0000", 2024-09-14T09:06:38.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.221e-06 2024-09-14T09:06:38.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068849+0000", 2024-09-14T09:06:38.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4659999999999998e-06 2024-09-14T09:06:38.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068861+0000", 2024-09-14T09:06:38.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2204e-05 2024-09-14T09:06:38.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068871+0000", 2024-09-14T09:06:38.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0318e-05 2024-09-14T09:06:38.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.068942+0000", 2024-09-14T09:06:38.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0438999999999994e-05 2024-09-14T09:06:38.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:38.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.069005+0000", 2024-09-14T09:06:38.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3076000000000004e-05 2024-09-14T09:06:38.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:38.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.074957+0000", 2024-09-14T09:06:38.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059524549999999997 2024-09-14T09:06:38.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:38.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163479+0000", 2024-09-14T09:06:38.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.088521676999999993 2024-09-14T09:06:38.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:38.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163633+0000", 2024-09-14T09:06:38.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000154301 2024-09-14T09:06:38.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:38.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163733+0000", 2024-09-14T09:06:38.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100089 2024-09-14T09:06:38.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163811+0000", 2024-09-14T09:06:38.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7206000000000006e-05 2024-09-14T09:06:38.572 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.573 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.573 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.573 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:65035 3.1d e65/23)", 2024-09-14T09:06:38.573 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.087665+0000", 2024-09-14T09:06:38.573 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 17.454766848999999, 2024-09-14T09:06:38.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.076160555000000005, 2024-09-14T09:06:38.573 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.573 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.573 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:38.573 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087665+0000", 2024-09-14T09:06:38.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087665+0000", 2024-09-14T09:06:38.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087667+0000", 2024-09-14T09:06:38.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.818e-06 2024-09-14T09:06:38.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087673+0000", 2024-09-14T09:06:38.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9259999999999997e-06 2024-09-14T09:06:38.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087692+0000", 2024-09-14T09:06:38.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9715e-05 2024-09-14T09:06:38.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.087703+0000", 2024-09-14T09:06:38.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.094e-05 2024-09-14T09:06:38.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163595+0000", 2024-09-14T09:06:38.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.075892121000000007 2024-09-14T09:06:38.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:38.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163621+0000", 2024-09-14T09:06:38.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5383000000000001e-05 2024-09-14T09:06:38.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.163826+0000", 2024-09-14T09:06:38.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020465199999999999 2024-09-14T09:06:38.577 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.577 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.577 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.577 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:71031 3.1f 3:f9e41410:::rbd_data.115ebd4a9f36.0000000000000003:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:06:38.577 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:29.576817+0000", 2024-09-14T09:06:38.577 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.9656143719999992, 2024-09-14T09:06:38.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.087207509000000002, 2024-09-14T09:06:38.577 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.577 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.577 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:38.578 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:38.578 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:38.578 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:38.578 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 71031 2024-09-14T09:06:38.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.578 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.576817+0000", 2024-09-14T09:06:38.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.576817+0000", 2024-09-14T09:06:38.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.576820+0000", 2024-09-14T09:06:38.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.283e-06 2024-09-14T09:06:38.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.576832+0000", 2024-09-14T09:06:38.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1843e-05 2024-09-14T09:06:38.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.576861+0000", 2024-09-14T09:06:38.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9043e-05 2024-09-14T09:06:38.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.576876+0000", 2024-09-14T09:06:38.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5293000000000001e-05 2024-09-14T09:06:38.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.577000+0000", 2024-09-14T09:06:38.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000124473 2024-09-14T09:06:38.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:38.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.577126+0000", 2024-09-14T09:06:38.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012597199999999999 2024-09-14T09:06:38.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-09-14T09:06:38.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.585227+0000", 2024-09-14T09:06:38.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0081002810000000008 2024-09-14T09:06:38.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:38.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.663671+0000", 2024-09-14T09:06:38.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.078444609999999998 2024-09-14T09:06:38.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:38.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.663867+0000", 2024-09-14T09:06:38.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019541099999999999 2024-09-14T09:06:38.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:38.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.663960+0000", 2024-09-14T09:06:38.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3191999999999998e-05 2024-09-14T09:06:38.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.664025+0000", 2024-09-14T09:06:38.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5108000000000001e-05 2024-09-14T09:06:38.582 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.582 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.582 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.583 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:71067 3.1d 3:b9bb9b2f:::rbd_data.115ebd4a9f36.0000000000000006:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:06:38.583 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:29.920442+0000", 2024-09-14T09:06:38.583 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.6219897799999998, 2024-09-14T09:06:38.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13139427400000001, 2024-09-14T09:06:38.583 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.583 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.583 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:38.583 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:38.583 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:38.583 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:38.583 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 71067 2024-09-14T09:06:38.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.584 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920442+0000", 2024-09-14T09:06:38.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920442+0000", 2024-09-14T09:06:38.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920445+0000", 2024-09-14T09:06:38.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7099999999999999e-06 2024-09-14T09:06:38.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920461+0000", 2024-09-14T09:06:38.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6436e-05 2024-09-14T09:06:38.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920492+0000", 2024-09-14T09:06:38.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1069999999999999e-05 2024-09-14T09:06:38.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920508+0000", 2024-09-14T09:06:38.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5463000000000001e-05 2024-09-14T09:06:38.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920625+0000", 2024-09-14T09:06:38.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116932 2024-09-14T09:06:38.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:38.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920766+0000", 2024-09-14T09:06:38.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000141092 2024-09-14T09:06:38.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:38.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.929373+0000", 2024-09-14T09:06:38.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086074510000000003 2024-09-14T09:06:38.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:38.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:30.051654+0000", 2024-09-14T09:06:38.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.12228082999999999 2024-09-14T09:06:38.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:38.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:30.051711+0000", 2024-09-14T09:06:38.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7108000000000003e-05 2024-09-14T09:06:38.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:38.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:30.051779+0000", 2024-09-14T09:06:38.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7696999999999999e-05 2024-09-14T09:06:38.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:30.051836+0000", 2024-09-14T09:06:38.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7485000000000003e-05 2024-09-14T09:06:38.588 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.588 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.589 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.589 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:71067 3.1d e71/23 ondisk, result = 0)", 2024-09-14T09:06:38.589 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:29.966416+0000", 2024-09-14T09:06:38.589 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 8.5760161060000009, 2024-09-14T09:06:38.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.085428365000000006, 2024-09-14T09:06:38.589 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.589 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.589 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:38.589 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.966416+0000", 2024-09-14T09:06:38.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.966416+0000", 2024-09-14T09:06:38.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.966417+0000", 2024-09-14T09:06:38.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.226e-06 2024-09-14T09:06:38.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.966423+0000", 2024-09-14T09:06:38.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7300000000000002e-06 2024-09-14T09:06:38.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.966443+0000", 2024-09-14T09:06:38.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.022e-05 2024-09-14T09:06:38.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.966453+0000", 2024-09-14T09:06:38.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0468e-05 2024-09-14T09:06:38.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:30.051691+0000", 2024-09-14T09:06:38.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.085237948999999993 2024-09-14T09:06:38.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:38.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:30.051704+0000", 2024-09-14T09:06:38.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2737e-05 2024-09-14T09:06:38.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:30.051844+0000", 2024-09-14T09:06:38.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000140035 2024-09-14T09:06:38.592 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.592 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.593 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.593 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:74301 3.1c 3:3af796bc:::rbd_data.115ea8d131ff.0000000000000044:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:38.593 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:35.806972+0000", 2024-09-14T09:06:38.593 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 2.7354597869999999, 2024-09-14T09:06:38.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.080706138999999996, 2024-09-14T09:06:38.593 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.593 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.593 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:38.593 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:38.593 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:38.593 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:38.593 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 74301 2024-09-14T09:06:38.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.594 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.806972+0000", 2024-09-14T09:06:38.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.806972+0000", 2024-09-14T09:06:38.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.806975+0000", 2024-09-14T09:06:38.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4899999999999999e-06 2024-09-14T09:06:38.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.806988+0000", 2024-09-14T09:06:38.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2999999999999999e-05 2024-09-14T09:06:38.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.807024+0000", 2024-09-14T09:06:38.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6476000000000001e-05 2024-09-14T09:06:38.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.807054+0000", 2024-09-14T09:06:38.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.97e-05 2024-09-14T09:06:38.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.807222+0000", 2024-09-14T09:06:38.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016833199999999999 2024-09-14T09:06:38.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:38.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.807437+0000", 2024-09-14T09:06:38.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021451799999999999 2024-09-14T09:06:38.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:38.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.811691+0000", 2024-09-14T09:06:38.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042546930000000004 2024-09-14T09:06:38.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:38.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.887505+0000", 2024-09-14T09:06:38.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.075813494999999995 2024-09-14T09:06:38.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:38.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.887531+0000", 2024-09-14T09:06:38.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6259e-05 2024-09-14T09:06:38.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:38.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.887609+0000", 2024-09-14T09:06:38.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8312000000000005e-05 2024-09-14T09:06:38.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.887678+0000", 2024-09-14T09:06:38.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8863999999999997e-05 2024-09-14T09:06:38.598 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.598 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.598 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.599 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:75105 3.1c 3:39056f03:::rbd_data.115ebd4a9f36.0000000000000000:1b [read 16154624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:38.599 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:36.096348+0000", 2024-09-14T09:06:38.599 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 2.4460833590000002, 2024-09-14T09:06:38.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072329352999999999, 2024-09-14T09:06:38.599 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.599 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.599 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:38.599 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:38.599 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:38.599 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:38.599 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 75105 2024-09-14T09:06:38.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.599 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096348+0000", 2024-09-14T09:06:38.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096348+0000", 2024-09-14T09:06:38.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096349+0000", 2024-09-14T09:06:38.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6599999999999995e-07 2024-09-14T09:06:38.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096352+0000", 2024-09-14T09:06:38.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2799999999999999e-06 2024-09-14T09:06:38.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096361+0000", 2024-09-14T09:06:38.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8340000000000005e-06 2024-09-14T09:06:38.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096368+0000", 2024-09-14T09:06:38.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:06:38.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.168383+0000", 2024-09-14T09:06:38.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072014395999999994 2024-09-14T09:06:38.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:38.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.168530+0000", 2024-09-14T09:06:38.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014685499999999999 2024-09-14T09:06:38.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.168678+0000", 2024-09-14T09:06:38.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000148302 2024-09-14T09:06:38.602 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.602 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.603 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.603 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:75106 3.1c 3:39056f03:::rbd_data.115ebd4a9f36.0000000000000000:1b [read 16236544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:38.603 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:36.096377+0000", 2024-09-14T09:06:38.603 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 2.4460547930000001, 2024-09-14T09:06:38.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072663059000000002, 2024-09-14T09:06:38.603 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.603 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.603 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:38.603 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:38.603 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:38.603 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:38.603 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 75106 2024-09-14T09:06:38.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.604 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096377+0000", 2024-09-14T09:06:38.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096377+0000", 2024-09-14T09:06:38.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096378+0000", 2024-09-14T09:06:38.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5700000000000002e-07 2024-09-14T09:06:38.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096381+0000", 2024-09-14T09:06:38.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9330000000000001e-06 2024-09-14T09:06:38.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096389+0000", 2024-09-14T09:06:38.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7199999999999995e-06 2024-09-14T09:06:38.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096396+0000", 2024-09-14T09:06:38.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9299999999999997e-06 2024-09-14T09:06:38.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.168739+0000", 2024-09-14T09:06:38.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072342397000000003 2024-09-14T09:06:38.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:38.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.168897+0000", 2024-09-14T09:06:38.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000158367 2024-09-14T09:06:38.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.169040+0000", 2024-09-14T09:06:38.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000142955 2024-09-14T09:06:38.607 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.607 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.607 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.607 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:75107 3.1c 3:39056f03:::rbd_data.115ebd4a9f36.0000000000000000:1b [read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:38.607 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:36.096418+0000", 2024-09-14T09:06:38.607 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 2.4460141329999998, 2024-09-14T09:06:38.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.073035971000000005, 2024-09-14T09:06:38.607 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.608 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.608 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:38.608 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:38.608 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:38.608 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:38.608 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 75107 2024-09-14T09:06:38.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.608 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096418+0000", 2024-09-14T09:06:38.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096418+0000", 2024-09-14T09:06:38.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096419+0000", 2024-09-14T09:06:38.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5400000000000001e-06 2024-09-14T09:06:38.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096426+0000", 2024-09-14T09:06:38.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6599999999999998e-06 2024-09-14T09:06:38.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096448+0000", 2024-09-14T09:06:38.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1798999999999999e-05 2024-09-14T09:06:38.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096461+0000", 2024-09-14T09:06:38.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3657e-05 2024-09-14T09:06:38.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.169155+0000", 2024-09-14T09:06:38.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072693174999999999 2024-09-14T09:06:38.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:38.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.169296+0000", 2024-09-14T09:06:38.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000141822 2024-09-14T09:06:38.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.169454+0000", 2024-09-14T09:06:38.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015731799999999999 2024-09-14T09:06:38.612 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.612 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.612 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.612 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:75108 3.1c 3:39056f03:::rbd_data.115ebd4a9f36.0000000000000000:1b [read 16400384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:38.612 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:36.096484+0000", 2024-09-14T09:06:38.612 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 2.4459474370000001, 2024-09-14T09:06:38.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.073346651999999998, 2024-09-14T09:06:38.612 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.612 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.612 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:38.612 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:38.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:38.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:38.613 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 75108 2024-09-14T09:06:38.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.613 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096484+0000", 2024-09-14T09:06:38.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096484+0000", 2024-09-14T09:06:38.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096486+0000", 2024-09-14T09:06:38.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.207e-06 2024-09-14T09:06:38.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096491+0000", 2024-09-14T09:06:38.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3700000000000003e-06 2024-09-14T09:06:38.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096525+0000", 2024-09-14T09:06:38.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3809000000000001e-05 2024-09-14T09:06:38.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096542+0000", 2024-09-14T09:06:38.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7317e-05 2024-09-14T09:06:38.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.169539+0000", 2024-09-14T09:06:38.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.072996762000000007 2024-09-14T09:06:38.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:38.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.169685+0000", 2024-09-14T09:06:38.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000146568 2024-09-14T09:06:38.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.169831+0000", 2024-09-14T09:06:38.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014561899999999999 2024-09-14T09:06:38.616 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.616 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.616 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.616 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:75109 3.1c 3:39056f03:::rbd_data.115ebd4a9f36.0000000000000000:1b [read 16482304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:38.616 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:36.096585+0000", 2024-09-14T09:06:38.617 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 2.4458464279999999, 2024-09-14T09:06:38.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.073600077, 2024-09-14T09:06:38.617 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.617 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.617 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:38.617 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:38.617 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:38.617 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:38.617 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 75109 2024-09-14T09:06:38.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.617 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096585+0000", 2024-09-14T09:06:38.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096585+0000", 2024-09-14T09:06:38.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096587+0000", 2024-09-14T09:06:38.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2270000000000001e-06 2024-09-14T09:06:38.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096593+0000", 2024-09-14T09:06:38.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.156e-06 2024-09-14T09:06:38.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096611+0000", 2024-09-14T09:06:38.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8547e-05 2024-09-14T09:06:38.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096647+0000", 2024-09-14T09:06:38.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5392999999999998e-05 2024-09-14T09:06:38.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.169922+0000", 2024-09-14T09:06:38.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.073275356 2024-09-14T09:06:38.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:38.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.170053+0000", 2024-09-14T09:06:38.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000131032 2024-09-14T09:06:38.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.170185+0000", 2024-09-14T09:06:38.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000132366 2024-09-14T09:06:38.620 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.620 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.620 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.621 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:75110 3.1c 3:39056f03:::rbd_data.115ebd4a9f36.0000000000000000:1b [read 16564224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:38.621 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:36.096667+0000", 2024-09-14T09:06:38.621 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 2.4457645320000001, 2024-09-14T09:06:38.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.073894445000000003, 2024-09-14T09:06:38.621 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.621 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.621 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:38.621 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:38.621 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:38.621 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:38.621 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 75110 2024-09-14T09:06:38.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.621 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096667+0000", 2024-09-14T09:06:38.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096667+0000", 2024-09-14T09:06:38.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096669+0000", 2024-09-14T09:06:38.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.403e-06 2024-09-14T09:06:38.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096674+0000", 2024-09-14T09:06:38.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3669999999999996e-06 2024-09-14T09:06:38.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096697+0000", 2024-09-14T09:06:38.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3413000000000001e-05 2024-09-14T09:06:38.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096712+0000", 2024-09-14T09:06:38.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4197e-05 2024-09-14T09:06:38.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.170271+0000", 2024-09-14T09:06:38.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.073559711999999999 2024-09-14T09:06:38.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:38.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.170419+0000", 2024-09-14T09:06:38.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014766900000000001 2024-09-14T09:06:38.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.170562+0000", 2024-09-14T09:06:38.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014268399999999999 2024-09-14T09:06:38.624 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.624 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.625 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.625 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:75111 3.1c 3:39056f03:::rbd_data.115ebd4a9f36.0000000000000000:1b [read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:38.625 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:36.096730+0000", 2024-09-14T09:06:38.625 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 2.4457014519999998, 2024-09-14T09:06:38.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.074172582000000001, 2024-09-14T09:06:38.625 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.625 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.625 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:38.625 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:38.625 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:38.625 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:38.626 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 75111 2024-09-14T09:06:38.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.626 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096730+0000", 2024-09-14T09:06:38.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096730+0000", 2024-09-14T09:06:38.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096732+0000", 2024-09-14T09:06:38.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3200000000000001e-06 2024-09-14T09:06:38.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096737+0000", 2024-09-14T09:06:38.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3730000000000001e-06 2024-09-14T09:06:38.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096754+0000", 2024-09-14T09:06:38.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.713e-05 2024-09-14T09:06:38.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096768+0000", 2024-09-14T09:06:38.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3883000000000001e-05 2024-09-14T09:06:38.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.170646+0000", 2024-09-14T09:06:38.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.073878031999999996 2024-09-14T09:06:38.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:38.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.170774+0000", 2024-09-14T09:06:38.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000127859 2024-09-14T09:06:38.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.170903+0000", 2024-09-14T09:06:38.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128985 2024-09-14T09:06:38.629 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.629 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.629 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.629 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:75112 3.1c 3:39056f03:::rbd_data.115ebd4a9f36.0000000000000000:1b [read 16728064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:38.629 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:36.096788+0000", 2024-09-14T09:06:38.629 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 2.445643929, 2024-09-14T09:06:38.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.074470693000000004, 2024-09-14T09:06:38.629 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:38.629 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:38.629 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:38.629 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:38.630 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:38.630 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:38.630 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 75112 2024-09-14T09:06:38.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.630 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:38.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:38.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096788+0000", 2024-09-14T09:06:38.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:38.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096788+0000", 2024-09-14T09:06:38.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:38.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:38.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096789+0000", 2024-09-14T09:06:38.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.246e-06 2024-09-14T09:06:38.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:38.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096794+0000", 2024-09-14T09:06:38.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0030000000000002e-06 2024-09-14T09:06:38.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:38.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096812+0000", 2024-09-14T09:06:38.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7632999999999999e-05 2024-09-14T09:06:38.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:38.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.096826+0000", 2024-09-14T09:06:38.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3787e-05 2024-09-14T09:06:38.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:38.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.170961+0000", 2024-09-14T09:06:38.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.074135802000000001 2024-09-14T09:06:38.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:38.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.171111+0000", 2024-09-14T09:06:38.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000149185 2024-09-14T09:06:38.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:38.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:38.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:38.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:36.171259+0000", 2024-09-14T09:06:38.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014803700000000001 2024-09-14T09:06:38.633 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.633 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.633 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.633 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:38.633 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:38.633 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:38.634 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:38.635 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:06:38.636 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:38.636 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:38.635+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:38.736 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:38.739+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:38.763 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:38.763 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:38.763 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:38.763 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:38.777 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:06:38.863 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:38.867+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:38.938 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:38.939+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:38.945 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:38.945 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:38.945 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:38.945 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:38.945 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:38.959 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:06:39.047 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:39.051+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:39.131 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:39.131 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:39.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:39.131 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:39.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.131 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19520 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:39.132 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475212+0000", 2024-09-14T09:06:39.132 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.657760484000001, 2024-09-14T09:06:39.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067435885000000001, 2024-09-14T09:06:39.132 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.132 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.132 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:39.132 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.132 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.132 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.132 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19520 2024-09-14T09:06:39.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.132 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475212+0000", 2024-09-14T09:06:39.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475212+0000", 2024-09-14T09:06:39.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475213+0000", 2024-09-14T09:06:39.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0440000000000001e-06 2024-09-14T09:06:39.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475218+0000", 2024-09-14T09:06:39.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1930000000000001e-06 2024-09-14T09:06:39.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475237+0000", 2024-09-14T09:06:39.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9003000000000001e-05 2024-09-14T09:06:39.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475251+0000", 2024-09-14T09:06:39.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3720000000000001e-05 2024-09-14T09:06:39.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.541342+0000", 2024-09-14T09:06:39.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066091227000000002 2024-09-14T09:06:39.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.541492+0000", 2024-09-14T09:06:39.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000149221 2024-09-14T09:06:39.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542648+0000", 2024-09-14T09:06:39.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011564769999999999 2024-09-14T09:06:39.135 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.136 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.136 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.136 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19530 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:39.136 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475326+0000", 2024-09-14T09:06:39.136 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.657646301, 2024-09-14T09:06:39.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067030846000000005, 2024-09-14T09:06:39.136 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.136 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.136 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:39.136 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.136 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.137 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.137 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19530 2024-09-14T09:06:39.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.137 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475326+0000", 2024-09-14T09:06:39.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475326+0000", 2024-09-14T09:06:39.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475328+0000", 2024-09-14T09:06:39.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.133e-06 2024-09-14T09:06:39.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475358+0000", 2024-09-14T09:06:39.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0253e-05 2024-09-14T09:06:39.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475378+0000", 2024-09-14T09:06:39.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.033e-05 2024-09-14T09:06:39.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475405+0000", 2024-09-14T09:06:39.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6568999999999999e-05 2024-09-14T09:06:39.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.541989+0000", 2024-09-14T09:06:39.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066583956999999999 2024-09-14T09:06:39.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542215+0000", 2024-09-14T09:06:39.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022592200000000001 2024-09-14T09:06:39.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542357+0000", 2024-09-14T09:06:39.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014268199999999999 2024-09-14T09:06:39.140 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.140 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.140 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.140 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19531 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15073280~11919] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:39.140 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475425+0000", 2024-09-14T09:06:39.140 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.657548005999999, 2024-09-14T09:06:39.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067651759000000006, 2024-09-14T09:06:39.141 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.141 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.141 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:39.141 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.141 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.141 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.141 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19531 2024-09-14T09:06:39.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.141 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475425+0000", 2024-09-14T09:06:39.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475425+0000", 2024-09-14T09:06:39.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475426+0000", 2024-09-14T09:06:39.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2440000000000001e-06 2024-09-14T09:06:39.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475465+0000", 2024-09-14T09:06:39.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8643000000000003e-05 2024-09-14T09:06:39.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475485+0000", 2024-09-14T09:06:39.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0645999999999999e-05 2024-09-14T09:06:39.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475499+0000", 2024-09-14T09:06:39.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3312999999999999e-05 2024-09-14T09:06:39.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542389+0000", 2024-09-14T09:06:39.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066890885999999997 2024-09-14T09:06:39.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542481+0000", 2024-09-14T09:06:39.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1481e-05 2024-09-14T09:06:39.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543076+0000", 2024-09-14T09:06:39.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00059554599999999997 2024-09-14T09:06:39.144 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.144 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.144 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.144 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19532 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15335424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:39.144 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475521+0000", 2024-09-14T09:06:39.145 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.657451360000003, 2024-09-14T09:06:39.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067895159999999996, 2024-09-14T09:06:39.145 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.145 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.145 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:39.145 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.145 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.145 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.145 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19532 2024-09-14T09:06:39.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.145 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475521+0000", 2024-09-14T09:06:39.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475521+0000", 2024-09-14T09:06:39.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475523+0000", 2024-09-14T09:06:39.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.277e-06 2024-09-14T09:06:39.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475528+0000", 2024-09-14T09:06:39.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8629999999999996e-06 2024-09-14T09:06:39.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475546+0000", 2024-09-14T09:06:39.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7750000000000001e-05 2024-09-14T09:06:39.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475560+0000", 2024-09-14T09:06:39.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3763e-05 2024-09-14T09:06:39.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543144+0000", 2024-09-14T09:06:39.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067584489999999997 2024-09-14T09:06:39.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543261+0000", 2024-09-14T09:06:39.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011696699999999999 2024-09-14T09:06:39.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543416+0000", 2024-09-14T09:06:39.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015505 2024-09-14T09:06:39.148 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.148 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.149 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.149 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19533 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:39.149 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475578+0000", 2024-09-14T09:06:39.149 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.657394216999997, 2024-09-14T09:06:39.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068124984999999999, 2024-09-14T09:06:39.149 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.149 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.149 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:39.149 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.149 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.149 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.149 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19533 2024-09-14T09:06:39.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.150 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475578+0000", 2024-09-14T09:06:39.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475578+0000", 2024-09-14T09:06:39.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475580+0000", 2024-09-14T09:06:39.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.08e-06 2024-09-14T09:06:39.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475585+0000", 2024-09-14T09:06:39.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4970000000000001e-06 2024-09-14T09:06:39.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475602+0000", 2024-09-14T09:06:39.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7459999999999999e-05 2024-09-14T09:06:39.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475615+0000", 2024-09-14T09:06:39.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2339e-05 2024-09-14T09:06:39.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543466+0000", 2024-09-14T09:06:39.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067850745000000004 2024-09-14T09:06:39.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543577+0000", 2024-09-14T09:06:39.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000111908 2024-09-14T09:06:39.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543703+0000", 2024-09-14T09:06:39.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012595600000000001 2024-09-14T09:06:39.153 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.153 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.153 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.153 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19534 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:39.153 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475634+0000", 2024-09-14T09:06:39.153 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.657338441, 2024-09-14T09:06:39.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068358706000000005, 2024-09-14T09:06:39.153 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.153 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.153 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:39.153 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.154 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.154 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.154 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19534 2024-09-14T09:06:39.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.154 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475634+0000", 2024-09-14T09:06:39.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475634+0000", 2024-09-14T09:06:39.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475635+0000", 2024-09-14T09:06:39.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.187e-06 2024-09-14T09:06:39.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475641+0000", 2024-09-14T09:06:39.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5629999999999999e-06 2024-09-14T09:06:39.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475659+0000", 2024-09-14T09:06:39.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7649999999999999e-05 2024-09-14T09:06:39.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475671+0000", 2024-09-14T09:06:39.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2583e-05 2024-09-14T09:06:39.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543748+0000", 2024-09-14T09:06:39.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068076740999999996 2024-09-14T09:06:39.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543855+0000", 2024-09-14T09:06:39.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010676800000000001 2024-09-14T09:06:39.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543993+0000", 2024-09-14T09:06:39.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138214 2024-09-14T09:06:39.157 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.157 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.157 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.157 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19535 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:39.157 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475691+0000", 2024-09-14T09:06:39.157 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.657281431000001, 2024-09-14T09:06:39.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068607583999999999, 2024-09-14T09:06:39.158 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.158 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.158 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:39.158 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.158 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.158 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.158 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19535 2024-09-14T09:06:39.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.158 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475691+0000", 2024-09-14T09:06:39.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475691+0000", 2024-09-14T09:06:39.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475693+0000", 2024-09-14T09:06:39.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6270000000000001e-06 2024-09-14T09:06:39.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475699+0000", 2024-09-14T09:06:39.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9059999999999999e-06 2024-09-14T09:06:39.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475717+0000", 2024-09-14T09:06:39.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-09-14T09:06:39.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475730+0000", 2024-09-14T09:06:39.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.308e-05 2024-09-14T09:06:39.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544042+0000", 2024-09-14T09:06:39.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068312791999999997 2024-09-14T09:06:39.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544168+0000", 2024-09-14T09:06:39.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012549699999999999 2024-09-14T09:06:39.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544299+0000", 2024-09-14T09:06:39.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013094199999999999 2024-09-14T09:06:39.161 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.161 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.161 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.162 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19536 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:39.162 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475749+0000", 2024-09-14T09:06:39.162 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.657224151999998, 2024-09-14T09:06:39.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068917143, 2024-09-14T09:06:39.162 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.162 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.162 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:39.162 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.162 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.162 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.162 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19536 2024-09-14T09:06:39.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.163 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475749+0000", 2024-09-14T09:06:39.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475749+0000", 2024-09-14T09:06:39.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475750+0000", 2024-09-14T09:06:39.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0330000000000001e-06 2024-09-14T09:06:39.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475755+0000", 2024-09-14T09:06:39.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9740000000000001e-06 2024-09-14T09:06:39.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475772+0000", 2024-09-14T09:06:39.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7750000000000001e-05 2024-09-14T09:06:39.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475802+0000", 2024-09-14T09:06:39.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0190000000000001e-05 2024-09-14T09:06:39.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544345+0000", 2024-09-14T09:06:39.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068542140000000001 2024-09-14T09:06:39.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544454+0000", 2024-09-14T09:06:39.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109447 2024-09-14T09:06:39.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544666+0000", 2024-09-14T09:06:39.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000211609 2024-09-14T09:06:39.166 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.166 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.166 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.166 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19537 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:39.166 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475822+0000", 2024-09-14T09:06:39.166 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.657150195, 2024-09-14T09:06:39.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069185398999999995, 2024-09-14T09:06:39.166 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.166 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.167 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:39.167 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.167 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.167 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.167 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19537 2024-09-14T09:06:39.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.167 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475822+0000", 2024-09-14T09:06:39.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475822+0000", 2024-09-14T09:06:39.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475824+0000", 2024-09-14T09:06:39.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2559999999999999e-06 2024-09-14T09:06:39.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475829+0000", 2024-09-14T09:06:39.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6840000000000001e-06 2024-09-14T09:06:39.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475848+0000", 2024-09-14T09:06:39.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8583000000000001e-05 2024-09-14T09:06:39.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475861+0000", 2024-09-14T09:06:39.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2802999999999999e-05 2024-09-14T09:06:39.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544716+0000", 2024-09-14T09:06:39.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068855036999999994 2024-09-14T09:06:39.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544827+0000", 2024-09-14T09:06:39.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011113200000000001 2024-09-14T09:06:39.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545008+0000", 2024-09-14T09:06:39.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018090399999999999 2024-09-14T09:06:39.170 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.170 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.170 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.170 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19538 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:39.170 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475880+0000", 2024-09-14T09:06:39.170 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.657092519999999, 2024-09-14T09:06:39.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069740072, 2024-09-14T09:06:39.171 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.171 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.171 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:39.171 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.171 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.171 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.171 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19538 2024-09-14T09:06:39.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.171 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475880+0000", 2024-09-14T09:06:39.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475880+0000", 2024-09-14T09:06:39.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475881+0000", 2024-09-14T09:06:39.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.124e-06 2024-09-14T09:06:39.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475886+0000", 2024-09-14T09:06:39.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0130000000000001e-06 2024-09-14T09:06:39.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475904+0000", 2024-09-14T09:06:39.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7779999999999999e-05 2024-09-14T09:06:39.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475918+0000", 2024-09-14T09:06:39.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.385e-05 2024-09-14T09:06:39.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545065+0000", 2024-09-14T09:06:39.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069147307000000005 2024-09-14T09:06:39.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545431+0000", 2024-09-14T09:06:39.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036538400000000001 2024-09-14T09:06:39.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545620+0000", 2024-09-14T09:06:39.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000189614 2024-09-14T09:06:39.174 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.174 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.174 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.175 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19539 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:39.175 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475936+0000", 2024-09-14T09:06:39.175 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.657036935999997, 2024-09-14T09:06:39.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069860063999999999, 2024-09-14T09:06:39.175 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.175 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.175 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:39.175 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.175 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.175 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.175 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19539 2024-09-14T09:06:39.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.175 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475936+0000", 2024-09-14T09:06:39.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475936+0000", 2024-09-14T09:06:39.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475937+0000", 2024-09-14T09:06:39.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5960000000000001e-06 2024-09-14T09:06:39.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475943+0000", 2024-09-14T09:06:39.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2340000000000002e-06 2024-09-14T09:06:39.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475960+0000", 2024-09-14T09:06:39.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7592999999999999e-05 2024-09-14T09:06:39.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475986+0000", 2024-09-14T09:06:39.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6319e-05 2024-09-14T09:06:39.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545650+0000", 2024-09-14T09:06:39.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069663613999999999 2024-09-14T09:06:39.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545714+0000", 2024-09-14T09:06:39.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4209999999999997e-05 2024-09-14T09:06:39.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545796+0000", 2024-09-14T09:06:39.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1497999999999998e-05 2024-09-14T09:06:39.179 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.179 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.179 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.179 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19541 3.10 3:0ae7d8d9:::rbd_data.115e4c71d6e8.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 4194304~8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:39.179 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.497517+0000", 2024-09-14T09:06:39.179 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.635455864999997, 2024-09-14T09:06:39.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069874055000000004, 2024-09-14T09:06:39.179 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.179 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.179 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:39.180 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.180 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.180 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.180 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19541 2024-09-14T09:06:39.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.180 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.497517+0000", 2024-09-14T09:06:39.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.497517+0000", 2024-09-14T09:06:39.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.497520+0000", 2024-09-14T09:06:39.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0730000000000001e-06 2024-09-14T09:06:39.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.504875+0000", 2024-09-14T09:06:39.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007355136 2024-09-14T09:06:39.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.504926+0000", 2024-09-14T09:06:39.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1146000000000001e-05 2024-09-14T09:06:39.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.504950+0000", 2024-09-14T09:06:39.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3638999999999998e-05 2024-09-14T09:06:39.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.505139+0000", 2024-09-14T09:06:39.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018904300000000001 2024-09-14T09:06:39.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.505402+0000", 2024-09-14T09:06:39.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026354699999999999 2024-09-14T09:06:39.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:39.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.506208+0000", 2024-09-14T09:06:39.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00080583399999999996 2024-09-14T09:06:39.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:39.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.566904+0000", 2024-09-14T09:06:39.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.060695510000000001 2024-09-14T09:06:39.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:39.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.567219+0000", 2024-09-14T09:06:39.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031509000000000002 2024-09-14T09:06:39.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:39.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.567319+0000", 2024-09-14T09:06:39.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9804999999999997e-05 2024-09-14T09:06:39.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.567391+0000", 2024-09-14T09:06:39.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2231999999999996e-05 2024-09-14T09:06:39.184 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.184 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.184 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.185 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:30891 3.10 3:0ac3ecfb:::rbd_data.115ebc3c67d1.0000000000000004:c [read 212992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:39.185 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.069740+0000", 2024-09-14T09:06:39.185 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.063233076000003, 2024-09-14T09:06:39.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066940186999999998, 2024-09-14T09:06:39.185 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.185 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.185 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:39.185 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.185 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.185 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.185 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 30891 2024-09-14T09:06:39.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.186 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069740+0000", 2024-09-14T09:06:39.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069740+0000", 2024-09-14T09:06:39.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070603+0000", 2024-09-14T09:06:39.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00086369599999999995 2024-09-14T09:06:39.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070612+0000", 2024-09-14T09:06:39.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3170000000000002e-06 2024-09-14T09:06:39.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070631+0000", 2024-09-14T09:06:39.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8936999999999998e-05 2024-09-14T09:06:39.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070645+0000", 2024-09-14T09:06:39.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4545999999999999e-05 2024-09-14T09:06:39.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136603+0000", 2024-09-14T09:06:39.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.065957551000000003 2024-09-14T09:06:39.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136644+0000", 2024-09-14T09:06:39.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1671000000000001e-05 2024-09-14T09:06:39.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136680+0000", 2024-09-14T09:06:39.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5469000000000002e-05 2024-09-14T09:06:39.189 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.189 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.189 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.189 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:37542 3.10 3:0e658405:::rbd_data.115eb07a8abd.000000000000000d:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:39.189 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.443185+0000", 2024-09-14T09:06:39.189 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 42.689788049999997, 2024-09-14T09:06:39.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.091143627000000005, 2024-09-14T09:06:39.189 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.189 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.190 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:39.190 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.190 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.190 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.190 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37542 2024-09-14T09:06:39.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.190 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443185+0000", 2024-09-14T09:06:39.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443185+0000", 2024-09-14T09:06:39.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443187+0000", 2024-09-14T09:06:39.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.367e-06 2024-09-14T09:06:39.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443199+0000", 2024-09-14T09:06:39.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2153e-05 2024-09-14T09:06:39.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443225+0000", 2024-09-14T09:06:39.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5457e-05 2024-09-14T09:06:39.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443248+0000", 2024-09-14T09:06:39.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2963e-05 2024-09-14T09:06:39.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443400+0000", 2024-09-14T09:06:39.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015241199999999999 2024-09-14T09:06:39.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443542+0000", 2024-09-14T09:06:39.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014224600000000001 2024-09-14T09:06:39.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:39.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.448965+0000", 2024-09-14T09:06:39.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054226839999999997 2024-09-14T09:06:39.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:39.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534136+0000", 2024-09-14T09:06:39.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.085170939000000001 2024-09-14T09:06:39.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:39.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534164+0000", 2024-09-14T09:06:39.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7863000000000001e-05 2024-09-14T09:06:39.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:39.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534250+0000", 2024-09-14T09:06:39.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5876000000000002e-05 2024-09-14T09:06:39.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534328+0000", 2024-09-14T09:06:39.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8666999999999997e-05 2024-09-14T09:06:39.194 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.195 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.195 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.195 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:37542 3.10 e48/28 ondisk, result = 0)", 2024-09-14T09:06:39.195 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.464008+0000", 2024-09-14T09:06:39.195 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 42.668964641000002, 2024-09-14T09:06:39.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.070141355000000002, 2024-09-14T09:06:39.195 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.195 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.195 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:39.195 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464008+0000", 2024-09-14T09:06:39.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464008+0000", 2024-09-14T09:06:39.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464010+0000", 2024-09-14T09:06:39.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.547e-06 2024-09-14T09:06:39.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464017+0000", 2024-09-14T09:06:39.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4930000000000001e-06 2024-09-14T09:06:39.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464047+0000", 2024-09-14T09:06:39.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0090000000000002e-05 2024-09-14T09:06:39.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464061+0000", 2024-09-14T09:06:39.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4003e-05 2024-09-14T09:06:39.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534107+0000", 2024-09-14T09:06:39.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.070045590000000005 2024-09-14T09:06:39.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534126+0000", 2024-09-14T09:06:39.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8893e-05 2024-09-14T09:06:39.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534149+0000", 2024-09-14T09:06:39.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3739000000000001e-05 2024-09-14T09:06:39.198 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.198 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.199 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.199 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38003 3.10 3:0e658405:::rbd_data.115eb07a8abd.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:39.199 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:57.392070+0000", 2024-09-14T09:06:39.199 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.740902867000003, 2024-09-14T09:06:39.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11697451, 2024-09-14T09:06:39.199 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.199 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.199 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:39.199 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.199 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.199 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.199 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38003 2024-09-14T09:06:39.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.200 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392070+0000", 2024-09-14T09:06:39.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392070+0000", 2024-09-14T09:06:39.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392072+0000", 2024-09-14T09:06:39.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6529999999999998e-06 2024-09-14T09:06:39.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392086+0000", 2024-09-14T09:06:39.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.332e-05 2024-09-14T09:06:39.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392139+0000", 2024-09-14T09:06:39.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2815999999999997e-05 2024-09-14T09:06:39.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392155+0000", 2024-09-14T09:06:39.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6796999999999999e-05 2024-09-14T09:06:39.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392296+0000", 2024-09-14T09:06:39.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014107900000000001 2024-09-14T09:06:39.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392443+0000", 2024-09-14T09:06:39.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014654499999999999 2024-09-14T09:06:39.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:39.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.399418+0000", 2024-09-14T09:06:39.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069751049999999997 2024-09-14T09:06:39.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:39.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508607+0000", 2024-09-14T09:06:39.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10918908400000001 2024-09-14T09:06:39.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:39.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508748+0000", 2024-09-14T09:06:39.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014079699999999999 2024-09-14T09:06:39.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:39.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508921+0000", 2024-09-14T09:06:39.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017345899999999999 2024-09-14T09:06:39.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.509044+0000", 2024-09-14T09:06:39.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000122855 2024-09-14T09:06:39.204 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.204 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.204 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.205 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:38003 3.10 e48/28 ondisk, result = 0)", 2024-09-14T09:06:39.205 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:57.413977+0000", 2024-09-14T09:06:39.205 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.718995644000003, 2024-09-14T09:06:39.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.095088000000000006, 2024-09-14T09:06:39.205 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.205 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.205 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:39.205 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413977+0000", 2024-09-14T09:06:39.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413977+0000", 2024-09-14T09:06:39.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413979+0000", 2024-09-14T09:06:39.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.933e-06 2024-09-14T09:06:39.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413988+0000", 2024-09-14T09:06:39.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2769999999999994e-06 2024-09-14T09:06:39.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.414021+0000", 2024-09-14T09:06:39.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2589999999999998e-05 2024-09-14T09:06:39.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.414040+0000", 2024-09-14T09:06:39.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8796000000000001e-05 2024-09-14T09:06:39.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508701+0000", 2024-09-14T09:06:39.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.094661067000000002 2024-09-14T09:06:39.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508729+0000", 2024-09-14T09:06:39.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8677e-05 2024-09-14T09:06:39.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.509065+0000", 2024-09-14T09:06:39.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033565999999999999 2024-09-14T09:06:39.208 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.208 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.208 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.208 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65139 3.a 3:54c90b7d:::rbd_data.115e21c31fcf.0000000000000008:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:06:39.209 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:06:39.209 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 17.75325994, 2024-09-14T09:06:39.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14429895700000001, 2024-09-14T09:06:39.209 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.209 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.209 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:39.209 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.209 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.209 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.209 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65139 2024-09-14T09:06:39.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.209 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:06:39.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:06:39.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379715+0000", 2024-09-14T09:06:39.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.576e-06 2024-09-14T09:06:39.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379724+0000", 2024-09-14T09:06:39.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1779999999999992e-06 2024-09-14T09:06:39.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379746+0000", 2024-09-14T09:06:39.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1661000000000001e-05 2024-09-14T09:06:39.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379759+0000", 2024-09-14T09:06:39.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.324e-05 2024-09-14T09:06:39.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379882+0000", 2024-09-14T09:06:39.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012239400000000001 2024-09-14T09:06:39.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.380023+0000", 2024-09-14T09:06:39.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000141402 2024-09-14T09:06:39.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:39.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.391553+0000", 2024-09-14T09:06:39.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011530197000000001 2024-09-14T09:06:39.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:39.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523927+0000", 2024-09-14T09:06:39.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.132373935 2024-09-14T09:06:39.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:39.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523941+0000", 2024-09-14T09:06:39.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3326000000000001e-05 2024-09-14T09:06:39.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:39.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523977+0000", 2024-09-14T09:06:39.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6185e-05 2024-09-14T09:06:39.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.524012+0000", 2024-09-14T09:06:39.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4863000000000001e-05 2024-09-14T09:06:39.214 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.214 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.214 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.214 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:65139 3.a e65/34 ondisk, result = 0)", 2024-09-14T09:06:39.214 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.408501+0000", 2024-09-14T09:06:39.214 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 17.724471383000001, 2024-09-14T09:06:39.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.115431275, 2024-09-14T09:06:39.214 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.215 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.215 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:39.215 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408501+0000", 2024-09-14T09:06:39.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408501+0000", 2024-09-14T09:06:39.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408504+0000", 2024-09-14T09:06:39.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2819999999999999e-06 2024-09-14T09:06:39.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408512+0000", 2024-09-14T09:06:39.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7530000000000007e-06 2024-09-14T09:06:39.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408534+0000", 2024-09-14T09:06:39.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1993000000000001e-05 2024-09-14T09:06:39.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408550+0000", 2024-09-14T09:06:39.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5988e-05 2024-09-14T09:06:39.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523914+0000", 2024-09-14T09:06:39.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.115363542 2024-09-14T09:06:39.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523922+0000", 2024-09-14T09:06:39.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5529999999999993e-06 2024-09-14T09:06:39.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523933+0000", 2024-09-14T09:06:39.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0164e-05 2024-09-14T09:06:39.218 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.218 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.218 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.218 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:74302 3.a 3:57b145b8:::rbd_data.115ea8d131ff.0000000000000045:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:39.218 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:35.808590+0000", 2024-09-14T09:06:39.218 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 3.3243822629999999, 2024-09-14T09:06:39.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.080730308000000001, 2024-09-14T09:06:39.218 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.218 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.219 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:39.219 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.219 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.219 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.219 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 74302 2024-09-14T09:06:39.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.219 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.808590+0000", 2024-09-14T09:06:39.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.808590+0000", 2024-09-14T09:06:39.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.808593+0000", 2024-09-14T09:06:39.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8700000000000001e-06 2024-09-14T09:06:39.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.808605+0000", 2024-09-14T09:06:39.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2e-05 2024-09-14T09:06:39.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.808637+0000", 2024-09-14T09:06:39.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2023999999999997e-05 2024-09-14T09:06:39.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.808660+0000", 2024-09-14T09:06:39.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2506000000000001e-05 2024-09-14T09:06:39.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.808822+0000", 2024-09-14T09:06:39.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162509 2024-09-14T09:06:39.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.808963+0000", 2024-09-14T09:06:39.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000140765 2024-09-14T09:06:39.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:39.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.813070+0000", 2024-09-14T09:06:39.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004106831 2024-09-14T09:06:39.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:39.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.889186+0000", 2024-09-14T09:06:39.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.076116122999999994 2024-09-14T09:06:39.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:39.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.889233+0000", 2024-09-14T09:06:39.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6591000000000002e-05 2024-09-14T09:06:39.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:39.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.889281+0000", 2024-09-14T09:06:39.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8355e-05 2024-09-14T09:06:39.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.889321+0000", 2024-09-14T09:06:39.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9734000000000003e-05 2024-09-14T09:06:39.223 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.223 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.224 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.224 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.224 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.224 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:39.225 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:06:39.226 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:39.223+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:39.315 INFO:teuthology.orchestra.run.smithi176.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T09:06:39.318 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T09:06:39.326 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:39.327+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:39.343 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T09:06:39.398 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:39.398 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:39.399 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:39.399 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:39.413 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:06:39.428 INFO:teuthology.orchestra.run.smithi176.stdout:3544 punch from 0x77f49df to 0x77f9086, (0x46a7 bytes) 2024-09-14T09:06:39.428 INFO:teuthology.orchestra.run.smithi176.stdout:3545 read 0x338c985 thru 0x33983a6 (0xba22 bytes) 2024-09-14T09:06:39.428 INFO:teuthology.orchestra.run.smithi176.stdout:3548 write 0x8b512bd thru 0x8b5a662 (0x93a6 bytes) 2024-09-14T09:06:39.435 INFO:teuthology.orchestra.run.smithi176.stdout:3550 trunc from 0xda9ef49 to 0xd872b4d 2024-09-14T09:06:39.441 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:39.441 INFO:teuthology.orchestra.run.smithi176.stdout:3552 punch from 0x713f66f to 0x7149d0e, (0xa69f bytes) 2024-09-14T09:06:39.455 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:39.459+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:39.460 INFO:teuthology.orchestra.run.smithi176.stdout:3553 trunc from 0xd872b4d to 0x18e3d06 2024-09-14T09:06:39.514 INFO:teuthology.orchestra.run.smithi176.stdout:3555 write 0xab9eefb thru 0xaba3a50 (0x4b56 bytes) 2024-09-14T09:06:39.522 INFO:teuthology.orchestra.run.smithi176.stdout:3558 write 0x79241da thru 0x792ff9f (0xbdc6 bytes) 2024-09-14T09:06:39.527 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:39.527+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:39.531 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:39.531 INFO:teuthology.orchestra.run.smithi176.stdout:3560 read 0x7671554 thru 0x7674efb (0x39a8 bytes) 2024-09-14T09:06:39.531 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:39.531 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:39.531 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:39.540 INFO:teuthology.orchestra.run.smithi176.stdout:3565 write 0xc8c1c62 thru 0xc8cda35 (0xbdd4 bytes) 2024-09-14T09:06:39.551 INFO:teuthology.orchestra.run.smithi176.stdout:3566 read 0x3bc27e0 thru 0x3bc3fcb (0x17ec bytes) 2024-09-14T09:06:39.551 INFO:teuthology.orchestra.run.smithi176.stdout:3567 read 0xa4885fb thru 0xa494ac1 (0xc4c7 bytes) 2024-09-14T09:06:39.551 INFO:teuthology.orchestra.run.smithi176.stdout:3568 read 0x2bac003 thru 0x2bb4fa2 (0x8fa0 bytes) 2024-09-14T09:06:39.551 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:39.551 INFO:teuthology.orchestra.run.smithi176.stdout:3570 punch from 0x769128c to 0x769a2e9, (0x905d bytes) 2024-09-14T09:06:39.557 INFO:teuthology.orchestra.run.smithi176.stdout:3571 trunc from 0xc8cda36 to 0x8d98004 2024-09-14T09:06:39.594 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:39.594 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:39.594 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:39.594 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:39.594 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:39.601 INFO:teuthology.orchestra.run.smithi176.stdout:3572 writesame 0xa3dd40b thru 0xa3dd7b6 data_size 0xa(0x3ac bytes) 2024-09-14T09:06:39.609 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:06:39.615 INFO:teuthology.orchestra.run.smithi176.stdout:3573 write 0xd0023f2 thru 0xd00cfa5 (0xabb4 bytes) 2024-09-14T09:06:39.628 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:39.627+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:39.631 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:39.631 INFO:teuthology.orchestra.run.smithi176.stdout:3575 writesame 0xa34d3a1 thru 0xa35c577 data_size 0x9(0xf1d7 bytes) 2024-09-14T09:06:39.645 INFO:teuthology.orchestra.run.smithi176.stdout:3576 read 0xa894d46 thru 0xa8971fe (0x24b9 bytes) 2024-09-14T09:06:39.645 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:39.645 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:39.645 INFO:teuthology.orchestra.run.smithi176.stdout:3581 write 0x57c20c1 thru 0x57d0d7f (0xecbf bytes) 2024-09-14T09:06:39.662 INFO:teuthology.orchestra.run.smithi176.stdout:3585 read 0x88d52f3 thru 0x88e2209 (0xcf17 bytes) 2024-09-14T09:06:39.663 INFO:teuthology.orchestra.run.smithi176.stdout:3586 write 0xc19684 thru 0xc1b858 (0x21d5 bytes) 2024-09-14T09:06:39.665 INFO:teuthology.orchestra.run.smithi176.stdout:3587 writesame 0x54b29e thru 0x55a114 data_size 0xab(0xee77 bytes) 2024-09-14T09:06:39.669 INFO:teuthology.orchestra.run.smithi176.stdout:3588 write 0xcc70aea thru 0xcc7162b (0xb42 bytes) 2024-09-14T09:06:39.676 INFO:teuthology.orchestra.run.smithi176.stdout:3590 writesame 0x5596da8 thru 0x5598a54 data_size 0x1(0x1cad bytes) 2024-09-14T09:06:39.693 INFO:teuthology.orchestra.run.smithi176.stdout:3592 trunc from 0xd00cfa6 to 0x381315a 2024-09-14T09:06:39.730 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:39.731+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:39.733 INFO:teuthology.orchestra.run.smithi176.stdout:3594 write 0x9d0f63a thru 0x9d11771 (0x2138 bytes) 2024-09-14T09:06:39.735 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:39.735 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:39.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:39.735 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:39.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.735 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:19152 3.9 e39/34 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head v 39'12, mlcod=39'9)", 2024-09-14T09:06:39.735 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.333629+0000", 2024-09-14T09:06:39.735 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 55.404738369999997, 2024-09-14T09:06:39.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.1194095, 2024-09-14T09:06:39.736 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.736 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.736 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:39.736 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.333629+0000", 2024-09-14T09:06:39.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.333629+0000", 2024-09-14T09:06:39.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.333631+0000", 2024-09-14T09:06:39.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.007e-06 2024-09-14T09:06:39.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.350796+0000", 2024-09-14T09:06:39.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017164690999999999 2024-09-14T09:06:39.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.350865+0000", 2024-09-14T09:06:39.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8989e-05 2024-09-14T09:06:39.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.350884+0000", 2024-09-14T09:06:39.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9323000000000001e-05 2024-09-14T09:06:39.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.351045+0000", 2024-09-14T09:06:39.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016102900000000001 2024-09-14T09:06:39.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.351087+0000", 2024-09-14T09:06:39.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1430000000000001e-05 2024-09-14T09:06:39.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:39.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.452865+0000", 2024-09-14T09:06:39.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10177839900000001 2024-09-14T09:06:39.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.453039+0000", 2024-09-14T09:06:39.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017363200000000001 2024-09-14T09:06:39.739 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.739 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.739 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.740 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:37543 3.1b 3:d8e0f8d4:::rbd_data.115eb07a8abd.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:39.740 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.535280+0000", 2024-09-14T09:06:39.740 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 43.203087361999998, 2024-09-14T09:06:39.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.12853271799999999, 2024-09-14T09:06:39.740 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.740 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.740 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:39.740 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.740 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.740 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.740 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37543 2024-09-14T09:06:39.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.741 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535280+0000", 2024-09-14T09:06:39.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535280+0000", 2024-09-14T09:06:39.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535284+0000", 2024-09-14T09:06:39.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6770000000000001e-06 2024-09-14T09:06:39.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535303+0000", 2024-09-14T09:06:39.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9553000000000001e-05 2024-09-14T09:06:39.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535349+0000", 2024-09-14T09:06:39.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5833e-05 2024-09-14T09:06:39.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535368+0000", 2024-09-14T09:06:39.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8542999999999998e-05 2024-09-14T09:06:39.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535548+0000", 2024-09-14T09:06:39.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000180688 2024-09-14T09:06:39.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535814+0000", 2024-09-14T09:06:39.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026614100000000001 2024-09-14T09:06:39.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:39.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.543779+0000", 2024-09-14T09:06:39.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0079644720000000002 2024-09-14T09:06:39.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:39.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663456+0000", 2024-09-14T09:06:39.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.119676694 2024-09-14T09:06:39.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:39.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663626+0000", 2024-09-14T09:06:39.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170559 2024-09-14T09:06:39.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:39.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663728+0000", 2024-09-14T09:06:39.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101889 2024-09-14T09:06:39.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663813+0000", 2024-09-14T09:06:39.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4668999999999999e-05 2024-09-14T09:06:39.745 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.745 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.745 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.745 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:37543 3.1b e48/34 ondisk, result = 0)", 2024-09-14T09:06:39.745 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.566755+0000", 2024-09-14T09:06:39.746 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 43.171612824, 2024-09-14T09:06:39.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.096718023, 2024-09-14T09:06:39.746 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.746 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.746 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:39.746 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566755+0000", 2024-09-14T09:06:39.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566755+0000", 2024-09-14T09:06:39.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566757+0000", 2024-09-14T09:06:39.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9800000000000001e-06 2024-09-14T09:06:39.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566767+0000", 2024-09-14T09:06:39.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0036e-05 2024-09-14T09:06:39.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566797+0000", 2024-09-14T09:06:39.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.012e-05 2024-09-14T09:06:39.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566813+0000", 2024-09-14T09:06:39.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6575999999999999e-05 2024-09-14T09:06:39.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663408+0000", 2024-09-14T09:06:39.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.096594901999999996 2024-09-14T09:06:39.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663426+0000", 2024-09-14T09:06:39.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7765999999999999e-05 2024-09-14T09:06:39.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663473+0000", 2024-09-14T09:06:39.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6643e-05 2024-09-14T09:06:39.749 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.749 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.749 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.749 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38593 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [watch reconnect cookie 139718844451680 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:06:39.749 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:06:39.750 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.919949955, 2024-09-14T09:06:39.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0218296630000001, 2024-09-14T09:06:39.750 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.750 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.750 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:39.750 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.750 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.750 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.750 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38593 2024-09-14T09:06:39.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.750 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:06:39.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:06:39.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818420+0000", 2024-09-14T09:06:39.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8399999999999999e-06 2024-09-14T09:06:39.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818431+0000", 2024-09-14T09:06:39.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0859999999999999e-05 2024-09-14T09:06:39.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818466+0000", 2024-09-14T09:06:39.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4486999999999997e-05 2024-09-14T09:06:39.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818481+0000", 2024-09-14T09:06:39.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5362999999999998e-05 2024-09-14T09:06:39.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818659+0000", 2024-09-14T09:06:39.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017807500000000001 2024-09-14T09:06:39.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op must wait for map", 2024-09-14T09:06:39.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818681+0000", 2024-09-14T09:06:39.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2096999999999999e-05 2024-09-14T09:06:39.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825472+0000", 2024-09-14T09:06:39.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067905170000000003 2024-09-14T09:06:39.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:06:39.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825491+0000", 2024-09-14T09:06:39.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9372999999999999e-05 2024-09-14T09:06:39.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835402+0000", 2024-09-14T09:06:39.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0099111620000001 2024-09-14T09:06:39.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835548+0000", 2024-09-14T09:06:39.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014607300000000001 2024-09-14T09:06:39.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.840247+0000", 2024-09-14T09:06:39.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046988159999999998 2024-09-14T09:06:39.755 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.755 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.755 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.756 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38615 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:06:39.756 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:06:39.756 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.871584747, 2024-09-14T09:06:39.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.97014128899999996, 2024-09-14T09:06:39.756 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.756 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.756 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:39.756 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.756 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.756 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.756 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38615 2024-09-14T09:06:39.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.756 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:06:39.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:06:39.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866787+0000", 2024-09-14T09:06:39.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.476e-06 2024-09-14T09:06:39.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866806+0000", 2024-09-14T09:06:39.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8606000000000001e-05 2024-09-14T09:06:39.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866839+0000", 2024-09-14T09:06:39.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3510000000000003e-05 2024-09-14T09:06:39.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866867+0000", 2024-09-14T09:06:39.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7610000000000002e-05 2024-09-14T09:06:39.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867031+0000", 2024-09-14T09:06:39.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016414199999999999 2024-09-14T09:06:39.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:06:39.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867054+0000", 2024-09-14T09:06:39.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2693000000000001e-05 2024-09-14T09:06:39.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836462+0000", 2024-09-14T09:06:39.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.96940797700000003 2024-09-14T09:06:39.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836547+0000", 2024-09-14T09:06:39.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5294e-05 2024-09-14T09:06:39.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836924+0000", 2024-09-14T09:06:39.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037698099999999997 2024-09-14T09:06:39.760 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.760 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.760 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.761 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49230 3.d 3:b4a67b64:::rbd_data.115e51d92ed2.0000000000000003:12 [read 2621440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:39.761 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.523859+0000", 2024-09-14T09:06:39.761 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.214508156000001, 2024-09-14T09:06:39.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067232675000000006, 2024-09-14T09:06:39.761 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.761 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.761 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:39.761 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.761 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.761 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.761 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49230 2024-09-14T09:06:39.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.762 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523859+0000", 2024-09-14T09:06:39.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523859+0000", 2024-09-14T09:06:39.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523860+0000", 2024-09-14T09:06:39.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0130000000000001e-06 2024-09-14T09:06:39.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523865+0000", 2024-09-14T09:06:39.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4000000000000002e-06 2024-09-14T09:06:39.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523880+0000", 2024-09-14T09:06:39.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5287000000000001e-05 2024-09-14T09:06:39.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523892+0000", 2024-09-14T09:06:39.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2006e-05 2024-09-14T09:06:39.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.590817+0000", 2024-09-14T09:06:39.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066924692999999993 2024-09-14T09:06:39.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:39.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.590937+0000", 2024-09-14T09:06:39.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012081700000000001 2024-09-14T09:06:39.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591092+0000", 2024-09-14T09:06:39.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015445899999999999 2024-09-14T09:06:39.765 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.765 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.765 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.765 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49231 3.d 3:b4a67b64:::rbd_data.115e51d92ed2.0000000000000003:12 [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:39.765 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.523908+0000", 2024-09-14T09:06:39.765 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.214459696999999, 2024-09-14T09:06:39.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067373731000000006, 2024-09-14T09:06:39.765 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.765 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.765 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:39.766 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.766 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.766 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.766 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49231 2024-09-14T09:06:39.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.766 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523908+0000", 2024-09-14T09:06:39.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523908+0000", 2024-09-14T09:06:39.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523909+0000", 2024-09-14T09:06:39.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9699999999999994e-07 2024-09-14T09:06:39.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523913+0000", 2024-09-14T09:06:39.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6070000000000001e-06 2024-09-14T09:06:39.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523929+0000", 2024-09-14T09:06:39.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5415999999999999e-05 2024-09-14T09:06:39.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523940+0000", 2024-09-14T09:06:39.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1707e-05 2024-09-14T09:06:39.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591167+0000", 2024-09-14T09:06:39.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067226576999999996 2024-09-14T09:06:39.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591281+0000", 2024-09-14T09:06:39.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011442699999999999 2024-09-14T09:06:39.769 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.769 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.769 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.769 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49232 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:39.769 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.523956+0000", 2024-09-14T09:06:39.769 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.214411363, 2024-09-14T09:06:39.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067398703000000004, 2024-09-14T09:06:39.769 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.769 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.769 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:39.770 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.770 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.770 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.770 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49232 2024-09-14T09:06:39.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.770 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523956+0000", 2024-09-14T09:06:39.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523956+0000", 2024-09-14T09:06:39.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523957+0000", 2024-09-14T09:06:39.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8299999999999995e-07 2024-09-14T09:06:39.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523961+0000", 2024-09-14T09:06:39.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3329999999999998e-06 2024-09-14T09:06:39.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523977+0000", 2024-09-14T09:06:39.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.577e-05 2024-09-14T09:06:39.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523989+0000", 2024-09-14T09:06:39.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2089999999999999e-05 2024-09-14T09:06:39.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591340+0000", 2024-09-14T09:06:39.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067350278999999999 2024-09-14T09:06:39.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591355+0000", 2024-09-14T09:06:39.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5248e-05 2024-09-14T09:06:39.773 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.773 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.773 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.773 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49233 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:39.773 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.524004+0000", 2024-09-14T09:06:39.774 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.214363454000001, 2024-09-14T09:06:39.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067426573000000004, 2024-09-14T09:06:39.774 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.774 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.774 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:39.774 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.774 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.774 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.774 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49233 2024-09-14T09:06:39.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.774 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524004+0000", 2024-09-14T09:06:39.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524004+0000", 2024-09-14T09:06:39.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524005+0000", 2024-09-14T09:06:39.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3699999999999999e-07 2024-09-14T09:06:39.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524009+0000", 2024-09-14T09:06:39.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3359999999999997e-06 2024-09-14T09:06:39.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524026+0000", 2024-09-14T09:06:39.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6753e-05 2024-09-14T09:06:39.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524040+0000", 2024-09-14T09:06:39.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3694e-05 2024-09-14T09:06:39.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591411+0000", 2024-09-14T09:06:39.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067371767999999999 2024-09-14T09:06:39.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591431+0000", 2024-09-14T09:06:39.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9085000000000001e-05 2024-09-14T09:06:39.777 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.777 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.777 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.777 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49257 3.d 3.bb54482d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:39.777 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.524584+0000", 2024-09-14T09:06:39.778 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.213783551999999, 2024-09-14T09:06:39.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067046887999999999, 2024-09-14T09:06:39.778 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.778 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.778 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:39.778 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.778 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.778 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.778 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49257 2024-09-14T09:06:39.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.778 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524584+0000", 2024-09-14T09:06:39.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524584+0000", 2024-09-14T09:06:39.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524585+0000", 2024-09-14T09:06:39.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1699999999999997e-07 2024-09-14T09:06:39.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524609+0000", 2024-09-14T09:06:39.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4448999999999999e-05 2024-09-14T09:06:39.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524627+0000", 2024-09-14T09:06:39.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7543000000000001e-05 2024-09-14T09:06:39.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524634+0000", 2024-09-14T09:06:39.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7540000000000006e-06 2024-09-14T09:06:39.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591622+0000", 2024-09-14T09:06:39.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066987668 2024-09-14T09:06:39.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591631+0000", 2024-09-14T09:06:39.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7569999999999993e-06 2024-09-14T09:06:39.781 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.781 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.781 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.781 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49310 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:39.782 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525200+0000", 2024-09-14T09:06:39.782 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.213167256999999, 2024-09-14T09:06:39.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067082507, 2024-09-14T09:06:39.782 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.782 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.782 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:39.782 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.782 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.782 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.782 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49310 2024-09-14T09:06:39.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.782 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525200+0000", 2024-09-14T09:06:39.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525200+0000", 2024-09-14T09:06:39.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525201+0000", 2024-09-14T09:06:39.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.023e-06 2024-09-14T09:06:39.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525206+0000", 2024-09-14T09:06:39.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5000000000000001e-06 2024-09-14T09:06:39.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525242+0000", 2024-09-14T09:06:39.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6672999999999999e-05 2024-09-14T09:06:39.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525256+0000", 2024-09-14T09:06:39.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3360000000000001e-05 2024-09-14T09:06:39.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592266+0000", 2024-09-14T09:06:39.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067009868 2024-09-14T09:06:39.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592283+0000", 2024-09-14T09:06:39.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7082999999999999e-05 2024-09-14T09:06:39.785 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.785 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.785 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.785 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49311 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:39.786 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525275+0000", 2024-09-14T09:06:39.786 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.213092481, 2024-09-14T09:06:39.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067067112999999998, 2024-09-14T09:06:39.786 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.786 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.786 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:39.786 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.786 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.786 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.786 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49311 2024-09-14T09:06:39.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.786 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.786 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525275+0000", 2024-09-14T09:06:39.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525275+0000", 2024-09-14T09:06:39.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525276+0000", 2024-09-14T09:06:39.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.313e-06 2024-09-14T09:06:39.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525282+0000", 2024-09-14T09:06:39.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.127e-06 2024-09-14T09:06:39.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525299+0000", 2024-09-14T09:06:39.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6903000000000001e-05 2024-09-14T09:06:39.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525312+0000", 2024-09-14T09:06:39.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3193999999999999e-05 2024-09-14T09:06:39.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592323+0000", 2024-09-14T09:06:39.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067010504999999998 2024-09-14T09:06:39.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592342+0000", 2024-09-14T09:06:39.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9071000000000001e-05 2024-09-14T09:06:39.789 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.789 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.789 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.789 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49312 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:39.789 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525330+0000", 2024-09-14T09:06:39.790 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.213037088, 2024-09-14T09:06:39.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.06706268, 2024-09-14T09:06:39.790 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.790 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.790 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:39.790 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.790 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.790 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.790 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49312 2024-09-14T09:06:39.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.790 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525330+0000", 2024-09-14T09:06:39.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525330+0000", 2024-09-14T09:06:39.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525332+0000", 2024-09-14T09:06:39.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.187e-06 2024-09-14T09:06:39.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525350+0000", 2024-09-14T09:06:39.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8833000000000001e-05 2024-09-14T09:06:39.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525385+0000", 2024-09-14T09:06:39.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4686e-05 2024-09-14T09:06:39.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525399+0000", 2024-09-14T09:06:39.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4239999999999999e-05 2024-09-14T09:06:39.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592377+0000", 2024-09-14T09:06:39.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066977255999999999 2024-09-14T09:06:39.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592393+0000", 2024-09-14T09:06:39.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6478e-05 2024-09-14T09:06:39.793 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.793 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.793 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.793 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49313 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:39.793 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525423+0000", 2024-09-14T09:06:39.794 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.212944689, 2024-09-14T09:06:39.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067039319, 2024-09-14T09:06:39.794 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.794 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.794 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:39.794 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.794 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.794 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.794 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49313 2024-09-14T09:06:39.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.794 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525423+0000", 2024-09-14T09:06:39.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525423+0000", 2024-09-14T09:06:39.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525424+0000", 2024-09-14T09:06:39.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.257e-06 2024-09-14T09:06:39.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525430+0000", 2024-09-14T09:06:39.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8699999999999997e-06 2024-09-14T09:06:39.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525447+0000", 2024-09-14T09:06:39.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7560000000000001e-05 2024-09-14T09:06:39.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525460+0000", 2024-09-14T09:06:39.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2243e-05 2024-09-14T09:06:39.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592444+0000", 2024-09-14T09:06:39.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066984118999999995 2024-09-14T09:06:39.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592462+0000", 2024-09-14T09:06:39.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.827e-05 2024-09-14T09:06:39.797 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.797 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.797 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.797 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49314 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:39.797 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525478+0000", 2024-09-14T09:06:39.797 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.212888926000002, 2024-09-14T09:06:39.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.06702756, 2024-09-14T09:06:39.798 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.798 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.798 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:39.798 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.798 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.798 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.798 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49314 2024-09-14T09:06:39.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.798 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525478+0000", 2024-09-14T09:06:39.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525478+0000", 2024-09-14T09:06:39.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525480+0000", 2024-09-14T09:06:39.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.246e-06 2024-09-14T09:06:39.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525485+0000", 2024-09-14T09:06:39.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7440000000000004e-06 2024-09-14T09:06:39.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525504+0000", 2024-09-14T09:06:39.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8709999999999999e-05 2024-09-14T09:06:39.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525512+0000", 2024-09-14T09:06:39.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6260000000000003e-06 2024-09-14T09:06:39.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592491+0000", 2024-09-14T09:06:39.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066979274000000005 2024-09-14T09:06:39.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592506+0000", 2024-09-14T09:06:39.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4960000000000001e-05 2024-09-14T09:06:39.801 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.801 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.801 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.801 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49315 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:39.801 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525523+0000", 2024-09-14T09:06:39.801 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.212844130000001, 2024-09-14T09:06:39.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067036894, 2024-09-14T09:06:39.802 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.802 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.802 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:39.802 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.802 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.802 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.802 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49315 2024-09-14T09:06:39.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.802 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525523+0000", 2024-09-14T09:06:39.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525523+0000", 2024-09-14T09:06:39.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525524+0000", 2024-09-14T09:06:39.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2999999999999999e-07 2024-09-14T09:06:39.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525528+0000", 2024-09-14T09:06:39.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4000000000000001e-06 2024-09-14T09:06:39.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525537+0000", 2024-09-14T09:06:39.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2739999999999996e-06 2024-09-14T09:06:39.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525543+0000", 2024-09-14T09:06:39.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6329999999999999e-06 2024-09-14T09:06:39.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592548+0000", 2024-09-14T09:06:39.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067004806 2024-09-14T09:06:39.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592560+0000", 2024-09-14T09:06:39.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1951e-05 2024-09-14T09:06:39.805 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.805 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.805 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.805 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49316 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:39.805 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525553+0000", 2024-09-14T09:06:39.805 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.212814236, 2024-09-14T09:06:39.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067056432999999999, 2024-09-14T09:06:39.806 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.806 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.806 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:39.806 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.806 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.806 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.806 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49316 2024-09-14T09:06:39.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.806 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525553+0000", 2024-09-14T09:06:39.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525553+0000", 2024-09-14T09:06:39.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525554+0000", 2024-09-14T09:06:39.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0299999999999998e-07 2024-09-14T09:06:39.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525557+0000", 2024-09-14T09:06:39.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9129999999999999e-06 2024-09-14T09:06:39.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525566+0000", 2024-09-14T09:06:39.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8000000000000004e-06 2024-09-14T09:06:39.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525572+0000", 2024-09-14T09:06:39.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.72e-06 2024-09-14T09:06:39.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592598+0000", 2024-09-14T09:06:39.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067025682000000003 2024-09-14T09:06:39.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592610+0000", 2024-09-14T09:06:39.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1515000000000001e-05 2024-09-14T09:06:39.809 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.809 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.809 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.809 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49317 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:39.809 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:39.809 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.212785777000001, 2024-09-14T09:06:39.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067082547000000006, 2024-09-14T09:06:39.810 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.810 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.810 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:39.810 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.810 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.810 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.810 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49317 2024-09-14T09:06:39.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.810 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:39.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:39.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:39.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:06:39.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525585+0000", 2024-09-14T09:06:39.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8700000000000001e-06 2024-09-14T09:06:39.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525594+0000", 2024-09-14T09:06:39.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9740000000000008e-06 2024-09-14T09:06:39.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525601+0000", 2024-09-14T09:06:39.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9489999999999999e-06 2024-09-14T09:06:39.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592652+0000", 2024-09-14T09:06:39.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067050803000000006 2024-09-14T09:06:39.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592664+0000", 2024-09-14T09:06:39.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2198000000000001e-05 2024-09-14T09:06:39.813 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.813 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.813 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.813 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49318 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:39.813 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525642+0000", 2024-09-14T09:06:39.813 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.212725277000001, 2024-09-14T09:06:39.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067064055999999997, 2024-09-14T09:06:39.814 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.814 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.814 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:39.814 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.814 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.814 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.814 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49318 2024-09-14T09:06:39.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.814 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525642+0000", 2024-09-14T09:06:39.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525642+0000", 2024-09-14T09:06:39.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525644+0000", 2024-09-14T09:06:39.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0159999999999998e-06 2024-09-14T09:06:39.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525649+0000", 2024-09-14T09:06:39.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2739999999999998e-06 2024-09-14T09:06:39.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525666+0000", 2024-09-14T09:06:39.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6990000000000002e-05 2024-09-14T09:06:39.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525680+0000", 2024-09-14T09:06:39.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.331e-05 2024-09-14T09:06:39.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592697+0000", 2024-09-14T09:06:39.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067017367999999994 2024-09-14T09:06:39.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592706+0000", 2024-09-14T09:06:39.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.098e-06 2024-09-14T09:06:39.817 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.817 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.817 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.817 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49319 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:39.817 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525704+0000", 2024-09-14T09:06:39.817 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.212663871, 2024-09-14T09:06:39.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067049867999999999, 2024-09-14T09:06:39.818 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:39.818 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:39.818 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:39.818 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:39.818 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:39.818 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:39.818 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49319 2024-09-14T09:06:39.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.818 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:39.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:39.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525704+0000", 2024-09-14T09:06:39.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:39.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525704+0000", 2024-09-14T09:06:39.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:39.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:39.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525705+0000", 2024-09-14T09:06:39.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.113e-06 2024-09-14T09:06:39.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:39.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525710+0000", 2024-09-14T09:06:39.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.843e-06 2024-09-14T09:06:39.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:39.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525726+0000", 2024-09-14T09:06:39.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.679e-05 2024-09-14T09:06:39.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:39.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525739+0000", 2024-09-14T09:06:39.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2962999999999999e-05 2024-09-14T09:06:39.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:39.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592745+0000", 2024-09-14T09:06:39.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067005341999999996 2024-09-14T09:06:39.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:39.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:39.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:39.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592753+0000", 2024-09-14T09:06:39.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8170000000000005e-06 2024-09-14T09:06:39.821 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.821 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.821 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.821 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:39.821 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:39.821 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:39.822 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:39.822 INFO:teuthology.orchestra.run.smithi176.stdout:3596 punch from 0x1fe97bd to 0x1ff7e8c, (0xe6cf bytes) 2024-09-14T09:06:39.823 INFO:teuthology.orchestra.run.smithi176.stdout:3597 read 0x7626c22 thru 0x762eb92 (0x7f71 bytes) 2024-09-14T09:06:39.823 INFO:teuthology.orchestra.run.smithi176.stdout:3598 trunc from 0x9d11772 to 0xd80ceac 2024-09-14T09:06:39.823 INFO:teuthology.orchestra.run.smithi176.stdout:3599 clone 26 order 19 su 32768 sc 9 2024-09-14T09:06:39.823 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:06:39.866 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:39.867+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:39.931 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:39.935+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:39.941 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:39.942 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:39.942 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:39.942 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:39.955 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:06:40.032 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:40.035+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:40.118 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:40.118 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:40.118 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:40.118 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:40.119 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:40.132 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:06:40.133 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:40.135+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:40.233 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:40.231+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:40.252 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:40.252 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:40.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:40.252 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:40.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.252 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20858 3.17 3:ed5052d1:::rbd_data.115e889d4809.000000000000015c:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:40.252 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:06:40.253 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.821227557, 2024-09-14T09:06:40.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053381794000000003, 2024-09-14T09:06:40.253 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.253 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.253 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:40.253 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:40.253 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:40.253 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:40.253 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20858 2024-09-14T09:06:40.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.253 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:06:40.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:06:40.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429351+0000", 2024-09-14T09:06:40.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1149999999999998e-06 2024-09-14T09:06:40.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429363+0000", 2024-09-14T09:06:40.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1819e-05 2024-09-14T09:06:40.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429389+0000", 2024-09-14T09:06:40.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5605e-05 2024-09-14T09:06:40.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429402+0000", 2024-09-14T09:06:40.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3175e-05 2024-09-14T09:06:40.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429492+0000", 2024-09-14T09:06:40.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.988e-05 2024-09-14T09:06:40.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429597+0000", 2024-09-14T09:06:40.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010564 2024-09-14T09:06:40.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:40.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.433465+0000", 2024-09-14T09:06:40.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0038674249999999999 2024-09-14T09:06:40.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:40.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482483+0000", 2024-09-14T09:06:40.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049017999999999999 2024-09-14T09:06:40.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:40.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482625+0000", 2024-09-14T09:06:40.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000142447 2024-09-14T09:06:40.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:40.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482684+0000", 2024-09-14T09:06:40.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8522e-05 2024-09-14T09:06:40.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482730+0000", 2024-09-14T09:06:40.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6165999999999998e-05 2024-09-14T09:06:40.258 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.258 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.259 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.259 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:20858 3.17 e41/23 ondisk, result = 0)", 2024-09-14T09:06:40.259 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:06:40.259 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.806468113999998, 2024-09-14T09:06:40.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038631402000000002, 2024-09-14T09:06:40.259 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.259 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.259 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:40.259 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:06:40.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:06:40.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444109+0000", 2024-09-14T09:06:40.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.331e-06 2024-09-14T09:06:40.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444114+0000", 2024-09-14T09:06:40.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3709999999999999e-06 2024-09-14T09:06:40.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444130+0000", 2024-09-14T09:06:40.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6079e-05 2024-09-14T09:06:40.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444144+0000", 2024-09-14T09:06:40.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3331e-05 2024-09-14T09:06:40.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482538+0000", 2024-09-14T09:06:40.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038394878 2024-09-14T09:06:40.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482548+0000", 2024-09-14T09:06:40.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0093e-05 2024-09-14T09:06:40.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482739+0000", 2024-09-14T09:06:40.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019031899999999999 2024-09-14T09:06:40.262 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.262 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.263 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.263 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:37800 3.1 3:830c4670:::rbd_data.115eb07a8abd.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:40.263 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:06:40.263 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.220020147, 2024-09-14T09:06:40.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.068040724999999996, 2024-09-14T09:06:40.263 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.263 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.263 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:40.263 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:40.264 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:40.264 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:40.264 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 37800 2024-09-14T09:06:40.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.264 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:06:40.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:06:40.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030557+0000", 2024-09-14T09:06:40.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.888e-06 2024-09-14T09:06:40.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030566+0000", 2024-09-14T09:06:40.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8540000000000003e-06 2024-09-14T09:06:40.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030585+0000", 2024-09-14T09:06:40.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8878000000000001e-05 2024-09-14T09:06:40.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030593+0000", 2024-09-14T09:06:40.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5870000000000004e-06 2024-09-14T09:06:40.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030667+0000", 2024-09-14T09:06:40.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4207000000000003e-05 2024-09-14T09:06:40.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030761+0000", 2024-09-14T09:06:40.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4223999999999998e-05 2024-09-14T09:06:40.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:06:40.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.034400+0000", 2024-09-14T09:06:40.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0036387759999999998 2024-09-14T09:06:40.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:40.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098457+0000", 2024-09-14T09:06:40.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.064057501000000003 2024-09-14T09:06:40.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:40.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098518+0000", 2024-09-14T09:06:40.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0730999999999998e-05 2024-09-14T09:06:40.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:40.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098560+0000", 2024-09-14T09:06:40.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2267e-05 2024-09-14T09:06:40.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098596+0000", 2024-09-14T09:06:40.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5812000000000002e-05 2024-09-14T09:06:40.269 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.269 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.269 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.269 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:37800 3.1 e48/34 ondisk, result = 0)", 2024-09-14T09:06:40.269 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:06:40.269 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.209028531000001, 2024-09-14T09:06:40.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057056260999999997, 2024-09-14T09:06:40.270 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.270 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.270 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:40.270 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:06:40.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:06:40.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041548+0000", 2024-09-14T09:06:40.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8100000000000002e-07 2024-09-14T09:06:40.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041552+0000", 2024-09-14T09:06:40.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7419999999999999e-06 2024-09-14T09:06:40.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041563+0000", 2024-09-14T09:06:40.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1926000000000001e-05 2024-09-14T09:06:40.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041572+0000", 2024-09-14T09:06:40.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8610000000000004e-06 2024-09-14T09:06:40.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098503+0000", 2024-09-14T09:06:40.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056930827000000003 2024-09-14T09:06:40.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098512+0000", 2024-09-14T09:06:40.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5019999999999992e-06 2024-09-14T09:06:40.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098603+0000", 2024-09-14T09:06:40.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1621999999999998e-05 2024-09-14T09:06:40.273 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.273 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.273 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.274 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38523 3.3 3:c62e0486:::rbd_data.115e9e133b92.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 4194304~28499] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T09:06:40.274 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:06:40.274 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.796600832999999, 2024-09-14T09:06:40.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035654406999999999, 2024-09-14T09:06:40.274 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.274 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.274 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:40.274 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:40.274 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:40.274 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:40.274 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38523 2024-09-14T09:06:40.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.274 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:06:40.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:06:40.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453977+0000", 2024-09-14T09:06:40.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6299999999999998e-06 2024-09-14T09:06:40.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455684+0000", 2024-09-14T09:06:40.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001706843 2024-09-14T09:06:40.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455710+0000", 2024-09-14T09:06:40.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6302e-05 2024-09-14T09:06:40.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455722+0000", 2024-09-14T09:06:40.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1798e-05 2024-09-14T09:06:40.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455825+0000", 2024-09-14T09:06:40.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102822 2024-09-14T09:06:40.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.456032+0000", 2024-09-14T09:06:40.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020704400000000001 2024-09-14T09:06:40.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:06:40.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.456759+0000", 2024-09-14T09:06:40.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00072735699999999998 2024-09-14T09:06:40.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:40.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.477856+0000", 2024-09-14T09:06:40.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021096011000000001 2024-09-14T09:06:40.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:40.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489526+0000", 2024-09-14T09:06:40.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011670313 2024-09-14T09:06:40.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:40.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489572+0000", 2024-09-14T09:06:40.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6408e-05 2024-09-14T09:06:40.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489629+0000", 2024-09-14T09:06:40.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6879000000000002e-05 2024-09-14T09:06:40.279 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.279 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.279 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.279 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49715 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:40.279 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.524874+0000", 2024-09-14T09:06:40.279 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.725701891, 2024-09-14T09:06:40.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034330422999999999, 2024-09-14T09:06:40.280 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.280 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.280 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:40.280 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:40.280 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:40.280 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:40.280 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49715 2024-09-14T09:06:40.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.280 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524874+0000", 2024-09-14T09:06:40.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524874+0000", 2024-09-14T09:06:40.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524875+0000", 2024-09-14T09:06:40.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1500000000000003e-07 2024-09-14T09:06:40.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524878+0000", 2024-09-14T09:06:40.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9090000000000002e-06 2024-09-14T09:06:40.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524889+0000", 2024-09-14T09:06:40.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.007e-05 2024-09-14T09:06:40.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524895+0000", 2024-09-14T09:06:40.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.285e-06 2024-09-14T09:06:40.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.558620+0000", 2024-09-14T09:06:40.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.033725009 2024-09-14T09:06:40.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.558681+0000", 2024-09-14T09:06:40.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1051999999999998e-05 2024-09-14T09:06:40.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559204+0000", 2024-09-14T09:06:40.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00052318299999999998 2024-09-14T09:06:40.284 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.284 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.284 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.285 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49716 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:40.285 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525151+0000", 2024-09-14T09:06:40.285 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.725424172, 2024-09-14T09:06:40.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034564820000000003, 2024-09-14T09:06:40.285 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.285 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.285 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:40.285 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:40.285 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:40.285 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:40.285 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49716 2024-09-14T09:06:40.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.288 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525151+0000", 2024-09-14T09:06:40.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525151+0000", 2024-09-14T09:06:40.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525152+0000", 2024-09-14T09:06:40.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5799999999999998e-07 2024-09-14T09:06:40.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525157+0000", 2024-09-14T09:06:40.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6380000000000003e-06 2024-09-14T09:06:40.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525167+0000", 2024-09-14T09:06:40.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0457e-05 2024-09-14T09:06:40.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525174+0000", 2024-09-14T09:06:40.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6220000000000003e-06 2024-09-14T09:06:40.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559237+0000", 2024-09-14T09:06:40.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034063361 2024-09-14T09:06:40.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559308+0000", 2024-09-14T09:06:40.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1124999999999994e-05 2024-09-14T09:06:40.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559716+0000", 2024-09-14T09:06:40.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000407759 2024-09-14T09:06:40.292 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.292 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.292 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.292 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49717 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:40.292 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525390+0000", 2024-09-14T09:06:40.292 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.725185374999999, 2024-09-14T09:06:40.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034552639000000003, 2024-09-14T09:06:40.292 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.292 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.293 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:40.293 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:40.293 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:40.293 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:40.293 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49717 2024-09-14T09:06:40.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.293 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525390+0000", 2024-09-14T09:06:40.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525390+0000", 2024-09-14T09:06:40.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525391+0000", 2024-09-14T09:06:40.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3099999999999997e-07 2024-09-14T09:06:40.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525395+0000", 2024-09-14T09:06:40.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.267e-06 2024-09-14T09:06:40.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525406+0000", 2024-09-14T09:06:40.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1224e-05 2024-09-14T09:06:40.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525413+0000", 2024-09-14T09:06:40.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5420000000000002e-06 2024-09-14T09:06:40.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559760+0000", 2024-09-14T09:06:40.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034347177999999999 2024-09-14T09:06:40.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559838+0000", 2024-09-14T09:06:40.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7708999999999997e-05 2024-09-14T09:06:40.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559943+0000", 2024-09-14T09:06:40.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010498800000000001 2024-09-14T09:06:40.297 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.297 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.297 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.298 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49718 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:40.298 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525514+0000", 2024-09-14T09:06:40.298 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.725061570000001, 2024-09-14T09:06:40.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034602766, 2024-09-14T09:06:40.298 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.298 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.298 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:40.298 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:40.298 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:40.298 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:40.298 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49718 2024-09-14T09:06:40.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.299 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525514+0000", 2024-09-14T09:06:40.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525514+0000", 2024-09-14T09:06:40.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525515+0000", 2024-09-14T09:06:40.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-07 2024-09-14T09:06:40.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525518+0000", 2024-09-14T09:06:40.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7220000000000001e-06 2024-09-14T09:06:40.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525528+0000", 2024-09-14T09:06:40.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4639999999999995e-06 2024-09-14T09:06:40.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525534+0000", 2024-09-14T09:06:40.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2330000000000003e-06 2024-09-14T09:06:40.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559974+0000", 2024-09-14T09:06:40.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034439866 2024-09-14T09:06:40.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560041+0000", 2024-09-14T09:06:40.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7343999999999998e-05 2024-09-14T09:06:40.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560117+0000", 2024-09-14T09:06:40.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5415999999999998e-05 2024-09-14T09:06:40.303 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.303 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.303 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.303 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49719 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:40.303 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:06:40.303 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.724942232, 2024-09-14T09:06:40.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034680563999999997, 2024-09-14T09:06:40.303 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.303 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.303 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:40.303 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:40.304 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:40.304 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:40.304 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49719 2024-09-14T09:06:40.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.304 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:06:40.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:06:40.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525634+0000", 2024-09-14T09:06:40.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4000000000000001e-07 2024-09-14T09:06:40.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525638+0000", 2024-09-14T09:06:40.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5899999999999999e-06 2024-09-14T09:06:40.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525647+0000", 2024-09-14T09:06:40.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2029999999999998e-06 2024-09-14T09:06:40.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525653+0000", 2024-09-14T09:06:40.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9850000000000003e-06 2024-09-14T09:06:40.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560150+0000", 2024-09-14T09:06:40.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034496845999999998 2024-09-14T09:06:40.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560221+0000", 2024-09-14T09:06:40.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1400000000000001e-05 2024-09-14T09:06:40.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560314+0000", 2024-09-14T09:06:40.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2800000000000006e-05 2024-09-14T09:06:40.307 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.307 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.307 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.308 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49720 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:40.308 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:06:40.308 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.724690608, 2024-09-14T09:06:40.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034919095999999997, 2024-09-14T09:06:40.308 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.308 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.308 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:40.308 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:40.308 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:40.308 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:40.308 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49720 2024-09-14T09:06:40.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.309 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:06:40.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:06:40.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525886+0000", 2024-09-14T09:06:40.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6000000000000002e-07 2024-09-14T09:06:40.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525890+0000", 2024-09-14T09:06:40.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4519999999999999e-06 2024-09-14T09:06:40.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525899+0000", 2024-09-14T09:06:40.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1090000000000004e-06 2024-09-14T09:06:40.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525906+0000", 2024-09-14T09:06:40.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0219999999999999e-06 2024-09-14T09:06:40.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560344+0000", 2024-09-14T09:06:40.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034437970999999998 2024-09-14T09:06:40.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560422+0000", 2024-09-14T09:06:40.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7857999999999999e-05 2024-09-14T09:06:40.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560804+0000", 2024-09-14T09:06:40.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00038182400000000001 2024-09-14T09:06:40.312 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.312 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.312 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.312 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49721 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:40.312 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:06:40.313 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.724643126, 2024-09-14T09:06:40.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035347380999999997, 2024-09-14T09:06:40.313 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.313 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.313 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:40.313 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:40.313 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:40.313 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:40.313 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49721 2024-09-14T09:06:40.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.313 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:06:40.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:06:40.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525933+0000", 2024-09-14T09:06:40.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T09:06:40.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525936+0000", 2024-09-14T09:06:40.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6759999999999999e-06 2024-09-14T09:06:40.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525943+0000", 2024-09-14T09:06:40.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2459999999999997e-06 2024-09-14T09:06:40.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525948+0000", 2024-09-14T09:06:40.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9429999999999999e-06 2024-09-14T09:06:40.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560845+0000", 2024-09-14T09:06:40.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034897354999999998 2024-09-14T09:06:40.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560925+0000", 2024-09-14T09:06:40.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9804999999999999e-05 2024-09-14T09:06:40.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561280+0000", 2024-09-14T09:06:40.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035480100000000002 2024-09-14T09:06:40.317 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.317 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.317 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.317 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49722 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3112960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:40.317 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:40.317 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.724619480000001, 2024-09-14T09:06:40.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035750491000000002, 2024-09-14T09:06:40.317 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.317 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.317 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:40.318 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:40.318 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:40.318 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:40.318 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49722 2024-09-14T09:06:40.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.318 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:40.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:40.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:40.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.96e-07 2024-09-14T09:06:40.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525958+0000", 2024-09-14T09:06:40.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.024e-06 2024-09-14T09:06:40.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525964+0000", 2024-09-14T09:06:40.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6699999999999999e-06 2024-09-14T09:06:40.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525969+0000", 2024-09-14T09:06:40.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3939999999999998e-06 2024-09-14T09:06:40.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561319+0000", 2024-09-14T09:06:40.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.03535013 2024-09-14T09:06:40.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561390+0000", 2024-09-14T09:06:40.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0920000000000005e-05 2024-09-14T09:06:40.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561707+0000", 2024-09-14T09:06:40.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031695699999999998 2024-09-14T09:06:40.321 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.321 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.321 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.321 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49723 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3506176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:40.322 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:40.322 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.724600497000001, 2024-09-14T09:06:40.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036113064, 2024-09-14T09:06:40.322 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.322 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.322 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:40.322 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:40.322 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:40.322 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:40.322 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49723 2024-09-14T09:06:40.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.322 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:40.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:40.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:40.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1699999999999999e-07 2024-09-14T09:06:40.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525977+0000", 2024-09-14T09:06:40.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9250000000000002e-06 2024-09-14T09:06:40.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525983+0000", 2024-09-14T09:06:40.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4620000000000004e-06 2024-09-14T09:06:40.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525987+0000", 2024-09-14T09:06:40.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.279e-06 2024-09-14T09:06:40.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561749+0000", 2024-09-14T09:06:40.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035762222000000003 2024-09-14T09:06:40.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561796+0000", 2024-09-14T09:06:40.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6959999999999998e-05 2024-09-14T09:06:40.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562088+0000", 2024-09-14T09:06:40.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029179899999999998 2024-09-14T09:06:40.326 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.326 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.326 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.326 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49724 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:40.326 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:06:40.326 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.724582341000001, 2024-09-14T09:06:40.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036449388999999999, 2024-09-14T09:06:40.326 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.326 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.327 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:40.327 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:40.327 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:40.327 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:40.327 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49724 2024-09-14T09:06:40.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.327 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:06:40.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:06:40.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525994+0000", 2024-09-14T09:06:40.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3099999999999998e-07 2024-09-14T09:06:40.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525996+0000", 2024-09-14T09:06:40.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.939e-06 2024-09-14T09:06:40.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526005+0000", 2024-09-14T09:06:40.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8430000000000005e-06 2024-09-14T09:06:40.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526010+0000", 2024-09-14T09:06:40.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9389999999999996e-06 2024-09-14T09:06:40.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562114+0000", 2024-09-14T09:06:40.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036103510999999998 2024-09-14T09:06:40.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562189+0000", 2024-09-14T09:06:40.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5162999999999998e-05 2024-09-14T09:06:40.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562443+0000", 2024-09-14T09:06:40.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000253563 2024-09-14T09:06:40.330 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.330 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.330 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.330 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:64935 3.12 3:4cd6d7cb:::rbd_data.115e21c31fcf.0000000000000049:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:06:40.331 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:06:40.331 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.499216993000001, 2024-09-14T09:06:40.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037353866999999999, 2024-09-14T09:06:40.331 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.331 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.331 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:40.331 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:40.331 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:40.331 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:40.331 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64935 2024-09-14T09:06:40.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.331 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:06:40.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:06:40.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751362+0000", 2024-09-14T09:06:40.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6899999999999998e-06 2024-09-14T09:06:40.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751378+0000", 2024-09-14T09:06:40.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6147e-05 2024-09-14T09:06:40.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751421+0000", 2024-09-14T09:06:40.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2691999999999997e-05 2024-09-14T09:06:40.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751444+0000", 2024-09-14T09:06:40.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3237000000000001e-05 2024-09-14T09:06:40.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751594+0000", 2024-09-14T09:06:40.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014947800000000001 2024-09-14T09:06:40.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751733+0000", 2024-09-14T09:06:40.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013945200000000001 2024-09-14T09:06:40.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:40.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.753315+0000", 2024-09-14T09:06:40.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0015815969999999999 2024-09-14T09:06:40.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:40.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788546+0000", 2024-09-14T09:06:40.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035231449999999997 2024-09-14T09:06:40.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:40.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788570+0000", 2024-09-14T09:06:40.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3991000000000001e-05 2024-09-14T09:06:40.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:40.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788645+0000", 2024-09-14T09:06:40.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4379000000000001e-05 2024-09-14T09:06:40.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788712+0000", 2024-09-14T09:06:40.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7754000000000002e-05 2024-09-14T09:06:40.336 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.336 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.336 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.337 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:71052 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:06:40.337 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:06:40.337 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.513007741999999, 2024-09-14T09:06:40.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14854635599999999, 2024-09-14T09:06:40.337 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.337 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.337 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:40.337 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:40.337 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:40.337 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:40.337 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 71052 2024-09-14T09:06:40.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.337 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:06:40.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:06:40.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737570+0000", 2024-09-14T09:06:40.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4770000000000002e-06 2024-09-14T09:06:40.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737584+0000", 2024-09-14T09:06:40.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3457e-05 2024-09-14T09:06:40.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737608+0000", 2024-09-14T09:06:40.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3893999999999999e-05 2024-09-14T09:06:40.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737620+0000", 2024-09-14T09:06:40.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2157000000000001e-05 2024-09-14T09:06:40.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737738+0000", 2024-09-14T09:06:40.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118409 2024-09-14T09:06:40.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737892+0000", 2024-09-14T09:06:40.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154222 2024-09-14T09:06:40.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:06:40.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.749622+0000", 2024-09-14T09:06:40.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011729537 2024-09-14T09:06:40.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:40.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885934+0000", 2024-09-14T09:06:40.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13631222200000001 2024-09-14T09:06:40.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:40.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885996+0000", 2024-09-14T09:06:40.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1555000000000003e-05 2024-09-14T09:06:40.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:40.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886060+0000", 2024-09-14T09:06:40.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3980999999999996e-05 2024-09-14T09:06:40.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886114+0000", 2024-09-14T09:06:40.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4444999999999998e-05 2024-09-14T09:06:40.342 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.342 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.342 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.342 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:71052 3.1 e71/34 ondisk, result = 0)", 2024-09-14T09:06:40.342 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:06:40.342 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.470104364000001, 2024-09-14T09:06:40.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10565334799999999, 2024-09-14T09:06:40.343 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.343 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.343 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:40.343 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:06:40.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:06:40.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780473+0000", 2024-09-14T09:06:40.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8980000000000001e-06 2024-09-14T09:06:40.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780482+0000", 2024-09-14T09:06:40.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9080000000000001e-06 2024-09-14T09:06:40.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780511+0000", 2024-09-14T09:06:40.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9490999999999999e-05 2024-09-14T09:06:40.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780527+0000", 2024-09-14T09:06:40.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.508e-05 2024-09-14T09:06:40.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885978+0000", 2024-09-14T09:06:40.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.105451328 2024-09-14T09:06:40.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885989+0000", 2024-09-14T09:06:40.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0875e-05 2024-09-14T09:06:40.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886125+0000", 2024-09-14T09:06:40.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013576800000000001 2024-09-14T09:06:40.346 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.346 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.346 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.347 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73871 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:1b [read 8339456~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:40.347 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:06:40.347 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8246193560000004, 2024-09-14T09:06:40.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034605047, 2024-09-14T09:06:40.347 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.347 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.347 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:40.347 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:40.347 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:40.347 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:40.347 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73871 2024-09-14T09:06:40.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.347 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:06:40.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:06:40.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425957+0000", 2024-09-14T09:06:40.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T09:06:40.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425961+0000", 2024-09-14T09:06:40.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8279999999999996e-06 2024-09-14T09:06:40.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425985+0000", 2024-09-14T09:06:40.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4182e-05 2024-09-14T09:06:40.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425997+0000", 2024-09-14T09:06:40.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.132e-05 2024-09-14T09:06:40.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460070+0000", 2024-09-14T09:06:40.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034072913000000003 2024-09-14T09:06:40.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460298+0000", 2024-09-14T09:06:40.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022819 2024-09-14T09:06:40.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460561+0000", 2024-09-14T09:06:40.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026341400000000001 2024-09-14T09:06:40.351 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.351 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.351 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.351 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:74276 3.12 3:49b683dd:::rbd_data.115ea8d131ff.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1376256~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:40.351 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:06:40.351 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.5654722660000004, 2024-09-14T09:06:40.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036482939999999998, 2024-09-14T09:06:40.351 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:40.352 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:40.352 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:40.352 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:40.352 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:40.352 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:40.352 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74276 2024-09-14T09:06:40.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.352 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:40.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:40.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:06:40.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:40.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:06:40.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:40.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:40.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685105+0000", 2024-09-14T09:06:40.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9520000000000001e-06 2024-09-14T09:06:40.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:40.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686530+0000", 2024-09-14T09:06:40.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001424629 2024-09-14T09:06:40.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:40.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686550+0000", 2024-09-14T09:06:40.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0472e-05 2024-09-14T09:06:40.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:40.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686561+0000", 2024-09-14T09:06:40.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T09:06:40.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:40.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686671+0000", 2024-09-14T09:06:40.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110621 2024-09-14T09:06:40.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:40.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686919+0000", 2024-09-14T09:06:40.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024790199999999999 2024-09-14T09:06:40.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:40.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.687375+0000", 2024-09-14T09:06:40.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00045572100000000002 2024-09-14T09:06:40.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:40.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.710871+0000", 2024-09-14T09:06:40.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023496481999999999 2024-09-14T09:06:40.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:40.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721478+0000", 2024-09-14T09:06:40.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01060709 2024-09-14T09:06:40.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:40.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721534+0000", 2024-09-14T09:06:40.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5912999999999998e-05 2024-09-14T09:06:40.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:40.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:40.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:40.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721586+0000", 2024-09-14T09:06:40.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1792000000000002e-05 2024-09-14T09:06:40.357 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.357 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.357 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.357 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:40.357 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:40.357 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:40.359 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:06:40.360 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is set 2024-09-14T09:06:40.360 INFO:teuthology.orchestra.run.smithi176.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T09:06:40.360 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:40.363+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:40.372 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T09:06:40.459 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:40.459+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:40.531 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:40.531 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:40.531 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:40.531 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:40.547 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:06:40.590 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:40.591+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:40.637 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:06:40.661 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:40.663+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:40.703 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:40.704 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:40.704 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:40.704 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:40.704 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:40.717 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:06:40.762 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:40.763+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:40.769 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:40.769 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:40.786 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:40.786 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:40.802 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:40.803 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:40.819 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:40.819 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:40.831 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:40.831 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:40.839 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:40.840 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:40.845 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:06:40.861 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:40.861 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:40.862 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:40.863+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:40.881 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:40.881 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:40.901 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:40.905 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:40.958 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:40.958 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:40.963 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:40.963+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:40.971 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:06:41.064 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:41.067+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:41.128 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:41.128 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:41.143 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:06:41.190 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:41.191+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:41.258 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:41.258 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:41.265 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:41.263+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:41.270 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:06:41.329 INFO:teuthology.orchestra.run.smithi176.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T09:06:41.363 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T09:06:41.409 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:41.407+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:41.434 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:41.434 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:41.447 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:06:41.466 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:41.467+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:41.567 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:41.567+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:41.603 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:41.603 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:41.617 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:06:41.702 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:41.703+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:41.769 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:41.771+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:41.770 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:41.771 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:41.783 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:06:41.829 INFO:teuthology.orchestra.run.smithi176.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T09:06:41.848 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T09:06:41.894 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:41.895+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:41.919 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:41.919 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:41.933 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:06:41.979 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:41.983+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:42.050 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:42.050 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:42.064 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:06:42.107 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:42.111+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:42.173 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:42.171+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:42.231 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:42.231 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:42.246 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:06:42.291 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:42.295+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:42.364 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:42.364 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:42.374 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T09:06:42.375 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:42.375+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:42.378 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:06:42.475 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:42.479+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:42.543 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:42.543 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:42.558 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:06:42.577 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:42.575+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:42.678 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:42.679+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:42.725 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:42.725 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:42.739 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:06:42.779 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:42.779+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:42.842 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is unset 2024-09-14T09:06:42.843 INFO:teuthology.orchestra.run.smithi176.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T09:06:42.872 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T09:06:42.914 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:42.915+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:42.918 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:06:42.928 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:42.928 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:42.946 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:06:42.991 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:42.995+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:43.068 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:43.068 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:43.082 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:43.083+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:43.085 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:06:43.096 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:43.097 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:43.115 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:43.115 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:43.130 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-14T09:06:43.134 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:43.134 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:43.162 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:43.162 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:43.180 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:43.180 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:43.183 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:43.187+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:43.198 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:43.199 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:43.215 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:43.216 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:43.218 INFO:teuthology.orchestra.run.smithi176.stdout:3600 write 0x510032f thru 0x510e6bf (0xe391 bytes) 2024-09-14T09:06:43.231 INFO:teuthology.orchestra.run.smithi176.stdout:3601 punch from 0x5ce13ee to 0x5ce200c, (0xc1e bytes) 2024-09-14T09:06:43.235 INFO:teuthology.orchestra.run.smithi176.stdout:3603 trunc from 0xd80ceac to 0xae1268a 2024-09-14T09:06:43.237 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:43.237 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:43.259 INFO:teuthology.orchestra.run.smithi176.stdout:3604 write 0x253d850 thru 0x254467e (0x6e2f bytes) 2024-09-14T09:06:43.269 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:43.269 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:43.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:43.269 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:43.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.270 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19520 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:43.271 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475212+0000", 2024-09-14T09:06:43.271 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.797190565999998, 2024-09-14T09:06:43.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067435885000000001, 2024-09-14T09:06:43.271 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.271 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.272 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:43.272 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.272 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.272 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.272 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19520 2024-09-14T09:06:43.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.273 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475212+0000", 2024-09-14T09:06:43.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475212+0000", 2024-09-14T09:06:43.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475213+0000", 2024-09-14T09:06:43.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0440000000000001e-06 2024-09-14T09:06:43.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475218+0000", 2024-09-14T09:06:43.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1930000000000001e-06 2024-09-14T09:06:43.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475237+0000", 2024-09-14T09:06:43.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9003000000000001e-05 2024-09-14T09:06:43.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475251+0000", 2024-09-14T09:06:43.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3720000000000001e-05 2024-09-14T09:06:43.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.541342+0000", 2024-09-14T09:06:43.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066091227000000002 2024-09-14T09:06:43.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.541492+0000", 2024-09-14T09:06:43.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000149221 2024-09-14T09:06:43.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542648+0000", 2024-09-14T09:06:43.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011564769999999999 2024-09-14T09:06:43.281 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.282 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.282 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.282 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19530 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:43.283 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475326+0000", 2024-09-14T09:06:43.283 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.797076382999997, 2024-09-14T09:06:43.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067030846000000005, 2024-09-14T09:06:43.283 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.283 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.284 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:43.284 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.284 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.284 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.284 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19530 2024-09-14T09:06:43.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.285 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475326+0000", 2024-09-14T09:06:43.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475326+0000", 2024-09-14T09:06:43.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475328+0000", 2024-09-14T09:06:43.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.133e-06 2024-09-14T09:06:43.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475358+0000", 2024-09-14T09:06:43.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0253e-05 2024-09-14T09:06:43.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475378+0000", 2024-09-14T09:06:43.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.033e-05 2024-09-14T09:06:43.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475405+0000", 2024-09-14T09:06:43.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6568999999999999e-05 2024-09-14T09:06:43.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.541989+0000", 2024-09-14T09:06:43.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066583956999999999 2024-09-14T09:06:43.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542215+0000", 2024-09-14T09:06:43.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022592200000000001 2024-09-14T09:06:43.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542357+0000", 2024-09-14T09:06:43.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014268199999999999 2024-09-14T09:06:43.293 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.293 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.293 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.294 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19531 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15073280~11919] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:43.294 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475425+0000", 2024-09-14T09:06:43.294 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.796978088000003, 2024-09-14T09:06:43.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067651759000000006, 2024-09-14T09:06:43.295 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.295 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.295 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:43.295 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.295 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.295 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.296 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19531 2024-09-14T09:06:43.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.296 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475425+0000", 2024-09-14T09:06:43.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475425+0000", 2024-09-14T09:06:43.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475426+0000", 2024-09-14T09:06:43.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2440000000000001e-06 2024-09-14T09:06:43.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475465+0000", 2024-09-14T09:06:43.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8643000000000003e-05 2024-09-14T09:06:43.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475485+0000", 2024-09-14T09:06:43.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0645999999999999e-05 2024-09-14T09:06:43.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475499+0000", 2024-09-14T09:06:43.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3312999999999999e-05 2024-09-14T09:06:43.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542389+0000", 2024-09-14T09:06:43.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066890885999999997 2024-09-14T09:06:43.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.542481+0000", 2024-09-14T09:06:43.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1481e-05 2024-09-14T09:06:43.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543076+0000", 2024-09-14T09:06:43.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00059554599999999997 2024-09-14T09:06:43.300 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.301 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.301 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.301 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19532 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15335424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:43.301 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475521+0000", 2024-09-14T09:06:43.301 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.796881442, 2024-09-14T09:06:43.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067895159999999996, 2024-09-14T09:06:43.301 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.301 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.301 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:43.301 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.301 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.302 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.302 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19532 2024-09-14T09:06:43.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.302 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475521+0000", 2024-09-14T09:06:43.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475521+0000", 2024-09-14T09:06:43.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475523+0000", 2024-09-14T09:06:43.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.277e-06 2024-09-14T09:06:43.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475528+0000", 2024-09-14T09:06:43.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8629999999999996e-06 2024-09-14T09:06:43.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475546+0000", 2024-09-14T09:06:43.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7750000000000001e-05 2024-09-14T09:06:43.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475560+0000", 2024-09-14T09:06:43.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3763e-05 2024-09-14T09:06:43.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543144+0000", 2024-09-14T09:06:43.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067584489999999997 2024-09-14T09:06:43.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543261+0000", 2024-09-14T09:06:43.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011696699999999999 2024-09-14T09:06:43.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543416+0000", 2024-09-14T09:06:43.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015505 2024-09-14T09:06:43.305 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.305 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.305 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.306 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19533 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:43.306 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475578+0000", 2024-09-14T09:06:43.306 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.796824299000001, 2024-09-14T09:06:43.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068124984999999999, 2024-09-14T09:06:43.306 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.306 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.306 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:43.306 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.306 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.306 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.306 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19533 2024-09-14T09:06:43.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.306 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475578+0000", 2024-09-14T09:06:43.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475578+0000", 2024-09-14T09:06:43.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475580+0000", 2024-09-14T09:06:43.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.08e-06 2024-09-14T09:06:43.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475585+0000", 2024-09-14T09:06:43.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4970000000000001e-06 2024-09-14T09:06:43.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475602+0000", 2024-09-14T09:06:43.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7459999999999999e-05 2024-09-14T09:06:43.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475615+0000", 2024-09-14T09:06:43.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2339e-05 2024-09-14T09:06:43.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543466+0000", 2024-09-14T09:06:43.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067850745000000004 2024-09-14T09:06:43.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543577+0000", 2024-09-14T09:06:43.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000111908 2024-09-14T09:06:43.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543703+0000", 2024-09-14T09:06:43.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012595600000000001 2024-09-14T09:06:43.310 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.310 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.310 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.311 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19534 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:43.311 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475634+0000", 2024-09-14T09:06:43.311 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.796768522999997, 2024-09-14T09:06:43.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068358706000000005, 2024-09-14T09:06:43.311 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.311 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.311 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:43.311 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.311 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.311 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.311 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19534 2024-09-14T09:06:43.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.311 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475634+0000", 2024-09-14T09:06:43.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475634+0000", 2024-09-14T09:06:43.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475635+0000", 2024-09-14T09:06:43.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.187e-06 2024-09-14T09:06:43.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475641+0000", 2024-09-14T09:06:43.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5629999999999999e-06 2024-09-14T09:06:43.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475659+0000", 2024-09-14T09:06:43.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7649999999999999e-05 2024-09-14T09:06:43.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475671+0000", 2024-09-14T09:06:43.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2583e-05 2024-09-14T09:06:43.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543748+0000", 2024-09-14T09:06:43.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068076740999999996 2024-09-14T09:06:43.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543855+0000", 2024-09-14T09:06:43.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010676800000000001 2024-09-14T09:06:43.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.543993+0000", 2024-09-14T09:06:43.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138214 2024-09-14T09:06:43.315 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.315 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.315 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.315 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19535 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:43.315 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475691+0000", 2024-09-14T09:06:43.315 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.796711512999998, 2024-09-14T09:06:43.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068607583999999999, 2024-09-14T09:06:43.315 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.315 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.316 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:43.316 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.316 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.316 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.316 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19535 2024-09-14T09:06:43.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.316 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475691+0000", 2024-09-14T09:06:43.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475691+0000", 2024-09-14T09:06:43.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475693+0000", 2024-09-14T09:06:43.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6270000000000001e-06 2024-09-14T09:06:43.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475699+0000", 2024-09-14T09:06:43.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9059999999999999e-06 2024-09-14T09:06:43.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475717+0000", 2024-09-14T09:06:43.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-09-14T09:06:43.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475730+0000", 2024-09-14T09:06:43.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.308e-05 2024-09-14T09:06:43.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544042+0000", 2024-09-14T09:06:43.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068312791999999997 2024-09-14T09:06:43.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544168+0000", 2024-09-14T09:06:43.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012549699999999999 2024-09-14T09:06:43.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544299+0000", 2024-09-14T09:06:43.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013094199999999999 2024-09-14T09:06:43.319 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.319 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.320 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.320 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19536 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:43.320 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475749+0000", 2024-09-14T09:06:43.320 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.796654234000002, 2024-09-14T09:06:43.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068917143, 2024-09-14T09:06:43.320 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.320 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.320 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:43.320 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.320 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.320 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.320 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19536 2024-09-14T09:06:43.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.321 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475749+0000", 2024-09-14T09:06:43.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475749+0000", 2024-09-14T09:06:43.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475750+0000", 2024-09-14T09:06:43.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0330000000000001e-06 2024-09-14T09:06:43.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475755+0000", 2024-09-14T09:06:43.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9740000000000001e-06 2024-09-14T09:06:43.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475772+0000", 2024-09-14T09:06:43.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7750000000000001e-05 2024-09-14T09:06:43.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475802+0000", 2024-09-14T09:06:43.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0190000000000001e-05 2024-09-14T09:06:43.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544345+0000", 2024-09-14T09:06:43.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068542140000000001 2024-09-14T09:06:43.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544454+0000", 2024-09-14T09:06:43.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109447 2024-09-14T09:06:43.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544666+0000", 2024-09-14T09:06:43.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000211609 2024-09-14T09:06:43.324 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.324 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.324 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.324 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19537 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:43.324 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475822+0000", 2024-09-14T09:06:43.324 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.796580276999997, 2024-09-14T09:06:43.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069185398999999995, 2024-09-14T09:06:43.324 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.324 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.324 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:43.325 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.325 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.325 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.325 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19537 2024-09-14T09:06:43.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.325 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475822+0000", 2024-09-14T09:06:43.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475822+0000", 2024-09-14T09:06:43.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475824+0000", 2024-09-14T09:06:43.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2559999999999999e-06 2024-09-14T09:06:43.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475829+0000", 2024-09-14T09:06:43.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6840000000000001e-06 2024-09-14T09:06:43.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475848+0000", 2024-09-14T09:06:43.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8583000000000001e-05 2024-09-14T09:06:43.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475861+0000", 2024-09-14T09:06:43.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2802999999999999e-05 2024-09-14T09:06:43.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544716+0000", 2024-09-14T09:06:43.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.068855036999999994 2024-09-14T09:06:43.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.544827+0000", 2024-09-14T09:06:43.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011113200000000001 2024-09-14T09:06:43.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545008+0000", 2024-09-14T09:06:43.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018090399999999999 2024-09-14T09:06:43.328 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.328 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.328 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.328 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19538 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:43.328 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475880+0000", 2024-09-14T09:06:43.328 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.796522602000003, 2024-09-14T09:06:43.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069740072, 2024-09-14T09:06:43.329 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.329 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.329 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:43.329 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.329 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.329 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.329 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19538 2024-09-14T09:06:43.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.329 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475880+0000", 2024-09-14T09:06:43.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475880+0000", 2024-09-14T09:06:43.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475881+0000", 2024-09-14T09:06:43.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.124e-06 2024-09-14T09:06:43.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475886+0000", 2024-09-14T09:06:43.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0130000000000001e-06 2024-09-14T09:06:43.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475904+0000", 2024-09-14T09:06:43.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7779999999999999e-05 2024-09-14T09:06:43.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475918+0000", 2024-09-14T09:06:43.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.385e-05 2024-09-14T09:06:43.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545065+0000", 2024-09-14T09:06:43.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069147307000000005 2024-09-14T09:06:43.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545431+0000", 2024-09-14T09:06:43.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036538400000000001 2024-09-14T09:06:43.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545620+0000", 2024-09-14T09:06:43.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000189614 2024-09-14T09:06:43.333 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.333 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.333 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.333 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19539 3.a 3:5677e69d:::rbd_data.115e564e8eff.0000000000000001:7 [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:43.333 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.475936+0000", 2024-09-14T09:06:43.333 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.796467018000001, 2024-09-14T09:06:43.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069860063999999999, 2024-09-14T09:06:43.334 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.334 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.334 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:43.334 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.334 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.334 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.334 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19539 2024-09-14T09:06:43.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.334 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475936+0000", 2024-09-14T09:06:43.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475936+0000", 2024-09-14T09:06:43.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475937+0000", 2024-09-14T09:06:43.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5960000000000001e-06 2024-09-14T09:06:43.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475943+0000", 2024-09-14T09:06:43.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2340000000000002e-06 2024-09-14T09:06:43.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475960+0000", 2024-09-14T09:06:43.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7592999999999999e-05 2024-09-14T09:06:43.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.475986+0000", 2024-09-14T09:06:43.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6319e-05 2024-09-14T09:06:43.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545650+0000", 2024-09-14T09:06:43.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069663613999999999 2024-09-14T09:06:43.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545714+0000", 2024-09-14T09:06:43.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4209999999999997e-05 2024-09-14T09:06:43.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.545796+0000", 2024-09-14T09:06:43.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1497999999999998e-05 2024-09-14T09:06:43.338 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.338 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.338 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.338 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:19541 3.10 3:0ae7d8d9:::rbd_data.115e4c71d6e8.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 4194304~8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:43.338 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.497517+0000", 2024-09-14T09:06:43.338 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.774885947000001, 2024-09-14T09:06:43.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.069874055000000004, 2024-09-14T09:06:43.339 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.339 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.339 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:43.339 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.339 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.339 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.339 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 19541 2024-09-14T09:06:43.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.339 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.497517+0000", 2024-09-14T09:06:43.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.497517+0000", 2024-09-14T09:06:43.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.497520+0000", 2024-09-14T09:06:43.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0730000000000001e-06 2024-09-14T09:06:43.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.504875+0000", 2024-09-14T09:06:43.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007355136 2024-09-14T09:06:43.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.504926+0000", 2024-09-14T09:06:43.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1146000000000001e-05 2024-09-14T09:06:43.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.504950+0000", 2024-09-14T09:06:43.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3638999999999998e-05 2024-09-14T09:06:43.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.505139+0000", 2024-09-14T09:06:43.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018904300000000001 2024-09-14T09:06:43.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.505402+0000", 2024-09-14T09:06:43.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026354699999999999 2024-09-14T09:06:43.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:43.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.506208+0000", 2024-09-14T09:06:43.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00080583399999999996 2024-09-14T09:06:43.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:43.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.566904+0000", 2024-09-14T09:06:43.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.060695510000000001 2024-09-14T09:06:43.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:43.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.567219+0000", 2024-09-14T09:06:43.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031509000000000002 2024-09-14T09:06:43.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:43.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.567319+0000", 2024-09-14T09:06:43.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9804999999999997e-05 2024-09-14T09:06:43.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.567391+0000", 2024-09-14T09:06:43.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2231999999999996e-05 2024-09-14T09:06:43.344 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.344 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.344 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.344 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:30891 3.10 3:0ac3ecfb:::rbd_data.115ebc3c67d1.0000000000000004:c [read 212992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T09:06:43.345 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:53.069740+0000", 2024-09-14T09:06:43.345 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.202663158, 2024-09-14T09:06:43.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066940186999999998, 2024-09-14T09:06:43.345 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.345 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.345 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:43.345 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.345 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.345 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.345 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 30891 2024-09-14T09:06:43.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.346 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069740+0000", 2024-09-14T09:06:43.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.069740+0000", 2024-09-14T09:06:43.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070603+0000", 2024-09-14T09:06:43.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00086369599999999995 2024-09-14T09:06:43.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070612+0000", 2024-09-14T09:06:43.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3170000000000002e-06 2024-09-14T09:06:43.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070631+0000", 2024-09-14T09:06:43.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8936999999999998e-05 2024-09-14T09:06:43.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.070645+0000", 2024-09-14T09:06:43.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4545999999999999e-05 2024-09-14T09:06:43.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136603+0000", 2024-09-14T09:06:43.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.065957551000000003 2024-09-14T09:06:43.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136644+0000", 2024-09-14T09:06:43.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1671000000000001e-05 2024-09-14T09:06:43.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:53.136680+0000", 2024-09-14T09:06:43.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5469000000000002e-05 2024-09-14T09:06:43.352 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.352 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.353 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.353 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:37542 3.10 3:0e658405:::rbd_data.115eb07a8abd.000000000000000d:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:43.353 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.443185+0000", 2024-09-14T09:06:43.353 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.829218132000001, 2024-09-14T09:06:43.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.091143627000000005, 2024-09-14T09:06:43.353 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.353 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.353 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:43.353 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.353 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.353 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.353 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37542 2024-09-14T09:06:43.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.354 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443185+0000", 2024-09-14T09:06:43.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443185+0000", 2024-09-14T09:06:43.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443187+0000", 2024-09-14T09:06:43.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.367e-06 2024-09-14T09:06:43.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443199+0000", 2024-09-14T09:06:43.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2153e-05 2024-09-14T09:06:43.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443225+0000", 2024-09-14T09:06:43.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5457e-05 2024-09-14T09:06:43.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443248+0000", 2024-09-14T09:06:43.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2963e-05 2024-09-14T09:06:43.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443400+0000", 2024-09-14T09:06:43.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015241199999999999 2024-09-14T09:06:43.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.443542+0000", 2024-09-14T09:06:43.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014224600000000001 2024-09-14T09:06:43.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:43.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.448965+0000", 2024-09-14T09:06:43.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054226839999999997 2024-09-14T09:06:43.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:43.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534136+0000", 2024-09-14T09:06:43.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.085170939000000001 2024-09-14T09:06:43.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:43.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534164+0000", 2024-09-14T09:06:43.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7863000000000001e-05 2024-09-14T09:06:43.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:43.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534250+0000", 2024-09-14T09:06:43.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5876000000000002e-05 2024-09-14T09:06:43.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534328+0000", 2024-09-14T09:06:43.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8666999999999997e-05 2024-09-14T09:06:43.358 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.358 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.358 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.358 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:37542 3.10 e48/28 ondisk, result = 0)", 2024-09-14T09:06:43.359 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.464008+0000", 2024-09-14T09:06:43.359 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.808394722999999, 2024-09-14T09:06:43.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.070141355000000002, 2024-09-14T09:06:43.359 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.359 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.359 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:43.359 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464008+0000", 2024-09-14T09:06:43.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464008+0000", 2024-09-14T09:06:43.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464010+0000", 2024-09-14T09:06:43.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.547e-06 2024-09-14T09:06:43.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464017+0000", 2024-09-14T09:06:43.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4930000000000001e-06 2024-09-14T09:06:43.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464047+0000", 2024-09-14T09:06:43.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0090000000000002e-05 2024-09-14T09:06:43.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.464061+0000", 2024-09-14T09:06:43.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4003e-05 2024-09-14T09:06:43.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534107+0000", 2024-09-14T09:06:43.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.070045590000000005 2024-09-14T09:06:43.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534126+0000", 2024-09-14T09:06:43.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8893e-05 2024-09-14T09:06:43.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.534149+0000", 2024-09-14T09:06:43.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3739000000000001e-05 2024-09-14T09:06:43.362 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.362 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.362 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.362 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38003 3.10 3:0e658405:::rbd_data.115eb07a8abd.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:43.362 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:57.392070+0000", 2024-09-14T09:06:43.363 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.880332949, 2024-09-14T09:06:43.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11697451, 2024-09-14T09:06:43.363 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.363 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.363 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:43.363 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.363 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.363 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.363 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38003 2024-09-14T09:06:43.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.363 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392070+0000", 2024-09-14T09:06:43.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392070+0000", 2024-09-14T09:06:43.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392072+0000", 2024-09-14T09:06:43.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6529999999999998e-06 2024-09-14T09:06:43.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392086+0000", 2024-09-14T09:06:43.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.332e-05 2024-09-14T09:06:43.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392139+0000", 2024-09-14T09:06:43.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2815999999999997e-05 2024-09-14T09:06:43.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392155+0000", 2024-09-14T09:06:43.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6796999999999999e-05 2024-09-14T09:06:43.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392296+0000", 2024-09-14T09:06:43.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014107900000000001 2024-09-14T09:06:43.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.392443+0000", 2024-09-14T09:06:43.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014654499999999999 2024-09-14T09:06:43.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:43.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.399418+0000", 2024-09-14T09:06:43.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069751049999999997 2024-09-14T09:06:43.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:43.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508607+0000", 2024-09-14T09:06:43.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10918908400000001 2024-09-14T09:06:43.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:43.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508748+0000", 2024-09-14T09:06:43.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014079699999999999 2024-09-14T09:06:43.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:43.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508921+0000", 2024-09-14T09:06:43.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017345899999999999 2024-09-14T09:06:43.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.509044+0000", 2024-09-14T09:06:43.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000122855 2024-09-14T09:06:43.368 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.368 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.368 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.369 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:38003 3.10 e48/28 ondisk, result = 0)", 2024-09-14T09:06:43.369 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:57.413977+0000", 2024-09-14T09:06:43.369 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.858425726, 2024-09-14T09:06:43.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.095088000000000006, 2024-09-14T09:06:43.369 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.369 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.369 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:43.369 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413977+0000", 2024-09-14T09:06:43.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413977+0000", 2024-09-14T09:06:43.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413979+0000", 2024-09-14T09:06:43.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.933e-06 2024-09-14T09:06:43.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.413988+0000", 2024-09-14T09:06:43.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2769999999999994e-06 2024-09-14T09:06:43.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.414021+0000", 2024-09-14T09:06:43.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2589999999999998e-05 2024-09-14T09:06:43.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.414040+0000", 2024-09-14T09:06:43.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8796000000000001e-05 2024-09-14T09:06:43.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508701+0000", 2024-09-14T09:06:43.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.094661067000000002 2024-09-14T09:06:43.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.508729+0000", 2024-09-14T09:06:43.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8677e-05 2024-09-14T09:06:43.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:57.509065+0000", 2024-09-14T09:06:43.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033565999999999999 2024-09-14T09:06:43.373 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.373 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.373 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.373 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65139 3.a 3:54c90b7d:::rbd_data.115e21c31fcf.0000000000000008:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:06:43.373 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:06:43.373 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 21.892690022, 2024-09-14T09:06:43.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14429895700000001, 2024-09-14T09:06:43.374 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.374 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.374 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:43.374 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.374 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.374 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.374 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65139 2024-09-14T09:06:43.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.374 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:06:43.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:06:43.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379715+0000", 2024-09-14T09:06:43.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.576e-06 2024-09-14T09:06:43.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379724+0000", 2024-09-14T09:06:43.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1779999999999992e-06 2024-09-14T09:06:43.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379746+0000", 2024-09-14T09:06:43.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1661000000000001e-05 2024-09-14T09:06:43.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379759+0000", 2024-09-14T09:06:43.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.324e-05 2024-09-14T09:06:43.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379882+0000", 2024-09-14T09:06:43.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012239400000000001 2024-09-14T09:06:43.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.380023+0000", 2024-09-14T09:06:43.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000141402 2024-09-14T09:06:43.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:43.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.391553+0000", 2024-09-14T09:06:43.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011530197000000001 2024-09-14T09:06:43.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:43.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523927+0000", 2024-09-14T09:06:43.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.132373935 2024-09-14T09:06:43.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:43.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523941+0000", 2024-09-14T09:06:43.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3326000000000001e-05 2024-09-14T09:06:43.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:43.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523977+0000", 2024-09-14T09:06:43.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6185e-05 2024-09-14T09:06:43.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.524012+0000", 2024-09-14T09:06:43.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4863000000000001e-05 2024-09-14T09:06:43.379 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.379 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.379 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.379 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:65139 3.a e65/34 ondisk, result = 0)", 2024-09-14T09:06:43.379 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.408501+0000", 2024-09-14T09:06:43.379 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 21.863901465000001, 2024-09-14T09:06:43.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.115431275, 2024-09-14T09:06:43.379 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.379 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.379 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:43.379 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408501+0000", 2024-09-14T09:06:43.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408501+0000", 2024-09-14T09:06:43.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408504+0000", 2024-09-14T09:06:43.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2819999999999999e-06 2024-09-14T09:06:43.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408512+0000", 2024-09-14T09:06:43.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7530000000000007e-06 2024-09-14T09:06:43.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408534+0000", 2024-09-14T09:06:43.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1993000000000001e-05 2024-09-14T09:06:43.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.408550+0000", 2024-09-14T09:06:43.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5988e-05 2024-09-14T09:06:43.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523914+0000", 2024-09-14T09:06:43.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.115363542 2024-09-14T09:06:43.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523922+0000", 2024-09-14T09:06:43.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5529999999999993e-06 2024-09-14T09:06:43.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523933+0000", 2024-09-14T09:06:43.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0164e-05 2024-09-14T09:06:43.383 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.383 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.383 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.383 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:74302 3.a 3:57b145b8:::rbd_data.115ea8d131ff.0000000000000045:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:43.383 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:35.808590+0000", 2024-09-14T09:06:43.383 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 7.463812345, 2024-09-14T09:06:43.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.080730308000000001, 2024-09-14T09:06:43.383 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.383 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.383 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:43.384 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.384 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.384 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.384 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 74302 2024-09-14T09:06:43.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.384 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.808590+0000", 2024-09-14T09:06:43.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.808590+0000", 2024-09-14T09:06:43.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.808593+0000", 2024-09-14T09:06:43.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8700000000000001e-06 2024-09-14T09:06:43.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.808605+0000", 2024-09-14T09:06:43.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2e-05 2024-09-14T09:06:43.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.808637+0000", 2024-09-14T09:06:43.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2023999999999997e-05 2024-09-14T09:06:43.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.808660+0000", 2024-09-14T09:06:43.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2506000000000001e-05 2024-09-14T09:06:43.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.808822+0000", 2024-09-14T09:06:43.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162509 2024-09-14T09:06:43.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.808963+0000", 2024-09-14T09:06:43.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000140765 2024-09-14T09:06:43.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:43.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.813070+0000", 2024-09-14T09:06:43.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004106831 2024-09-14T09:06:43.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:43.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.889186+0000", 2024-09-14T09:06:43.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.076116122999999994 2024-09-14T09:06:43.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:43.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.889233+0000", 2024-09-14T09:06:43.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6591000000000002e-05 2024-09-14T09:06:43.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:43.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.889281+0000", 2024-09-14T09:06:43.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8355e-05 2024-09-14T09:06:43.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:35.889321+0000", 2024-09-14T09:06:43.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9734000000000003e-05 2024-09-14T09:06:43.388 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.389 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.389 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.389 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.389 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.389 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:43.391 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:06:43.391 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:43.391+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:43.392 INFO:teuthology.orchestra.run.smithi176.stdout:3605 trunc from 0xae1268a to 0x26b1cb1 2024-09-14T09:06:43.392 INFO:teuthology.orchestra.run.smithi176.stdout:3606 write 0x48da285 thru 0x48e879a (0xe516 bytes) 2024-09-14T09:06:43.392 INFO:teuthology.orchestra.run.smithi176.stdout:3607 read 0x3c94504 thru 0x3c961ab (0x1ca8 bytes) 2024-09-14T09:06:43.392 INFO:teuthology.orchestra.run.smithi176.stdout:3608 write 0x60156f8 thru 0x601591c (0x225 bytes) 2024-09-14T09:06:43.392 INFO:teuthology.orchestra.run.smithi176.stdout:3609 punch from 0x2ec2063 to 0x2ec77a5, (0x5742 bytes) 2024-09-14T09:06:43.393 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:43.393 INFO:teuthology.orchestra.run.smithi176.stdout:3611 read 0x1bf315f thru 0x1bfc4a8 (0x934a bytes) 2024-09-14T09:06:43.393 INFO:teuthology.orchestra.run.smithi176.stdout:3612 writesame 0xc781a5e thru 0xc7909a9 data_size 0x1e(0xef4c bytes) 2024-09-14T09:06:43.393 INFO:teuthology.orchestra.run.smithi176.stdout:3613 read 0x8e257f2 thru 0x8e345e0 (0xedef bytes) 2024-09-14T09:06:43.393 INFO:teuthology.orchestra.run.smithi176.stdout:3615 write 0x4e34ee thru 0x4f32cd (0xfde0 bytes) 2024-09-14T09:06:43.394 INFO:teuthology.orchestra.run.smithi176.stdout:3618 writesame 0xca0c2f4 thru 0xca1555c data_size 0x1(0x9269 bytes) 2024-09-14T09:06:43.394 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:43.394 INFO:teuthology.orchestra.run.smithi176.stdout:3620 write 0x7f8dfe3 thru 0x7f99387 (0xb3a5 bytes) 2024-09-14T09:06:43.403 INFO:teuthology.orchestra.run.smithi176.stdout:3621 write 0x7468508 thru 0x7476aed (0xe5e6 bytes) 2024-09-14T09:06:43.421 INFO:teuthology.orchestra.run.smithi176.stdout:3622 write 0xd76e00a thru 0xd77d031 (0xf028 bytes) 2024-09-14T09:06:43.429 INFO:teuthology.orchestra.run.smithi176.stdout:3623 write 0xa0daf49 thru 0xa0e6b4c (0xbc04 bytes) 2024-09-14T09:06:43.437 INFO:teuthology.orchestra.run.smithi176.stdout:3625 read 0x1c710bf thru 0x1c78091 (0x6fd3 bytes) 2024-09-14T09:06:43.438 INFO:teuthology.orchestra.run.smithi176.stdout:3627 write 0x7b5f038 thru 0x7b6e6e2 (0xf6ab bytes) 2024-09-14T09:06:43.448 INFO:teuthology.orchestra.run.smithi176.stdout:3628 read 0x979e3dc thru 0x97acdd7 (0xe9fc bytes) 2024-09-14T09:06:43.448 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:43.458 INFO:teuthology.orchestra.run.smithi176.stdout:3630 write 0xdd0f0c2 thru 0xdd194ef (0xa42e bytes) 2024-09-14T09:06:43.465 INFO:teuthology.orchestra.run.smithi176.stdout:3631 writesame 0x263f73c thru 0x2643538 data_size 0x1(0x3dfd bytes) 2024-09-14T09:06:43.491 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:43.491+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:43.509 INFO:teuthology.orchestra.run.smithi176.stdout:3633 write 0x63f3a28 thru 0x63f7e66 (0x443f bytes) 2024-09-14T09:06:43.516 INFO:teuthology.orchestra.run.smithi176.stdout:3634 punch from 0x7aa2c2e to 0x7aa43c5, (0x1797 bytes) 2024-09-14T09:06:43.516 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:43.516 INFO:teuthology.orchestra.run.smithi176.stdout:3638 punch from 0x62ff686 to 0x6306927, (0x72a1 bytes) 2024-09-14T09:06:43.517 INFO:teuthology.orchestra.run.smithi176.stdout:3640 punch from 0xdc9df25 to 0xdca3cd6, (0x5db1 bytes) 2024-09-14T09:06:43.517 INFO:teuthology.orchestra.run.smithi176.stdout:3641 write 0xa9fa535 thru 0xa9fbc4c (0x1718 bytes) 2024-09-14T09:06:43.522 INFO:teuthology.orchestra.run.smithi176.stdout:3642 read 0x79e4f1f thru 0x79ece5c (0x7f3e bytes) 2024-09-14T09:06:43.522 INFO:teuthology.orchestra.run.smithi176.stdout:3643 write 0x7f4d168 thru 0x7f58a3a (0xb8d3 bytes) 2024-09-14T09:06:43.529 INFO:teuthology.orchestra.run.smithi176.stdout:3645 read 0x1043ada thru 0x1052185 (0xe6ac bytes) 2024-09-14T09:06:43.560 INFO:teuthology.orchestra.run.smithi176.stdout:3646 trunc from 0xdd194f0 to 0x1f0f7df 2024-09-14T09:06:43.562 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:43.562 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:43.563 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:06:43.563 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:43.582 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:06:43.594 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:43.595+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:43.632 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:43.645 INFO:teuthology.orchestra.run.smithi176.stdout:3648 writesame 0x40f54c6 thru 0x4100d50 data_size 0x11(0xb88b bytes) 2024-09-14T09:06:43.676 INFO:teuthology.orchestra.run.smithi176.stdout:3649 write 0xe3918a4 thru 0xe391d3f (0x49c bytes) 2024-09-14T09:06:43.680 INFO:teuthology.orchestra.run.smithi176.stdout:3650 writesame 0x1876163 thru 0x187f6c2 data_size 0x50(0x9560 bytes) 2024-09-14T09:06:43.692 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:43.695+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:43.695 INFO:teuthology.orchestra.run.smithi176.stdout:3651 writesame 0x53ba1b8 thru 0x53c1cbb data_size 0x2(0x7b04 bytes) 2024-09-14T09:06:43.715 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:43.716 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:06:43.716 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:06:43.716 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:06:43.716 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:43.726 INFO:teuthology.orchestra.run.smithi176.stdout:3653 punch from 0x4ab6b65 to 0x4ac563c, (0xead7 bytes) 2024-09-14T09:06:43.734 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:06:43.737 INFO:teuthology.orchestra.run.smithi176.stdout:3654 write 0xec77c26 thru 0xec81c39 (0xa014 bytes) 2024-09-14T09:06:43.747 INFO:teuthology.orchestra.run.smithi176.stdout:3655 punch from 0x15d400 to 0x15d789, (0x389 bytes) 2024-09-14T09:06:43.747 INFO:teuthology.orchestra.run.smithi176.stdout:3656 punch from 0xa707ad8 to 0xa716ecb, (0xf3f3 bytes) 2024-09-14T09:06:43.747 INFO:teuthology.orchestra.run.smithi176.stdout:3657 write 0x863e0c1 thru 0x8647201 (0x9141 bytes) 2024-09-14T09:06:43.758 INFO:teuthology.orchestra.run.smithi176.stdout:3660 punch from 0x8051e95 to 0x805d106, (0xb271 bytes) 2024-09-14T09:06:43.789 INFO:teuthology.orchestra.run.smithi176.stdout:3661 trunc from 0xec81c3a to 0x254ddd1 2024-09-14T09:06:43.793 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:43.791+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:43.810 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:43.813 INFO:teuthology.orchestra.run.smithi176.stdout:3663 trunc from 0x254ddd1 to 0x3b7d6a6 2024-09-14T09:06:43.825 INFO:teuthology.orchestra.run.smithi176.stdout:3664 trunc from 0x3b7d6a6 to 0x315d089 2024-09-14T09:06:43.837 INFO:teuthology.orchestra.run.smithi176.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T09:06:43.867 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T09:06:43.867 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:06:43.914 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:43.915+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:43.922 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:43.922 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:43.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:43.922 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:43.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.922 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:19152 3.9 e39/34 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head v 39'12, mlcod=39'9)", 2024-09-14T09:06:43.922 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.333629+0000", 2024-09-14T09:06:43.923 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.589952140999998, 2024-09-14T09:06:43.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.1194095, 2024-09-14T09:06:43.923 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.923 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.923 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:43.923 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.333629+0000", 2024-09-14T09:06:43.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.333629+0000", 2024-09-14T09:06:43.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.333631+0000", 2024-09-14T09:06:43.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.007e-06 2024-09-14T09:06:43.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.350796+0000", 2024-09-14T09:06:43.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017164690999999999 2024-09-14T09:06:43.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.350865+0000", 2024-09-14T09:06:43.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8989e-05 2024-09-14T09:06:43.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.350884+0000", 2024-09-14T09:06:43.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9323000000000001e-05 2024-09-14T09:06:43.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.351045+0000", 2024-09-14T09:06:43.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016102900000000001 2024-09-14T09:06:43.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.351087+0000", 2024-09-14T09:06:43.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1430000000000001e-05 2024-09-14T09:06:43.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:43.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.452865+0000", 2024-09-14T09:06:43.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10177839900000001 2024-09-14T09:06:43.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.453039+0000", 2024-09-14T09:06:43.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017363200000000001 2024-09-14T09:06:43.927 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.927 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.927 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.927 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:37543 3.1b 3:d8e0f8d4:::rbd_data.115eb07a8abd.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:43.927 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.535280+0000", 2024-09-14T09:06:43.927 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.388301132999999, 2024-09-14T09:06:43.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.12853271799999999, 2024-09-14T09:06:43.928 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.928 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.928 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:43.928 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.928 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.928 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.928 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37543 2024-09-14T09:06:43.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.928 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535280+0000", 2024-09-14T09:06:43.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535280+0000", 2024-09-14T09:06:43.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535284+0000", 2024-09-14T09:06:43.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6770000000000001e-06 2024-09-14T09:06:43.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535303+0000", 2024-09-14T09:06:43.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9553000000000001e-05 2024-09-14T09:06:43.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535349+0000", 2024-09-14T09:06:43.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5833e-05 2024-09-14T09:06:43.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535368+0000", 2024-09-14T09:06:43.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8542999999999998e-05 2024-09-14T09:06:43.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535548+0000", 2024-09-14T09:06:43.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000180688 2024-09-14T09:06:43.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535814+0000", 2024-09-14T09:06:43.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026614100000000001 2024-09-14T09:06:43.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:43.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.543779+0000", 2024-09-14T09:06:43.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0079644720000000002 2024-09-14T09:06:43.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:43.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663456+0000", 2024-09-14T09:06:43.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.119676694 2024-09-14T09:06:43.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:43.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663626+0000", 2024-09-14T09:06:43.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170559 2024-09-14T09:06:43.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:43.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663728+0000", 2024-09-14T09:06:43.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101889 2024-09-14T09:06:43.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663813+0000", 2024-09-14T09:06:43.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4668999999999999e-05 2024-09-14T09:06:43.933 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.933 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.933 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.933 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:37543 3.1b e48/34 ondisk, result = 0)", 2024-09-14T09:06:43.933 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.566755+0000", 2024-09-14T09:06:43.934 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.356826595000001, 2024-09-14T09:06:43.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.096718023, 2024-09-14T09:06:43.934 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.934 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.934 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:43.934 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566755+0000", 2024-09-14T09:06:43.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566755+0000", 2024-09-14T09:06:43.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566757+0000", 2024-09-14T09:06:43.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9800000000000001e-06 2024-09-14T09:06:43.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566767+0000", 2024-09-14T09:06:43.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0036e-05 2024-09-14T09:06:43.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566797+0000", 2024-09-14T09:06:43.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.012e-05 2024-09-14T09:06:43.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566813+0000", 2024-09-14T09:06:43.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6575999999999999e-05 2024-09-14T09:06:43.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663408+0000", 2024-09-14T09:06:43.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.096594901999999996 2024-09-14T09:06:43.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663426+0000", 2024-09-14T09:06:43.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7765999999999999e-05 2024-09-14T09:06:43.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663473+0000", 2024-09-14T09:06:43.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6643e-05 2024-09-14T09:06:43.937 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.937 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.937 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.938 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38593 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [watch reconnect cookie 139718844451680 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:06:43.938 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:06:43.938 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 42.105163726000001, 2024-09-14T09:06:43.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0218296630000001, 2024-09-14T09:06:43.938 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.938 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.938 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:43.938 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.938 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.938 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.938 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38593 2024-09-14T09:06:43.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.938 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:06:43.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:06:43.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818420+0000", 2024-09-14T09:06:43.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8399999999999999e-06 2024-09-14T09:06:43.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818431+0000", 2024-09-14T09:06:43.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0859999999999999e-05 2024-09-14T09:06:43.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818466+0000", 2024-09-14T09:06:43.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4486999999999997e-05 2024-09-14T09:06:43.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818481+0000", 2024-09-14T09:06:43.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5362999999999998e-05 2024-09-14T09:06:43.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818659+0000", 2024-09-14T09:06:43.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017807500000000001 2024-09-14T09:06:43.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op must wait for map", 2024-09-14T09:06:43.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818681+0000", 2024-09-14T09:06:43.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2096999999999999e-05 2024-09-14T09:06:43.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825472+0000", 2024-09-14T09:06:43.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067905170000000003 2024-09-14T09:06:43.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:06:43.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825491+0000", 2024-09-14T09:06:43.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9372999999999999e-05 2024-09-14T09:06:43.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835402+0000", 2024-09-14T09:06:43.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0099111620000001 2024-09-14T09:06:43.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835548+0000", 2024-09-14T09:06:43.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014607300000000001 2024-09-14T09:06:43.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.840247+0000", 2024-09-14T09:06:43.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046988159999999998 2024-09-14T09:06:43.943 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.943 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.943 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.944 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38615 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:06:43.944 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:06:43.944 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 42.056798518000001, 2024-09-14T09:06:43.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.97014128899999996, 2024-09-14T09:06:43.944 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.944 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.944 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:43.945 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.945 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.945 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.945 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38615 2024-09-14T09:06:43.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.945 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:06:43.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:06:43.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866787+0000", 2024-09-14T09:06:43.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.476e-06 2024-09-14T09:06:43.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866806+0000", 2024-09-14T09:06:43.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8606000000000001e-05 2024-09-14T09:06:43.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866839+0000", 2024-09-14T09:06:43.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3510000000000003e-05 2024-09-14T09:06:43.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866867+0000", 2024-09-14T09:06:43.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7610000000000002e-05 2024-09-14T09:06:43.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867031+0000", 2024-09-14T09:06:43.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016414199999999999 2024-09-14T09:06:43.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:06:43.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867054+0000", 2024-09-14T09:06:43.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2693000000000001e-05 2024-09-14T09:06:43.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836462+0000", 2024-09-14T09:06:43.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.96940797700000003 2024-09-14T09:06:43.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836547+0000", 2024-09-14T09:06:43.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5294e-05 2024-09-14T09:06:43.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836924+0000", 2024-09-14T09:06:43.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037698099999999997 2024-09-14T09:06:43.949 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.949 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.949 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.950 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49230 3.d 3:b4a67b64:::rbd_data.115e51d92ed2.0000000000000003:12 [read 2621440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:43.950 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.523859+0000", 2024-09-14T09:06:43.950 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.399721927000002, 2024-09-14T09:06:43.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067232675000000006, 2024-09-14T09:06:43.950 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.950 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.950 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:43.950 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.950 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.950 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.950 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49230 2024-09-14T09:06:43.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.950 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523859+0000", 2024-09-14T09:06:43.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523859+0000", 2024-09-14T09:06:43.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523860+0000", 2024-09-14T09:06:43.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0130000000000001e-06 2024-09-14T09:06:43.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523865+0000", 2024-09-14T09:06:43.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4000000000000002e-06 2024-09-14T09:06:43.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523880+0000", 2024-09-14T09:06:43.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5287000000000001e-05 2024-09-14T09:06:43.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523892+0000", 2024-09-14T09:06:43.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2006e-05 2024-09-14T09:06:43.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.590817+0000", 2024-09-14T09:06:43.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066924692999999993 2024-09-14T09:06:43.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:43.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.590937+0000", 2024-09-14T09:06:43.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012081700000000001 2024-09-14T09:06:43.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591092+0000", 2024-09-14T09:06:43.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015445899999999999 2024-09-14T09:06:43.954 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.954 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.954 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.954 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49231 3.d 3:b4a67b64:::rbd_data.115e51d92ed2.0000000000000003:12 [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:43.954 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.523908+0000", 2024-09-14T09:06:43.954 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.399673468000003, 2024-09-14T09:06:43.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067373731000000006, 2024-09-14T09:06:43.954 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.954 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.954 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:43.955 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.955 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.955 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.955 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49231 2024-09-14T09:06:43.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.955 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523908+0000", 2024-09-14T09:06:43.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523908+0000", 2024-09-14T09:06:43.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523909+0000", 2024-09-14T09:06:43.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9699999999999994e-07 2024-09-14T09:06:43.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523913+0000", 2024-09-14T09:06:43.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6070000000000001e-06 2024-09-14T09:06:43.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523929+0000", 2024-09-14T09:06:43.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5415999999999999e-05 2024-09-14T09:06:43.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523940+0000", 2024-09-14T09:06:43.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1707e-05 2024-09-14T09:06:43.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591167+0000", 2024-09-14T09:06:43.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067226576999999996 2024-09-14T09:06:43.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591281+0000", 2024-09-14T09:06:43.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011442699999999999 2024-09-14T09:06:43.958 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.958 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.958 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.958 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49232 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:43.959 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.523956+0000", 2024-09-14T09:06:43.959 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.399625133999997, 2024-09-14T09:06:43.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067398703000000004, 2024-09-14T09:06:43.959 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.959 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.959 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:43.959 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.959 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.959 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.960 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49232 2024-09-14T09:06:43.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.960 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523956+0000", 2024-09-14T09:06:43.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523956+0000", 2024-09-14T09:06:43.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523957+0000", 2024-09-14T09:06:43.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8299999999999995e-07 2024-09-14T09:06:43.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523961+0000", 2024-09-14T09:06:43.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3329999999999998e-06 2024-09-14T09:06:43.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523977+0000", 2024-09-14T09:06:43.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.577e-05 2024-09-14T09:06:43.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523989+0000", 2024-09-14T09:06:43.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2089999999999999e-05 2024-09-14T09:06:43.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591340+0000", 2024-09-14T09:06:43.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067350278999999999 2024-09-14T09:06:43.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591355+0000", 2024-09-14T09:06:43.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5248e-05 2024-09-14T09:06:43.963 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.963 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.963 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.963 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49233 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:43.963 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.524004+0000", 2024-09-14T09:06:43.964 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.399577225000002, 2024-09-14T09:06:43.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067426573000000004, 2024-09-14T09:06:43.964 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.964 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.964 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:43.964 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.964 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.964 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.964 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49233 2024-09-14T09:06:43.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.964 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524004+0000", 2024-09-14T09:06:43.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524004+0000", 2024-09-14T09:06:43.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524005+0000", 2024-09-14T09:06:43.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3699999999999999e-07 2024-09-14T09:06:43.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524009+0000", 2024-09-14T09:06:43.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3359999999999997e-06 2024-09-14T09:06:43.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524026+0000", 2024-09-14T09:06:43.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6753e-05 2024-09-14T09:06:43.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524040+0000", 2024-09-14T09:06:43.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3694e-05 2024-09-14T09:06:43.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591411+0000", 2024-09-14T09:06:43.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067371767999999999 2024-09-14T09:06:43.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591431+0000", 2024-09-14T09:06:43.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9085000000000001e-05 2024-09-14T09:06:43.967 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.968 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.968 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.968 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49257 3.d 3.bb54482d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:43.968 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.524584+0000", 2024-09-14T09:06:43.969 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.398997323000003, 2024-09-14T09:06:43.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067046887999999999, 2024-09-14T09:06:43.969 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.969 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.969 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:43.969 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.970 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.970 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.970 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49257 2024-09-14T09:06:43.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.970 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524584+0000", 2024-09-14T09:06:43.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524584+0000", 2024-09-14T09:06:43.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524585+0000", 2024-09-14T09:06:43.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1699999999999997e-07 2024-09-14T09:06:43.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.974 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524609+0000", 2024-09-14T09:06:43.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4448999999999999e-05 2024-09-14T09:06:43.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524627+0000", 2024-09-14T09:06:43.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7543000000000001e-05 2024-09-14T09:06:43.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524634+0000", 2024-09-14T09:06:43.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7540000000000006e-06 2024-09-14T09:06:43.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591622+0000", 2024-09-14T09:06:43.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066987668 2024-09-14T09:06:43.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591631+0000", 2024-09-14T09:06:43.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7569999999999993e-06 2024-09-14T09:06:43.978 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.978 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.978 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.978 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49310 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:43.978 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525200+0000", 2024-09-14T09:06:43.979 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.398381028000003, 2024-09-14T09:06:43.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067082507, 2024-09-14T09:06:43.979 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.979 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.979 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:43.979 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.980 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.980 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.980 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49310 2024-09-14T09:06:43.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.980 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.981 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525200+0000", 2024-09-14T09:06:43.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525200+0000", 2024-09-14T09:06:43.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525201+0000", 2024-09-14T09:06:43.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.023e-06 2024-09-14T09:06:43.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525206+0000", 2024-09-14T09:06:43.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5000000000000001e-06 2024-09-14T09:06:43.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525242+0000", 2024-09-14T09:06:43.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6672999999999999e-05 2024-09-14T09:06:43.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525256+0000", 2024-09-14T09:06:43.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3360000000000001e-05 2024-09-14T09:06:43.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.986 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592266+0000", 2024-09-14T09:06:43.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067009868 2024-09-14T09:06:43.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592283+0000", 2024-09-14T09:06:43.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7082999999999999e-05 2024-09-14T09:06:43.987 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.987 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.988 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.988 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49311 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:43.988 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525275+0000", 2024-09-14T09:06:43.988 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.398306251999998, 2024-09-14T09:06:43.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067067112999999998, 2024-09-14T09:06:43.989 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.989 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.989 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:43.989 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.989 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.990 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.990 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49311 2024-09-14T09:06:43.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.990 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:43.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:43.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525275+0000", 2024-09-14T09:06:43.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:43.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525275+0000", 2024-09-14T09:06:43.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:43.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:43.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525276+0000", 2024-09-14T09:06:43.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.313e-06 2024-09-14T09:06:43.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:43.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525282+0000", 2024-09-14T09:06:43.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.127e-06 2024-09-14T09:06:43.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:43.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525299+0000", 2024-09-14T09:06:43.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6903000000000001e-05 2024-09-14T09:06:43.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:43.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525312+0000", 2024-09-14T09:06:43.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3193999999999999e-05 2024-09-14T09:06:43.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:43.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592323+0000", 2024-09-14T09:06:43.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067010504999999998 2024-09-14T09:06:43.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:43.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592342+0000", 2024-09-14T09:06:43.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9071000000000001e-05 2024-09-14T09:06:43.997 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.997 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:43.997 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:43.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:43.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:43.998 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49312 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:43.998 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525330+0000", 2024-09-14T09:06:43.998 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.398250859000001, 2024-09-14T09:06:43.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.06706268, 2024-09-14T09:06:43.998 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:43.999 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:43.999 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:43.999 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:43.999 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:43.999 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:43.999 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49312 2024-09-14T09:06:43.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.000 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:44.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:44.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525330+0000", 2024-09-14T09:06:44.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:44.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:44.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525330+0000", 2024-09-14T09:06:44.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:44.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:44.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525332+0000", 2024-09-14T09:06:44.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.187e-06 2024-09-14T09:06:44.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:44.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525350+0000", 2024-09-14T09:06:44.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8833000000000001e-05 2024-09-14T09:06:44.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:44.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525385+0000", 2024-09-14T09:06:44.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4686e-05 2024-09-14T09:06:44.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.005 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525399+0000", 2024-09-14T09:06:44.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4239999999999999e-05 2024-09-14T09:06:44.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:44.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592377+0000", 2024-09-14T09:06:44.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066977255999999999 2024-09-14T09:06:44.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:44.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592393+0000", 2024-09-14T09:06:44.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6478e-05 2024-09-14T09:06:44.007 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:44.007 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:44.007 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:44.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.008 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49313 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:44.008 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525423+0000", 2024-09-14T09:06:44.008 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.398158459999998, 2024-09-14T09:06:44.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067039319, 2024-09-14T09:06:44.009 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:44.009 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:44.009 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:44.009 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:44.009 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:44.010 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.010 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49313 2024-09-14T09:06:44.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.010 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:44.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:44.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525423+0000", 2024-09-14T09:06:44.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:44.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:44.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525423+0000", 2024-09-14T09:06:44.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:44.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:44.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525424+0000", 2024-09-14T09:06:44.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.257e-06 2024-09-14T09:06:44.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:44.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525430+0000", 2024-09-14T09:06:44.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8699999999999997e-06 2024-09-14T09:06:44.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:44.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525447+0000", 2024-09-14T09:06:44.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7560000000000001e-05 2024-09-14T09:06:44.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525460+0000", 2024-09-14T09:06:44.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2243e-05 2024-09-14T09:06:44.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:44.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592444+0000", 2024-09-14T09:06:44.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066984118999999995 2024-09-14T09:06:44.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:44.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592462+0000", 2024-09-14T09:06:44.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.827e-05 2024-09-14T09:06:44.017 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:44.017 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:44.018 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:44.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.018 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49314 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:44.018 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525478+0000", 2024-09-14T09:06:44.019 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.398102696999999, 2024-09-14T09:06:44.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.06702756, 2024-09-14T09:06:44.019 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:44.019 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:44.019 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:44.019 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:44.020 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:44.020 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.020 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49314 2024-09-14T09:06:44.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.020 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:44.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:44.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525478+0000", 2024-09-14T09:06:44.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:44.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:44.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525478+0000", 2024-09-14T09:06:44.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:44.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:44.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525480+0000", 2024-09-14T09:06:44.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.246e-06 2024-09-14T09:06:44.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:44.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525485+0000", 2024-09-14T09:06:44.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7440000000000004e-06 2024-09-14T09:06:44.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:44.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525504+0000", 2024-09-14T09:06:44.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8709999999999999e-05 2024-09-14T09:06:44.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525512+0000", 2024-09-14T09:06:44.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6260000000000003e-06 2024-09-14T09:06:44.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:44.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592491+0000", 2024-09-14T09:06:44.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066979274000000005 2024-09-14T09:06:44.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:44.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592506+0000", 2024-09-14T09:06:44.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4960000000000001e-05 2024-09-14T09:06:44.025 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:44.025 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:44.025 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:44.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.026 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49315 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:44.026 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525523+0000", 2024-09-14T09:06:44.026 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.398057901000001, 2024-09-14T09:06:44.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067036894, 2024-09-14T09:06:44.026 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:44.026 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:44.026 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:44.026 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:44.026 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:44.026 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.026 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49315 2024-09-14T09:06:44.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.026 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:44.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:44.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525523+0000", 2024-09-14T09:06:44.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:44.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:44.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525523+0000", 2024-09-14T09:06:44.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:44.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:44.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525524+0000", 2024-09-14T09:06:44.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2999999999999999e-07 2024-09-14T09:06:44.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:44.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525528+0000", 2024-09-14T09:06:44.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4000000000000001e-06 2024-09-14T09:06:44.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:44.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525537+0000", 2024-09-14T09:06:44.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2739999999999996e-06 2024-09-14T09:06:44.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525543+0000", 2024-09-14T09:06:44.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6329999999999999e-06 2024-09-14T09:06:44.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:44.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592548+0000", 2024-09-14T09:06:44.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067004806 2024-09-14T09:06:44.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:44.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592560+0000", 2024-09-14T09:06:44.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1951e-05 2024-09-14T09:06:44.029 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:44.029 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:44.029 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:44.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.030 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49316 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:44.030 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525553+0000", 2024-09-14T09:06:44.030 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.398028007000001, 2024-09-14T09:06:44.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067056432999999999, 2024-09-14T09:06:44.030 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:44.030 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:44.030 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:44.030 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:44.030 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:44.030 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.030 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49316 2024-09-14T09:06:44.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.031 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:44.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:44.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525553+0000", 2024-09-14T09:06:44.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:44.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:44.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525553+0000", 2024-09-14T09:06:44.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:44.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:44.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525554+0000", 2024-09-14T09:06:44.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0299999999999998e-07 2024-09-14T09:06:44.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:44.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525557+0000", 2024-09-14T09:06:44.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9129999999999999e-06 2024-09-14T09:06:44.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:44.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525566+0000", 2024-09-14T09:06:44.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8000000000000004e-06 2024-09-14T09:06:44.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525572+0000", 2024-09-14T09:06:44.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.72e-06 2024-09-14T09:06:44.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:44.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592598+0000", 2024-09-14T09:06:44.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067025682000000003 2024-09-14T09:06:44.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:44.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592610+0000", 2024-09-14T09:06:44.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1515000000000001e-05 2024-09-14T09:06:44.033 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:44.034 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:44.034 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:44.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.034 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49317 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:44.034 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:44.034 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.397999548000001, 2024-09-14T09:06:44.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067082547000000006, 2024-09-14T09:06:44.034 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:44.034 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:44.034 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:44.034 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:44.034 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:44.034 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.035 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49317 2024-09-14T09:06:44.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.035 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:44.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:44.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:44.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:44.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:44.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:44.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:44.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:44.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:44.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:06:44.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:44.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525585+0000", 2024-09-14T09:06:44.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8700000000000001e-06 2024-09-14T09:06:44.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:44.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525594+0000", 2024-09-14T09:06:44.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9740000000000008e-06 2024-09-14T09:06:44.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525601+0000", 2024-09-14T09:06:44.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9489999999999999e-06 2024-09-14T09:06:44.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:44.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592652+0000", 2024-09-14T09:06:44.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067050803000000006 2024-09-14T09:06:44.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:44.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592664+0000", 2024-09-14T09:06:44.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2198000000000001e-05 2024-09-14T09:06:44.038 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:44.038 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:44.038 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:44.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.038 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49318 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:44.038 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525642+0000", 2024-09-14T09:06:44.038 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.397939047999998, 2024-09-14T09:06:44.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067064055999999997, 2024-09-14T09:06:44.038 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:44.038 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:44.038 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:44.038 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:44.038 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:44.038 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.039 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49318 2024-09-14T09:06:44.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.039 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:44.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:44.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525642+0000", 2024-09-14T09:06:44.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:44.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:44.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525642+0000", 2024-09-14T09:06:44.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:44.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:44.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525644+0000", 2024-09-14T09:06:44.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0159999999999998e-06 2024-09-14T09:06:44.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:44.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525649+0000", 2024-09-14T09:06:44.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2739999999999998e-06 2024-09-14T09:06:44.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:44.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525666+0000", 2024-09-14T09:06:44.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6990000000000002e-05 2024-09-14T09:06:44.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525680+0000", 2024-09-14T09:06:44.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.331e-05 2024-09-14T09:06:44.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:44.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592697+0000", 2024-09-14T09:06:44.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067017367999999994 2024-09-14T09:06:44.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:44.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592706+0000", 2024-09-14T09:06:44.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.098e-06 2024-09-14T09:06:44.041 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:44.042 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:44.042 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:44.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.042 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49319 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:44.042 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525704+0000", 2024-09-14T09:06:44.042 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.397877641999997, 2024-09-14T09:06:44.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067049867999999999, 2024-09-14T09:06:44.042 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:44.042 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:44.042 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:44.042 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:44.042 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:44.042 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.043 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49319 2024-09-14T09:06:44.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.043 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:44.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:44.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525704+0000", 2024-09-14T09:06:44.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:44.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:44.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525704+0000", 2024-09-14T09:06:44.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:44.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:44.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525705+0000", 2024-09-14T09:06:44.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.113e-06 2024-09-14T09:06:44.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:44.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525710+0000", 2024-09-14T09:06:44.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.843e-06 2024-09-14T09:06:44.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:44.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525726+0000", 2024-09-14T09:06:44.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.679e-05 2024-09-14T09:06:44.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525739+0000", 2024-09-14T09:06:44.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2962999999999999e-05 2024-09-14T09:06:44.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:44.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592745+0000", 2024-09-14T09:06:44.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067005341999999996 2024-09-14T09:06:44.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:44.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:44.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:44.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592753+0000", 2024-09-14T09:06:44.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8170000000000005e-06 2024-09-14T09:06:44.046 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:44.046 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:44.046 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:44.046 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:44.046 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:44.046 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:44.048 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:44.051+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:44.048 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:06:44.148 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:44.151+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:44.162 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:44.163 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:44.163 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:44.163 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:44.177 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:06:44.226 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:06:44.227 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:06:44.246 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2024-09-14T09:06:43.983361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1036797639,"num_objects":1200,"num_object_clones":367,"num_object_copies":2400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1200,"num_whiteouts":440,"num_read":102637,"num_read_kb":539394,"num_write":12660,"num_write_kb":221220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":23018534,"num_keys_recovered":66,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9946,"ondisk_log_size":9946,"up":82,"acting":77,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":338732,"kb_used_data":42036,"kb_used_omap":12,"kb_used_meta":296499,"kb_avail":754635988,"statfs":{"total":773094113280,"available":772747251712,"internally_reserved":0,"allocated":43044864,"data_stored":307259437,"data_compressed":9045170,"data_compressed_allocated":25071616,"data_compressed_original":297113030,"omap_allocated":12751,"internal_metadata":303615537},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19947990,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":687,"num_read_kb":1287,"num_write":338,"num_write_kb":1841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":10481218,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.649309"},"pg_stats":[{"pgid":"3.1f","version":"81'1211","reported_seq":4111,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.832160+0000","last_change":"2024-09-14T09:06:41.348384+0000","last_active":"2024-09-14T09:06:41.832160+0000","last_peered":"2024-09-14T09:06:41.832160+0000","last_clean":"2024-09-14T09:06:41.832160+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:06:41.832160+0000","last_undegraded":"2024-09-14T09:06:41.832160+0000","last_fullsized":"2024-09-14T09:06:41.832160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1211,"log_dups_size":0,"ondisk_log_size":1211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:40:01.002873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001387512,"stat_sum":{"num_bytes":17479846,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":15,"num_read":8652,"num_read_kb":13323,"num_write":2090,"num_write_kb":4784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"79'195","reported_seq":2404,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344064+0000","last_change":"2024-09-14T09:05:29.039503+0000","last_active":"2024-09-14T09:06:40.344064+0000","last_peered":"2024-09-14T09:06:40.344064+0000","last_clean":"2024-09-14T09:06:40.344064+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:06:40.344064+0000","last_undegraded":"2024-09-14T09:06:40.344064+0000","last_fullsized":"2024-09-14T09:06:40.344064+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:25:36.908116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001657176,"stat_sum":{"num_bytes":31504440,"num_objects":51,"num_object_clones":18,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":20,"num_read":2150,"num_read_kb":22915,"num_write":231,"num_write_kb":8634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"79'360","reported_seq":3728,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.830471+0000","last_change":"2024-09-14T09:05:28.835298+0000","last_active":"2024-09-14T09:06:41.830471+0000","last_peered":"2024-09-14T09:06:41.830471+0000","last_clean":"2024-09-14T09:06:41.830471+0000","last_became_active":"2024-09-14T09:05:26.835372+0000","last_became_peered":"2024-09-14T09:05:26.835372+0000","last_unstale":"2024-09-14T09:06:41.830471+0000","last_undegraded":"2024-09-14T09:06:41.830471+0000","last_fullsized":"2024-09-14T09:06:41.830471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:38.418901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014149329999999999,"stat_sum":{"num_bytes":57821985,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":10,"num_read":3993,"num_read_kb":52947,"num_write":505,"num_write_kb":8809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"79'518","reported_seq":3036,"reported_epoch":81,"state":"remapped+peering","last_fresh":"2024-09-14T09:06:41.345959+0000","last_change":"2024-09-14T09:06:41.831155+0000","last_active":"2024-09-14T09:06:41.345959+0000","last_peered":"2024-09-14T09:06:40.344845+0000","last_clean":"2024-09-14T09:06:40.344845+0000","last_became_active":"2024-09-14T09:05:39.918593+0000","last_became_peered":"2024-09-14T09:05:39.918593+0000","last_unstale":"2024-09-14T09:06:41.345959+0000","last_undegraded":"2024-09-14T09:06:41.345959+0000","last_fullsized":"2024-09-14T09:06:41.345959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014661419999999999,"stat_sum":{"num_bytes":38979686,"num_objects":37,"num_object_clones":12,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":5418,"num_read_kb":20602,"num_write":735,"num_write_kb":6091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":17,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"79'205","reported_seq":2345,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.832877+0000","last_change":"2024-09-14T09:06:41.347478+0000","last_active":"2024-09-14T09:06:41.832877+0000","last_peered":"2024-09-14T09:06:41.832877+0000","last_clean":"2024-09-14T09:06:41.832877+0000","last_became_active":"2024-09-14T09:06:41.346397+0000","last_became_peered":"2024-09-14T09:06:41.346397+0000","last_unstale":"2024-09-14T09:06:41.832877+0000","last_undegraded":"2024-09-14T09:06:41.832877+0000","last_fullsized":"2024-09-14T09:06:41.832877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:30:10.950021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060373399999999998,"stat_sum":{"num_bytes":30069643,"num_objects":40,"num_object_clones":16,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":1888,"num_read_kb":5564,"num_write":210,"num_write_kb":11506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":6073202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"79'477","reported_seq":3286,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344261+0000","last_change":"2024-09-14T09:05:39.919093+0000","last_active":"2024-09-14T09:06:40.344261+0000","last_peered":"2024-09-14T09:06:40.344261+0000","last_clean":"2024-09-14T09:06:40.344261+0000","last_became_active":"2024-09-14T09:05:39.905126+0000","last_became_peered":"2024-09-14T09:05:39.905126+0000","last_unstale":"2024-09-14T09:06:40.344261+0000","last_undegraded":"2024-09-14T09:06:40.344261+0000","last_fullsized":"2024-09-14T09:06:40.344261+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:21:58.882897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020845550000000001,"stat_sum":{"num_bytes":33943451,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":4393,"num_read_kb":18572,"num_write":762,"num_write_kb":10051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":92669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"79'206","reported_seq":2074,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343852+0000","last_change":"2024-09-14T09:05:28.822565+0000","last_active":"2024-09-14T09:06:40.343852+0000","last_peered":"2024-09-14T09:06:40.343852+0000","last_clean":"2024-09-14T09:06:40.343852+0000","last_became_active":"2024-09-14T09:05:26.816132+0000","last_became_peered":"2024-09-14T09:05:26.816132+0000","last_unstale":"2024-09-14T09:06:40.343852+0000","last_undegraded":"2024-09-14T09:06:40.343852+0000","last_fullsized":"2024-09-14T09:06:40.343852+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:01:39.934279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092420299999999996,"stat_sum":{"num_bytes":21650738,"num_objects":32,"num_object_clones":11,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":12,"num_read":1798,"num_read_kb":6256,"num_write":243,"num_write_kb":4114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"79'283","reported_seq":3314,"reported_epoch":79,"state":"peering","last_fresh":"2024-09-14T09:06:39.775896+0000","last_change":"2024-09-14T09:06:40.341188+0000","last_active":"2024-09-14T09:06:40.341188+0000","last_peered":"2024-09-14T09:06:39.775896+0000","last_clean":"2024-09-14T09:06:37.214421+0000","last_became_active":"2024-09-14T09:06:39.405393+0000","last_became_peered":"2024-09-14T09:06:39.405393+0000","last_unstale":"2024-09-14T09:06:39.775896+0000","last_undegraded":"2024-09-14T09:06:39.775896+0000","last_fullsized":"2024-09-14T09:06:39.314493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081579899999999997,"stat_sum":{"num_bytes":49898996,"num_objects":46,"num_object_clones":17,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":14,"num_read":3640,"num_read_kb":7929,"num_write":273,"num_write_kb":3948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"79'355","reported_seq":4347,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.832529+0000","last_change":"2024-09-14T09:05:28.839928+0000","last_active":"2024-09-14T09:06:41.832529+0000","last_peered":"2024-09-14T09:06:41.832529+0000","last_clean":"2024-09-14T09:06:41.832529+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:06:41.832529+0000","last_undegraded":"2024-09-14T09:06:41.832529+0000","last_fullsized":"2024-09-14T09:06:41.832529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:36:52.750689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014343089999999999,"stat_sum":{"num_bytes":37014757,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":4961,"num_read_kb":22170,"num_write":438,"num_write_kb":4862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"79'374","reported_seq":1522,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.830446+0000","last_change":"2024-09-14T09:05:28.836370+0000","last_active":"2024-09-14T09:06:41.830446+0000","last_peered":"2024-09-14T09:06:41.830446+0000","last_clean":"2024-09-14T09:06:41.830446+0000","last_became_active":"2024-09-14T09:05:26.836744+0000","last_became_peered":"2024-09-14T09:05:26.836744+0000","last_unstale":"2024-09-14T09:06:41.830446+0000","last_undegraded":"2024-09-14T09:06:41.830446+0000","last_fullsized":"2024-09-14T09:06:41.830446+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:11:06.446635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015277190000000001,"stat_sum":{"num_bytes":15473207,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":9,"num_read":3094,"num_read_kb":7897,"num_write":504,"num_write_kb":3528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"79'506","reported_seq":1630,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343594+0000","last_change":"2024-09-14T09:05:28.851587+0000","last_active":"2024-09-14T09:06:40.343594+0000","last_peered":"2024-09-14T09:06:40.343594+0000","last_clean":"2024-09-14T09:06:40.343594+0000","last_became_active":"2024-09-14T09:05:26.840893+0000","last_became_peered":"2024-09-14T09:05:26.840893+0000","last_unstale":"2024-09-14T09:06:40.343594+0000","last_undegraded":"2024-09-14T09:06:40.343594+0000","last_fullsized":"2024-09-14T09:06:40.343594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:35:19.280259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098594000000000004,"stat_sum":{"num_bytes":8828839,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":3906,"num_read_kb":3852,"num_write":772,"num_write_kb":2737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"79'323","reported_seq":1557,"reported_epoch":82,"state":"remapped+peering","last_fresh":"2024-09-14T09:06:41.833669+0000","last_change":"2024-09-14T09:06:41.833669+0000","last_active":"2024-09-14T09:06:41.833669+0000","last_peered":"2024-09-14T09:06:39.647923+0000","last_clean":"2024-09-14T09:06:39.647923+0000","last_became_active":"2024-09-14T09:05:42.829504+0000","last_became_peered":"2024-09-14T09:05:42.829504+0000","last_unstale":"2024-09-14T09:06:41.833669+0000","last_undegraded":"2024-09-14T09:06:41.833669+0000","last_fullsized":"2024-09-14T09:06:41.833669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014079679999999999,"stat_sum":{"num_bytes":20404479,"num_objects":40,"num_object_clones":13,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":2269,"num_read_kb":3974,"num_write":371,"num_write_kb":897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":53852,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.833601+0000","last_change":"2024-09-14T09:05:19.793674+0000","last_active":"2024-09-14T09:06:41.833601+0000","last_peered":"2024-09-14T09:06:41.833601+0000","last_clean":"2024-09-14T09:06:41.833601+0000","last_became_active":"2024-09-14T09:05:17.284977+0000","last_became_peered":"2024-09-14T09:05:17.284977+0000","last_unstale":"2024-09-14T09:06:41.833601+0000","last_undegraded":"2024-09-14T09:06:41.833601+0000","last_fullsized":"2024-09-14T09:06:41.833601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:30:38.832027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"79'219","reported_seq":3515,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.832069+0000","last_change":"2024-09-14T09:05:28.839614+0000","last_active":"2024-09-14T09:06:41.832069+0000","last_peered":"2024-09-14T09:06:41.832069+0000","last_clean":"2024-09-14T09:06:41.832069+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:06:41.832069+0000","last_undegraded":"2024-09-14T09:06:41.832069+0000","last_fullsized":"2024-09-14T09:06:41.832069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:30:20.294407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014245869999999999,"stat_sum":{"num_bytes":39824462,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":12,"num_read":4241,"num_read_kb":38806,"num_write":236,"num_write_kb":5668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344323+0000","last_change":"2024-09-14T09:05:19.314029+0000","last_active":"2024-09-14T09:06:40.344323+0000","last_peered":"2024-09-14T09:06:40.344323+0000","last_clean":"2024-09-14T09:06:40.344323+0000","last_became_active":"2024-09-14T09:05:17.282890+0000","last_became_peered":"2024-09-14T09:05:17.282890+0000","last_unstale":"2024-09-14T09:06:40.344323+0000","last_undegraded":"2024-09-14T09:06:40.344323+0000","last_fullsized":"2024-09-14T09:06:40.344323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:25.304105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"79'359","reported_seq":2348,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.342863+0000","last_change":"2024-09-14T09:05:29.039154+0000","last_active":"2024-09-14T09:06:40.342863+0000","last_peered":"2024-09-14T09:06:40.342863+0000","last_clean":"2024-09-14T09:06:40.342863+0000","last_became_active":"2024-09-14T09:05:26.829855+0000","last_became_peered":"2024-09-14T09:05:26.829855+0000","last_unstale":"2024-09-14T09:06:40.342863+0000","last_undegraded":"2024-09-14T09:06:40.342863+0000","last_fullsized":"2024-09-14T09:06:40.342863+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:12:56.716058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019213749999999999,"stat_sum":{"num_bytes":37534250,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":3271,"num_read_kb":12689,"num_write":469,"num_write_kb":3865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":136,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344083+0000","last_change":"2024-09-14T09:05:19.794165+0000","last_active":"2024-09-14T09:06:40.344083+0000","last_peered":"2024-09-14T09:06:40.344083+0000","last_clean":"2024-09-14T09:06:40.344083+0000","last_became_active":"2024-09-14T09:05:17.295864+0000","last_became_peered":"2024-09-14T09:05:17.295864+0000","last_unstale":"2024-09-14T09:06:40.344083+0000","last_undegraded":"2024-09-14T09:06:40.344083+0000","last_fullsized":"2024-09-14T09:06:40.344083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:53.901865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011399719999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"79'262","reported_seq":2624,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344256+0000","last_change":"2024-09-14T09:06:35.286509+0000","last_active":"2024-09-14T09:06:40.344256+0000","last_peered":"2024-09-14T09:06:40.344256+0000","last_clean":"2024-09-14T09:06:40.344256+0000","last_became_active":"2024-09-14T09:06:35.282469+0000","last_became_peered":"2024-09-14T09:06:35.282469+0000","last_unstale":"2024-09-14T09:06:40.344256+0000","last_undegraded":"2024-09-14T09:06:40.344256+0000","last_fullsized":"2024-09-14T09:06:40.344256+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:56:24.711903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014087959999999999,"stat_sum":{"num_bytes":19969479,"num_objects":43,"num_object_clones":15,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":18,"num_read":3211,"num_read_kb":14319,"num_write":271,"num_write_kb":5415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":1142719,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":136,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344423+0000","last_change":"2024-09-14T09:05:19.314129+0000","last_active":"2024-09-14T09:06:40.344423+0000","last_peered":"2024-09-14T09:06:40.344423+0000","last_clean":"2024-09-14T09:06:40.344423+0000","last_became_active":"2024-09-14T09:05:17.295196+0000","last_became_peered":"2024-09-14T09:05:17.295196+0000","last_unstale":"2024-09-14T09:06:40.344423+0000","last_undegraded":"2024-09-14T09:06:40.344423+0000","last_fullsized":"2024-09-14T09:06:40.344423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:35.269115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011613800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"71'162","reported_seq":855,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.830615+0000","last_change":"2024-09-14T09:05:28.835976+0000","last_active":"2024-09-14T09:06:41.830615+0000","last_peered":"2024-09-14T09:06:41.830615+0000","last_clean":"2024-09-14T09:06:41.830615+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:06:41.830615+0000","last_undegraded":"2024-09-14T09:06:41.830615+0000","last_fullsized":"2024-09-14T09:06:41.830615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:21:06.765695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017480779999999999,"stat_sum":{"num_bytes":52867963,"num_objects":45,"num_object_clones":16,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":21,"num_read":596,"num_read_kb":9794,"num_write":149,"num_write_kb":9015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"80'343","reported_seq":1831,"reported_epoch":80,"state":"remapped+peering","last_fresh":"2024-09-14T09:06:40.888318+0000","last_change":"2024-09-14T09:06:41.832454+0000","last_active":"2024-09-14T09:06:41.832454+0000","last_peered":"2024-09-14T09:06:40.888318+0000","last_clean":"2024-09-14T09:06:40.888318+0000","last_became_active":"2024-09-14T09:05:39.914913+0000","last_became_peered":"2024-09-14T09:05:39.914913+0000","last_unstale":"2024-09-14T09:06:40.888318+0000","last_undegraded":"2024-09-14T09:06:40.888318+0000","last_fullsized":"2024-09-14T09:06:40.888318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014128,"stat_sum":{"num_bytes":25020722,"num_objects":35,"num_object_clones":12,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":2596,"num_read_kb":15720,"num_write":427,"num_write_kb":8882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":4,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.830325+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:06:41.830325+0000","last_peered":"2024-09-14T09:06:41.830325+0000","last_clean":"2024-09-14T09:06:41.830325+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:06:41.830325+0000","last_undegraded":"2024-09-14T09:06:41.830325+0000","last_fullsized":"2024-09-14T09:06:41.830325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"79'168","reported_seq":902,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344632+0000","last_change":"2024-09-14T09:05:28.822791+0000","last_active":"2024-09-14T09:06:40.344632+0000","last_peered":"2024-09-14T09:06:40.344632+0000","last_clean":"2024-09-14T09:06:40.344632+0000","last_became_active":"2024-09-14T09:05:26.817109+0000","last_became_peered":"2024-09-14T09:05:26.817109+0000","last_unstale":"2024-09-14T09:06:40.344632+0000","last_undegraded":"2024-09-14T09:06:40.344632+0000","last_fullsized":"2024-09-14T09:06:40.344632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:08:41.364341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011728439999999999,"stat_sum":{"num_bytes":14678273,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":791,"num_read_kb":3583,"num_write":179,"num_write_kb":1360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.827234+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:06:41.827234+0000","last_peered":"2024-09-14T09:06:41.827234+0000","last_clean":"2024-09-14T09:06:41.827234+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:06:41.827234+0000","last_undegraded":"2024-09-14T09:06:41.827234+0000","last_fullsized":"2024-09-14T09:06:41.827234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"79'201","reported_seq":1805,"reported_epoch":81,"state":"remapped+peering","last_fresh":"2024-09-14T09:06:41.339775+0000","last_change":"2024-09-14T09:06:41.832680+0000","last_active":"2024-09-14T09:06:41.339775+0000","last_peered":"2024-09-14T09:06:40.335804+0000","last_clean":"2024-09-14T09:06:40.335804+0000","last_became_active":"2024-09-14T09:05:39.905252+0000","last_became_peered":"2024-09-14T09:05:39.905252+0000","last_unstale":"2024-09-14T09:06:41.339775+0000","last_undegraded":"2024-09-14T09:06:41.339775+0000","last_fullsized":"2024-09-14T09:06:41.339775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074815999999999999,"stat_sum":{"num_bytes":28865016,"num_objects":30,"num_object_clones":11,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":13,"num_read":1487,"num_read_kb":15691,"num_write":203,"num_write_kb":9083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2097152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":136,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343102+0000","last_change":"2024-09-14T09:05:19.793999+0000","last_active":"2024-09-14T09:06:40.343102+0000","last_peered":"2024-09-14T09:06:40.343102+0000","last_clean":"2024-09-14T09:06:40.343102+0000","last_became_active":"2024-09-14T09:05:17.284122+0000","last_became_peered":"2024-09-14T09:05:17.284122+0000","last_unstale":"2024-09-14T09:06:40.343102+0000","last_undegraded":"2024-09-14T09:06:40.343102+0000","last_fullsized":"2024-09-14T09:06:40.343102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:12:01.133215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010234269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"68'195","reported_seq":1109,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.342756+0000","last_change":"2024-09-14T09:05:29.038389+0000","last_active":"2024-09-14T09:06:40.342756+0000","last_peered":"2024-09-14T09:06:40.342756+0000","last_clean":"2024-09-14T09:06:40.342756+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:06:40.342756+0000","last_undegraded":"2024-09-14T09:06:40.342756+0000","last_fullsized":"2024-09-14T09:06:40.342756+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:42.547444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001479076,"stat_sum":{"num_bytes":19439154,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":1071,"num_read_kb":4172,"num_write":238,"num_write_kb":2073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.342699+0000","last_change":"2024-09-14T09:06:25.227644+0000","last_active":"2024-09-14T09:06:40.342699+0000","last_peered":"2024-09-14T09:06:40.342699+0000","last_clean":"2024-09-14T09:06:40.342699+0000","last_became_active":"2024-09-14T09:06:25.222344+0000","last_became_peered":"2024-09-14T09:06:25.222344+0000","last_unstale":"2024-09-14T09:06:40.342699+0000","last_undegraded":"2024-09-14T09:06:40.342699+0000","last_fullsized":"2024-09-14T09:06:40.342699+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:23:33.414859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":227,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343412+0000","last_change":"2024-09-14T09:05:13.239808+0000","last_active":"2024-09-14T09:06:40.343412+0000","last_peered":"2024-09-14T09:06:40.343412+0000","last_clean":"2024-09-14T09:06:40.343412+0000","last_became_active":"2024-09-14T09:05:13.239064+0000","last_became_peered":"2024-09-14T09:05:13.239064+0000","last_unstale":"2024-09-14T09:06:40.343412+0000","last_undegraded":"2024-09-14T09:06:40.343412+0000","last_fullsized":"2024-09-14T09:06:40.343412+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:40:11.559161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"79'231","reported_seq":3284,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343364+0000","last_change":"2024-09-14T09:05:29.037923+0000","last_active":"2024-09-14T09:06:40.343364+0000","last_peered":"2024-09-14T09:06:40.343364+0000","last_clean":"2024-09-14T09:06:40.343364+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:06:40.343364+0000","last_undegraded":"2024-09-14T09:06:40.343364+0000","last_fullsized":"2024-09-14T09:06:40.343364+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:30:17.623019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099199800000000001,"stat_sum":{"num_bytes":42354761,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":3488,"num_read_kb":29604,"num_write":249,"num_write_kb":7446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"79'189","reported_seq":1415,"reported_epoch":80,"state":"remapped+peering","last_fresh":"2024-09-14T09:06:40.340320+0000","last_change":"2024-09-14T09:06:41.827712+0000","last_active":"2024-09-14T09:06:41.827712+0000","last_peered":"2024-09-14T09:06:40.340320+0000","last_clean":"2024-09-14T09:06:40.340320+0000","last_became_active":"2024-09-14T09:05:39.924025+0000","last_became_peered":"2024-09-14T09:05:39.924025+0000","last_unstale":"2024-09-14T09:06:40.340320+0000","last_undegraded":"2024-09-14T09:06:40.340320+0000","last_fullsized":"2024-09-14T09:06:40.340320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001266028,"stat_sum":{"num_bytes":27628229,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":1134,"num_read_kb":6881,"num_write":171,"num_write_kb":14037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7],"up_primary":4,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"79'257","reported_seq":3062,"reported_epoch":81,"state":"remapped+peering","last_fresh":"2024-09-14T09:06:41.344987+0000","last_change":"2024-09-14T09:06:41.833516+0000","last_active":"2024-09-14T09:06:41.344987+0000","last_peered":"2024-09-14T09:06:40.352617+0000","last_clean":"2024-09-14T09:06:40.352617+0000","last_became_active":"2024-09-14T09:05:39.918526+0000","last_became_peered":"2024-09-14T09:05:39.918526+0000","last_unstale":"2024-09-14T09:06:41.344987+0000","last_undegraded":"2024-09-14T09:06:41.344987+0000","last_fullsized":"2024-09-14T09:06:41.344987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185731,"stat_sum":{"num_bytes":48761384,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":3293,"num_read_kb":31783,"num_write":289,"num_write_kb":16306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":2159421,"num_keys_recovered":3,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"79'303","reported_seq":7865,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.342949+0000","last_change":"2024-09-14T09:06:02.837388+0000","last_active":"2024-09-14T09:06:40.342949+0000","last_peered":"2024-09-14T09:06:40.342949+0000","last_clean":"2024-09-14T09:06:40.342949+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:06:40.342949+0000","last_undegraded":"2024-09-14T09:06:40.342949+0000","last_fullsized":"2024-09-14T09:06:40.342949+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:55:06.321684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001585295,"stat_sum":{"num_bytes":39029359,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":8749,"num_read_kb":52948,"num_write":301,"num_write_kb":7483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"76'207","reported_seq":1173,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.830762+0000","last_change":"2024-09-14T09:05:28.834647+0000","last_active":"2024-09-14T09:06:41.830762+0000","last_peered":"2024-09-14T09:06:41.830762+0000","last_clean":"2024-09-14T09:06:41.830762+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:06:41.830762+0000","last_undegraded":"2024-09-14T09:06:41.830762+0000","last_fullsized":"2024-09-14T09:06:41.830762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:29:56.253705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244290000000001,"stat_sum":{"num_bytes":26698314,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":1353,"num_read_kb":7206,"num_write":220,"num_write_kb":7151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"79'211","reported_seq":2820,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343788+0000","last_change":"2024-09-14T09:05:28.851561+0000","last_active":"2024-09-14T09:06:40.343788+0000","last_peered":"2024-09-14T09:06:40.343788+0000","last_clean":"2024-09-14T09:06:40.343788+0000","last_became_active":"2024-09-14T09:05:26.843780+0000","last_became_peered":"2024-09-14T09:05:26.843780+0000","last_unstale":"2024-09-14T09:06:40.343788+0000","last_undegraded":"2024-09-14T09:06:40.343788+0000","last_fullsized":"2024-09-14T09:06:40.343788+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:55:09.169516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072987200000000001,"stat_sum":{"num_bytes":28500713,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":2826,"num_read_kb":21160,"num_write":266,"num_write_kb":13640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"79'164","reported_seq":1218,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343718+0000","last_change":"2024-09-14T09:05:40.010535+0000","last_active":"2024-09-14T09:06:40.343718+0000","last_peered":"2024-09-14T09:06:40.343718+0000","last_clean":"2024-09-14T09:06:40.343718+0000","last_became_active":"2024-09-14T09:05:39.921575+0000","last_became_peered":"2024-09-14T09:05:39.921575+0000","last_unstale":"2024-09-14T09:06:40.343718+0000","last_undegraded":"2024-09-14T09:06:40.343718+0000","last_fullsized":"2024-09-14T09:06:40.343718+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:34.630587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057596100000000005,"stat_sum":{"num_bytes":35589399,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":13,"num_read":919,"num_read_kb":9018,"num_write":150,"num_write_kb":4457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":111591,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"79'206","reported_seq":1026,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344064+0000","last_change":"2024-09-14T09:06:02.836145+0000","last_active":"2024-09-14T09:06:40.344064+0000","last_peered":"2024-09-14T09:06:40.344064+0000","last_clean":"2024-09-14T09:06:40.344064+0000","last_became_active":"2024-09-14T09:06:02.834684+0000","last_became_peered":"2024-09-14T09:06:02.834684+0000","last_unstale":"2024-09-14T09:06:40.344064+0000","last_undegraded":"2024-09-14T09:06:40.344064+0000","last_fullsized":"2024-09-14T09:06:40.344064+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:33:28.824221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000909463,"stat_sum":{"num_bytes":29699032,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":14,"num_read":1074,"num_read_kb":3425,"num_write":191,"num_write_kb":7631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":259265,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"79'226","reported_seq":4022,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.832839+0000","last_change":"2024-09-14T09:06:00.166357+0000","last_active":"2024-09-14T09:06:41.832839+0000","last_peered":"2024-09-14T09:06:41.832839+0000","last_clean":"2024-09-14T09:06:41.832839+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:06:41.832839+0000","last_undegraded":"2024-09-14T09:06:41.832839+0000","last_fullsized":"2024-09-14T09:06:41.832839+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:58:28.963004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011381609999999999,"stat_sum":{"num_bytes":85650268,"num_objects":42,"num_object_clones":13,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":17,"num_read":3660,"num_read_kb":42918,"num_write":284,"num_write_kb":15069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"79'313","reported_seq":2525,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343379+0000","last_change":"2024-09-14T09:05:40.038345+0000","last_active":"2024-09-14T09:06:40.343379+0000","last_peered":"2024-09-14T09:06:40.343379+0000","last_clean":"2024-09-14T09:06:40.343379+0000","last_became_active":"2024-09-14T09:05:39.923017+0000","last_became_peered":"2024-09-14T09:05:39.923017+0000","last_unstale":"2024-09-14T09:06:40.343379+0000","last_undegraded":"2024-09-14T09:06:40.343379+0000","last_fullsized":"2024-09-14T09:06:40.343379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:12:14.761360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092949299999999999,"stat_sum":{"num_bytes":3898608,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":3511,"num_read_kb":3930,"num_write":421,"num_write_kb":729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"76'364","reported_seq":2563,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.832024+0000","last_change":"2024-09-14T09:05:28.839774+0000","last_active":"2024-09-14T09:06:41.832024+0000","last_peered":"2024-09-14T09:06:41.832024+0000","last_clean":"2024-09-14T09:06:41.832024+0000","last_became_active":"2024-09-14T09:05:26.826600+0000","last_became_peered":"2024-09-14T09:05:26.826600+0000","last_unstale":"2024-09-14T09:06:41.832024+0000","last_undegraded":"2024-09-14T09:06:41.832024+0000","last_fullsized":"2024-09-14T09:06:41.832024+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:37:44.244140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018357099999999999,"stat_sum":{"num_bytes":43950102,"num_objects":41,"num_object_clones":13,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":3652,"num_read_kb":3385,"num_write":457,"num_write_kb":3258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"79'278","reported_seq":4381,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344780+0000","last_change":"2024-09-14T09:05:39.938958+0000","last_active":"2024-09-14T09:06:40.344780+0000","last_peered":"2024-09-14T09:06:40.344780+0000","last_clean":"2024-09-14T09:06:40.344780+0000","last_became_active":"2024-09-14T09:05:39.921023+0000","last_became_peered":"2024-09-14T09:05:39.921023+0000","last_unstale":"2024-09-14T09:06:40.344780+0000","last_undegraded":"2024-09-14T09:06:40.344780+0000","last_fullsized":"2024-09-14T09:06:40.344780+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:46:00.475180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061978900000000004,"stat_sum":{"num_bytes":23177697,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":5486,"num_read_kb":26306,"num_write":252,"num_write_kb":7061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":120888,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1036207252,"num_objects":1196,"num_object_clones":367,"num_object_copies":2392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1196,"num_whiteouts":440,"num_read":102571,"num_read_kb":539339,"num_write":12557,"num_write_kb":219590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":481,"num_bytes_recovered":23018534,"num_keys_recovered":66,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35364864,"data_stored":302647760,"data_compressed":9032746,"data_compressed_allocated":24997888,"data_compressed_original":295966150,"omap_allocated":0,"internal_metadata":0},"log_size":9871,"ondisk_log_size":9871,"up":64,"acting":59,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12424,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40460,"kb_used_data":5104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331380,"statfs":{"total":96636764160,"available":96595333120,"internally_reserved":0,"allocated":5226496,"data_stored":38232434,"data_compressed":1181978,"data_compressed_allocated":3141632,"data_compressed_original":37081632,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50800,"kb_used_data":5724,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94321040,"statfs":{"total":96636764160,"available":96584744960,"internally_reserved":0,"allocated":5861376,"data_stored":41776063,"data_compressed":1136197,"data_compressed_allocated":3391488,"data_compressed_original":40415084,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509463,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42424,"kb_used_data":2916,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329416,"statfs":{"total":96636764160,"available":96593321984,"internally_reserved":0,"allocated":2985984,"data_stored":19263970,"data_compressed":506678,"data_compressed_allocated":1609728,"data_compressed_original":18588448,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46304,"kb_used_data":6792,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325536,"statfs":{"total":96636764160,"available":96589348864,"internally_reserved":0,"allocated":6955008,"data_stored":53317915,"data_compressed":1517281,"data_compressed_allocated":4202496,"data_compressed_original":51760338,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33520,"kb_used_data":6492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338320,"statfs":{"total":96636764160,"available":96602439680,"internally_reserved":0,"allocated":6647808,"data_stored":48313146,"data_compressed":1220402,"data_compressed_allocated":3915776,"data_compressed_original":46621097,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542166,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2166784,"data_stored":12132788,"data_compressed":364162,"data_compressed_allocated":999424,"data_compressed_original":11568392,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542166,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45468,"kb_used_data":5956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326372,"statfs":{"total":96636764160,"available":96590204928,"internally_reserved":0,"allocated":6098944,"data_stored":47202911,"data_compressed":1328461,"data_compressed_allocated":3547136,"data_compressed_original":45737754,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542167,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46452,"kb_used_data":6936,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325388,"statfs":{"total":96636764160,"available":96589197312,"internally_reserved":0,"allocated":7102464,"data_stored":47020210,"data_compressed":1790011,"data_compressed_allocated":4263936,"data_compressed_original":45340285,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5951488,"data_stored":52298055,"data_compressed":1511069,"data_compressed_allocated":4165632,"data_compressed_original":51186898,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6164480,"data_stored":46596785,"data_compressed":1790011,"data_compressed_allocated":4263936,"data_compressed_original":45340285,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":18828041,"data_compressed":506678,"data_compressed_allocated":1609728,"data_compressed_original":18588448,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5165056,"data_stored":46779505,"data_compressed":1328461,"data_compressed_allocated":3547136,"data_compressed_original":45737754,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1232896,"data_stored":11709382,"data_compressed":364162,"data_compressed_allocated":999424,"data_compressed_original":11568392,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4898816,"data_stored":41340115,"data_compressed":1136197,"data_compressed_allocated":3391488,"data_compressed_original":40415084,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4268032,"data_stored":37796505,"data_compressed":1181978,"data_compressed_allocated":3141632,"data_compressed_original":37081632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5656576,"data_stored":47299372,"data_compressed":1214190,"data_compressed_allocated":3878912,"data_compressed_original":46047657,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:06:44.248 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:06:44.250 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:44.251+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:44.300 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:44.300 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:44.300 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:44.301 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:44.301 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:44.315 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:06:44.351 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:44.355+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:44.436 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:44.437 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:44.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:44.437 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:44.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.437 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20858 3.17 3:ed5052d1:::rbd_data.115e889d4809.000000000000015c:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:44.437 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:06:44.437 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.006351963, 2024-09-14T09:06:44.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053381794000000003, 2024-09-14T09:06:44.437 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.437 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.437 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:44.438 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:44.438 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:44.438 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.438 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20858 2024-09-14T09:06:44.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.438 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:06:44.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:06:44.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429351+0000", 2024-09-14T09:06:44.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1149999999999998e-06 2024-09-14T09:06:44.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429363+0000", 2024-09-14T09:06:44.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1819e-05 2024-09-14T09:06:44.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429389+0000", 2024-09-14T09:06:44.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5605e-05 2024-09-14T09:06:44.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429402+0000", 2024-09-14T09:06:44.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3175e-05 2024-09-14T09:06:44.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429492+0000", 2024-09-14T09:06:44.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.988e-05 2024-09-14T09:06:44.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429597+0000", 2024-09-14T09:06:44.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010564 2024-09-14T09:06:44.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:44.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.433465+0000", 2024-09-14T09:06:44.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0038674249999999999 2024-09-14T09:06:44.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:44.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482483+0000", 2024-09-14T09:06:44.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049017999999999999 2024-09-14T09:06:44.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:44.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482625+0000", 2024-09-14T09:06:44.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000142447 2024-09-14T09:06:44.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:44.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482684+0000", 2024-09-14T09:06:44.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8522e-05 2024-09-14T09:06:44.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482730+0000", 2024-09-14T09:06:44.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6165999999999998e-05 2024-09-14T09:06:44.442 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.443 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.443 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.443 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:20858 3.17 e41/23 ondisk, result = 0)", 2024-09-14T09:06:44.443 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:06:44.443 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.991592519999998, 2024-09-14T09:06:44.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038631402000000002, 2024-09-14T09:06:44.443 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.443 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.443 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:44.443 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:06:44.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:06:44.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444109+0000", 2024-09-14T09:06:44.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.331e-06 2024-09-14T09:06:44.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444114+0000", 2024-09-14T09:06:44.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3709999999999999e-06 2024-09-14T09:06:44.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444130+0000", 2024-09-14T09:06:44.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6079e-05 2024-09-14T09:06:44.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444144+0000", 2024-09-14T09:06:44.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3331e-05 2024-09-14T09:06:44.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482538+0000", 2024-09-14T09:06:44.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038394878 2024-09-14T09:06:44.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482548+0000", 2024-09-14T09:06:44.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0093e-05 2024-09-14T09:06:44.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482739+0000", 2024-09-14T09:06:44.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019031899999999999 2024-09-14T09:06:44.446 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.446 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.446 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.447 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:37800 3.1 3:830c4670:::rbd_data.115eb07a8abd.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:44.447 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:06:44.447 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.405144553, 2024-09-14T09:06:44.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.068040724999999996, 2024-09-14T09:06:44.447 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.447 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.447 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:44.447 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:44.447 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:44.447 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.447 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 37800 2024-09-14T09:06:44.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.447 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:06:44.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:06:44.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030557+0000", 2024-09-14T09:06:44.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.888e-06 2024-09-14T09:06:44.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030566+0000", 2024-09-14T09:06:44.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8540000000000003e-06 2024-09-14T09:06:44.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030585+0000", 2024-09-14T09:06:44.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8878000000000001e-05 2024-09-14T09:06:44.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030593+0000", 2024-09-14T09:06:44.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5870000000000004e-06 2024-09-14T09:06:44.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030667+0000", 2024-09-14T09:06:44.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4207000000000003e-05 2024-09-14T09:06:44.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030761+0000", 2024-09-14T09:06:44.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4223999999999998e-05 2024-09-14T09:06:44.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:06:44.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.034400+0000", 2024-09-14T09:06:44.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0036387759999999998 2024-09-14T09:06:44.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:44.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098457+0000", 2024-09-14T09:06:44.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.064057501000000003 2024-09-14T09:06:44.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:44.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098518+0000", 2024-09-14T09:06:44.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0730999999999998e-05 2024-09-14T09:06:44.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:44.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098560+0000", 2024-09-14T09:06:44.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2267e-05 2024-09-14T09:06:44.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098596+0000", 2024-09-14T09:06:44.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5812000000000002e-05 2024-09-14T09:06:44.452 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.452 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.452 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.452 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:37800 3.1 e48/34 ondisk, result = 0)", 2024-09-14T09:06:44.452 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:06:44.452 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.394152937000001, 2024-09-14T09:06:44.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057056260999999997, 2024-09-14T09:06:44.452 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.452 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.452 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:44.452 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:06:44.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:06:44.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041548+0000", 2024-09-14T09:06:44.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8100000000000002e-07 2024-09-14T09:06:44.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041552+0000", 2024-09-14T09:06:44.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7419999999999999e-06 2024-09-14T09:06:44.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041563+0000", 2024-09-14T09:06:44.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1926000000000001e-05 2024-09-14T09:06:44.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041572+0000", 2024-09-14T09:06:44.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8610000000000004e-06 2024-09-14T09:06:44.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098503+0000", 2024-09-14T09:06:44.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056930827000000003 2024-09-14T09:06:44.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098512+0000", 2024-09-14T09:06:44.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5019999999999992e-06 2024-09-14T09:06:44.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098603+0000", 2024-09-14T09:06:44.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1621999999999998e-05 2024-09-14T09:06:44.456 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.456 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.456 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.456 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38523 3.3 3:c62e0486:::rbd_data.115e9e133b92.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 4194304~28499] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T09:06:44.456 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:06:44.456 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.981725238999999, 2024-09-14T09:06:44.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035654406999999999, 2024-09-14T09:06:44.456 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.456 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.456 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:44.456 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:44.457 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:44.457 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.457 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38523 2024-09-14T09:06:44.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.457 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:06:44.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:06:44.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453977+0000", 2024-09-14T09:06:44.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6299999999999998e-06 2024-09-14T09:06:44.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455684+0000", 2024-09-14T09:06:44.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001706843 2024-09-14T09:06:44.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455710+0000", 2024-09-14T09:06:44.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6302e-05 2024-09-14T09:06:44.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455722+0000", 2024-09-14T09:06:44.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1798e-05 2024-09-14T09:06:44.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455825+0000", 2024-09-14T09:06:44.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102822 2024-09-14T09:06:44.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.456032+0000", 2024-09-14T09:06:44.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020704400000000001 2024-09-14T09:06:44.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:06:44.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.456759+0000", 2024-09-14T09:06:44.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00072735699999999998 2024-09-14T09:06:44.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:44.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.477856+0000", 2024-09-14T09:06:44.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021096011000000001 2024-09-14T09:06:44.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:44.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489526+0000", 2024-09-14T09:06:44.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011670313 2024-09-14T09:06:44.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:44.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489572+0000", 2024-09-14T09:06:44.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6408e-05 2024-09-14T09:06:44.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489629+0000", 2024-09-14T09:06:44.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6879000000000002e-05 2024-09-14T09:06:44.461 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.461 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.461 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.462 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49715 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:44.462 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.524874+0000", 2024-09-14T09:06:44.462 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.910826297, 2024-09-14T09:06:44.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034330422999999999, 2024-09-14T09:06:44.462 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.462 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.462 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:44.462 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:44.462 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:44.462 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.462 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49715 2024-09-14T09:06:44.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.462 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524874+0000", 2024-09-14T09:06:44.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524874+0000", 2024-09-14T09:06:44.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524875+0000", 2024-09-14T09:06:44.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1500000000000003e-07 2024-09-14T09:06:44.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524878+0000", 2024-09-14T09:06:44.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9090000000000002e-06 2024-09-14T09:06:44.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524889+0000", 2024-09-14T09:06:44.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.007e-05 2024-09-14T09:06:44.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524895+0000", 2024-09-14T09:06:44.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.285e-06 2024-09-14T09:06:44.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.558620+0000", 2024-09-14T09:06:44.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.033725009 2024-09-14T09:06:44.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.558681+0000", 2024-09-14T09:06:44.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1051999999999998e-05 2024-09-14T09:06:44.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559204+0000", 2024-09-14T09:06:44.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00052318299999999998 2024-09-14T09:06:44.466 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.466 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.466 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.466 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49716 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:44.466 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525151+0000", 2024-09-14T09:06:44.466 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.910548577999997, 2024-09-14T09:06:44.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034564820000000003, 2024-09-14T09:06:44.466 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.466 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.466 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:44.466 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:44.466 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:44.467 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.467 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49716 2024-09-14T09:06:44.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.467 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525151+0000", 2024-09-14T09:06:44.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525151+0000", 2024-09-14T09:06:44.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525152+0000", 2024-09-14T09:06:44.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5799999999999998e-07 2024-09-14T09:06:44.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525157+0000", 2024-09-14T09:06:44.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6380000000000003e-06 2024-09-14T09:06:44.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525167+0000", 2024-09-14T09:06:44.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0457e-05 2024-09-14T09:06:44.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525174+0000", 2024-09-14T09:06:44.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6220000000000003e-06 2024-09-14T09:06:44.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559237+0000", 2024-09-14T09:06:44.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034063361 2024-09-14T09:06:44.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559308+0000", 2024-09-14T09:06:44.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1124999999999994e-05 2024-09-14T09:06:44.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559716+0000", 2024-09-14T09:06:44.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000407759 2024-09-14T09:06:44.470 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.470 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.470 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.470 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49717 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:44.471 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525390+0000", 2024-09-14T09:06:44.471 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.910309781000002, 2024-09-14T09:06:44.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034552639000000003, 2024-09-14T09:06:44.471 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.471 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.471 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:44.471 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:44.471 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:44.471 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.471 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49717 2024-09-14T09:06:44.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.471 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525390+0000", 2024-09-14T09:06:44.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525390+0000", 2024-09-14T09:06:44.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525391+0000", 2024-09-14T09:06:44.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3099999999999997e-07 2024-09-14T09:06:44.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525395+0000", 2024-09-14T09:06:44.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.267e-06 2024-09-14T09:06:44.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525406+0000", 2024-09-14T09:06:44.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1224e-05 2024-09-14T09:06:44.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525413+0000", 2024-09-14T09:06:44.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5420000000000002e-06 2024-09-14T09:06:44.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559760+0000", 2024-09-14T09:06:44.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034347177999999999 2024-09-14T09:06:44.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559838+0000", 2024-09-14T09:06:44.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7708999999999997e-05 2024-09-14T09:06:44.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559943+0000", 2024-09-14T09:06:44.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010498800000000001 2024-09-14T09:06:44.475 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.475 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.475 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.475 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49718 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:44.475 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525514+0000", 2024-09-14T09:06:44.475 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.910185976000001, 2024-09-14T09:06:44.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034602766, 2024-09-14T09:06:44.476 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.476 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.476 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:44.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:44.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:44.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.476 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49718 2024-09-14T09:06:44.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.476 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525514+0000", 2024-09-14T09:06:44.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525514+0000", 2024-09-14T09:06:44.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525515+0000", 2024-09-14T09:06:44.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-07 2024-09-14T09:06:44.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525518+0000", 2024-09-14T09:06:44.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7220000000000001e-06 2024-09-14T09:06:44.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525528+0000", 2024-09-14T09:06:44.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4639999999999995e-06 2024-09-14T09:06:44.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525534+0000", 2024-09-14T09:06:44.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2330000000000003e-06 2024-09-14T09:06:44.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559974+0000", 2024-09-14T09:06:44.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034439866 2024-09-14T09:06:44.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560041+0000", 2024-09-14T09:06:44.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7343999999999998e-05 2024-09-14T09:06:44.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560117+0000", 2024-09-14T09:06:44.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5415999999999998e-05 2024-09-14T09:06:44.479 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.479 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.480 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.480 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49719 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:44.480 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:06:44.480 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.910066638000004, 2024-09-14T09:06:44.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034680563999999997, 2024-09-14T09:06:44.480 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.480 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.480 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:44.480 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:44.480 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:44.480 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.480 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49719 2024-09-14T09:06:44.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.481 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:06:44.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:06:44.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525634+0000", 2024-09-14T09:06:44.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4000000000000001e-07 2024-09-14T09:06:44.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525638+0000", 2024-09-14T09:06:44.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5899999999999999e-06 2024-09-14T09:06:44.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525647+0000", 2024-09-14T09:06:44.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2029999999999998e-06 2024-09-14T09:06:44.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525653+0000", 2024-09-14T09:06:44.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9850000000000003e-06 2024-09-14T09:06:44.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560150+0000", 2024-09-14T09:06:44.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034496845999999998 2024-09-14T09:06:44.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560221+0000", 2024-09-14T09:06:44.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1400000000000001e-05 2024-09-14T09:06:44.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560314+0000", 2024-09-14T09:06:44.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2800000000000006e-05 2024-09-14T09:06:44.484 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.484 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.484 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.484 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49720 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:44.484 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:06:44.484 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.909815014000003, 2024-09-14T09:06:44.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034919095999999997, 2024-09-14T09:06:44.484 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.485 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.485 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:44.485 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:44.485 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:44.485 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.485 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49720 2024-09-14T09:06:44.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.485 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:06:44.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:06:44.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525886+0000", 2024-09-14T09:06:44.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6000000000000002e-07 2024-09-14T09:06:44.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525890+0000", 2024-09-14T09:06:44.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4519999999999999e-06 2024-09-14T09:06:44.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525899+0000", 2024-09-14T09:06:44.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1090000000000004e-06 2024-09-14T09:06:44.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525906+0000", 2024-09-14T09:06:44.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0219999999999999e-06 2024-09-14T09:06:44.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560344+0000", 2024-09-14T09:06:44.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034437970999999998 2024-09-14T09:06:44.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560422+0000", 2024-09-14T09:06:44.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7857999999999999e-05 2024-09-14T09:06:44.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560804+0000", 2024-09-14T09:06:44.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00038182400000000001 2024-09-14T09:06:44.488 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.488 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.488 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.488 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49721 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:44.489 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:06:44.489 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.909767532000004, 2024-09-14T09:06:44.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035347380999999997, 2024-09-14T09:06:44.489 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.489 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.489 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:44.489 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:44.489 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:44.489 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.489 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49721 2024-09-14T09:06:44.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.489 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:06:44.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:06:44.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525933+0000", 2024-09-14T09:06:44.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T09:06:44.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525936+0000", 2024-09-14T09:06:44.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6759999999999999e-06 2024-09-14T09:06:44.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525943+0000", 2024-09-14T09:06:44.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2459999999999997e-06 2024-09-14T09:06:44.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525948+0000", 2024-09-14T09:06:44.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9429999999999999e-06 2024-09-14T09:06:44.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560845+0000", 2024-09-14T09:06:44.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034897354999999998 2024-09-14T09:06:44.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560925+0000", 2024-09-14T09:06:44.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9804999999999999e-05 2024-09-14T09:06:44.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561280+0000", 2024-09-14T09:06:44.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035480100000000002 2024-09-14T09:06:44.493 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.493 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.493 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.493 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49722 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3112960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:44.493 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:44.493 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.909743886000001, 2024-09-14T09:06:44.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035750491000000002, 2024-09-14T09:06:44.493 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.493 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.493 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:44.494 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:44.494 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:44.494 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.494 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49722 2024-09-14T09:06:44.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.494 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:44.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:44.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:44.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.96e-07 2024-09-14T09:06:44.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525958+0000", 2024-09-14T09:06:44.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.024e-06 2024-09-14T09:06:44.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525964+0000", 2024-09-14T09:06:44.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6699999999999999e-06 2024-09-14T09:06:44.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525969+0000", 2024-09-14T09:06:44.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3939999999999998e-06 2024-09-14T09:06:44.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561319+0000", 2024-09-14T09:06:44.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.03535013 2024-09-14T09:06:44.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561390+0000", 2024-09-14T09:06:44.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0920000000000005e-05 2024-09-14T09:06:44.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561707+0000", 2024-09-14T09:06:44.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031695699999999998 2024-09-14T09:06:44.497 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.497 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.497 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.497 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49723 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3506176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:44.497 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:44.497 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.909724902999997, 2024-09-14T09:06:44.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036113064, 2024-09-14T09:06:44.498 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.498 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.498 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:44.498 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:44.498 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:44.498 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.498 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49723 2024-09-14T09:06:44.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.498 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:44.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:44.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:44.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1699999999999999e-07 2024-09-14T09:06:44.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525977+0000", 2024-09-14T09:06:44.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9250000000000002e-06 2024-09-14T09:06:44.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525983+0000", 2024-09-14T09:06:44.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4620000000000004e-06 2024-09-14T09:06:44.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525987+0000", 2024-09-14T09:06:44.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.279e-06 2024-09-14T09:06:44.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561749+0000", 2024-09-14T09:06:44.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035762222000000003 2024-09-14T09:06:44.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561796+0000", 2024-09-14T09:06:44.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6959999999999998e-05 2024-09-14T09:06:44.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562088+0000", 2024-09-14T09:06:44.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029179899999999998 2024-09-14T09:06:44.501 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.501 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.501 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.501 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49724 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:44.502 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:06:44.502 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.909706747000001, 2024-09-14T09:06:44.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036449388999999999, 2024-09-14T09:06:44.502 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.502 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.502 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:44.502 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:44.502 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:44.502 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.502 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49724 2024-09-14T09:06:44.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.502 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:06:44.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:06:44.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525994+0000", 2024-09-14T09:06:44.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3099999999999998e-07 2024-09-14T09:06:44.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525996+0000", 2024-09-14T09:06:44.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.939e-06 2024-09-14T09:06:44.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526005+0000", 2024-09-14T09:06:44.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8430000000000005e-06 2024-09-14T09:06:44.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526010+0000", 2024-09-14T09:06:44.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9389999999999996e-06 2024-09-14T09:06:44.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562114+0000", 2024-09-14T09:06:44.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036103510999999998 2024-09-14T09:06:44.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562189+0000", 2024-09-14T09:06:44.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5162999999999998e-05 2024-09-14T09:06:44.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562443+0000", 2024-09-14T09:06:44.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000253563 2024-09-14T09:06:44.505 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.505 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.505 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.506 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:64935 3.12 3:4cd6d7cb:::rbd_data.115e21c31fcf.0000000000000049:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:06:44.506 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:06:44.506 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.684341399000001, 2024-09-14T09:06:44.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037353866999999999, 2024-09-14T09:06:44.506 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.506 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.506 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:44.506 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:44.506 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:44.506 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.506 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64935 2024-09-14T09:06:44.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.506 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:06:44.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:06:44.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751362+0000", 2024-09-14T09:06:44.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6899999999999998e-06 2024-09-14T09:06:44.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751378+0000", 2024-09-14T09:06:44.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6147e-05 2024-09-14T09:06:44.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751421+0000", 2024-09-14T09:06:44.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2691999999999997e-05 2024-09-14T09:06:44.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751444+0000", 2024-09-14T09:06:44.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3237000000000001e-05 2024-09-14T09:06:44.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751594+0000", 2024-09-14T09:06:44.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014947800000000001 2024-09-14T09:06:44.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751733+0000", 2024-09-14T09:06:44.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013945200000000001 2024-09-14T09:06:44.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:44.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.753315+0000", 2024-09-14T09:06:44.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0015815969999999999 2024-09-14T09:06:44.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:44.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788546+0000", 2024-09-14T09:06:44.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035231449999999997 2024-09-14T09:06:44.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:44.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788570+0000", 2024-09-14T09:06:44.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3991000000000001e-05 2024-09-14T09:06:44.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:44.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788645+0000", 2024-09-14T09:06:44.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4379000000000001e-05 2024-09-14T09:06:44.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788712+0000", 2024-09-14T09:06:44.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7754000000000002e-05 2024-09-14T09:06:44.511 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.511 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.511 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.512 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:71052 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:06:44.512 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:06:44.512 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.698132147999999, 2024-09-14T09:06:44.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14854635599999999, 2024-09-14T09:06:44.512 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.512 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.512 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:44.512 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:44.512 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:44.512 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.512 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 71052 2024-09-14T09:06:44.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.513 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:06:44.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:06:44.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737570+0000", 2024-09-14T09:06:44.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4770000000000002e-06 2024-09-14T09:06:44.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737584+0000", 2024-09-14T09:06:44.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3457e-05 2024-09-14T09:06:44.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737608+0000", 2024-09-14T09:06:44.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3893999999999999e-05 2024-09-14T09:06:44.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737620+0000", 2024-09-14T09:06:44.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2157000000000001e-05 2024-09-14T09:06:44.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737738+0000", 2024-09-14T09:06:44.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118409 2024-09-14T09:06:44.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737892+0000", 2024-09-14T09:06:44.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154222 2024-09-14T09:06:44.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:06:44.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.749622+0000", 2024-09-14T09:06:44.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011729537 2024-09-14T09:06:44.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:44.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885934+0000", 2024-09-14T09:06:44.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13631222200000001 2024-09-14T09:06:44.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:44.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885996+0000", 2024-09-14T09:06:44.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1555000000000003e-05 2024-09-14T09:06:44.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:44.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886060+0000", 2024-09-14T09:06:44.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3980999999999996e-05 2024-09-14T09:06:44.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886114+0000", 2024-09-14T09:06:44.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4444999999999998e-05 2024-09-14T09:06:44.517 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.517 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.517 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.517 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:71052 3.1 e71/34 ondisk, result = 0)", 2024-09-14T09:06:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:06:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.655228770000001, 2024-09-14T09:06:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10565334799999999, 2024-09-14T09:06:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:06:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:06:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780473+0000", 2024-09-14T09:06:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8980000000000001e-06 2024-09-14T09:06:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780482+0000", 2024-09-14T09:06:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9080000000000001e-06 2024-09-14T09:06:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780511+0000", 2024-09-14T09:06:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9490999999999999e-05 2024-09-14T09:06:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780527+0000", 2024-09-14T09:06:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.508e-05 2024-09-14T09:06:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885978+0000", 2024-09-14T09:06:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.105451328 2024-09-14T09:06:44.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885989+0000", 2024-09-14T09:06:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0875e-05 2024-09-14T09:06:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886125+0000", 2024-09-14T09:06:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013576800000000001 2024-09-14T09:06:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73871 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:1b [read 8339456~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:06:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.0097437619999994, 2024-09-14T09:06:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034605047, 2024-09-14T09:06:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73871 2024-09-14T09:06:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:06:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:06:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425957+0000", 2024-09-14T09:06:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T09:06:44.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425961+0000", 2024-09-14T09:06:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8279999999999996e-06 2024-09-14T09:06:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425985+0000", 2024-09-14T09:06:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4182e-05 2024-09-14T09:06:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425997+0000", 2024-09-14T09:06:44.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.132e-05 2024-09-14T09:06:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460070+0000", 2024-09-14T09:06:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034072913000000003 2024-09-14T09:06:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460298+0000", 2024-09-14T09:06:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022819 2024-09-14T09:06:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460561+0000", 2024-09-14T09:06:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026341400000000001 2024-09-14T09:06:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:74276 3.12 3:49b683dd:::rbd_data.115ea8d131ff.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1376256~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:06:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.7505966720000004, 2024-09-14T09:06:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036482939999999998, 2024-09-14T09:06:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:44.526 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74276 2024-09-14T09:06:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:06:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:06:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:44.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685105+0000", 2024-09-14T09:06:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9520000000000001e-06 2024-09-14T09:06:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686530+0000", 2024-09-14T09:06:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001424629 2024-09-14T09:06:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686550+0000", 2024-09-14T09:06:44.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0472e-05 2024-09-14T09:06:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686561+0000", 2024-09-14T09:06:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T09:06:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686671+0000", 2024-09-14T09:06:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110621 2024-09-14T09:06:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:44.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686919+0000", 2024-09-14T09:06:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024790199999999999 2024-09-14T09:06:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.687375+0000", 2024-09-14T09:06:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00045572100000000002 2024-09-14T09:06:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.710871+0000", 2024-09-14T09:06:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023496481999999999 2024-09-14T09:06:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721478+0000", 2024-09-14T09:06:44.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01060709 2024-09-14T09:06:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721534+0000", 2024-09-14T09:06:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5912999999999998e-05 2024-09-14T09:06:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721586+0000", 2024-09-14T09:06:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1792000000000002e-05 2024-09-14T09:06:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.531 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:44.532 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:44.532 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:44.533 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:06:44.534 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:44.535+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:44.553 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:06:44.554 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:06:44.573 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2024-09-14T09:06:43.983361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1036797639,"num_objects":1200,"num_object_clones":367,"num_object_copies":2400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1200,"num_whiteouts":440,"num_read":102637,"num_read_kb":539394,"num_write":12660,"num_write_kb":221220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":23018534,"num_keys_recovered":66,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9946,"ondisk_log_size":9946,"up":82,"acting":77,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":338732,"kb_used_data":42036,"kb_used_omap":12,"kb_used_meta":296499,"kb_avail":754635988,"statfs":{"total":773094113280,"available":772747251712,"internally_reserved":0,"allocated":43044864,"data_stored":307259437,"data_compressed":9045170,"data_compressed_allocated":25071616,"data_compressed_original":297113030,"omap_allocated":12751,"internal_metadata":303615537},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19947990,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":687,"num_read_kb":1287,"num_write":338,"num_write_kb":1841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":10481218,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.649309"},"pg_stats":[{"pgid":"3.1f","version":"81'1211","reported_seq":4111,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.832160+0000","last_change":"2024-09-14T09:06:41.348384+0000","last_active":"2024-09-14T09:06:41.832160+0000","last_peered":"2024-09-14T09:06:41.832160+0000","last_clean":"2024-09-14T09:06:41.832160+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:06:41.832160+0000","last_undegraded":"2024-09-14T09:06:41.832160+0000","last_fullsized":"2024-09-14T09:06:41.832160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1211,"log_dups_size":0,"ondisk_log_size":1211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:40:01.002873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001387512,"stat_sum":{"num_bytes":17479846,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":15,"num_read":8652,"num_read_kb":13323,"num_write":2090,"num_write_kb":4784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"79'195","reported_seq":2404,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344064+0000","last_change":"2024-09-14T09:05:29.039503+0000","last_active":"2024-09-14T09:06:40.344064+0000","last_peered":"2024-09-14T09:06:40.344064+0000","last_clean":"2024-09-14T09:06:40.344064+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:06:40.344064+0000","last_undegraded":"2024-09-14T09:06:40.344064+0000","last_fullsized":"2024-09-14T09:06:40.344064+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:25:36.908116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001657176,"stat_sum":{"num_bytes":31504440,"num_objects":51,"num_object_clones":18,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":20,"num_read":2150,"num_read_kb":22915,"num_write":231,"num_write_kb":8634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"79'360","reported_seq":3728,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.830471+0000","last_change":"2024-09-14T09:05:28.835298+0000","last_active":"2024-09-14T09:06:41.830471+0000","last_peered":"2024-09-14T09:06:41.830471+0000","last_clean":"2024-09-14T09:06:41.830471+0000","last_became_active":"2024-09-14T09:05:26.835372+0000","last_became_peered":"2024-09-14T09:05:26.835372+0000","last_unstale":"2024-09-14T09:06:41.830471+0000","last_undegraded":"2024-09-14T09:06:41.830471+0000","last_fullsized":"2024-09-14T09:06:41.830471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:38.418901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014149329999999999,"stat_sum":{"num_bytes":57821985,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":10,"num_read":3993,"num_read_kb":52947,"num_write":505,"num_write_kb":8809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"79'518","reported_seq":3036,"reported_epoch":81,"state":"remapped+peering","last_fresh":"2024-09-14T09:06:41.345959+0000","last_change":"2024-09-14T09:06:41.831155+0000","last_active":"2024-09-14T09:06:41.345959+0000","last_peered":"2024-09-14T09:06:40.344845+0000","last_clean":"2024-09-14T09:06:40.344845+0000","last_became_active":"2024-09-14T09:05:39.918593+0000","last_became_peered":"2024-09-14T09:05:39.918593+0000","last_unstale":"2024-09-14T09:06:41.345959+0000","last_undegraded":"2024-09-14T09:06:41.345959+0000","last_fullsized":"2024-09-14T09:06:41.345959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014661419999999999,"stat_sum":{"num_bytes":38979686,"num_objects":37,"num_object_clones":12,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":5418,"num_read_kb":20602,"num_write":735,"num_write_kb":6091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":17,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"79'205","reported_seq":2345,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.832877+0000","last_change":"2024-09-14T09:06:41.347478+0000","last_active":"2024-09-14T09:06:41.832877+0000","last_peered":"2024-09-14T09:06:41.832877+0000","last_clean":"2024-09-14T09:06:41.832877+0000","last_became_active":"2024-09-14T09:06:41.346397+0000","last_became_peered":"2024-09-14T09:06:41.346397+0000","last_unstale":"2024-09-14T09:06:41.832877+0000","last_undegraded":"2024-09-14T09:06:41.832877+0000","last_fullsized":"2024-09-14T09:06:41.832877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:30:10.950021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060373399999999998,"stat_sum":{"num_bytes":30069643,"num_objects":40,"num_object_clones":16,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":1888,"num_read_kb":5564,"num_write":210,"num_write_kb":11506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":6073202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"79'477","reported_seq":3286,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344261+0000","last_change":"2024-09-14T09:05:39.919093+0000","last_active":"2024-09-14T09:06:40.344261+0000","last_peered":"2024-09-14T09:06:40.344261+0000","last_clean":"2024-09-14T09:06:40.344261+0000","last_became_active":"2024-09-14T09:05:39.905126+0000","last_became_peered":"2024-09-14T09:05:39.905126+0000","last_unstale":"2024-09-14T09:06:40.344261+0000","last_undegraded":"2024-09-14T09:06:40.344261+0000","last_fullsized":"2024-09-14T09:06:40.344261+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:21:58.882897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020845550000000001,"stat_sum":{"num_bytes":33943451,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":4393,"num_read_kb":18572,"num_write":762,"num_write_kb":10051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":92669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"79'206","reported_seq":2074,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343852+0000","last_change":"2024-09-14T09:05:28.822565+0000","last_active":"2024-09-14T09:06:40.343852+0000","last_peered":"2024-09-14T09:06:40.343852+0000","last_clean":"2024-09-14T09:06:40.343852+0000","last_became_active":"2024-09-14T09:05:26.816132+0000","last_became_peered":"2024-09-14T09:05:26.816132+0000","last_unstale":"2024-09-14T09:06:40.343852+0000","last_undegraded":"2024-09-14T09:06:40.343852+0000","last_fullsized":"2024-09-14T09:06:40.343852+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:01:39.934279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092420299999999996,"stat_sum":{"num_bytes":21650738,"num_objects":32,"num_object_clones":11,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":12,"num_read":1798,"num_read_kb":6256,"num_write":243,"num_write_kb":4114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"79'283","reported_seq":3314,"reported_epoch":79,"state":"peering","last_fresh":"2024-09-14T09:06:39.775896+0000","last_change":"2024-09-14T09:06:40.341188+0000","last_active":"2024-09-14T09:06:40.341188+0000","last_peered":"2024-09-14T09:06:39.775896+0000","last_clean":"2024-09-14T09:06:37.214421+0000","last_became_active":"2024-09-14T09:06:39.405393+0000","last_became_peered":"2024-09-14T09:06:39.405393+0000","last_unstale":"2024-09-14T09:06:39.775896+0000","last_undegraded":"2024-09-14T09:06:39.775896+0000","last_fullsized":"2024-09-14T09:06:39.314493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081579899999999997,"stat_sum":{"num_bytes":49898996,"num_objects":46,"num_object_clones":17,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":14,"num_read":3640,"num_read_kb":7929,"num_write":273,"num_write_kb":3948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"79'355","reported_seq":4347,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.832529+0000","last_change":"2024-09-14T09:05:28.839928+0000","last_active":"2024-09-14T09:06:41.832529+0000","last_peered":"2024-09-14T09:06:41.832529+0000","last_clean":"2024-09-14T09:06:41.832529+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:06:41.832529+0000","last_undegraded":"2024-09-14T09:06:41.832529+0000","last_fullsized":"2024-09-14T09:06:41.832529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:36:52.750689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014343089999999999,"stat_sum":{"num_bytes":37014757,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":4961,"num_read_kb":22170,"num_write":438,"num_write_kb":4862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"79'374","reported_seq":1522,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.830446+0000","last_change":"2024-09-14T09:05:28.836370+0000","last_active":"2024-09-14T09:06:41.830446+0000","last_peered":"2024-09-14T09:06:41.830446+0000","last_clean":"2024-09-14T09:06:41.830446+0000","last_became_active":"2024-09-14T09:05:26.836744+0000","last_became_peered":"2024-09-14T09:05:26.836744+0000","last_unstale":"2024-09-14T09:06:41.830446+0000","last_undegraded":"2024-09-14T09:06:41.830446+0000","last_fullsized":"2024-09-14T09:06:41.830446+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:11:06.446635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015277190000000001,"stat_sum":{"num_bytes":15473207,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":9,"num_read":3094,"num_read_kb":7897,"num_write":504,"num_write_kb":3528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"79'506","reported_seq":1630,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343594+0000","last_change":"2024-09-14T09:05:28.851587+0000","last_active":"2024-09-14T09:06:40.343594+0000","last_peered":"2024-09-14T09:06:40.343594+0000","last_clean":"2024-09-14T09:06:40.343594+0000","last_became_active":"2024-09-14T09:05:26.840893+0000","last_became_peered":"2024-09-14T09:05:26.840893+0000","last_unstale":"2024-09-14T09:06:40.343594+0000","last_undegraded":"2024-09-14T09:06:40.343594+0000","last_fullsized":"2024-09-14T09:06:40.343594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:35:19.280259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098594000000000004,"stat_sum":{"num_bytes":8828839,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":3906,"num_read_kb":3852,"num_write":772,"num_write_kb":2737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"79'323","reported_seq":1557,"reported_epoch":82,"state":"remapped+peering","last_fresh":"2024-09-14T09:06:41.833669+0000","last_change":"2024-09-14T09:06:41.833669+0000","last_active":"2024-09-14T09:06:41.833669+0000","last_peered":"2024-09-14T09:06:39.647923+0000","last_clean":"2024-09-14T09:06:39.647923+0000","last_became_active":"2024-09-14T09:05:42.829504+0000","last_became_peered":"2024-09-14T09:05:42.829504+0000","last_unstale":"2024-09-14T09:06:41.833669+0000","last_undegraded":"2024-09-14T09:06:41.833669+0000","last_fullsized":"2024-09-14T09:06:41.833669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014079679999999999,"stat_sum":{"num_bytes":20404479,"num_objects":40,"num_object_clones":13,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":2269,"num_read_kb":3974,"num_write":371,"num_write_kb":897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":53852,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.833601+0000","last_change":"2024-09-14T09:05:19.793674+0000","last_active":"2024-09-14T09:06:41.833601+0000","last_peered":"2024-09-14T09:06:41.833601+0000","last_clean":"2024-09-14T09:06:41.833601+0000","last_became_active":"2024-09-14T09:05:17.284977+0000","last_became_peered":"2024-09-14T09:05:17.284977+0000","last_unstale":"2024-09-14T09:06:41.833601+0000","last_undegraded":"2024-09-14T09:06:41.833601+0000","last_fullsized":"2024-09-14T09:06:41.833601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:30:38.832027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"79'219","reported_seq":3515,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.832069+0000","last_change":"2024-09-14T09:05:28.839614+0000","last_active":"2024-09-14T09:06:41.832069+0000","last_peered":"2024-09-14T09:06:41.832069+0000","last_clean":"2024-09-14T09:06:41.832069+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:06:41.832069+0000","last_undegraded":"2024-09-14T09:06:41.832069+0000","last_fullsized":"2024-09-14T09:06:41.832069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:30:20.294407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014245869999999999,"stat_sum":{"num_bytes":39824462,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":12,"num_read":4241,"num_read_kb":38806,"num_write":236,"num_write_kb":5668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344323+0000","last_change":"2024-09-14T09:05:19.314029+0000","last_active":"2024-09-14T09:06:40.344323+0000","last_peered":"2024-09-14T09:06:40.344323+0000","last_clean":"2024-09-14T09:06:40.344323+0000","last_became_active":"2024-09-14T09:05:17.282890+0000","last_became_peered":"2024-09-14T09:05:17.282890+0000","last_unstale":"2024-09-14T09:06:40.344323+0000","last_undegraded":"2024-09-14T09:06:40.344323+0000","last_fullsized":"2024-09-14T09:06:40.344323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:25.304105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"79'359","reported_seq":2348,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.342863+0000","last_change":"2024-09-14T09:05:29.039154+0000","last_active":"2024-09-14T09:06:40.342863+0000","last_peered":"2024-09-14T09:06:40.342863+0000","last_clean":"2024-09-14T09:06:40.342863+0000","last_became_active":"2024-09-14T09:05:26.829855+0000","last_became_peered":"2024-09-14T09:05:26.829855+0000","last_unstale":"2024-09-14T09:06:40.342863+0000","last_undegraded":"2024-09-14T09:06:40.342863+0000","last_fullsized":"2024-09-14T09:06:40.342863+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:12:56.716058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019213749999999999,"stat_sum":{"num_bytes":37534250,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":3271,"num_read_kb":12689,"num_write":469,"num_write_kb":3865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":136,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344083+0000","last_change":"2024-09-14T09:05:19.794165+0000","last_active":"2024-09-14T09:06:40.344083+0000","last_peered":"2024-09-14T09:06:40.344083+0000","last_clean":"2024-09-14T09:06:40.344083+0000","last_became_active":"2024-09-14T09:05:17.295864+0000","last_became_peered":"2024-09-14T09:05:17.295864+0000","last_unstale":"2024-09-14T09:06:40.344083+0000","last_undegraded":"2024-09-14T09:06:40.344083+0000","last_fullsized":"2024-09-14T09:06:40.344083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:53.901865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011399719999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"79'262","reported_seq":2624,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344256+0000","last_change":"2024-09-14T09:06:35.286509+0000","last_active":"2024-09-14T09:06:40.344256+0000","last_peered":"2024-09-14T09:06:40.344256+0000","last_clean":"2024-09-14T09:06:40.344256+0000","last_became_active":"2024-09-14T09:06:35.282469+0000","last_became_peered":"2024-09-14T09:06:35.282469+0000","last_unstale":"2024-09-14T09:06:40.344256+0000","last_undegraded":"2024-09-14T09:06:40.344256+0000","last_fullsized":"2024-09-14T09:06:40.344256+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:56:24.711903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014087959999999999,"stat_sum":{"num_bytes":19969479,"num_objects":43,"num_object_clones":15,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":18,"num_read":3211,"num_read_kb":14319,"num_write":271,"num_write_kb":5415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":1142719,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":136,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344423+0000","last_change":"2024-09-14T09:05:19.314129+0000","last_active":"2024-09-14T09:06:40.344423+0000","last_peered":"2024-09-14T09:06:40.344423+0000","last_clean":"2024-09-14T09:06:40.344423+0000","last_became_active":"2024-09-14T09:05:17.295196+0000","last_became_peered":"2024-09-14T09:05:17.295196+0000","last_unstale":"2024-09-14T09:06:40.344423+0000","last_undegraded":"2024-09-14T09:06:40.344423+0000","last_fullsized":"2024-09-14T09:06:40.344423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:35.269115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011613800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"71'162","reported_seq":855,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.830615+0000","last_change":"2024-09-14T09:05:28.835976+0000","last_active":"2024-09-14T09:06:41.830615+0000","last_peered":"2024-09-14T09:06:41.830615+0000","last_clean":"2024-09-14T09:06:41.830615+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:06:41.830615+0000","last_undegraded":"2024-09-14T09:06:41.830615+0000","last_fullsized":"2024-09-14T09:06:41.830615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:21:06.765695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017480779999999999,"stat_sum":{"num_bytes":52867963,"num_objects":45,"num_object_clones":16,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":21,"num_read":596,"num_read_kb":9794,"num_write":149,"num_write_kb":9015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"80'343","reported_seq":1831,"reported_epoch":80,"state":"remapped+peering","last_fresh":"2024-09-14T09:06:40.888318+0000","last_change":"2024-09-14T09:06:41.832454+0000","last_active":"2024-09-14T09:06:41.832454+0000","last_peered":"2024-09-14T09:06:40.888318+0000","last_clean":"2024-09-14T09:06:40.888318+0000","last_became_active":"2024-09-14T09:05:39.914913+0000","last_became_peered":"2024-09-14T09:05:39.914913+0000","last_unstale":"2024-09-14T09:06:40.888318+0000","last_undegraded":"2024-09-14T09:06:40.888318+0000","last_fullsized":"2024-09-14T09:06:40.888318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014128,"stat_sum":{"num_bytes":25020722,"num_objects":35,"num_object_clones":12,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":2596,"num_read_kb":15720,"num_write":427,"num_write_kb":8882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":4,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.830325+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:06:41.830325+0000","last_peered":"2024-09-14T09:06:41.830325+0000","last_clean":"2024-09-14T09:06:41.830325+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:06:41.830325+0000","last_undegraded":"2024-09-14T09:06:41.830325+0000","last_fullsized":"2024-09-14T09:06:41.830325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"79'168","reported_seq":902,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344632+0000","last_change":"2024-09-14T09:05:28.822791+0000","last_active":"2024-09-14T09:06:40.344632+0000","last_peered":"2024-09-14T09:06:40.344632+0000","last_clean":"2024-09-14T09:06:40.344632+0000","last_became_active":"2024-09-14T09:05:26.817109+0000","last_became_peered":"2024-09-14T09:05:26.817109+0000","last_unstale":"2024-09-14T09:06:40.344632+0000","last_undegraded":"2024-09-14T09:06:40.344632+0000","last_fullsized":"2024-09-14T09:06:40.344632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:08:41.364341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011728439999999999,"stat_sum":{"num_bytes":14678273,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":791,"num_read_kb":3583,"num_write":179,"num_write_kb":1360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.827234+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:06:41.827234+0000","last_peered":"2024-09-14T09:06:41.827234+0000","last_clean":"2024-09-14T09:06:41.827234+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:06:41.827234+0000","last_undegraded":"2024-09-14T09:06:41.827234+0000","last_fullsized":"2024-09-14T09:06:41.827234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"79'201","reported_seq":1805,"reported_epoch":81,"state":"remapped+peering","last_fresh":"2024-09-14T09:06:41.339775+0000","last_change":"2024-09-14T09:06:41.832680+0000","last_active":"2024-09-14T09:06:41.339775+0000","last_peered":"2024-09-14T09:06:40.335804+0000","last_clean":"2024-09-14T09:06:40.335804+0000","last_became_active":"2024-09-14T09:05:39.905252+0000","last_became_peered":"2024-09-14T09:05:39.905252+0000","last_unstale":"2024-09-14T09:06:41.339775+0000","last_undegraded":"2024-09-14T09:06:41.339775+0000","last_fullsized":"2024-09-14T09:06:41.339775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074815999999999999,"stat_sum":{"num_bytes":28865016,"num_objects":30,"num_object_clones":11,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":13,"num_read":1487,"num_read_kb":15691,"num_write":203,"num_write_kb":9083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2097152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":136,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343102+0000","last_change":"2024-09-14T09:05:19.793999+0000","last_active":"2024-09-14T09:06:40.343102+0000","last_peered":"2024-09-14T09:06:40.343102+0000","last_clean":"2024-09-14T09:06:40.343102+0000","last_became_active":"2024-09-14T09:05:17.284122+0000","last_became_peered":"2024-09-14T09:05:17.284122+0000","last_unstale":"2024-09-14T09:06:40.343102+0000","last_undegraded":"2024-09-14T09:06:40.343102+0000","last_fullsized":"2024-09-14T09:06:40.343102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:12:01.133215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010234269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"68'195","reported_seq":1109,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.342756+0000","last_change":"2024-09-14T09:05:29.038389+0000","last_active":"2024-09-14T09:06:40.342756+0000","last_peered":"2024-09-14T09:06:40.342756+0000","last_clean":"2024-09-14T09:06:40.342756+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:06:40.342756+0000","last_undegraded":"2024-09-14T09:06:40.342756+0000","last_fullsized":"2024-09-14T09:06:40.342756+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:42.547444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001479076,"stat_sum":{"num_bytes":19439154,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":1071,"num_read_kb":4172,"num_write":238,"num_write_kb":2073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.342699+0000","last_change":"2024-09-14T09:06:25.227644+0000","last_active":"2024-09-14T09:06:40.342699+0000","last_peered":"2024-09-14T09:06:40.342699+0000","last_clean":"2024-09-14T09:06:40.342699+0000","last_became_active":"2024-09-14T09:06:25.222344+0000","last_became_peered":"2024-09-14T09:06:25.222344+0000","last_unstale":"2024-09-14T09:06:40.342699+0000","last_undegraded":"2024-09-14T09:06:40.342699+0000","last_fullsized":"2024-09-14T09:06:40.342699+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:23:33.414859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":227,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343412+0000","last_change":"2024-09-14T09:05:13.239808+0000","last_active":"2024-09-14T09:06:40.343412+0000","last_peered":"2024-09-14T09:06:40.343412+0000","last_clean":"2024-09-14T09:06:40.343412+0000","last_became_active":"2024-09-14T09:05:13.239064+0000","last_became_peered":"2024-09-14T09:05:13.239064+0000","last_unstale":"2024-09-14T09:06:40.343412+0000","last_undegraded":"2024-09-14T09:06:40.343412+0000","last_fullsized":"2024-09-14T09:06:40.343412+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:40:11.559161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"79'231","reported_seq":3284,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343364+0000","last_change":"2024-09-14T09:05:29.037923+0000","last_active":"2024-09-14T09:06:40.343364+0000","last_peered":"2024-09-14T09:06:40.343364+0000","last_clean":"2024-09-14T09:06:40.343364+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:06:40.343364+0000","last_undegraded":"2024-09-14T09:06:40.343364+0000","last_fullsized":"2024-09-14T09:06:40.343364+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:30:17.623019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099199800000000001,"stat_sum":{"num_bytes":42354761,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":3488,"num_read_kb":29604,"num_write":249,"num_write_kb":7446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"79'189","reported_seq":1415,"reported_epoch":80,"state":"remapped+peering","last_fresh":"2024-09-14T09:06:40.340320+0000","last_change":"2024-09-14T09:06:41.827712+0000","last_active":"2024-09-14T09:06:41.827712+0000","last_peered":"2024-09-14T09:06:40.340320+0000","last_clean":"2024-09-14T09:06:40.340320+0000","last_became_active":"2024-09-14T09:05:39.924025+0000","last_became_peered":"2024-09-14T09:05:39.924025+0000","last_unstale":"2024-09-14T09:06:40.340320+0000","last_undegraded":"2024-09-14T09:06:40.340320+0000","last_fullsized":"2024-09-14T09:06:40.340320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001266028,"stat_sum":{"num_bytes":27628229,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":1134,"num_read_kb":6881,"num_write":171,"num_write_kb":14037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7],"up_primary":4,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"79'257","reported_seq":3062,"reported_epoch":81,"state":"remapped+peering","last_fresh":"2024-09-14T09:06:41.344987+0000","last_change":"2024-09-14T09:06:41.833516+0000","last_active":"2024-09-14T09:06:41.344987+0000","last_peered":"2024-09-14T09:06:40.352617+0000","last_clean":"2024-09-14T09:06:40.352617+0000","last_became_active":"2024-09-14T09:05:39.918526+0000","last_became_peered":"2024-09-14T09:05:39.918526+0000","last_unstale":"2024-09-14T09:06:41.344987+0000","last_undegraded":"2024-09-14T09:06:41.344987+0000","last_fullsized":"2024-09-14T09:06:41.344987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185731,"stat_sum":{"num_bytes":48761384,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":3293,"num_read_kb":31783,"num_write":289,"num_write_kb":16306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":2159421,"num_keys_recovered":3,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"79'303","reported_seq":7865,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.342949+0000","last_change":"2024-09-14T09:06:02.837388+0000","last_active":"2024-09-14T09:06:40.342949+0000","last_peered":"2024-09-14T09:06:40.342949+0000","last_clean":"2024-09-14T09:06:40.342949+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:06:40.342949+0000","last_undegraded":"2024-09-14T09:06:40.342949+0000","last_fullsized":"2024-09-14T09:06:40.342949+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:55:06.321684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001585295,"stat_sum":{"num_bytes":39029359,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":8749,"num_read_kb":52948,"num_write":301,"num_write_kb":7483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"76'207","reported_seq":1173,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.830762+0000","last_change":"2024-09-14T09:05:28.834647+0000","last_active":"2024-09-14T09:06:41.830762+0000","last_peered":"2024-09-14T09:06:41.830762+0000","last_clean":"2024-09-14T09:06:41.830762+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:06:41.830762+0000","last_undegraded":"2024-09-14T09:06:41.830762+0000","last_fullsized":"2024-09-14T09:06:41.830762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:29:56.253705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244290000000001,"stat_sum":{"num_bytes":26698314,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":1353,"num_read_kb":7206,"num_write":220,"num_write_kb":7151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"79'211","reported_seq":2820,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343788+0000","last_change":"2024-09-14T09:05:28.851561+0000","last_active":"2024-09-14T09:06:40.343788+0000","last_peered":"2024-09-14T09:06:40.343788+0000","last_clean":"2024-09-14T09:06:40.343788+0000","last_became_active":"2024-09-14T09:05:26.843780+0000","last_became_peered":"2024-09-14T09:05:26.843780+0000","last_unstale":"2024-09-14T09:06:40.343788+0000","last_undegraded":"2024-09-14T09:06:40.343788+0000","last_fullsized":"2024-09-14T09:06:40.343788+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:55:09.169516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072987200000000001,"stat_sum":{"num_bytes":28500713,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":2826,"num_read_kb":21160,"num_write":266,"num_write_kb":13640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"79'164","reported_seq":1218,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343718+0000","last_change":"2024-09-14T09:05:40.010535+0000","last_active":"2024-09-14T09:06:40.343718+0000","last_peered":"2024-09-14T09:06:40.343718+0000","last_clean":"2024-09-14T09:06:40.343718+0000","last_became_active":"2024-09-14T09:05:39.921575+0000","last_became_peered":"2024-09-14T09:05:39.921575+0000","last_unstale":"2024-09-14T09:06:40.343718+0000","last_undegraded":"2024-09-14T09:06:40.343718+0000","last_fullsized":"2024-09-14T09:06:40.343718+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:34.630587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057596100000000005,"stat_sum":{"num_bytes":35589399,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":13,"num_read":919,"num_read_kb":9018,"num_write":150,"num_write_kb":4457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":111591,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"79'206","reported_seq":1026,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344064+0000","last_change":"2024-09-14T09:06:02.836145+0000","last_active":"2024-09-14T09:06:40.344064+0000","last_peered":"2024-09-14T09:06:40.344064+0000","last_clean":"2024-09-14T09:06:40.344064+0000","last_became_active":"2024-09-14T09:06:02.834684+0000","last_became_peered":"2024-09-14T09:06:02.834684+0000","last_unstale":"2024-09-14T09:06:40.344064+0000","last_undegraded":"2024-09-14T09:06:40.344064+0000","last_fullsized":"2024-09-14T09:06:40.344064+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:33:28.824221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000909463,"stat_sum":{"num_bytes":29699032,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":14,"num_read":1074,"num_read_kb":3425,"num_write":191,"num_write_kb":7631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":259265,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"79'226","reported_seq":4022,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.832839+0000","last_change":"2024-09-14T09:06:00.166357+0000","last_active":"2024-09-14T09:06:41.832839+0000","last_peered":"2024-09-14T09:06:41.832839+0000","last_clean":"2024-09-14T09:06:41.832839+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:06:41.832839+0000","last_undegraded":"2024-09-14T09:06:41.832839+0000","last_fullsized":"2024-09-14T09:06:41.832839+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:58:28.963004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011381609999999999,"stat_sum":{"num_bytes":85650268,"num_objects":42,"num_object_clones":13,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":17,"num_read":3660,"num_read_kb":42918,"num_write":284,"num_write_kb":15069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"79'313","reported_seq":2525,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343379+0000","last_change":"2024-09-14T09:05:40.038345+0000","last_active":"2024-09-14T09:06:40.343379+0000","last_peered":"2024-09-14T09:06:40.343379+0000","last_clean":"2024-09-14T09:06:40.343379+0000","last_became_active":"2024-09-14T09:05:39.923017+0000","last_became_peered":"2024-09-14T09:05:39.923017+0000","last_unstale":"2024-09-14T09:06:40.343379+0000","last_undegraded":"2024-09-14T09:06:40.343379+0000","last_fullsized":"2024-09-14T09:06:40.343379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:12:14.761360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092949299999999999,"stat_sum":{"num_bytes":3898608,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":3511,"num_read_kb":3930,"num_write":421,"num_write_kb":729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"76'364","reported_seq":2563,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.832024+0000","last_change":"2024-09-14T09:05:28.839774+0000","last_active":"2024-09-14T09:06:41.832024+0000","last_peered":"2024-09-14T09:06:41.832024+0000","last_clean":"2024-09-14T09:06:41.832024+0000","last_became_active":"2024-09-14T09:05:26.826600+0000","last_became_peered":"2024-09-14T09:05:26.826600+0000","last_unstale":"2024-09-14T09:06:41.832024+0000","last_undegraded":"2024-09-14T09:06:41.832024+0000","last_fullsized":"2024-09-14T09:06:41.832024+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:37:44.244140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018357099999999999,"stat_sum":{"num_bytes":43950102,"num_objects":41,"num_object_clones":13,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":3652,"num_read_kb":3385,"num_write":457,"num_write_kb":3258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"79'278","reported_seq":4381,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344780+0000","last_change":"2024-09-14T09:05:39.938958+0000","last_active":"2024-09-14T09:06:40.344780+0000","last_peered":"2024-09-14T09:06:40.344780+0000","last_clean":"2024-09-14T09:06:40.344780+0000","last_became_active":"2024-09-14T09:05:39.921023+0000","last_became_peered":"2024-09-14T09:05:39.921023+0000","last_unstale":"2024-09-14T09:06:40.344780+0000","last_undegraded":"2024-09-14T09:06:40.344780+0000","last_fullsized":"2024-09-14T09:06:40.344780+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:46:00.475180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061978900000000004,"stat_sum":{"num_bytes":23177697,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":5486,"num_read_kb":26306,"num_write":252,"num_write_kb":7061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":120888,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1036207252,"num_objects":1196,"num_object_clones":367,"num_object_copies":2392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1196,"num_whiteouts":440,"num_read":102571,"num_read_kb":539339,"num_write":12557,"num_write_kb":219590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":481,"num_bytes_recovered":23018534,"num_keys_recovered":66,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35364864,"data_stored":302647760,"data_compressed":9032746,"data_compressed_allocated":24997888,"data_compressed_original":295966150,"omap_allocated":0,"internal_metadata":0},"log_size":9871,"ondisk_log_size":9871,"up":64,"acting":59,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12424,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40460,"kb_used_data":5104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331380,"statfs":{"total":96636764160,"available":96595333120,"internally_reserved":0,"allocated":5226496,"data_stored":38232434,"data_compressed":1181978,"data_compressed_allocated":3141632,"data_compressed_original":37081632,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50800,"kb_used_data":5724,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94321040,"statfs":{"total":96636764160,"available":96584744960,"internally_reserved":0,"allocated":5861376,"data_stored":41776063,"data_compressed":1136197,"data_compressed_allocated":3391488,"data_compressed_original":40415084,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509463,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42424,"kb_used_data":2916,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329416,"statfs":{"total":96636764160,"available":96593321984,"internally_reserved":0,"allocated":2985984,"data_stored":19263970,"data_compressed":506678,"data_compressed_allocated":1609728,"data_compressed_original":18588448,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46304,"kb_used_data":6792,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325536,"statfs":{"total":96636764160,"available":96589348864,"internally_reserved":0,"allocated":6955008,"data_stored":53317915,"data_compressed":1517281,"data_compressed_allocated":4202496,"data_compressed_original":51760338,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33520,"kb_used_data":6492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338320,"statfs":{"total":96636764160,"available":96602439680,"internally_reserved":0,"allocated":6647808,"data_stored":48313146,"data_compressed":1220402,"data_compressed_allocated":3915776,"data_compressed_original":46621097,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542166,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2166784,"data_stored":12132788,"data_compressed":364162,"data_compressed_allocated":999424,"data_compressed_original":11568392,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542166,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45468,"kb_used_data":5956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326372,"statfs":{"total":96636764160,"available":96590204928,"internally_reserved":0,"allocated":6098944,"data_stored":47202911,"data_compressed":1328461,"data_compressed_allocated":3547136,"data_compressed_original":45737754,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542167,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46452,"kb_used_data":6936,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325388,"statfs":{"total":96636764160,"available":96589197312,"internally_reserved":0,"allocated":7102464,"data_stored":47020210,"data_compressed":1790011,"data_compressed_allocated":4263936,"data_compressed_original":45340285,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5951488,"data_stored":52298055,"data_compressed":1511069,"data_compressed_allocated":4165632,"data_compressed_original":51186898,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6164480,"data_stored":46596785,"data_compressed":1790011,"data_compressed_allocated":4263936,"data_compressed_original":45340285,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":18828041,"data_compressed":506678,"data_compressed_allocated":1609728,"data_compressed_original":18588448,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5165056,"data_stored":46779505,"data_compressed":1328461,"data_compressed_allocated":3547136,"data_compressed_original":45737754,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1232896,"data_stored":11709382,"data_compressed":364162,"data_compressed_allocated":999424,"data_compressed_original":11568392,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4898816,"data_stored":41340115,"data_compressed":1136197,"data_compressed_allocated":3391488,"data_compressed_original":40415084,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4268032,"data_stored":37796505,"data_compressed":1181978,"data_compressed_allocated":3141632,"data_compressed_original":37081632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5656576,"data_stored":47299372,"data_compressed":1214190,"data_compressed_allocated":3878912,"data_compressed_original":46047657,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:06:44.576 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-14T09:06:44.662 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:44.663+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:44.668 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:44.668 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:44.668 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:44.668 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:44.685 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:06:44.735 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:44.735+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:44.836 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:44.835+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:44.851 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:44.851 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:44.852 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:44.852 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:44.852 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:44.869 INFO:teuthology.orchestra.run.smithi176.stdout:3665 writesame 0x59757b3 thru 0x59844ea data_size 0x4(0xed38 bytes) 2024-09-14T09:06:44.872 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:06:44.873 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T09:06:44.876 INFO:teuthology.orchestra.run.smithi176.stdout:3666 trunc from 0x59844eb to 0x1f778d9 2024-09-14T09:06:44.893 INFO:teuthology.orchestra.run.smithi176.stdout:3667 punch from 0xf04729 to 0xf08367, (0x3c3e bytes) 2024-09-14T09:06:44.893 INFO:teuthology.orchestra.run.smithi176.stdout:3668 read 0x1aa660a thru 0x1aadf4c (0x7943 bytes) 2024-09-14T09:06:44.894 INFO:teuthology.orchestra.run.smithi176.stdout:3669 read 0x14ba9b6 thru 0x14c0912 (0x5f5d bytes) 2024-09-14T09:06:44.912 INFO:teuthology.orchestra.run.smithi176.stdout:3671 writesame 0x5b1fa26 thru 0x5b278d3 data_size 0x5e(0x7eae bytes) 2024-09-14T09:06:44.929 INFO:teuthology.orchestra.run.smithi176.stdout:3672 write 0x8716e2e thru 0x871cb39 (0x5d0c bytes) 2024-09-14T09:06:44.937 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:44.935+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:44.938 INFO:teuthology.orchestra.run.smithi176.stdout:3673 writesame 0x59a6318 thru 0x59af23f data_size 0xc(0x8f28 bytes) 2024-09-14T09:06:44.947 INFO:teuthology.orchestra.run.smithi176.stdout:3674 writesame 0xbc8886 thru 0xbd6d97 data_size 0x2(0xe512 bytes) 2024-09-14T09:06:45.007 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:45.008 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:45.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:45.008 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:45.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.008 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73540 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15155200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.008 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:06:45.008 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.738805752999999, 2024-09-14T09:06:45.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.022930347, 2024-09-14T09:06:45.008 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.008 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.008 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.008 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.009 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.009 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.009 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73540 2024-09-14T09:06:45.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.009 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:06:45.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:06:45.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268665+0000", 2024-09-14T09:06:45.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6599999999999996e-07 2024-09-14T09:06:45.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268668+0000", 2024-09-14T09:06:45.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8760000000000001e-06 2024-09-14T09:06:45.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268676+0000", 2024-09-14T09:06:45.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7879999999999995e-06 2024-09-14T09:06:45.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268683+0000", 2024-09-14T09:06:45.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3180000000000003e-06 2024-09-14T09:06:45.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282017+0000", 2024-09-14T09:06:45.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01333409 2024-09-14T09:06:45.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282020+0000", 2024-09-14T09:06:45.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1609999999999999e-06 2024-09-14T09:06:45.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291158+0000", 2024-09-14T09:06:45.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0091376430000001 2024-09-14T09:06:45.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291383+0000", 2024-09-14T09:06:45.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022491000000000001 2024-09-14T09:06:45.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291595+0000", 2024-09-14T09:06:45.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021199500000000001 2024-09-14T09:06:45.013 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.013 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.013 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.013 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73541 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15237120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.013 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:06:45.013 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.738778332999999, 2024-09-14T09:06:45.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0232607330000001, 2024-09-14T09:06:45.013 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.013 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.013 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.013 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.013 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.013 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.014 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73541 2024-09-14T09:06:45.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.014 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:06:45.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:06:45.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:06:45.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5400000000000001e-07 2024-09-14T09:06:45.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268695+0000", 2024-09-14T09:06:45.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5799999999999999e-06 2024-09-14T09:06:45.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268703+0000", 2024-09-14T09:06:45.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5480000000000002e-06 2024-09-14T09:06:45.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268713+0000", 2024-09-14T09:06:45.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8640000000000008e-06 2024-09-14T09:06:45.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282032+0000", 2024-09-14T09:06:45.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013318831999999999 2024-09-14T09:06:45.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282035+0000", 2024-09-14T09:06:45.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4010000000000001e-06 2024-09-14T09:06:45.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291701+0000", 2024-09-14T09:06:45.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0096654199999999 2024-09-14T09:06:45.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291807+0000", 2024-09-14T09:06:45.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105852 2024-09-14T09:06:45.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291952+0000", 2024-09-14T09:06:45.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014568200000000001 2024-09-14T09:06:45.017 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.017 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.017 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.018 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73542 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15319040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.018 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:06:45.018 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.738708722, 2024-09-14T09:06:45.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0237091359999999, 2024-09-14T09:06:45.018 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.018 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.018 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.018 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.018 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.018 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.018 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73542 2024-09-14T09:06:45.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.018 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:06:45.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:06:45.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268762+0000", 2024-09-14T09:06:45.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:06:45.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268766+0000", 2024-09-14T09:06:45.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5880000000000002e-06 2024-09-14T09:06:45.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268776+0000", 2024-09-14T09:06:45.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0168e-05 2024-09-14T09:06:45.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268789+0000", 2024-09-14T09:06:45.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:06:45.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.021 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.021 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282053+0000", 2024-09-14T09:06:45.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013264496000000001 2024-09-14T09:06:45.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.021 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.021 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282057+0000", 2024-09-14T09:06:45.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6090000000000001e-06 2024-09-14T09:06:45.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.021 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.021 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292052+0000", 2024-09-14T09:06:45.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0099950070000001 2024-09-14T09:06:45.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292262+0000", 2024-09-14T09:06:45.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021009800000000001 2024-09-14T09:06:45.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292470+0000", 2024-09-14T09:06:45.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000208314 2024-09-14T09:06:45.022 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.022 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.022 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.023 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73543 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.023 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:06:45.023 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.738669932000001, 2024-09-14T09:06:45.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023985473, 2024-09-14T09:06:45.023 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.023 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.023 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.023 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.023 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.023 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.023 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73543 2024-09-14T09:06:45.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.024 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:06:45.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:06:45.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268801+0000", 2024-09-14T09:06:45.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:06:45.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268803+0000", 2024-09-14T09:06:45.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7439999999999999e-06 2024-09-14T09:06:45.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268812+0000", 2024-09-14T09:06:45.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7080000000000003e-06 2024-09-14T09:06:45.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268819+0000", 2024-09-14T09:06:45.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T09:06:45.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282068+0000", 2024-09-14T09:06:45.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013249540000000001 2024-09-14T09:06:45.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282072+0000", 2024-09-14T09:06:45.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0960000000000001e-06 2024-09-14T09:06:45.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292573+0000", 2024-09-14T09:06:45.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0105014560000001 2024-09-14T09:06:45.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292679+0000", 2024-09-14T09:06:45.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010545899999999999 2024-09-14T09:06:45.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292786+0000", 2024-09-14T09:06:45.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107052 2024-09-14T09:06:45.028 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.028 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.028 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.028 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73544 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.028 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:06:45.028 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.73864223, 2024-09-14T09:06:45.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0244830549999999, 2024-09-14T09:06:45.028 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.028 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.029 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.029 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.029 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.029 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.029 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73544 2024-09-14T09:06:45.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.029 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:06:45.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:06:45.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:06:45.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5599999999999995e-07 2024-09-14T09:06:45.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268831+0000", 2024-09-14T09:06:45.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.57e-06 2024-09-14T09:06:45.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268844+0000", 2024-09-14T09:06:45.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2811e-05 2024-09-14T09:06:45.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268851+0000", 2024-09-14T09:06:45.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9140000000000002e-06 2024-09-14T09:06:45.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282083+0000", 2024-09-14T09:06:45.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013232111 2024-09-14T09:06:45.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282086+0000", 2024-09-14T09:06:45.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2909999999999999e-06 2024-09-14T09:06:45.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292884+0000", 2024-09-14T09:06:45.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0107975069999999 2024-09-14T09:06:45.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293100+0000", 2024-09-14T09:06:45.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021639399999999999 2024-09-14T09:06:45.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293311+0000", 2024-09-14T09:06:45.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021090099999999999 2024-09-14T09:06:45.033 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.033 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.033 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.033 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73545 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15564800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.033 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:06:45.033 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.738591045, 2024-09-14T09:06:45.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0249746740000001, 2024-09-14T09:06:45.034 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.034 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.034 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.034 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.034 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.034 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.034 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73545 2024-09-14T09:06:45.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.034 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.034 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:06:45.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:06:45.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268880+0000", 2024-09-14T09:06:45.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5600000000000005e-07 2024-09-14T09:06:45.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268889+0000", 2024-09-14T09:06:45.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8559999999999996e-06 2024-09-14T09:06:45.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268913+0000", 2024-09-14T09:06:45.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3958e-05 2024-09-14T09:06:45.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268921+0000", 2024-09-14T09:06:45.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7479999999999999e-06 2024-09-14T09:06:45.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282098+0000", 2024-09-14T09:06:45.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013176297999999999 2024-09-14T09:06:45.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282101+0000", 2024-09-14T09:06:45.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2329999999999998e-06 2024-09-14T09:06:45.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293435+0000", 2024-09-14T09:06:45.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0113339459999999 2024-09-14T09:06:45.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293648+0000", 2024-09-14T09:06:45.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021307100000000001 2024-09-14T09:06:45.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293854+0000", 2024-09-14T09:06:45.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020580800000000001 2024-09-14T09:06:45.038 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.038 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.038 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.038 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73546 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15646720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.038 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:06:45.039 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.738537173999999, 2024-09-14T09:06:45.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0254846360000001, 2024-09-14T09:06:45.039 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.039 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.039 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.039 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.039 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.039 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.039 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73546 2024-09-14T09:06:45.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.039 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:06:45.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:06:45.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268934+0000", 2024-09-14T09:06:45.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7999999999999995e-07 2024-09-14T09:06:45.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268937+0000", 2024-09-14T09:06:45.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9840000000000001e-06 2024-09-14T09:06:45.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.041 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.041 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268947+0000", 2024-09-14T09:06:45.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0522e-05 2024-09-14T09:06:45.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.041 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.041 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268955+0000", 2024-09-14T09:06:45.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8059999999999995e-06 2024-09-14T09:06:45.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.041 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.041 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282112+0000", 2024-09-14T09:06:45.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013157189 2024-09-14T09:06:45.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282115+0000", 2024-09-14T09:06:45.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2949999999999998e-06 2024-09-14T09:06:45.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293968+0000", 2024-09-14T09:06:45.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0118530990000001 2024-09-14T09:06:45.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294197+0000", 2024-09-14T09:06:45.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228507 2024-09-14T09:06:45.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294417+0000", 2024-09-14T09:06:45.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000220554 2024-09-14T09:06:45.043 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.043 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.043 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.043 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73547 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.043 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:06:45.043 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.738504728000001, 2024-09-14T09:06:45.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.025970525, 2024-09-14T09:06:45.043 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.043 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.044 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.044 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.044 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.044 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.044 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73547 2024-09-14T09:06:45.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.044 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:06:45.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:06:45.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268966+0000", 2024-09-14T09:06:45.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3799999999999997e-07 2024-09-14T09:06:45.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268969+0000", 2024-09-14T09:06:45.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8739999999999999e-06 2024-09-14T09:06:45.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268979+0000", 2024-09-14T09:06:45.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9939999999999995e-06 2024-09-14T09:06:45.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268986+0000", 2024-09-14T09:06:45.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3459999999999996e-06 2024-09-14T09:06:45.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282127+0000", 2024-09-14T09:06:45.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01314039 2024-09-14T09:06:45.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282130+0000", 2024-09-14T09:06:45.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2219999999999998e-06 2024-09-14T09:06:45.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294540+0000", 2024-09-14T09:06:45.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124105880000001 2024-09-14T09:06:45.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294759+0000", 2024-09-14T09:06:45.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000218798 2024-09-14T09:06:45.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294936+0000", 2024-09-14T09:06:45.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000176675 2024-09-14T09:06:45.048 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.048 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.048 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.048 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73548 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15810560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.048 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:06:45.048 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.738473616, 2024-09-14T09:06:45.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0265045129999999, 2024-09-14T09:06:45.048 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.048 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.048 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.048 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.049 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.049 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.049 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73548 2024-09-14T09:06:45.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.049 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:06:45.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:06:45.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268997+0000", 2024-09-14T09:06:45.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7599999999999996e-07 2024-09-14T09:06:45.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269000+0000", 2024-09-14T09:06:45.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0580000000000002e-06 2024-09-14T09:06:45.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269010+0000", 2024-09-14T09:06:45.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124e-05 2024-09-14T09:06:45.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269018+0000", 2024-09-14T09:06:45.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5399999999999998e-06 2024-09-14T09:06:45.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282141+0000", 2024-09-14T09:06:45.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013123012 2024-09-14T09:06:45.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282144+0000", 2024-09-14T09:06:45.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.196e-06 2024-09-14T09:06:45.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295042+0000", 2024-09-14T09:06:45.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0128975689999999 2024-09-14T09:06:45.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295272+0000", 2024-09-14T09:06:45.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023048099999999999 2024-09-14T09:06:45.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295501+0000", 2024-09-14T09:06:45.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228657 2024-09-14T09:06:45.052 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.052 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.053 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.053 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73549 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15892480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.053 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:06:45.053 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.738427737, 2024-09-14T09:06:45.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0268354099999999, 2024-09-14T09:06:45.053 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.053 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.053 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.053 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.053 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.053 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.054 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73549 2024-09-14T09:06:45.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.054 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:06:45.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:06:45.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269043+0000", 2024-09-14T09:06:45.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6199999999999997e-07 2024-09-14T09:06:45.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269046+0000", 2024-09-14T09:06:45.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2480000000000001e-06 2024-09-14T09:06:45.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269057+0000", 2024-09-14T09:06:45.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0436e-05 2024-09-14T09:06:45.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269064+0000", 2024-09-14T09:06:45.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5179999999999998e-06 2024-09-14T09:06:45.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282155+0000", 2024-09-14T09:06:45.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013091190000000001 2024-09-14T09:06:45.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282159+0000", 2024-09-14T09:06:45.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.366e-06 2024-09-14T09:06:45.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295599+0000", 2024-09-14T09:06:45.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0134405559999999 2024-09-14T09:06:45.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295765+0000", 2024-09-14T09:06:45.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016575499999999999 2024-09-14T09:06:45.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295878+0000", 2024-09-14T09:06:45.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112679 2024-09-14T09:06:45.057 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.057 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.057 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.058 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73550 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15974400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.058 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:06:45.058 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.738395136999999, 2024-09-14T09:06:45.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027357657, 2024-09-14T09:06:45.058 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.058 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.058 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.058 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.058 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.058 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.058 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73550 2024-09-14T09:06:45.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.059 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:06:45.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:06:45.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269076+0000", 2024-09-14T09:06:45.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:06:45.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269079+0000", 2024-09-14T09:06:45.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:06:45.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269098+0000", 2024-09-14T09:06:45.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9698e-05 2024-09-14T09:06:45.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269106+0000", 2024-09-14T09:06:45.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6179999999999997e-06 2024-09-14T09:06:45.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282170+0000", 2024-09-14T09:06:45.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013063874 2024-09-14T09:06:45.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282173+0000", 2024-09-14T09:06:45.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1999999999999999e-06 2024-09-14T09:06:45.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295996+0000", 2024-09-14T09:06:45.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.013823151 2024-09-14T09:06:45.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296214+0000", 2024-09-14T09:06:45.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021817700000000001 2024-09-14T09:06:45.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296433+0000", 2024-09-14T09:06:45.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021829099999999999 2024-09-14T09:06:45.063 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.063 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.063 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.063 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73551 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.063 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:06:45.063 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.738353719999999, 2024-09-14T09:06:45.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027730276, 2024-09-14T09:06:45.063 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.063 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.063 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.063 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.063 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.064 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.064 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73551 2024-09-14T09:06:45.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.064 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:06:45.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:06:45.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269117+0000", 2024-09-14T09:06:45.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:06:45.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269120+0000", 2024-09-14T09:06:45.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.056e-06 2024-09-14T09:06:45.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269130+0000", 2024-09-14T09:06:45.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0084e-05 2024-09-14T09:06:45.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269137+0000", 2024-09-14T09:06:45.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1359999999999999e-06 2024-09-14T09:06:45.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282184+0000", 2024-09-14T09:06:45.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013046537 2024-09-14T09:06:45.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282188+0000", 2024-09-14T09:06:45.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3220000000000003e-06 2024-09-14T09:06:45.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296529+0000", 2024-09-14T09:06:45.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0143406109999999 2024-09-14T09:06:45.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296695+0000", 2024-09-14T09:06:45.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016604799999999999 2024-09-14T09:06:45.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296847+0000", 2024-09-14T09:06:45.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000151822 2024-09-14T09:06:45.068 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.068 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.068 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.068 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73552 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.068 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:06:45.068 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.73832314, 2024-09-14T09:06:45.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0281767500000001, 2024-09-14T09:06:45.069 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.069 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.069 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.069 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.069 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.069 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.069 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73552 2024-09-14T09:06:45.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.069 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:06:45.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:06:45.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:06:45.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:06:45.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269150+0000", 2024-09-14T09:06:45.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9119999999999998e-06 2024-09-14T09:06:45.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269160+0000", 2024-09-14T09:06:45.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6819999999999998e-06 2024-09-14T09:06:45.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269168+0000", 2024-09-14T09:06:45.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6079999999999998e-06 2024-09-14T09:06:45.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282205+0000", 2024-09-14T09:06:45.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013036982000000001 2024-09-14T09:06:45.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282208+0000", 2024-09-14T09:06:45.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4620000000000001e-06 2024-09-14T09:06:45.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296968+0000", 2024-09-14T09:06:45.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0147601209999999 2024-09-14T09:06:45.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297115+0000", 2024-09-14T09:06:45.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014673900000000001 2024-09-14T09:06:45.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297324+0000", 2024-09-14T09:06:45.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020866400000000001 2024-09-14T09:06:45.073 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.073 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.073 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.073 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73553 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.074 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:06:45.074 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.738293076, 2024-09-14T09:06:45.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0286556739999999, 2024-09-14T09:06:45.074 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.074 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.074 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.074 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.074 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.074 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.074 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73553 2024-09-14T09:06:45.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.075 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:06:45.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:06:45.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269178+0000", 2024-09-14T09:06:45.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:06:45.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269180+0000", 2024-09-14T09:06:45.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T09:06:45.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269189+0000", 2024-09-14T09:06:45.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2e-06 2024-09-14T09:06:45.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269197+0000", 2024-09-14T09:06:45.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2980000000000003e-06 2024-09-14T09:06:45.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282219+0000", 2024-09-14T09:06:45.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013022687 2024-09-14T09:06:45.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282223+0000", 2024-09-14T09:06:45.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.241e-06 2024-09-14T09:06:45.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297470+0000", 2024-09-14T09:06:45.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.015247778 2024-09-14T09:06:45.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297704+0000", 2024-09-14T09:06:45.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000233367 2024-09-14T09:06:45.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297833+0000", 2024-09-14T09:06:45.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012888900000000001 2024-09-14T09:06:45.079 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.079 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.080 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.080 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73554 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.080 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:06:45.080 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.738263759000001, 2024-09-14T09:06:45.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.029005113, 2024-09-14T09:06:45.080 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.080 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.080 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.080 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.081 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.081 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.081 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73554 2024-09-14T09:06:45.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.081 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:06:45.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:06:45.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269207+0000", 2024-09-14T09:06:45.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.68e-07 2024-09-14T09:06:45.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269210+0000", 2024-09-14T09:06:45.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8100000000000002e-06 2024-09-14T09:06:45.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269219+0000", 2024-09-14T09:06:45.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:06:45.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269227+0000", 2024-09-14T09:06:45.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3839999999999999e-06 2024-09-14T09:06:45.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282234+0000", 2024-09-14T09:06:45.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013007187 2024-09-14T09:06:45.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282237+0000", 2024-09-14T09:06:45.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1200000000000002e-06 2024-09-14T09:06:45.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297942+0000", 2024-09-14T09:06:45.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0157048040000001 2024-09-14T09:06:45.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298087+0000", 2024-09-14T09:06:45.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014499099999999999 2024-09-14T09:06:45.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298211+0000", 2024-09-14T09:06:45.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012461899999999999 2024-09-14T09:06:45.085 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.085 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.085 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.085 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73555 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.085 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:06:45.085 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.738233911, 2024-09-14T09:06:45.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.02941558, 2024-09-14T09:06:45.085 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.085 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.085 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.085 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.085 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.085 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.085 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73555 2024-09-14T09:06:45.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.086 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:06:45.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:06:45.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269237+0000", 2024-09-14T09:06:45.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3399999999999999e-07 2024-09-14T09:06:45.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269240+0000", 2024-09-14T09:06:45.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8480000000000001e-06 2024-09-14T09:06:45.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269249+0000", 2024-09-14T09:06:45.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1880000000000008e-06 2024-09-14T09:06:45.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269256+0000", 2024-09-14T09:06:45.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:06:45.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282248+0000", 2024-09-14T09:06:45.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012992327 2024-09-14T09:06:45.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282251+0000", 2024-09-14T09:06:45.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2160000000000002e-06 2024-09-14T09:06:45.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298299+0000", 2024-09-14T09:06:45.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0160479330000001 2024-09-14T09:06:45.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.089 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.089 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298446+0000", 2024-09-14T09:06:45.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000146525 2024-09-14T09:06:45.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.089 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.089 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298652+0000", 2024-09-14T09:06:45.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020584100000000001 2024-09-14T09:06:45.089 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.089 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.089 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.090 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73556 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.090 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:06:45.090 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.738204164999999, 2024-09-14T09:06:45.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0299854239999999, 2024-09-14T09:06:45.090 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.091 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.091 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.091 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.091 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.091 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.091 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73556 2024-09-14T09:06:45.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.092 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:06:45.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:06:45.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:06:45.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:06:45.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269270+0000", 2024-09-14T09:06:45.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0920000000000002e-06 2024-09-14T09:06:45.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269280+0000", 2024-09-14T09:06:45.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0013999999999999e-05 2024-09-14T09:06:45.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.094 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269288+0000", 2024-09-14T09:06:45.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:06:45.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.094 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.094 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282263+0000", 2024-09-14T09:06:45.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012974703000000001 2024-09-14T09:06:45.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.094 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.094 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282266+0000", 2024-09-14T09:06:45.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1520000000000001e-06 2024-09-14T09:06:45.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298729+0000", 2024-09-14T09:06:45.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.016463506 2024-09-14T09:06:45.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298884+0000", 2024-09-14T09:06:45.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015450800000000001 2024-09-14T09:06:45.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299251+0000", 2024-09-14T09:06:45.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036759899999999997 2024-09-14T09:06:45.096 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.096 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.096 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.096 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73557 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16547840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.096 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:06:45.096 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.738157514999999, 2024-09-14T09:06:45.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0307969480000001, 2024-09-14T09:06:45.096 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.096 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.096 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.097 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.097 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.097 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.097 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73557 2024-09-14T09:06:45.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.097 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:06:45.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:06:45.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269313+0000", 2024-09-14T09:06:45.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.92e-07 2024-09-14T09:06:45.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269336+0000", 2024-09-14T09:06:45.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3118000000000001e-05 2024-09-14T09:06:45.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269346+0000", 2024-09-14T09:06:45.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4069999999999999e-06 2024-09-14T09:06:45.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269352+0000", 2024-09-14T09:06:45.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5810000000000002e-06 2024-09-14T09:06:45.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282277+0000", 2024-09-14T09:06:45.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012924658 2024-09-14T09:06:45.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282280+0000", 2024-09-14T09:06:45.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2880000000000001e-06 2024-09-14T09:06:45.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299343+0000", 2024-09-14T09:06:45.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0170630190000001 2024-09-14T09:06:45.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299554+0000", 2024-09-14T09:06:45.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021085099999999999 2024-09-14T09:06:45.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300109+0000", 2024-09-14T09:06:45.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00055533399999999999 2024-09-14T09:06:45.101 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.101 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.101 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.101 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73558 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.101 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:06:45.101 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.738105892, 2024-09-14T09:06:45.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0314849340000001, 2024-09-14T09:06:45.102 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.102 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.102 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.102 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.102 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.102 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.102 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73558 2024-09-14T09:06:45.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.102 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:06:45.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:06:45.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269365+0000", 2024-09-14T09:06:45.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7400000000000003e-07 2024-09-14T09:06:45.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269367+0000", 2024-09-14T09:06:45.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4930000000000001e-06 2024-09-14T09:06:45.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269374+0000", 2024-09-14T09:06:45.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9519999999999997e-06 2024-09-14T09:06:45.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269379+0000", 2024-09-14T09:06:45.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9710000000000003e-06 2024-09-14T09:06:45.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282292+0000", 2024-09-14T09:06:45.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012912704000000001 2024-09-14T09:06:45.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282295+0000", 2024-09-14T09:06:45.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.179e-06 2024-09-14T09:06:45.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300195+0000", 2024-09-14T09:06:45.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.017899975 2024-09-14T09:06:45.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300345+0000", 2024-09-14T09:06:45.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014957499999999999 2024-09-14T09:06:45.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300849+0000", 2024-09-14T09:06:45.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050451100000000002 2024-09-14T09:06:45.106 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.106 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.106 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.106 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73559 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:45.106 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:06:45.106 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.738083483, 2024-09-14T09:06:45.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.031916343, 2024-09-14T09:06:45.106 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:45.106 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:45.106 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:45.107 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:45.107 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:45.107 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:45.107 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73559 2024-09-14T09:06:45.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.107 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:45.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:45.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:06:45.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:45.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:06:45.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:45.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:45.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:06:45.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3500000000000002e-07 2024-09-14T09:06:45.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:45.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269390+0000", 2024-09-14T09:06:45.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5890000000000001e-06 2024-09-14T09:06:45.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:45.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269396+0000", 2024-09-14T09:06:45.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3300000000000004e-06 2024-09-14T09:06:45.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:45.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269401+0000", 2024-09-14T09:06:45.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0340000000000004e-06 2024-09-14T09:06:45.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282311+0000", 2024-09-14T09:06:45.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012910204 2024-09-14T09:06:45.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:45.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282317+0000", 2024-09-14T09:06:45.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2639999999999997e-06 2024-09-14T09:06:45.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:45.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300933+0000", 2024-09-14T09:06:45.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0186160710000001 2024-09-14T09:06:45.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:45.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301158+0000", 2024-09-14T09:06:45.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225021 2024-09-14T09:06:45.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:45.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:45.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:45.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301303+0000", 2024-09-14T09:06:45.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014439499999999999 2024-09-14T09:06:45.110 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.111 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.111 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.111 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:45.111 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:45.111 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:45.112 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:06:45.112 INFO:teuthology.orchestra.run.smithi176.stdout:{"fsid":"c635667d-994e-454f-bcea-51b45637b2a2","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"nodeep-scrub flag(s) set","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":107,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":85,"num_osds":8,"num_up_osds":8,"osd_up_since":1726304711,"num_in_osds":8,"osd_in_since":1726304699,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":34},{"state_name":"remapped+peering","count":6},{"state_name":"peering","count":1}],"num_pgs":41,"num_pools":3,"num_objects":1200,"data_bytes":1036797639,"bytes_used":346861568,"bytes_avail":772747251712,"bytes_total":773094113280,"inactive_pgs_ratio":0.17073170840740204,"recovering_objects_per_sec":58,"recovering_bytes_per_sec":2872109,"recovering_keys_per_sec":3,"num_objects_recovered":214,"num_bytes_recovered":10481218,"num_keys_recovered":11,"read_bytes_sec":361133,"write_bytes_sec":516586,"read_op_per_sec":188,"write_op_per_sec":92},"fsmap":{"epoch":1,"btime":"2024-09-14T09:04:57:032443+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-14T09:06:13.975920+0000","services":{}},"progress_events":{}} 2024-09-14T09:06:45.112 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:06:45.113 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-14T09:06:45.113 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:06:45.114 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:45.111+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:45.213 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:45.215+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:45.257 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:06:45.285 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:45.286 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:45.286 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:45.286 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:45.302 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:06:45.313 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:45.315+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:45.414 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:45.414 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:45.414 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:45.415+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:45.431 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:45.431 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:45.449 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:45.449 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:45.464 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:45.464 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:45.466 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:45.466 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:45.479 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:06:45.480 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:06:45.482 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:06:45.488 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:45.489 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:45.501 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-14T09:06:44.850507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1036797639,"num_objects":1200,"num_object_clones":367,"num_object_copies":2400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1200,"num_whiteouts":440,"num_read":102637,"num_read_kb":539394,"num_write":12660,"num_write_kb":221220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":23018534,"num_keys_recovered":66,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9946,"ondisk_log_size":9946,"up":82,"acting":77,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":338732,"kb_used_data":42036,"kb_used_omap":12,"kb_used_meta":296499,"kb_avail":754635988,"statfs":{"total":773094113280,"available":772747251712,"internally_reserved":0,"allocated":43044864,"data_stored":307259437,"data_compressed":9045170,"data_compressed_allocated":25071616,"data_compressed_original":297113030,"omap_allocated":12751,"internal_metadata":303615537},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11881771,"num_objects":-15,"num_object_clones":0,"num_object_copies":-30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-15,"num_whiteouts":0,"num_read":488,"num_read_kb":444,"num_write":169,"num_write_kb":400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":7481313,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.514258"},"pg_stats":[{"pgid":"3.1f","version":"81'1211","reported_seq":4111,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.832160+0000","last_change":"2024-09-14T09:06:41.348384+0000","last_active":"2024-09-14T09:06:41.832160+0000","last_peered":"2024-09-14T09:06:41.832160+0000","last_clean":"2024-09-14T09:06:41.832160+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:06:41.832160+0000","last_undegraded":"2024-09-14T09:06:41.832160+0000","last_fullsized":"2024-09-14T09:06:41.832160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1211,"log_dups_size":0,"ondisk_log_size":1211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:40:01.002873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001387512,"stat_sum":{"num_bytes":17479846,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":15,"num_read":8652,"num_read_kb":13323,"num_write":2090,"num_write_kb":4784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"79'195","reported_seq":2404,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344064+0000","last_change":"2024-09-14T09:05:29.039503+0000","last_active":"2024-09-14T09:06:40.344064+0000","last_peered":"2024-09-14T09:06:40.344064+0000","last_clean":"2024-09-14T09:06:40.344064+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:06:40.344064+0000","last_undegraded":"2024-09-14T09:06:40.344064+0000","last_fullsized":"2024-09-14T09:06:40.344064+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:25:36.908116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001657176,"stat_sum":{"num_bytes":31504440,"num_objects":51,"num_object_clones":18,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":20,"num_read":2150,"num_read_kb":22915,"num_write":231,"num_write_kb":8634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"79'360","reported_seq":3728,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.830471+0000","last_change":"2024-09-14T09:05:28.835298+0000","last_active":"2024-09-14T09:06:41.830471+0000","last_peered":"2024-09-14T09:06:41.830471+0000","last_clean":"2024-09-14T09:06:41.830471+0000","last_became_active":"2024-09-14T09:05:26.835372+0000","last_became_peered":"2024-09-14T09:05:26.835372+0000","last_unstale":"2024-09-14T09:06:41.830471+0000","last_undegraded":"2024-09-14T09:06:41.830471+0000","last_fullsized":"2024-09-14T09:06:41.830471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:38.418901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014149329999999999,"stat_sum":{"num_bytes":57821985,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":10,"num_read":3993,"num_read_kb":52947,"num_write":505,"num_write_kb":8809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"79'518","reported_seq":3036,"reported_epoch":81,"state":"remapped+peering","last_fresh":"2024-09-14T09:06:41.345959+0000","last_change":"2024-09-14T09:06:41.831155+0000","last_active":"2024-09-14T09:06:41.345959+0000","last_peered":"2024-09-14T09:06:40.344845+0000","last_clean":"2024-09-14T09:06:40.344845+0000","last_became_active":"2024-09-14T09:05:39.918593+0000","last_became_peered":"2024-09-14T09:05:39.918593+0000","last_unstale":"2024-09-14T09:06:41.345959+0000","last_undegraded":"2024-09-14T09:06:41.345959+0000","last_fullsized":"2024-09-14T09:06:41.345959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014661419999999999,"stat_sum":{"num_bytes":38979686,"num_objects":37,"num_object_clones":12,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":5418,"num_read_kb":20602,"num_write":735,"num_write_kb":6091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":17,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"79'205","reported_seq":2345,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.832877+0000","last_change":"2024-09-14T09:06:41.347478+0000","last_active":"2024-09-14T09:06:41.832877+0000","last_peered":"2024-09-14T09:06:41.832877+0000","last_clean":"2024-09-14T09:06:41.832877+0000","last_became_active":"2024-09-14T09:06:41.346397+0000","last_became_peered":"2024-09-14T09:06:41.346397+0000","last_unstale":"2024-09-14T09:06:41.832877+0000","last_undegraded":"2024-09-14T09:06:41.832877+0000","last_fullsized":"2024-09-14T09:06:41.832877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:30:10.950021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060373399999999998,"stat_sum":{"num_bytes":30069643,"num_objects":40,"num_object_clones":16,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":1888,"num_read_kb":5564,"num_write":210,"num_write_kb":11506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":6073202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"79'477","reported_seq":3286,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344261+0000","last_change":"2024-09-14T09:05:39.919093+0000","last_active":"2024-09-14T09:06:40.344261+0000","last_peered":"2024-09-14T09:06:40.344261+0000","last_clean":"2024-09-14T09:06:40.344261+0000","last_became_active":"2024-09-14T09:05:39.905126+0000","last_became_peered":"2024-09-14T09:05:39.905126+0000","last_unstale":"2024-09-14T09:06:40.344261+0000","last_undegraded":"2024-09-14T09:06:40.344261+0000","last_fullsized":"2024-09-14T09:06:40.344261+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:21:58.882897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020845550000000001,"stat_sum":{"num_bytes":33943451,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":4393,"num_read_kb":18572,"num_write":762,"num_write_kb":10051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":92669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"79'206","reported_seq":2074,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343852+0000","last_change":"2024-09-14T09:05:28.822565+0000","last_active":"2024-09-14T09:06:40.343852+0000","last_peered":"2024-09-14T09:06:40.343852+0000","last_clean":"2024-09-14T09:06:40.343852+0000","last_became_active":"2024-09-14T09:05:26.816132+0000","last_became_peered":"2024-09-14T09:05:26.816132+0000","last_unstale":"2024-09-14T09:06:40.343852+0000","last_undegraded":"2024-09-14T09:06:40.343852+0000","last_fullsized":"2024-09-14T09:06:40.343852+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:01:39.934279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092420299999999996,"stat_sum":{"num_bytes":21650738,"num_objects":32,"num_object_clones":11,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":12,"num_read":1798,"num_read_kb":6256,"num_write":243,"num_write_kb":4114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"79'283","reported_seq":3314,"reported_epoch":79,"state":"peering","last_fresh":"2024-09-14T09:06:39.775896+0000","last_change":"2024-09-14T09:06:40.341188+0000","last_active":"2024-09-14T09:06:40.341188+0000","last_peered":"2024-09-14T09:06:39.775896+0000","last_clean":"2024-09-14T09:06:37.214421+0000","last_became_active":"2024-09-14T09:06:39.405393+0000","last_became_peered":"2024-09-14T09:06:39.405393+0000","last_unstale":"2024-09-14T09:06:39.775896+0000","last_undegraded":"2024-09-14T09:06:39.775896+0000","last_fullsized":"2024-09-14T09:06:39.314493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081579899999999997,"stat_sum":{"num_bytes":49898996,"num_objects":46,"num_object_clones":17,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":14,"num_read":3640,"num_read_kb":7929,"num_write":273,"num_write_kb":3948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"79'355","reported_seq":4347,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.832529+0000","last_change":"2024-09-14T09:05:28.839928+0000","last_active":"2024-09-14T09:06:41.832529+0000","last_peered":"2024-09-14T09:06:41.832529+0000","last_clean":"2024-09-14T09:06:41.832529+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:06:41.832529+0000","last_undegraded":"2024-09-14T09:06:41.832529+0000","last_fullsized":"2024-09-14T09:06:41.832529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:36:52.750689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014343089999999999,"stat_sum":{"num_bytes":37014757,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":4961,"num_read_kb":22170,"num_write":438,"num_write_kb":4862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"79'374","reported_seq":1522,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.830446+0000","last_change":"2024-09-14T09:05:28.836370+0000","last_active":"2024-09-14T09:06:41.830446+0000","last_peered":"2024-09-14T09:06:41.830446+0000","last_clean":"2024-09-14T09:06:41.830446+0000","last_became_active":"2024-09-14T09:05:26.836744+0000","last_became_peered":"2024-09-14T09:05:26.836744+0000","last_unstale":"2024-09-14T09:06:41.830446+0000","last_undegraded":"2024-09-14T09:06:41.830446+0000","last_fullsized":"2024-09-14T09:06:41.830446+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:11:06.446635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015277190000000001,"stat_sum":{"num_bytes":15473207,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":9,"num_read":3094,"num_read_kb":7897,"num_write":504,"num_write_kb":3528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"79'506","reported_seq":1630,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343594+0000","last_change":"2024-09-14T09:05:28.851587+0000","last_active":"2024-09-14T09:06:40.343594+0000","last_peered":"2024-09-14T09:06:40.343594+0000","last_clean":"2024-09-14T09:06:40.343594+0000","last_became_active":"2024-09-14T09:05:26.840893+0000","last_became_peered":"2024-09-14T09:05:26.840893+0000","last_unstale":"2024-09-14T09:06:40.343594+0000","last_undegraded":"2024-09-14T09:06:40.343594+0000","last_fullsized":"2024-09-14T09:06:40.343594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:35:19.280259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098594000000000004,"stat_sum":{"num_bytes":8828839,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":3906,"num_read_kb":3852,"num_write":772,"num_write_kb":2737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"79'323","reported_seq":1557,"reported_epoch":82,"state":"remapped+peering","last_fresh":"2024-09-14T09:06:41.833669+0000","last_change":"2024-09-14T09:06:41.833669+0000","last_active":"2024-09-14T09:06:41.833669+0000","last_peered":"2024-09-14T09:06:39.647923+0000","last_clean":"2024-09-14T09:06:39.647923+0000","last_became_active":"2024-09-14T09:05:42.829504+0000","last_became_peered":"2024-09-14T09:05:42.829504+0000","last_unstale":"2024-09-14T09:06:41.833669+0000","last_undegraded":"2024-09-14T09:06:41.833669+0000","last_fullsized":"2024-09-14T09:06:41.833669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014079679999999999,"stat_sum":{"num_bytes":20404479,"num_objects":40,"num_object_clones":13,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":2269,"num_read_kb":3974,"num_write":371,"num_write_kb":897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":53852,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.833601+0000","last_change":"2024-09-14T09:05:19.793674+0000","last_active":"2024-09-14T09:06:41.833601+0000","last_peered":"2024-09-14T09:06:41.833601+0000","last_clean":"2024-09-14T09:06:41.833601+0000","last_became_active":"2024-09-14T09:05:17.284977+0000","last_became_peered":"2024-09-14T09:05:17.284977+0000","last_unstale":"2024-09-14T09:06:41.833601+0000","last_undegraded":"2024-09-14T09:06:41.833601+0000","last_fullsized":"2024-09-14T09:06:41.833601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:30:38.832027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"79'219","reported_seq":3515,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.832069+0000","last_change":"2024-09-14T09:05:28.839614+0000","last_active":"2024-09-14T09:06:41.832069+0000","last_peered":"2024-09-14T09:06:41.832069+0000","last_clean":"2024-09-14T09:06:41.832069+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:06:41.832069+0000","last_undegraded":"2024-09-14T09:06:41.832069+0000","last_fullsized":"2024-09-14T09:06:41.832069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:30:20.294407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014245869999999999,"stat_sum":{"num_bytes":39824462,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":12,"num_read":4241,"num_read_kb":38806,"num_write":236,"num_write_kb":5668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344323+0000","last_change":"2024-09-14T09:05:19.314029+0000","last_active":"2024-09-14T09:06:40.344323+0000","last_peered":"2024-09-14T09:06:40.344323+0000","last_clean":"2024-09-14T09:06:40.344323+0000","last_became_active":"2024-09-14T09:05:17.282890+0000","last_became_peered":"2024-09-14T09:05:17.282890+0000","last_unstale":"2024-09-14T09:06:40.344323+0000","last_undegraded":"2024-09-14T09:06:40.344323+0000","last_fullsized":"2024-09-14T09:06:40.344323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:25.304105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"79'359","reported_seq":2348,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.342863+0000","last_change":"2024-09-14T09:05:29.039154+0000","last_active":"2024-09-14T09:06:40.342863+0000","last_peered":"2024-09-14T09:06:40.342863+0000","last_clean":"2024-09-14T09:06:40.342863+0000","last_became_active":"2024-09-14T09:05:26.829855+0000","last_became_peered":"2024-09-14T09:05:26.829855+0000","last_unstale":"2024-09-14T09:06:40.342863+0000","last_undegraded":"2024-09-14T09:06:40.342863+0000","last_fullsized":"2024-09-14T09:06:40.342863+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:12:56.716058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019213749999999999,"stat_sum":{"num_bytes":37534250,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":3271,"num_read_kb":12689,"num_write":469,"num_write_kb":3865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":136,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344083+0000","last_change":"2024-09-14T09:05:19.794165+0000","last_active":"2024-09-14T09:06:40.344083+0000","last_peered":"2024-09-14T09:06:40.344083+0000","last_clean":"2024-09-14T09:06:40.344083+0000","last_became_active":"2024-09-14T09:05:17.295864+0000","last_became_peered":"2024-09-14T09:05:17.295864+0000","last_unstale":"2024-09-14T09:06:40.344083+0000","last_undegraded":"2024-09-14T09:06:40.344083+0000","last_fullsized":"2024-09-14T09:06:40.344083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:53.901865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011399719999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"79'262","reported_seq":2624,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344256+0000","last_change":"2024-09-14T09:06:35.286509+0000","last_active":"2024-09-14T09:06:40.344256+0000","last_peered":"2024-09-14T09:06:40.344256+0000","last_clean":"2024-09-14T09:06:40.344256+0000","last_became_active":"2024-09-14T09:06:35.282469+0000","last_became_peered":"2024-09-14T09:06:35.282469+0000","last_unstale":"2024-09-14T09:06:40.344256+0000","last_undegraded":"2024-09-14T09:06:40.344256+0000","last_fullsized":"2024-09-14T09:06:40.344256+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:56:24.711903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014087959999999999,"stat_sum":{"num_bytes":19969479,"num_objects":43,"num_object_clones":15,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":18,"num_read":3211,"num_read_kb":14319,"num_write":271,"num_write_kb":5415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":1142719,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":136,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344423+0000","last_change":"2024-09-14T09:05:19.314129+0000","last_active":"2024-09-14T09:06:40.344423+0000","last_peered":"2024-09-14T09:06:40.344423+0000","last_clean":"2024-09-14T09:06:40.344423+0000","last_became_active":"2024-09-14T09:05:17.295196+0000","last_became_peered":"2024-09-14T09:05:17.295196+0000","last_unstale":"2024-09-14T09:06:40.344423+0000","last_undegraded":"2024-09-14T09:06:40.344423+0000","last_fullsized":"2024-09-14T09:06:40.344423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:35.269115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011613800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"71'162","reported_seq":855,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.830615+0000","last_change":"2024-09-14T09:05:28.835976+0000","last_active":"2024-09-14T09:06:41.830615+0000","last_peered":"2024-09-14T09:06:41.830615+0000","last_clean":"2024-09-14T09:06:41.830615+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:06:41.830615+0000","last_undegraded":"2024-09-14T09:06:41.830615+0000","last_fullsized":"2024-09-14T09:06:41.830615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:21:06.765695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017480779999999999,"stat_sum":{"num_bytes":52867963,"num_objects":45,"num_object_clones":16,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":21,"num_read":596,"num_read_kb":9794,"num_write":149,"num_write_kb":9015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"80'343","reported_seq":1831,"reported_epoch":80,"state":"remapped+peering","last_fresh":"2024-09-14T09:06:40.888318+0000","last_change":"2024-09-14T09:06:41.832454+0000","last_active":"2024-09-14T09:06:41.832454+0000","last_peered":"2024-09-14T09:06:40.888318+0000","last_clean":"2024-09-14T09:06:40.888318+0000","last_became_active":"2024-09-14T09:05:39.914913+0000","last_became_peered":"2024-09-14T09:05:39.914913+0000","last_unstale":"2024-09-14T09:06:40.888318+0000","last_undegraded":"2024-09-14T09:06:40.888318+0000","last_fullsized":"2024-09-14T09:06:40.888318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014128,"stat_sum":{"num_bytes":25020722,"num_objects":35,"num_object_clones":12,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":2596,"num_read_kb":15720,"num_write":427,"num_write_kb":8882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":4,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.830325+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:06:41.830325+0000","last_peered":"2024-09-14T09:06:41.830325+0000","last_clean":"2024-09-14T09:06:41.830325+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:06:41.830325+0000","last_undegraded":"2024-09-14T09:06:41.830325+0000","last_fullsized":"2024-09-14T09:06:41.830325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"79'168","reported_seq":902,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344632+0000","last_change":"2024-09-14T09:05:28.822791+0000","last_active":"2024-09-14T09:06:40.344632+0000","last_peered":"2024-09-14T09:06:40.344632+0000","last_clean":"2024-09-14T09:06:40.344632+0000","last_became_active":"2024-09-14T09:05:26.817109+0000","last_became_peered":"2024-09-14T09:05:26.817109+0000","last_unstale":"2024-09-14T09:06:40.344632+0000","last_undegraded":"2024-09-14T09:06:40.344632+0000","last_fullsized":"2024-09-14T09:06:40.344632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:08:41.364341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011728439999999999,"stat_sum":{"num_bytes":14678273,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":791,"num_read_kb":3583,"num_write":179,"num_write_kb":1360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.827234+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:06:41.827234+0000","last_peered":"2024-09-14T09:06:41.827234+0000","last_clean":"2024-09-14T09:06:41.827234+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:06:41.827234+0000","last_undegraded":"2024-09-14T09:06:41.827234+0000","last_fullsized":"2024-09-14T09:06:41.827234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"79'201","reported_seq":1805,"reported_epoch":81,"state":"remapped+peering","last_fresh":"2024-09-14T09:06:41.339775+0000","last_change":"2024-09-14T09:06:41.832680+0000","last_active":"2024-09-14T09:06:41.339775+0000","last_peered":"2024-09-14T09:06:40.335804+0000","last_clean":"2024-09-14T09:06:40.335804+0000","last_became_active":"2024-09-14T09:05:39.905252+0000","last_became_peered":"2024-09-14T09:05:39.905252+0000","last_unstale":"2024-09-14T09:06:41.339775+0000","last_undegraded":"2024-09-14T09:06:41.339775+0000","last_fullsized":"2024-09-14T09:06:41.339775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074815999999999999,"stat_sum":{"num_bytes":28865016,"num_objects":30,"num_object_clones":11,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":13,"num_read":1487,"num_read_kb":15691,"num_write":203,"num_write_kb":9083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2097152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":136,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343102+0000","last_change":"2024-09-14T09:05:19.793999+0000","last_active":"2024-09-14T09:06:40.343102+0000","last_peered":"2024-09-14T09:06:40.343102+0000","last_clean":"2024-09-14T09:06:40.343102+0000","last_became_active":"2024-09-14T09:05:17.284122+0000","last_became_peered":"2024-09-14T09:05:17.284122+0000","last_unstale":"2024-09-14T09:06:40.343102+0000","last_undegraded":"2024-09-14T09:06:40.343102+0000","last_fullsized":"2024-09-14T09:06:40.343102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:12:01.133215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010234269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"68'195","reported_seq":1109,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.342756+0000","last_change":"2024-09-14T09:05:29.038389+0000","last_active":"2024-09-14T09:06:40.342756+0000","last_peered":"2024-09-14T09:06:40.342756+0000","last_clean":"2024-09-14T09:06:40.342756+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:06:40.342756+0000","last_undegraded":"2024-09-14T09:06:40.342756+0000","last_fullsized":"2024-09-14T09:06:40.342756+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:42.547444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001479076,"stat_sum":{"num_bytes":19439154,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":1071,"num_read_kb":4172,"num_write":238,"num_write_kb":2073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.342699+0000","last_change":"2024-09-14T09:06:25.227644+0000","last_active":"2024-09-14T09:06:40.342699+0000","last_peered":"2024-09-14T09:06:40.342699+0000","last_clean":"2024-09-14T09:06:40.342699+0000","last_became_active":"2024-09-14T09:06:25.222344+0000","last_became_peered":"2024-09-14T09:06:25.222344+0000","last_unstale":"2024-09-14T09:06:40.342699+0000","last_undegraded":"2024-09-14T09:06:40.342699+0000","last_fullsized":"2024-09-14T09:06:40.342699+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:23:33.414859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":227,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343412+0000","last_change":"2024-09-14T09:05:13.239808+0000","last_active":"2024-09-14T09:06:40.343412+0000","last_peered":"2024-09-14T09:06:40.343412+0000","last_clean":"2024-09-14T09:06:40.343412+0000","last_became_active":"2024-09-14T09:05:13.239064+0000","last_became_peered":"2024-09-14T09:05:13.239064+0000","last_unstale":"2024-09-14T09:06:40.343412+0000","last_undegraded":"2024-09-14T09:06:40.343412+0000","last_fullsized":"2024-09-14T09:06:40.343412+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:40:11.559161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"79'231","reported_seq":3284,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343364+0000","last_change":"2024-09-14T09:05:29.037923+0000","last_active":"2024-09-14T09:06:40.343364+0000","last_peered":"2024-09-14T09:06:40.343364+0000","last_clean":"2024-09-14T09:06:40.343364+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:06:40.343364+0000","last_undegraded":"2024-09-14T09:06:40.343364+0000","last_fullsized":"2024-09-14T09:06:40.343364+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:30:17.623019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099199800000000001,"stat_sum":{"num_bytes":42354761,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":3488,"num_read_kb":29604,"num_write":249,"num_write_kb":7446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"79'189","reported_seq":1415,"reported_epoch":80,"state":"remapped+peering","last_fresh":"2024-09-14T09:06:40.340320+0000","last_change":"2024-09-14T09:06:41.827712+0000","last_active":"2024-09-14T09:06:41.827712+0000","last_peered":"2024-09-14T09:06:40.340320+0000","last_clean":"2024-09-14T09:06:40.340320+0000","last_became_active":"2024-09-14T09:05:39.924025+0000","last_became_peered":"2024-09-14T09:05:39.924025+0000","last_unstale":"2024-09-14T09:06:40.340320+0000","last_undegraded":"2024-09-14T09:06:40.340320+0000","last_fullsized":"2024-09-14T09:06:40.340320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001266028,"stat_sum":{"num_bytes":27628229,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":1134,"num_read_kb":6881,"num_write":171,"num_write_kb":14037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7],"up_primary":4,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"79'257","reported_seq":3062,"reported_epoch":81,"state":"remapped+peering","last_fresh":"2024-09-14T09:06:41.344987+0000","last_change":"2024-09-14T09:06:41.833516+0000","last_active":"2024-09-14T09:06:41.344987+0000","last_peered":"2024-09-14T09:06:40.352617+0000","last_clean":"2024-09-14T09:06:40.352617+0000","last_became_active":"2024-09-14T09:05:39.918526+0000","last_became_peered":"2024-09-14T09:05:39.918526+0000","last_unstale":"2024-09-14T09:06:41.344987+0000","last_undegraded":"2024-09-14T09:06:41.344987+0000","last_fullsized":"2024-09-14T09:06:41.344987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185731,"stat_sum":{"num_bytes":48761384,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":3293,"num_read_kb":31783,"num_write":289,"num_write_kb":16306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":2159421,"num_keys_recovered":3,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"79'303","reported_seq":7865,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.342949+0000","last_change":"2024-09-14T09:06:02.837388+0000","last_active":"2024-09-14T09:06:40.342949+0000","last_peered":"2024-09-14T09:06:40.342949+0000","last_clean":"2024-09-14T09:06:40.342949+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:06:40.342949+0000","last_undegraded":"2024-09-14T09:06:40.342949+0000","last_fullsized":"2024-09-14T09:06:40.342949+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:55:06.321684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001585295,"stat_sum":{"num_bytes":39029359,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":8749,"num_read_kb":52948,"num_write":301,"num_write_kb":7483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"76'207","reported_seq":1173,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.830762+0000","last_change":"2024-09-14T09:05:28.834647+0000","last_active":"2024-09-14T09:06:41.830762+0000","last_peered":"2024-09-14T09:06:41.830762+0000","last_clean":"2024-09-14T09:06:41.830762+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:06:41.830762+0000","last_undegraded":"2024-09-14T09:06:41.830762+0000","last_fullsized":"2024-09-14T09:06:41.830762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:29:56.253705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244290000000001,"stat_sum":{"num_bytes":26698314,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":1353,"num_read_kb":7206,"num_write":220,"num_write_kb":7151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"79'211","reported_seq":2820,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343788+0000","last_change":"2024-09-14T09:05:28.851561+0000","last_active":"2024-09-14T09:06:40.343788+0000","last_peered":"2024-09-14T09:06:40.343788+0000","last_clean":"2024-09-14T09:06:40.343788+0000","last_became_active":"2024-09-14T09:05:26.843780+0000","last_became_peered":"2024-09-14T09:05:26.843780+0000","last_unstale":"2024-09-14T09:06:40.343788+0000","last_undegraded":"2024-09-14T09:06:40.343788+0000","last_fullsized":"2024-09-14T09:06:40.343788+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:55:09.169516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072987200000000001,"stat_sum":{"num_bytes":28500713,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":2826,"num_read_kb":21160,"num_write":266,"num_write_kb":13640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"79'164","reported_seq":1218,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343718+0000","last_change":"2024-09-14T09:05:40.010535+0000","last_active":"2024-09-14T09:06:40.343718+0000","last_peered":"2024-09-14T09:06:40.343718+0000","last_clean":"2024-09-14T09:06:40.343718+0000","last_became_active":"2024-09-14T09:05:39.921575+0000","last_became_peered":"2024-09-14T09:05:39.921575+0000","last_unstale":"2024-09-14T09:06:40.343718+0000","last_undegraded":"2024-09-14T09:06:40.343718+0000","last_fullsized":"2024-09-14T09:06:40.343718+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:34.630587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057596100000000005,"stat_sum":{"num_bytes":35589399,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":13,"num_read":919,"num_read_kb":9018,"num_write":150,"num_write_kb":4457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":111591,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"79'206","reported_seq":1026,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344064+0000","last_change":"2024-09-14T09:06:02.836145+0000","last_active":"2024-09-14T09:06:40.344064+0000","last_peered":"2024-09-14T09:06:40.344064+0000","last_clean":"2024-09-14T09:06:40.344064+0000","last_became_active":"2024-09-14T09:06:02.834684+0000","last_became_peered":"2024-09-14T09:06:02.834684+0000","last_unstale":"2024-09-14T09:06:40.344064+0000","last_undegraded":"2024-09-14T09:06:40.344064+0000","last_fullsized":"2024-09-14T09:06:40.344064+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:33:28.824221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000909463,"stat_sum":{"num_bytes":29699032,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":14,"num_read":1074,"num_read_kb":3425,"num_write":191,"num_write_kb":7631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":259265,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"79'226","reported_seq":4022,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.832839+0000","last_change":"2024-09-14T09:06:00.166357+0000","last_active":"2024-09-14T09:06:41.832839+0000","last_peered":"2024-09-14T09:06:41.832839+0000","last_clean":"2024-09-14T09:06:41.832839+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:06:41.832839+0000","last_undegraded":"2024-09-14T09:06:41.832839+0000","last_fullsized":"2024-09-14T09:06:41.832839+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:58:28.963004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011381609999999999,"stat_sum":{"num_bytes":85650268,"num_objects":42,"num_object_clones":13,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":17,"num_read":3660,"num_read_kb":42918,"num_write":284,"num_write_kb":15069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"79'313","reported_seq":2525,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.343379+0000","last_change":"2024-09-14T09:05:40.038345+0000","last_active":"2024-09-14T09:06:40.343379+0000","last_peered":"2024-09-14T09:06:40.343379+0000","last_clean":"2024-09-14T09:06:40.343379+0000","last_became_active":"2024-09-14T09:05:39.923017+0000","last_became_peered":"2024-09-14T09:05:39.923017+0000","last_unstale":"2024-09-14T09:06:40.343379+0000","last_undegraded":"2024-09-14T09:06:40.343379+0000","last_fullsized":"2024-09-14T09:06:40.343379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:12:14.761360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092949299999999999,"stat_sum":{"num_bytes":3898608,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":3511,"num_read_kb":3930,"num_write":421,"num_write_kb":729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"76'364","reported_seq":2563,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T09:06:41.832024+0000","last_change":"2024-09-14T09:05:28.839774+0000","last_active":"2024-09-14T09:06:41.832024+0000","last_peered":"2024-09-14T09:06:41.832024+0000","last_clean":"2024-09-14T09:06:41.832024+0000","last_became_active":"2024-09-14T09:05:26.826600+0000","last_became_peered":"2024-09-14T09:05:26.826600+0000","last_unstale":"2024-09-14T09:06:41.832024+0000","last_undegraded":"2024-09-14T09:06:41.832024+0000","last_fullsized":"2024-09-14T09:06:41.832024+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:37:44.244140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018357099999999999,"stat_sum":{"num_bytes":43950102,"num_objects":41,"num_object_clones":13,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":3652,"num_read_kb":3385,"num_write":457,"num_write_kb":3258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"79'278","reported_seq":4381,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T09:06:40.344780+0000","last_change":"2024-09-14T09:05:39.938958+0000","last_active":"2024-09-14T09:06:40.344780+0000","last_peered":"2024-09-14T09:06:40.344780+0000","last_clean":"2024-09-14T09:06:40.344780+0000","last_became_active":"2024-09-14T09:05:39.921023+0000","last_became_peered":"2024-09-14T09:05:39.921023+0000","last_unstale":"2024-09-14T09:06:40.344780+0000","last_undegraded":"2024-09-14T09:06:40.344780+0000","last_fullsized":"2024-09-14T09:06:40.344780+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:46:00.475180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061978900000000004,"stat_sum":{"num_bytes":23177697,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":5486,"num_read_kb":26306,"num_write":252,"num_write_kb":7061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":120888,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1036207252,"num_objects":1196,"num_object_clones":367,"num_object_copies":2392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1196,"num_whiteouts":440,"num_read":102571,"num_read_kb":539339,"num_write":12557,"num_write_kb":219590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":481,"num_bytes_recovered":23018534,"num_keys_recovered":66,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35364864,"data_stored":302647760,"data_compressed":9032746,"data_compressed_allocated":24997888,"data_compressed_original":295966150,"omap_allocated":0,"internal_metadata":0},"log_size":9871,"ondisk_log_size":9871,"up":64,"acting":59,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12424,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40460,"kb_used_data":5104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331380,"statfs":{"total":96636764160,"available":96595333120,"internally_reserved":0,"allocated":5226496,"data_stored":38232434,"data_compressed":1181978,"data_compressed_allocated":3141632,"data_compressed_original":37081632,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50800,"kb_used_data":5724,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94321040,"statfs":{"total":96636764160,"available":96584744960,"internally_reserved":0,"allocated":5861376,"data_stored":41776063,"data_compressed":1136197,"data_compressed_allocated":3391488,"data_compressed_original":40415084,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509463,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42424,"kb_used_data":2916,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329416,"statfs":{"total":96636764160,"available":96593321984,"internally_reserved":0,"allocated":2985984,"data_stored":19263970,"data_compressed":506678,"data_compressed_allocated":1609728,"data_compressed_original":18588448,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46304,"kb_used_data":6792,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325536,"statfs":{"total":96636764160,"available":96589348864,"internally_reserved":0,"allocated":6955008,"data_stored":53317915,"data_compressed":1517281,"data_compressed_allocated":4202496,"data_compressed_original":51760338,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33520,"kb_used_data":6492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338320,"statfs":{"total":96636764160,"available":96602439680,"internally_reserved":0,"allocated":6647808,"data_stored":48313146,"data_compressed":1220402,"data_compressed_allocated":3915776,"data_compressed_original":46621097,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542166,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2166784,"data_stored":12132788,"data_compressed":364162,"data_compressed_allocated":999424,"data_compressed_original":11568392,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542166,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45468,"kb_used_data":5956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326372,"statfs":{"total":96636764160,"available":96590204928,"internally_reserved":0,"allocated":6098944,"data_stored":47202911,"data_compressed":1328461,"data_compressed_allocated":3547136,"data_compressed_original":45737754,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542167,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46452,"kb_used_data":6936,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325388,"statfs":{"total":96636764160,"available":96589197312,"internally_reserved":0,"allocated":7102464,"data_stored":47020210,"data_compressed":1790011,"data_compressed_allocated":4263936,"data_compressed_original":45340285,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5951488,"data_stored":52298055,"data_compressed":1511069,"data_compressed_allocated":4165632,"data_compressed_original":51186898,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6164480,"data_stored":46596785,"data_compressed":1790011,"data_compressed_allocated":4263936,"data_compressed_original":45340285,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":18828041,"data_compressed":506678,"data_compressed_allocated":1609728,"data_compressed_original":18588448,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5165056,"data_stored":46779505,"data_compressed":1328461,"data_compressed_allocated":3547136,"data_compressed_original":45737754,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1232896,"data_stored":11709382,"data_compressed":364162,"data_compressed_allocated":999424,"data_compressed_original":11568392,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4898816,"data_stored":41340115,"data_compressed":1136197,"data_compressed_allocated":3391488,"data_compressed_original":40415084,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4268032,"data_stored":37796505,"data_compressed":1181978,"data_compressed_allocated":3141632,"data_compressed_original":37081632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5656576,"data_stored":47299372,"data_compressed":1214190,"data_compressed_allocated":3878912,"data_compressed_original":46047657,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:06:45.515 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:45.519+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:45.525 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:45.526 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:45.561 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:45.562 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:45.597 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:45.598 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:45.615 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:45.619+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:45.627 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:45.628 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:45.642 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:06:45.730 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:45.731+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:45.809 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:45.809 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:45.817 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:45.819+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:45.822 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:06:45.847 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is unset 2024-09-14T09:06:45.919 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:45.919+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:45.947 INFO:teuthology.orchestra.run.smithi176.stdout:3676 read 0x4333b21 thru 0x433dde1 (0xa2c1 bytes) 2024-09-14T09:06:45.947 INFO:teuthology.orchestra.run.smithi176.stdout:3680 writesame 0x6b68e83 thru 0x6b69bd2 data_size 0x10(0xd50 bytes) 2024-09-14T09:06:45.979 INFO:teuthology.orchestra.run.smithi176.stdout:3681 writesame 0xdf2c6e2 thru 0xdf371ba data_size 0x3d(0xaad9 bytes) 2024-09-14T09:06:45.979 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:45.979 INFO:teuthology.orchestra.run.smithi176.stdout:3683 read 0x93bc9d0 thru 0x93c51cc (0x87fd bytes) 2024-09-14T09:06:45.979 INFO:teuthology.orchestra.run.smithi176.stdout:3686 write 0xc63a51 thru 0xc6d326 (0x98d6 bytes) 2024-09-14T09:06:45.980 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:45.980 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:45.996 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:06:46.019 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:46.023+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:46.117 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:46.117 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:46.120 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:46.123+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:46.131 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:06:46.222 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:46.219+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:46.260 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:46.260 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:46.274 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:06:46.323 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:46.323+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:46.396 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:46.397 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:46.410 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:06:46.455 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:46.459+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:46.524 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:46.527+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:46.529 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:46.529 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:46.543 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:06:46.625 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:46.623+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:46.716 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:46.717 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:46.726 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:46.727+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:46.731 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:06:46.827 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:46.831+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:46.901 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:46.901 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:46.903 INFO:teuthology.orchestra.run.smithi176.stdout:3689 writesame 0x62698e0 thru 0x626ad84 data_size 0x7(0x14a5 bytes) 2024-09-14T09:06:46.913 INFO:teuthology.orchestra.run.smithi176.stdout:3690 punch from 0x6dd5e60 to 0x6dd896c, (0x2b0c bytes) 2024-09-14T09:06:46.913 INFO:teuthology.orchestra.run.smithi176.stdout:3691 read 0x141c470 thru 0x141d6eb (0x127c bytes) 2024-09-14T09:06:46.914 INFO:teuthology.orchestra.run.smithi176.stdout:3692 read 0x765a41a thru 0x7660a12 (0x65f9 bytes) 2024-09-14T09:06:46.914 INFO:teuthology.orchestra.run.smithi176.stdout:3695 write 0x1c31bcd thru 0x1c3ac4e (0x9082 bytes) 2024-09-14T09:06:46.916 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:06:46.928 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:46.927+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:46.931 INFO:teuthology.orchestra.run.smithi176.stdout:3696 write 0xdf44b06 thru 0xdf4dca8 (0x91a3 bytes) 2024-09-14T09:06:46.956 INFO:teuthology.orchestra.run.smithi176.stdout:3697 trunc from 0xdf4dca9 to 0x87fbdbd 2024-09-14T09:06:46.981 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:46.981 INFO:teuthology.orchestra.run.smithi176.stdout:3699 read 0x563adc6 thru 0x564752d (0xc768 bytes) 2024-09-14T09:06:46.992 INFO:teuthology.orchestra.run.smithi176.stdout:3700 write 0x8d4de0e thru 0x8d4e50e (0x701 bytes) 2024-09-14T09:06:47.019 INFO:teuthology.orchestra.run.smithi176.stdout:3702 write 0xc29f096 thru 0xc2a4c34 (0x5b9f bytes) 2024-09-14T09:06:47.028 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:47.028 INFO:teuthology.orchestra.run.smithi176.stdout:3705 punch from 0xa8f1de3 to 0xa8f98b3, (0x7ad0 bytes) 2024-09-14T09:06:47.028 INFO:teuthology.orchestra.run.smithi176.stdout:3706 write 0x3f563bd thru 0x3f653aa (0xefee bytes) 2024-09-14T09:06:47.028 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:47.027+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:47.057 INFO:teuthology.orchestra.run.smithi176.stdout:3708 write 0xe369836 thru 0xe3752fc (0xbac7 bytes) 2024-09-14T09:06:47.067 INFO:teuthology.orchestra.run.smithi176.stdout:3709 writesame 0xbaaf9b9 thru 0xbab828f data_size 0x1(0x88d7 bytes) 2024-09-14T09:06:47.097 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:47.097 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:47.113 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:06:47.129 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:47.127+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:47.203 INFO:teuthology.orchestra.run.smithi176.stdout:3710 trunc from 0xe3752fd to 0x5d8dd24 2024-09-14T09:06:47.228 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:47.230 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:47.231+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:47.237 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:47.237 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:47.251 INFO:teuthology.orchestra.run.smithi176.stdout:3712 writesame 0xa925533 thru 0xa933be6 data_size 0xa(0xe6b4 bytes) 2024-09-14T09:06:47.251 INFO:teuthology.orchestra.run.smithi176.stdout:3713 write 0xaeea28 thru 0xaf58b0 (0x6e89 bytes) 2024-09-14T09:06:47.252 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:06:47.281 INFO:teuthology.orchestra.run.smithi176.stdout:3714 trunc from 0xa933be7 to 0x5752e29 2024-09-14T09:06:47.293 INFO:teuthology.orchestra.run.smithi176.stdout:3715 read 0x1af6576 thru 0x1b00c77 (0xa702 bytes) 2024-09-14T09:06:47.316 INFO:teuthology.orchestra.run.smithi176.stdout:3717 write 0xa54c1ad thru 0xa552012 (0x5e66 bytes) 2024-09-14T09:06:47.330 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:47.331+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:47.349 INFO:teuthology.orchestra.run.smithi176.stdout:3718 writesame 0xe9f14d9 thru 0xe9fed5f data_size 0x9(0xd887 bytes) 2024-09-14T09:06:47.349 INFO:teuthology.orchestra.run.smithi176.stdout:3719 read 0x22ca75e thru 0x22d2e8b (0x872e bytes) 2024-09-14T09:06:47.349 INFO:teuthology.orchestra.run.smithi176.stdout:3720 writesame 0xabb0b7b thru 0xabb64bb data_size 0x1(0x5941 bytes) 2024-09-14T09:06:47.416 INFO:teuthology.orchestra.run.smithi176.stdout:3721 trunc from 0xe9fed60 to 0xb40f054 2024-09-14T09:06:47.424 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:47.424 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:47.431 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:47.431 INFO:teuthology.orchestra.run.smithi176.stdout:3723 writesame 0x820ad95 thru 0x8211008 data_size 0x2(0x6274 bytes) 2024-09-14T09:06:47.431 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:47.435+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:47.439 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:06:47.452 INFO:teuthology.orchestra.run.smithi176.stdout:3724 write 0x886527 thru 0x89576d (0xf247 bytes) 2024-09-14T09:06:47.477 INFO:teuthology.orchestra.run.smithi176.stdout:3725 punch from 0x56002f4 to 0x560f1fc, (0xef08 bytes) 2024-09-14T09:06:47.477 INFO:teuthology.orchestra.run.smithi176.stdout:3727 read 0x89a3a55 thru 0x89ab5c0 (0x7b6c bytes) 2024-09-14T09:06:47.477 INFO:teuthology.orchestra.run.smithi176.stdout:3728 punch from 0x747a2db to 0x7488843, (0xe568 bytes) 2024-09-14T09:06:47.477 INFO:teuthology.orchestra.run.smithi176.stdout:3729 write 0x9c00cd2 thru 0x9c0baba (0xade9 bytes) 2024-09-14T09:06:47.504 INFO:teuthology.orchestra.run.smithi176.stdout:3730 writesame 0xc5d9337 thru 0xc5e6ac7 data_size 0x41(0xd791 bytes) 2024-09-14T09:06:47.504 INFO:teuthology.orchestra.run.smithi176.stdout:3732 write 0x2512520 thru 0x25174f3 (0x4fd4 bytes) 2024-09-14T09:06:47.512 INFO:teuthology.orchestra.run.smithi176.stdout:3734 write 0x49f400a thru 0x49fc1ce (0x81c5 bytes) 2024-09-14T09:06:47.523 INFO:teuthology.orchestra.run.smithi176.stdout:3735 punch from 0x52d7957 to 0x52dfe58, (0x8501 bytes) 2024-09-14T09:06:47.523 INFO:teuthology.orchestra.run.smithi176.stdout:3736 writesame 0x6bd607a thru 0x6bdf210 data_size 0x2f(0x9197 bytes) 2024-09-14T09:06:47.533 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:47.531+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:47.534 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:47.534 INFO:teuthology.orchestra.run.smithi176.stdout:3740 clone 27 order 18 su 8192 sc 10 2024-09-14T09:06:47.610 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:47.610 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:47.616 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:06:47.623 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:06:47.666 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:47.667+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:47.734 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:47.735+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:47.746 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:47.747 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:47.762 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:06:47.770 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:47.770 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:47.787 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:47.787 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:47.805 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:47.805 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:47.821 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:47.821 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:47.835 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:47.839+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:47.840 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:47.841 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:47.859 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:47.859 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:47.876 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T09:06:47.879 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:47.880 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:47.890 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:47.890 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:47.903 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:47.903 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:47.905 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:06:47.951 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:47.955+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:47.970 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone26 from 0xc5e6ac8 (overlap 0x1f0f7df) to 0x0 2024-09-14T09:06:48.036 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:48.035+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:48.134 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:06:48.134 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:06:48.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:06:48.134 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:06:48.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.134 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(client.4446.0:19152 3.9 e39/34 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head v 39'12, mlcod=39'9)", 2024-09-14T09:06:48.134 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:44.333629+0000", 2024-09-14T09:06:48.134 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.802861120999999, 2024-09-14T09:06:48.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.1194095, 2024-09-14T09:06:48.134 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.135 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.135 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:48.135 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.333629+0000", 2024-09-14T09:06:48.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.333629+0000", 2024-09-14T09:06:48.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.333631+0000", 2024-09-14T09:06:48.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.007e-06 2024-09-14T09:06:48.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.350796+0000", 2024-09-14T09:06:48.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017164690999999999 2024-09-14T09:06:48.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.350865+0000", 2024-09-14T09:06:48.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8989e-05 2024-09-14T09:06:48.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.350884+0000", 2024-09-14T09:06:48.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9323000000000001e-05 2024-09-14T09:06:48.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.351045+0000", 2024-09-14T09:06:48.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016102900000000001 2024-09-14T09:06:48.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:48.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.351087+0000", 2024-09-14T09:06:48.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1430000000000001e-05 2024-09-14T09:06:48.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:48.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.452865+0000", 2024-09-14T09:06:48.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10177839900000001 2024-09-14T09:06:48.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:44.453039+0000", 2024-09-14T09:06:48.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017363200000000001 2024-09-14T09:06:48.138 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.138 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.139 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.139 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:37543 3.1b 3:d8e0f8d4:::rbd_data.115eb07a8abd.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:48.139 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.535280+0000", 2024-09-14T09:06:48.139 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.601210113, 2024-09-14T09:06:48.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.12853271799999999, 2024-09-14T09:06:48.139 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.139 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.139 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:48.139 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:48.139 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:48.139 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.139 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37543 2024-09-14T09:06:48.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.140 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535280+0000", 2024-09-14T09:06:48.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535280+0000", 2024-09-14T09:06:48.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535284+0000", 2024-09-14T09:06:48.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6770000000000001e-06 2024-09-14T09:06:48.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535303+0000", 2024-09-14T09:06:48.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9553000000000001e-05 2024-09-14T09:06:48.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535349+0000", 2024-09-14T09:06:48.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5833e-05 2024-09-14T09:06:48.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535368+0000", 2024-09-14T09:06:48.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8542999999999998e-05 2024-09-14T09:06:48.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535548+0000", 2024-09-14T09:06:48.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000180688 2024-09-14T09:06:48.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:48.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.535814+0000", 2024-09-14T09:06:48.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026614100000000001 2024-09-14T09:06:48.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:48.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.543779+0000", 2024-09-14T09:06:48.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0079644720000000002 2024-09-14T09:06:48.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:48.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663456+0000", 2024-09-14T09:06:48.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.119676694 2024-09-14T09:06:48.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:06:48.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663626+0000", 2024-09-14T09:06:48.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170559 2024-09-14T09:06:48.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:06:48.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663728+0000", 2024-09-14T09:06:48.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101889 2024-09-14T09:06:48.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663813+0000", 2024-09-14T09:06:48.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4668999999999999e-05 2024-09-14T09:06:48.144 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.144 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.145 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.145 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:37543 3.1b e48/34 ondisk, result = 0)", 2024-09-14T09:06:48.145 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:56.566755+0000", 2024-09-14T09:06:48.145 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.569735575000003, 2024-09-14T09:06:48.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.096718023, 2024-09-14T09:06:48.145 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.145 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.145 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:48.145 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566755+0000", 2024-09-14T09:06:48.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566755+0000", 2024-09-14T09:06:48.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566757+0000", 2024-09-14T09:06:48.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9800000000000001e-06 2024-09-14T09:06:48.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566767+0000", 2024-09-14T09:06:48.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0036e-05 2024-09-14T09:06:48.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566797+0000", 2024-09-14T09:06:48.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.012e-05 2024-09-14T09:06:48.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.566813+0000", 2024-09-14T09:06:48.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6575999999999999e-05 2024-09-14T09:06:48.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663408+0000", 2024-09-14T09:06:48.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.096594901999999996 2024-09-14T09:06:48.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:48.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663426+0000", 2024-09-14T09:06:48.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7765999999999999e-05 2024-09-14T09:06:48.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:56.663473+0000", 2024-09-14T09:06:48.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6643e-05 2024-09-14T09:06:48.149 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.149 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.149 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.149 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38593 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [watch reconnect cookie 139718844451680 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:06:48.149 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:06:48.149 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.318072706000002, 2024-09-14T09:06:48.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0218296630000001, 2024-09-14T09:06:48.149 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.149 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.149 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:48.150 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:48.150 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:48.150 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.150 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38593 2024-09-14T09:06:48.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.150 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:06:48.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:06:48.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818420+0000", 2024-09-14T09:06:48.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8399999999999999e-06 2024-09-14T09:06:48.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818431+0000", 2024-09-14T09:06:48.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0859999999999999e-05 2024-09-14T09:06:48.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818466+0000", 2024-09-14T09:06:48.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4486999999999997e-05 2024-09-14T09:06:48.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818481+0000", 2024-09-14T09:06:48.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5362999999999998e-05 2024-09-14T09:06:48.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818659+0000", 2024-09-14T09:06:48.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017807500000000001 2024-09-14T09:06:48.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op must wait for map", 2024-09-14T09:06:48.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818681+0000", 2024-09-14T09:06:48.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2096999999999999e-05 2024-09-14T09:06:48.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825472+0000", 2024-09-14T09:06:48.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067905170000000003 2024-09-14T09:06:48.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:06:48.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825491+0000", 2024-09-14T09:06:48.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9372999999999999e-05 2024-09-14T09:06:48.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835402+0000", 2024-09-14T09:06:48.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0099111620000001 2024-09-14T09:06:48.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:48.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835548+0000", 2024-09-14T09:06:48.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014607300000000001 2024-09-14T09:06:48.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.840247+0000", 2024-09-14T09:06:48.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046988159999999998 2024-09-14T09:06:48.155 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.155 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.155 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.155 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38615 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:06:48.155 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:06:48.155 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.269707498000002, 2024-09-14T09:06:48.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.97014128899999996, 2024-09-14T09:06:48.155 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.155 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.155 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:48.155 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:48.155 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:48.156 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.156 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38615 2024-09-14T09:06:48.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.156 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:06:48.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:06:48.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866787+0000", 2024-09-14T09:06:48.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.476e-06 2024-09-14T09:06:48.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866806+0000", 2024-09-14T09:06:48.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8606000000000001e-05 2024-09-14T09:06:48.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866839+0000", 2024-09-14T09:06:48.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3510000000000003e-05 2024-09-14T09:06:48.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866867+0000", 2024-09-14T09:06:48.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7610000000000002e-05 2024-09-14T09:06:48.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867031+0000", 2024-09-14T09:06:48.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016414199999999999 2024-09-14T09:06:48.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:06:48.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867054+0000", 2024-09-14T09:06:48.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2693000000000001e-05 2024-09-14T09:06:48.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836462+0000", 2024-09-14T09:06:48.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.96940797700000003 2024-09-14T09:06:48.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:48.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836547+0000", 2024-09-14T09:06:48.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5294e-05 2024-09-14T09:06:48.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836924+0000", 2024-09-14T09:06:48.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037698099999999997 2024-09-14T09:06:48.160 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.160 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.160 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.160 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49230 3.d 3:b4a67b64:::rbd_data.115e51d92ed2.0000000000000003:12 [read 2621440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.160 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.523859+0000", 2024-09-14T09:06:48.160 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.612630907000003, 2024-09-14T09:06:48.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067232675000000006, 2024-09-14T09:06:48.160 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.160 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.161 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:06:48.161 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:48.161 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:48.161 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.161 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49230 2024-09-14T09:06:48.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.161 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523859+0000", 2024-09-14T09:06:48.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523859+0000", 2024-09-14T09:06:48.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523860+0000", 2024-09-14T09:06:48.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0130000000000001e-06 2024-09-14T09:06:48.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523865+0000", 2024-09-14T09:06:48.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4000000000000002e-06 2024-09-14T09:06:48.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523880+0000", 2024-09-14T09:06:48.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5287000000000001e-05 2024-09-14T09:06:48.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523892+0000", 2024-09-14T09:06:48.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2006e-05 2024-09-14T09:06:48.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.590817+0000", 2024-09-14T09:06:48.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066924692999999993 2024-09-14T09:06:48.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:06:48.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.590937+0000", 2024-09-14T09:06:48.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012081700000000001 2024-09-14T09:06:48.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591092+0000", 2024-09-14T09:06:48.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015445899999999999 2024-09-14T09:06:48.164 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.164 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.165 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.165 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49231 3.d 3:b4a67b64:::rbd_data.115e51d92ed2.0000000000000003:12 [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.165 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.523908+0000", 2024-09-14T09:06:48.165 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.612582447999998, 2024-09-14T09:06:48.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067373731000000006, 2024-09-14T09:06:48.165 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.165 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.165 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:48.165 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:48.165 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:48.165 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.166 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49231 2024-09-14T09:06:48.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.166 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523908+0000", 2024-09-14T09:06:48.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523908+0000", 2024-09-14T09:06:48.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523909+0000", 2024-09-14T09:06:48.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9699999999999994e-07 2024-09-14T09:06:48.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523913+0000", 2024-09-14T09:06:48.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6070000000000001e-06 2024-09-14T09:06:48.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523929+0000", 2024-09-14T09:06:48.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5415999999999999e-05 2024-09-14T09:06:48.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523940+0000", 2024-09-14T09:06:48.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1707e-05 2024-09-14T09:06:48.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591167+0000", 2024-09-14T09:06:48.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067226576999999996 2024-09-14T09:06:48.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591281+0000", 2024-09-14T09:06:48.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011442699999999999 2024-09-14T09:06:48.169 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.169 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.169 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.169 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49232 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.169 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.523956+0000", 2024-09-14T09:06:48.169 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.612534113999999, 2024-09-14T09:06:48.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067398703000000004, 2024-09-14T09:06:48.169 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.169 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.169 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:48.169 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:48.170 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:48.170 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.170 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49232 2024-09-14T09:06:48.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.170 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523956+0000", 2024-09-14T09:06:48.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523956+0000", 2024-09-14T09:06:48.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523957+0000", 2024-09-14T09:06:48.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8299999999999995e-07 2024-09-14T09:06:48.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523961+0000", 2024-09-14T09:06:48.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3329999999999998e-06 2024-09-14T09:06:48.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523977+0000", 2024-09-14T09:06:48.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.577e-05 2024-09-14T09:06:48.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.523989+0000", 2024-09-14T09:06:48.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2089999999999999e-05 2024-09-14T09:06:48.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591340+0000", 2024-09-14T09:06:48.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067350278999999999 2024-09-14T09:06:48.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591355+0000", 2024-09-14T09:06:48.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5248e-05 2024-09-14T09:06:48.173 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.173 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.173 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.173 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49233 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.173 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.524004+0000", 2024-09-14T09:06:48.173 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.612486205000003, 2024-09-14T09:06:48.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067426573000000004, 2024-09-14T09:06:48.173 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.174 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.174 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:48.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:48.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:48.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.174 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49233 2024-09-14T09:06:48.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.174 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524004+0000", 2024-09-14T09:06:48.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524004+0000", 2024-09-14T09:06:48.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524005+0000", 2024-09-14T09:06:48.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3699999999999999e-07 2024-09-14T09:06:48.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524009+0000", 2024-09-14T09:06:48.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3359999999999997e-06 2024-09-14T09:06:48.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524026+0000", 2024-09-14T09:06:48.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6753e-05 2024-09-14T09:06:48.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524040+0000", 2024-09-14T09:06:48.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3694e-05 2024-09-14T09:06:48.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591411+0000", 2024-09-14T09:06:48.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067371767999999999 2024-09-14T09:06:48.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591431+0000", 2024-09-14T09:06:48.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9085000000000001e-05 2024-09-14T09:06:48.177 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.177 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.177 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.177 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49257 3.d 3.bb54482d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.177 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.524584+0000", 2024-09-14T09:06:48.177 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.611906302999998, 2024-09-14T09:06:48.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067046887999999999, 2024-09-14T09:06:48.178 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.178 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.178 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:48.178 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:48.178 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:48.178 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.178 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49257 2024-09-14T09:06:48.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.178 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524584+0000", 2024-09-14T09:06:48.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524584+0000", 2024-09-14T09:06:48.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524585+0000", 2024-09-14T09:06:48.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1699999999999997e-07 2024-09-14T09:06:48.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524609+0000", 2024-09-14T09:06:48.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4448999999999999e-05 2024-09-14T09:06:48.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524627+0000", 2024-09-14T09:06:48.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7543000000000001e-05 2024-09-14T09:06:48.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.524634+0000", 2024-09-14T09:06:48.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7540000000000006e-06 2024-09-14T09:06:48.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591622+0000", 2024-09-14T09:06:48.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066987668 2024-09-14T09:06:48.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.591631+0000", 2024-09-14T09:06:48.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7569999999999993e-06 2024-09-14T09:06:48.181 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.181 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.181 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.181 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49310 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.181 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525200+0000", 2024-09-14T09:06:48.181 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.611290007999997, 2024-09-14T09:06:48.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067082507, 2024-09-14T09:06:48.182 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.182 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.182 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:48.182 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:48.182 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:48.182 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.182 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49310 2024-09-14T09:06:48.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.182 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525200+0000", 2024-09-14T09:06:48.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525200+0000", 2024-09-14T09:06:48.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525201+0000", 2024-09-14T09:06:48.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.023e-06 2024-09-14T09:06:48.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525206+0000", 2024-09-14T09:06:48.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5000000000000001e-06 2024-09-14T09:06:48.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525242+0000", 2024-09-14T09:06:48.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6672999999999999e-05 2024-09-14T09:06:48.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525256+0000", 2024-09-14T09:06:48.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3360000000000001e-05 2024-09-14T09:06:48.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592266+0000", 2024-09-14T09:06:48.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067009868 2024-09-14T09:06:48.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592283+0000", 2024-09-14T09:06:48.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7082999999999999e-05 2024-09-14T09:06:48.185 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.185 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.185 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.186 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49311 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.186 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525275+0000", 2024-09-14T09:06:48.186 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.611215231999999, 2024-09-14T09:06:48.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067067112999999998, 2024-09-14T09:06:48.186 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.186 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.186 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:48.186 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:48.186 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:48.186 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.186 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49311 2024-09-14T09:06:48.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.186 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525275+0000", 2024-09-14T09:06:48.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525275+0000", 2024-09-14T09:06:48.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525276+0000", 2024-09-14T09:06:48.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.313e-06 2024-09-14T09:06:48.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525282+0000", 2024-09-14T09:06:48.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.127e-06 2024-09-14T09:06:48.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525299+0000", 2024-09-14T09:06:48.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6903000000000001e-05 2024-09-14T09:06:48.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525312+0000", 2024-09-14T09:06:48.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3193999999999999e-05 2024-09-14T09:06:48.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592323+0000", 2024-09-14T09:06:48.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067010504999999998 2024-09-14T09:06:48.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592342+0000", 2024-09-14T09:06:48.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9071000000000001e-05 2024-09-14T09:06:48.190 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.190 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.190 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.190 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49312 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.190 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525330+0000", 2024-09-14T09:06:48.190 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.611159839000003, 2024-09-14T09:06:48.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.06706268, 2024-09-14T09:06:48.190 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.190 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.190 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:48.190 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:48.190 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:48.191 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.191 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49312 2024-09-14T09:06:48.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.191 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525330+0000", 2024-09-14T09:06:48.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525330+0000", 2024-09-14T09:06:48.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525332+0000", 2024-09-14T09:06:48.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.187e-06 2024-09-14T09:06:48.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525350+0000", 2024-09-14T09:06:48.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8833000000000001e-05 2024-09-14T09:06:48.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525385+0000", 2024-09-14T09:06:48.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4686e-05 2024-09-14T09:06:48.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525399+0000", 2024-09-14T09:06:48.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4239999999999999e-05 2024-09-14T09:06:48.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592377+0000", 2024-09-14T09:06:48.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066977255999999999 2024-09-14T09:06:48.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592393+0000", 2024-09-14T09:06:48.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6478e-05 2024-09-14T09:06:48.194 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.194 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.194 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.194 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49313 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.194 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525423+0000", 2024-09-14T09:06:48.194 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.611067439999999, 2024-09-14T09:06:48.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067039319, 2024-09-14T09:06:48.195 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.195 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.195 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:48.195 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:48.195 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:48.195 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.195 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49313 2024-09-14T09:06:48.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.195 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525423+0000", 2024-09-14T09:06:48.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525423+0000", 2024-09-14T09:06:48.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525424+0000", 2024-09-14T09:06:48.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.257e-06 2024-09-14T09:06:48.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525430+0000", 2024-09-14T09:06:48.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8699999999999997e-06 2024-09-14T09:06:48.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525447+0000", 2024-09-14T09:06:48.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7560000000000001e-05 2024-09-14T09:06:48.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525460+0000", 2024-09-14T09:06:48.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2243e-05 2024-09-14T09:06:48.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592444+0000", 2024-09-14T09:06:48.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066984118999999995 2024-09-14T09:06:48.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592462+0000", 2024-09-14T09:06:48.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.827e-05 2024-09-14T09:06:48.198 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.198 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.198 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.199 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49314 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.199 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525478+0000", 2024-09-14T09:06:48.199 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.611011677, 2024-09-14T09:06:48.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.06702756, 2024-09-14T09:06:48.199 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.199 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.199 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:48.199 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:48.199 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:48.199 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.199 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49314 2024-09-14T09:06:48.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.200 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525478+0000", 2024-09-14T09:06:48.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525478+0000", 2024-09-14T09:06:48.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525480+0000", 2024-09-14T09:06:48.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.246e-06 2024-09-14T09:06:48.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525485+0000", 2024-09-14T09:06:48.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7440000000000004e-06 2024-09-14T09:06:48.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525504+0000", 2024-09-14T09:06:48.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8709999999999999e-05 2024-09-14T09:06:48.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525512+0000", 2024-09-14T09:06:48.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6260000000000003e-06 2024-09-14T09:06:48.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592491+0000", 2024-09-14T09:06:48.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.066979274000000005 2024-09-14T09:06:48.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592506+0000", 2024-09-14T09:06:48.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4960000000000001e-05 2024-09-14T09:06:48.202 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.203 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.203 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.203 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49315 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.203 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525523+0000", 2024-09-14T09:06:48.203 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.610966881000003, 2024-09-14T09:06:48.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067036894, 2024-09-14T09:06:48.203 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.203 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.203 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:48.203 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:48.203 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:48.203 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.204 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49315 2024-09-14T09:06:48.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.204 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525523+0000", 2024-09-14T09:06:48.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525523+0000", 2024-09-14T09:06:48.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525524+0000", 2024-09-14T09:06:48.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2999999999999999e-07 2024-09-14T09:06:48.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525528+0000", 2024-09-14T09:06:48.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4000000000000001e-06 2024-09-14T09:06:48.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525537+0000", 2024-09-14T09:06:48.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2739999999999996e-06 2024-09-14T09:06:48.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525543+0000", 2024-09-14T09:06:48.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6329999999999999e-06 2024-09-14T09:06:48.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592548+0000", 2024-09-14T09:06:48.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067004806 2024-09-14T09:06:48.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592560+0000", 2024-09-14T09:06:48.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1951e-05 2024-09-14T09:06:48.207 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.207 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.207 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.207 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49316 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.207 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525553+0000", 2024-09-14T09:06:48.207 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.610936987000002, 2024-09-14T09:06:48.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067056432999999999, 2024-09-14T09:06:48.207 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.207 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.207 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:48.207 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:48.207 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:48.208 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.208 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49316 2024-09-14T09:06:48.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.208 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525553+0000", 2024-09-14T09:06:48.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525553+0000", 2024-09-14T09:06:48.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525554+0000", 2024-09-14T09:06:48.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0299999999999998e-07 2024-09-14T09:06:48.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525557+0000", 2024-09-14T09:06:48.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9129999999999999e-06 2024-09-14T09:06:48.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525566+0000", 2024-09-14T09:06:48.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8000000000000004e-06 2024-09-14T09:06:48.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525572+0000", 2024-09-14T09:06:48.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.72e-06 2024-09-14T09:06:48.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592598+0000", 2024-09-14T09:06:48.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067025682000000003 2024-09-14T09:06:48.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592610+0000", 2024-09-14T09:06:48.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1515000000000001e-05 2024-09-14T09:06:48.211 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.211 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.211 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.211 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49317 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.211 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:48.211 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.610908528000003, 2024-09-14T09:06:48.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067082547000000006, 2024-09-14T09:06:48.211 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.211 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.211 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:48.212 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:48.212 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:48.212 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.212 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49317 2024-09-14T09:06:48.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.212 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:48.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:48.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525582+0000", 2024-09-14T09:06:48.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:06:48.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525585+0000", 2024-09-14T09:06:48.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8700000000000001e-06 2024-09-14T09:06:48.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525594+0000", 2024-09-14T09:06:48.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9740000000000008e-06 2024-09-14T09:06:48.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525601+0000", 2024-09-14T09:06:48.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9489999999999999e-06 2024-09-14T09:06:48.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592652+0000", 2024-09-14T09:06:48.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067050803000000006 2024-09-14T09:06:48.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592664+0000", 2024-09-14T09:06:48.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2198000000000001e-05 2024-09-14T09:06:48.215 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.215 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.215 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.215 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49318 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.215 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525642+0000", 2024-09-14T09:06:48.215 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.610848027999999, 2024-09-14T09:06:48.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067064055999999997, 2024-09-14T09:06:48.215 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.215 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.216 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:48.216 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:48.216 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:48.216 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.216 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49318 2024-09-14T09:06:48.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.216 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525642+0000", 2024-09-14T09:06:48.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525642+0000", 2024-09-14T09:06:48.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525644+0000", 2024-09-14T09:06:48.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0159999999999998e-06 2024-09-14T09:06:48.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525649+0000", 2024-09-14T09:06:48.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2739999999999998e-06 2024-09-14T09:06:48.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525666+0000", 2024-09-14T09:06:48.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6990000000000002e-05 2024-09-14T09:06:48.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525680+0000", 2024-09-14T09:06:48.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.331e-05 2024-09-14T09:06:48.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592697+0000", 2024-09-14T09:06:48.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067017367999999994 2024-09-14T09:06:48.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592706+0000", 2024-09-14T09:06:48.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.098e-06 2024-09-14T09:06:48.219 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.219 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.219 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.219 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:49319 3.d 3.26de652d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.219 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:10.525704+0000", 2024-09-14T09:06:48.219 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.610786621999999, 2024-09-14T09:06:48.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067049867999999999, 2024-09-14T09:06:48.219 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:06:48.219 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:06:48.220 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:06:48.220 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:06:48.220 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:06:48.220 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.220 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 49319 2024-09-14T09:06:48.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.220 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:06:48.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:06:48.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525704+0000", 2024-09-14T09:06:48.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:06:48.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525704+0000", 2024-09-14T09:06:48.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:06:48.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:06:48.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525705+0000", 2024-09-14T09:06:48.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.113e-06 2024-09-14T09:06:48.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:06:48.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525710+0000", 2024-09-14T09:06:48.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.843e-06 2024-09-14T09:06:48.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:06:48.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525726+0000", 2024-09-14T09:06:48.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.679e-05 2024-09-14T09:06:48.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.525739+0000", 2024-09-14T09:06:48.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2962999999999999e-05 2024-09-14T09:06:48.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:06:48.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592745+0000", 2024-09-14T09:06:48.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.067005341999999996 2024-09-14T09:06:48.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:06:48.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:06:48.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:06:48.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:10.592753+0000", 2024-09-14T09:06:48.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8170000000000005e-06 2024-09-14T09:06:48.223 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.223 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.223 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.223 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:06:48.223 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:06:48.223 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:06:48.224 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-14T09:06:48.225 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:48.227+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:48.225 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:06:48.325 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:48.327+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:48.356 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:48.356 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:48.356 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:48.356 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:48.380 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:06:48.426 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:48.427+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:48.504 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:06:48.507 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:48.507 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:48.507 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:48.507 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:48.507 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:48.523 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:06:48.527 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:48.531+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:48.628 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:48.631+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:48.633 INFO:teuthology.orchestra.run.smithi176.stdout:3741 trunc from 0xc5e6ac8 to 0x7da45a7 2024-09-14T09:06:48.666 INFO:teuthology.orchestra.run.smithi176.stdout:3742 write 0x7d28f25 thru 0x7d2fc93 (0x6d6f bytes) 2024-09-14T09:06:48.671 INFO:teuthology.orchestra.run.smithi176.stdout:3745 writesame 0xec4ce7 thru 0xec7a66 data_size 0x10(0x2d80 bytes) 2024-09-14T09:06:48.688 INFO:teuthology.orchestra.run.smithi176.stdout:3746 read 0x6653f9b thru 0x6660702 (0xc768 bytes) 2024-09-14T09:06:48.688 INFO:teuthology.orchestra.run.smithi176.stdout:3747 write 0x4756284 thru 0x47595bd (0x333a bytes) 2024-09-14T09:06:48.699 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20858 3.17 3:ed5052d1:::rbd_data.115e889d4809.000000000000015c:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:06:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.270100466000002, 2024-09-14T09:06:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053381794000000003, 2024-09-14T09:06:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:48.700 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20858 2024-09-14T09:06:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:06:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:06:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429351+0000", 2024-09-14T09:06:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1149999999999998e-06 2024-09-14T09:06:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429363+0000", 2024-09-14T09:06:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1819e-05 2024-09-14T09:06:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429389+0000", 2024-09-14T09:06:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5605e-05 2024-09-14T09:06:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429402+0000", 2024-09-14T09:06:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3175e-05 2024-09-14T09:06:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429492+0000", 2024-09-14T09:06:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.988e-05 2024-09-14T09:06:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429597+0000", 2024-09-14T09:06:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010564 2024-09-14T09:06:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.433465+0000", 2024-09-14T09:06:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0038674249999999999 2024-09-14T09:06:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482483+0000", 2024-09-14T09:06:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049017999999999999 2024-09-14T09:06:48.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482625+0000", 2024-09-14T09:06:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000142447 2024-09-14T09:06:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482684+0000", 2024-09-14T09:06:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8522e-05 2024-09-14T09:06:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482730+0000", 2024-09-14T09:06:48.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6165999999999998e-05 2024-09-14T09:06:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:20858 3.17 e41/23 ondisk, result = 0)", 2024-09-14T09:06:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:06:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.255341023, 2024-09-14T09:06:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038631402000000002, 2024-09-14T09:06:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:06:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:06:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444109+0000", 2024-09-14T09:06:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.331e-06 2024-09-14T09:06:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444114+0000", 2024-09-14T09:06:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3709999999999999e-06 2024-09-14T09:06:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444130+0000", 2024-09-14T09:06:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6079e-05 2024-09-14T09:06:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444144+0000", 2024-09-14T09:06:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3331e-05 2024-09-14T09:06:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482538+0000", 2024-09-14T09:06:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038394878 2024-09-14T09:06:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482548+0000", 2024-09-14T09:06:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0093e-05 2024-09-14T09:06:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482739+0000", 2024-09-14T09:06:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019031899999999999 2024-09-14T09:06:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.709 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:37800 3.1 3:830c4670:::rbd_data.115eb07a8abd.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:06:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.668893056000002, 2024-09-14T09:06:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.068040724999999996, 2024-09-14T09:06:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 37800 2024-09-14T09:06:48.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:06:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:06:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030557+0000", 2024-09-14T09:06:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.888e-06 2024-09-14T09:06:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030566+0000", 2024-09-14T09:06:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8540000000000003e-06 2024-09-14T09:06:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030585+0000", 2024-09-14T09:06:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8878000000000001e-05 2024-09-14T09:06:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030593+0000", 2024-09-14T09:06:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5870000000000004e-06 2024-09-14T09:06:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030667+0000", 2024-09-14T09:06:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4207000000000003e-05 2024-09-14T09:06:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030761+0000", 2024-09-14T09:06:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4223999999999998e-05 2024-09-14T09:06:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:06:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.034400+0000", 2024-09-14T09:06:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0036387759999999998 2024-09-14T09:06:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098457+0000", 2024-09-14T09:06:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.064057501000000003 2024-09-14T09:06:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098518+0000", 2024-09-14T09:06:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0730999999999998e-05 2024-09-14T09:06:48.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098560+0000", 2024-09-14T09:06:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2267e-05 2024-09-14T09:06:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098596+0000", 2024-09-14T09:06:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5812000000000002e-05 2024-09-14T09:06:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:37800 3.1 e48/34 ondisk, result = 0)", 2024-09-14T09:06:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:06:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.657901440000003, 2024-09-14T09:06:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057056260999999997, 2024-09-14T09:06:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:06:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:06:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041548+0000", 2024-09-14T09:06:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8100000000000002e-07 2024-09-14T09:06:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041552+0000", 2024-09-14T09:06:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7419999999999999e-06 2024-09-14T09:06:48.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041563+0000", 2024-09-14T09:06:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1926000000000001e-05 2024-09-14T09:06:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041572+0000", 2024-09-14T09:06:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8610000000000004e-06 2024-09-14T09:06:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098503+0000", 2024-09-14T09:06:48.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056930827000000003 2024-09-14T09:06:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098512+0000", 2024-09-14T09:06:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5019999999999992e-06 2024-09-14T09:06:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098603+0000", 2024-09-14T09:06:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1621999999999998e-05 2024-09-14T09:06:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.719 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38523 3.3 3:c62e0486:::rbd_data.115e9e133b92.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 4194304~28499] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T09:06:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:06:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 50.245473742000001, 2024-09-14T09:06:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035654406999999999, 2024-09-14T09:06:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38523 2024-09-14T09:06:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.720 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:06:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:06:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453977+0000", 2024-09-14T09:06:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6299999999999998e-06 2024-09-14T09:06:48.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455684+0000", 2024-09-14T09:06:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001706843 2024-09-14T09:06:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455710+0000", 2024-09-14T09:06:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6302e-05 2024-09-14T09:06:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455722+0000", 2024-09-14T09:06:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1798e-05 2024-09-14T09:06:48.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455825+0000", 2024-09-14T09:06:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102822 2024-09-14T09:06:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.456032+0000", 2024-09-14T09:06:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020704400000000001 2024-09-14T09:06:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:06:48.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.456759+0000", 2024-09-14T09:06:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00072735699999999998 2024-09-14T09:06:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.477856+0000", 2024-09-14T09:06:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021096011000000001 2024-09-14T09:06:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489526+0000", 2024-09-14T09:06:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011670313 2024-09-14T09:06:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:48.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489572+0000", 2024-09-14T09:06:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6408e-05 2024-09-14T09:06:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489629+0000", 2024-09-14T09:06:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6879000000000002e-05 2024-09-14T09:06:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49715 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.524874+0000", 2024-09-14T09:06:48.725 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 38.174574800000002, 2024-09-14T09:06:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034330422999999999, 2024-09-14T09:06:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49715 2024-09-14T09:06:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524874+0000", 2024-09-14T09:06:48.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524874+0000", 2024-09-14T09:06:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524875+0000", 2024-09-14T09:06:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1500000000000003e-07 2024-09-14T09:06:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524878+0000", 2024-09-14T09:06:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9090000000000002e-06 2024-09-14T09:06:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524889+0000", 2024-09-14T09:06:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.007e-05 2024-09-14T09:06:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524895+0000", 2024-09-14T09:06:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.285e-06 2024-09-14T09:06:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.558620+0000", 2024-09-14T09:06:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.033725009 2024-09-14T09:06:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.558681+0000", 2024-09-14T09:06:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1051999999999998e-05 2024-09-14T09:06:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559204+0000", 2024-09-14T09:06:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00052318299999999998 2024-09-14T09:06:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49716 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525151+0000", 2024-09-14T09:06:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 38.174297080999999, 2024-09-14T09:06:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034564820000000003, 2024-09-14T09:06:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.730 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49716 2024-09-14T09:06:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525151+0000", 2024-09-14T09:06:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525151+0000", 2024-09-14T09:06:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525152+0000", 2024-09-14T09:06:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5799999999999998e-07 2024-09-14T09:06:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525157+0000", 2024-09-14T09:06:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6380000000000003e-06 2024-09-14T09:06:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525167+0000", 2024-09-14T09:06:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0457e-05 2024-09-14T09:06:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525174+0000", 2024-09-14T09:06:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6220000000000003e-06 2024-09-14T09:06:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559237+0000", 2024-09-14T09:06:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034063361 2024-09-14T09:06:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559308+0000", 2024-09-14T09:06:48.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1124999999999994e-05 2024-09-14T09:06:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559716+0000", 2024-09-14T09:06:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000407759 2024-09-14T09:06:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49717 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525390+0000", 2024-09-14T09:06:48.734 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 38.174058283999997, 2024-09-14T09:06:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034552639000000003, 2024-09-14T09:06:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49717 2024-09-14T09:06:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525390+0000", 2024-09-14T09:06:48.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525390+0000", 2024-09-14T09:06:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525391+0000", 2024-09-14T09:06:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3099999999999997e-07 2024-09-14T09:06:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525395+0000", 2024-09-14T09:06:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.267e-06 2024-09-14T09:06:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525406+0000", 2024-09-14T09:06:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1224e-05 2024-09-14T09:06:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525413+0000", 2024-09-14T09:06:48.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5420000000000002e-06 2024-09-14T09:06:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559760+0000", 2024-09-14T09:06:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034347177999999999 2024-09-14T09:06:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559838+0000", 2024-09-14T09:06:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7708999999999997e-05 2024-09-14T09:06:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559943+0000", 2024-09-14T09:06:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010498800000000001 2024-09-14T09:06:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49718 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525514+0000", 2024-09-14T09:06:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 38.173934479000003, 2024-09-14T09:06:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034602766, 2024-09-14T09:06:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.739 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49718 2024-09-14T09:06:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525514+0000", 2024-09-14T09:06:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525514+0000", 2024-09-14T09:06:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525515+0000", 2024-09-14T09:06:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-07 2024-09-14T09:06:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525518+0000", 2024-09-14T09:06:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7220000000000001e-06 2024-09-14T09:06:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525528+0000", 2024-09-14T09:06:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4639999999999995e-06 2024-09-14T09:06:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525534+0000", 2024-09-14T09:06:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2330000000000003e-06 2024-09-14T09:06:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559974+0000", 2024-09-14T09:06:48.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034439866 2024-09-14T09:06:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560041+0000", 2024-09-14T09:06:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7343999999999998e-05 2024-09-14T09:06:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560117+0000", 2024-09-14T09:06:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5415999999999998e-05 2024-09-14T09:06:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.743 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49719 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:06:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 38.173815140999999, 2024-09-14T09:06:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034680563999999997, 2024-09-14T09:06:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49719 2024-09-14T09:06:48.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:06:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:06:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525634+0000", 2024-09-14T09:06:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4000000000000001e-07 2024-09-14T09:06:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525638+0000", 2024-09-14T09:06:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5899999999999999e-06 2024-09-14T09:06:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525647+0000", 2024-09-14T09:06:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2029999999999998e-06 2024-09-14T09:06:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525653+0000", 2024-09-14T09:06:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9850000000000003e-06 2024-09-14T09:06:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560150+0000", 2024-09-14T09:06:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034496845999999998 2024-09-14T09:06:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560221+0000", 2024-09-14T09:06:48.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1400000000000001e-05 2024-09-14T09:06:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560314+0000", 2024-09-14T09:06:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2800000000000006e-05 2024-09-14T09:06:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49720 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:06:48.748 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 38.173563516999998, 2024-09-14T09:06:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034919095999999997, 2024-09-14T09:06:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49720 2024-09-14T09:06:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:06:48.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:06:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525886+0000", 2024-09-14T09:06:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6000000000000002e-07 2024-09-14T09:06:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525890+0000", 2024-09-14T09:06:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4519999999999999e-06 2024-09-14T09:06:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525899+0000", 2024-09-14T09:06:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1090000000000004e-06 2024-09-14T09:06:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525906+0000", 2024-09-14T09:06:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0219999999999999e-06 2024-09-14T09:06:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560344+0000", 2024-09-14T09:06:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034437970999999998 2024-09-14T09:06:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560422+0000", 2024-09-14T09:06:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7857999999999999e-05 2024-09-14T09:06:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560804+0000", 2024-09-14T09:06:48.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00038182400000000001 2024-09-14T09:06:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49721 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:06:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 38.173516034999999, 2024-09-14T09:06:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035347380999999997, 2024-09-14T09:06:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:48.753 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49721 2024-09-14T09:06:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:06:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:06:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525933+0000", 2024-09-14T09:06:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T09:06:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525936+0000", 2024-09-14T09:06:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6759999999999999e-06 2024-09-14T09:06:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525943+0000", 2024-09-14T09:06:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2459999999999997e-06 2024-09-14T09:06:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525948+0000", 2024-09-14T09:06:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9429999999999999e-06 2024-09-14T09:06:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560845+0000", 2024-09-14T09:06:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034897354999999998 2024-09-14T09:06:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560925+0000", 2024-09-14T09:06:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9804999999999999e-05 2024-09-14T09:06:48.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561280+0000", 2024-09-14T09:06:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035480100000000002 2024-09-14T09:06:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49722 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3112960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 38.173492389000003, 2024-09-14T09:06:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035750491000000002, 2024-09-14T09:06:48.757 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.758 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.758 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:48.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:48.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:48.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.758 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49722 2024-09-14T09:06:48.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.758 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:48.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:48.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:48.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.96e-07 2024-09-14T09:06:48.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525958+0000", 2024-09-14T09:06:48.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.024e-06 2024-09-14T09:06:48.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525964+0000", 2024-09-14T09:06:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6699999999999999e-06 2024-09-14T09:06:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525969+0000", 2024-09-14T09:06:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3939999999999998e-06 2024-09-14T09:06:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561319+0000", 2024-09-14T09:06:48.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.03535013 2024-09-14T09:06:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561390+0000", 2024-09-14T09:06:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0920000000000005e-05 2024-09-14T09:06:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561707+0000", 2024-09-14T09:06:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031695699999999998 2024-09-14T09:06:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49723 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3506176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 38.173473405999999, 2024-09-14T09:06:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036113064, 2024-09-14T09:06:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49723 2024-09-14T09:06:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1699999999999999e-07 2024-09-14T09:06:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525977+0000", 2024-09-14T09:06:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9250000000000002e-06 2024-09-14T09:06:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525983+0000", 2024-09-14T09:06:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4620000000000004e-06 2024-09-14T09:06:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525987+0000", 2024-09-14T09:06:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.279e-06 2024-09-14T09:06:48.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561749+0000", 2024-09-14T09:06:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035762222000000003 2024-09-14T09:06:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561796+0000", 2024-09-14T09:06:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6959999999999998e-05 2024-09-14T09:06:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562088+0000", 2024-09-14T09:06:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029179899999999998 2024-09-14T09:06:48.765 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49724 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:06:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 38.173455250000004, 2024-09-14T09:06:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036449388999999999, 2024-09-14T09:06:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.766 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49724 2024-09-14T09:06:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:06:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:06:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525994+0000", 2024-09-14T09:06:48.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3099999999999998e-07 2024-09-14T09:06:48.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525996+0000", 2024-09-14T09:06:48.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.939e-06 2024-09-14T09:06:48.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526005+0000", 2024-09-14T09:06:48.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8430000000000005e-06 2024-09-14T09:06:48.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526010+0000", 2024-09-14T09:06:48.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9389999999999996e-06 2024-09-14T09:06:48.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562114+0000", 2024-09-14T09:06:48.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036103510999999998 2024-09-14T09:06:48.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562189+0000", 2024-09-14T09:06:48.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5162999999999998e-05 2024-09-14T09:06:48.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562443+0000", 2024-09-14T09:06:48.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000253563 2024-09-14T09:06:48.770 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.770 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.770 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.770 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:64935 3.12 3:4cd6d7cb:::rbd_data.115e21c31fcf.0000000000000049:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:06:48.770 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:06:48.770 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.948089902, 2024-09-14T09:06:48.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037353866999999999, 2024-09-14T09:06:48.770 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.770 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.770 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:48.771 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:48.771 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:48.771 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.771 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64935 2024-09-14T09:06:48.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.771 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:06:48.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:06:48.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751362+0000", 2024-09-14T09:06:48.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6899999999999998e-06 2024-09-14T09:06:48.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751378+0000", 2024-09-14T09:06:48.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6147e-05 2024-09-14T09:06:48.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751421+0000", 2024-09-14T09:06:48.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2691999999999997e-05 2024-09-14T09:06:48.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751444+0000", 2024-09-14T09:06:48.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3237000000000001e-05 2024-09-14T09:06:48.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751594+0000", 2024-09-14T09:06:48.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014947800000000001 2024-09-14T09:06:48.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751733+0000", 2024-09-14T09:06:48.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013945200000000001 2024-09-14T09:06:48.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:48.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.753315+0000", 2024-09-14T09:06:48.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0015815969999999999 2024-09-14T09:06:48.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:48.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788546+0000", 2024-09-14T09:06:48.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035231449999999997 2024-09-14T09:06:48.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:48.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788570+0000", 2024-09-14T09:06:48.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3991000000000001e-05 2024-09-14T09:06:48.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:48.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788645+0000", 2024-09-14T09:06:48.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4379000000000001e-05 2024-09-14T09:06:48.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788712+0000", 2024-09-14T09:06:48.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7754000000000002e-05 2024-09-14T09:06:48.776 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.776 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.776 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.776 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:71052 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:06:48.776 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:06:48.776 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.961880651000001, 2024-09-14T09:06:48.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14854635599999999, 2024-09-14T09:06:48.776 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.776 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.776 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:48.777 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:48.777 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:48.777 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.777 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 71052 2024-09-14T09:06:48.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.777 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:06:48.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:06:48.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737570+0000", 2024-09-14T09:06:48.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4770000000000002e-06 2024-09-14T09:06:48.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737584+0000", 2024-09-14T09:06:48.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3457e-05 2024-09-14T09:06:48.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737608+0000", 2024-09-14T09:06:48.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3893999999999999e-05 2024-09-14T09:06:48.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737620+0000", 2024-09-14T09:06:48.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2157000000000001e-05 2024-09-14T09:06:48.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737738+0000", 2024-09-14T09:06:48.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118409 2024-09-14T09:06:48.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737892+0000", 2024-09-14T09:06:48.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154222 2024-09-14T09:06:48.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:06:48.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.749622+0000", 2024-09-14T09:06:48.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011729537 2024-09-14T09:06:48.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:48.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885934+0000", 2024-09-14T09:06:48.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13631222200000001 2024-09-14T09:06:48.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:48.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885996+0000", 2024-09-14T09:06:48.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1555000000000003e-05 2024-09-14T09:06:48.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:48.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886060+0000", 2024-09-14T09:06:48.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3980999999999996e-05 2024-09-14T09:06:48.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886114+0000", 2024-09-14T09:06:48.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4444999999999998e-05 2024-09-14T09:06:48.782 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.782 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.782 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.782 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:71052 3.1 e71/34 ondisk, result = 0)", 2024-09-14T09:06:48.782 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:06:48.782 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.918977272999999, 2024-09-14T09:06:48.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10565334799999999, 2024-09-14T09:06:48.783 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.783 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.783 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:48.783 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:06:48.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:06:48.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780473+0000", 2024-09-14T09:06:48.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8980000000000001e-06 2024-09-14T09:06:48.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780482+0000", 2024-09-14T09:06:48.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9080000000000001e-06 2024-09-14T09:06:48.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780511+0000", 2024-09-14T09:06:48.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9490999999999999e-05 2024-09-14T09:06:48.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780527+0000", 2024-09-14T09:06:48.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.508e-05 2024-09-14T09:06:48.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885978+0000", 2024-09-14T09:06:48.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.105451328 2024-09-14T09:06:48.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885989+0000", 2024-09-14T09:06:48.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0875e-05 2024-09-14T09:06:48.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886125+0000", 2024-09-14T09:06:48.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013576800000000001 2024-09-14T09:06:48.786 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.786 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.786 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.787 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73871 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:1b [read 8339456~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:48.787 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:06:48.787 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 13.273492265, 2024-09-14T09:06:48.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034605047, 2024-09-14T09:06:48.787 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.787 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.787 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:48.787 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:48.787 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:48.787 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.787 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73871 2024-09-14T09:06:48.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.787 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:06:48.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:06:48.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425957+0000", 2024-09-14T09:06:48.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T09:06:48.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425961+0000", 2024-09-14T09:06:48.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8279999999999996e-06 2024-09-14T09:06:48.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425985+0000", 2024-09-14T09:06:48.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4182e-05 2024-09-14T09:06:48.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425997+0000", 2024-09-14T09:06:48.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.132e-05 2024-09-14T09:06:48.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460070+0000", 2024-09-14T09:06:48.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034072913000000003 2024-09-14T09:06:48.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460298+0000", 2024-09-14T09:06:48.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022819 2024-09-14T09:06:48.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460561+0000", 2024-09-14T09:06:48.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026341400000000001 2024-09-14T09:06:48.791 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.791 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.791 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.792 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:74276 3.12 3:49b683dd:::rbd_data.115ea8d131ff.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1376256~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:48.792 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:06:48.792 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 13.014345175000001, 2024-09-14T09:06:48.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036482939999999998, 2024-09-14T09:06:48.792 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.792 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.792 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:48.792 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:48.792 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:48.792 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.793 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74276 2024-09-14T09:06:48.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.793 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:06:48.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:06:48.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685105+0000", 2024-09-14T09:06:48.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9520000000000001e-06 2024-09-14T09:06:48.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686530+0000", 2024-09-14T09:06:48.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001424629 2024-09-14T09:06:48.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686550+0000", 2024-09-14T09:06:48.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0472e-05 2024-09-14T09:06:48.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686561+0000", 2024-09-14T09:06:48.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T09:06:48.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686671+0000", 2024-09-14T09:06:48.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110621 2024-09-14T09:06:48.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686919+0000", 2024-09-14T09:06:48.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024790199999999999 2024-09-14T09:06:48.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:48.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.687375+0000", 2024-09-14T09:06:48.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00045572100000000002 2024-09-14T09:06:48.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:48.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.710871+0000", 2024-09-14T09:06:48.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023496481999999999 2024-09-14T09:06:48.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:48.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721478+0000", 2024-09-14T09:06:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01060709 2024-09-14T09:06:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721534+0000", 2024-09-14T09:06:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5912999999999998e-05 2024-09-14T09:06:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721586+0000", 2024-09-14T09:06:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1792000000000002e-05 2024-09-14T09:06:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.797 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.798 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.798 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.798 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:48.798 INFO:teuthology.orchestra.run.smithi176.stdout:3749 trunc from 0x7da45a7 to 0x5ed77f 2024-09-14T09:06:48.799 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:06:48.800 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:48.800 INFO:teuthology.orchestra.run.smithi176.stdout:3753 write 0x5ec1fce thru 0x5ecf84b (0xd87e bytes) 2024-09-14T09:06:48.800 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:48.799+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:48.818 INFO:teuthology.orchestra.run.smithi176.stdout:3754 trunc from 0x5ecf84c to 0xe104d03 2024-09-14T09:06:48.826 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:48.839 INFO:teuthology.orchestra.run.smithi176.stdout:3756 trunc from 0xe104d03 to 0x381a79a 2024-09-14T09:06:48.852 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:06:48.853 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:06:48.857 INFO:teuthology.orchestra.run.smithi176.stdout:3757 read 0x2017932 thru 0x202191d (0x9fec bytes) 2024-09-14T09:06:48.873 INFO:teuthology.orchestra.run.smithi176.stdout:3760 write 0x9f8960c thru 0x9f91771 (0x8166 bytes) 2024-09-14T09:06:48.878 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-14T09:06:47.984168+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1045368526,"num_objects":1227,"num_object_clones":367,"num_object_copies":2454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1227,"num_whiteouts":440,"num_read":104448,"num_read_kb":546595,"num_write":13061,"num_write_kb":223909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1216,"num_bytes_recovered":72785688,"num_keys_recovered":252,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10248,"ondisk_log_size":10248,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":349540,"kb_used_data":44532,"kb_used_omap":12,"kb_used_meta":304819,"kb_avail":754625180,"statfs":{"total":773094113280,"available":772736184320,"internally_reserved":0,"allocated":45600768,"data_stored":324876064,"data_compressed":9529226,"data_compressed_allocated":26660864,"data_compressed_original":314194383,"omap_allocated":12751,"internal_metadata":312135217},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8570887,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1811,"num_read_kb":7201,"num_write":401,"num_write_kb":2689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":734,"num_bytes_recovered":49767154,"num_keys_recovered":186,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000806"},"pg_stats":[{"pgid":"3.1f","version":"87'1344","reported_seq":4285,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:47.223591+0000","last_change":"2024-09-14T09:06:41.348384+0000","last_active":"2024-09-14T09:06:47.223591+0000","last_peered":"2024-09-14T09:06:47.223591+0000","last_clean":"2024-09-14T09:06:47.223591+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:06:47.223591+0000","last_undegraded":"2024-09-14T09:06:47.223591+0000","last_fullsized":"2024-09-14T09:06:47.223591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1344,"log_dups_size":0,"ondisk_log_size":1344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:40:01.002873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001387512,"stat_sum":{"num_bytes":17479966,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":15,"num_read":9351,"num_read_kb":14328,"num_write":2320,"num_write_kb":5034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"85'201","reported_seq":2424,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.854797+0000","last_change":"2024-09-14T09:05:29.039503+0000","last_active":"2024-09-14T09:06:45.854797+0000","last_peered":"2024-09-14T09:06:45.854797+0000","last_clean":"2024-09-14T09:06:45.854797+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:06:45.854797+0000","last_undegraded":"2024-09-14T09:06:45.854797+0000","last_fullsized":"2024-09-14T09:06:45.854797+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:25:36.908116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001657176,"stat_sum":{"num_bytes":32516728,"num_objects":53,"num_object_clones":18,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":20,"num_read":2154,"num_read_kb":22916,"num_write":239,"num_write_kb":9189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"87'367","reported_seq":3745,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.873849+0000","last_change":"2024-09-14T09:05:28.835298+0000","last_active":"2024-09-14T09:06:46.873849+0000","last_peered":"2024-09-14T09:06:46.873849+0000","last_clean":"2024-09-14T09:06:46.873849+0000","last_became_active":"2024-09-14T09:05:26.835372+0000","last_became_peered":"2024-09-14T09:05:26.835372+0000","last_unstale":"2024-09-14T09:06:46.873849+0000","last_undegraded":"2024-09-14T09:06:46.873849+0000","last_fullsized":"2024-09-14T09:06:46.873849+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:38.418901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014149329999999999,"stat_sum":{"num_bytes":58835428,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":10,"num_read":3995,"num_read_kb":52947,"num_write":512,"num_write_kb":8922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"83'525","reported_seq":3197,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.874459+0000","last_change":"2024-09-14T09:06:44.867658+0000","last_active":"2024-09-14T09:06:46.874459+0000","last_peered":"2024-09-14T09:06:46.874459+0000","last_clean":"2024-09-14T09:06:46.874459+0000","last_became_active":"2024-09-14T09:06:44.862945+0000","last_became_peered":"2024-09-14T09:06:44.862945+0000","last_unstale":"2024-09-14T09:06:46.874459+0000","last_undegraded":"2024-09-14T09:06:46.874459+0000","last_fullsized":"2024-09-14T09:06:46.874459+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:13:41.872396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014661419999999999,"stat_sum":{"num_bytes":38979686,"num_objects":37,"num_object_clones":12,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":5424,"num_read_kb":22099,"num_write":742,"num_write_kb":6156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"79'205","reported_seq":2402,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.875685+0000","last_change":"2024-09-14T09:06:41.347478+0000","last_active":"2024-09-14T09:06:46.875685+0000","last_peered":"2024-09-14T09:06:46.875685+0000","last_clean":"2024-09-14T09:06:46.875685+0000","last_became_active":"2024-09-14T09:06:41.346397+0000","last_became_peered":"2024-09-14T09:06:41.346397+0000","last_unstale":"2024-09-14T09:06:46.875685+0000","last_undegraded":"2024-09-14T09:06:46.875685+0000","last_fullsized":"2024-09-14T09:06:46.875685+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:30:10.950021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060373399999999998,"stat_sum":{"num_bytes":30069643,"num_objects":40,"num_object_clones":16,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":1935,"num_read_kb":6316,"num_write":210,"num_write_kb":11506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":6073202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"85'484","reported_seq":3433,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.985628+0000","last_change":"2024-09-14T09:06:44.872988+0000","last_active":"2024-09-14T09:06:45.985628+0000","last_peered":"2024-09-14T09:06:45.985628+0000","last_clean":"2024-09-14T09:06:45.985628+0000","last_became_active":"2024-09-14T09:06:44.869103+0000","last_became_peered":"2024-09-14T09:06:44.869103+0000","last_unstale":"2024-09-14T09:06:45.985628+0000","last_undegraded":"2024-09-14T09:06:45.985628+0000","last_fullsized":"2024-09-14T09:06:45.985628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:50:16.711214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020845550000000001,"stat_sum":{"num_bytes":34465908,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":4445,"num_read_kb":18966,"num_write":769,"num_write_kb":10088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"83'209","reported_seq":2089,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.885547+0000","last_change":"2024-09-14T09:05:28.822565+0000","last_active":"2024-09-14T09:06:45.885547+0000","last_peered":"2024-09-14T09:06:45.885547+0000","last_clean":"2024-09-14T09:06:45.885547+0000","last_became_active":"2024-09-14T09:05:26.816132+0000","last_became_peered":"2024-09-14T09:05:26.816132+0000","last_unstale":"2024-09-14T09:06:45.885547+0000","last_undegraded":"2024-09-14T09:06:45.885547+0000","last_fullsized":"2024-09-14T09:06:45.885547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:01:39.934279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092420299999999996,"stat_sum":{"num_bytes":22175026,"num_objects":33,"num_object_clones":11,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":12,"num_read":1799,"num_read_kb":6256,"num_write":248,"num_write_kb":4631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"83'285","reported_seq":3392,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.854365+0000","last_change":"2024-09-14T09:06:41.342724+0000","last_active":"2024-09-14T09:06:45.854365+0000","last_peered":"2024-09-14T09:06:45.854365+0000","last_clean":"2024-09-14T09:06:45.854365+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:06:45.854365+0000","last_undegraded":"2024-09-14T09:06:45.854365+0000","last_fullsized":"2024-09-14T09:06:45.854365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:31:28.047943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081579899999999997,"stat_sum":{"num_bytes":50357810,"num_objects":48,"num_object_clones":17,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":14,"num_read":3701,"num_read_kb":8060,"num_write":275,"num_write_kb":3949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"83'357","reported_seq":4359,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.876315+0000","last_change":"2024-09-14T09:05:28.839928+0000","last_active":"2024-09-14T09:06:46.876315+0000","last_peered":"2024-09-14T09:06:46.876315+0000","last_clean":"2024-09-14T09:06:46.876315+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:06:46.876315+0000","last_undegraded":"2024-09-14T09:06:46.876315+0000","last_fullsized":"2024-09-14T09:06:46.876315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:36:52.750689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014343089999999999,"stat_sum":{"num_bytes":37014757,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":4962,"num_read_kb":22170,"num_write":440,"num_write_kb":4862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"86'377","reported_seq":1535,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.873865+0000","last_change":"2024-09-14T09:05:28.836370+0000","last_active":"2024-09-14T09:06:46.873865+0000","last_peered":"2024-09-14T09:06:46.873865+0000","last_clean":"2024-09-14T09:06:46.873865+0000","last_became_active":"2024-09-14T09:05:26.836744+0000","last_became_peered":"2024-09-14T09:05:26.836744+0000","last_unstale":"2024-09-14T09:06:46.873865+0000","last_undegraded":"2024-09-14T09:06:46.873865+0000","last_fullsized":"2024-09-14T09:06:46.873865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:11:06.446635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015277190000000001,"stat_sum":{"num_bytes":15800887,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":9,"num_read":3095,"num_read_kb":7897,"num_write":507,"num_write_kb":3559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"83'514","reported_seq":1650,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.860602+0000","last_change":"2024-09-14T09:05:28.851587+0000","last_active":"2024-09-14T09:06:45.860602+0000","last_peered":"2024-09-14T09:06:45.860602+0000","last_clean":"2024-09-14T09:06:45.860602+0000","last_became_active":"2024-09-14T09:05:26.840893+0000","last_became_peered":"2024-09-14T09:05:26.840893+0000","last_unstale":"2024-09-14T09:06:45.860602+0000","last_undegraded":"2024-09-14T09:06:45.860602+0000","last_fullsized":"2024-09-14T09:06:45.860602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:35:19.280259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098594000000000004,"stat_sum":{"num_bytes":9058215,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":13,"num_read":3911,"num_read_kb":3852,"num_write":780,"num_write_kb":2759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"85'326","reported_seq":1723,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.871881+0000","last_change":"2024-09-14T09:06:44.867267+0000","last_active":"2024-09-14T09:06:45.871881+0000","last_peered":"2024-09-14T09:06:45.871881+0000","last_clean":"2024-09-14T09:06:45.871881+0000","last_became_active":"2024-09-14T09:06:44.865413+0000","last_became_peered":"2024-09-14T09:06:44.865413+0000","last_unstale":"2024-09-14T09:06:45.871881+0000","last_undegraded":"2024-09-14T09:06:45.871881+0000","last_fullsized":"2024-09-14T09:06:45.871881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:19:03.069884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014079679999999999,"stat_sum":{"num_bytes":20928767,"num_objects":42,"num_object_clones":13,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":16,"num_read":2271,"num_read_kb":3974,"num_write":374,"num_write_kb":902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":253148,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":150,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:47.006915+0000","last_change":"2024-09-14T09:05:19.793674+0000","last_active":"2024-09-14T09:06:47.006915+0000","last_peered":"2024-09-14T09:06:47.006915+0000","last_clean":"2024-09-14T09:06:47.006915+0000","last_became_active":"2024-09-14T09:05:17.284977+0000","last_became_peered":"2024-09-14T09:05:17.284977+0000","last_unstale":"2024-09-14T09:06:47.006915+0000","last_undegraded":"2024-09-14T09:06:47.006915+0000","last_fullsized":"2024-09-14T09:06:47.006915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:30:38.832027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"87'227","reported_seq":3533,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.914300+0000","last_change":"2024-09-14T09:05:28.839614+0000","last_active":"2024-09-14T09:06:46.914300+0000","last_peered":"2024-09-14T09:06:46.914300+0000","last_clean":"2024-09-14T09:06:46.914300+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:06:46.914300+0000","last_undegraded":"2024-09-14T09:06:46.914300+0000","last_fullsized":"2024-09-14T09:06:46.914300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:30:20.294407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014245869999999999,"stat_sum":{"num_bytes":40394524,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":4246,"num_read_kb":38806,"num_write":246,"num_write_kb":6221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":148,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.890630+0000","last_change":"2024-09-14T09:05:19.314029+0000","last_active":"2024-09-14T09:06:45.890630+0000","last_peered":"2024-09-14T09:06:45.890630+0000","last_clean":"2024-09-14T09:06:45.890630+0000","last_became_active":"2024-09-14T09:05:17.282890+0000","last_became_peered":"2024-09-14T09:05:17.282890+0000","last_unstale":"2024-09-14T09:06:45.890630+0000","last_undegraded":"2024-09-14T09:06:45.890630+0000","last_fullsized":"2024-09-14T09:06:45.890630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:25.304105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"85'362","reported_seq":2363,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.854777+0000","last_change":"2024-09-14T09:05:29.039154+0000","last_active":"2024-09-14T09:06:45.854777+0000","last_peered":"2024-09-14T09:06:45.854777+0000","last_clean":"2024-09-14T09:06:45.854777+0000","last_became_active":"2024-09-14T09:05:26.829855+0000","last_became_peered":"2024-09-14T09:05:26.829855+0000","last_unstale":"2024-09-14T09:06:45.854777+0000","last_undegraded":"2024-09-14T09:06:45.854777+0000","last_fullsized":"2024-09-14T09:06:45.854777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:12:56.716058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019213749999999999,"stat_sum":{"num_bytes":37553508,"num_objects":29,"num_object_clones":9,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":9,"num_read":3271,"num_read_kb":12689,"num_write":472,"num_write_kb":3916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":148,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.860869+0000","last_change":"2024-09-14T09:05:19.794165+0000","last_active":"2024-09-14T09:06:45.860869+0000","last_peered":"2024-09-14T09:06:45.860869+0000","last_clean":"2024-09-14T09:06:45.860869+0000","last_became_active":"2024-09-14T09:05:17.295864+0000","last_became_peered":"2024-09-14T09:05:17.295864+0000","last_unstale":"2024-09-14T09:06:45.860869+0000","last_undegraded":"2024-09-14T09:06:45.860869+0000","last_fullsized":"2024-09-14T09:06:45.860869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:53.901865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011399719999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"83'270","reported_seq":2708,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.885402+0000","last_change":"2024-09-14T09:06:35.286509+0000","last_active":"2024-09-14T09:06:45.885402+0000","last_peered":"2024-09-14T09:06:45.885402+0000","last_clean":"2024-09-14T09:06:45.885402+0000","last_became_active":"2024-09-14T09:06:35.282469+0000","last_became_peered":"2024-09-14T09:06:35.282469+0000","last_unstale":"2024-09-14T09:06:45.885402+0000","last_undegraded":"2024-09-14T09:06:45.885402+0000","last_fullsized":"2024-09-14T09:06:45.885402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:56:24.711903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014087959999999999,"stat_sum":{"num_bytes":20460999,"num_objects":44,"num_object_clones":15,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":18,"num_read":3301,"num_read_kb":15228,"num_write":277,"num_write_kb":5460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":1142719,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.885604+0000","last_change":"2024-09-14T09:05:19.314129+0000","last_active":"2024-09-14T09:06:45.885604+0000","last_peered":"2024-09-14T09:06:45.885604+0000","last_clean":"2024-09-14T09:06:45.885604+0000","last_became_active":"2024-09-14T09:05:17.295196+0000","last_became_peered":"2024-09-14T09:05:17.295196+0000","last_unstale":"2024-09-14T09:06:45.885604+0000","last_undegraded":"2024-09-14T09:06:45.885604+0000","last_fullsized":"2024-09-14T09:06:45.885604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:35.269115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011613800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"83'168","reported_seq":871,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.873896+0000","last_change":"2024-09-14T09:05:28.835976+0000","last_active":"2024-09-14T09:06:46.873896+0000","last_peered":"2024-09-14T09:06:46.873896+0000","last_clean":"2024-09-14T09:06:46.873896+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:06:46.873896+0000","last_undegraded":"2024-09-14T09:06:46.873896+0000","last_fullsized":"2024-09-14T09:06:46.873896+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:21:06.765695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017480779999999999,"stat_sum":{"num_bytes":52867963,"num_objects":45,"num_object_clones":16,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":21,"num_read":596,"num_read_kb":9794,"num_write":155,"num_write_kb":9028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"83'358","reported_seq":1901,"reported_epoch":86,"state":"creating+peering","last_fresh":"2024-09-14T09:06:44.949645+0000","last_change":"2024-09-14T09:06:46.149268+0000","last_active":"2024-09-14T09:06:46.149268+0000","last_peered":"2024-09-14T09:06:44.949645+0000","last_clean":"2024-09-14T09:06:40.888318+0000","last_became_active":"2024-09-14T09:06:42.894671+0000","last_became_peered":"2024-09-14T09:06:42.894671+0000","last_unstale":"2024-09-14T09:06:44.949645+0000","last_undegraded":"2024-09-14T09:06:44.949645+0000","last_fullsized":"2024-09-14T09:06:42.847670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014128,"stat_sum":{"num_bytes":25020722,"num_objects":35,"num_object_clones":12,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":2744,"num_read_kb":15841,"num_write":439,"num_write_kb":8915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":152,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.873713+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:06:46.873713+0000","last_peered":"2024-09-14T09:06:46.873713+0000","last_clean":"2024-09-14T09:06:46.873713+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:06:46.873713+0000","last_undegraded":"2024-09-14T09:06:46.873713+0000","last_fullsized":"2024-09-14T09:06:46.873713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"83'173","reported_seq":975,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.885476+0000","last_change":"2024-09-14T09:05:28.822791+0000","last_active":"2024-09-14T09:06:45.885476+0000","last_peered":"2024-09-14T09:06:45.885476+0000","last_clean":"2024-09-14T09:06:45.885476+0000","last_became_active":"2024-09-14T09:05:26.817109+0000","last_became_peered":"2024-09-14T09:05:26.817109+0000","last_unstale":"2024-09-14T09:06:45.885476+0000","last_undegraded":"2024-09-14T09:06:45.885476+0000","last_fullsized":"2024-09-14T09:06:45.885476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:08:41.364341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011728439999999999,"stat_sum":{"num_bytes":14678273,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":849,"num_read_kb":3701,"num_write":184,"num_write_kb":1405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":150,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.870865+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:06:46.870865+0000","last_peered":"2024-09-14T09:06:46.870865+0000","last_clean":"2024-09-14T09:06:46.870865+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:06:46.870865+0000","last_undegraded":"2024-09-14T09:06:46.870865+0000","last_fullsized":"2024-09-14T09:06:46.870865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"87'205","reported_seq":1968,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.875774+0000","last_change":"2024-09-14T09:06:45.868939+0000","last_active":"2024-09-14T09:06:46.875774+0000","last_peered":"2024-09-14T09:06:46.875774+0000","last_clean":"2024-09-14T09:06:46.875774+0000","last_became_active":"2024-09-14T09:06:45.868194+0000","last_became_peered":"2024-09-14T09:06:45.868194+0000","last_unstale":"2024-09-14T09:06:46.875774+0000","last_undegraded":"2024-09-14T09:06:46.875774+0000","last_fullsized":"2024-09-14T09:06:46.875774+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:06:02.859882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074815999999999999,"stat_sum":{"num_bytes":29354455,"num_objects":31,"num_object_clones":11,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":1490,"num_read_kb":15691,"num_write":207,"num_write_kb":9103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":148,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.859972+0000","last_change":"2024-09-14T09:05:19.793999+0000","last_active":"2024-09-14T09:06:45.859972+0000","last_peered":"2024-09-14T09:06:45.859972+0000","last_clean":"2024-09-14T09:06:45.859972+0000","last_became_active":"2024-09-14T09:05:17.284122+0000","last_became_peered":"2024-09-14T09:05:17.284122+0000","last_unstale":"2024-09-14T09:06:45.859972+0000","last_undegraded":"2024-09-14T09:06:45.859972+0000","last_fullsized":"2024-09-14T09:06:45.859972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:12:01.133215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010234269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"68'195","reported_seq":1121,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.854815+0000","last_change":"2024-09-14T09:05:29.038389+0000","last_active":"2024-09-14T09:06:45.854815+0000","last_peered":"2024-09-14T09:06:45.854815+0000","last_clean":"2024-09-14T09:06:45.854815+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:06:45.854815+0000","last_undegraded":"2024-09-14T09:06:45.854815+0000","last_fullsized":"2024-09-14T09:06:45.854815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:42.547444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001479076,"stat_sum":{"num_bytes":19439154,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":1071,"num_read_kb":4172,"num_write":238,"num_write_kb":2073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.854623+0000","last_change":"2024-09-14T09:06:25.227644+0000","last_active":"2024-09-14T09:06:45.854623+0000","last_peered":"2024-09-14T09:06:45.854623+0000","last_clean":"2024-09-14T09:06:45.854623+0000","last_became_active":"2024-09-14T09:06:25.222344+0000","last_became_peered":"2024-09-14T09:06:25.222344+0000","last_unstale":"2024-09-14T09:06:45.854623+0000","last_undegraded":"2024-09-14T09:06:45.854623+0000","last_fullsized":"2024-09-14T09:06:45.854623+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:23:33.414859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":239,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.860270+0000","last_change":"2024-09-14T09:05:13.239808+0000","last_active":"2024-09-14T09:06:45.860270+0000","last_peered":"2024-09-14T09:06:45.860270+0000","last_clean":"2024-09-14T09:06:45.860270+0000","last_became_active":"2024-09-14T09:05:13.239064+0000","last_became_peered":"2024-09-14T09:05:13.239064+0000","last_unstale":"2024-09-14T09:06:45.860270+0000","last_undegraded":"2024-09-14T09:06:45.860270+0000","last_fullsized":"2024-09-14T09:06:45.860270+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:40:11.559161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"83'239","reported_seq":3304,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.854385+0000","last_change":"2024-09-14T09:05:29.037923+0000","last_active":"2024-09-14T09:06:45.854385+0000","last_peered":"2024-09-14T09:06:45.854385+0000","last_clean":"2024-09-14T09:06:45.854385+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:06:45.854385+0000","last_undegraded":"2024-09-14T09:06:45.854385+0000","last_fullsized":"2024-09-14T09:06:45.854385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:30:17.623019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099199800000000001,"stat_sum":{"num_bytes":42354761,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":3489,"num_read_kb":29604,"num_write":257,"num_write_kb":7537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"83'195","reported_seq":1588,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.870685+0000","last_change":"2024-09-14T09:06:44.868332+0000","last_active":"2024-09-14T09:06:45.870685+0000","last_peered":"2024-09-14T09:06:45.870685+0000","last_clean":"2024-09-14T09:06:45.870685+0000","last_became_active":"2024-09-14T09:06:44.866346+0000","last_became_peered":"2024-09-14T09:06:44.866346+0000","last_unstale":"2024-09-14T09:06:45.870685+0000","last_undegraded":"2024-09-14T09:06:45.870685+0000","last_fullsized":"2024-09-14T09:06:45.870685+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:47:10.301740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001266028,"stat_sum":{"num_bytes":27628229,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":1136,"num_read_kb":6881,"num_write":177,"num_write_kb":14095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":4589732,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"83'262","reported_seq":3293,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:47.006902+0000","last_change":"2024-09-14T09:06:45.870752+0000","last_active":"2024-09-14T09:06:47.006902+0000","last_peered":"2024-09-14T09:06:47.006902+0000","last_clean":"2024-09-14T09:06:47.006902+0000","last_became_active":"2024-09-14T09:06:45.869266+0000","last_became_peered":"2024-09-14T09:06:45.869266+0000","last_unstale":"2024-09-14T09:06:47.006902+0000","last_undegraded":"2024-09-14T09:06:47.006902+0000","last_fullsized":"2024-09-14T09:06:47.006902+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:44:26.704179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185731,"stat_sum":{"num_bytes":48761384,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":3347,"num_read_kb":31896,"num_write":294,"num_write_kb":16339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"79'303","reported_seq":7931,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.854973+0000","last_change":"2024-09-14T09:06:02.837388+0000","last_active":"2024-09-14T09:06:45.854973+0000","last_peered":"2024-09-14T09:06:45.854973+0000","last_clean":"2024-09-14T09:06:45.854973+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:06:45.854973+0000","last_undegraded":"2024-09-14T09:06:45.854973+0000","last_fullsized":"2024-09-14T09:06:45.854973+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:55:06.321684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001585295,"stat_sum":{"num_bytes":39029359,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":8803,"num_read_kb":53060,"num_write":301,"num_write_kb":7483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"83'209","reported_seq":1185,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.873838+0000","last_change":"2024-09-14T09:05:28.834647+0000","last_active":"2024-09-14T09:06:46.873838+0000","last_peered":"2024-09-14T09:06:46.873838+0000","last_clean":"2024-09-14T09:06:46.873838+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:06:46.873838+0000","last_undegraded":"2024-09-14T09:06:46.873838+0000","last_fullsized":"2024-09-14T09:06:46.873838+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:29:56.253705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244290000000001,"stat_sum":{"num_bytes":26698314,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":1353,"num_read_kb":7206,"num_write":222,"num_write_kb":7172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"86'238","reported_seq":3024,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.983415+0000","last_change":"2024-09-14T09:05:28.851561+0000","last_active":"2024-09-14T09:06:45.983415+0000","last_peered":"2024-09-14T09:06:45.983415+0000","last_clean":"2024-09-14T09:06:45.983415+0000","last_became_active":"2024-09-14T09:05:26.843780+0000","last_became_peered":"2024-09-14T09:05:26.843780+0000","last_unstale":"2024-09-14T09:06:45.983415+0000","last_undegraded":"2024-09-14T09:06:45.983415+0000","last_fullsized":"2024-09-14T09:06:45.983415+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:55:09.169516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072987200000000001,"stat_sum":{"num_bytes":28500713,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":3240,"num_read_kb":21867,"num_write":294,"num_write_kb":13667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"83'169","reported_seq":1399,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.987002+0000","last_change":"2024-09-14T09:06:45.872735+0000","last_active":"2024-09-14T09:06:45.987002+0000","last_peered":"2024-09-14T09:06:45.987002+0000","last_clean":"2024-09-14T09:06:45.987002+0000","last_became_active":"2024-09-14T09:06:45.868989+0000","last_became_peered":"2024-09-14T09:06:45.868989+0000","last_unstale":"2024-09-14T09:06:45.987002+0000","last_undegraded":"2024-09-14T09:06:45.987002+0000","last_fullsized":"2024-09-14T09:06:45.987002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:06:36.272971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057596100000000005,"stat_sum":{"num_bytes":35589399,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":13,"num_read":938,"num_read_kb":9274,"num_write":155,"num_write_kb":4467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":3897102,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"83'209","reported_seq":1206,"reported_epoch":85,"state":"peering","last_fresh":"2024-09-14T09:06:45.209516+0000","last_change":"2024-09-14T09:06:45.861860+0000","last_active":"2024-09-14T09:06:45.861860+0000","last_peered":"2024-09-14T09:06:45.209516+0000","last_clean":"2024-09-14T09:06:40.344351+0000","last_became_active":"2024-09-14T09:06:42.890786+0000","last_became_peered":"2024-09-14T09:06:42.890786+0000","last_unstale":"2024-09-14T09:06:45.209516+0000","last_undegraded":"2024-09-14T09:06:45.209516+0000","last_fullsized":"2024-09-14T09:06:42.850359+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000909463,"stat_sum":{"num_bytes":29699048,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":14,"num_read":1105,"num_read_kb":3779,"num_write":194,"num_write_kb":7632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":3341023,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"86'231","reported_seq":4069,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:47.006835+0000","last_change":"2024-09-14T09:06:00.166357+0000","last_active":"2024-09-14T09:06:47.006835+0000","last_peered":"2024-09-14T09:06:47.006835+0000","last_clean":"2024-09-14T09:06:47.006835+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:06:47.006835+0000","last_undegraded":"2024-09-14T09:06:47.006835+0000","last_fullsized":"2024-09-14T09:06:47.006835+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:58:28.963004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011381609999999999,"stat_sum":{"num_bytes":86083375,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":17,"num_read":3692,"num_read_kb":43423,"num_write":289,"num_write_kb":15097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"85'318","reported_seq":2643,"reported_epoch":85,"state":"peering","last_fresh":"2024-09-14T09:06:45.361251+0000","last_change":"2024-09-14T09:06:45.862138+0000","last_active":"2024-09-14T09:06:45.862138+0000","last_peered":"2024-09-14T09:06:45.361251+0000","last_clean":"2024-09-14T09:06:40.343697+0000","last_became_active":"2024-09-14T09:06:42.896330+0000","last_became_peered":"2024-09-14T09:06:42.896330+0000","last_unstale":"2024-09-14T09:06:45.361251+0000","last_undegraded":"2024-09-14T09:06:45.361251+0000","last_fullsized":"2024-09-14T09:06:42.848361+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092949299999999999,"stat_sum":{"num_bytes":5340400,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":13,"num_read":3514,"num_read_kb":3930,"num_write":426,"num_write_kb":737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":30867,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"87'370","reported_seq":2594,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.875813+0000","last_change":"2024-09-14T09:05:28.839774+0000","last_active":"2024-09-14T09:06:46.875813+0000","last_peered":"2024-09-14T09:06:46.875813+0000","last_clean":"2024-09-14T09:06:46.875813+0000","last_became_active":"2024-09-14T09:05:26.826600+0000","last_became_peered":"2024-09-14T09:05:26.826600+0000","last_unstale":"2024-09-14T09:06:46.875813+0000","last_undegraded":"2024-09-14T09:06:46.875813+0000","last_fullsized":"2024-09-14T09:06:46.875813+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:37:44.244140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018357099999999999,"stat_sum":{"num_bytes":44463041,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":17,"num_read":3668,"num_read_kb":3611,"num_write":463,"num_write_kb":3314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"79'278","reported_seq":4529,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.860081+0000","last_change":"2024-09-14T09:06:44.869874+0000","last_active":"2024-09-14T09:06:45.860081+0000","last_peered":"2024-09-14T09:06:45.860081+0000","last_clean":"2024-09-14T09:06:45.860081+0000","last_became_active":"2024-09-14T09:06:44.865881+0000","last_became_peered":"2024-09-14T09:06:44.865881+0000","last_unstale":"2024-09-14T09:06:45.860081+0000","last_undegraded":"2024-09-14T09:06:45.860081+0000","last_fullsized":"2024-09-14T09:06:45.860081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:42:49.191314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061978900000000004,"stat_sum":{"num_bytes":23177697,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":5486,"num_read_kb":26306,"num_write":252,"num_write_kb":7061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":6761444,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1044778139,"num_objects":1223,"num_object_clones":367,"num_object_copies":2446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1223,"num_whiteouts":440,"num_read":104382,"num_read_kb":546540,"num_write":12958,"num_write_kb":222279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1215,"num_bytes_recovered":72785688,"num_keys_recovered":252,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":34783232,"data_stored":296834935,"data_compressed":8893997,"data_compressed_allocated":24612864,"data_compressed_original":290295055,"omap_allocated":0,"internal_metadata":0},"log_size":10173,"ondisk_log_size":10173,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12424,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39656,"kb_used_data":4308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332184,"statfs":{"total":96636764160,"available":96596156416,"internally_reserved":0,"allocated":4411392,"data_stored":31752236,"data_compressed":825018,"data_compressed_allocated":2605056,"data_compressed_original":30800251,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50876,"kb_used_data":5804,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94320964,"statfs":{"total":96636764160,"available":96584667136,"internally_reserved":0,"allocated":5943296,"data_stored":41867259,"data_compressed":1136476,"data_compressed_allocated":3399680,"data_compressed_original":40472428,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509464,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41264,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330576,"statfs":{"total":96636764160,"available":96594509824,"internally_reserved":0,"allocated":1789952,"data_stored":6039141,"data_compressed":257074,"data_compressed_allocated":589824,"data_compressed_original":5468960,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509466,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46440,"kb_used_data":6932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325400,"statfs":{"total":96636764160,"available":96589209600,"internally_reserved":0,"allocated":7098368,"data_stored":53889428,"data_compressed":1551176,"data_compressed_allocated":4263936,"data_compressed_original":52295459,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542166,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":5524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":5656576,"data_stored":38413556,"data_compressed":904624,"data_compressed_allocated":3022848,"data_compressed_original":36788332,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542167,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47768,"kb_used_data":8264,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94324072,"statfs":{"total":96636764160,"available":96587849728,"internally_reserved":0,"allocated":8462336,"data_stored":66103371,"data_compressed":2043905,"data_compressed_allocated":5562368,"data_compressed_original":64406792,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542167,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45656,"kb_used_data":6140,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326184,"statfs":{"total":96636764160,"available":96590012416,"internally_reserved":0,"allocated":6287360,"data_stored":48338724,"data_compressed":1364959,"data_compressed_allocated":3653632,"data_compressed_original":46831386,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542168,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45320,"kb_used_data":5812,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326520,"statfs":{"total":96636764160,"available":96590356480,"internally_reserved":0,"allocated":5951488,"data_stored":38472349,"data_compressed":1445994,"data_compressed_allocated":3563520,"data_compressed_original":37130775,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5632000,"data_stored":49786473,"data_compressed":1463694,"data_compressed_allocated":3952640,"data_compressed_original":48752419,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3145728,"data_stored":23886926,"data_compressed":928721,"data_compressed_allocated":2195456,"data_compressed_original":23266647,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":770048,"data_stored":5573138,"data_compressed":257074,"data_compressed_allocated":589824,"data_compressed_original":5468960,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4837376,"data_stored":41823211,"data_compressed":1318925,"data_compressed_allocated":3293184,"data_compressed_original":40847130,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7467008,"data_stored":65649133,"data_compressed":2043905,"data_compressed_allocated":5562368,"data_compressed_original":64406792,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4947968,"data_stored":41466789,"data_compressed":1158248,"data_compressed_allocated":3428352,"data_compressed_original":40537964,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3391488,"data_stored":31286233,"data_compressed":825018,"data_compressed_allocated":2605056,"data_compressed_original":30800251,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4591616,"data_stored":37363032,"data_compressed":898412,"data_compressed_allocated":2985984,"data_compressed_original":36214892,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:06:48.881 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-14T09:06:48.886 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is set 2024-09-14T09:06:48.894 INFO:teuthology.orchestra.run.smithi176.stdout:3761 punch from 0x9d8c02f to 0x9d96806, (0xa7d7 bytes) 2024-09-14T09:06:48.894 INFO:teuthology.orchestra.run.smithi176.stdout:3762 write 0x4d741c3 thru 0x4d76ed2 (0x2d10 bytes) 2024-09-14T09:06:48.899 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:48.899+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:48.901 INFO:teuthology.orchestra.run.smithi176.stdout:3763 read 0x85506c1 thru 0x855f767 (0xf0a7 bytes) 2024-09-14T09:06:48.901 INFO:teuthology.orchestra.run.smithi176.stdout:3765 writesame 0x686aeed thru 0x6873302 data_size 0x3a(0x8416 bytes) 2024-09-14T09:06:48.914 INFO:teuthology.orchestra.run.smithi176.stdout:3766 read 0x1cc8d62 thru 0x1ccc5a6 (0x3845 bytes) 2024-09-14T09:06:48.914 INFO:teuthology.orchestra.run.smithi176.stdout:3767 read 0xfdc2de thru 0xfe9073 (0xcd96 bytes) 2024-09-14T09:06:48.927 INFO:teuthology.orchestra.run.smithi176.stdout:3769 trunc from 0x9f91772 to 0x1094bf2 2024-09-14T09:06:48.950 INFO:teuthology.orchestra.run.smithi176.stdout:3770 read 0xcfd20e thru 0xcfe6a2 (0x1495 bytes) 2024-09-14T09:06:48.969 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:48.969 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:48.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.969 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:79709 3.4 3:22052275:::rbd_header.115ee10e61a5:head [call rbd.set_size in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-14T09:06:48.969 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:48.969819+0000", 2024-09-14T09:06:48.969 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.0010961359999999999, 2024-09-14T09:06:48.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0011145700000000001, 2024-09-14T09:06:48.970 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:48.970 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:48.970 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "waiting for sub ops", 2024-09-14T09:06:48.970 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:48.970 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:48.970 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:48.970 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 79709 2024-09-14T09:06:48.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.970 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:48.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:48.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:48.969819+0000", 2024-09-14T09:06:48.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:48.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:48.969819+0000", 2024-09-14T09:06:48.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:48.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:48.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:48.969821+0000", 2024-09-14T09:06:48.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2450000000000001e-06 2024-09-14T09:06:48.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:48.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:48.969828+0000", 2024-09-14T09:06:48.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2049999999999996e-06 2024-09-14T09:06:48.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:48.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:48.969846+0000", 2024-09-14T09:06:48.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8082999999999999e-05 2024-09-14T09:06:48.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:48.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:48.969854+0000", 2024-09-14T09:06:48.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7130000000000005e-06 2024-09-14T09:06:48.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:48.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:48.969926+0000", 2024-09-14T09:06:48.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1935999999999997e-05 2024-09-14T09:06:48.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:48.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:48.970030+0000", 2024-09-14T09:06:48.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010447500000000001 2024-09-14T09:06:48.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:48.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:48.970252+0000", 2024-09-14T09:06:48.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022143099999999999 2024-09-14T09:06:48.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:48.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:48.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:48.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:48.970878+0000", 2024-09-14T09:06:48.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062587299999999999 2024-09-14T09:06:48.974 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.974 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:48.974 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.974 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:48.974 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:06:48.974 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 1 2024-09-14T09:06:48.974 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:48.975 INFO:teuthology.orchestra.run.smithi176.stdout:3771 write 0xbe1f823 thru 0xbe2866a (0x8e48 bytes) 2024-09-14T09:06:48.987 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:06:48.988 INFO:teuthology.orchestra.run.smithi176.stdout:3772 punch from 0x96513e1 to 0x9659c67, (0x8886 bytes) 2024-09-14T09:06:48.988 INFO:teuthology.orchestra.run.smithi176.stdout:3773 writesame 0x3fe7d1e thru 0x3ff21e3 data_size 0x2e(0xa4c6 bytes) 2024-09-14T09:06:49.000 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:49.003+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:49.005 INFO:teuthology.orchestra.run.smithi176.stdout:3775 punch from 0x8bf596a to 0x8bfd000, (0x7696 bytes) 2024-09-14T09:06:49.005 INFO:teuthology.orchestra.run.smithi176.stdout:3776 write 0x5c03a23 thru 0x5c0e634 (0xac12 bytes) 2024-09-14T09:06:49.018 INFO:teuthology.orchestra.run.smithi176.stdout:3777 writesame 0x5bdb13 thru 0x5c3e33 data_size 0xb(0x6321 bytes) 2024-09-14T09:06:49.021 INFO:teuthology.orchestra.run.smithi176.stdout:3778 write 0x571e6ce thru 0x5724303 (0x5c36 bytes) 2024-09-14T09:06:49.034 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:49.035 INFO:teuthology.orchestra.run.smithi176.stdout:3780 write 0xb6b2d60 thru 0xb6c05a0 (0xd841 bytes) 2024-09-14T09:06:49.060 INFO:teuthology.orchestra.run.smithi176.stdout:3782 writesame 0xbecb54a thru 0xbed7548 data_size 0x1(0xbfff bytes) 2024-09-14T09:06:49.063 INFO:teuthology.orchestra.run.smithi176.stdout:3783 punch from 0x515e97 to 0x522d80, (0xcee9 bytes) 2024-09-14T09:06:49.063 INFO:teuthology.orchestra.run.smithi176.stdout:3784 punch from 0x95f47d3 to 0x95f4d79, (0x5a6 bytes) 2024-09-14T09:06:49.064 INFO:teuthology.orchestra.run.smithi176.stdout:3787 read 0x2d65b74 thru 0x2d75388 (0xf815 bytes) 2024-09-14T09:06:49.064 INFO:teuthology.orchestra.run.smithi176.stdout:3788 read 0x3da15de thru 0x3da2dbc (0x17df bytes) 2024-09-14T09:06:49.064 INFO:teuthology.orchestra.run.smithi176.stdout:3789 read 0xac1ac55 thru 0xac28a52 (0xddfe bytes) 2024-09-14T09:06:49.075 INFO:teuthology.orchestra.run.smithi176.stdout:3790 trunc from 0xbed7549 to 0x288de2a 2024-09-14T09:06:49.101 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:49.099+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:49.104 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:49.116 INFO:teuthology.orchestra.run.smithi176.stdout:3793 write 0x5d064d4 thru 0x5d0fd50 (0x987d bytes) 2024-09-14T09:06:49.123 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:49.123 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:49.124 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:49.124 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:49.124 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:49.142 INFO:teuthology.orchestra.run.smithi176.stdout:3794 write 0xaf6dc3d thru 0xaf7d781 (0xfb45 bytes) 2024-09-14T09:06:49.146 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:06:49.156 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:49.156 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:49.156 INFO:teuthology.orchestra.run.smithi176.stdout:3798 writesame 0x916f707 thru 0x917a245 data_size 0x3(0xab3f bytes) 2024-09-14T09:06:49.169 INFO:teuthology.orchestra.run.smithi176.stdout:3799 writesame 0x6b4f2a7 thru 0x6b520a5 data_size 0x19(0x2dff bytes) 2024-09-14T09:06:49.178 INFO:teuthology.orchestra.run.smithi176.stdout:3800 write 0x6e9fea2 thru 0x6ea6f34 (0x7093 bytes) 2024-09-14T09:06:49.189 INFO:teuthology.orchestra.run.smithi176.stdout:3801 punch from 0xa40c17 to 0xa42fac, (0x2395 bytes) 2024-09-14T09:06:49.190 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:49.198 INFO:teuthology.orchestra.run.smithi176.stdout:3803 write 0xc0cec3e thru 0xc0d6663 (0x7a26 bytes) 2024-09-14T09:06:49.202 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:49.203+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:49.218 INFO:teuthology.orchestra.run.smithi176.stdout:3804 write 0xe163623 thru 0xe166116 (0x2af4 bytes) 2024-09-14T09:06:49.225 INFO:teuthology.orchestra.run.smithi176.stdout:3805 read 0xa63f478 thru 0xa64a449 (0xafd2 bytes) 2024-09-14T09:06:49.225 INFO:teuthology.orchestra.run.smithi176.stdout:3806 read 0x13c8023 thru 0x13ca025 (0x2003 bytes) 2024-09-14T09:06:49.225 INFO:teuthology.orchestra.run.smithi176.stdout:3808 read 0xd537a3c thru 0xd53ceb0 (0x5475 bytes) 2024-09-14T09:06:49.239 INFO:teuthology.orchestra.run.smithi176.stdout:3809 trunc from 0xe166117 to 0x38960c2 2024-09-14T09:06:49.267 INFO:teuthology.orchestra.run.smithi176.stdout:3810 writesame 0x1b706e9 thru 0x1b71c44 data_size 0x2(0x155c bytes) 2024-09-14T09:06:49.272 INFO:teuthology.orchestra.run.smithi176.stdout:3813 writesame 0x1779fb6 thru 0x177c7f6 data_size 0xf(0x2841 bytes) 2024-09-14T09:06:49.284 INFO:teuthology.orchestra.run.smithi176.stdout:3814 trunc from 0x38960c2 to 0x9d6468 2024-09-14T09:06:49.290 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:49.290 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:49.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:49.290 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:49.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73540 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15155200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:06:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.021851243, 2024-09-14T09:06:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.022930347, 2024-09-14T09:06:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73540 2024-09-14T09:06:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.291 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:06:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:06:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268665+0000", 2024-09-14T09:06:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6599999999999996e-07 2024-09-14T09:06:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268668+0000", 2024-09-14T09:06:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8760000000000001e-06 2024-09-14T09:06:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268676+0000", 2024-09-14T09:06:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7879999999999995e-06 2024-09-14T09:06:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268683+0000", 2024-09-14T09:06:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3180000000000003e-06 2024-09-14T09:06:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282017+0000", 2024-09-14T09:06:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01333409 2024-09-14T09:06:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282020+0000", 2024-09-14T09:06:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1609999999999999e-06 2024-09-14T09:06:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291158+0000", 2024-09-14T09:06:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0091376430000001 2024-09-14T09:06:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291383+0000", 2024-09-14T09:06:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022491000000000001 2024-09-14T09:06:49.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291595+0000", 2024-09-14T09:06:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021199500000000001 2024-09-14T09:06:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73541 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15237120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:06:49.296 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.021823823, 2024-09-14T09:06:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0232607330000001, 2024-09-14T09:06:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73541 2024-09-14T09:06:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:06:49.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:06:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:06:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5400000000000001e-07 2024-09-14T09:06:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268695+0000", 2024-09-14T09:06:49.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5799999999999999e-06 2024-09-14T09:06:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268703+0000", 2024-09-14T09:06:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5480000000000002e-06 2024-09-14T09:06:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268713+0000", 2024-09-14T09:06:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8640000000000008e-06 2024-09-14T09:06:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282032+0000", 2024-09-14T09:06:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013318831999999999 2024-09-14T09:06:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282035+0000", 2024-09-14T09:06:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4010000000000001e-06 2024-09-14T09:06:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291701+0000", 2024-09-14T09:06:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0096654199999999 2024-09-14T09:06:49.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291807+0000", 2024-09-14T09:06:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105852 2024-09-14T09:06:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291952+0000", 2024-09-14T09:06:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014568200000000001 2024-09-14T09:06:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73542 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15319040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:06:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.021754211999999, 2024-09-14T09:06:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0237091359999999, 2024-09-14T09:06:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73542 2024-09-14T09:06:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.302 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:06:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:06:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268762+0000", 2024-09-14T09:06:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:06:49.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268766+0000", 2024-09-14T09:06:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5880000000000002e-06 2024-09-14T09:06:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268776+0000", 2024-09-14T09:06:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0168e-05 2024-09-14T09:06:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268789+0000", 2024-09-14T09:06:49.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:06:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282053+0000", 2024-09-14T09:06:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013264496000000001 2024-09-14T09:06:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282057+0000", 2024-09-14T09:06:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6090000000000001e-06 2024-09-14T09:06:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292052+0000", 2024-09-14T09:06:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0099950070000001 2024-09-14T09:06:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292262+0000", 2024-09-14T09:06:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021009800000000001 2024-09-14T09:06:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292470+0000", 2024-09-14T09:06:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000208314 2024-09-14T09:06:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.306 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73543 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:06:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.021715422, 2024-09-14T09:06:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023985473, 2024-09-14T09:06:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73543 2024-09-14T09:06:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:06:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:06:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.308 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268801+0000", 2024-09-14T09:06:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:06:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268803+0000", 2024-09-14T09:06:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7439999999999999e-06 2024-09-14T09:06:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268812+0000", 2024-09-14T09:06:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7080000000000003e-06 2024-09-14T09:06:49.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268819+0000", 2024-09-14T09:06:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T09:06:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282068+0000", 2024-09-14T09:06:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013249540000000001 2024-09-14T09:06:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282072+0000", 2024-09-14T09:06:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0960000000000001e-06 2024-09-14T09:06:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292573+0000", 2024-09-14T09:06:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0105014560000001 2024-09-14T09:06:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292679+0000", 2024-09-14T09:06:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010545899999999999 2024-09-14T09:06:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292786+0000", 2024-09-14T09:06:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107052 2024-09-14T09:06:49.311 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73544 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:06:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.021687719999999, 2024-09-14T09:06:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0244830549999999, 2024-09-14T09:06:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.312 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73544 2024-09-14T09:06:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:06:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:06:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:06:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5599999999999995e-07 2024-09-14T09:06:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268831+0000", 2024-09-14T09:06:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.57e-06 2024-09-14T09:06:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268844+0000", 2024-09-14T09:06:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2811e-05 2024-09-14T09:06:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.314 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268851+0000", 2024-09-14T09:06:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9140000000000002e-06 2024-09-14T09:06:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282083+0000", 2024-09-14T09:06:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013232111 2024-09-14T09:06:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282086+0000", 2024-09-14T09:06:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2909999999999999e-06 2024-09-14T09:06:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292884+0000", 2024-09-14T09:06:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0107975069999999 2024-09-14T09:06:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293100+0000", 2024-09-14T09:06:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021639399999999999 2024-09-14T09:06:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293311+0000", 2024-09-14T09:06:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021090099999999999 2024-09-14T09:06:49.316 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73545 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15564800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:06:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.021636535000001, 2024-09-14T09:06:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0249746740000001, 2024-09-14T09:06:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.317 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73545 2024-09-14T09:06:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:06:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:06:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268880+0000", 2024-09-14T09:06:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5600000000000005e-07 2024-09-14T09:06:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268889+0000", 2024-09-14T09:06:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8559999999999996e-06 2024-09-14T09:06:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268913+0000", 2024-09-14T09:06:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3958e-05 2024-09-14T09:06:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268921+0000", 2024-09-14T09:06:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7479999999999999e-06 2024-09-14T09:06:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282098+0000", 2024-09-14T09:06:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013176297999999999 2024-09-14T09:06:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282101+0000", 2024-09-14T09:06:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2329999999999998e-06 2024-09-14T09:06:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293435+0000", 2024-09-14T09:06:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0113339459999999 2024-09-14T09:06:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293648+0000", 2024-09-14T09:06:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021307100000000001 2024-09-14T09:06:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293854+0000", 2024-09-14T09:06:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020580800000000001 2024-09-14T09:06:49.321 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73546 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15646720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:06:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.021582664, 2024-09-14T09:06:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0254846360000001, 2024-09-14T09:06:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.322 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73546 2024-09-14T09:06:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:06:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:06:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268934+0000", 2024-09-14T09:06:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7999999999999995e-07 2024-09-14T09:06:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268937+0000", 2024-09-14T09:06:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9840000000000001e-06 2024-09-14T09:06:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268947+0000", 2024-09-14T09:06:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0522e-05 2024-09-14T09:06:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268955+0000", 2024-09-14T09:06:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8059999999999995e-06 2024-09-14T09:06:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282112+0000", 2024-09-14T09:06:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013157189 2024-09-14T09:06:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282115+0000", 2024-09-14T09:06:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2949999999999998e-06 2024-09-14T09:06:49.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293968+0000", 2024-09-14T09:06:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0118530990000001 2024-09-14T09:06:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294197+0000", 2024-09-14T09:06:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228507 2024-09-14T09:06:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294417+0000", 2024-09-14T09:06:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000220554 2024-09-14T09:06:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73547 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:06:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.021550218, 2024-09-14T09:06:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.025970525, 2024-09-14T09:06:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.327 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.328 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.328 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.328 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73547 2024-09-14T09:06:49.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.328 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:06:49.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:06:49.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268966+0000", 2024-09-14T09:06:49.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3799999999999997e-07 2024-09-14T09:06:49.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268969+0000", 2024-09-14T09:06:49.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8739999999999999e-06 2024-09-14T09:06:49.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268979+0000", 2024-09-14T09:06:49.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9939999999999995e-06 2024-09-14T09:06:49.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268986+0000", 2024-09-14T09:06:49.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3459999999999996e-06 2024-09-14T09:06:49.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282127+0000", 2024-09-14T09:06:49.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01314039 2024-09-14T09:06:49.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282130+0000", 2024-09-14T09:06:49.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2219999999999998e-06 2024-09-14T09:06:49.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294540+0000", 2024-09-14T09:06:49.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124105880000001 2024-09-14T09:06:49.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294759+0000", 2024-09-14T09:06:49.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000218798 2024-09-14T09:06:49.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294936+0000", 2024-09-14T09:06:49.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000176675 2024-09-14T09:06:49.332 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.332 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.332 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.332 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73548 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15810560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.332 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:06:49.332 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.021519106, 2024-09-14T09:06:49.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0265045129999999, 2024-09-14T09:06:49.332 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.332 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.332 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.332 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.333 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.333 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.333 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73548 2024-09-14T09:06:49.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.333 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:06:49.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:06:49.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268997+0000", 2024-09-14T09:06:49.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7599999999999996e-07 2024-09-14T09:06:49.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269000+0000", 2024-09-14T09:06:49.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0580000000000002e-06 2024-09-14T09:06:49.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269010+0000", 2024-09-14T09:06:49.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124e-05 2024-09-14T09:06:49.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269018+0000", 2024-09-14T09:06:49.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5399999999999998e-06 2024-09-14T09:06:49.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282141+0000", 2024-09-14T09:06:49.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013123012 2024-09-14T09:06:49.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282144+0000", 2024-09-14T09:06:49.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.196e-06 2024-09-14T09:06:49.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295042+0000", 2024-09-14T09:06:49.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0128975689999999 2024-09-14T09:06:49.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295272+0000", 2024-09-14T09:06:49.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023048099999999999 2024-09-14T09:06:49.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295501+0000", 2024-09-14T09:06:49.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228657 2024-09-14T09:06:49.337 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.337 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.337 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.337 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73549 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15892480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.337 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:06:49.338 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.021473227, 2024-09-14T09:06:49.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0268354099999999, 2024-09-14T09:06:49.338 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.338 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.338 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.338 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.338 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.338 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.338 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73549 2024-09-14T09:06:49.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.338 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:06:49.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:06:49.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269043+0000", 2024-09-14T09:06:49.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6199999999999997e-07 2024-09-14T09:06:49.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269046+0000", 2024-09-14T09:06:49.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2480000000000001e-06 2024-09-14T09:06:49.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269057+0000", 2024-09-14T09:06:49.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0436e-05 2024-09-14T09:06:49.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269064+0000", 2024-09-14T09:06:49.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5179999999999998e-06 2024-09-14T09:06:49.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282155+0000", 2024-09-14T09:06:49.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013091190000000001 2024-09-14T09:06:49.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282159+0000", 2024-09-14T09:06:49.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.366e-06 2024-09-14T09:06:49.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295599+0000", 2024-09-14T09:06:49.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0134405559999999 2024-09-14T09:06:49.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295765+0000", 2024-09-14T09:06:49.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016575499999999999 2024-09-14T09:06:49.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295878+0000", 2024-09-14T09:06:49.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112679 2024-09-14T09:06:49.342 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.342 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.342 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.343 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73550 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15974400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.343 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:06:49.343 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.021440627, 2024-09-14T09:06:49.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027357657, 2024-09-14T09:06:49.343 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.343 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.343 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.343 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73550 2024-09-14T09:06:49.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.343 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:06:49.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:06:49.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269076+0000", 2024-09-14T09:06:49.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:06:49.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269079+0000", 2024-09-14T09:06:49.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:06:49.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269098+0000", 2024-09-14T09:06:49.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9698e-05 2024-09-14T09:06:49.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269106+0000", 2024-09-14T09:06:49.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6179999999999997e-06 2024-09-14T09:06:49.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282170+0000", 2024-09-14T09:06:49.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013063874 2024-09-14T09:06:49.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282173+0000", 2024-09-14T09:06:49.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1999999999999999e-06 2024-09-14T09:06:49.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295996+0000", 2024-09-14T09:06:49.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.013823151 2024-09-14T09:06:49.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296214+0000", 2024-09-14T09:06:49.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021817700000000001 2024-09-14T09:06:49.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296433+0000", 2024-09-14T09:06:49.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021829099999999999 2024-09-14T09:06:49.347 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.348 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.348 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.348 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73551 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.348 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:06:49.348 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.02139921, 2024-09-14T09:06:49.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027730276, 2024-09-14T09:06:49.348 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.348 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.348 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.348 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.348 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.348 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.349 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73551 2024-09-14T09:06:49.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.349 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:06:49.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:06:49.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269117+0000", 2024-09-14T09:06:49.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:06:49.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269120+0000", 2024-09-14T09:06:49.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.056e-06 2024-09-14T09:06:49.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269130+0000", 2024-09-14T09:06:49.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0084e-05 2024-09-14T09:06:49.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269137+0000", 2024-09-14T09:06:49.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1359999999999999e-06 2024-09-14T09:06:49.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282184+0000", 2024-09-14T09:06:49.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013046537 2024-09-14T09:06:49.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282188+0000", 2024-09-14T09:06:49.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3220000000000003e-06 2024-09-14T09:06:49.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296529+0000", 2024-09-14T09:06:49.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0143406109999999 2024-09-14T09:06:49.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296695+0000", 2024-09-14T09:06:49.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016604799999999999 2024-09-14T09:06:49.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296847+0000", 2024-09-14T09:06:49.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000151822 2024-09-14T09:06:49.353 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.353 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.353 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.353 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73552 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.353 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:06:49.353 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.02136863, 2024-09-14T09:06:49.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0281767500000001, 2024-09-14T09:06:49.353 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.353 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.353 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.353 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.354 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.354 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.354 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73552 2024-09-14T09:06:49.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.354 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:06:49.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:06:49.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:06:49.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:06:49.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269150+0000", 2024-09-14T09:06:49.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9119999999999998e-06 2024-09-14T09:06:49.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269160+0000", 2024-09-14T09:06:49.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6819999999999998e-06 2024-09-14T09:06:49.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269168+0000", 2024-09-14T09:06:49.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6079999999999998e-06 2024-09-14T09:06:49.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282205+0000", 2024-09-14T09:06:49.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013036982000000001 2024-09-14T09:06:49.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282208+0000", 2024-09-14T09:06:49.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4620000000000001e-06 2024-09-14T09:06:49.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296968+0000", 2024-09-14T09:06:49.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0147601209999999 2024-09-14T09:06:49.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297115+0000", 2024-09-14T09:06:49.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014673900000000001 2024-09-14T09:06:49.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297324+0000", 2024-09-14T09:06:49.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020866400000000001 2024-09-14T09:06:49.358 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.358 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.358 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.358 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73553 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.358 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:06:49.358 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.021338566000001, 2024-09-14T09:06:49.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0286556739999999, 2024-09-14T09:06:49.358 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.358 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.358 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.359 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.359 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.359 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.359 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73553 2024-09-14T09:06:49.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.359 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:06:49.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:06:49.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269178+0000", 2024-09-14T09:06:49.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:06:49.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269180+0000", 2024-09-14T09:06:49.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T09:06:49.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269189+0000", 2024-09-14T09:06:49.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2e-06 2024-09-14T09:06:49.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269197+0000", 2024-09-14T09:06:49.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2980000000000003e-06 2024-09-14T09:06:49.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282219+0000", 2024-09-14T09:06:49.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013022687 2024-09-14T09:06:49.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282223+0000", 2024-09-14T09:06:49.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.241e-06 2024-09-14T09:06:49.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297470+0000", 2024-09-14T09:06:49.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.015247778 2024-09-14T09:06:49.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297704+0000", 2024-09-14T09:06:49.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000233367 2024-09-14T09:06:49.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297833+0000", 2024-09-14T09:06:49.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012888900000000001 2024-09-14T09:06:49.363 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.363 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.363 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.363 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73554 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.363 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:06:49.363 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.021309249, 2024-09-14T09:06:49.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.029005113, 2024-09-14T09:06:49.364 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.364 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.364 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.364 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.364 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.364 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.364 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73554 2024-09-14T09:06:49.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.364 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:06:49.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:06:49.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269207+0000", 2024-09-14T09:06:49.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.68e-07 2024-09-14T09:06:49.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269210+0000", 2024-09-14T09:06:49.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8100000000000002e-06 2024-09-14T09:06:49.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269219+0000", 2024-09-14T09:06:49.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:06:49.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269227+0000", 2024-09-14T09:06:49.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3839999999999999e-06 2024-09-14T09:06:49.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282234+0000", 2024-09-14T09:06:49.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013007187 2024-09-14T09:06:49.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282237+0000", 2024-09-14T09:06:49.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1200000000000002e-06 2024-09-14T09:06:49.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297942+0000", 2024-09-14T09:06:49.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0157048040000001 2024-09-14T09:06:49.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298087+0000", 2024-09-14T09:06:49.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014499099999999999 2024-09-14T09:06:49.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298211+0000", 2024-09-14T09:06:49.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012461899999999999 2024-09-14T09:06:49.368 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.368 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.368 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.368 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73555 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.368 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:06:49.369 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.021279400999999, 2024-09-14T09:06:49.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.02941558, 2024-09-14T09:06:49.369 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.369 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.369 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.369 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.369 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.369 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.369 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73555 2024-09-14T09:06:49.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.369 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:06:49.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:06:49.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269237+0000", 2024-09-14T09:06:49.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3399999999999999e-07 2024-09-14T09:06:49.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269240+0000", 2024-09-14T09:06:49.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8480000000000001e-06 2024-09-14T09:06:49.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269249+0000", 2024-09-14T09:06:49.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1880000000000008e-06 2024-09-14T09:06:49.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269256+0000", 2024-09-14T09:06:49.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:06:49.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282248+0000", 2024-09-14T09:06:49.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012992327 2024-09-14T09:06:49.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282251+0000", 2024-09-14T09:06:49.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2160000000000002e-06 2024-09-14T09:06:49.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298299+0000", 2024-09-14T09:06:49.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0160479330000001 2024-09-14T09:06:49.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298446+0000", 2024-09-14T09:06:49.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000146525 2024-09-14T09:06:49.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298652+0000", 2024-09-14T09:06:49.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020584100000000001 2024-09-14T09:06:49.373 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.373 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.373 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.373 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73556 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.373 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:06:49.373 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.021249655, 2024-09-14T09:06:49.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0299854239999999, 2024-09-14T09:06:49.374 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.374 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.374 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.374 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73556 2024-09-14T09:06:49.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.374 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:06:49.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:06:49.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:06:49.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:06:49.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269270+0000", 2024-09-14T09:06:49.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0920000000000002e-06 2024-09-14T09:06:49.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269280+0000", 2024-09-14T09:06:49.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0013999999999999e-05 2024-09-14T09:06:49.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269288+0000", 2024-09-14T09:06:49.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:06:49.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282263+0000", 2024-09-14T09:06:49.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012974703000000001 2024-09-14T09:06:49.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282266+0000", 2024-09-14T09:06:49.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1520000000000001e-06 2024-09-14T09:06:49.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298729+0000", 2024-09-14T09:06:49.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.016463506 2024-09-14T09:06:49.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298884+0000", 2024-09-14T09:06:49.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015450800000000001 2024-09-14T09:06:49.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299251+0000", 2024-09-14T09:06:49.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036759899999999997 2024-09-14T09:06:49.378 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.378 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.378 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.378 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73557 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16547840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.378 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:06:49.378 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.021203005, 2024-09-14T09:06:49.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0307969480000001, 2024-09-14T09:06:49.379 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.379 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.379 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.379 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.379 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.379 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.379 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73557 2024-09-14T09:06:49.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.379 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:06:49.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:06:49.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269313+0000", 2024-09-14T09:06:49.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.92e-07 2024-09-14T09:06:49.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269336+0000", 2024-09-14T09:06:49.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3118000000000001e-05 2024-09-14T09:06:49.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269346+0000", 2024-09-14T09:06:49.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4069999999999999e-06 2024-09-14T09:06:49.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269352+0000", 2024-09-14T09:06:49.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5810000000000002e-06 2024-09-14T09:06:49.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282277+0000", 2024-09-14T09:06:49.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012924658 2024-09-14T09:06:49.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282280+0000", 2024-09-14T09:06:49.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2880000000000001e-06 2024-09-14T09:06:49.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299343+0000", 2024-09-14T09:06:49.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0170630190000001 2024-09-14T09:06:49.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299554+0000", 2024-09-14T09:06:49.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021085099999999999 2024-09-14T09:06:49.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300109+0000", 2024-09-14T09:06:49.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00055533399999999999 2024-09-14T09:06:49.383 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.383 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.383 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.383 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73558 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.384 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:06:49.384 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.021151381999999, 2024-09-14T09:06:49.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0314849340000001, 2024-09-14T09:06:49.384 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.384 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.384 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.384 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.384 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.384 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.384 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73558 2024-09-14T09:06:49.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.384 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:06:49.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:06:49.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269365+0000", 2024-09-14T09:06:49.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7400000000000003e-07 2024-09-14T09:06:49.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269367+0000", 2024-09-14T09:06:49.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4930000000000001e-06 2024-09-14T09:06:49.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269374+0000", 2024-09-14T09:06:49.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9519999999999997e-06 2024-09-14T09:06:49.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269379+0000", 2024-09-14T09:06:49.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9710000000000003e-06 2024-09-14T09:06:49.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282292+0000", 2024-09-14T09:06:49.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012912704000000001 2024-09-14T09:06:49.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282295+0000", 2024-09-14T09:06:49.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.179e-06 2024-09-14T09:06:49.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300195+0000", 2024-09-14T09:06:49.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.017899975 2024-09-14T09:06:49.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300345+0000", 2024-09-14T09:06:49.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014957499999999999 2024-09-14T09:06:49.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300849+0000", 2024-09-14T09:06:49.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050451100000000002 2024-09-14T09:06:49.388 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.389 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.389 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.389 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73559 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:49.389 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:06:49.389 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.021128973, 2024-09-14T09:06:49.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.031916343, 2024-09-14T09:06:49.389 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.389 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.389 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.390 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.390 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73559 2024-09-14T09:06:49.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.390 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:06:49.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:06:49.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:06:49.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3500000000000002e-07 2024-09-14T09:06:49.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269390+0000", 2024-09-14T09:06:49.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5890000000000001e-06 2024-09-14T09:06:49.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269396+0000", 2024-09-14T09:06:49.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3300000000000004e-06 2024-09-14T09:06:49.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269401+0000", 2024-09-14T09:06:49.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0340000000000004e-06 2024-09-14T09:06:49.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282311+0000", 2024-09-14T09:06:49.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012910204 2024-09-14T09:06:49.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:49.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282317+0000", 2024-09-14T09:06:49.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2639999999999997e-06 2024-09-14T09:06:49.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300933+0000", 2024-09-14T09:06:49.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0186160710000001 2024-09-14T09:06:49.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301158+0000", 2024-09-14T09:06:49.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225021 2024-09-14T09:06:49.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301303+0000", 2024-09-14T09:06:49.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014439499999999999 2024-09-14T09:06:49.394 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.394 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.394 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.394 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.394 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.394 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:49.395 INFO:teuthology.orchestra.run.smithi176.stdout:3815 punch from 0x5da960 to 0x5e5cfd, (0xb39d bytes) 2024-09-14T09:06:49.395 INFO:teuthology.orchestra.run.smithi176.stdout:3816 read 0x3d2459 thru 0x3dbe4e (0x99f6 bytes) 2024-09-14T09:06:49.396 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:06:49.397 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:06:49.397 INFO:teuthology.orchestra.run.smithi176.stdout:{"fsid":"c635667d-994e-454f-bcea-51b45637b2a2","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub flag(s) set","count":7},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":112,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":89,"num_osds":8,"num_up_osds":8,"osd_up_since":1726304711,"num_in_osds":8,"osd_in_since":1726304699,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":38},{"state_name":"peering","count":2},{"state_name":"creating+peering","count":1}],"num_pgs":41,"num_pools":3,"num_objects":1227,"data_bytes":1045368526,"bytes_used":357928960,"bytes_avail":772736184320,"bytes_total":773094113280,"inactive_pgs_ratio":0.073170728981494904,"recovering_objects_per_sec":183,"recovering_bytes_per_sec":12439279,"recovering_keys_per_sec":46,"num_objects_recovered":734,"num_bytes_recovered":49767154,"num_keys_recovered":186,"read_bytes_sec":1843084,"write_bytes_sec":688245,"read_op_per_sec":452,"write_op_per_sec":100},"fsmap":{"epoch":1,"btime":"2024-09-14T09:04:57:032443+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-14T09:06:13.975920+0000","services":{}},"progress_events":{}} 2024-09-14T09:06:49.397 INFO:teuthology.orchestra.run.smithi176.stdout:3817 write 0xd69c697 thru 0xd6aa9e0 (0xe34a bytes) 2024-09-14T09:06:49.398 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-14T09:06:49.398 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:06:49.398 INFO:teuthology.orchestra.run.smithi176.stdout:3821 read 0xd45705c thru 0xd459a41 (0x29e6 bytes) 2024-09-14T09:06:49.399 INFO:teuthology.orchestra.run.smithi176.stdout:3822 flatten 2024-09-14T09:06:49.399 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:49.395+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:49.447 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:49.447 INFO:teuthology.orchestra.run.smithi176.stdout:3825 write 0x6276344 thru 0x627bc70 (0x592d bytes) 2024-09-14T09:06:49.465 INFO:teuthology.orchestra.run.smithi176.stdout:3826 trunc from 0xd6aa9e1 to 0x5def113 2024-09-14T09:06:49.480 INFO:teuthology.orchestra.run.smithi176.stdout:3829 writesame 0x2e0903a thru 0x2e167c0 data_size 0x5(0xd787 bytes) 2024-09-14T09:06:49.494 INFO:teuthology.orchestra.run.smithi176.stdout:3830 punch from 0x413b990 to 0x41455da, (0x9c4a bytes) 2024-09-14T09:06:49.494 INFO:teuthology.orchestra.run.smithi176.stdout:3831 read 0x336f221 thru 0x33771fc (0x7fdc bytes) 2024-09-14T09:06:49.494 INFO:teuthology.orchestra.run.smithi176.stdout:3833 punch from 0x4c516bd to 0x4c5f1fc, (0xdb3f bytes) 2024-09-14T09:06:49.497 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:49.499+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:49.499 INFO:teuthology.orchestra.run.smithi176.stdout:3834 trunc from 0x5def113 to 0x18f62c2 2024-09-14T09:06:49.513 INFO:teuthology.orchestra.run.smithi176.stdout:3835 punch from 0x61dc47 to 0x627b60, (0x9f19 bytes) 2024-09-14T09:06:49.522 INFO:teuthology.orchestra.run.smithi176.stdout:3837 trunc from 0x18f62c2 to 0x94d05c3 2024-09-14T09:06:49.540 INFO:teuthology.orchestra.run.smithi176.stdout:3838 trunc from 0x94d05c3 to 0x5f2959 2024-09-14T09:06:49.571 INFO:teuthology.orchestra.run.smithi176.stdout:3840 writesame 0xcda99da thru 0xcdb5612 data_size 0x17(0xbc39 bytes) 2024-09-14T09:06:49.572 INFO:teuthology.orchestra.run.smithi176.stdout:3843 read 0xc57e01e thru 0xc589cb6 (0xbc99 bytes) 2024-09-14T09:06:49.572 INFO:teuthology.orchestra.run.smithi176.stdout:3845 punch from 0x7d072d6 to 0x7d13b8a, (0xc8b4 bytes) 2024-09-14T09:06:49.572 INFO:teuthology.orchestra.run.smithi176.stdout:3846 writesame 0x45236f2 thru 0x4530a9d data_size 0x4c(0xd3ac bytes) 2024-09-14T09:06:49.578 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:49.578 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:49.578 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:49.578 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:49.588 INFO:teuthology.orchestra.run.smithi176.stdout:3848 write 0x656e0 thru 0x659d4 (0x2f5 bytes) 2024-09-14T09:06:49.590 INFO:teuthology.orchestra.run.smithi176.stdout:3849 write 0x1cbba22 thru 0x1cc185d (0x5e3c bytes) 2024-09-14T09:06:49.596 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:06:49.600 INFO:teuthology.orchestra.run.smithi176.stdout:3851 clone 28 order 21 su 16384 sc 7 2024-09-14T09:06:49.600 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:49.599+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:49.698 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:49.699+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:49.711 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:49.711 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:49.711 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:49.711 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:49.711 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:49.725 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:06:49.759 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:06:49.760 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:06:49.777 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-14T09:06:48.891676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1045368526,"num_objects":1227,"num_object_clones":367,"num_object_copies":2454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1227,"num_whiteouts":440,"num_read":104448,"num_read_kb":546595,"num_write":13061,"num_write_kb":223909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1216,"num_bytes_recovered":72785688,"num_keys_recovered":252,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10248,"ondisk_log_size":10248,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":349540,"kb_used_data":44532,"kb_used_omap":12,"kb_used_meta":304819,"kb_avail":754625180,"statfs":{"total":773094113280,"available":772736184320,"internally_reserved":0,"allocated":45600768,"data_stored":324876064,"data_compressed":9529226,"data_compressed_allocated":26660864,"data_compressed_original":314194383,"omap_allocated":12751,"internal_metadata":312135217},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8570887,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1811,"num_read_kb":7201,"num_write":401,"num_write_kb":2689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":734,"num_bytes_recovered":49767154,"num_keys_recovered":186,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.041168"},"pg_stats":[{"pgid":"3.1f","version":"87'1344","reported_seq":4285,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:47.223591+0000","last_change":"2024-09-14T09:06:41.348384+0000","last_active":"2024-09-14T09:06:47.223591+0000","last_peered":"2024-09-14T09:06:47.223591+0000","last_clean":"2024-09-14T09:06:47.223591+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:06:47.223591+0000","last_undegraded":"2024-09-14T09:06:47.223591+0000","last_fullsized":"2024-09-14T09:06:47.223591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1344,"log_dups_size":0,"ondisk_log_size":1344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:40:01.002873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001387512,"stat_sum":{"num_bytes":17479966,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":15,"num_read":9351,"num_read_kb":14328,"num_write":2320,"num_write_kb":5034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"85'201","reported_seq":2424,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.854797+0000","last_change":"2024-09-14T09:05:29.039503+0000","last_active":"2024-09-14T09:06:45.854797+0000","last_peered":"2024-09-14T09:06:45.854797+0000","last_clean":"2024-09-14T09:06:45.854797+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:06:45.854797+0000","last_undegraded":"2024-09-14T09:06:45.854797+0000","last_fullsized":"2024-09-14T09:06:45.854797+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:25:36.908116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001657176,"stat_sum":{"num_bytes":32516728,"num_objects":53,"num_object_clones":18,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":20,"num_read":2154,"num_read_kb":22916,"num_write":239,"num_write_kb":9189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"87'367","reported_seq":3745,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.873849+0000","last_change":"2024-09-14T09:05:28.835298+0000","last_active":"2024-09-14T09:06:46.873849+0000","last_peered":"2024-09-14T09:06:46.873849+0000","last_clean":"2024-09-14T09:06:46.873849+0000","last_became_active":"2024-09-14T09:05:26.835372+0000","last_became_peered":"2024-09-14T09:05:26.835372+0000","last_unstale":"2024-09-14T09:06:46.873849+0000","last_undegraded":"2024-09-14T09:06:46.873849+0000","last_fullsized":"2024-09-14T09:06:46.873849+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:38.418901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014149329999999999,"stat_sum":{"num_bytes":58835428,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":10,"num_read":3995,"num_read_kb":52947,"num_write":512,"num_write_kb":8922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"83'525","reported_seq":3197,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.874459+0000","last_change":"2024-09-14T09:06:44.867658+0000","last_active":"2024-09-14T09:06:46.874459+0000","last_peered":"2024-09-14T09:06:46.874459+0000","last_clean":"2024-09-14T09:06:46.874459+0000","last_became_active":"2024-09-14T09:06:44.862945+0000","last_became_peered":"2024-09-14T09:06:44.862945+0000","last_unstale":"2024-09-14T09:06:46.874459+0000","last_undegraded":"2024-09-14T09:06:46.874459+0000","last_fullsized":"2024-09-14T09:06:46.874459+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:13:41.872396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014661419999999999,"stat_sum":{"num_bytes":38979686,"num_objects":37,"num_object_clones":12,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":5424,"num_read_kb":22099,"num_write":742,"num_write_kb":6156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"79'205","reported_seq":2402,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.875685+0000","last_change":"2024-09-14T09:06:41.347478+0000","last_active":"2024-09-14T09:06:46.875685+0000","last_peered":"2024-09-14T09:06:46.875685+0000","last_clean":"2024-09-14T09:06:46.875685+0000","last_became_active":"2024-09-14T09:06:41.346397+0000","last_became_peered":"2024-09-14T09:06:41.346397+0000","last_unstale":"2024-09-14T09:06:46.875685+0000","last_undegraded":"2024-09-14T09:06:46.875685+0000","last_fullsized":"2024-09-14T09:06:46.875685+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:30:10.950021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060373399999999998,"stat_sum":{"num_bytes":30069643,"num_objects":40,"num_object_clones":16,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":1935,"num_read_kb":6316,"num_write":210,"num_write_kb":11506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":6073202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"85'484","reported_seq":3433,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.985628+0000","last_change":"2024-09-14T09:06:44.872988+0000","last_active":"2024-09-14T09:06:45.985628+0000","last_peered":"2024-09-14T09:06:45.985628+0000","last_clean":"2024-09-14T09:06:45.985628+0000","last_became_active":"2024-09-14T09:06:44.869103+0000","last_became_peered":"2024-09-14T09:06:44.869103+0000","last_unstale":"2024-09-14T09:06:45.985628+0000","last_undegraded":"2024-09-14T09:06:45.985628+0000","last_fullsized":"2024-09-14T09:06:45.985628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:50:16.711214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020845550000000001,"stat_sum":{"num_bytes":34465908,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":4445,"num_read_kb":18966,"num_write":769,"num_write_kb":10088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"83'209","reported_seq":2089,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.885547+0000","last_change":"2024-09-14T09:05:28.822565+0000","last_active":"2024-09-14T09:06:45.885547+0000","last_peered":"2024-09-14T09:06:45.885547+0000","last_clean":"2024-09-14T09:06:45.885547+0000","last_became_active":"2024-09-14T09:05:26.816132+0000","last_became_peered":"2024-09-14T09:05:26.816132+0000","last_unstale":"2024-09-14T09:06:45.885547+0000","last_undegraded":"2024-09-14T09:06:45.885547+0000","last_fullsized":"2024-09-14T09:06:45.885547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:01:39.934279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092420299999999996,"stat_sum":{"num_bytes":22175026,"num_objects":33,"num_object_clones":11,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":12,"num_read":1799,"num_read_kb":6256,"num_write":248,"num_write_kb":4631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"83'285","reported_seq":3392,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.854365+0000","last_change":"2024-09-14T09:06:41.342724+0000","last_active":"2024-09-14T09:06:45.854365+0000","last_peered":"2024-09-14T09:06:45.854365+0000","last_clean":"2024-09-14T09:06:45.854365+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:06:45.854365+0000","last_undegraded":"2024-09-14T09:06:45.854365+0000","last_fullsized":"2024-09-14T09:06:45.854365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:31:28.047943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081579899999999997,"stat_sum":{"num_bytes":50357810,"num_objects":48,"num_object_clones":17,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":14,"num_read":3701,"num_read_kb":8060,"num_write":275,"num_write_kb":3949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"83'357","reported_seq":4359,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.876315+0000","last_change":"2024-09-14T09:05:28.839928+0000","last_active":"2024-09-14T09:06:46.876315+0000","last_peered":"2024-09-14T09:06:46.876315+0000","last_clean":"2024-09-14T09:06:46.876315+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:06:46.876315+0000","last_undegraded":"2024-09-14T09:06:46.876315+0000","last_fullsized":"2024-09-14T09:06:46.876315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:36:52.750689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014343089999999999,"stat_sum":{"num_bytes":37014757,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":4962,"num_read_kb":22170,"num_write":440,"num_write_kb":4862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"86'377","reported_seq":1535,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.873865+0000","last_change":"2024-09-14T09:05:28.836370+0000","last_active":"2024-09-14T09:06:46.873865+0000","last_peered":"2024-09-14T09:06:46.873865+0000","last_clean":"2024-09-14T09:06:46.873865+0000","last_became_active":"2024-09-14T09:05:26.836744+0000","last_became_peered":"2024-09-14T09:05:26.836744+0000","last_unstale":"2024-09-14T09:06:46.873865+0000","last_undegraded":"2024-09-14T09:06:46.873865+0000","last_fullsized":"2024-09-14T09:06:46.873865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:11:06.446635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015277190000000001,"stat_sum":{"num_bytes":15800887,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":9,"num_read":3095,"num_read_kb":7897,"num_write":507,"num_write_kb":3559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"83'514","reported_seq":1650,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.860602+0000","last_change":"2024-09-14T09:05:28.851587+0000","last_active":"2024-09-14T09:06:45.860602+0000","last_peered":"2024-09-14T09:06:45.860602+0000","last_clean":"2024-09-14T09:06:45.860602+0000","last_became_active":"2024-09-14T09:05:26.840893+0000","last_became_peered":"2024-09-14T09:05:26.840893+0000","last_unstale":"2024-09-14T09:06:45.860602+0000","last_undegraded":"2024-09-14T09:06:45.860602+0000","last_fullsized":"2024-09-14T09:06:45.860602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:35:19.280259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098594000000000004,"stat_sum":{"num_bytes":9058215,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":13,"num_read":3911,"num_read_kb":3852,"num_write":780,"num_write_kb":2759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"85'326","reported_seq":1723,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.871881+0000","last_change":"2024-09-14T09:06:44.867267+0000","last_active":"2024-09-14T09:06:45.871881+0000","last_peered":"2024-09-14T09:06:45.871881+0000","last_clean":"2024-09-14T09:06:45.871881+0000","last_became_active":"2024-09-14T09:06:44.865413+0000","last_became_peered":"2024-09-14T09:06:44.865413+0000","last_unstale":"2024-09-14T09:06:45.871881+0000","last_undegraded":"2024-09-14T09:06:45.871881+0000","last_fullsized":"2024-09-14T09:06:45.871881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:19:03.069884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014079679999999999,"stat_sum":{"num_bytes":20928767,"num_objects":42,"num_object_clones":13,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":16,"num_read":2271,"num_read_kb":3974,"num_write":374,"num_write_kb":902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":253148,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":150,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:47.006915+0000","last_change":"2024-09-14T09:05:19.793674+0000","last_active":"2024-09-14T09:06:47.006915+0000","last_peered":"2024-09-14T09:06:47.006915+0000","last_clean":"2024-09-14T09:06:47.006915+0000","last_became_active":"2024-09-14T09:05:17.284977+0000","last_became_peered":"2024-09-14T09:05:17.284977+0000","last_unstale":"2024-09-14T09:06:47.006915+0000","last_undegraded":"2024-09-14T09:06:47.006915+0000","last_fullsized":"2024-09-14T09:06:47.006915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:30:38.832027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"87'227","reported_seq":3533,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.914300+0000","last_change":"2024-09-14T09:05:28.839614+0000","last_active":"2024-09-14T09:06:46.914300+0000","last_peered":"2024-09-14T09:06:46.914300+0000","last_clean":"2024-09-14T09:06:46.914300+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:06:46.914300+0000","last_undegraded":"2024-09-14T09:06:46.914300+0000","last_fullsized":"2024-09-14T09:06:46.914300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:30:20.294407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014245869999999999,"stat_sum":{"num_bytes":40394524,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":4246,"num_read_kb":38806,"num_write":246,"num_write_kb":6221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":148,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.890630+0000","last_change":"2024-09-14T09:05:19.314029+0000","last_active":"2024-09-14T09:06:45.890630+0000","last_peered":"2024-09-14T09:06:45.890630+0000","last_clean":"2024-09-14T09:06:45.890630+0000","last_became_active":"2024-09-14T09:05:17.282890+0000","last_became_peered":"2024-09-14T09:05:17.282890+0000","last_unstale":"2024-09-14T09:06:45.890630+0000","last_undegraded":"2024-09-14T09:06:45.890630+0000","last_fullsized":"2024-09-14T09:06:45.890630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:25.304105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"85'362","reported_seq":2363,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.854777+0000","last_change":"2024-09-14T09:05:29.039154+0000","last_active":"2024-09-14T09:06:45.854777+0000","last_peered":"2024-09-14T09:06:45.854777+0000","last_clean":"2024-09-14T09:06:45.854777+0000","last_became_active":"2024-09-14T09:05:26.829855+0000","last_became_peered":"2024-09-14T09:05:26.829855+0000","last_unstale":"2024-09-14T09:06:45.854777+0000","last_undegraded":"2024-09-14T09:06:45.854777+0000","last_fullsized":"2024-09-14T09:06:45.854777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:12:56.716058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019213749999999999,"stat_sum":{"num_bytes":37553508,"num_objects":29,"num_object_clones":9,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":9,"num_read":3271,"num_read_kb":12689,"num_write":472,"num_write_kb":3916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":148,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.860869+0000","last_change":"2024-09-14T09:05:19.794165+0000","last_active":"2024-09-14T09:06:45.860869+0000","last_peered":"2024-09-14T09:06:45.860869+0000","last_clean":"2024-09-14T09:06:45.860869+0000","last_became_active":"2024-09-14T09:05:17.295864+0000","last_became_peered":"2024-09-14T09:05:17.295864+0000","last_unstale":"2024-09-14T09:06:45.860869+0000","last_undegraded":"2024-09-14T09:06:45.860869+0000","last_fullsized":"2024-09-14T09:06:45.860869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:53.901865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011399719999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"83'270","reported_seq":2708,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.885402+0000","last_change":"2024-09-14T09:06:35.286509+0000","last_active":"2024-09-14T09:06:45.885402+0000","last_peered":"2024-09-14T09:06:45.885402+0000","last_clean":"2024-09-14T09:06:45.885402+0000","last_became_active":"2024-09-14T09:06:35.282469+0000","last_became_peered":"2024-09-14T09:06:35.282469+0000","last_unstale":"2024-09-14T09:06:45.885402+0000","last_undegraded":"2024-09-14T09:06:45.885402+0000","last_fullsized":"2024-09-14T09:06:45.885402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:56:24.711903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014087959999999999,"stat_sum":{"num_bytes":20460999,"num_objects":44,"num_object_clones":15,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":18,"num_read":3301,"num_read_kb":15228,"num_write":277,"num_write_kb":5460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":1142719,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.885604+0000","last_change":"2024-09-14T09:05:19.314129+0000","last_active":"2024-09-14T09:06:45.885604+0000","last_peered":"2024-09-14T09:06:45.885604+0000","last_clean":"2024-09-14T09:06:45.885604+0000","last_became_active":"2024-09-14T09:05:17.295196+0000","last_became_peered":"2024-09-14T09:05:17.295196+0000","last_unstale":"2024-09-14T09:06:45.885604+0000","last_undegraded":"2024-09-14T09:06:45.885604+0000","last_fullsized":"2024-09-14T09:06:45.885604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:35.269115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011613800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"83'168","reported_seq":871,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.873896+0000","last_change":"2024-09-14T09:05:28.835976+0000","last_active":"2024-09-14T09:06:46.873896+0000","last_peered":"2024-09-14T09:06:46.873896+0000","last_clean":"2024-09-14T09:06:46.873896+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:06:46.873896+0000","last_undegraded":"2024-09-14T09:06:46.873896+0000","last_fullsized":"2024-09-14T09:06:46.873896+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:21:06.765695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017480779999999999,"stat_sum":{"num_bytes":52867963,"num_objects":45,"num_object_clones":16,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":21,"num_read":596,"num_read_kb":9794,"num_write":155,"num_write_kb":9028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"83'358","reported_seq":1901,"reported_epoch":86,"state":"creating+peering","last_fresh":"2024-09-14T09:06:44.949645+0000","last_change":"2024-09-14T09:06:46.149268+0000","last_active":"2024-09-14T09:06:46.149268+0000","last_peered":"2024-09-14T09:06:44.949645+0000","last_clean":"2024-09-14T09:06:40.888318+0000","last_became_active":"2024-09-14T09:06:42.894671+0000","last_became_peered":"2024-09-14T09:06:42.894671+0000","last_unstale":"2024-09-14T09:06:44.949645+0000","last_undegraded":"2024-09-14T09:06:44.949645+0000","last_fullsized":"2024-09-14T09:06:42.847670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014128,"stat_sum":{"num_bytes":25020722,"num_objects":35,"num_object_clones":12,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":2744,"num_read_kb":15841,"num_write":439,"num_write_kb":8915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":152,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.873713+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:06:46.873713+0000","last_peered":"2024-09-14T09:06:46.873713+0000","last_clean":"2024-09-14T09:06:46.873713+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:06:46.873713+0000","last_undegraded":"2024-09-14T09:06:46.873713+0000","last_fullsized":"2024-09-14T09:06:46.873713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"83'173","reported_seq":975,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.885476+0000","last_change":"2024-09-14T09:05:28.822791+0000","last_active":"2024-09-14T09:06:45.885476+0000","last_peered":"2024-09-14T09:06:45.885476+0000","last_clean":"2024-09-14T09:06:45.885476+0000","last_became_active":"2024-09-14T09:05:26.817109+0000","last_became_peered":"2024-09-14T09:05:26.817109+0000","last_unstale":"2024-09-14T09:06:45.885476+0000","last_undegraded":"2024-09-14T09:06:45.885476+0000","last_fullsized":"2024-09-14T09:06:45.885476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:08:41.364341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011728439999999999,"stat_sum":{"num_bytes":14678273,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":849,"num_read_kb":3701,"num_write":184,"num_write_kb":1405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":150,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.870865+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:06:46.870865+0000","last_peered":"2024-09-14T09:06:46.870865+0000","last_clean":"2024-09-14T09:06:46.870865+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:06:46.870865+0000","last_undegraded":"2024-09-14T09:06:46.870865+0000","last_fullsized":"2024-09-14T09:06:46.870865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"87'205","reported_seq":1968,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.875774+0000","last_change":"2024-09-14T09:06:45.868939+0000","last_active":"2024-09-14T09:06:46.875774+0000","last_peered":"2024-09-14T09:06:46.875774+0000","last_clean":"2024-09-14T09:06:46.875774+0000","last_became_active":"2024-09-14T09:06:45.868194+0000","last_became_peered":"2024-09-14T09:06:45.868194+0000","last_unstale":"2024-09-14T09:06:46.875774+0000","last_undegraded":"2024-09-14T09:06:46.875774+0000","last_fullsized":"2024-09-14T09:06:46.875774+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:06:02.859882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074815999999999999,"stat_sum":{"num_bytes":29354455,"num_objects":31,"num_object_clones":11,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":1490,"num_read_kb":15691,"num_write":207,"num_write_kb":9103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":148,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.859972+0000","last_change":"2024-09-14T09:05:19.793999+0000","last_active":"2024-09-14T09:06:45.859972+0000","last_peered":"2024-09-14T09:06:45.859972+0000","last_clean":"2024-09-14T09:06:45.859972+0000","last_became_active":"2024-09-14T09:05:17.284122+0000","last_became_peered":"2024-09-14T09:05:17.284122+0000","last_unstale":"2024-09-14T09:06:45.859972+0000","last_undegraded":"2024-09-14T09:06:45.859972+0000","last_fullsized":"2024-09-14T09:06:45.859972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:12:01.133215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010234269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"68'195","reported_seq":1121,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.854815+0000","last_change":"2024-09-14T09:05:29.038389+0000","last_active":"2024-09-14T09:06:45.854815+0000","last_peered":"2024-09-14T09:06:45.854815+0000","last_clean":"2024-09-14T09:06:45.854815+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:06:45.854815+0000","last_undegraded":"2024-09-14T09:06:45.854815+0000","last_fullsized":"2024-09-14T09:06:45.854815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:42.547444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001479076,"stat_sum":{"num_bytes":19439154,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":1071,"num_read_kb":4172,"num_write":238,"num_write_kb":2073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.854623+0000","last_change":"2024-09-14T09:06:25.227644+0000","last_active":"2024-09-14T09:06:45.854623+0000","last_peered":"2024-09-14T09:06:45.854623+0000","last_clean":"2024-09-14T09:06:45.854623+0000","last_became_active":"2024-09-14T09:06:25.222344+0000","last_became_peered":"2024-09-14T09:06:25.222344+0000","last_unstale":"2024-09-14T09:06:45.854623+0000","last_undegraded":"2024-09-14T09:06:45.854623+0000","last_fullsized":"2024-09-14T09:06:45.854623+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:23:33.414859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":239,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.860270+0000","last_change":"2024-09-14T09:05:13.239808+0000","last_active":"2024-09-14T09:06:45.860270+0000","last_peered":"2024-09-14T09:06:45.860270+0000","last_clean":"2024-09-14T09:06:45.860270+0000","last_became_active":"2024-09-14T09:05:13.239064+0000","last_became_peered":"2024-09-14T09:05:13.239064+0000","last_unstale":"2024-09-14T09:06:45.860270+0000","last_undegraded":"2024-09-14T09:06:45.860270+0000","last_fullsized":"2024-09-14T09:06:45.860270+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:40:11.559161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"83'239","reported_seq":3304,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.854385+0000","last_change":"2024-09-14T09:05:29.037923+0000","last_active":"2024-09-14T09:06:45.854385+0000","last_peered":"2024-09-14T09:06:45.854385+0000","last_clean":"2024-09-14T09:06:45.854385+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:06:45.854385+0000","last_undegraded":"2024-09-14T09:06:45.854385+0000","last_fullsized":"2024-09-14T09:06:45.854385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:30:17.623019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099199800000000001,"stat_sum":{"num_bytes":42354761,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":3489,"num_read_kb":29604,"num_write":257,"num_write_kb":7537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"83'195","reported_seq":1588,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.870685+0000","last_change":"2024-09-14T09:06:44.868332+0000","last_active":"2024-09-14T09:06:45.870685+0000","last_peered":"2024-09-14T09:06:45.870685+0000","last_clean":"2024-09-14T09:06:45.870685+0000","last_became_active":"2024-09-14T09:06:44.866346+0000","last_became_peered":"2024-09-14T09:06:44.866346+0000","last_unstale":"2024-09-14T09:06:45.870685+0000","last_undegraded":"2024-09-14T09:06:45.870685+0000","last_fullsized":"2024-09-14T09:06:45.870685+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:47:10.301740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001266028,"stat_sum":{"num_bytes":27628229,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":1136,"num_read_kb":6881,"num_write":177,"num_write_kb":14095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":4589732,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"83'262","reported_seq":3293,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:47.006902+0000","last_change":"2024-09-14T09:06:45.870752+0000","last_active":"2024-09-14T09:06:47.006902+0000","last_peered":"2024-09-14T09:06:47.006902+0000","last_clean":"2024-09-14T09:06:47.006902+0000","last_became_active":"2024-09-14T09:06:45.869266+0000","last_became_peered":"2024-09-14T09:06:45.869266+0000","last_unstale":"2024-09-14T09:06:47.006902+0000","last_undegraded":"2024-09-14T09:06:47.006902+0000","last_fullsized":"2024-09-14T09:06:47.006902+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:44:26.704179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185731,"stat_sum":{"num_bytes":48761384,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":3347,"num_read_kb":31896,"num_write":294,"num_write_kb":16339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"79'303","reported_seq":7931,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.854973+0000","last_change":"2024-09-14T09:06:02.837388+0000","last_active":"2024-09-14T09:06:45.854973+0000","last_peered":"2024-09-14T09:06:45.854973+0000","last_clean":"2024-09-14T09:06:45.854973+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:06:45.854973+0000","last_undegraded":"2024-09-14T09:06:45.854973+0000","last_fullsized":"2024-09-14T09:06:45.854973+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:55:06.321684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001585295,"stat_sum":{"num_bytes":39029359,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":8803,"num_read_kb":53060,"num_write":301,"num_write_kb":7483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"83'209","reported_seq":1185,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.873838+0000","last_change":"2024-09-14T09:05:28.834647+0000","last_active":"2024-09-14T09:06:46.873838+0000","last_peered":"2024-09-14T09:06:46.873838+0000","last_clean":"2024-09-14T09:06:46.873838+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:06:46.873838+0000","last_undegraded":"2024-09-14T09:06:46.873838+0000","last_fullsized":"2024-09-14T09:06:46.873838+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:29:56.253705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244290000000001,"stat_sum":{"num_bytes":26698314,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":1353,"num_read_kb":7206,"num_write":222,"num_write_kb":7172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"86'238","reported_seq":3024,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.983415+0000","last_change":"2024-09-14T09:05:28.851561+0000","last_active":"2024-09-14T09:06:45.983415+0000","last_peered":"2024-09-14T09:06:45.983415+0000","last_clean":"2024-09-14T09:06:45.983415+0000","last_became_active":"2024-09-14T09:05:26.843780+0000","last_became_peered":"2024-09-14T09:05:26.843780+0000","last_unstale":"2024-09-14T09:06:45.983415+0000","last_undegraded":"2024-09-14T09:06:45.983415+0000","last_fullsized":"2024-09-14T09:06:45.983415+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:55:09.169516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072987200000000001,"stat_sum":{"num_bytes":28500713,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":3240,"num_read_kb":21867,"num_write":294,"num_write_kb":13667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"83'169","reported_seq":1399,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.987002+0000","last_change":"2024-09-14T09:06:45.872735+0000","last_active":"2024-09-14T09:06:45.987002+0000","last_peered":"2024-09-14T09:06:45.987002+0000","last_clean":"2024-09-14T09:06:45.987002+0000","last_became_active":"2024-09-14T09:06:45.868989+0000","last_became_peered":"2024-09-14T09:06:45.868989+0000","last_unstale":"2024-09-14T09:06:45.987002+0000","last_undegraded":"2024-09-14T09:06:45.987002+0000","last_fullsized":"2024-09-14T09:06:45.987002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:06:36.272971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057596100000000005,"stat_sum":{"num_bytes":35589399,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":13,"num_read":938,"num_read_kb":9274,"num_write":155,"num_write_kb":4467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":3897102,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"83'209","reported_seq":1206,"reported_epoch":85,"state":"peering","last_fresh":"2024-09-14T09:06:45.209516+0000","last_change":"2024-09-14T09:06:45.861860+0000","last_active":"2024-09-14T09:06:45.861860+0000","last_peered":"2024-09-14T09:06:45.209516+0000","last_clean":"2024-09-14T09:06:40.344351+0000","last_became_active":"2024-09-14T09:06:42.890786+0000","last_became_peered":"2024-09-14T09:06:42.890786+0000","last_unstale":"2024-09-14T09:06:45.209516+0000","last_undegraded":"2024-09-14T09:06:45.209516+0000","last_fullsized":"2024-09-14T09:06:42.850359+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000909463,"stat_sum":{"num_bytes":29699048,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":14,"num_read":1105,"num_read_kb":3779,"num_write":194,"num_write_kb":7632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":3341023,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"86'231","reported_seq":4069,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:47.006835+0000","last_change":"2024-09-14T09:06:00.166357+0000","last_active":"2024-09-14T09:06:47.006835+0000","last_peered":"2024-09-14T09:06:47.006835+0000","last_clean":"2024-09-14T09:06:47.006835+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:06:47.006835+0000","last_undegraded":"2024-09-14T09:06:47.006835+0000","last_fullsized":"2024-09-14T09:06:47.006835+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:58:28.963004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011381609999999999,"stat_sum":{"num_bytes":86083375,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":17,"num_read":3692,"num_read_kb":43423,"num_write":289,"num_write_kb":15097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"85'318","reported_seq":2643,"reported_epoch":85,"state":"peering","last_fresh":"2024-09-14T09:06:45.361251+0000","last_change":"2024-09-14T09:06:45.862138+0000","last_active":"2024-09-14T09:06:45.862138+0000","last_peered":"2024-09-14T09:06:45.361251+0000","last_clean":"2024-09-14T09:06:40.343697+0000","last_became_active":"2024-09-14T09:06:42.896330+0000","last_became_peered":"2024-09-14T09:06:42.896330+0000","last_unstale":"2024-09-14T09:06:45.361251+0000","last_undegraded":"2024-09-14T09:06:45.361251+0000","last_fullsized":"2024-09-14T09:06:42.848361+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092949299999999999,"stat_sum":{"num_bytes":5340400,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":13,"num_read":3514,"num_read_kb":3930,"num_write":426,"num_write_kb":737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":30867,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"87'370","reported_seq":2594,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.875813+0000","last_change":"2024-09-14T09:05:28.839774+0000","last_active":"2024-09-14T09:06:46.875813+0000","last_peered":"2024-09-14T09:06:46.875813+0000","last_clean":"2024-09-14T09:06:46.875813+0000","last_became_active":"2024-09-14T09:05:26.826600+0000","last_became_peered":"2024-09-14T09:05:26.826600+0000","last_unstale":"2024-09-14T09:06:46.875813+0000","last_undegraded":"2024-09-14T09:06:46.875813+0000","last_fullsized":"2024-09-14T09:06:46.875813+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:37:44.244140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018357099999999999,"stat_sum":{"num_bytes":44463041,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":17,"num_read":3668,"num_read_kb":3611,"num_write":463,"num_write_kb":3314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"79'278","reported_seq":4529,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-14T09:06:45.860081+0000","last_change":"2024-09-14T09:06:44.869874+0000","last_active":"2024-09-14T09:06:45.860081+0000","last_peered":"2024-09-14T09:06:45.860081+0000","last_clean":"2024-09-14T09:06:45.860081+0000","last_became_active":"2024-09-14T09:06:44.865881+0000","last_became_peered":"2024-09-14T09:06:44.865881+0000","last_unstale":"2024-09-14T09:06:45.860081+0000","last_undegraded":"2024-09-14T09:06:45.860081+0000","last_fullsized":"2024-09-14T09:06:45.860081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:42:49.191314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061978900000000004,"stat_sum":{"num_bytes":23177697,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":5486,"num_read_kb":26306,"num_write":252,"num_write_kb":7061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":6761444,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1044778139,"num_objects":1223,"num_object_clones":367,"num_object_copies":2446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1223,"num_whiteouts":440,"num_read":104382,"num_read_kb":546540,"num_write":12958,"num_write_kb":222279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1215,"num_bytes_recovered":72785688,"num_keys_recovered":252,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":34783232,"data_stored":296834935,"data_compressed":8893997,"data_compressed_allocated":24612864,"data_compressed_original":290295055,"omap_allocated":0,"internal_metadata":0},"log_size":10173,"ondisk_log_size":10173,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12424,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39656,"kb_used_data":4308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332184,"statfs":{"total":96636764160,"available":96596156416,"internally_reserved":0,"allocated":4411392,"data_stored":31752236,"data_compressed":825018,"data_compressed_allocated":2605056,"data_compressed_original":30800251,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50876,"kb_used_data":5804,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94320964,"statfs":{"total":96636764160,"available":96584667136,"internally_reserved":0,"allocated":5943296,"data_stored":41867259,"data_compressed":1136476,"data_compressed_allocated":3399680,"data_compressed_original":40472428,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509464,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41264,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330576,"statfs":{"total":96636764160,"available":96594509824,"internally_reserved":0,"allocated":1789952,"data_stored":6039141,"data_compressed":257074,"data_compressed_allocated":589824,"data_compressed_original":5468960,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509466,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46440,"kb_used_data":6932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325400,"statfs":{"total":96636764160,"available":96589209600,"internally_reserved":0,"allocated":7098368,"data_stored":53889428,"data_compressed":1551176,"data_compressed_allocated":4263936,"data_compressed_original":52295459,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542166,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":5524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":5656576,"data_stored":38413556,"data_compressed":904624,"data_compressed_allocated":3022848,"data_compressed_original":36788332,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542167,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47768,"kb_used_data":8264,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94324072,"statfs":{"total":96636764160,"available":96587849728,"internally_reserved":0,"allocated":8462336,"data_stored":66103371,"data_compressed":2043905,"data_compressed_allocated":5562368,"data_compressed_original":64406792,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542167,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45656,"kb_used_data":6140,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326184,"statfs":{"total":96636764160,"available":96590012416,"internally_reserved":0,"allocated":6287360,"data_stored":48338724,"data_compressed":1364959,"data_compressed_allocated":3653632,"data_compressed_original":46831386,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542168,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45320,"kb_used_data":5812,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326520,"statfs":{"total":96636764160,"available":96590356480,"internally_reserved":0,"allocated":5951488,"data_stored":38472349,"data_compressed":1445994,"data_compressed_allocated":3563520,"data_compressed_original":37130775,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5632000,"data_stored":49786473,"data_compressed":1463694,"data_compressed_allocated":3952640,"data_compressed_original":48752419,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3145728,"data_stored":23886926,"data_compressed":928721,"data_compressed_allocated":2195456,"data_compressed_original":23266647,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":770048,"data_stored":5573138,"data_compressed":257074,"data_compressed_allocated":589824,"data_compressed_original":5468960,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4837376,"data_stored":41823211,"data_compressed":1318925,"data_compressed_allocated":3293184,"data_compressed_original":40847130,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7467008,"data_stored":65649133,"data_compressed":2043905,"data_compressed_allocated":5562368,"data_compressed_original":64406792,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4947968,"data_stored":41466789,"data_compressed":1158248,"data_compressed_allocated":3428352,"data_compressed_original":40537964,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3391488,"data_stored":31286233,"data_compressed":825018,"data_compressed_allocated":2605056,"data_compressed_original":30800251,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4591616,"data_stored":37363032,"data_compressed":898412,"data_compressed_allocated":2985984,"data_compressed_original":36214892,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:06:49.822 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:49.823+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:49.894 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:49.894 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:49.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:49.894 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:49.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.894 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18383 3.9 3:9252f7c9:::rbd_data.115e3abd3b2c.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:06:49.894 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:49.895 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.732590845999994, 2024-09-14T09:06:49.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056433830999999997, 2024-09-14T09:06:49.895 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.895 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.895 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:49.895 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.895 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.896 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.896 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18383 2024-09-14T09:06:49.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.896 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:49.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:49.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158423+0000", 2024-09-14T09:06:49.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T09:06:49.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158429+0000", 2024-09-14T09:06:49.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0179999999999998e-06 2024-09-14T09:06:49.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158440+0000", 2024-09-14T09:06:49.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0769e-05 2024-09-14T09:06:49.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158445+0000", 2024-09-14T09:06:49.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5840000000000002e-06 2024-09-14T09:06:49.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158495+0000", 2024-09-14T09:06:49.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9129999999999999e-05 2024-09-14T09:06:49.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158598+0000", 2024-09-14T09:06:49.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102938 2024-09-14T09:06:49.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:49.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.161048+0000", 2024-09-14T09:06:49.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002450336 2024-09-14T09:06:49.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:49.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214605+0000", 2024-09-14T09:06:49.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053557526000000001 2024-09-14T09:06:49.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:49.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214634+0000", 2024-09-14T09:06:49.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8483000000000002e-05 2024-09-14T09:06:49.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:49.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214731+0000", 2024-09-14T09:06:49.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7026000000000002e-05 2024-09-14T09:06:49.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214856+0000", 2024-09-14T09:06:49.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012474799999999999 2024-09-14T09:06:49.903 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.903 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.903 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.904 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18383 3.9 e38/34 ondisk, result = 0)", 2024-09-14T09:06:49.904 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:49.904 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.722274365000004, 2024-09-14T09:06:49.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045880575999999999, 2024-09-14T09:06:49.904 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.904 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.904 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.904 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:49.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:49.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168739+0000", 2024-09-14T09:06:49.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3900000000000004e-07 2024-09-14T09:06:49.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168744+0000", 2024-09-14T09:06:49.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5929999999999999e-06 2024-09-14T09:06:49.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168754+0000", 2024-09-14T09:06:49.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0686999999999999e-05 2024-09-14T09:06:49.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168761+0000", 2024-09-14T09:06:49.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6340000000000003e-06 2024-09-14T09:06:49.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214578+0000", 2024-09-14T09:06:49.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045817443999999999 2024-09-14T09:06:49.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214596+0000", 2024-09-14T09:06:49.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7159e-05 2024-09-14T09:06:49.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214619+0000", 2024-09-14T09:06:49.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.332e-05 2024-09-14T09:06:49.907 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.908 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.908 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.908 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19152 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14159058~8940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:49.908 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:49.908 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 65.568361909000004, 2024-09-14T09:06:49.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12891994500000001, 2024-09-14T09:06:49.908 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.908 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.908 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:49.908 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.908 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.908 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.909 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19152 2024-09-14T09:06:49.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.909 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:49.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:49.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322653+0000", 2024-09-14T09:06:49.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3750000000000001e-06 2024-09-14T09:06:49.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329647+0000", 2024-09-14T09:06:49.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069941960000000003 2024-09-14T09:06:49.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329673+0000", 2024-09-14T09:06:49.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5760000000000001e-05 2024-09-14T09:06:49.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329684+0000", 2024-09-14T09:06:49.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0474e-05 2024-09-14T09:06:49.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329776+0000", 2024-09-14T09:06:49.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2467999999999996e-05 2024-09-14T09:06:49.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329987+0000", 2024-09-14T09:06:49.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021133 2024-09-14T09:06:49.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:49.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.331303+0000", 2024-09-14T09:06:49.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001315629 2024-09-14T09:06:49.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:49.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.411090+0000", 2024-09-14T09:06:49.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079787477999999995 2024-09-14T09:06:49.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:49.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451463+0000", 2024-09-14T09:06:49.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040372312 2024-09-14T09:06:49.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:49.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451523+0000", 2024-09-14T09:06:49.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0420000000000001e-05 2024-09-14T09:06:49.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451571+0000", 2024-09-14T09:06:49.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7503000000000002e-05 2024-09-14T09:06:49.914 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.914 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.914 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.914 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20926 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:49.914 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:49.914 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.219029909, 2024-09-14T09:06:49.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018572207, 2024-09-14T09:06:49.914 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.914 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.914 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.914 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.914 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.915 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.915 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20926 2024-09-14T09:06:49.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.915 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:49.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:49.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:49.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:06:49.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671990+0000", 2024-09-14T09:06:49.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5769999999999999e-06 2024-09-14T09:06:49.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672001+0000", 2024-09-14T09:06:49.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1293e-05 2024-09-14T09:06:49.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672008+0000", 2024-09-14T09:06:49.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9099999999999999e-06 2024-09-14T09:06:49.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689147+0000", 2024-09-14T09:06:49.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017139060000000001 2024-09-14T09:06:49.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689324+0000", 2024-09-14T09:06:49.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017640200000000001 2024-09-14T09:06:49.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690555+0000", 2024-09-14T09:06:49.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001231212 2024-09-14T09:06:49.918 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.918 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.918 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.918 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20927 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2621440~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:49.918 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:49.919 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.218992943000003, 2024-09-14T09:06:49.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020002227000000001, 2024-09-14T09:06:49.919 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.919 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.919 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.919 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.919 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.919 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.919 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20927 2024-09-14T09:06:49.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.919 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:49.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:49.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:49.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9699999999999995e-07 2024-09-14T09:06:49.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672024+0000", 2024-09-14T09:06:49.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:06:49.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672036+0000", 2024-09-14T09:06:49.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.119e-05 2024-09-14T09:06:49.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672044+0000", 2024-09-14T09:06:49.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8370000000000003e-06 2024-09-14T09:06:49.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690602+0000", 2024-09-14T09:06:49.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018557648 2024-09-14T09:06:49.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690805+0000", 2024-09-14T09:06:49.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020282899999999999 2024-09-14T09:06:49.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692022+0000", 2024-09-14T09:06:49.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012170429999999999 2024-09-14T09:06:49.922 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.923 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.923 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.923 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20928 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 917504~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:49.923 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:49.923 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.218956802999998, 2024-09-14T09:06:49.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021623738, 2024-09-14T09:06:49.923 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.923 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.923 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.923 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.923 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.923 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.924 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20928 2024-09-14T09:06:49.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.924 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:49.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:49.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672057+0000", 2024-09-14T09:06:49.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:06:49.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672059+0000", 2024-09-14T09:06:49.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9270000000000001e-06 2024-09-14T09:06:49.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672070+0000", 2024-09-14T09:06:49.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0773e-05 2024-09-14T09:06:49.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672077+0000", 2024-09-14T09:06:49.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9600000000000003e-06 2024-09-14T09:06:49.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692154+0000", 2024-09-14T09:06:49.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020076482 2024-09-14T09:06:49.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692419+0000", 2024-09-14T09:06:49.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026517400000000002 2024-09-14T09:06:49.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693680+0000", 2024-09-14T09:06:49.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012607390000000001 2024-09-14T09:06:49.927 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.927 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.927 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.927 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20929 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2752512~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:49.927 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:49.927 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.218923760999999, 2024-09-14T09:06:49.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022898728, 2024-09-14T09:06:49.927 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.928 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.928 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.928 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.928 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.928 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.928 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20929 2024-09-14T09:06:49.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.928 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:49.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:49.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672090+0000", 2024-09-14T09:06:49.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:06:49.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672092+0000", 2024-09-14T09:06:49.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8940000000000002e-06 2024-09-14T09:06:49.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672103+0000", 2024-09-14T09:06:49.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0815999999999999e-05 2024-09-14T09:06:49.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672110+0000", 2024-09-14T09:06:49.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9469999999999997e-06 2024-09-14T09:06:49.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693737+0000", 2024-09-14T09:06:49.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021626467999999999 2024-09-14T09:06:49.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693962+0000", 2024-09-14T09:06:49.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225474 2024-09-14T09:06:49.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.694988+0000", 2024-09-14T09:06:49.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001025469 2024-09-14T09:06:49.931 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.931 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.931 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.932 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20930 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:49.932 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:49.932 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.218890666999997, 2024-09-14T09:06:49.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024303286, 2024-09-14T09:06:49.932 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.932 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.932 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.932 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.932 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.932 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.933 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20930 2024-09-14T09:06:49.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.933 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:49.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:49.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672123+0000", 2024-09-14T09:06:49.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:06:49.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672125+0000", 2024-09-14T09:06:49.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7300000000000001e-06 2024-09-14T09:06:49.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672134+0000", 2024-09-14T09:06:49.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7930000000000003e-06 2024-09-14T09:06:49.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672141+0000", 2024-09-14T09:06:49.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9029999999999998e-06 2024-09-14T09:06:49.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695087+0000", 2024-09-14T09:06:49.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022946013000000001 2024-09-14T09:06:49.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695350+0000", 2024-09-14T09:06:49.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026266000000000001 2024-09-14T09:06:49.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696425+0000", 2024-09-14T09:06:49.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001075527 2024-09-14T09:06:49.936 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.936 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.936 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.936 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20931 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:49.936 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:49.936 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.218857438000001, 2024-09-14T09:06:49.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025331519, 2024-09-14T09:06:49.937 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.937 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.937 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.937 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.937 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.937 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.937 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20931 2024-09-14T09:06:49.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.937 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:49.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:49.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672156+0000", 2024-09-14T09:06:49.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4300000000000002e-07 2024-09-14T09:06:49.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672159+0000", 2024-09-14T09:06:49.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T09:06:49.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672170+0000", 2024-09-14T09:06:49.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1192999999999999e-05 2024-09-14T09:06:49.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672177+0000", 2024-09-14T09:06:49.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9059999999999996e-06 2024-09-14T09:06:49.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696532+0000", 2024-09-14T09:06:49.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024355291000000001 2024-09-14T09:06:49.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696755+0000", 2024-09-14T09:06:49.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022297699999999999 2024-09-14T09:06:49.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697487+0000", 2024-09-14T09:06:49.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00073171499999999997 2024-09-14T09:06:49.941 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.941 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.941 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.941 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20932 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1179648~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:49.941 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:49.941 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.218823995000001, 2024-09-14T09:06:49.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026104083, 2024-09-14T09:06:49.941 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.941 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.942 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.942 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.942 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.942 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.942 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20932 2024-09-14T09:06:49.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.942 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:49.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:49.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:49.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0699999999999996e-07 2024-09-14T09:06:49.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672192+0000", 2024-09-14T09:06:49.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7700000000000002e-06 2024-09-14T09:06:49.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672203+0000", 2024-09-14T09:06:49.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0956e-05 2024-09-14T09:06:49.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672211+0000", 2024-09-14T09:06:49.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3769999999999996e-06 2024-09-14T09:06:49.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697530+0000", 2024-09-14T09:06:49.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025318411999999998 2024-09-14T09:06:49.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697640+0000", 2024-09-14T09:06:49.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011045299999999999 2024-09-14T09:06:49.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698293+0000", 2024-09-14T09:06:49.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065240800000000005 2024-09-14T09:06:49.945 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.945 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.946 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.946 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20933 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3014656~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:49.946 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:49.946 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.218789792000003, 2024-09-14T09:06:49.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026558875999999999, 2024-09-14T09:06:49.946 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.946 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.946 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.946 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20933 2024-09-14T09:06:49.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.947 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:49.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:49.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672224+0000", 2024-09-14T09:06:49.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2399999999999997e-07 2024-09-14T09:06:49.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672234+0000", 2024-09-14T09:06:49.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0793e-05 2024-09-14T09:06:49.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672246+0000", 2024-09-14T09:06:49.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659e-05 2024-09-14T09:06:49.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672255+0000", 2024-09-14T09:06:49.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5730000000000008e-06 2024-09-14T09:06:49.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698326+0000", 2024-09-14T09:06:49.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026071414000000001 2024-09-14T09:06:49.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698374+0000", 2024-09-14T09:06:49.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8032999999999999e-05 2024-09-14T09:06:49.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698782+0000", 2024-09-14T09:06:49.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040768 2024-09-14T09:06:49.950 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.950 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.950 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.950 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20934 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1310720~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:49.950 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:49.950 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.218747915999998, 2024-09-14T09:06:49.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027292067999999999, 2024-09-14T09:06:49.950 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.950 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.950 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.951 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.951 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.951 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.951 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20934 2024-09-14T09:06:49.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.951 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:49.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:49.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672266+0000", 2024-09-14T09:06:49.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6299999999999993e-07 2024-09-14T09:06:49.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672269+0000", 2024-09-14T09:06:49.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8569999999999999e-06 2024-09-14T09:06:49.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672280+0000", 2024-09-14T09:06:49.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1236999999999999e-05 2024-09-14T09:06:49.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672287+0000", 2024-09-14T09:06:49.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8260000000000004e-06 2024-09-14T09:06:49.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698820+0000", 2024-09-14T09:06:49.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026533055999999999 2024-09-14T09:06:49.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698933+0000", 2024-09-14T09:06:49.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011291 2024-09-14T09:06:49.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699557+0000", 2024-09-14T09:06:49.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062421899999999997 2024-09-14T09:06:49.954 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.954 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.954 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.954 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20935 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:49.954 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:49.954 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.218714155999997, 2024-09-14T09:06:49.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028108610999999999, 2024-09-14T09:06:49.955 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.955 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.955 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.955 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.955 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.955 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.955 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20935 2024-09-14T09:06:49.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.955 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:49.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:49.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672299+0000", 2024-09-14T09:06:49.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T09:06:49.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672302+0000", 2024-09-14T09:06:49.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7640000000000001e-06 2024-09-14T09:06:49.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672313+0000", 2024-09-14T09:06:49.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1080000000000001e-05 2024-09-14T09:06:49.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672321+0000", 2024-09-14T09:06:49.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6490000000000008e-06 2024-09-14T09:06:49.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699587+0000", 2024-09-14T09:06:49.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027265884000000001 2024-09-14T09:06:49.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699631+0000", 2024-09-14T09:06:49.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4947999999999997e-05 2024-09-14T09:06:49.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700407+0000", 2024-09-14T09:06:49.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00077559299999999999 2024-09-14T09:06:49.958 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.958 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.958 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.958 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20936 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1441792~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:49.958 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:49.958 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.218679979999997, 2024-09-14T09:06:49.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028784750000000001, 2024-09-14T09:06:49.959 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.959 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.959 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.959 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20936 2024-09-14T09:06:49.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.959 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:49.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:49.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:49.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4700000000000001e-07 2024-09-14T09:06:49.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672336+0000", 2024-09-14T09:06:49.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6730000000000001e-06 2024-09-14T09:06:49.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672347+0000", 2024-09-14T09:06:49.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0710000000000001e-05 2024-09-14T09:06:49.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672354+0000", 2024-09-14T09:06:49.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4630000000000004e-06 2024-09-14T09:06:49.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700442+0000", 2024-09-14T09:06:49.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028087951 2024-09-14T09:06:49.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700557+0000", 2024-09-14T09:06:49.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011461800000000001 2024-09-14T09:06:49.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701117+0000", 2024-09-14T09:06:49.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056058799999999999 2024-09-14T09:06:49.962 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.962 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.962 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.963 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20937 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:49.963 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:49.963 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.218646853999999, 2024-09-14T09:06:49.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029493028000000001, 2024-09-14T09:06:49.963 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.963 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.963 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.963 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20937 2024-09-14T09:06:49.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.964 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:49.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:49.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:49.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T09:06:49.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672384+0000", 2024-09-14T09:06:49.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7127000000000001e-05 2024-09-14T09:06:49.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672395+0000", 2024-09-14T09:06:49.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759999999999999e-05 2024-09-14T09:06:49.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672402+0000", 2024-09-14T09:06:49.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0960000000000003e-06 2024-09-14T09:06:49.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701147+0000", 2024-09-14T09:06:49.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028744241 2024-09-14T09:06:49.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701246+0000", 2024-09-14T09:06:49.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9031999999999996e-05 2024-09-14T09:06:49.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701859+0000", 2024-09-14T09:06:49.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061310499999999997 2024-09-14T09:06:49.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.967 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.967 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20938 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:49.967 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:49.967 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.218599693999998, 2024-09-14T09:06:49.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030109219999999999, 2024-09-14T09:06:49.967 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.967 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.967 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.968 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20938 2024-09-14T09:06:49.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.968 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:49.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:49.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672414+0000", 2024-09-14T09:06:49.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-07 2024-09-14T09:06:49.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672417+0000", 2024-09-14T09:06:49.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:06:49.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672428+0000", 2024-09-14T09:06:49.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0873000000000001e-05 2024-09-14T09:06:49.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672435+0000", 2024-09-14T09:06:49.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:06:49.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701884+0000", 2024-09-14T09:06:49.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029449811999999999 2024-09-14T09:06:49.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701934+0000", 2024-09-14T09:06:49.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9703000000000002e-05 2024-09-14T09:06:49.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702522+0000", 2024-09-14T09:06:49.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00058811200000000001 2024-09-14T09:06:49.971 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.971 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.971 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.972 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20939 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:49.972 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:49.972 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.218567561, 2024-09-14T09:06:49.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030779259, 2024-09-14T09:06:49.972 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.972 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.972 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.972 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.972 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.972 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.972 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20939 2024-09-14T09:06:49.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.972 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:49.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:49.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672446+0000", 2024-09-14T09:06:49.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:06:49.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672449+0000", 2024-09-14T09:06:49.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7930000000000002e-06 2024-09-14T09:06:49.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672461+0000", 2024-09-14T09:06:49.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T09:06:49.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672471+0000", 2024-09-14T09:06:49.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0807e-05 2024-09-14T09:06:49.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702551+0000", 2024-09-14T09:06:49.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030079228 2024-09-14T09:06:49.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702664+0000", 2024-09-14T09:06:49.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011351800000000001 2024-09-14T09:06:49.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.703224+0000", 2024-09-14T09:06:49.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056026999999999997 2024-09-14T09:06:49.976 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.976 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.976 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.976 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:72788 3.1f e74/28 3:fa166c1d:::rbd_data.115ea8d131ff.0000000000000003:head v 74'947, mlcod=74'945)", 2024-09-14T09:06:49.976 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.121215+0000", 2024-09-14T09:06:49.976 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.769797505, 2024-09-14T09:06:49.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017555599000000002, 2024-09-14T09:06:49.976 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.977 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.977 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:49.977 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.121215+0000", 2024-09-14T09:06:49.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.121215+0000", 2024-09-14T09:06:49.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.121216+0000", 2024-09-14T09:06:49.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9599999999999998e-07 2024-09-14T09:06:49.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.123528+0000", 2024-09-14T09:06:49.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0023118050000000001 2024-09-14T09:06:49.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.123544+0000", 2024-09-14T09:06:49.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6294e-05 2024-09-14T09:06:49.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.123552+0000", 2024-09-14T09:06:49.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9659999999999996e-06 2024-09-14T09:06:49.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.123624+0000", 2024-09-14T09:06:49.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1766000000000004e-05 2024-09-14T09:06:49.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.123653+0000", 2024-09-14T09:06:49.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8858000000000001e-05 2024-09-14T09:06:49.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:49.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.138740+0000", 2024-09-14T09:06:49.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015087194 2024-09-14T09:06:49.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.138771+0000", 2024-09-14T09:06:49.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.082e-05 2024-09-14T09:06:49.980 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.980 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.980 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.980 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:74298 3.9 3:95c8282a:::rbd_data.115ea8d131ff.0000000000000041:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:49.980 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:06:49.981 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.093257549, 2024-09-14T09:06:49.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042668064999999998, 2024-09-14T09:06:49.981 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.981 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.981 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:49.981 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:49.981 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:49.981 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:49.981 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74298 2024-09-14T09:06:49.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.981 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:06:49.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:06:49.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797758+0000", 2024-09-14T09:06:49.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9799999999999998e-06 2024-09-14T09:06:49.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797770+0000", 2024-09-14T09:06:49.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1667e-05 2024-09-14T09:06:49.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797797+0000", 2024-09-14T09:06:49.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7212999999999998e-05 2024-09-14T09:06:49.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797811+0000", 2024-09-14T09:06:49.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4256e-05 2024-09-14T09:06:49.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797882+0000", 2024-09-14T09:06:49.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0622000000000002e-05 2024-09-14T09:06:49.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797943+0000", 2024-09-14T09:06:49.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0893000000000001e-05 2024-09-14T09:06:49.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:49.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.799676+0000", 2024-09-14T09:06:49.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0017331619999999999 2024-09-14T09:06:49.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:49.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840109+0000", 2024-09-14T09:06:49.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040432994 2024-09-14T09:06:49.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:49.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840204+0000", 2024-09-14T09:06:49.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5247000000000005e-05 2024-09-14T09:06:49.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:49.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840346+0000", 2024-09-14T09:06:49.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014156499999999999 2024-09-14T09:06:49.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840423+0000", 2024-09-14T09:06:49.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7466000000000007e-05 2024-09-14T09:06:49.986 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.986 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.986 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.986 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:74298 3.9 e76/34 ondisk, result = 0)", 2024-09-14T09:06:49.986 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:06:49.986 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.086862906, 2024-09-14T09:06:49.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036289318000000001, 2024-09-14T09:06:49.986 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:49.986 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:49.987 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:49.987 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:49.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:49.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:06:49.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:49.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:06:49.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:49.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:49.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804151+0000", 2024-09-14T09:06:49.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6700000000000001e-06 2024-09-14T09:06:49.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:49.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804163+0000", 2024-09-14T09:06:49.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.148e-05 2024-09-14T09:06:49.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:49.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804197+0000", 2024-09-14T09:06:49.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3909999999999999e-05 2024-09-14T09:06:49.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:49.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804217+0000", 2024-09-14T09:06:49.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0287000000000001e-05 2024-09-14T09:06:49.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:49.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840176+0000", 2024-09-14T09:06:49.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035958641 2024-09-14T09:06:49.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:49.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840193+0000", 2024-09-14T09:06:49.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6986e-05 2024-09-14T09:06:49.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:49.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:49.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:49.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840439+0000", 2024-09-14T09:06:49.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024634400000000001 2024-09-14T09:06:49.990 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.990 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.990 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.990 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:49.990 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:49.990 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:49.990 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:06:49.992 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:06:49.992 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:49.991+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:50.092 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:50.091+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:50.158 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:50.158 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:50.158 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:50.158 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:50.173 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:06:50.177 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:50.177 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:50.178 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-14T09:06:50.181 INFO:teuthology.orchestra.run.smithi176.stdout:3854 read 0x1dd372c thru 0x1dd52c0 (0x1b95 bytes) 2024-09-14T09:06:50.182 INFO:teuthology.orchestra.run.smithi176.stdout:3855 writesame 0x3ead711 thru 0x3eb894e data_size 0x5a(0xb23e bytes) 2024-09-14T09:06:50.192 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:50.191+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:50.200 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:50.200 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:50.219 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:50.219 INFO:teuthology.orchestra.run.smithi176.stdout:3859 read 0x938d0ba thru 0x9394fc9 (0x7f10 bytes) 2024-09-14T09:06:50.219 INFO:teuthology.orchestra.run.smithi176.stdout:3860 punch from 0xd5a201 to 0xd67662, (0xd461 bytes) 2024-09-14T09:06:50.219 INFO:teuthology.orchestra.run.smithi176.stdout:3861 write 0x4afca93 thru 0x4b04885 (0x7df3 bytes) 2024-09-14T09:06:50.228 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:50.228 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:50.231 INFO:teuthology.orchestra.run.smithi176.stdout:3862 punch from 0x8c20717 to 0x8c2ea56, (0xe33f bytes) 2024-09-14T09:06:50.231 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:50.231 INFO:teuthology.orchestra.run.smithi176.stdout:3865 punch from 0x6b11675 to 0x6b1a39a, (0x8d25 bytes) 2024-09-14T09:06:50.231 INFO:teuthology.orchestra.run.smithi176.stdout:3866 read 0x452d4b1 thru 0x4533561 (0x60b1 bytes) 2024-09-14T09:06:50.232 INFO:teuthology.orchestra.run.smithi176.stdout:3867 write 0x7ee7b73 thru 0x7ef749a (0xf928 bytes) 2024-09-14T09:06:50.245 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:50.246 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:50.250 INFO:teuthology.orchestra.run.smithi176.stdout:3869 write 0x88ddf7 thru 0x89356e (0x5778 bytes) 2024-09-14T09:06:50.274 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:50.274 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:50.294 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:50.295+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:50.297 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:50.297 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:50.302 INFO:teuthology.orchestra.run.smithi176.stdout:3870 read 0xcc80a9e thru 0xcc89238 (0x879b bytes) 2024-09-14T09:06:50.302 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:50.302 INFO:teuthology.orchestra.run.smithi176.stdout:3873 write 0x644aaf5 thru 0x64571a9 (0xc6b5 bytes) 2024-09-14T09:06:50.308 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:50.308 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:50.320 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:50.320 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:50.321 INFO:teuthology.orchestra.run.smithi176.stdout:3874 trunc from 0xcdb5613 to 0x7fab53c 2024-09-14T09:06:50.330 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:06:50.335 INFO:teuthology.orchestra.run.smithi176.stdout:3875 write 0x483426 thru 0x4913fb (0xdfd6 bytes) 2024-09-14T09:06:50.363 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:50.363 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:50.392 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:50.393 INFO:teuthology.orchestra.run.smithi176.stdout:3877 read 0x2419d13 thru 0x241c27e (0x256c bytes) 2024-09-14T09:06:50.393 INFO:teuthology.orchestra.run.smithi176.stdout:3878 write 0x2ce63f1 thru 0x2cf2a98 (0xc6a8 bytes) 2024-09-14T09:06:50.395 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:50.399+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:50.403 INFO:teuthology.orchestra.run.smithi176.stdout:3879 read 0x4675144 thru 0x467cafb (0x79b8 bytes) 2024-09-14T09:06:50.403 INFO:teuthology.orchestra.run.smithi176.stdout:3880 clone 29 order 19 su 65536 sc 7 2024-09-14T09:06:50.455 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:50.455 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:50.468 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:06:50.515 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:50.519+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:50.596 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:50.595+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:50.599 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:50.599 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:50.614 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:06:50.703 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:50.707+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:50.778 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:50.778 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:50.792 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:06:50.839 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:50.843+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:50.899 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:50.903+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:50.910 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T09:06:50.970 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:50.970 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:50.986 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:06:51.017 INFO:teuthology.orchestra.run.smithi176.stdout:leaving image image_client.0-clone28 intact 2024-09-14T09:06:51.031 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:51.035+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:51.100 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:51.103+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:51.149 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-14T09:06:51.158 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:51.158 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:51.173 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:06:51.215 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:51.219+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:51.302 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:51.303+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:51.349 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:51.349 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:51.363 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:06:51.404 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:51.403+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:51.505 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:51.507+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:51.532 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:51.532 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:51.546 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:06:51.571 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:51.590 INFO:teuthology.orchestra.run.smithi176.stdout:3883 write 0xb17be4d thru 0xb17f345 (0x34f9 bytes) 2024-09-14T09:06:51.598 INFO:teuthology.orchestra.run.smithi176.stdout:3884 trunc from 0xb17f346 to 0xb994382 2024-09-14T09:06:51.606 INFO:teuthology.orchestra.run.smithi176.stdout:3885 read 0x27384b4 thru 0x2741624 (0x9171 bytes) 2024-09-14T09:06:51.606 INFO:teuthology.orchestra.run.smithi176.stdout:3886 write 0x40274a1 thru 0x4031b8c (0xa6ec bytes) 2024-09-14T09:06:51.621 INFO:teuthology.orchestra.run.smithi176.stdout:3887 read 0x5220476 thru 0x522468d (0x4218 bytes) 2024-09-14T09:06:51.622 INFO:teuthology.orchestra.run.smithi176.stdout:3891 read 0x1674048 thru 0x168210f (0xe0c8 bytes) 2024-09-14T09:06:51.631 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:51.635+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:51.632 INFO:teuthology.orchestra.run.smithi176.stdout:3892 write 0xc647838 thru 0xc651960 (0xa129 bytes) 2024-09-14T09:06:51.639 INFO:teuthology.orchestra.run.smithi176.stdout:3893 punch from 0x44bac05 to 0x44c4845, (0x9c40 bytes) 2024-09-14T09:06:51.646 INFO:teuthology.orchestra.run.smithi176.stdout:3894 trunc from 0xc651961 to 0x91677f4 2024-09-14T09:06:51.664 INFO:teuthology.orchestra.run.smithi176.stdout:3895 write 0x8b67c09 thru 0x8b70739 (0x8b31 bytes) 2024-09-14T09:06:51.673 INFO:teuthology.orchestra.run.smithi176.stdout:3896 read 0x76f90c6 thru 0x76fad56 (0x1c91 bytes) 2024-09-14T09:06:51.674 INFO:teuthology.orchestra.run.smithi176.stdout:3897 punch from 0x50f0950 to 0x50fa63c, (0x9cec bytes) 2024-09-14T09:06:51.696 INFO:teuthology.orchestra.run.smithi176.stdout:3898 writesame 0xed00934 thru 0xed0ff16 data_size 0x1(0xf5e3 bytes) 2024-09-14T09:06:51.696 INFO:teuthology.orchestra.run.smithi176.stdout:3899 punch from 0xd1e9121 to 0xd1efc65, (0x6b44 bytes) 2024-09-14T09:06:51.696 INFO:teuthology.orchestra.run.smithi176.stdout:3901 write 0x3bea03 thru 0x3c14b9 (0x2ab7 bytes) 2024-09-14T09:06:51.705 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:51.707+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:51.718 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:51.718 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:51.723 INFO:teuthology.orchestra.run.smithi176.stdout:3903 punch from 0x1ed5034 to 0x1edf4c8, (0xa494 bytes) 2024-09-14T09:06:51.723 INFO:teuthology.orchestra.run.smithi176.stdout:3904 write 0xbe37673 thru 0xbe455d1 (0xdf5f bytes) 2024-09-14T09:06:51.733 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:06:51.734 INFO:teuthology.orchestra.run.smithi176.stdout:3905 read 0xc2d23ba thru 0xc2d512f (0x2d76 bytes) 2024-09-14T09:06:51.734 INFO:teuthology.orchestra.run.smithi176.stdout:3906 read 0x89ed49f thru 0x89fd30a (0xfe6c bytes) 2024-09-14T09:06:51.757 INFO:teuthology.orchestra.run.smithi176.stdout:3907 trunc from 0xed0ff17 to 0x616cf2f 2024-09-14T09:06:51.814 INFO:teuthology.orchestra.run.smithi176.stdout:3908 trunc from 0x616cf2f to 0xad38b45 2024-09-14T09:06:51.823 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:51.827+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:51.827 INFO:teuthology.orchestra.run.smithi176.stdout:3910 write 0x36ca99e thru 0x36cede5 (0x4448 bytes) 2024-09-14T09:06:51.837 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:51.837 INFO:teuthology.orchestra.run.smithi176.stdout:3914 read 0x1f4440a thru 0x1f4c463 (0x805a bytes) 2024-09-14T09:06:51.837 INFO:teuthology.orchestra.run.smithi176.stdout:3916 punch from 0x419aa38 to 0x41aa5a4, (0xfb6c bytes) 2024-09-14T09:06:51.854 INFO:teuthology.orchestra.run.smithi176.stdout:3917 write 0xc47b203 thru 0xc47d256 (0x2054 bytes) 2024-09-14T09:06:51.861 INFO:teuthology.orchestra.run.smithi176.stdout:3918 read 0x92bbc76 thru 0x92c8eab (0xd236 bytes) 2024-09-14T09:06:51.861 INFO:teuthology.orchestra.run.smithi176.stdout:3919 read 0x5184657 thru 0x5185ebf (0x1869 bytes) 2024-09-14T09:06:51.861 INFO:teuthology.orchestra.run.smithi176.stdout:3920 read 0xaf77edf thru 0xaf82f8e (0xb0b0 bytes) 2024-09-14T09:06:51.863 INFO:teuthology.orchestra.run.smithi176.stdout:3922 trunc from 0xc47d257 to 0xd087998 2024-09-14T09:06:51.877 INFO:teuthology.orchestra.run.smithi176.stdout:3923 read 0x8343823 thru 0x834de3b (0xa619 bytes) 2024-09-14T09:06:51.877 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:51.892 INFO:teuthology.orchestra.run.smithi176.stdout:3927 write 0xd71c0dd thru 0xd71fe69 (0x3d8d bytes) 2024-09-14T09:06:51.904 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:51.905 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:51.907 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:51.907+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:51.910 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is set 2024-09-14T09:06:51.918 INFO:teuthology.orchestra.run.smithi176.stdout:3928 write 0xda4c610 thru 0xda5b467 (0xee58 bytes) 2024-09-14T09:06:51.922 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:06:51.925 INFO:teuthology.orchestra.run.smithi176.stdout:3929 punch from 0x97f6e49 to 0x97fb984, (0x4b3b bytes) 2024-09-14T09:06:51.925 INFO:teuthology.orchestra.run.smithi176.stdout:3930 read 0x371ecbf thru 0x372430a (0x564c bytes) 2024-09-14T09:06:51.926 INFO:teuthology.orchestra.run.smithi176.stdout:3931 write 0x4199a7e thru 0x41a08c1 (0x6e44 bytes) 2024-09-14T09:06:51.937 INFO:teuthology.orchestra.run.smithi176.stdout:3932 write 0x8884a05 thru 0x888b73b (0x6d37 bytes) 2024-09-14T09:06:51.941 INFO:teuthology.orchestra.run.smithi176.stdout:3933 read 0x198f505 thru 0x1993c6e (0x476a bytes) 2024-09-14T09:06:51.941 INFO:teuthology.orchestra.run.smithi176.stdout:3935 write 0x6b619d3 thru 0x6b637d5 (0x1e03 bytes) 2024-09-14T09:06:51.945 INFO:teuthology.orchestra.run.smithi176.stdout:3936 write 0x11a9872 thru 0x11b0114 (0x68a3 bytes) 2024-09-14T09:06:51.952 INFO:teuthology.orchestra.run.smithi176.stdout:3939 write 0xbc2cc7b thru 0xbc3c612 (0xf998 bytes) 2024-09-14T09:06:51.960 INFO:teuthology.orchestra.run.smithi176.stdout:3942 read 0xafd8a57 thru 0xafdd9e8 (0x4f92 bytes) 2024-09-14T09:06:51.974 INFO:teuthology.orchestra.run.smithi176.stdout:3943 trunc from 0xda5b468 to 0x50a3c45 2024-09-14T09:06:52.001 INFO:teuthology.orchestra.run.smithi176.stdout:3944 punch from 0x4728067 to 0x47334f1, (0xb48a bytes) 2024-09-14T09:06:52.009 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:52.007+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:52.021 INFO:teuthology.orchestra.run.smithi176.stdout:3945 writesame 0xbb41a3d thru 0xbb440fc data_size 0x28(0x26c0 bytes) 2024-09-14T09:06:52.044 INFO:teuthology.orchestra.run.smithi176.stdout:3946 trunc from 0xbb440fd to 0x10be6a5 2024-09-14T09:06:52.089 INFO:teuthology.orchestra.run.smithi176.stdout:3947 punch from 0x9a263 to 0xa8e7c, (0xec19 bytes) 2024-09-14T09:06:52.092 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:52.092 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:52.106 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:06:52.110 INFO:teuthology.orchestra.run.smithi176.stdout:3948 write 0x4903afa thru 0x490a898 (0x6d9f bytes) 2024-09-14T09:06:52.110 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:52.111+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:52.136 INFO:teuthology.orchestra.run.smithi176.stdout:3949 write 0x7b6166a thru 0x7b70e87 (0xf81e bytes) 2024-09-14T09:06:52.145 INFO:teuthology.orchestra.run.smithi176.stdout:3950 read 0x3e4fcef thru 0x3e50e21 (0x1133 bytes) 2024-09-14T09:06:52.148 INFO:teuthology.orchestra.run.smithi176.stdout:3953 trunc from 0x7b70e88 to 0x8aacae0 2024-09-14T09:06:52.188 INFO:teuthology.orchestra.run.smithi176.stdout:3954 write 0xd6ca79e thru 0xd6cd844 (0x30a7 bytes) 2024-09-14T09:06:52.211 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:52.211+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:52.215 INFO:teuthology.orchestra.run.smithi176.stdout:3955 trunc from 0xd6cd845 to 0x5134ba4 2024-09-14T09:06:52.233 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:52.233 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:52.238 INFO:teuthology.orchestra.run.smithi176.stdout:3956 write 0x97923e thru 0x97cddf (0x3ba2 bytes) 2024-09-14T09:06:52.248 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:06:52.255 INFO:teuthology.orchestra.run.smithi176.stdout:3957 punch from 0xc7b816 to 0xc7d2d0, (0x1aba bytes) 2024-09-14T09:06:52.255 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:52.266 INFO:teuthology.orchestra.run.smithi176.stdout:3960 trunc from 0x5134ba4 to 0xd4a896 2024-09-14T09:06:52.311 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:52.311+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:52.347 INFO:teuthology.orchestra.run.smithi176.stdout:3961 write 0x7c2071f thru 0x7c27aa6 (0x7388 bytes) 2024-09-14T09:06:52.374 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:52.374 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:52.381 INFO:teuthology.orchestra.run.smithi176.stdout:3963 writesame 0xdf1d7c3 thru 0xdf21b3a data_size 0x22(0x4378 bytes) 2024-09-14T09:06:52.381 INFO:teuthology.orchestra.run.smithi176.stdout:3964 punch from 0x32ba731 to 0x32c3835, (0x9104 bytes) 2024-09-14T09:06:52.381 INFO:teuthology.orchestra.run.smithi176.stdout:3966 writesame 0x4002319 thru 0x40083a8 data_size 0x67(0x6090 bytes) 2024-09-14T09:06:52.388 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:06:52.389 INFO:teuthology.orchestra.run.smithi176.stdout:3967 writesame 0x590d9e3 thru 0x5916ef2 data_size 0x48(0x9510 bytes) 2024-09-14T09:06:52.390 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:06:52.397 INFO:teuthology.orchestra.run.smithi176.stdout:3968 write 0x89c6928 thru 0x89c7fe8 (0x16c1 bytes) 2024-09-14T09:06:52.402 INFO:teuthology.orchestra.run.smithi176.stdout:3969 read 0xc0214c3 thru 0xc02d539 (0xc077 bytes) 2024-09-14T09:06:52.409 INFO:teuthology.orchestra.run.smithi176.stdout:3970 trunc from 0xdf21b3b to 0xb25d372 2024-09-14T09:06:52.417 INFO:teuthology.orchestra.run.smithi176.stdout:3972 write 0x7252a0b thru 0x7256455 (0x3a4b bytes) 2024-09-14T09:06:52.435 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:52.439+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:52.437 INFO:teuthology.orchestra.run.smithi176.stdout:3973 writesame 0xeb32273 thru 0xeb3bd2f data_size 0x1(0x9abd bytes) 2024-09-14T09:06:52.437 INFO:teuthology.orchestra.run.smithi176.stdout:3974 write 0xaa456a2 thru 0xaa4ee90 (0x97ef bytes) 2024-09-14T09:06:52.443 INFO:teuthology.orchestra.run.smithi176.stdout:3975 writesame 0xa20c6a8 thru 0xa20ce8d data_size 0x3(0x7e6 bytes) 2024-09-14T09:06:52.448 INFO:teuthology.orchestra.run.smithi176.stdout:3976 write 0xf3a39c thru 0xf3e274 (0x3ed9 bytes) 2024-09-14T09:06:52.456 INFO:teuthology.orchestra.run.smithi176.stdout:3977 trunc from 0xeb3bd30 to 0xd999db6 2024-09-14T09:06:52.466 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:52.466 INFO:teuthology.orchestra.run.smithi176.stdout:3980 read 0xa6151a8 thru 0xa615482 (0x2db bytes) 2024-09-14T09:06:52.466 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:52.466 INFO:teuthology.orchestra.run.smithi176.stdout:3982 punch from 0xa9b3fa6 to 0xa9b5ffa, (0x2054 bytes) 2024-09-14T09:06:52.466 INFO:teuthology.orchestra.run.smithi176.stdout:3983 trunc from 0xd999db6 to 0xd29ba61 2024-09-14T09:06:52.475 INFO:teuthology.orchestra.run.smithi176.stdout:3985 write 0x500888a thru 0x500e45c (0x5bd3 bytes) 2024-09-14T09:06:52.483 INFO:teuthology.orchestra.run.smithi176.stdout:3990 read 0x2c66ef7 thru 0x2c747b2 (0xd8bc bytes) 2024-09-14T09:06:52.483 INFO:teuthology.orchestra.run.smithi176.stdout:3991 read 0x422040 thru 0x426897 (0x4858 bytes) 2024-09-14T09:06:52.484 INFO:teuthology.orchestra.run.smithi176.stdout:3992 write 0xe265c2 thru 0xe2a265 (0x3ca4 bytes) 2024-09-14T09:06:52.490 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:52.490 INFO:teuthology.orchestra.run.smithi176.stdout:3994 write 0x719af76 thru 0x71a5ba0 (0xac2b bytes) 2024-09-14T09:06:52.499 INFO:teuthology.orchestra.run.smithi176.stdout:3996 read 0xa4e198b thru 0xa4ee796 (0xce0c bytes) 2024-09-14T09:06:52.506 INFO:teuthology.orchestra.run.smithi176.stdout:3998 writesame 0xd3cd3bd thru 0xd3d6903 data_size 0x1(0x9547 bytes) 2024-09-14T09:06:52.507 INFO:teuthology.orchestra.run.smithi176.stdout:3999 write 0xc482282 thru 0xc48639b (0x411a bytes) 2024-09-14T09:06:52.512 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:52.512 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:52.513 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:52.513 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:52.513 INFO:teuthology.orchestra.run.smithi176.stdout:4003 read 0x7060426 thru 0x706c761 (0xc33c bytes) 2024-09-14T09:06:52.513 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:52.515+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:52.527 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:06:52.534 INFO:teuthology.orchestra.run.smithi176.stdout:4004 trunc from 0xd3d6904 to 0x3d1dc4f 2024-09-14T09:06:52.560 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:52.560 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:52.564 INFO:teuthology.orchestra.run.smithi176.stdout:4005 write 0x981aa25 thru 0x982133e (0x691a bytes) 2024-09-14T09:06:52.569 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:52.570 INFO:teuthology.orchestra.run.smithi176.stdout:4008 punch from 0x556843d to 0x5571783, (0x9346 bytes) 2024-09-14T09:06:52.579 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:52.580 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:52.580 INFO:teuthology.orchestra.run.smithi176.stdout:4009 trunc from 0x982133f to 0x2c2045b 2024-09-14T09:06:52.594 INFO:teuthology.orchestra.run.smithi176.stdout:4010 punch from 0x248a3a to 0x2529c0, (0x9f86 bytes) 2024-09-14T09:06:52.594 INFO:teuthology.orchestra.run.smithi176.stdout:4012 read 0x275ac55 thru 0x2768a99 (0xde45 bytes) 2024-09-14T09:06:52.599 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:52.599 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:52.614 INFO:teuthology.orchestra.run.smithi176.stdout:4013 write 0xb8ea60a thru 0xb8ed7b5 (0x31ac bytes) 2024-09-14T09:06:52.614 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:52.615+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:52.618 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:52.618 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:52.619 INFO:teuthology.orchestra.run.smithi176.stdout:4014 read 0x271a337 thru 0x271b142 (0xe0c bytes) 2024-09-14T09:06:52.619 INFO:teuthology.orchestra.run.smithi176.stdout:4015 writesame 0x8e70a87 thru 0x8e7d43c data_size 0x1(0xc9b6 bytes) 2024-09-14T09:06:52.637 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:52.637 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:52.658 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:52.659 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:52.659 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:52.659 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:52.659 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:52.659 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:52.676 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:06:52.680 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:52.680 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:52.701 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:52.702 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:52.715 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:52.715+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:52.738 INFO:teuthology.orchestra.run.smithi176.stdout:4017 writesame 0x267c8ab thru 0x2680bc4 data_size 0x15(0x431a bytes) 2024-09-14T09:06:52.746 INFO:teuthology.orchestra.run.smithi176.stdout:4018 write 0x92363f5 thru 0x923777f (0x138b bytes) 2024-09-14T09:06:52.752 INFO:teuthology.orchestra.run.smithi176.stdout:4019 punch from 0x92117b7 to 0x921291a, (0x1163 bytes) 2024-09-14T09:06:52.754 INFO:teuthology.orchestra.run.smithi176.stdout:4020 trunc from 0xb8ed7b6 to 0xc9ae0a1 2024-09-14T09:06:52.761 INFO:teuthology.orchestra.run.smithi176.stdout:4021 punch from 0x4855683 to 0x485ab47, (0x54c4 bytes) 2024-09-14T09:06:52.762 INFO:teuthology.orchestra.run.smithi176.stdout:4022 write 0x2da82f4 thru 0x2da8f33 (0xc40 bytes) 2024-09-14T09:06:52.768 INFO:teuthology.orchestra.run.smithi176.stdout:4023 read 0x2859929 thru 0x2864c36 (0xb30e bytes) 2024-09-14T09:06:52.768 INFO:teuthology.orchestra.run.smithi176.stdout:4025 write 0xa96bbd2 thru 0xa971372 (0x57a1 bytes) 2024-09-14T09:06:52.776 INFO:teuthology.orchestra.run.smithi176.stdout:4026 punch from 0x55d6861 to 0x55e3a8e, (0xd22d bytes) 2024-09-14T09:06:52.780 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:06:52.783 INFO:teuthology.orchestra.run.smithi176.stdout:4027 trunc from 0xc9ae0a1 to 0x7e827dc 2024-09-14T09:06:52.801 INFO:teuthology.orchestra.run.smithi176.stdout:4028 write 0x631e178 thru 0x63290f4 (0xaf7d bytes) 2024-09-14T09:06:52.805 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:52.805 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:52.805 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:52.806 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:52.806 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:52.815 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:52.816 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:52.815+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:52.823 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:06:52.826 INFO:teuthology.orchestra.run.smithi176.stdout:4030 write 0xaf3c961 thru 0xaf3e502 (0x1ba2 bytes) 2024-09-14T09:06:52.832 INFO:teuthology.orchestra.run.smithi176.stdout:4031 punch from 0x53489d2 to 0x534d5c3, (0x4bf1 bytes) 2024-09-14T09:06:52.832 INFO:teuthology.orchestra.run.smithi176.stdout:4032 read 0xa318986 thru 0xa327897 (0xef12 bytes) 2024-09-14T09:06:52.844 INFO:teuthology.orchestra.run.smithi176.stdout:4033 writesame 0xd19a23a thru 0xd1a92cc data_size 0x1(0xf093 bytes) 2024-09-14T09:06:52.844 INFO:teuthology.orchestra.run.smithi176.stdout:4035 read 0x351222b thru 0x351936e (0x7144 bytes) 2024-09-14T09:06:52.844 INFO:teuthology.orchestra.run.smithi176.stdout:4036 punch from 0x5ecbb51 to 0x5ecc997, (0xe46 bytes) 2024-09-14T09:06:52.851 INFO:teuthology.orchestra.run.smithi176.stdout:4037 write 0xd4a47de thru 0xd4ab18b (0x69ae bytes) 2024-09-14T09:06:52.858 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:52.858 INFO:teuthology.orchestra.run.smithi176.stdout:4039 write 0x837070d thru 0x8372b48 (0x243c bytes) 2024-09-14T09:06:52.864 INFO:teuthology.orchestra.run.smithi176.stdout:4040 read 0x2342bad thru 0x23486cd (0x5b21 bytes) 2024-09-14T09:06:52.864 INFO:teuthology.orchestra.run.smithi176.stdout:4041 read 0x1baedde thru 0x1bb1d1b (0x2f3e bytes) 2024-09-14T09:06:52.864 INFO:teuthology.orchestra.run.smithi176.stdout:4043 write 0x3924cd7 thru 0x392f2a2 (0xa5cc bytes) 2024-09-14T09:06:52.889 INFO:teuthology.orchestra.run.smithi176.stdout:4044 trunc from 0xd4ab18c to 0x306c15a 2024-09-14T09:06:52.916 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:52.919+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:52.922 INFO:teuthology.orchestra.run.smithi176.stdout:4045 write 0x5dad5d5 thru 0x5db9b4c (0xc578 bytes) 2024-09-14T09:06:52.931 INFO:teuthology.orchestra.run.smithi176.stdout:4047 read 0x595a8c thru 0x5a4c03 (0xf178 bytes) 2024-09-14T09:06:52.933 INFO:teuthology.orchestra.run.smithi176.stdout:4050 punch from 0x48b9e01 to 0x48c76f3, (0xd8f2 bytes) 2024-09-14T09:06:52.944 INFO:teuthology.orchestra.run.smithi176.stdout:4051 trunc from 0x5db9b4d to 0xd3f23e5 2024-09-14T09:06:52.951 INFO:teuthology.orchestra.run.smithi176.stdout:4052 writesame 0xc776ebb thru 0xc777c85 data_size 0xb(0xdcb bytes) 2024-09-14T09:06:52.957 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:52.957 INFO:teuthology.orchestra.run.smithi176.stdout:4054 read 0xb6576ab thru 0xb664335 (0xcc8b bytes) 2024-09-14T09:06:52.957 INFO:teuthology.orchestra.run.smithi176.stdout:4056 write 0x1a2389a thru 0x1a331b4 (0xf91b bytes) 2024-09-14T09:06:52.969 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:52.969 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:52.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:52.969 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:52.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.970 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20858 3.17 3:ed5052d1:::rbd_data.115e889d4809.000000000000015c:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:52.970 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:06:52.970 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 65.539525949999998, 2024-09-14T09:06:52.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053381794000000003, 2024-09-14T09:06:52.970 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:52.970 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:52.970 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:52.970 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:52.970 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:52.970 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:52.970 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20858 2024-09-14T09:06:52.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.970 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:52.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:52.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:06:52.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:52.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:52.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:06:52.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:52.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:52.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429351+0000", 2024-09-14T09:06:52.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1149999999999998e-06 2024-09-14T09:06:52.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:52.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429363+0000", 2024-09-14T09:06:52.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1819e-05 2024-09-14T09:06:52.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:52.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429389+0000", 2024-09-14T09:06:52.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5605e-05 2024-09-14T09:06:52.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:52.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429402+0000", 2024-09-14T09:06:52.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3175e-05 2024-09-14T09:06:52.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:52.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429492+0000", 2024-09-14T09:06:52.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.988e-05 2024-09-14T09:06:52.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:52.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429597+0000", 2024-09-14T09:06:52.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010564 2024-09-14T09:06:52.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:52.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.433465+0000", 2024-09-14T09:06:52.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0038674249999999999 2024-09-14T09:06:52.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:52.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482483+0000", 2024-09-14T09:06:52.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049017999999999999 2024-09-14T09:06:52.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:52.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482625+0000", 2024-09-14T09:06:52.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000142447 2024-09-14T09:06:52.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:52.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482684+0000", 2024-09-14T09:06:52.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8522e-05 2024-09-14T09:06:52.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:52.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482730+0000", 2024-09-14T09:06:52.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6165999999999998e-05 2024-09-14T09:06:52.975 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:52.975 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:52.975 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:52.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.975 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:20858 3.17 e41/23 ondisk, result = 0)", 2024-09-14T09:06:52.976 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:06:52.976 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 65.524766506999995, 2024-09-14T09:06:52.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038631402000000002, 2024-09-14T09:06:52.976 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:52.976 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:52.976 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:52.976 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:52.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:52.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:06:52.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:52.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:52.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:06:52.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:52.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:52.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444109+0000", 2024-09-14T09:06:52.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.331e-06 2024-09-14T09:06:52.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:52.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444114+0000", 2024-09-14T09:06:52.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3709999999999999e-06 2024-09-14T09:06:52.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:52.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444130+0000", 2024-09-14T09:06:52.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6079e-05 2024-09-14T09:06:52.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:52.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444144+0000", 2024-09-14T09:06:52.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3331e-05 2024-09-14T09:06:52.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:52.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482538+0000", 2024-09-14T09:06:52.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038394878 2024-09-14T09:06:52.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:52.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482548+0000", 2024-09-14T09:06:52.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0093e-05 2024-09-14T09:06:52.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:52.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482739+0000", 2024-09-14T09:06:52.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019031899999999999 2024-09-14T09:06:52.979 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:52.979 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:52.979 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:52.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.980 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:37800 3.1 3:830c4670:::rbd_data.115eb07a8abd.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:52.980 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:06:52.980 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 55.938318539999997, 2024-09-14T09:06:52.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.068040724999999996, 2024-09-14T09:06:52.980 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:52.980 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:52.980 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:52.980 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:52.980 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:52.980 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:52.980 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 37800 2024-09-14T09:06:52.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.980 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:52.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:52.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:06:52.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:52.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:52.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:06:52.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:52.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:52.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030557+0000", 2024-09-14T09:06:52.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.888e-06 2024-09-14T09:06:52.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:52.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030566+0000", 2024-09-14T09:06:52.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8540000000000003e-06 2024-09-14T09:06:52.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:52.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030585+0000", 2024-09-14T09:06:52.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8878000000000001e-05 2024-09-14T09:06:52.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:52.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030593+0000", 2024-09-14T09:06:52.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5870000000000004e-06 2024-09-14T09:06:52.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:52.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030667+0000", 2024-09-14T09:06:52.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4207000000000003e-05 2024-09-14T09:06:52.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:52.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030761+0000", 2024-09-14T09:06:52.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4223999999999998e-05 2024-09-14T09:06:52.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:06:52.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.034400+0000", 2024-09-14T09:06:52.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0036387759999999998 2024-09-14T09:06:52.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:52.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098457+0000", 2024-09-14T09:06:52.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.064057501000000003 2024-09-14T09:06:52.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:52.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098518+0000", 2024-09-14T09:06:52.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0730999999999998e-05 2024-09-14T09:06:52.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:52.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098560+0000", 2024-09-14T09:06:52.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2267e-05 2024-09-14T09:06:52.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:52.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098596+0000", 2024-09-14T09:06:52.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5812000000000002e-05 2024-09-14T09:06:52.985 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:52.985 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:52.985 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:52.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.985 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:37800 3.1 e48/34 ondisk, result = 0)", 2024-09-14T09:06:52.985 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:06:52.986 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 55.927326923999999, 2024-09-14T09:06:52.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057056260999999997, 2024-09-14T09:06:52.986 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:52.986 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:52.986 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:52.986 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:52.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:52.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:06:52.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:52.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:52.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:06:52.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:52.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:52.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041548+0000", 2024-09-14T09:06:52.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8100000000000002e-07 2024-09-14T09:06:52.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:52.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041552+0000", 2024-09-14T09:06:52.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7419999999999999e-06 2024-09-14T09:06:52.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:52.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041563+0000", 2024-09-14T09:06:52.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1926000000000001e-05 2024-09-14T09:06:52.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:52.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041572+0000", 2024-09-14T09:06:52.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8610000000000004e-06 2024-09-14T09:06:52.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:52.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098503+0000", 2024-09-14T09:06:52.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056930827000000003 2024-09-14T09:06:52.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:52.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098512+0000", 2024-09-14T09:06:52.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5019999999999992e-06 2024-09-14T09:06:52.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:52.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098603+0000", 2024-09-14T09:06:52.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1621999999999998e-05 2024-09-14T09:06:52.990 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:52.990 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:52.990 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:52.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.991 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38523 3.3 3:c62e0486:::rbd_data.115e9e133b92.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 4194304~28499] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T09:06:52.991 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:06:52.991 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 54.514899225999997, 2024-09-14T09:06:52.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035654406999999999, 2024-09-14T09:06:52.991 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:52.992 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:52.992 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:52.992 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:52.992 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:52.992 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:52.992 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38523 2024-09-14T09:06:52.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.992 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:52.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:52.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:06:52.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:52.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:52.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:06:52.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:52.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:52.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453977+0000", 2024-09-14T09:06:52.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6299999999999998e-06 2024-09-14T09:06:52.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:52.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455684+0000", 2024-09-14T09:06:52.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001706843 2024-09-14T09:06:52.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:52.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455710+0000", 2024-09-14T09:06:52.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6302e-05 2024-09-14T09:06:52.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:52.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455722+0000", 2024-09-14T09:06:52.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1798e-05 2024-09-14T09:06:52.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:52.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455825+0000", 2024-09-14T09:06:52.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102822 2024-09-14T09:06:52.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:52.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.456032+0000", 2024-09-14T09:06:52.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020704400000000001 2024-09-14T09:06:52.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:06:52.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.456759+0000", 2024-09-14T09:06:52.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00072735699999999998 2024-09-14T09:06:52.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:52.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.477856+0000", 2024-09-14T09:06:52.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021096011000000001 2024-09-14T09:06:52.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:52.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489526+0000", 2024-09-14T09:06:52.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011670313 2024-09-14T09:06:52.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:52.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489572+0000", 2024-09-14T09:06:52.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6408e-05 2024-09-14T09:06:52.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:52.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489629+0000", 2024-09-14T09:06:52.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6879000000000002e-05 2024-09-14T09:06:52.997 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:52.997 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:52.997 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:52.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.997 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49715 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:52.997 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.524874+0000", 2024-09-14T09:06:52.997 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.444000283999998, 2024-09-14T09:06:52.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034330422999999999, 2024-09-14T09:06:52.997 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:52.997 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:52.998 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:52.998 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:52.998 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:52.998 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:52.998 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49715 2024-09-14T09:06:52.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.998 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:52.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:52.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524874+0000", 2024-09-14T09:06:52.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:52.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:52.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524874+0000", 2024-09-14T09:06:52.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:52.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:52.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524875+0000", 2024-09-14T09:06:52.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1500000000000003e-07 2024-09-14T09:06:52.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:52.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:52.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:52.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524878+0000", 2024-09-14T09:06:53.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9090000000000002e-06 2024-09-14T09:06:53.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524889+0000", 2024-09-14T09:06:53.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.007e-05 2024-09-14T09:06:53.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.524895+0000", 2024-09-14T09:06:53.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.285e-06 2024-09-14T09:06:53.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.558620+0000", 2024-09-14T09:06:53.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.033725009 2024-09-14T09:06:53.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.558681+0000", 2024-09-14T09:06:53.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1051999999999998e-05 2024-09-14T09:06:53.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559204+0000", 2024-09-14T09:06:53.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00052318299999999998 2024-09-14T09:06:53.001 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.001 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.002 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.002 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49716 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:53.002 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525151+0000", 2024-09-14T09:06:53.002 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.443722565000002, 2024-09-14T09:06:53.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034564820000000003, 2024-09-14T09:06:53.002 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.002 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.002 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.002 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.002 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.002 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.002 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49716 2024-09-14T09:06:53.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.003 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525151+0000", 2024-09-14T09:06:53.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525151+0000", 2024-09-14T09:06:53.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525152+0000", 2024-09-14T09:06:53.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5799999999999998e-07 2024-09-14T09:06:53.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525157+0000", 2024-09-14T09:06:53.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6380000000000003e-06 2024-09-14T09:06:53.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525167+0000", 2024-09-14T09:06:53.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0457e-05 2024-09-14T09:06:53.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525174+0000", 2024-09-14T09:06:53.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6220000000000003e-06 2024-09-14T09:06:53.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559237+0000", 2024-09-14T09:06:53.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034063361 2024-09-14T09:06:53.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559308+0000", 2024-09-14T09:06:53.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1124999999999994e-05 2024-09-14T09:06:53.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559716+0000", 2024-09-14T09:06:53.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000407759 2024-09-14T09:06:53.006 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.006 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.006 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.007 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49717 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:53.007 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525390+0000", 2024-09-14T09:06:53.007 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.443483768, 2024-09-14T09:06:53.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034552639000000003, 2024-09-14T09:06:53.007 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.007 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.007 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.007 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.007 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.007 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.007 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49717 2024-09-14T09:06:53.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.007 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525390+0000", 2024-09-14T09:06:53.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525390+0000", 2024-09-14T09:06:53.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525391+0000", 2024-09-14T09:06:53.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3099999999999997e-07 2024-09-14T09:06:53.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525395+0000", 2024-09-14T09:06:53.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.267e-06 2024-09-14T09:06:53.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525406+0000", 2024-09-14T09:06:53.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1224e-05 2024-09-14T09:06:53.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525413+0000", 2024-09-14T09:06:53.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5420000000000002e-06 2024-09-14T09:06:53.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559760+0000", 2024-09-14T09:06:53.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034347177999999999 2024-09-14T09:06:53.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559838+0000", 2024-09-14T09:06:53.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7708999999999997e-05 2024-09-14T09:06:53.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559943+0000", 2024-09-14T09:06:53.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010498800000000001 2024-09-14T09:06:53.011 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.011 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.011 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.011 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49718 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:53.011 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525514+0000", 2024-09-14T09:06:53.011 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.443359962999999, 2024-09-14T09:06:53.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034602766, 2024-09-14T09:06:53.011 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.011 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.011 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.011 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49718 2024-09-14T09:06:53.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.012 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525514+0000", 2024-09-14T09:06:53.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525514+0000", 2024-09-14T09:06:53.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525515+0000", 2024-09-14T09:06:53.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-07 2024-09-14T09:06:53.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525518+0000", 2024-09-14T09:06:53.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7220000000000001e-06 2024-09-14T09:06:53.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525528+0000", 2024-09-14T09:06:53.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4639999999999995e-06 2024-09-14T09:06:53.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525534+0000", 2024-09-14T09:06:53.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2330000000000003e-06 2024-09-14T09:06:53.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559974+0000", 2024-09-14T09:06:53.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034439866 2024-09-14T09:06:53.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560041+0000", 2024-09-14T09:06:53.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7343999999999998e-05 2024-09-14T09:06:53.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560117+0000", 2024-09-14T09:06:53.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5415999999999998e-05 2024-09-14T09:06:53.015 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.015 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.015 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.016 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49719 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:53.016 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:06:53.016 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.443240625000001, 2024-09-14T09:06:53.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034680563999999997, 2024-09-14T09:06:53.016 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.016 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.016 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.016 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.016 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.016 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.016 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49719 2024-09-14T09:06:53.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.016 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:06:53.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:06:53.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525634+0000", 2024-09-14T09:06:53.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4000000000000001e-07 2024-09-14T09:06:53.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525638+0000", 2024-09-14T09:06:53.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5899999999999999e-06 2024-09-14T09:06:53.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525647+0000", 2024-09-14T09:06:53.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2029999999999998e-06 2024-09-14T09:06:53.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525653+0000", 2024-09-14T09:06:53.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9850000000000003e-06 2024-09-14T09:06:53.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560150+0000", 2024-09-14T09:06:53.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034496845999999998 2024-09-14T09:06:53.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560221+0000", 2024-09-14T09:06:53.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1400000000000001e-05 2024-09-14T09:06:53.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560314+0000", 2024-09-14T09:06:53.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2800000000000006e-05 2024-09-14T09:06:53.020 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.020 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.020 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.020 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49720 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:53.020 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:06:53.020 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.442989001000001, 2024-09-14T09:06:53.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034919095999999997, 2024-09-14T09:06:53.021 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.021 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.021 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.021 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.021 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.021 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.021 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49720 2024-09-14T09:06:53.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.021 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.021 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.021 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:06:53.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:06:53.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525886+0000", 2024-09-14T09:06:53.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6000000000000002e-07 2024-09-14T09:06:53.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525890+0000", 2024-09-14T09:06:53.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4519999999999999e-06 2024-09-14T09:06:53.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525899+0000", 2024-09-14T09:06:53.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1090000000000004e-06 2024-09-14T09:06:53.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525906+0000", 2024-09-14T09:06:53.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0219999999999999e-06 2024-09-14T09:06:53.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560344+0000", 2024-09-14T09:06:53.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034437970999999998 2024-09-14T09:06:53.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560422+0000", 2024-09-14T09:06:53.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7857999999999999e-05 2024-09-14T09:06:53.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560804+0000", 2024-09-14T09:06:53.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00038182400000000001 2024-09-14T09:06:53.024 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.024 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.024 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.025 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49721 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:53.025 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:06:53.025 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.442941519000001, 2024-09-14T09:06:53.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035347380999999997, 2024-09-14T09:06:53.025 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.025 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.025 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.025 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.025 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.025 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.025 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49721 2024-09-14T09:06:53.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.025 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:06:53.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:06:53.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525933+0000", 2024-09-14T09:06:53.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T09:06:53.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525936+0000", 2024-09-14T09:06:53.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6759999999999999e-06 2024-09-14T09:06:53.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525943+0000", 2024-09-14T09:06:53.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2459999999999997e-06 2024-09-14T09:06:53.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525948+0000", 2024-09-14T09:06:53.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9429999999999999e-06 2024-09-14T09:06:53.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560845+0000", 2024-09-14T09:06:53.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034897354999999998 2024-09-14T09:06:53.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560925+0000", 2024-09-14T09:06:53.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9804999999999999e-05 2024-09-14T09:06:53.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561280+0000", 2024-09-14T09:06:53.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035480100000000002 2024-09-14T09:06:53.029 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.029 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.030 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.030 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49722 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3112960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:53.030 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:53.030 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.442917872999999, 2024-09-14T09:06:53.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035750491000000002, 2024-09-14T09:06:53.030 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.030 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.030 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.030 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.030 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.030 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.030 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49722 2024-09-14T09:06:53.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.031 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:53.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:53.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:53.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.96e-07 2024-09-14T09:06:53.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525958+0000", 2024-09-14T09:06:53.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.024e-06 2024-09-14T09:06:53.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525964+0000", 2024-09-14T09:06:53.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6699999999999999e-06 2024-09-14T09:06:53.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525969+0000", 2024-09-14T09:06:53.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3939999999999998e-06 2024-09-14T09:06:53.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561319+0000", 2024-09-14T09:06:53.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.03535013 2024-09-14T09:06:53.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.034 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561390+0000", 2024-09-14T09:06:53.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0920000000000005e-05 2024-09-14T09:06:53.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.034 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561707+0000", 2024-09-14T09:06:53.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031695699999999998 2024-09-14T09:06:53.034 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.034 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.034 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.035 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49723 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3506176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:53.035 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:53.035 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.442898890000002, 2024-09-14T09:06:53.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036113064, 2024-09-14T09:06:53.035 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.035 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.035 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.035 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.035 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.035 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.035 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49723 2024-09-14T09:06:53.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.035 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:53.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:53.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:53.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1699999999999999e-07 2024-09-14T09:06:53.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525977+0000", 2024-09-14T09:06:53.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9250000000000002e-06 2024-09-14T09:06:53.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525983+0000", 2024-09-14T09:06:53.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4620000000000004e-06 2024-09-14T09:06:53.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525987+0000", 2024-09-14T09:06:53.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.279e-06 2024-09-14T09:06:53.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561749+0000", 2024-09-14T09:06:53.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035762222000000003 2024-09-14T09:06:53.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561796+0000", 2024-09-14T09:06:53.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6959999999999998e-05 2024-09-14T09:06:53.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562088+0000", 2024-09-14T09:06:53.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029179899999999998 2024-09-14T09:06:53.039 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.039 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.039 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.039 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49724 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:53.039 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:06:53.039 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 42.442880733999999, 2024-09-14T09:06:53.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036449388999999999, 2024-09-14T09:06:53.039 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.039 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.039 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.039 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.040 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.040 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.040 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49724 2024-09-14T09:06:53.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.040 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:06:53.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.041 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.041 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:06:53.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.041 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.041 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525994+0000", 2024-09-14T09:06:53.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3099999999999998e-07 2024-09-14T09:06:53.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525996+0000", 2024-09-14T09:06:53.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.939e-06 2024-09-14T09:06:53.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526005+0000", 2024-09-14T09:06:53.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8430000000000005e-06 2024-09-14T09:06:53.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526010+0000", 2024-09-14T09:06:53.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9389999999999996e-06 2024-09-14T09:06:53.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562114+0000", 2024-09-14T09:06:53.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036103510999999998 2024-09-14T09:06:53.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562189+0000", 2024-09-14T09:06:53.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5162999999999998e-05 2024-09-14T09:06:53.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562443+0000", 2024-09-14T09:06:53.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000253563 2024-09-14T09:06:53.044 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.044 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.044 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.044 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:64935 3.12 3:4cd6d7cb:::rbd_data.115e21c31fcf.0000000000000049:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:06:53.044 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:06:53.044 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.217515386000002, 2024-09-14T09:06:53.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037353866999999999, 2024-09-14T09:06:53.045 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.045 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.045 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:53.045 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.045 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.045 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.045 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64935 2024-09-14T09:06:53.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.045 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:06:53.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:06:53.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751362+0000", 2024-09-14T09:06:53.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6899999999999998e-06 2024-09-14T09:06:53.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751378+0000", 2024-09-14T09:06:53.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6147e-05 2024-09-14T09:06:53.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751421+0000", 2024-09-14T09:06:53.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2691999999999997e-05 2024-09-14T09:06:53.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751444+0000", 2024-09-14T09:06:53.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3237000000000001e-05 2024-09-14T09:06:53.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751594+0000", 2024-09-14T09:06:53.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014947800000000001 2024-09-14T09:06:53.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751733+0000", 2024-09-14T09:06:53.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013945200000000001 2024-09-14T09:06:53.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:53.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.753315+0000", 2024-09-14T09:06:53.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0015815969999999999 2024-09-14T09:06:53.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:53.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788546+0000", 2024-09-14T09:06:53.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035231449999999997 2024-09-14T09:06:53.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:53.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788570+0000", 2024-09-14T09:06:53.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3991000000000001e-05 2024-09-14T09:06:53.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:53.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788645+0000", 2024-09-14T09:06:53.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4379000000000001e-05 2024-09-14T09:06:53.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788712+0000", 2024-09-14T09:06:53.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7754000000000002e-05 2024-09-14T09:06:53.050 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.050 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.050 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.050 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:71052 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:06:53.050 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:06:53.050 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.231306135000001, 2024-09-14T09:06:53.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14854635599999999, 2024-09-14T09:06:53.050 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.051 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.051 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:53.051 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.051 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.051 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.051 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 71052 2024-09-14T09:06:53.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.051 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:06:53.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:06:53.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737570+0000", 2024-09-14T09:06:53.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4770000000000002e-06 2024-09-14T09:06:53.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737584+0000", 2024-09-14T09:06:53.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3457e-05 2024-09-14T09:06:53.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737608+0000", 2024-09-14T09:06:53.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3893999999999999e-05 2024-09-14T09:06:53.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737620+0000", 2024-09-14T09:06:53.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2157000000000001e-05 2024-09-14T09:06:53.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737738+0000", 2024-09-14T09:06:53.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118409 2024-09-14T09:06:53.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737892+0000", 2024-09-14T09:06:53.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154222 2024-09-14T09:06:53.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:06:53.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.749622+0000", 2024-09-14T09:06:53.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011729537 2024-09-14T09:06:53.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:53.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885934+0000", 2024-09-14T09:06:53.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13631222200000001 2024-09-14T09:06:53.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:53.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885996+0000", 2024-09-14T09:06:53.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1555000000000003e-05 2024-09-14T09:06:53.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:53.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886060+0000", 2024-09-14T09:06:53.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3980999999999996e-05 2024-09-14T09:06:53.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886114+0000", 2024-09-14T09:06:53.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4444999999999998e-05 2024-09-14T09:06:53.056 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.056 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.056 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.056 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:71052 3.1 e71/34 ondisk, result = 0)", 2024-09-14T09:06:53.057 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:06:53.057 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.188402756999999, 2024-09-14T09:06:53.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10565334799999999, 2024-09-14T09:06:53.057 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.057 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.057 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.057 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:06:53.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:06:53.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780473+0000", 2024-09-14T09:06:53.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8980000000000001e-06 2024-09-14T09:06:53.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780482+0000", 2024-09-14T09:06:53.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9080000000000001e-06 2024-09-14T09:06:53.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780511+0000", 2024-09-14T09:06:53.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9490999999999999e-05 2024-09-14T09:06:53.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780527+0000", 2024-09-14T09:06:53.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.508e-05 2024-09-14T09:06:53.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885978+0000", 2024-09-14T09:06:53.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.105451328 2024-09-14T09:06:53.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885989+0000", 2024-09-14T09:06:53.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0875e-05 2024-09-14T09:06:53.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886125+0000", 2024-09-14T09:06:53.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013576800000000001 2024-09-14T09:06:53.060 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.060 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.060 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.061 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73871 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:1b [read 8339456~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:53.061 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:06:53.061 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.542917749000001, 2024-09-14T09:06:53.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034605047, 2024-09-14T09:06:53.061 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.061 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.061 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.061 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.061 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.061 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.061 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73871 2024-09-14T09:06:53.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.062 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:06:53.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:06:53.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425957+0000", 2024-09-14T09:06:53.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T09:06:53.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425961+0000", 2024-09-14T09:06:53.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8279999999999996e-06 2024-09-14T09:06:53.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425985+0000", 2024-09-14T09:06:53.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4182e-05 2024-09-14T09:06:53.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425997+0000", 2024-09-14T09:06:53.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.132e-05 2024-09-14T09:06:53.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460070+0000", 2024-09-14T09:06:53.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034072913000000003 2024-09-14T09:06:53.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460298+0000", 2024-09-14T09:06:53.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022819 2024-09-14T09:06:53.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460561+0000", 2024-09-14T09:06:53.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026341400000000001 2024-09-14T09:06:53.065 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.065 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.065 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.066 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:74276 3.12 3:49b683dd:::rbd_data.115ea8d131ff.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1376256~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:53.066 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:06:53.066 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.283770658999998, 2024-09-14T09:06:53.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036482939999999998, 2024-09-14T09:06:53.066 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.066 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.066 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:53.066 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.066 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.066 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.066 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74276 2024-09-14T09:06:53.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.067 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:06:53.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:06:53.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685105+0000", 2024-09-14T09:06:53.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9520000000000001e-06 2024-09-14T09:06:53.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686530+0000", 2024-09-14T09:06:53.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001424629 2024-09-14T09:06:53.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686550+0000", 2024-09-14T09:06:53.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0472e-05 2024-09-14T09:06:53.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686561+0000", 2024-09-14T09:06:53.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T09:06:53.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686671+0000", 2024-09-14T09:06:53.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110621 2024-09-14T09:06:53.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686919+0000", 2024-09-14T09:06:53.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024790199999999999 2024-09-14T09:06:53.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:53.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.687375+0000", 2024-09-14T09:06:53.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00045572100000000002 2024-09-14T09:06:53.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:53.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.710871+0000", 2024-09-14T09:06:53.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023496481999999999 2024-09-14T09:06:53.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:53.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721478+0000", 2024-09-14T09:06:53.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01060709 2024-09-14T09:06:53.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:53.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721534+0000", 2024-09-14T09:06:53.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5912999999999998e-05 2024-09-14T09:06:53.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721586+0000", 2024-09-14T09:06:53.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1792000000000002e-05 2024-09-14T09:06:53.071 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.071 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.071 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.071 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.071 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.071 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:53.072 INFO:teuthology.orchestra.run.smithi176.stdout:4058 writesame 0x38e0c3c thru 0x38eee27 data_size 0x26(0xe1ec bytes) 2024-09-14T09:06:53.072 INFO:teuthology.orchestra.run.smithi176.stdout:4059 write 0xa7128aa thru 0xa718654 (0x5dab bytes) 2024-09-14T09:06:53.073 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:06:53.074 INFO:teuthology.orchestra.run.smithi176.stdout:4061 read 0xed44ad thru 0xee1aa2 (0xd5f6 bytes) 2024-09-14T09:06:53.074 INFO:teuthology.orchestra.run.smithi176.stdout:4063 read 0xcc468d4 thru 0xcc539ee (0xd11b bytes) 2024-09-14T09:06:53.074 INFO:teuthology.orchestra.run.smithi176.stdout:4065 trunc from 0xd3f23e5 to 0x7e8fc5a 2024-09-14T09:06:53.075 INFO:teuthology.orchestra.run.smithi176.stdout:4066 read 0x3e04c2c thru 0x3e0a706 (0x5adb bytes) 2024-09-14T09:06:53.075 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:53.075 INFO:teuthology.orchestra.run.smithi176.stdout:4070 write 0xdfc3a70 thru 0xdfd1d4d (0xe2de bytes) 2024-09-14T09:06:53.075 INFO:teuthology.orchestra.run.smithi176.stdout:4072 trunc from 0xdfd1d4e to 0xe223d12 2024-09-14T09:06:53.075 INFO:teuthology.orchestra.run.smithi176.stdout:4073 trunc from 0xe223d12 to 0x6c6f143 2024-09-14T09:06:53.075 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:53.075+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:53.088 INFO:teuthology.orchestra.run.smithi176.stdout:4074 trunc from 0x6c6f143 to 0xae86f6b 2024-09-14T09:06:53.098 INFO:teuthology.orchestra.run.smithi176.stdout:4075 writesame 0x39f3f93 thru 0x39f9053 data_size 0x3(0x50c1 bytes) 2024-09-14T09:06:53.125 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:06:53.126 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:06:53.141 INFO:teuthology.orchestra.run.smithi176.stdout:4076 writesame 0x67a8b3f thru 0x67b021e data_size 0x30(0x76e0 bytes) 2024-09-14T09:06:53.150 INFO:teuthology.orchestra.run.smithi176.stdout:4077 write 0x6a3560c thru 0x6a3dbfb (0x85f0 bytes) 2024-09-14T09:06:53.151 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-14T09:06:51.985114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1069216166,"num_objects":1343,"num_object_clones":404,"num_object_copies":2686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1343,"num_whiteouts":488,"num_read":108059,"num_read_kb":574019,"num_write":13555,"num_write_kb":238890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1216,"num_bytes_recovered":72785688,"num_keys_recovered":252,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10732,"ondisk_log_size":10732,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":350672,"kb_used_data":45652,"kb_used_omap":12,"kb_used_meta":304819,"kb_avail":754624048,"statfs":{"total":773094113280,"available":772735025152,"internally_reserved":0,"allocated":46747648,"data_stored":335448999,"data_compressed":9894934,"data_compressed_allocated":27795456,"data_compressed_original":324821136,"omap_allocated":12751,"internal_metadata":312135217},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23847640,"num_objects":116,"num_object_clones":37,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":48,"num_read":3611,"num_read_kb":27424,"num_write":494,"num_write_kb":14981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000945"},"pg_stats":[{"pgid":"3.1f","version":"87'1344","reported_seq":4285,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:47.223591+0000","last_change":"2024-09-14T09:06:41.348384+0000","last_active":"2024-09-14T09:06:47.223591+0000","last_peered":"2024-09-14T09:06:47.223591+0000","last_clean":"2024-09-14T09:06:47.223591+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:06:47.223591+0000","last_undegraded":"2024-09-14T09:06:47.223591+0000","last_fullsized":"2024-09-14T09:06:47.223591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1344,"log_dups_size":0,"ondisk_log_size":1344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:40:01.002873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001387512,"stat_sum":{"num_bytes":17479966,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":15,"num_read":9351,"num_read_kb":14328,"num_write":2320,"num_write_kb":5034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"90'233","reported_seq":2578,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.911662+0000","last_change":"2024-09-14T09:05:29.039503+0000","last_active":"2024-09-14T09:06:50.911662+0000","last_peered":"2024-09-14T09:06:50.911662+0000","last_clean":"2024-09-14T09:06:50.911662+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:06:50.911662+0000","last_undegraded":"2024-09-14T09:06:50.911662+0000","last_fullsized":"2024-09-14T09:06:50.911662+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:05:46.072775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001657176,"stat_sum":{"num_bytes":34267862,"num_objects":62,"num_object_clones":22,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":24,"num_read":2272,"num_read_kb":23767,"num_write":270,"num_write_kb":10428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"87'367","reported_seq":3745,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.873849+0000","last_change":"2024-09-14T09:05:28.835298+0000","last_active":"2024-09-14T09:06:46.873849+0000","last_peered":"2024-09-14T09:06:46.873849+0000","last_clean":"2024-09-14T09:06:46.873849+0000","last_became_active":"2024-09-14T09:05:26.835372+0000","last_became_peered":"2024-09-14T09:05:26.835372+0000","last_unstale":"2024-09-14T09:06:46.873849+0000","last_undegraded":"2024-09-14T09:06:46.873849+0000","last_fullsized":"2024-09-14T09:06:46.873849+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:38.418901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014149329999999999,"stat_sum":{"num_bytes":58835428,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":10,"num_read":3995,"num_read_kb":52947,"num_write":512,"num_write_kb":8922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"83'525","reported_seq":3197,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.874459+0000","last_change":"2024-09-14T09:06:44.867658+0000","last_active":"2024-09-14T09:06:46.874459+0000","last_peered":"2024-09-14T09:06:46.874459+0000","last_clean":"2024-09-14T09:06:46.874459+0000","last_became_active":"2024-09-14T09:06:44.862945+0000","last_became_peered":"2024-09-14T09:06:44.862945+0000","last_unstale":"2024-09-14T09:06:46.874459+0000","last_undegraded":"2024-09-14T09:06:46.874459+0000","last_fullsized":"2024-09-14T09:06:46.874459+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:13:41.872396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014661419999999999,"stat_sum":{"num_bytes":38979686,"num_objects":37,"num_object_clones":12,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":5424,"num_read_kb":22099,"num_write":742,"num_write_kb":6156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"79'205","reported_seq":2402,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.875685+0000","last_change":"2024-09-14T09:06:41.347478+0000","last_active":"2024-09-14T09:06:46.875685+0000","last_peered":"2024-09-14T09:06:46.875685+0000","last_clean":"2024-09-14T09:06:46.875685+0000","last_became_active":"2024-09-14T09:06:41.346397+0000","last_became_peered":"2024-09-14T09:06:41.346397+0000","last_unstale":"2024-09-14T09:06:46.875685+0000","last_undegraded":"2024-09-14T09:06:46.875685+0000","last_fullsized":"2024-09-14T09:06:46.875685+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:30:10.950021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060373399999999998,"stat_sum":{"num_bytes":30069643,"num_objects":40,"num_object_clones":16,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":1935,"num_read_kb":6316,"num_write":210,"num_write_kb":11506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":6073202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"90'495","reported_seq":3641,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:51.134932+0000","last_change":"2024-09-14T09:06:44.872988+0000","last_active":"2024-09-14T09:06:51.134932+0000","last_peered":"2024-09-14T09:06:51.134932+0000","last_clean":"2024-09-14T09:06:51.134932+0000","last_became_active":"2024-09-14T09:06:44.869103+0000","last_became_peered":"2024-09-14T09:06:44.869103+0000","last_unstale":"2024-09-14T09:06:51.134932+0000","last_undegraded":"2024-09-14T09:06:51.134932+0000","last_fullsized":"2024-09-14T09:06:51.134932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:18:58.336567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020845550000000001,"stat_sum":{"num_bytes":34728052,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":13,"num_read":4632,"num_read_kb":22831,"num_write":779,"num_write_kb":10112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"89'225","reported_seq":2183,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.917106+0000","last_change":"2024-09-14T09:05:28.822565+0000","last_active":"2024-09-14T09:06:50.917106+0000","last_peered":"2024-09-14T09:06:50.917106+0000","last_clean":"2024-09-14T09:06:50.917106+0000","last_became_active":"2024-09-14T09:05:26.816132+0000","last_became_peered":"2024-09-14T09:05:26.816132+0000","last_unstale":"2024-09-14T09:06:50.917106+0000","last_undegraded":"2024-09-14T09:06:50.917106+0000","last_fullsized":"2024-09-14T09:06:50.917106+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:52:37.782050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092420299999999996,"stat_sum":{"num_bytes":22797618,"num_objects":38,"num_object_clones":13,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":15,"num_read":1872,"num_read_kb":6672,"num_write":262,"num_write_kb":5303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"90'302","reported_seq":3553,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.911075+0000","last_change":"2024-09-14T09:06:41.342724+0000","last_active":"2024-09-14T09:06:50.911075+0000","last_peered":"2024-09-14T09:06:50.911075+0000","last_clean":"2024-09-14T09:06:50.911075+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:06:50.911075+0000","last_undegraded":"2024-09-14T09:06:50.911075+0000","last_fullsized":"2024-09-14T09:06:50.911075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:17.184898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081579899999999997,"stat_sum":{"num_bytes":51553842,"num_objects":57,"num_object_clones":19,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":18,"num_read":3846,"num_read_kb":9241,"num_write":291,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"83'357","reported_seq":4359,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.876315+0000","last_change":"2024-09-14T09:05:28.839928+0000","last_active":"2024-09-14T09:06:46.876315+0000","last_peered":"2024-09-14T09:06:46.876315+0000","last_clean":"2024-09-14T09:06:46.876315+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:06:46.876315+0000","last_undegraded":"2024-09-14T09:06:46.876315+0000","last_fullsized":"2024-09-14T09:06:46.876315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:36:52.750689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014343089999999999,"stat_sum":{"num_bytes":37014757,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":4962,"num_read_kb":22170,"num_write":440,"num_write_kb":4862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"86'377","reported_seq":1535,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.873865+0000","last_change":"2024-09-14T09:05:28.836370+0000","last_active":"2024-09-14T09:06:46.873865+0000","last_peered":"2024-09-14T09:06:46.873865+0000","last_clean":"2024-09-14T09:06:46.873865+0000","last_became_active":"2024-09-14T09:05:26.836744+0000","last_became_peered":"2024-09-14T09:05:26.836744+0000","last_unstale":"2024-09-14T09:06:46.873865+0000","last_undegraded":"2024-09-14T09:06:46.873865+0000","last_fullsized":"2024-09-14T09:06:46.873865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:11:06.446635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015277190000000001,"stat_sum":{"num_bytes":15800887,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":9,"num_read":3095,"num_read_kb":7897,"num_write":507,"num_write_kb":3559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'532","reported_seq":1705,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.923111+0000","last_change":"2024-09-14T09:05:28.851587+0000","last_active":"2024-09-14T09:06:50.923111+0000","last_peered":"2024-09-14T09:06:50.923111+0000","last_clean":"2024-09-14T09:06:50.923111+0000","last_became_active":"2024-09-14T09:05:26.840893+0000","last_became_peered":"2024-09-14T09:05:26.840893+0000","last_unstale":"2024-09-14T09:06:50.923111+0000","last_undegraded":"2024-09-14T09:06:50.923111+0000","last_fullsized":"2024-09-14T09:06:50.923111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:56:28.741207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098594000000000004,"stat_sum":{"num_bytes":9844647,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":15,"num_read":3941,"num_read_kb":4052,"num_write":799,"num_write_kb":3575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"90'349","reported_seq":1821,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.909595+0000","last_change":"2024-09-14T09:06:44.867267+0000","last_active":"2024-09-14T09:06:50.909595+0000","last_peered":"2024-09-14T09:06:50.909595+0000","last_clean":"2024-09-14T09:06:50.909595+0000","last_became_active":"2024-09-14T09:06:44.865413+0000","last_became_peered":"2024-09-14T09:06:44.865413+0000","last_unstale":"2024-09-14T09:06:50.909595+0000","last_undegraded":"2024-09-14T09:06:50.909595+0000","last_fullsized":"2024-09-14T09:06:50.909595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:32:42.409994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014079679999999999,"stat_sum":{"num_bytes":21911807,"num_objects":47,"num_object_clones":15,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":2341,"num_read_kb":4486,"num_write":393,"num_write_kb":1460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":253148,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":150,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:47.006915+0000","last_change":"2024-09-14T09:05:19.793674+0000","last_active":"2024-09-14T09:06:47.006915+0000","last_peered":"2024-09-14T09:06:47.006915+0000","last_clean":"2024-09-14T09:06:47.006915+0000","last_became_active":"2024-09-14T09:05:17.284977+0000","last_became_peered":"2024-09-14T09:05:17.284977+0000","last_unstale":"2024-09-14T09:06:47.006915+0000","last_undegraded":"2024-09-14T09:06:47.006915+0000","last_fullsized":"2024-09-14T09:06:47.006915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:30:38.832027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"87'227","reported_seq":3533,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.914300+0000","last_change":"2024-09-14T09:05:28.839614+0000","last_active":"2024-09-14T09:06:46.914300+0000","last_peered":"2024-09-14T09:06:46.914300+0000","last_clean":"2024-09-14T09:06:46.914300+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:06:46.914300+0000","last_undegraded":"2024-09-14T09:06:46.914300+0000","last_fullsized":"2024-09-14T09:06:46.914300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:30:20.294407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014245869999999999,"stat_sum":{"num_bytes":40394524,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":4246,"num_read_kb":38806,"num_write":246,"num_write_kb":6221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.917587+0000","last_change":"2024-09-14T09:05:19.314029+0000","last_active":"2024-09-14T09:06:50.917587+0000","last_peered":"2024-09-14T09:06:50.917587+0000","last_clean":"2024-09-14T09:06:50.917587+0000","last_became_active":"2024-09-14T09:05:17.282890+0000","last_became_peered":"2024-09-14T09:05:17.282890+0000","last_unstale":"2024-09-14T09:06:50.917587+0000","last_undegraded":"2024-09-14T09:06:50.917587+0000","last_fullsized":"2024-09-14T09:06:50.917587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:25.304105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"89'382","reported_seq":2396,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.911507+0000","last_change":"2024-09-14T09:05:29.039154+0000","last_active":"2024-09-14T09:06:50.911507+0000","last_peered":"2024-09-14T09:06:50.911507+0000","last_clean":"2024-09-14T09:06:50.911507+0000","last_became_active":"2024-09-14T09:05:26.829855+0000","last_became_peered":"2024-09-14T09:05:26.829855+0000","last_unstale":"2024-09-14T09:06:50.911507+0000","last_undegraded":"2024-09-14T09:06:50.911507+0000","last_fullsized":"2024-09-14T09:06:50.911507+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:27:03.532681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019213749999999999,"stat_sum":{"num_bytes":37534250,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":3276,"num_read_kb":12689,"num_write":497,"num_write_kb":4216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.923311+0000","last_change":"2024-09-14T09:05:19.794165+0000","last_active":"2024-09-14T09:06:50.923311+0000","last_peered":"2024-09-14T09:06:50.923311+0000","last_clean":"2024-09-14T09:06:50.923311+0000","last_became_active":"2024-09-14T09:05:17.295864+0000","last_became_peered":"2024-09-14T09:05:17.295864+0000","last_unstale":"2024-09-14T09:06:50.923311+0000","last_undegraded":"2024-09-14T09:06:50.923311+0000","last_fullsized":"2024-09-14T09:06:50.923311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:53.901865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011399719999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"90'339","reported_seq":3107,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:51.148220+0000","last_change":"2024-09-14T09:06:35.286509+0000","last_active":"2024-09-14T09:06:51.148220+0000","last_peered":"2024-09-14T09:06:51.148220+0000","last_clean":"2024-09-14T09:06:51.148220+0000","last_became_active":"2024-09-14T09:06:35.282469+0000","last_became_peered":"2024-09-14T09:06:35.282469+0000","last_unstale":"2024-09-14T09:06:51.148220+0000","last_undegraded":"2024-09-14T09:06:51.148220+0000","last_fullsized":"2024-09-14T09:06:51.148220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:39:27.059412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014087959999999999,"stat_sum":{"num_bytes":22196354,"num_objects":56,"num_object_clones":19,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":25,"num_read":4104,"num_read_kb":17360,"num_write":337,"num_write_kb":6610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":1142719,"num_keys_recovered":23,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.917479+0000","last_change":"2024-09-14T09:05:19.314129+0000","last_active":"2024-09-14T09:06:50.917479+0000","last_peered":"2024-09-14T09:06:50.917479+0000","last_clean":"2024-09-14T09:06:50.917479+0000","last_became_active":"2024-09-14T09:05:17.295196+0000","last_became_peered":"2024-09-14T09:05:17.295196+0000","last_unstale":"2024-09-14T09:06:50.917479+0000","last_undegraded":"2024-09-14T09:06:50.917479+0000","last_fullsized":"2024-09-14T09:06:50.917479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:35.269115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011613800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"83'168","reported_seq":871,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.873896+0000","last_change":"2024-09-14T09:05:28.835976+0000","last_active":"2024-09-14T09:06:46.873896+0000","last_peered":"2024-09-14T09:06:46.873896+0000","last_clean":"2024-09-14T09:06:46.873896+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:06:46.873896+0000","last_undegraded":"2024-09-14T09:06:46.873896+0000","last_fullsized":"2024-09-14T09:06:46.873896+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:21:06.765695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017480779999999999,"stat_sum":{"num_bytes":52867963,"num_objects":45,"num_object_clones":16,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":21,"num_read":596,"num_read_kb":9794,"num_write":155,"num_write_kb":9028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"89'387","reported_seq":2046,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.909806+0000","last_change":"2024-09-14T09:06:46.882194+0000","last_active":"2024-09-14T09:06:50.909806+0000","last_peered":"2024-09-14T09:06:50.909806+0000","last_clean":"2024-09-14T09:06:50.909806+0000","last_became_active":"2024-09-14T09:06:46.880786+0000","last_became_peered":"2024-09-14T09:06:46.880786+0000","last_unstale":"2024-09-14T09:06:50.909806+0000","last_undegraded":"2024-09-14T09:06:50.909806+0000","last_fullsized":"2024-09-14T09:06:50.909806+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:01:15.626388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014128,"stat_sum":{"num_bytes":26481291,"num_objects":42,"num_object_clones":14,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":14,"num_read":2920,"num_read_kb":16318,"num_write":467,"num_write_kb":9830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":152,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.873713+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:06:46.873713+0000","last_peered":"2024-09-14T09:06:46.873713+0000","last_clean":"2024-09-14T09:06:46.873713+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:06:46.873713+0000","last_undegraded":"2024-09-14T09:06:46.873713+0000","last_fullsized":"2024-09-14T09:06:46.873713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"89'191","reported_seq":1122,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.917349+0000","last_change":"2024-09-14T09:05:28.822791+0000","last_active":"2024-09-14T09:06:50.917349+0000","last_peered":"2024-09-14T09:06:50.917349+0000","last_clean":"2024-09-14T09:06:50.917349+0000","last_became_active":"2024-09-14T09:05:26.817109+0000","last_became_peered":"2024-09-14T09:05:26.817109+0000","last_unstale":"2024-09-14T09:06:50.917349+0000","last_undegraded":"2024-09-14T09:06:50.917349+0000","last_fullsized":"2024-09-14T09:06:50.917349+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:18:41.388655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011728439999999999,"stat_sum":{"num_bytes":15644929,"num_objects":44,"num_object_clones":13,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":18,"num_read":970,"num_read_kb":5438,"num_write":200,"num_write_kb":2208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":150,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.870865+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:06:46.870865+0000","last_peered":"2024-09-14T09:06:46.870865+0000","last_clean":"2024-09-14T09:06:46.870865+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:06:46.870865+0000","last_undegraded":"2024-09-14T09:06:46.870865+0000","last_fullsized":"2024-09-14T09:06:46.870865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"87'205","reported_seq":1968,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.875774+0000","last_change":"2024-09-14T09:06:45.868939+0000","last_active":"2024-09-14T09:06:46.875774+0000","last_peered":"2024-09-14T09:06:46.875774+0000","last_clean":"2024-09-14T09:06:46.875774+0000","last_became_active":"2024-09-14T09:06:45.868194+0000","last_became_peered":"2024-09-14T09:06:45.868194+0000","last_unstale":"2024-09-14T09:06:46.875774+0000","last_undegraded":"2024-09-14T09:06:46.875774+0000","last_fullsized":"2024-09-14T09:06:46.875774+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:06:02.859882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074815999999999999,"stat_sum":{"num_bytes":29354455,"num_objects":31,"num_object_clones":11,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":1490,"num_read_kb":15691,"num_write":207,"num_write_kb":9103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":158,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.922768+0000","last_change":"2024-09-14T09:05:19.793999+0000","last_active":"2024-09-14T09:06:50.922768+0000","last_peered":"2024-09-14T09:06:50.922768+0000","last_clean":"2024-09-14T09:06:50.922768+0000","last_became_active":"2024-09-14T09:05:17.284122+0000","last_became_peered":"2024-09-14T09:05:17.284122+0000","last_unstale":"2024-09-14T09:06:50.922768+0000","last_undegraded":"2024-09-14T09:06:50.922768+0000","last_fullsized":"2024-09-14T09:06:50.922768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:12:01.133215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010234269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'213","reported_seq":1261,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.911201+0000","last_change":"2024-09-14T09:05:29.038389+0000","last_active":"2024-09-14T09:06:50.911201+0000","last_peered":"2024-09-14T09:06:50.911201+0000","last_clean":"2024-09-14T09:06:50.911201+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:06:50.911201+0000","last_undegraded":"2024-09-14T09:06:50.911201+0000","last_fullsized":"2024-09-14T09:06:50.911201+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:02:19.602333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001479076,"stat_sum":{"num_bytes":22732338,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":17,"num_read":1187,"num_read_kb":5060,"num_write":260,"num_write_kb":3573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.911033+0000","last_change":"2024-09-14T09:06:25.227644+0000","last_active":"2024-09-14T09:06:50.911033+0000","last_peered":"2024-09-14T09:06:50.911033+0000","last_clean":"2024-09-14T09:06:50.911033+0000","last_became_active":"2024-09-14T09:06:25.222344+0000","last_became_peered":"2024-09-14T09:06:25.222344+0000","last_unstale":"2024-09-14T09:06:50.911033+0000","last_undegraded":"2024-09-14T09:06:50.911033+0000","last_fullsized":"2024-09-14T09:06:50.911033+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:23:33.414859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":249,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.922787+0000","last_change":"2024-09-14T09:05:13.239808+0000","last_active":"2024-09-14T09:06:50.922787+0000","last_peered":"2024-09-14T09:06:50.922787+0000","last_clean":"2024-09-14T09:06:50.922787+0000","last_became_active":"2024-09-14T09:05:13.239064+0000","last_became_peered":"2024-09-14T09:05:13.239064+0000","last_unstale":"2024-09-14T09:06:50.922787+0000","last_undegraded":"2024-09-14T09:06:50.922787+0000","last_fullsized":"2024-09-14T09:06:50.922787+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:40:11.559161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"91'270","reported_seq":3558,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:51.132105+0000","last_change":"2024-09-14T09:05:29.037923+0000","last_active":"2024-09-14T09:06:51.132105+0000","last_peered":"2024-09-14T09:06:51.132105+0000","last_clean":"2024-09-14T09:06:51.132105+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:06:51.132105+0000","last_undegraded":"2024-09-14T09:06:51.132105+0000","last_fullsized":"2024-09-14T09:06:51.132105+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:26:18.317368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099199800000000001,"stat_sum":{"num_bytes":43866210,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":16,"num_read":3864,"num_read_kb":30877,"num_write":284,"num_write_kb":8596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"91'209","reported_seq":1689,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:51.123811+0000","last_change":"2024-09-14T09:06:44.868332+0000","last_active":"2024-09-14T09:06:51.123811+0000","last_peered":"2024-09-14T09:06:51.123811+0000","last_clean":"2024-09-14T09:06:51.123811+0000","last_became_active":"2024-09-14T09:06:44.866346+0000","last_became_peered":"2024-09-14T09:06:44.866346+0000","last_unstale":"2024-09-14T09:06:51.123811+0000","last_undegraded":"2024-09-14T09:06:51.123811+0000","last_fullsized":"2024-09-14T09:06:51.123811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:27:44.824297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001266028,"stat_sum":{"num_bytes":28381909,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":14,"num_read":1219,"num_read_kb":7315,"num_write":194,"num_write_kb":14728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":4589732,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"83'262","reported_seq":3293,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:47.006902+0000","last_change":"2024-09-14T09:06:45.870752+0000","last_active":"2024-09-14T09:06:47.006902+0000","last_peered":"2024-09-14T09:06:47.006902+0000","last_clean":"2024-09-14T09:06:47.006902+0000","last_became_active":"2024-09-14T09:06:45.869266+0000","last_became_peered":"2024-09-14T09:06:45.869266+0000","last_unstale":"2024-09-14T09:06:47.006902+0000","last_undegraded":"2024-09-14T09:06:47.006902+0000","last_fullsized":"2024-09-14T09:06:47.006902+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:44:26.704179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185731,"stat_sum":{"num_bytes":48761384,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":3347,"num_read_kb":31896,"num_write":294,"num_write_kb":16339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"90'315","reported_seq":8061,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.966928+0000","last_change":"2024-09-14T09:06:02.837388+0000","last_active":"2024-09-14T09:06:50.966928+0000","last_peered":"2024-09-14T09:06:50.966928+0000","last_clean":"2024-09-14T09:06:50.966928+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:06:50.966928+0000","last_undegraded":"2024-09-14T09:06:50.966928+0000","last_fullsized":"2024-09-14T09:06:50.966928+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:55:11.858581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001585295,"stat_sum":{"num_bytes":40094561,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":15,"num_read":8915,"num_read_kb":54668,"num_write":312,"num_write_kb":7534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"83'209","reported_seq":1185,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.873838+0000","last_change":"2024-09-14T09:05:28.834647+0000","last_active":"2024-09-14T09:06:46.873838+0000","last_peered":"2024-09-14T09:06:46.873838+0000","last_clean":"2024-09-14T09:06:46.873838+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:06:46.873838+0000","last_undegraded":"2024-09-14T09:06:46.873838+0000","last_fullsized":"2024-09-14T09:06:46.873838+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:29:56.253705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244290000000001,"stat_sum":{"num_bytes":26698314,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":1353,"num_read_kb":7206,"num_write":222,"num_write_kb":7172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"90'278","reported_seq":3376,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.923106+0000","last_change":"2024-09-14T09:05:28.851561+0000","last_active":"2024-09-14T09:06:50.923106+0000","last_peered":"2024-09-14T09:06:50.923106+0000","last_clean":"2024-09-14T09:06:50.923106+0000","last_became_active":"2024-09-14T09:05:26.843780+0000","last_became_peered":"2024-09-14T09:05:26.843780+0000","last_unstale":"2024-09-14T09:06:50.923106+0000","last_undegraded":"2024-09-14T09:06:50.923106+0000","last_fullsized":"2024-09-14T09:06:50.923106+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:11:00.153191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072987200000000001,"stat_sum":{"num_bytes":29532905,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":13,"num_read":3806,"num_read_kb":26419,"num_write":329,"num_write_kb":14265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"89'188","reported_seq":1595,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.923557+0000","last_change":"2024-09-14T09:06:45.872735+0000","last_active":"2024-09-14T09:06:50.923557+0000","last_peered":"2024-09-14T09:06:50.923557+0000","last_clean":"2024-09-14T09:06:50.923557+0000","last_became_active":"2024-09-14T09:06:45.868989+0000","last_became_peered":"2024-09-14T09:06:45.868989+0000","last_unstale":"2024-09-14T09:06:50.923557+0000","last_undegraded":"2024-09-14T09:06:50.923557+0000","last_fullsized":"2024-09-14T09:06:50.923557+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:05:15.424159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057596100000000005,"stat_sum":{"num_bytes":36900119,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":16,"num_read":1112,"num_read_kb":11331,"num_write":173,"num_write_kb":5375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":3897102,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"89'248","reported_seq":1411,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.923133+0000","last_change":"2024-09-14T09:06:46.884414+0000","last_active":"2024-09-14T09:06:50.923133+0000","last_peered":"2024-09-14T09:06:50.923133+0000","last_clean":"2024-09-14T09:06:50.923133+0000","last_became_active":"2024-09-14T09:06:46.881561+0000","last_became_peered":"2024-09-14T09:06:46.881561+0000","last_unstale":"2024-09-14T09:06:50.923133+0000","last_undegraded":"2024-09-14T09:06:50.923133+0000","last_fullsized":"2024-09-14T09:06:50.923133+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:48:33.986404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000909463,"stat_sum":{"num_bytes":29805677,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":15,"num_read":1387,"num_read_kb":7823,"num_write":261,"num_write_kb":7808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":3341023,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"86'231","reported_seq":4069,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:47.006835+0000","last_change":"2024-09-14T09:06:00.166357+0000","last_active":"2024-09-14T09:06:47.006835+0000","last_peered":"2024-09-14T09:06:47.006835+0000","last_clean":"2024-09-14T09:06:47.006835+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:06:47.006835+0000","last_undegraded":"2024-09-14T09:06:47.006835+0000","last_fullsized":"2024-09-14T09:06:47.006835+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:58:28.963004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011381609999999999,"stat_sum":{"num_bytes":86083375,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":17,"num_read":3692,"num_read_kb":43423,"num_write":289,"num_write_kb":15097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"90'354","reported_seq":2763,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.922810+0000","last_change":"2024-09-14T09:06:46.882214+0000","last_active":"2024-09-14T09:06:50.922810+0000","last_peered":"2024-09-14T09:06:50.922810+0000","last_clean":"2024-09-14T09:06:50.922810+0000","last_became_active":"2024-09-14T09:06:46.880416+0000","last_became_peered":"2024-09-14T09:06:46.880416+0000","last_unstale":"2024-09-14T09:06:50.922810+0000","last_undegraded":"2024-09-14T09:06:50.922810+0000","last_fullsized":"2024-09-14T09:06:50.922810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:12:00.641498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092949299999999999,"stat_sum":{"num_bytes":7519472,"num_objects":47,"num_object_clones":15,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":20,"num_read":3594,"num_read_kb":4516,"num_write":457,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":30867,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"87'370","reported_seq":2594,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-14T09:06:46.875813+0000","last_change":"2024-09-14T09:05:28.839774+0000","last_active":"2024-09-14T09:06:46.875813+0000","last_peered":"2024-09-14T09:06:46.875813+0000","last_clean":"2024-09-14T09:06:46.875813+0000","last_became_active":"2024-09-14T09:05:26.826600+0000","last_became_peered":"2024-09-14T09:05:26.826600+0000","last_unstale":"2024-09-14T09:06:46.875813+0000","last_undegraded":"2024-09-14T09:06:46.875813+0000","last_fullsized":"2024-09-14T09:06:46.875813+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:37:44.244140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018357099999999999,"stat_sum":{"num_bytes":44463041,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":17,"num_read":3668,"num_read_kb":3611,"num_write":463,"num_write_kb":3314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"90'300","reported_seq":4653,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T09:06:50.922812+0000","last_change":"2024-09-14T09:06:44.869874+0000","last_active":"2024-09-14T09:06:50.922812+0000","last_peered":"2024-09-14T09:06:50.922812+0000","last_clean":"2024-09-14T09:06:50.922812+0000","last_became_active":"2024-09-14T09:06:44.865881+0000","last_became_peered":"2024-09-14T09:06:44.865881+0000","last_unstale":"2024-09-14T09:06:50.922812+0000","last_undegraded":"2024-09-14T09:06:50.922812+0000","last_fullsized":"2024-09-14T09:06:50.922812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:18:52.986772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061978900000000004,"stat_sum":{"num_bytes":26028513,"num_objects":41,"num_object_clones":12,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":13,"num_read":5581,"num_read_kb":26917,"num_write":280,"num_write_kb":7924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":6761444,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1068625779,"num_objects":1339,"num_object_clones":404,"num_object_copies":2678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1339,"num_whiteouts":488,"num_read":107993,"num_read_kb":573964,"num_write":13452,"num_write_kb":237260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1215,"num_bytes_recovered":72785688,"num_keys_recovered":252,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38121472,"data_stored":329266248,"data_compressed":9886944,"data_compressed_allocated":27643904,"data_compressed_original":322556048,"omap_allocated":0,"internal_metadata":0},"log_size":10657,"ondisk_log_size":10657,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12424,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39656,"kb_used_data":4308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332184,"statfs":{"total":96636764160,"available":96596156416,"internally_reserved":0,"allocated":4411392,"data_stored":31752236,"data_compressed":825018,"data_compressed_allocated":2605056,"data_compressed_original":30800251,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50876,"kb_used_data":5804,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94320964,"statfs":{"total":96636764160,"available":96584667136,"internally_reserved":0,"allocated":5943296,"data_stored":41867259,"data_compressed":1136476,"data_compressed_allocated":3399680,"data_compressed_original":40472428,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509464,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41264,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330576,"statfs":{"total":96636764160,"available":96594509824,"internally_reserved":0,"allocated":1789952,"data_stored":6039141,"data_compressed":257074,"data_compressed_allocated":589824,"data_compressed_original":5468960,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509466,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46440,"kb_used_data":6932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325400,"statfs":{"total":96636764160,"available":96589209600,"internally_reserved":0,"allocated":7098368,"data_stored":53889428,"data_compressed":1551176,"data_compressed_allocated":4263936,"data_compressed_original":52295459,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542167,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33620,"kb_used_data":6568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338220,"statfs":{"total":96636764160,"available":96602337280,"internally_reserved":0,"allocated":6725632,"data_stored":48226976,"data_compressed":1238830,"data_compressed_allocated":3969024,"data_compressed_original":46539647,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48860,"kb_used_data":9352,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322980,"statfs":{"total":96636764160,"available":96586731520,"internally_reserved":0,"allocated":9576448,"data_stored":76764888,"data_compressed":2289579,"data_compressed_allocated":6508544,"data_compressed_original":74970177,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45952,"kb_used_data":6440,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325888,"statfs":{"total":96636764160,"available":96589709312,"internally_reserved":0,"allocated":6594560,"data_stored":48440912,"data_compressed":1537844,"data_compressed_allocated":3899392,"data_compressed_original":46931587,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542169,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44004,"kb_used_data":4500,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327836,"statfs":{"total":96636764160,"available":96591704064,"internally_reserved":0,"allocated":4608000,"data_stored":28468159,"data_compressed":1058937,"data_compressed_allocated":2560000,"data_compressed_original":27342627,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5632000,"data_stored":49786473,"data_compressed":1463694,"data_compressed_allocated":3952640,"data_compressed_original":48752419,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3547136,"data_stored":27983968,"data_compressed":1058937,"data_compressed_allocated":2560000,"data_compressed_original":27342627,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":770048,"data_stored":5573138,"data_compressed":257074,"data_compressed_allocated":589824,"data_compressed_original":5468960,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5709824,"data_stored":49742648,"data_compressed":1601776,"data_compressed_allocated":4067328,"data_compressed_original":48717443,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8523776,"data_stored":76280732,"data_compressed":2289579,"data_compressed_allocated":6508544,"data_compressed_original":74970177,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4947968,"data_stored":41466789,"data_compressed":1158248,"data_compressed_allocated":3428352,"data_compressed_original":40537964,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3391488,"data_stored":31286233,"data_compressed":825018,"data_compressed_allocated":2605056,"data_compressed_original":30800251,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5599232,"data_stored":47146267,"data_compressed":1232618,"data_compressed_allocated":3932160,"data_compressed_original":45966207,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:06:53.154 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T09:06:53.154 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:06:53.154 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-14T09:06:53.154 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T09:06:53.154 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T09:06:53.155 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-14T09:06:53.159 INFO:teuthology.orchestra.run.smithi176.stdout:4078 read 0x1a6727a thru 0x1a684dd (0x1264 bytes) 2024-09-14T09:06:53.160 INFO:teuthology.orchestra.run.smithi176.stdout:4080 write 0xa6eb27b thru 0xa6f77cc (0xc552 bytes) 2024-09-14T09:06:53.171 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:53.171 INFO:teuthology.orchestra.run.smithi176.stdout:4082 writesame 0x24c2bbd thru 0x24ce3e6 data_size 0xb(0xb82a bytes) 2024-09-14T09:06:53.173 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:53.175+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:53.183 INFO:teuthology.orchestra.run.smithi176.stdout:4083 write 0x4b35738 thru 0x4b3aba5 (0x546e bytes) 2024-09-14T09:06:53.189 INFO:teuthology.orchestra.run.smithi176.stdout:4084 write 0x743b3c2 thru 0x7447502 (0xc141 bytes) 2024-09-14T09:06:53.200 INFO:teuthology.orchestra.run.smithi176.stdout:4086 write 0x22a081 thru 0x236845 (0xc7c5 bytes) 2024-09-14T09:06:53.226 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:53.226 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:53.226 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:53.226 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:53.234 INFO:teuthology.orchestra.run.smithi176.stdout:4087 write 0xfd749d thru 0xfdde5f (0x69c3 bytes) 2024-09-14T09:06:53.243 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:06:53.246 INFO:teuthology.orchestra.run.smithi176.stdout:4088 trunc from 0xae86f6b to 0x750d39b 2024-09-14T09:06:53.277 INFO:teuthology.orchestra.run.smithi176.stdout:4089 write 0x982865c thru 0x9836c86 (0xe62b bytes) 2024-09-14T09:06:53.277 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:53.275+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:53.284 INFO:teuthology.orchestra.run.smithi176.stdout:4090 write 0x7334bbb thru 0x734187d (0xccc3 bytes) 2024-09-14T09:06:53.300 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:53.300 INFO:teuthology.orchestra.run.smithi176.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T09:06:53.300 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:53.308 INFO:teuthology.orchestra.run.smithi176.stdout:4091 writesame 0xaf0ac67 thru 0xaf0ea03 data_size 0x1(0x3d9d bytes) 2024-09-14T09:06:53.308 INFO:teuthology.orchestra.run.smithi176.stdout:4092 trunc from 0xaf0ea04 to 0xa9e6f1d 2024-09-14T09:06:53.318 INFO:teuthology.orchestra.run.smithi176.stdout:4093 writesame 0x7070d9b thru 0x707d9b5 data_size 0x1(0xcc1b bytes) 2024-09-14T09:06:53.369 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:53.370 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:53.370 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:53.370 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:53.370 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:53.375 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:53.379+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:53.386 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:06:53.434 INFO:teuthology.orchestra.run.smithi176.stdout:4095 punch from 0x485e1f7 to 0x4860487, (0x2290 bytes) 2024-09-14T09:06:53.448 INFO:teuthology.orchestra.run.smithi176.stdout:4096 write 0xe6eb957 thru 0xe6fb66d (0xfd17 bytes) 2024-09-14T09:06:53.462 INFO:teuthology.orchestra.run.smithi176.stdout:4097 trunc from 0xe6fb66e to 0x9acd2e7 2024-09-14T09:06:53.476 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:53.475+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:53.479 INFO:teuthology.orchestra.run.smithi176.stdout:4098 writesame 0x7e32e94 thru 0x7e3601b data_size 0x14(0x3188 bytes) 2024-09-14T09:06:53.486 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:53.486 INFO:teuthology.orchestra.run.smithi176.stdout:4100 punch from 0x622eda to 0x62e8c0, (0xb9e6 bytes) 2024-09-14T09:06:53.486 INFO:teuthology.orchestra.run.smithi176.stdout:4101 read 0x98f4748 thru 0x99025c7 (0xde80 bytes) 2024-09-14T09:06:53.492 INFO:teuthology.orchestra.run.smithi176.stdout:4102 trunc from 0x9acd2e7 to 0x6aa78b3 2024-09-14T09:06:53.515 INFO:teuthology.orchestra.run.smithi176.stdout:4103 write 0x611b9b7 thru 0x612a349 (0xe993 bytes) 2024-09-14T09:06:53.523 INFO:teuthology.orchestra.run.smithi176.stdout:4104 read 0x35c37b thru 0x36ad26 (0xe9ac bytes) 2024-09-14T09:06:53.526 INFO:teuthology.orchestra.run.smithi176.stdout:4105 punch from 0x505d419 to 0x5065386, (0x7f6d bytes) 2024-09-14T09:06:53.526 INFO:teuthology.orchestra.run.smithi176.stdout:4106 punch from 0x2c0bc67 to 0x2c14d55, (0x90ee bytes) 2024-09-14T09:06:53.537 INFO:teuthology.orchestra.run.smithi176.stdout:4111 write 0x72485c9 thru 0x724c98f (0x43c7 bytes) 2024-09-14T09:06:53.545 INFO:teuthology.orchestra.run.smithi176.stdout:4112 read 0x18d50ff thru 0x18dd0a7 (0x7fa9 bytes) 2024-09-14T09:06:53.546 INFO:teuthology.orchestra.run.smithi176.stdout:4113 read 0x646e015 thru 0x6476f15 (0x8f01 bytes) 2024-09-14T09:06:53.553 INFO:teuthology.orchestra.run.smithi176.stdout:4114 trunc from 0x724c990 to 0x40a2d1b 2024-09-14T09:06:53.576 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:53.577 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:53.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:53.577 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:53.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.577 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73540 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15155200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.577 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:06:53.577 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.305940681999999, 2024-09-14T09:06:53.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.022930347, 2024-09-14T09:06:53.577 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.577 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.577 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.577 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.577 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.578 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.578 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73540 2024-09-14T09:06:53.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.578 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:06:53.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:06:53.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268665+0000", 2024-09-14T09:06:53.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6599999999999996e-07 2024-09-14T09:06:53.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268668+0000", 2024-09-14T09:06:53.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8760000000000001e-06 2024-09-14T09:06:53.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268676+0000", 2024-09-14T09:06:53.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7879999999999995e-06 2024-09-14T09:06:53.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268683+0000", 2024-09-14T09:06:53.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3180000000000003e-06 2024-09-14T09:06:53.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282017+0000", 2024-09-14T09:06:53.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01333409 2024-09-14T09:06:53.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282020+0000", 2024-09-14T09:06:53.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1609999999999999e-06 2024-09-14T09:06:53.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291158+0000", 2024-09-14T09:06:53.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0091376430000001 2024-09-14T09:06:53.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291383+0000", 2024-09-14T09:06:53.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022491000000000001 2024-09-14T09:06:53.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291595+0000", 2024-09-14T09:06:53.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021199500000000001 2024-09-14T09:06:53.582 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.582 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.582 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.582 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73541 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15237120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.582 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:06:53.582 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.305913262000001, 2024-09-14T09:06:53.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0232607330000001, 2024-09-14T09:06:53.583 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.583 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.583 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.583 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73541 2024-09-14T09:06:53.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.583 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:06:53.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:06:53.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:06:53.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5400000000000001e-07 2024-09-14T09:06:53.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268695+0000", 2024-09-14T09:06:53.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5799999999999999e-06 2024-09-14T09:06:53.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268703+0000", 2024-09-14T09:06:53.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5480000000000002e-06 2024-09-14T09:06:53.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268713+0000", 2024-09-14T09:06:53.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8640000000000008e-06 2024-09-14T09:06:53.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282032+0000", 2024-09-14T09:06:53.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013318831999999999 2024-09-14T09:06:53.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282035+0000", 2024-09-14T09:06:53.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4010000000000001e-06 2024-09-14T09:06:53.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291701+0000", 2024-09-14T09:06:53.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0096654199999999 2024-09-14T09:06:53.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291807+0000", 2024-09-14T09:06:53.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105852 2024-09-14T09:06:53.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291952+0000", 2024-09-14T09:06:53.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014568200000000001 2024-09-14T09:06:53.587 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.587 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.587 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.588 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73542 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15319040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.588 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:06:53.588 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.305843651, 2024-09-14T09:06:53.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0237091359999999, 2024-09-14T09:06:53.588 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.588 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.588 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.588 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.588 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.588 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.588 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73542 2024-09-14T09:06:53.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.589 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:06:53.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:06:53.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268762+0000", 2024-09-14T09:06:53.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:06:53.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268766+0000", 2024-09-14T09:06:53.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5880000000000002e-06 2024-09-14T09:06:53.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268776+0000", 2024-09-14T09:06:53.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0168e-05 2024-09-14T09:06:53.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268789+0000", 2024-09-14T09:06:53.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:06:53.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282053+0000", 2024-09-14T09:06:53.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013264496000000001 2024-09-14T09:06:53.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282057+0000", 2024-09-14T09:06:53.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6090000000000001e-06 2024-09-14T09:06:53.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292052+0000", 2024-09-14T09:06:53.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0099950070000001 2024-09-14T09:06:53.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292262+0000", 2024-09-14T09:06:53.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021009800000000001 2024-09-14T09:06:53.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292470+0000", 2024-09-14T09:06:53.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000208314 2024-09-14T09:06:53.593 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.593 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.593 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.594 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73543 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.594 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:06:53.594 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.305804860999999, 2024-09-14T09:06:53.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023985473, 2024-09-14T09:06:53.594 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.594 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.594 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.594 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.594 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.594 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.594 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73543 2024-09-14T09:06:53.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.595 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:06:53.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:06:53.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268801+0000", 2024-09-14T09:06:53.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:06:53.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268803+0000", 2024-09-14T09:06:53.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7439999999999999e-06 2024-09-14T09:06:53.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268812+0000", 2024-09-14T09:06:53.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7080000000000003e-06 2024-09-14T09:06:53.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268819+0000", 2024-09-14T09:06:53.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T09:06:53.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282068+0000", 2024-09-14T09:06:53.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013249540000000001 2024-09-14T09:06:53.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282072+0000", 2024-09-14T09:06:53.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0960000000000001e-06 2024-09-14T09:06:53.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292573+0000", 2024-09-14T09:06:53.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0105014560000001 2024-09-14T09:06:53.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292679+0000", 2024-09-14T09:06:53.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010545899999999999 2024-09-14T09:06:53.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292786+0000", 2024-09-14T09:06:53.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107052 2024-09-14T09:06:53.599 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.599 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.599 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.599 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73544 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.599 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:06:53.599 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.305777159000002, 2024-09-14T09:06:53.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0244830549999999, 2024-09-14T09:06:53.600 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.600 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.600 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.600 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.600 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.600 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.600 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73544 2024-09-14T09:06:53.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.600 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:06:53.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:06:53.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:06:53.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5599999999999995e-07 2024-09-14T09:06:53.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268831+0000", 2024-09-14T09:06:53.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.57e-06 2024-09-14T09:06:53.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268844+0000", 2024-09-14T09:06:53.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2811e-05 2024-09-14T09:06:53.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268851+0000", 2024-09-14T09:06:53.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9140000000000002e-06 2024-09-14T09:06:53.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282083+0000", 2024-09-14T09:06:53.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013232111 2024-09-14T09:06:53.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282086+0000", 2024-09-14T09:06:53.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2909999999999999e-06 2024-09-14T09:06:53.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292884+0000", 2024-09-14T09:06:53.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0107975069999999 2024-09-14T09:06:53.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293100+0000", 2024-09-14T09:06:53.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021639399999999999 2024-09-14T09:06:53.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293311+0000", 2024-09-14T09:06:53.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021090099999999999 2024-09-14T09:06:53.604 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.604 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.605 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.605 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73545 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15564800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.605 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:06:53.605 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.305725974000001, 2024-09-14T09:06:53.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0249746740000001, 2024-09-14T09:06:53.605 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.605 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.605 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.606 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.606 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73545 2024-09-14T09:06:53.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.606 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:06:53.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:06:53.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268880+0000", 2024-09-14T09:06:53.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5600000000000005e-07 2024-09-14T09:06:53.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268889+0000", 2024-09-14T09:06:53.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8559999999999996e-06 2024-09-14T09:06:53.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268913+0000", 2024-09-14T09:06:53.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3958e-05 2024-09-14T09:06:53.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268921+0000", 2024-09-14T09:06:53.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7479999999999999e-06 2024-09-14T09:06:53.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282098+0000", 2024-09-14T09:06:53.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013176297999999999 2024-09-14T09:06:53.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282101+0000", 2024-09-14T09:06:53.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2329999999999998e-06 2024-09-14T09:06:53.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293435+0000", 2024-09-14T09:06:53.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0113339459999999 2024-09-14T09:06:53.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293648+0000", 2024-09-14T09:06:53.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021307100000000001 2024-09-14T09:06:53.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293854+0000", 2024-09-14T09:06:53.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020580800000000001 2024-09-14T09:06:53.611 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.611 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.611 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.612 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73546 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15646720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.612 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:06:53.612 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.305672102999999, 2024-09-14T09:06:53.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0254846360000001, 2024-09-14T09:06:53.612 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.612 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.612 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.612 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.612 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.612 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.612 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73546 2024-09-14T09:06:53.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.612 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:06:53.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:06:53.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268934+0000", 2024-09-14T09:06:53.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7999999999999995e-07 2024-09-14T09:06:53.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268937+0000", 2024-09-14T09:06:53.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9840000000000001e-06 2024-09-14T09:06:53.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268947+0000", 2024-09-14T09:06:53.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0522e-05 2024-09-14T09:06:53.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268955+0000", 2024-09-14T09:06:53.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8059999999999995e-06 2024-09-14T09:06:53.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282112+0000", 2024-09-14T09:06:53.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013157189 2024-09-14T09:06:53.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282115+0000", 2024-09-14T09:06:53.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2949999999999998e-06 2024-09-14T09:06:53.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293968+0000", 2024-09-14T09:06:53.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0118530990000001 2024-09-14T09:06:53.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294197+0000", 2024-09-14T09:06:53.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228507 2024-09-14T09:06:53.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294417+0000", 2024-09-14T09:06:53.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000220554 2024-09-14T09:06:53.617 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.617 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.617 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.617 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73547 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.617 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:06:53.617 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.305639657, 2024-09-14T09:06:53.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.025970525, 2024-09-14T09:06:53.617 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.617 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.617 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.618 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.618 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.618 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.618 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73547 2024-09-14T09:06:53.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.618 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:06:53.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:06:53.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268966+0000", 2024-09-14T09:06:53.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3799999999999997e-07 2024-09-14T09:06:53.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268969+0000", 2024-09-14T09:06:53.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8739999999999999e-06 2024-09-14T09:06:53.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268979+0000", 2024-09-14T09:06:53.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9939999999999995e-06 2024-09-14T09:06:53.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268986+0000", 2024-09-14T09:06:53.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3459999999999996e-06 2024-09-14T09:06:53.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282127+0000", 2024-09-14T09:06:53.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01314039 2024-09-14T09:06:53.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282130+0000", 2024-09-14T09:06:53.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2219999999999998e-06 2024-09-14T09:06:53.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294540+0000", 2024-09-14T09:06:53.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124105880000001 2024-09-14T09:06:53.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294759+0000", 2024-09-14T09:06:53.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000218798 2024-09-14T09:06:53.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294936+0000", 2024-09-14T09:06:53.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000176675 2024-09-14T09:06:53.622 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.622 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.622 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.623 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73548 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15810560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.623 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:06:53.623 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.305608544999998, 2024-09-14T09:06:53.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0265045129999999, 2024-09-14T09:06:53.623 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.623 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.623 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.623 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.623 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.624 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.624 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73548 2024-09-14T09:06:53.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.624 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:06:53.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:06:53.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268997+0000", 2024-09-14T09:06:53.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7599999999999996e-07 2024-09-14T09:06:53.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269000+0000", 2024-09-14T09:06:53.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0580000000000002e-06 2024-09-14T09:06:53.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269010+0000", 2024-09-14T09:06:53.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124e-05 2024-09-14T09:06:53.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269018+0000", 2024-09-14T09:06:53.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5399999999999998e-06 2024-09-14T09:06:53.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282141+0000", 2024-09-14T09:06:53.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013123012 2024-09-14T09:06:53.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282144+0000", 2024-09-14T09:06:53.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.196e-06 2024-09-14T09:06:53.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295042+0000", 2024-09-14T09:06:53.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0128975689999999 2024-09-14T09:06:53.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295272+0000", 2024-09-14T09:06:53.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023048099999999999 2024-09-14T09:06:53.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295501+0000", 2024-09-14T09:06:53.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228657 2024-09-14T09:06:53.628 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.628 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.628 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.628 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73549 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15892480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.628 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:06:53.629 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.305562666, 2024-09-14T09:06:53.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0268354099999999, 2024-09-14T09:06:53.629 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.629 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.629 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.629 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.629 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.629 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.629 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73549 2024-09-14T09:06:53.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.629 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:06:53.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:06:53.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269043+0000", 2024-09-14T09:06:53.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6199999999999997e-07 2024-09-14T09:06:53.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269046+0000", 2024-09-14T09:06:53.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2480000000000001e-06 2024-09-14T09:06:53.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269057+0000", 2024-09-14T09:06:53.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0436e-05 2024-09-14T09:06:53.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269064+0000", 2024-09-14T09:06:53.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5179999999999998e-06 2024-09-14T09:06:53.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282155+0000", 2024-09-14T09:06:53.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013091190000000001 2024-09-14T09:06:53.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282159+0000", 2024-09-14T09:06:53.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.366e-06 2024-09-14T09:06:53.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295599+0000", 2024-09-14T09:06:53.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0134405559999999 2024-09-14T09:06:53.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295765+0000", 2024-09-14T09:06:53.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016575499999999999 2024-09-14T09:06:53.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295878+0000", 2024-09-14T09:06:53.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112679 2024-09-14T09:06:53.634 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.634 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.634 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.635 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73550 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15974400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.635 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:06:53.635 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.305530065999999, 2024-09-14T09:06:53.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027357657, 2024-09-14T09:06:53.635 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.635 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.636 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.636 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.636 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.636 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.636 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73550 2024-09-14T09:06:53.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.637 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:06:53.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:06:53.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269076+0000", 2024-09-14T09:06:53.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:06:53.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269079+0000", 2024-09-14T09:06:53.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:06:53.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269098+0000", 2024-09-14T09:06:53.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9698e-05 2024-09-14T09:06:53.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269106+0000", 2024-09-14T09:06:53.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6179999999999997e-06 2024-09-14T09:06:53.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282170+0000", 2024-09-14T09:06:53.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013063874 2024-09-14T09:06:53.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282173+0000", 2024-09-14T09:06:53.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1999999999999999e-06 2024-09-14T09:06:53.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295996+0000", 2024-09-14T09:06:53.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.013823151 2024-09-14T09:06:53.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296214+0000", 2024-09-14T09:06:53.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021817700000000001 2024-09-14T09:06:53.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296433+0000", 2024-09-14T09:06:53.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021829099999999999 2024-09-14T09:06:53.646 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.646 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.646 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.647 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73551 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.647 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:06:53.647 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.305488649000001, 2024-09-14T09:06:53.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027730276, 2024-09-14T09:06:53.647 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.647 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.648 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.648 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.648 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.648 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.648 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73551 2024-09-14T09:06:53.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.649 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:06:53.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:06:53.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269117+0000", 2024-09-14T09:06:53.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:06:53.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269120+0000", 2024-09-14T09:06:53.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.056e-06 2024-09-14T09:06:53.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269130+0000", 2024-09-14T09:06:53.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0084e-05 2024-09-14T09:06:53.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269137+0000", 2024-09-14T09:06:53.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1359999999999999e-06 2024-09-14T09:06:53.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282184+0000", 2024-09-14T09:06:53.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013046537 2024-09-14T09:06:53.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282188+0000", 2024-09-14T09:06:53.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3220000000000003e-06 2024-09-14T09:06:53.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296529+0000", 2024-09-14T09:06:53.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0143406109999999 2024-09-14T09:06:53.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296695+0000", 2024-09-14T09:06:53.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016604799999999999 2024-09-14T09:06:53.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296847+0000", 2024-09-14T09:06:53.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000151822 2024-09-14T09:06:53.658 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.658 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.658 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.658 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73552 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.658 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:06:53.658 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.305458069, 2024-09-14T09:06:53.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0281767500000001, 2024-09-14T09:06:53.659 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.659 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.659 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.659 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.659 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.659 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.659 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73552 2024-09-14T09:06:53.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.659 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:06:53.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:06:53.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:06:53.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:06:53.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269150+0000", 2024-09-14T09:06:53.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9119999999999998e-06 2024-09-14T09:06:53.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269160+0000", 2024-09-14T09:06:53.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6819999999999998e-06 2024-09-14T09:06:53.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269168+0000", 2024-09-14T09:06:53.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6079999999999998e-06 2024-09-14T09:06:53.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282205+0000", 2024-09-14T09:06:53.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013036982000000001 2024-09-14T09:06:53.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282208+0000", 2024-09-14T09:06:53.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4620000000000001e-06 2024-09-14T09:06:53.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296968+0000", 2024-09-14T09:06:53.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0147601209999999 2024-09-14T09:06:53.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297115+0000", 2024-09-14T09:06:53.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014673900000000001 2024-09-14T09:06:53.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297324+0000", 2024-09-14T09:06:53.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020866400000000001 2024-09-14T09:06:53.664 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.664 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.664 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.664 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73553 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.665 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:06:53.665 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.305428005, 2024-09-14T09:06:53.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0286556739999999, 2024-09-14T09:06:53.665 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.665 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.665 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.665 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.665 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.665 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.665 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73553 2024-09-14T09:06:53.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.666 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:06:53.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:06:53.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269178+0000", 2024-09-14T09:06:53.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:06:53.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269180+0000", 2024-09-14T09:06:53.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T09:06:53.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269189+0000", 2024-09-14T09:06:53.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2e-06 2024-09-14T09:06:53.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269197+0000", 2024-09-14T09:06:53.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2980000000000003e-06 2024-09-14T09:06:53.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282219+0000", 2024-09-14T09:06:53.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013022687 2024-09-14T09:06:53.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282223+0000", 2024-09-14T09:06:53.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.241e-06 2024-09-14T09:06:53.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297470+0000", 2024-09-14T09:06:53.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.015247778 2024-09-14T09:06:53.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297704+0000", 2024-09-14T09:06:53.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000233367 2024-09-14T09:06:53.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297833+0000", 2024-09-14T09:06:53.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012888900000000001 2024-09-14T09:06:53.670 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.670 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.670 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.670 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73554 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.670 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:06:53.670 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.305398688, 2024-09-14T09:06:53.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.029005113, 2024-09-14T09:06:53.670 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.671 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.671 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.671 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.671 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.671 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.671 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73554 2024-09-14T09:06:53.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.671 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:06:53.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:06:53.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269207+0000", 2024-09-14T09:06:53.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.68e-07 2024-09-14T09:06:53.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269210+0000", 2024-09-14T09:06:53.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8100000000000002e-06 2024-09-14T09:06:53.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269219+0000", 2024-09-14T09:06:53.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:06:53.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269227+0000", 2024-09-14T09:06:53.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3839999999999999e-06 2024-09-14T09:06:53.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282234+0000", 2024-09-14T09:06:53.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013007187 2024-09-14T09:06:53.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282237+0000", 2024-09-14T09:06:53.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1200000000000002e-06 2024-09-14T09:06:53.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297942+0000", 2024-09-14T09:06:53.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0157048040000001 2024-09-14T09:06:53.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298087+0000", 2024-09-14T09:06:53.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014499099999999999 2024-09-14T09:06:53.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298211+0000", 2024-09-14T09:06:53.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012461899999999999 2024-09-14T09:06:53.675 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.675 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.676 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.676 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73555 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.676 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:06:53.676 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.30536884, 2024-09-14T09:06:53.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.02941558, 2024-09-14T09:06:53.676 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.676 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.676 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.676 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.676 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.676 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.677 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73555 2024-09-14T09:06:53.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.677 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:06:53.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:06:53.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269237+0000", 2024-09-14T09:06:53.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3399999999999999e-07 2024-09-14T09:06:53.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269240+0000", 2024-09-14T09:06:53.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8480000000000001e-06 2024-09-14T09:06:53.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269249+0000", 2024-09-14T09:06:53.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1880000000000008e-06 2024-09-14T09:06:53.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269256+0000", 2024-09-14T09:06:53.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:06:53.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282248+0000", 2024-09-14T09:06:53.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012992327 2024-09-14T09:06:53.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282251+0000", 2024-09-14T09:06:53.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2160000000000002e-06 2024-09-14T09:06:53.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298299+0000", 2024-09-14T09:06:53.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0160479330000001 2024-09-14T09:06:53.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298446+0000", 2024-09-14T09:06:53.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000146525 2024-09-14T09:06:53.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298652+0000", 2024-09-14T09:06:53.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020584100000000001 2024-09-14T09:06:53.681 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.681 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.681 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.681 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73556 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.681 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:06:53.682 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.305339094000001, 2024-09-14T09:06:53.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0299854239999999, 2024-09-14T09:06:53.682 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.682 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.682 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.682 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.682 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.682 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.682 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73556 2024-09-14T09:06:53.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.682 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:06:53.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:06:53.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:06:53.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:06:53.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269270+0000", 2024-09-14T09:06:53.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0920000000000002e-06 2024-09-14T09:06:53.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269280+0000", 2024-09-14T09:06:53.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0013999999999999e-05 2024-09-14T09:06:53.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269288+0000", 2024-09-14T09:06:53.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:06:53.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282263+0000", 2024-09-14T09:06:53.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012974703000000001 2024-09-14T09:06:53.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282266+0000", 2024-09-14T09:06:53.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1520000000000001e-06 2024-09-14T09:06:53.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298729+0000", 2024-09-14T09:06:53.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.016463506 2024-09-14T09:06:53.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298884+0000", 2024-09-14T09:06:53.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015450800000000001 2024-09-14T09:06:53.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299251+0000", 2024-09-14T09:06:53.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036759899999999997 2024-09-14T09:06:53.686 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.686 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.687 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.687 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73557 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16547840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.687 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:06:53.687 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.305292443999999, 2024-09-14T09:06:53.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0307969480000001, 2024-09-14T09:06:53.687 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.687 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.687 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.687 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.687 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.687 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.687 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73557 2024-09-14T09:06:53.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.688 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:06:53.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:06:53.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269313+0000", 2024-09-14T09:06:53.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.92e-07 2024-09-14T09:06:53.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269336+0000", 2024-09-14T09:06:53.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3118000000000001e-05 2024-09-14T09:06:53.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269346+0000", 2024-09-14T09:06:53.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4069999999999999e-06 2024-09-14T09:06:53.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269352+0000", 2024-09-14T09:06:53.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5810000000000002e-06 2024-09-14T09:06:53.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282277+0000", 2024-09-14T09:06:53.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012924658 2024-09-14T09:06:53.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282280+0000", 2024-09-14T09:06:53.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2880000000000001e-06 2024-09-14T09:06:53.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299343+0000", 2024-09-14T09:06:53.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0170630190000001 2024-09-14T09:06:53.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299554+0000", 2024-09-14T09:06:53.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021085099999999999 2024-09-14T09:06:53.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300109+0000", 2024-09-14T09:06:53.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00055533399999999999 2024-09-14T09:06:53.692 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.692 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.692 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.693 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73558 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.693 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:06:53.693 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.305240821000002, 2024-09-14T09:06:53.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0314849340000001, 2024-09-14T09:06:53.693 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.693 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.693 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.693 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.693 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.693 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.693 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73558 2024-09-14T09:06:53.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.694 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:06:53.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:06:53.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269365+0000", 2024-09-14T09:06:53.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7400000000000003e-07 2024-09-14T09:06:53.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269367+0000", 2024-09-14T09:06:53.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4930000000000001e-06 2024-09-14T09:06:53.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269374+0000", 2024-09-14T09:06:53.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9519999999999997e-06 2024-09-14T09:06:53.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269379+0000", 2024-09-14T09:06:53.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9710000000000003e-06 2024-09-14T09:06:53.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282292+0000", 2024-09-14T09:06:53.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012912704000000001 2024-09-14T09:06:53.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282295+0000", 2024-09-14T09:06:53.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.179e-06 2024-09-14T09:06:53.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300195+0000", 2024-09-14T09:06:53.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.017899975 2024-09-14T09:06:53.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300345+0000", 2024-09-14T09:06:53.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014957499999999999 2024-09-14T09:06:53.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300849+0000", 2024-09-14T09:06:53.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050451100000000002 2024-09-14T09:06:53.698 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.698 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.698 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.698 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73559 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:53.698 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:06:53.698 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.305218411999999, 2024-09-14T09:06:53.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.031916343, 2024-09-14T09:06:53.699 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:53.699 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:53.699 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:53.699 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:53.699 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:53.699 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:53.699 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73559 2024-09-14T09:06:53.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.699 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:53.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:53.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:06:53.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:53.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:06:53.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:53.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:53.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:06:53.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3500000000000002e-07 2024-09-14T09:06:53.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:53.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269390+0000", 2024-09-14T09:06:53.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5890000000000001e-06 2024-09-14T09:06:53.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:53.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269396+0000", 2024-09-14T09:06:53.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3300000000000004e-06 2024-09-14T09:06:53.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:53.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269401+0000", 2024-09-14T09:06:53.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0340000000000004e-06 2024-09-14T09:06:53.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282311+0000", 2024-09-14T09:06:53.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012910204 2024-09-14T09:06:53.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:53.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282317+0000", 2024-09-14T09:06:53.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2639999999999997e-06 2024-09-14T09:06:53.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:53.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300933+0000", 2024-09-14T09:06:53.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0186160710000001 2024-09-14T09:06:53.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:53.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301158+0000", 2024-09-14T09:06:53.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225021 2024-09-14T09:06:53.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:53.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:53.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:53.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301303+0000", 2024-09-14T09:06:53.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014439499999999999 2024-09-14T09:06:53.703 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.703 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.703 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.703 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:53.703 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:53.703 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:53.704 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:53.579+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:53.705 INFO:teuthology.orchestra.run.smithi176.stdout:4115 punch from 0x1fca512 to 0x1fd00bc, (0x5baa bytes) 2024-09-14T09:06:53.705 INFO:teuthology.orchestra.run.smithi176.stdout:4117 read 0x28a5b74 thru 0x28abe82 (0x630f bytes) 2024-09-14T09:06:53.705 INFO:teuthology.orchestra.run.smithi176.stdout:4118 write 0x730df29 thru 0x7317a88 (0x9b60 bytes) 2024-09-14T09:06:53.706 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:06:53.707 INFO:teuthology.orchestra.run.smithi176.stdout:4119 punch from 0x1c2cc82 to 0x1c312d7, (0x4655 bytes) 2024-09-14T09:06:53.707 INFO:teuthology.orchestra.run.smithi176.stdout:4120 trunc from 0x7317a89 to 0x4227309 2024-09-14T09:06:53.708 INFO:teuthology.orchestra.run.smithi176.stdout:4121 trunc from 0x4227309 to 0x5a8ece1 2024-09-14T09:06:53.708 INFO:teuthology.orchestra.run.smithi176.stdout:4123 punch from 0x15d116d to 0x15e00e5, (0xef78 bytes) 2024-09-14T09:06:53.708 INFO:teuthology.orchestra.run.smithi176.stdout:4124 read 0x516676e thru 0x5174e48 (0xe6db bytes) 2024-09-14T09:06:53.708 INFO:teuthology.orchestra.run.smithi176.stdout:4126 writesame 0xc536534 thru 0xc53cedb data_size 0x45(0x69a8 bytes) 2024-09-14T09:06:53.708 INFO:teuthology.orchestra.run.smithi176.stdout:4127 writesame 0x90d3c70 thru 0x90e2f27 data_size 0x24(0xf2b8 bytes) 2024-09-14T09:06:53.708 INFO:teuthology.orchestra.run.smithi176.stdout:4128 writesame 0x6242524 thru 0x6246bb8 data_size 0x13(0x4695 bytes) 2024-09-14T09:06:53.708 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:53.708 INFO:teuthology.orchestra.run.smithi176.stdout:4130 writesame 0x225a8cc thru 0x2267cae data_size 0x93(0xd3e3 bytes) 2024-09-14T09:06:53.708 INFO:teuthology.orchestra.run.smithi176.stdout:4132 read 0x4f37ef7 thru 0x4f454d9 (0xd5e3 bytes) 2024-09-14T09:06:53.709 INFO:teuthology.orchestra.run.smithi176.stdout:4133 read 0x6845555 thru 0x68494e9 (0x3f95 bytes) 2024-09-14T09:06:53.709 INFO:teuthology.orchestra.run.smithi176.stdout:4134 read 0x1144aaf thru 0x11478a2 (0x2df4 bytes) 2024-09-14T09:06:53.709 INFO:teuthology.orchestra.run.smithi176.stdout:4135 trunc from 0xc53cedc to 0x3f6a6a5 2024-09-14T09:06:53.709 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:53.707+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:53.723 INFO:teuthology.orchestra.run.smithi176.stdout:4136 read 0x11c0c4a thru 0x11c7916 (0x6ccd bytes) 2024-09-14T09:06:53.737 INFO:teuthology.orchestra.run.smithi176.stdout:4137 trunc from 0x3f6a6a5 to 0xa4a8032 2024-09-14T09:06:53.746 INFO:teuthology.orchestra.run.smithi176.stdout:4138 write 0x53b5f0 thru 0x53c664 (0x1075 bytes) 2024-09-14T09:06:53.773 INFO:teuthology.orchestra.run.smithi176.stdout:4141 read 0x77917da thru 0x779eab7 (0xd2de bytes) 2024-09-14T09:06:53.773 INFO:teuthology.orchestra.run.smithi176.stdout:4142 write 0x12ddb88 thru 0x12e064b (0x2ac4 bytes) 2024-09-14T09:06:53.793 INFO:teuthology.orchestra.run.smithi176.stdout:4144 write 0xb3c2006 thru 0xb3ce8b4 (0xc8af bytes) 2024-09-14T09:06:53.800 INFO:teuthology.orchestra.run.smithi176.stdout:4145 read 0x18b3b18 thru 0x18b4920 (0xe09 bytes) 2024-09-14T09:06:53.800 INFO:teuthology.orchestra.run.smithi176.stdout:4147 punch from 0x5a3ddc2 to 0x5a4b40a, (0xd648 bytes) 2024-09-14T09:06:53.806 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:53.803+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:53.809 INFO:teuthology.orchestra.run.smithi176.stdout:4148 trunc from 0xb3ce8b5 to 0x67569b5 2024-09-14T09:06:53.825 INFO:teuthology.orchestra.run.smithi176.stdout:4149 punch from 0x24e4c3c to 0x24e62a0, (0x1664 bytes) 2024-09-14T09:06:53.830 INFO:teuthology.orchestra.run.smithi176.stdout:4153 trunc from 0x67569b5 to 0x3d84b5f 2024-09-14T09:06:53.838 INFO:teuthology.orchestra.run.smithi176.stdout:4155 read 0x22f3018 thru 0x2302dc2 (0xfdab bytes) 2024-09-14T09:06:53.839 INFO:teuthology.orchestra.run.smithi176.stdout:4156 read 0x14595b0 thru 0x14676a1 (0xe0f2 bytes) 2024-09-14T09:06:53.839 INFO:teuthology.orchestra.run.smithi176.stdout:4157 read 0xbf6e37 thru 0xbfda55 (0x6c1f bytes) 2024-09-14T09:06:53.840 INFO:teuthology.orchestra.run.smithi176.stdout:4158 read 0x3ab2e7e thru 0x3abb339 (0x84bc bytes) 2024-09-14T09:06:53.840 INFO:teuthology.orchestra.run.smithi176.stdout:4160 read 0x13d9cd8 thru 0x13df41a (0x5743 bytes) 2024-09-14T09:06:53.840 INFO:teuthology.orchestra.run.smithi176.stdout:4161 punch from 0x3744d61 to 0x374559f, (0x83e bytes) 2024-09-14T09:06:53.864 INFO:teuthology.orchestra.run.smithi176.stdout:4162 write 0xb1d3983 thru 0xb1e15c2 (0xdc40 bytes) 2024-09-14T09:06:53.873 INFO:teuthology.orchestra.run.smithi176.stdout:4163 writesame 0xc6baad thru 0xc77b71 data_size 0x1(0xc0c5 bytes) 2024-09-14T09:06:53.883 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:53.883 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:53.883 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:53.900 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:06:53.906 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:53.907+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:53.935 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T09:06:54.002 INFO:teuthology.orchestra.run.smithi176.stdout:4164 read 0x80f67a thru 0x81e092 (0xea19 bytes) 2024-09-14T09:06:54.003 INFO:teuthology.orchestra.run.smithi176.stdout:4165 read 0xa9ad8a1 thru 0xa9b675a (0x8eba bytes) 2024-09-14T09:06:54.003 INFO:teuthology.orchestra.run.smithi176.stdout:4166 write 0x75d324a thru 0x75db1e8 (0x7f9f bytes) 2024-09-14T09:06:54.007 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:54.007+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:54.009 INFO:teuthology.orchestra.run.smithi176.stdout:4167 read 0x3bfbf3a thru 0x3bffcb6 (0x3d7d bytes) 2024-09-14T09:06:54.009 INFO:teuthology.orchestra.run.smithi176.stdout:4168 write 0xa46924f thru 0xa46b886 (0x2638 bytes) 2024-09-14T09:06:54.026 INFO:teuthology.orchestra.run.smithi176.stdout:4171 trunc from 0xb1e15c3 to 0x45a7f66 2024-09-14T09:06:54.046 INFO:teuthology.orchestra.run.smithi176.stdout:4172 punch from 0x1a76529 to 0x1a824d9, (0xbfb0 bytes) 2024-09-14T09:06:54.046 INFO:teuthology.orchestra.run.smithi176.stdout:4173 writesame 0x2d3637e thru 0x2d41462 data_size 0x5(0xb0e5 bytes) 2024-09-14T09:06:54.058 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:54.069 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:54.069 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:54.069 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:54.069 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:54.069 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:54.073 INFO:teuthology.orchestra.run.smithi176.stdout:4176 trunc from 0x45a7f66 to 0xe71ce73 2024-09-14T09:06:54.082 INFO:teuthology.orchestra.run.smithi176.stdout:4179 write 0xd0831de thru 0xd084241 (0x1064 bytes) 2024-09-14T09:06:54.088 INFO:teuthology.orchestra.run.smithi176.stdout:4180 write 0x6d46146 thru 0x6d56123 (0xffde bytes) 2024-09-14T09:06:54.089 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:06:54.098 INFO:teuthology.orchestra.run.smithi176.stdout:4181 read 0x3787d4f thru 0x3794ad8 (0xcd8a bytes) 2024-09-14T09:06:54.098 INFO:teuthology.orchestra.run.smithi176.stdout:4182 writesame 0x9ce1b45 thru 0x9ceb9cb data_size 0x1(0x9e87 bytes) 2024-09-14T09:06:54.142 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:54.143+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:54.209 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:54.211+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:54.213 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:54.213 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:54.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:54.214 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:54.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.214 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18383 3.9 3:9252f7c9:::rbd_data.115e3abd3b2c.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:06:54.214 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:54.214 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.055323001999994, 2024-09-14T09:06:54.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056433830999999997, 2024-09-14T09:06:54.214 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.215 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.215 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:54.215 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.215 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.215 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.215 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18383 2024-09-14T09:06:54.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.215 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:54.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:54.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158423+0000", 2024-09-14T09:06:54.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T09:06:54.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158429+0000", 2024-09-14T09:06:54.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0179999999999998e-06 2024-09-14T09:06:54.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158440+0000", 2024-09-14T09:06:54.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0769e-05 2024-09-14T09:06:54.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158445+0000", 2024-09-14T09:06:54.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5840000000000002e-06 2024-09-14T09:06:54.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158495+0000", 2024-09-14T09:06:54.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9129999999999999e-05 2024-09-14T09:06:54.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158598+0000", 2024-09-14T09:06:54.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102938 2024-09-14T09:06:54.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:54.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.161048+0000", 2024-09-14T09:06:54.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002450336 2024-09-14T09:06:54.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:54.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214605+0000", 2024-09-14T09:06:54.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053557526000000001 2024-09-14T09:06:54.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:54.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214634+0000", 2024-09-14T09:06:54.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8483000000000002e-05 2024-09-14T09:06:54.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:54.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214731+0000", 2024-09-14T09:06:54.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7026000000000002e-05 2024-09-14T09:06:54.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214856+0000", 2024-09-14T09:06:54.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012474799999999999 2024-09-14T09:06:54.219 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.219 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.219 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.220 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18383 3.9 e38/34 ondisk, result = 0)", 2024-09-14T09:06:54.220 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:54.220 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.045006521000005, 2024-09-14T09:06:54.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045880575999999999, 2024-09-14T09:06:54.220 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.220 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.220 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.220 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:54.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:54.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168739+0000", 2024-09-14T09:06:54.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3900000000000004e-07 2024-09-14T09:06:54.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168744+0000", 2024-09-14T09:06:54.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5929999999999999e-06 2024-09-14T09:06:54.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168754+0000", 2024-09-14T09:06:54.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0686999999999999e-05 2024-09-14T09:06:54.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168761+0000", 2024-09-14T09:06:54.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6340000000000003e-06 2024-09-14T09:06:54.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214578+0000", 2024-09-14T09:06:54.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045817443999999999 2024-09-14T09:06:54.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214596+0000", 2024-09-14T09:06:54.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7159e-05 2024-09-14T09:06:54.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214619+0000", 2024-09-14T09:06:54.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.332e-05 2024-09-14T09:06:54.224 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.224 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.224 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.224 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19152 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14159058~8940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:54.224 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:54.224 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 69.891094065000004, 2024-09-14T09:06:54.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12891994500000001, 2024-09-14T09:06:54.224 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.224 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.224 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:54.224 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.225 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.225 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.225 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19152 2024-09-14T09:06:54.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.225 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:54.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:54.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322653+0000", 2024-09-14T09:06:54.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3750000000000001e-06 2024-09-14T09:06:54.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329647+0000", 2024-09-14T09:06:54.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069941960000000003 2024-09-14T09:06:54.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329673+0000", 2024-09-14T09:06:54.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5760000000000001e-05 2024-09-14T09:06:54.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329684+0000", 2024-09-14T09:06:54.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0474e-05 2024-09-14T09:06:54.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329776+0000", 2024-09-14T09:06:54.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2467999999999996e-05 2024-09-14T09:06:54.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329987+0000", 2024-09-14T09:06:54.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021133 2024-09-14T09:06:54.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:54.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.331303+0000", 2024-09-14T09:06:54.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001315629 2024-09-14T09:06:54.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:54.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.411090+0000", 2024-09-14T09:06:54.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079787477999999995 2024-09-14T09:06:54.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:54.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451463+0000", 2024-09-14T09:06:54.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040372312 2024-09-14T09:06:54.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:54.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451523+0000", 2024-09-14T09:06:54.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0420000000000001e-05 2024-09-14T09:06:54.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451571+0000", 2024-09-14T09:06:54.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7503000000000002e-05 2024-09-14T09:06:54.230 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.230 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.230 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.230 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20926 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:54.230 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:54.230 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.541762065, 2024-09-14T09:06:54.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018572207, 2024-09-14T09:06:54.230 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.230 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.230 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.231 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.231 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.231 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.231 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20926 2024-09-14T09:06:54.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.231 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:54.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:54.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:54.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:06:54.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671990+0000", 2024-09-14T09:06:54.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5769999999999999e-06 2024-09-14T09:06:54.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672001+0000", 2024-09-14T09:06:54.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1293e-05 2024-09-14T09:06:54.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672008+0000", 2024-09-14T09:06:54.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9099999999999999e-06 2024-09-14T09:06:54.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689147+0000", 2024-09-14T09:06:54.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017139060000000001 2024-09-14T09:06:54.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689324+0000", 2024-09-14T09:06:54.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017640200000000001 2024-09-14T09:06:54.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690555+0000", 2024-09-14T09:06:54.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001231212 2024-09-14T09:06:54.234 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.234 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.234 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.235 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20927 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2621440~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:54.235 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:54.235 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.541725099000004, 2024-09-14T09:06:54.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020002227000000001, 2024-09-14T09:06:54.235 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.235 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.235 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.235 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.235 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.235 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.235 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20927 2024-09-14T09:06:54.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.235 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:54.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:54.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:54.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9699999999999995e-07 2024-09-14T09:06:54.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672024+0000", 2024-09-14T09:06:54.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:06:54.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672036+0000", 2024-09-14T09:06:54.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.119e-05 2024-09-14T09:06:54.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672044+0000", 2024-09-14T09:06:54.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8370000000000003e-06 2024-09-14T09:06:54.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690602+0000", 2024-09-14T09:06:54.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018557648 2024-09-14T09:06:54.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690805+0000", 2024-09-14T09:06:54.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020282899999999999 2024-09-14T09:06:54.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692022+0000", 2024-09-14T09:06:54.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012170429999999999 2024-09-14T09:06:54.239 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.239 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.239 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.239 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20928 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 917504~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:54.239 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:54.239 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.541688958999998, 2024-09-14T09:06:54.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021623738, 2024-09-14T09:06:54.239 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.239 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.239 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.239 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.239 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.239 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.240 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20928 2024-09-14T09:06:54.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.240 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:54.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:54.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672057+0000", 2024-09-14T09:06:54.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:06:54.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672059+0000", 2024-09-14T09:06:54.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9270000000000001e-06 2024-09-14T09:06:54.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672070+0000", 2024-09-14T09:06:54.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0773e-05 2024-09-14T09:06:54.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672077+0000", 2024-09-14T09:06:54.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9600000000000003e-06 2024-09-14T09:06:54.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692154+0000", 2024-09-14T09:06:54.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020076482 2024-09-14T09:06:54.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692419+0000", 2024-09-14T09:06:54.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026517400000000002 2024-09-14T09:06:54.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693680+0000", 2024-09-14T09:06:54.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012607390000000001 2024-09-14T09:06:54.243 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.243 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.243 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.243 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20929 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2752512~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:54.243 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:54.243 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.541655917, 2024-09-14T09:06:54.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022898728, 2024-09-14T09:06:54.244 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.244 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.244 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.244 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.244 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.244 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.244 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20929 2024-09-14T09:06:54.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.244 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:54.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:54.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672090+0000", 2024-09-14T09:06:54.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:06:54.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672092+0000", 2024-09-14T09:06:54.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8940000000000002e-06 2024-09-14T09:06:54.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672103+0000", 2024-09-14T09:06:54.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0815999999999999e-05 2024-09-14T09:06:54.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672110+0000", 2024-09-14T09:06:54.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9469999999999997e-06 2024-09-14T09:06:54.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693737+0000", 2024-09-14T09:06:54.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021626467999999999 2024-09-14T09:06:54.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693962+0000", 2024-09-14T09:06:54.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225474 2024-09-14T09:06:54.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.694988+0000", 2024-09-14T09:06:54.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001025469 2024-09-14T09:06:54.247 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.247 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.247 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.248 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20930 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:54.248 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:54.248 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.541622822999997, 2024-09-14T09:06:54.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024303286, 2024-09-14T09:06:54.248 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.248 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.248 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.248 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.248 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.248 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.248 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20930 2024-09-14T09:06:54.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.248 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:54.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:54.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672123+0000", 2024-09-14T09:06:54.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:06:54.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672125+0000", 2024-09-14T09:06:54.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7300000000000001e-06 2024-09-14T09:06:54.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672134+0000", 2024-09-14T09:06:54.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7930000000000003e-06 2024-09-14T09:06:54.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672141+0000", 2024-09-14T09:06:54.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9029999999999998e-06 2024-09-14T09:06:54.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695087+0000", 2024-09-14T09:06:54.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022946013000000001 2024-09-14T09:06:54.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695350+0000", 2024-09-14T09:06:54.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026266000000000001 2024-09-14T09:06:54.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696425+0000", 2024-09-14T09:06:54.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001075527 2024-09-14T09:06:54.251 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.252 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.252 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.252 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20931 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:54.252 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:54.252 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.541589594000001, 2024-09-14T09:06:54.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025331519, 2024-09-14T09:06:54.252 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.252 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.252 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.252 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.252 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.252 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.252 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20931 2024-09-14T09:06:54.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.253 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:54.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:54.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672156+0000", 2024-09-14T09:06:54.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4300000000000002e-07 2024-09-14T09:06:54.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672159+0000", 2024-09-14T09:06:54.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T09:06:54.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672170+0000", 2024-09-14T09:06:54.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1192999999999999e-05 2024-09-14T09:06:54.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672177+0000", 2024-09-14T09:06:54.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9059999999999996e-06 2024-09-14T09:06:54.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696532+0000", 2024-09-14T09:06:54.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024355291000000001 2024-09-14T09:06:54.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696755+0000", 2024-09-14T09:06:54.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022297699999999999 2024-09-14T09:06:54.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697487+0000", 2024-09-14T09:06:54.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00073171499999999997 2024-09-14T09:06:54.256 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.256 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.256 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.256 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20932 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1179648~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:54.256 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:54.256 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.541556150999995, 2024-09-14T09:06:54.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026104083, 2024-09-14T09:06:54.256 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.257 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.257 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.257 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.257 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.257 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.257 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20932 2024-09-14T09:06:54.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.257 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:54.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:54.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:54.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0699999999999996e-07 2024-09-14T09:06:54.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672192+0000", 2024-09-14T09:06:54.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7700000000000002e-06 2024-09-14T09:06:54.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672203+0000", 2024-09-14T09:06:54.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0956e-05 2024-09-14T09:06:54.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672211+0000", 2024-09-14T09:06:54.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3769999999999996e-06 2024-09-14T09:06:54.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697530+0000", 2024-09-14T09:06:54.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025318411999999998 2024-09-14T09:06:54.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697640+0000", 2024-09-14T09:06:54.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011045299999999999 2024-09-14T09:06:54.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698293+0000", 2024-09-14T09:06:54.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065240800000000005 2024-09-14T09:06:54.260 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.260 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.260 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.261 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20933 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3014656~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:54.261 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:54.261 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.541521947999996, 2024-09-14T09:06:54.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026558875999999999, 2024-09-14T09:06:54.261 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.261 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.261 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.261 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.261 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.261 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.261 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20933 2024-09-14T09:06:54.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.261 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:54.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:54.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672224+0000", 2024-09-14T09:06:54.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2399999999999997e-07 2024-09-14T09:06:54.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672234+0000", 2024-09-14T09:06:54.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0793e-05 2024-09-14T09:06:54.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672246+0000", 2024-09-14T09:06:54.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659e-05 2024-09-14T09:06:54.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672255+0000", 2024-09-14T09:06:54.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5730000000000008e-06 2024-09-14T09:06:54.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698326+0000", 2024-09-14T09:06:54.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026071414000000001 2024-09-14T09:06:54.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698374+0000", 2024-09-14T09:06:54.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8032999999999999e-05 2024-09-14T09:06:54.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698782+0000", 2024-09-14T09:06:54.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040768 2024-09-14T09:06:54.265 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.265 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.265 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.265 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20934 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1310720~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:54.265 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:54.265 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.541480071999999, 2024-09-14T09:06:54.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027292067999999999, 2024-09-14T09:06:54.265 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.265 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.265 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20934 2024-09-14T09:06:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672266+0000", 2024-09-14T09:06:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6299999999999993e-07 2024-09-14T09:06:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672269+0000", 2024-09-14T09:06:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8569999999999999e-06 2024-09-14T09:06:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672280+0000", 2024-09-14T09:06:54.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1236999999999999e-05 2024-09-14T09:06:54.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672287+0000", 2024-09-14T09:06:54.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8260000000000004e-06 2024-09-14T09:06:54.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698820+0000", 2024-09-14T09:06:54.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026533055999999999 2024-09-14T09:06:54.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698933+0000", 2024-09-14T09:06:54.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011291 2024-09-14T09:06:54.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699557+0000", 2024-09-14T09:06:54.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062421899999999997 2024-09-14T09:06:54.269 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.269 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.269 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.269 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20935 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:54.270 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:54.270 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.541446312000005, 2024-09-14T09:06:54.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028108610999999999, 2024-09-14T09:06:54.270 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.270 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.270 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.270 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.270 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.270 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.270 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20935 2024-09-14T09:06:54.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.270 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:54.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:54.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672299+0000", 2024-09-14T09:06:54.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T09:06:54.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672302+0000", 2024-09-14T09:06:54.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7640000000000001e-06 2024-09-14T09:06:54.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672313+0000", 2024-09-14T09:06:54.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1080000000000001e-05 2024-09-14T09:06:54.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672321+0000", 2024-09-14T09:06:54.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6490000000000008e-06 2024-09-14T09:06:54.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699587+0000", 2024-09-14T09:06:54.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027265884000000001 2024-09-14T09:06:54.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699631+0000", 2024-09-14T09:06:54.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4947999999999997e-05 2024-09-14T09:06:54.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700407+0000", 2024-09-14T09:06:54.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00077559299999999999 2024-09-14T09:06:54.274 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.274 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.274 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.274 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20936 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1441792~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:54.274 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:54.274 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.541412136000005, 2024-09-14T09:06:54.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028784750000000001, 2024-09-14T09:06:54.274 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.274 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.274 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.274 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.275 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.275 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.275 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20936 2024-09-14T09:06:54.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.275 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:54.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:54.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:54.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4700000000000001e-07 2024-09-14T09:06:54.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672336+0000", 2024-09-14T09:06:54.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6730000000000001e-06 2024-09-14T09:06:54.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672347+0000", 2024-09-14T09:06:54.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0710000000000001e-05 2024-09-14T09:06:54.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672354+0000", 2024-09-14T09:06:54.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4630000000000004e-06 2024-09-14T09:06:54.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700442+0000", 2024-09-14T09:06:54.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028087951 2024-09-14T09:06:54.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700557+0000", 2024-09-14T09:06:54.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011461800000000001 2024-09-14T09:06:54.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701117+0000", 2024-09-14T09:06:54.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056058799999999999 2024-09-14T09:06:54.278 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.278 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.278 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.278 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20937 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:54.278 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:54.278 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.54137901, 2024-09-14T09:06:54.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029493028000000001, 2024-09-14T09:06:54.278 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.278 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.279 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.279 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.279 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.279 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.279 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20937 2024-09-14T09:06:54.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.279 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:54.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:54.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:54.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T09:06:54.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672384+0000", 2024-09-14T09:06:54.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7127000000000001e-05 2024-09-14T09:06:54.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672395+0000", 2024-09-14T09:06:54.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759999999999999e-05 2024-09-14T09:06:54.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672402+0000", 2024-09-14T09:06:54.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0960000000000003e-06 2024-09-14T09:06:54.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701147+0000", 2024-09-14T09:06:54.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028744241 2024-09-14T09:06:54.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701246+0000", 2024-09-14T09:06:54.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9031999999999996e-05 2024-09-14T09:06:54.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701859+0000", 2024-09-14T09:06:54.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061310499999999997 2024-09-14T09:06:54.282 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.282 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.282 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.282 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20938 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:54.282 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:54.282 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.541331850000006, 2024-09-14T09:06:54.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030109219999999999, 2024-09-14T09:06:54.282 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.283 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.283 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.283 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.283 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.283 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.283 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20938 2024-09-14T09:06:54.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.283 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:54.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:54.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672414+0000", 2024-09-14T09:06:54.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-07 2024-09-14T09:06:54.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672417+0000", 2024-09-14T09:06:54.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:06:54.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672428+0000", 2024-09-14T09:06:54.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0873000000000001e-05 2024-09-14T09:06:54.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672435+0000", 2024-09-14T09:06:54.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:06:54.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701884+0000", 2024-09-14T09:06:54.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029449811999999999 2024-09-14T09:06:54.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701934+0000", 2024-09-14T09:06:54.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9703000000000002e-05 2024-09-14T09:06:54.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702522+0000", 2024-09-14T09:06:54.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00058811200000000001 2024-09-14T09:06:54.286 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.286 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.286 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.286 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20939 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:54.286 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:54.286 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.541299717000001, 2024-09-14T09:06:54.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030779259, 2024-09-14T09:06:54.287 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.287 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.287 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.287 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.287 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.287 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.287 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20939 2024-09-14T09:06:54.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.287 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:54.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:54.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672446+0000", 2024-09-14T09:06:54.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:06:54.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672449+0000", 2024-09-14T09:06:54.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7930000000000002e-06 2024-09-14T09:06:54.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672461+0000", 2024-09-14T09:06:54.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T09:06:54.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672471+0000", 2024-09-14T09:06:54.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0807e-05 2024-09-14T09:06:54.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702551+0000", 2024-09-14T09:06:54.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030079228 2024-09-14T09:06:54.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702664+0000", 2024-09-14T09:06:54.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011351800000000001 2024-09-14T09:06:54.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.703224+0000", 2024-09-14T09:06:54.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056026999999999997 2024-09-14T09:06:54.290 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.291 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.291 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.291 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:72788 3.1f e74/28 3:fa166c1d:::rbd_data.115ea8d131ff.0000000000000003:head v 74'947, mlcod=74'945)", 2024-09-14T09:06:54.291 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.121215+0000", 2024-09-14T09:06:54.291 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.092529661, 2024-09-14T09:06:54.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017555599000000002, 2024-09-14T09:06:54.291 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.291 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.291 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:54.291 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.121215+0000", 2024-09-14T09:06:54.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.121215+0000", 2024-09-14T09:06:54.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.121216+0000", 2024-09-14T09:06:54.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9599999999999998e-07 2024-09-14T09:06:54.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.123528+0000", 2024-09-14T09:06:54.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0023118050000000001 2024-09-14T09:06:54.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.123544+0000", 2024-09-14T09:06:54.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6294e-05 2024-09-14T09:06:54.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.123552+0000", 2024-09-14T09:06:54.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9659999999999996e-06 2024-09-14T09:06:54.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.123624+0000", 2024-09-14T09:06:54.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1766000000000004e-05 2024-09-14T09:06:54.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.123653+0000", 2024-09-14T09:06:54.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8858000000000001e-05 2024-09-14T09:06:54.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:54.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.138740+0000", 2024-09-14T09:06:54.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.015087194 2024-09-14T09:06:54.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.138771+0000", 2024-09-14T09:06:54.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.082e-05 2024-09-14T09:06:54.295 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.295 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.295 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.295 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:74298 3.9 3:95c8282a:::rbd_data.115ea8d131ff.0000000000000041:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:54.295 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:06:54.295 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.415989705000001, 2024-09-14T09:06:54.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042668064999999998, 2024-09-14T09:06:54.295 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.296 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.296 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:54.296 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.296 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.296 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.296 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74298 2024-09-14T09:06:54.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.296 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:06:54.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:06:54.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797758+0000", 2024-09-14T09:06:54.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9799999999999998e-06 2024-09-14T09:06:54.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797770+0000", 2024-09-14T09:06:54.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1667e-05 2024-09-14T09:06:54.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797797+0000", 2024-09-14T09:06:54.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7212999999999998e-05 2024-09-14T09:06:54.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797811+0000", 2024-09-14T09:06:54.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4256e-05 2024-09-14T09:06:54.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797882+0000", 2024-09-14T09:06:54.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0622000000000002e-05 2024-09-14T09:06:54.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797943+0000", 2024-09-14T09:06:54.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0893000000000001e-05 2024-09-14T09:06:54.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:54.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.799676+0000", 2024-09-14T09:06:54.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0017331619999999999 2024-09-14T09:06:54.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:54.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840109+0000", 2024-09-14T09:06:54.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040432994 2024-09-14T09:06:54.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:54.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840204+0000", 2024-09-14T09:06:54.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5247000000000005e-05 2024-09-14T09:06:54.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:54.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840346+0000", 2024-09-14T09:06:54.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014156499999999999 2024-09-14T09:06:54.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.301 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840423+0000", 2024-09-14T09:06:54.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7466000000000007e-05 2024-09-14T09:06:54.301 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.301 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.301 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.301 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:74298 3.9 e76/34 ondisk, result = 0)", 2024-09-14T09:06:54.301 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:06:54.302 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.409595062000001, 2024-09-14T09:06:54.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036289318000000001, 2024-09-14T09:06:54.302 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.302 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.302 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.302 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:06:54.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:06:54.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804151+0000", 2024-09-14T09:06:54.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6700000000000001e-06 2024-09-14T09:06:54.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.303 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.303 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804163+0000", 2024-09-14T09:06:54.303 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.148e-05 2024-09-14T09:06:54.303 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.303 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804197+0000", 2024-09-14T09:06:54.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3909999999999999e-05 2024-09-14T09:06:54.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804217+0000", 2024-09-14T09:06:54.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0287000000000001e-05 2024-09-14T09:06:54.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840176+0000", 2024-09-14T09:06:54.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035958641 2024-09-14T09:06:54.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840193+0000", 2024-09-14T09:06:54.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6986e-05 2024-09-14T09:06:54.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840439+0000", 2024-09-14T09:06:54.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024634400000000001 2024-09-14T09:06:54.305 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.305 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.305 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.305 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.305 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.306 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:54.307 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:06:54.308 INFO:teuthology.orchestra.run.smithi176.stdout:4183 punch from 0x8ec3789 to 0x8ecc774, (0x8feb bytes) 2024-09-14T09:06:54.308 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:54.308 INFO:teuthology.orchestra.run.smithi176.stdout:4185 read 0xae241cb thru 0xae2b7ca (0x7600 bytes) 2024-09-14T09:06:54.308 INFO:teuthology.orchestra.run.smithi176.stdout:4186 punch from 0x920b88a to 0x921259c, (0x6d12 bytes) 2024-09-14T09:06:54.308 INFO:teuthology.orchestra.run.smithi176.stdout:4188 read 0xbe82610 thru 0xbe86c38 (0x4629 bytes) 2024-09-14T09:06:54.308 INFO:teuthology.orchestra.run.smithi176.stdout:4189 write 0x40496a4 thru 0x404fef8 (0x6855 bytes) 2024-09-14T09:06:54.309 INFO:teuthology.orchestra.run.smithi176.stdout:4190 read 0xd4c28a thru 0xd57cc7 (0xba3e bytes) 2024-09-14T09:06:54.309 INFO:teuthology.orchestra.run.smithi176.stdout:4191 write 0x96a5e4 thru 0x978e58 (0xe875 bytes) 2024-09-14T09:06:54.309 INFO:teuthology.orchestra.run.smithi176.stdout:4192 read 0x2d80d88 thru 0x2d81322 (0x59b bytes) 2024-09-14T09:06:54.309 INFO:teuthology.orchestra.run.smithi176.stdout:4193 writesame 0x5ea48ec thru 0x5ea5c28 data_size 0x5(0x133d bytes) 2024-09-14T09:06:54.309 INFO:teuthology.orchestra.run.smithi176.stdout:4194 write 0xcef14ca thru 0xceff9ea (0xe521 bytes) 2024-09-14T09:06:54.309 INFO:teuthology.orchestra.run.smithi176.stdout:4195 punch from 0x43e2a36 to 0x43ee013, (0xb5dd bytes) 2024-09-14T09:06:54.309 INFO:teuthology.orchestra.run.smithi176.stdout:4196 write 0x103aef thru 0x10a7e6 (0x6cf8 bytes) 2024-09-14T09:06:54.311 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:54.311+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:54.328 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:54.328 INFO:teuthology.orchestra.run.smithi176.stdout:4199 write 0x4d5015d thru 0x4d53cdb (0x3b7f bytes) 2024-09-14T09:06:54.336 INFO:teuthology.orchestra.run.smithi176.stdout:4200 writesame 0xdeaf3f9 thru 0xdeb8c5d data_size 0x1(0x9865 bytes) 2024-09-14T09:06:54.411 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:54.415+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:54.456 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:54.457 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:54.457 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:54.457 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:54.462 INFO:teuthology.orchestra.run.smithi176.stdout:4203 trunc from 0xe71ce73 to 0x1105bb8 2024-09-14T09:06:54.478 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:06:54.490 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:54.498 INFO:teuthology.orchestra.run.smithi176.stdout:4205 writesame 0x16ba376 thru 0x16bb445 data_size 0x4(0x10d0 bytes) 2024-09-14T09:06:54.512 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:54.515+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:54.519 INFO:teuthology.orchestra.run.smithi176.stdout:4206 writesame 0x98839fd thru 0x9884074 data_size 0x18(0x678 bytes) 2024-09-14T09:06:54.519 INFO:teuthology.orchestra.run.smithi176.stdout:4208 punch from 0x3c015d2 to 0x3c034c1, (0x1eef bytes) 2024-09-14T09:06:54.519 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:54.528 INFO:teuthology.orchestra.run.smithi176.stdout:4211 trunc from 0x9884075 to 0xe30244f 2024-09-14T09:06:54.536 INFO:teuthology.orchestra.run.smithi176.stdout:4212 write 0xd78fc50 thru 0xd79088b (0xc3c bytes) 2024-09-14T09:06:54.544 INFO:teuthology.orchestra.run.smithi176.stdout:4214 writesame 0x83afdcf thru 0x83b4c86 data_size 0x2(0x4eb8 bytes) 2024-09-14T09:06:54.556 INFO:teuthology.orchestra.run.smithi176.stdout:4215 writesame 0x44b2335 thru 0x44ba19e data_size 0xb(0x7e6a bytes) 2024-09-14T09:06:54.570 INFO:teuthology.orchestra.run.smithi176.stdout:4217 write 0x6ac4d1b thru 0x6ac7ec7 (0x31ad bytes) 2024-09-14T09:06:54.584 INFO:teuthology.orchestra.run.smithi176.stdout:4218 trunc from 0xe30244f to 0xb6a4889 2024-09-14T09:06:54.609 INFO:teuthology.orchestra.run.smithi176.stdout:4219 read 0xa41f583 thru 0xa4273a2 (0x7e20 bytes) 2024-09-14T09:06:54.609 INFO:teuthology.orchestra.run.smithi176.stdout:4220 read 0x4d25d47 thru 0x4d2e64d (0x8907 bytes) 2024-09-14T09:06:54.609 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:54.609 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:54.609 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:54.610 INFO:teuthology.orchestra.run.smithi176.stdout:4225 punch from 0x12b876 to 0x134723, (0x8ead bytes) 2024-09-14T09:06:54.610 INFO:teuthology.orchestra.run.smithi176.stdout:4227 write 0xb516e1f thru 0xb51a26f (0x3451 bytes) 2024-09-14T09:06:54.613 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:54.615+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:54.616 INFO:teuthology.orchestra.run.smithi176.stdout:4228 read 0xc2bc84 thru 0xc30908 (0x4c85 bytes) 2024-09-14T09:06:54.628 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:54.629 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:54.629 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:54.629 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:54.629 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:54.631 INFO:teuthology.orchestra.run.smithi176.stdout:4229 write 0xd093b82 thru 0xd0a1a0c (0xde8b bytes) 2024-09-14T09:06:54.641 INFO:teuthology.orchestra.run.smithi176.stdout:4230 clone 30 order 19 su 2048 sc 11 2024-09-14T09:06:54.645 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:06:54.715 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:54.719+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:54.723 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:06:54.812 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:54.812 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:54.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:54.813 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:54.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.813 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:54.813 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:54.813 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 81.727854496999996, 2024-09-14T09:06:54.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:06:54.813 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.813 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.814 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.814 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.814 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.815 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.816 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:06:54.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.816 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:54.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:54.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:06:54.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:06:54.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:06:54.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:06:54.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:06:54.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:06:54.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:06:54.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:06:54.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:06:54.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:06:54.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:54.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:06:54.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:06:54.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:06:54.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:06:54.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:06:54.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:06:54.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:06:54.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:06:54.820 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.821 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.821 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.821 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:54.821 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:54.821 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 81.727799838999999, 2024-09-14T09:06:54.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:06:54.821 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.821 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.821 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.821 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.821 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.821 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.822 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:06:54.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.822 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:54.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:54.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:54.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:06:54.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:06:54.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:06:54.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:06:54.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:06:54.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:06:54.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:06:54.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:06:54.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:06:54.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:54.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:06:54.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:06:54.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:06:54.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:06:54.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:06:54.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:06:54.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:06:54.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:06:54.826 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.826 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.826 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.826 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:54.826 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:54.826 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 81.727694464999999, 2024-09-14T09:06:54.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:06:54.827 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.827 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.827 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.827 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.827 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.827 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.827 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:06:54.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.827 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:54.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:54.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:06:54.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:06:54.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:06:54.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:06:54.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:06:54.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:06:54.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:06:54.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:06:54.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:06:54.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:06:54.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:54.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:06:54.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:06:54.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:06:54.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:06:54.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:06:54.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:06:54.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:06:54.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:06:54.831 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.832 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.832 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.832 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:54.832 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:54.832 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 81.727337700999996, 2024-09-14T09:06:54.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:06:54.832 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.832 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.832 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.832 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.832 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.832 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.832 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:06:54.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.833 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:54.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:54.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:06:54.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:06:54.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:06:54.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:06:54.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:06:54.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:06:54.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:06:54.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:06:54.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:06:54.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:06:54.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:54.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:06:54.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:06:54.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:06:54.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:06:54.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:06:54.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:06:54.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:06:54.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:06:54.837 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.837 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.837 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.837 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:54.837 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:54.837 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 81.727252962999998, 2024-09-14T09:06:54.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:06:54.837 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.837 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.838 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.838 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.838 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.838 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.838 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:06:54.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.838 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:54.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:54.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:06:54.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:06:54.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:06:54.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:06:54.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:06:54.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:06:54.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:06:54.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:06:54.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:06:54.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:06:54.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:54.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:06:54.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:06:54.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:06:54.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:06:54.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:06:54.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:06:54.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:06:54.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:06:54.842 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.842 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.842 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.842 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:54.843 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:54.843 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 81.727165276999997, 2024-09-14T09:06:54.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:06:54.843 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.843 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.843 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.843 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.843 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.843 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.843 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:06:54.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.843 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:54.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:54.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:06:54.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:06:54.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:06:54.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:06:54.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:06:54.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:06:54.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:06:54.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:06:54.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:06:54.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:06:54.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:54.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:06:54.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:06:54.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:06:54.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:06:54.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:06:54.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:06:54.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:06:54.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:06:54.847 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.847 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.847 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.848 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18388 3.18 3:19e2be55:::rbd_data.115e3abd3b2c.0000000000000021:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:06:54.848 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:54.848 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.591251966000002, 2024-09-14T09:06:54.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042293648000000003, 2024-09-14T09:06:54.848 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.848 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.848 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:54.848 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.848 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.848 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.848 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18388 2024-09-14T09:06:54.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.848 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:54.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:54.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220236+0000", 2024-09-14T09:06:54.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1339999999999998e-06 2024-09-14T09:06:54.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220244+0000", 2024-09-14T09:06:54.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1950000000000003e-06 2024-09-14T09:06:54.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220262+0000", 2024-09-14T09:06:54.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8017e-05 2024-09-14T09:06:54.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220270+0000", 2024-09-14T09:06:54.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9480000000000008e-06 2024-09-14T09:06:54.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220357+0000", 2024-09-14T09:06:54.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7185e-05 2024-09-14T09:06:54.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220576+0000", 2024-09-14T09:06:54.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021912200000000001 2024-09-14T09:06:54.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:54.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.222435+0000", 2024-09-14T09:06:54.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018592350000000001 2024-09-14T09:06:54.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:54.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262439+0000", 2024-09-14T09:06:54.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040003368999999997 2024-09-14T09:06:54.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:54.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262452+0000", 2024-09-14T09:06:54.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2941e-05 2024-09-14T09:06:54.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:54.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262490+0000", 2024-09-14T09:06:54.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8478e-05 2024-09-14T09:06:54.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262527+0000", 2024-09-14T09:06:54.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7024000000000003e-05 2024-09-14T09:06:54.853 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.853 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.853 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.854 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:19543 3.5 e39/23 3:a536ff9b:::rbd_data.115e4c71d6e8.0000000000000002:head v 39'8, mlcod=39'6)", 2024-09-14T09:06:54.854 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:54.854 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 70.288397305999993, 2024-09-14T09:06:54.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.039280019999999999, 2024-09-14T09:06:54.854 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.854 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.854 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:54.854 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:54.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523088+0000", 2024-09-14T09:06:54.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.523090+0000", 2024-09-14T09:06:54.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.773e-06 2024-09-14T09:06:54.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537290+0000", 2024-09-14T09:06:54.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.014200007000000001 2024-09-14T09:06:54.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537317+0000", 2024-09-14T09:06:54.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6704999999999999e-05 2024-09-14T09:06:54.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537328+0000", 2024-09-14T09:06:54.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1299e-05 2024-09-14T09:06:54.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537417+0000", 2024-09-14T09:06:54.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8869999999999994e-05 2024-09-14T09:06:54.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.537445+0000", 2024-09-14T09:06:54.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8377999999999998e-05 2024-09-14T09:06:54.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:54.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562324+0000", 2024-09-14T09:06:54.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024879307999999999 2024-09-14T09:06:54.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.562368+0000", 2024-09-14T09:06:54.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3680000000000002e-05 2024-09-14T09:06:54.858 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.858 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.858 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.858 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38002 3.1e 3:7df715aa:::rbd_data.115eb07a8abd.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:54.858 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:54.858 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.421734059999999, 2024-09-14T09:06:54.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.065148304000000004, 2024-09-14T09:06:54.858 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.859 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.859 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:54.859 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.859 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.859 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.859 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38002 2024-09-14T09:06:54.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.859 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:54.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:54.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389752+0000", 2024-09-14T09:06:54.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1829999999999999e-06 2024-09-14T09:06:54.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389758+0000", 2024-09-14T09:06:54.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0919999999999997e-06 2024-09-14T09:06:54.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389769+0000", 2024-09-14T09:06:54.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759e-05 2024-09-14T09:06:54.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389775+0000", 2024-09-14T09:06:54.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8819999999999998e-06 2024-09-14T09:06:54.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389824+0000", 2024-09-14T09:06:54.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8573999999999999e-05 2024-09-14T09:06:54.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389876+0000", 2024-09-14T09:06:54.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2643999999999999e-05 2024-09-14T09:06:54.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:06:54.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.392640+0000", 2024-09-14T09:06:54.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0027636409999999998 2024-09-14T09:06:54.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:54.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454672+0000", 2024-09-14T09:06:54.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062031703000000001 2024-09-14T09:06:54.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:54.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454765+0000", 2024-09-14T09:06:54.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2862000000000006e-05 2024-09-14T09:06:54.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:54.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454841+0000", 2024-09-14T09:06:54.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5857000000000002e-05 2024-09-14T09:06:54.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454900+0000", 2024-09-14T09:06:54.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9107000000000002e-05 2024-09-14T09:06:54.864 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.864 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.864 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.864 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:38002 3.1e e48/23 ondisk, result = 0)", 2024-09-14T09:06:54.864 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:54.864 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.411417471, 2024-09-14T09:06:54.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054841741999999999, 2024-09-14T09:06:54.865 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.865 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.865 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.865 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:54.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:54.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400069+0000", 2024-09-14T09:06:54.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.339e-06 2024-09-14T09:06:54.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400076+0000", 2024-09-14T09:06:54.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7800000000000003e-06 2024-09-14T09:06:54.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400093+0000", 2024-09-14T09:06:54.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6915999999999999e-05 2024-09-14T09:06:54.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400106+0000", 2024-09-14T09:06:54.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2785e-05 2024-09-14T09:06:54.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454713+0000", 2024-09-14T09:06:54.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054607041000000002 2024-09-14T09:06:54.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454724+0000", 2024-09-14T09:06:54.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1202e-05 2024-09-14T09:06:54.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454910+0000", 2024-09-14T09:06:54.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018567900000000001 2024-09-14T09:06:54.868 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.868 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.868 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.868 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38100 3.b 3:d0f67c24:::rbd_data.115eb07a8abd.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:54.868 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:54.868 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.165563810999998, 2024-09-14T09:06:54.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053701964999999997, 2024-09-14T09:06:54.869 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.869 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.869 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:54.869 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.869 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.869 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.869 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38100 2024-09-14T09:06:54.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.869 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:54.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:54.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645923+0000", 2024-09-14T09:06:54.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:06:54.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645928+0000", 2024-09-14T09:06:54.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0699999999999997e-06 2024-09-14T09:06:54.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645942+0000", 2024-09-14T09:06:54.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4069e-05 2024-09-14T09:06:54.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645951+0000", 2024-09-14T09:06:54.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3230000000000004e-06 2024-09-14T09:06:54.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646008+0000", 2024-09-14T09:06:54.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6583999999999999e-05 2024-09-14T09:06:54.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646089+0000", 2024-09-14T09:06:54.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1557999999999994e-05 2024-09-14T09:06:54.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:54.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.650355+0000", 2024-09-14T09:06:54.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004265525 2024-09-14T09:06:54.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:54.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699439+0000", 2024-09-14T09:06:54.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049083904999999997 2024-09-14T09:06:54.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:54.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699467+0000", 2024-09-14T09:06:54.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8198999999999999e-05 2024-09-14T09:06:54.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:54.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699552+0000", 2024-09-14T09:06:54.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T09:06:54.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699624+0000", 2024-09-14T09:06:54.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1725999999999997e-05 2024-09-14T09:06:54.874 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.874 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.874 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.874 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:06:54.874 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:54.874 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.65087329, 2024-09-14T09:06:54.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:06:54.874 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.874 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.874 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:54.874 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.874 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.875 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:06:54.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.875 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:54.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:54.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:06:54.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:06:54.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:06:54.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:06:54.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:06:54.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:06:54.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:06:54.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:06:54.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:06:54.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:06:54.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:06:54.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:06:54.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:06:54.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:06:54.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:06:54.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:54.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:06:54.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:06:54.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:54.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:06:54.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:06:54.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:54.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:06:54.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:06:54.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:06:54.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:06:54.879 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.880 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.880 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.880 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:39401 3.b e56/53)", 2024-09-14T09:06:54.880 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:54.880 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.624977223999998, 2024-09-14T09:06:54.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10443659600000001, 2024-09-14T09:06:54.880 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.880 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.880 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.880 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:54.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:54.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186509+0000", 2024-09-14T09:06:54.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2640000000000001e-06 2024-09-14T09:06:54.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186516+0000", 2024-09-14T09:06:54.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1070000000000001e-06 2024-09-14T09:06:54.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186533+0000", 2024-09-14T09:06:54.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7818000000000001e-05 2024-09-14T09:06:54.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186543+0000", 2024-09-14T09:06:54.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5170000000000005e-06 2024-09-14T09:06:54.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290885+0000", 2024-09-14T09:06:54.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.104342221 2024-09-14T09:06:54.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290893+0000", 2024-09-14T09:06:54.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2859999999999999e-06 2024-09-14T09:06:54.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290945+0000", 2024-09-14T09:06:54.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1382999999999999e-05 2024-09-14T09:06:54.883 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.884 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.884 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.884 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49710 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:54.884 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526924+0000", 2024-09-14T09:06:54.884 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 44.284561062000002, 2024-09-14T09:06:54.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037006095000000003, 2024-09-14T09:06:54.884 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.884 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.884 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.884 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.884 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.884 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.884 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49710 2024-09-14T09:06:54.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.885 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526924+0000", 2024-09-14T09:06:54.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526924+0000", 2024-09-14T09:06:54.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526925+0000", 2024-09-14T09:06:54.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3599999999999999e-07 2024-09-14T09:06:54.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526926+0000", 2024-09-14T09:06:54.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.795e-06 2024-09-14T09:06:54.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526932+0000", 2024-09-14T09:06:54.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0839999999999999e-06 2024-09-14T09:06:54.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526936+0000", 2024-09-14T09:06:54.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3440000000000003e-06 2024-09-14T09:06:54.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563593+0000", 2024-09-14T09:06:54.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036657041000000001 2024-09-14T09:06:54.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563648+0000", 2024-09-14T09:06:54.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5152999999999999e-05 2024-09-14T09:06:54.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563930+0000", 2024-09-14T09:06:54.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028234199999999999 2024-09-14T09:06:54.888 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.888 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.888 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.888 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49711 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:54.888 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:54.888 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 44.284543542999998, 2024-09-14T09:06:54.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037330945999999997, 2024-09-14T09:06:54.888 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.888 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.888 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.888 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.888 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.888 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.889 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49711 2024-09-14T09:06:54.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.889 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:54.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:54.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526942+0000", 2024-09-14T09:06:54.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1800000000000002e-07 2024-09-14T09:06:54.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526944+0000", 2024-09-14T09:06:54.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.821e-06 2024-09-14T09:06:54.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526950+0000", 2024-09-14T09:06:54.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5600000000000001e-06 2024-09-14T09:06:54.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526954+0000", 2024-09-14T09:06:54.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.437e-06 2024-09-14T09:06:54.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.563965+0000", 2024-09-14T09:06:54.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037010534999999997 2024-09-14T09:06:54.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564021+0000", 2024-09-14T09:06:54.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6728000000000003e-05 2024-09-14T09:06:54.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564273+0000", 2024-09-14T09:06:54.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00025154700000000002 2024-09-14T09:06:54.892 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.892 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.892 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.892 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49712 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 3112960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:54.892 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:54.892 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 44.284525334999998, 2024-09-14T09:06:54.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037795625999999999, 2024-09-14T09:06:54.893 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.893 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.893 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.893 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.893 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.893 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.893 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49712 2024-09-14T09:06:54.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.893 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:54.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:54.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526960+0000", 2024-09-14T09:06:54.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4799999999999999e-07 2024-09-14T09:06:54.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526962+0000", 2024-09-14T09:06:54.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8449999999999999e-06 2024-09-14T09:06:54.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526967+0000", 2024-09-14T09:06:54.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1059999999999999e-06 2024-09-14T09:06:54.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526971+0000", 2024-09-14T09:06:54.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8310000000000003e-06 2024-09-14T09:06:54.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564324+0000", 2024-09-14T09:06:54.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037352926000000002 2024-09-14T09:06:54.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564411+0000", 2024-09-14T09:06:54.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7052000000000006e-05 2024-09-14T09:06:54.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564756+0000", 2024-09-14T09:06:54.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034451799999999998 2024-09-14T09:06:54.896 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.896 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.896 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.897 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49713 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 3506176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:54.897 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:54.897 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 44.284508662999997, 2024-09-14T09:06:54.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038135466, 2024-09-14T09:06:54.897 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.897 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.897 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.897 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.897 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.897 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.897 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49713 2024-09-14T09:06:54.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.897 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:54.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:54.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526977+0000", 2024-09-14T09:06:54.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1899999999999998e-07 2024-09-14T09:06:54.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526979+0000", 2024-09-14T09:06:54.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8959999999999999e-06 2024-09-14T09:06:54.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526986+0000", 2024-09-14T09:06:54.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1720000000000001e-06 2024-09-14T09:06:54.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526991+0000", 2024-09-14T09:06:54.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.729e-06 2024-09-14T09:06:54.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564781+0000", 2024-09-14T09:06:54.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037790345000000003 2024-09-14T09:06:54.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.564844+0000", 2024-09-14T09:06:54.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2656000000000005e-05 2024-09-14T09:06:54.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565112+0000", 2024-09-14T09:06:54.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026834899999999997 2024-09-14T09:06:54.901 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.901 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.901 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.901 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49714 3.8 3:13d730f4:::rbd_data.115e51d92ed2.0000000000000000:12 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:54.901 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.526997+0000", 2024-09-14T09:06:54.901 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 44.284488054000001, 2024-09-14T09:06:54.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038531469999999998, 2024-09-14T09:06:54.901 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.901 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.902 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.902 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.902 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.902 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.902 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49714 2024-09-14T09:06:54.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.902 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526997+0000", 2024-09-14T09:06:54.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526997+0000", 2024-09-14T09:06:54.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526998+0000", 2024-09-14T09:06:54.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7800000000000002e-07 2024-09-14T09:06:54.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.527000+0000", 2024-09-14T09:06:54.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9080000000000002e-06 2024-09-14T09:06:54.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.527007+0000", 2024-09-14T09:06:54.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1609999999999997e-06 2024-09-14T09:06:54.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.527011+0000", 2024-09-14T09:06:54.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0929999999999996e-06 2024-09-14T09:06:54.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565152+0000", 2024-09-14T09:06:54.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038141520999999998 2024-09-14T09:06:54.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565236+0000", 2024-09-14T09:06:54.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3498999999999995e-05 2024-09-14T09:06:54.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.565529+0000", 2024-09-14T09:06:54.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029291000000000001 2024-09-14T09:06:54.905 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.905 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.906 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.906 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:84137 3.7 3:e15b49d9:::rbd_data.115e77b799c3.00000000000000e0:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:06:54.906 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:06:54.906 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.488125892, 2024-09-14T09:06:54.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11205707500000001, 2024-09-14T09:06:54.906 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.906 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.906 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:54.906 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:54.906 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:54.906 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:54.906 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 84137 2024-09-14T09:06:54.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.907 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:06:54.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:06:54.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323362+0000", 2024-09-14T09:06:54.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0839999999999999e-06 2024-09-14T09:06:54.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323370+0000", 2024-09-14T09:06:54.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0180000000000001e-06 2024-09-14T09:06:54.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323388+0000", 2024-09-14T09:06:54.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8326999999999999e-05 2024-09-14T09:06:54.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323397+0000", 2024-09-14T09:06:54.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1940000000000004e-06 2024-09-14T09:06:54.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323452+0000", 2024-09-14T09:06:54.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5043000000000001e-05 2024-09-14T09:06:54.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323550+0000", 2024-09-14T09:06:54.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8201000000000004e-05 2024-09-14T09:06:54.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:54.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.328273+0000", 2024-09-14T09:06:54.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0047227079999999999 2024-09-14T09:06:54.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:54.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435232+0000", 2024-09-14T09:06:54.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.106959339 2024-09-14T09:06:54.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:54.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435256+0000", 2024-09-14T09:06:54.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3774e-05 2024-09-14T09:06:54.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:54.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435314+0000", 2024-09-14T09:06:54.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7636999999999997e-05 2024-09-14T09:06:54.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435417+0000", 2024-09-14T09:06:54.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010275 2024-09-14T09:06:54.912 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.912 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.912 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.912 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:84137 3.7 e92/23 ondisk, result = 0)", 2024-09-14T09:06:54.912 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:53.341180+0000", 2024-09-14T09:06:54.912 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.4703053079999999, 2024-09-14T09:06:54.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.094062506000000004, 2024-09-14T09:06:54.913 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:54.913 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:54.913 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:54.913 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:54.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:54.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341180+0000", 2024-09-14T09:06:54.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:54.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341180+0000", 2024-09-14T09:06:54.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:54.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:54.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341181+0000", 2024-09-14T09:06:54.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.192e-06 2024-09-14T09:06:54.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:54.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341187+0000", 2024-09-14T09:06:54.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0070000000000003e-06 2024-09-14T09:06:54.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:54.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341208+0000", 2024-09-14T09:06:54.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0321000000000001e-05 2024-09-14T09:06:54.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:54.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341219+0000", 2024-09-14T09:06:54.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1764e-05 2024-09-14T09:06:54.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:54.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435214+0000", 2024-09-14T09:06:54.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.093994438999999999 2024-09-14T09:06:54.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:54.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435225+0000", 2024-09-14T09:06:54.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1123e-05 2024-09-14T09:06:54.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:54.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:54.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:54.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435243+0000", 2024-09-14T09:06:54.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.766e-05 2024-09-14T09:06:54.916 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.916 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.916 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.916 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:54.916 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:54.916 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:54.918 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:06:54.918 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:54.919 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:54.919 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:54.920 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:54.920 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:54.920 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:54.920 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:54.915+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:54.923 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:54.924 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:54.931 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is unset 2024-09-14T09:06:54.945 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:54.945 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:54.964 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:54.965 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:54.982 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:54.983 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:55.003 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:55.004 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:55.019 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:55.023+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:55.028 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone29 from 0xd0a1a0d (overlap 0xd4a896) to 0x0 2024-09-14T09:06:55.073 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:55.073 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:55.096 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:06:55.120 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:55.119+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:55.196 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-14T09:06:55.222 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:55.223+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:55.255 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:55.256 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:55.270 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:06:55.359 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:55.363+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:55.423 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:55.427+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:55.436 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:55.436 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:55.450 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:06:55.497 INFO:teuthology.orchestra.run.smithi176.stdout:4234 punch from 0x9973f62 to 0x997855e, (0x45fc bytes) 2024-09-14T09:06:55.505 INFO:teuthology.orchestra.run.smithi176.stdout:4235 read 0xa3a9a01 thru 0xa3b0622 (0x6c22 bytes) 2024-09-14T09:06:55.524 INFO:teuthology.orchestra.run.smithi176.stdout:4236 trunc from 0xd0a1a0d to 0x8a62c7 2024-09-14T09:06:55.524 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:55.523+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:55.625 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:55.627+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:55.665 INFO:teuthology.orchestra.run.smithi176.stdout:4239 write 0x17ab40a thru 0x17b3c0d (0x8804 bytes) 2024-09-14T09:06:55.668 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:55.668 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:55.682 INFO:teuthology.orchestra.run.smithi176.stdout:4240 punch from 0xbd07c8 to 0xbd6d66, (0x659e bytes) 2024-09-14T09:06:55.685 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:06:55.703 INFO:teuthology.orchestra.run.smithi176.stdout:4242 writesame 0x8da0dbb thru 0x8da6fe5 data_size 0x1(0x622b bytes) 2024-09-14T09:06:55.713 INFO:teuthology.orchestra.run.smithi176.stdout:4243 write 0x9c8a170 thru 0x9c93e91 (0x9d22 bytes) 2024-09-14T09:06:55.726 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:55.727+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:55.737 INFO:teuthology.orchestra.run.smithi176.stdout:4244 trunc from 0x9c93e92 to 0xe7968c3 2024-09-14T09:06:55.746 INFO:teuthology.orchestra.run.smithi176.stdout:4245 read 0xaa10da0 thru 0xaa18676 (0x78d7 bytes) 2024-09-14T09:06:55.747 INFO:teuthology.orchestra.run.smithi176.stdout:4246 trunc from 0xe7968c3 to 0xebdb07d 2024-09-14T09:06:55.755 INFO:teuthology.orchestra.run.smithi176.stdout:4247 write 0xb9073cd thru 0xb910a47 (0x967b bytes) 2024-09-14T09:06:55.774 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:55.774 INFO:teuthology.orchestra.run.smithi176.stdout:4253 punch from 0x19b52c2 to 0x19c0a84, (0xb7c2 bytes) 2024-09-14T09:06:55.774 INFO:teuthology.orchestra.run.smithi176.stdout:4255 write 0xd2a2d83 thru 0xd2af6f1 (0xc96f bytes) 2024-09-14T09:06:55.788 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:55.788 INFO:teuthology.orchestra.run.smithi176.stdout:4258 write 0x547f265 thru 0x548675f (0x74fb bytes) 2024-09-14T09:06:55.825 INFO:teuthology.orchestra.run.smithi176.stdout:4259 trunc from 0xebdb07d to 0x3a005dd 2024-09-14T09:06:55.827 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:55.827+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:55.852 INFO:teuthology.orchestra.run.smithi176.stdout:4260 read 0xf35b7f thru 0xf424dc (0xc95e bytes) 2024-09-14T09:06:55.857 INFO:teuthology.orchestra.run.smithi176.stdout:4262 trunc from 0x3a005dd to 0x6504c74 2024-09-14T09:06:55.858 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:55.858 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:55.866 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:55.873 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:06:55.878 INFO:teuthology.orchestra.run.smithi176.stdout:4265 writesame 0x844e56f thru 0x845a282 data_size 0x2(0xbd14 bytes) 2024-09-14T09:06:55.878 INFO:teuthology.orchestra.run.smithi176.stdout:4266 write 0x70df3f thru 0x717fdf (0xa0a1 bytes) 2024-09-14T09:06:55.887 INFO:teuthology.orchestra.run.smithi176.stdout:4267 read 0x5181281 thru 0x51908f7 (0xf677 bytes) 2024-09-14T09:06:55.893 INFO:teuthology.orchestra.run.smithi176.stdout:4268 trunc from 0x845a283 to 0x59240cc 2024-09-14T09:06:55.905 INFO:teuthology.orchestra.run.smithi176.stdout:4270 punch from 0xe542e to 0xedf44, (0x8b16 bytes) 2024-09-14T09:06:55.905 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:55.905 INFO:teuthology.orchestra.run.smithi176.stdout:4274 read 0x4a979ca thru 0x4a99443 (0x1a7a bytes) 2024-09-14T09:06:55.909 INFO:teuthology.orchestra.run.smithi176.stdout:4275 trunc from 0x59240cc to 0x7d4997a 2024-09-14T09:06:55.918 INFO:teuthology.orchestra.run.smithi176.stdout:4276 read 0x3bcd9e thru 0x3ccbd5 (0xfe38 bytes) 2024-09-14T09:06:55.928 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:55.931+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:55.942 INFO:teuthology.orchestra.run.smithi176.stdout:4277 trunc from 0x7d4997a to 0xd656712 2024-09-14T09:06:55.968 INFO:teuthology.orchestra.run.smithi176.stdout:4279 write 0xe763d44 thru 0xe77237c (0xe639 bytes) 2024-09-14T09:06:55.986 INFO:teuthology.orchestra.run.smithi176.stdout:4281 read 0x12eb07d thru 0x12f705e (0xbfe2 bytes) 2024-09-14T09:06:55.986 INFO:teuthology.orchestra.run.smithi176.stdout:4283 punch from 0x77902d8 to 0x7798031, (0x7d59 bytes) 2024-09-14T09:06:55.986 INFO:teuthology.orchestra.run.smithi176.stdout:4284 read 0x343187 thru 0x3460c3 (0x2f3d bytes) 2024-09-14T09:06:55.990 INFO:teuthology.orchestra.run.smithi176.stdout:4285 read 0x8a5488c thru 0x8a5ca5b (0x81d0 bytes) 2024-09-14T09:06:55.990 INFO:teuthology.orchestra.run.smithi176.stdout:4286 read 0x39119ef thru 0x3912778 (0xd8a bytes) 2024-09-14T09:06:55.990 INFO:teuthology.orchestra.run.smithi176.stdout:4287 read 0xfe0706 thru 0xfed6c8 (0xcfc3 bytes) 2024-09-14T09:06:55.990 INFO:teuthology.orchestra.run.smithi176.stdout:4288 read 0x97d785d thru 0x97e0156 (0x88fa bytes) 2024-09-14T09:06:55.991 INFO:teuthology.orchestra.run.smithi176.stdout:4289 punch from 0x25d6119 to 0x25dca32, (0x6919 bytes) 2024-09-14T09:06:55.991 INFO:teuthology.orchestra.run.smithi176.stdout:4290 write 0x8614b0f thru 0x861d0f6 (0x85e8 bytes) 2024-09-14T09:06:56.010 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:56.010 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:56.024 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:06:56.029 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:56.027+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:56.040 INFO:teuthology.orchestra.run.smithi176.stdout:4291 trunc from 0xe77237d to 0x1a145d9 2024-09-14T09:06:56.073 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:56.092 INFO:teuthology.orchestra.run.smithi176.stdout:4294 trunc from 0x1a145d9 to 0xb17cb57 2024-09-14T09:06:56.104 INFO:teuthology.orchestra.run.smithi176.stdout:4295 writesame 0x39b5387 thru 0x39c4787 data_size 0x5(0xf401 bytes) 2024-09-14T09:06:56.128 INFO:teuthology.orchestra.run.smithi176.stdout:4297 writesame 0x99d9ae thru 0x9a5e73 data_size 0x37(0x84c6 bytes) 2024-09-14T09:06:56.130 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:56.131+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:56.139 INFO:teuthology.orchestra.run.smithi176.stdout:4298 writesame 0x199dfe5 thru 0x19a851c data_size 0x44(0xa538 bytes) 2024-09-14T09:06:56.146 INFO:teuthology.orchestra.run.smithi176.stdout:4300 read 0x1993b55 thru 0x199c8d2 (0x8d7e bytes) 2024-09-14T09:06:56.147 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:56.147 INFO:teuthology.orchestra.run.smithi176.stdout:4303 read 0x78c3473 thru 0x78d219b (0xed29 bytes) 2024-09-14T09:06:56.163 INFO:teuthology.orchestra.run.smithi176.stdout:4304 write 0xd6ecd35 thru 0xd6f4068 (0x7334 bytes) 2024-09-14T09:06:56.181 INFO:teuthology.orchestra.run.smithi176.stdout:4306 write 0x65e9e52 thru 0x65eb7f6 (0x19a5 bytes) 2024-09-14T09:06:56.191 INFO:teuthology.orchestra.run.smithi176.stdout:4307 read 0x544703 thru 0x547ede (0x37dc bytes) 2024-09-14T09:06:56.194 INFO:teuthology.orchestra.run.smithi176.stdout:4308 read 0x1cdd00a thru 0x1cebf8b (0xef82 bytes) 2024-09-14T09:06:56.194 INFO:teuthology.orchestra.run.smithi176.stdout:4309 write 0xa982355 thru 0xa98f20a (0xceb6 bytes) 2024-09-14T09:06:56.213 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:56.213 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:56.216 INFO:teuthology.orchestra.run.smithi176.stdout:4310 punch from 0x134bfd2 to 0x135b69b, (0xf6c9 bytes) 2024-09-14T09:06:56.228 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:06:56.233 INFO:teuthology.orchestra.run.smithi176.stdout:4312 writesame 0xe577e1f thru 0xe5797d5 data_size 0x1(0x19b7 bytes) 2024-09-14T09:06:56.233 INFO:teuthology.orchestra.run.smithi176.stdout:4315 write 0xb0eed35 thru 0xb0fb887 (0xcb53 bytes) 2024-09-14T09:06:56.259 INFO:teuthology.orchestra.run.smithi176.stdout:4317 writesame 0xcd2a947 thru 0xcd37770 data_size 0xb(0xce2a bytes) 2024-09-14T09:06:56.271 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:56.275+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:56.282 INFO:teuthology.orchestra.run.smithi176.stdout:4318 punch from 0xa7118f8 to 0xa712264, (0x96c bytes) 2024-09-14T09:06:56.282 INFO:teuthology.orchestra.run.smithi176.stdout:4319 write 0x1c05459 thru 0x1c10477 (0xb01f bytes) 2024-09-14T09:06:56.332 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:56.335+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:56.332 INFO:teuthology.orchestra.run.smithi176.stdout:4320 trunc from 0xe5797d6 to 0x1efaea1 2024-09-14T09:06:56.406 INFO:teuthology.orchestra.run.smithi176.stdout:4322 trunc from 0x1efaea1 to 0xcb0ac23 2024-09-14T09:06:56.406 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:56.406 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:56.418 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:56.418 INFO:teuthology.orchestra.run.smithi176.stdout:4324 punch from 0xac00473 to 0xac0823b, (0x7dc8 bytes) 2024-09-14T09:06:56.418 INFO:teuthology.orchestra.run.smithi176.stdout:4326 punch from 0xe0efb0 to 0xe1c3c6, (0xd416 bytes) 2024-09-14T09:06:56.420 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:06:56.434 INFO:teuthology.orchestra.run.smithi176.stdout:4329 trunc from 0xcb0ac23 to 0x66a49ac 2024-09-14T09:06:56.448 INFO:teuthology.orchestra.run.smithi176.stdout:4331 punch from 0x63150a2 to 0x631eae1, (0x9a3f bytes) 2024-09-14T09:06:56.512 INFO:teuthology.orchestra.run.smithi176.stdout:4334 write 0x618441d thru 0x61937cb (0xf3af bytes) 2024-09-14T09:06:56.513 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:56.467+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:56.514 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:56.514 INFO:teuthology.orchestra.run.smithi176.stdout:4339 trunc from 0x66a49ac to 0x3a395e2 2024-09-14T09:06:56.514 INFO:teuthology.orchestra.run.smithi176.stdout:4341 read 0x15f2f57 thru 0x15fddeb (0xae95 bytes) 2024-09-14T09:06:56.514 INFO:teuthology.orchestra.run.smithi176.stdout:4343 write 0x14caf7a thru 0x14d3edc (0x8f63 bytes) 2024-09-14T09:06:56.526 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:56.526 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:56.526 INFO:teuthology.orchestra.run.smithi176.stdout:4347 punch from 0x237a607 to 0x237fc71, (0x566a bytes) 2024-09-14T09:06:56.526 INFO:teuthology.orchestra.run.smithi176.stdout:4349 read 0x2cb7d2a thru 0x2cb944c (0x1723 bytes) 2024-09-14T09:06:56.526 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:56.526 INFO:teuthology.orchestra.run.smithi176.stdout:4352 write 0xc40902 thru 0xc4813a (0x7839 bytes) 2024-09-14T09:06:56.563 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:56.567+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:56.571 INFO:teuthology.orchestra.run.smithi176.stdout:4353 writesame 0x630fc53 thru 0x63152f9 data_size 0x1(0x56a7 bytes) 2024-09-14T09:06:56.578 INFO:teuthology.orchestra.run.smithi176.stdout:4354 trunc from 0x63152fa to 0x3ab5609 2024-09-14T09:06:56.623 INFO:teuthology.orchestra.run.smithi176.stdout:4356 trunc from 0x3ab5609 to 0xe883ee7 2024-09-14T09:06:56.633 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:56.635+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:56.637 INFO:teuthology.orchestra.run.smithi176.stdout:4357 clone 31 order 23 su 131072 sc 15 2024-09-14T09:06:56.639 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:56.639 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:56.653 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:06:56.733 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:56.731+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:56.821 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:56.821 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:56.834 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:06:56.835 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:56.835+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:56.936 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:56.935+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:56.953 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T09:06:57.001 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:57.001 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:57.015 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:06:57.022 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:06:57.059 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:57.063+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:57.069 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone30 from 0xe883ee7 (overlap 0x8a62c7) to 0x0 2024-09-14T09:06:57.136 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:57.135+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:57.180 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:57.180 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:57.201 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:06:57.229 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:57.229 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:57.237 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:06:57.235+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:06:57.253 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:57.253 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:57.270 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:57.270 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:57.287 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:57.287 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:57.305 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:57.306 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:57.324 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:57.324 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:57.337 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:57.335+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:57.342 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:57.342 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:57.361 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:57.361 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:57.365 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:57.365 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:57.365 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:57.365 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:57.381 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:06:57.466 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:57.467+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:57.540 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:57.543+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:57.556 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:57.556 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:57.556 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:57.556 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:57.556 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:57.572 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:06:57.577 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-14T09:06:57.579 INFO:teuthology.orchestra.run.smithi176.stdout:4358 write 0xb01dd4d thru 0xb021de7 (0x409b bytes) 2024-09-14T09:06:57.601 INFO:teuthology.orchestra.run.smithi176.stdout:4361 write 0x5a7304a thru 0x5a7d2a1 (0xa258 bytes) 2024-09-14T09:06:57.640 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:57.643+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:57.652 INFO:teuthology.orchestra.run.smithi176.stdout:4362 trunc from 0xe883ee7 to 0xa271031 2024-09-14T09:06:57.719 INFO:teuthology.orchestra.run.smithi176.stdout:4363 trunc from 0xa271031 to 0xe873205 2024-09-14T09:06:57.727 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20858 3.17 3:ed5052d1:::rbd_data.115e889d4809.000000000000015c:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:06:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 70.297061829, 2024-09-14T09:06:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053381794000000003, 2024-09-14T09:06:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:57.727 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20858 2024-09-14T09:06:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:06:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:06:57.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429351+0000", 2024-09-14T09:06:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1149999999999998e-06 2024-09-14T09:06:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429363+0000", 2024-09-14T09:06:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1819e-05 2024-09-14T09:06:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429389+0000", 2024-09-14T09:06:57.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5605e-05 2024-09-14T09:06:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429402+0000", 2024-09-14T09:06:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3175e-05 2024-09-14T09:06:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429492+0000", 2024-09-14T09:06:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.988e-05 2024-09-14T09:06:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429597+0000", 2024-09-14T09:06:57.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010564 2024-09-14T09:06:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:06:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.433465+0000", 2024-09-14T09:06:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0038674249999999999 2024-09-14T09:06:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482483+0000", 2024-09-14T09:06:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049017999999999999 2024-09-14T09:06:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:57.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482625+0000", 2024-09-14T09:06:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000142447 2024-09-14T09:06:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482684+0000", 2024-09-14T09:06:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8522e-05 2024-09-14T09:06:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482730+0000", 2024-09-14T09:06:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6165999999999998e-05 2024-09-14T09:06:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.732 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:20858 3.17 e41/23 ondisk, result = 0)", 2024-09-14T09:06:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:06:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 70.282302385999998, 2024-09-14T09:06:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038631402000000002, 2024-09-14T09:06:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:06:57.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:06:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444109+0000", 2024-09-14T09:06:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.331e-06 2024-09-14T09:06:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444114+0000", 2024-09-14T09:06:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3709999999999999e-06 2024-09-14T09:06:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444130+0000", 2024-09-14T09:06:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6079e-05 2024-09-14T09:06:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444144+0000", 2024-09-14T09:06:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3331e-05 2024-09-14T09:06:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482538+0000", 2024-09-14T09:06:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038394878 2024-09-14T09:06:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482548+0000", 2024-09-14T09:06:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0093e-05 2024-09-14T09:06:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482739+0000", 2024-09-14T09:06:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019031899999999999 2024-09-14T09:06:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.736 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:37800 3.1 3:830c4670:::rbd_data.115eb07a8abd.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:06:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.695854419, 2024-09-14T09:06:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.068040724999999996, 2024-09-14T09:06:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:57.737 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 37800 2024-09-14T09:06:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:06:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:06:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030557+0000", 2024-09-14T09:06:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.888e-06 2024-09-14T09:06:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030566+0000", 2024-09-14T09:06:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8540000000000003e-06 2024-09-14T09:06:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030585+0000", 2024-09-14T09:06:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8878000000000001e-05 2024-09-14T09:06:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030593+0000", 2024-09-14T09:06:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5870000000000004e-06 2024-09-14T09:06:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030667+0000", 2024-09-14T09:06:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4207000000000003e-05 2024-09-14T09:06:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030761+0000", 2024-09-14T09:06:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4223999999999998e-05 2024-09-14T09:06:57.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:06:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.034400+0000", 2024-09-14T09:06:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0036387759999999998 2024-09-14T09:06:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098457+0000", 2024-09-14T09:06:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.064057501000000003 2024-09-14T09:06:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098518+0000", 2024-09-14T09:06:57.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0730999999999998e-05 2024-09-14T09:06:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098560+0000", 2024-09-14T09:06:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2267e-05 2024-09-14T09:06:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098596+0000", 2024-09-14T09:06:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5812000000000002e-05 2024-09-14T09:06:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:37800 3.1 e48/34 ondisk, result = 0)", 2024-09-14T09:06:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:06:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 60.684862803000001, 2024-09-14T09:06:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057056260999999997, 2024-09-14T09:06:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:06:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:06:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041548+0000", 2024-09-14T09:06:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8100000000000002e-07 2024-09-14T09:06:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041552+0000", 2024-09-14T09:06:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7419999999999999e-06 2024-09-14T09:06:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041563+0000", 2024-09-14T09:06:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1926000000000001e-05 2024-09-14T09:06:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041572+0000", 2024-09-14T09:06:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8610000000000004e-06 2024-09-14T09:06:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098503+0000", 2024-09-14T09:06:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056930827000000003 2024-09-14T09:06:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098512+0000", 2024-09-14T09:06:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5019999999999992e-06 2024-09-14T09:06:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098603+0000", 2024-09-14T09:06:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1621999999999998e-05 2024-09-14T09:06:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38523 3.3 3:c62e0486:::rbd_data.115e9e133b92.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 4194304~28499] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T09:06:57.746 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:06:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 59.272435105, 2024-09-14T09:06:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035654406999999999, 2024-09-14T09:06:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38523 2024-09-14T09:06:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:06:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:06:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453977+0000", 2024-09-14T09:06:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6299999999999998e-06 2024-09-14T09:06:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455684+0000", 2024-09-14T09:06:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001706843 2024-09-14T09:06:57.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455710+0000", 2024-09-14T09:06:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6302e-05 2024-09-14T09:06:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455722+0000", 2024-09-14T09:06:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1798e-05 2024-09-14T09:06:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455825+0000", 2024-09-14T09:06:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102822 2024-09-14T09:06:57.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.456032+0000", 2024-09-14T09:06:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020704400000000001 2024-09-14T09:06:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:06:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.456759+0000", 2024-09-14T09:06:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00072735699999999998 2024-09-14T09:06:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.477856+0000", 2024-09-14T09:06:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021096011000000001 2024-09-14T09:06:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489526+0000", 2024-09-14T09:06:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011670313 2024-09-14T09:06:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489572+0000", 2024-09-14T09:06:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6408e-05 2024-09-14T09:06:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489629+0000", 2024-09-14T09:06:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6879000000000002e-05 2024-09-14T09:06:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.751 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49716 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525151+0000", 2024-09-14T09:06:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.201258443999997, 2024-09-14T09:06:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034564820000000003, 2024-09-14T09:06:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:57.752 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49716 2024-09-14T09:06:57.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.753 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525151+0000", 2024-09-14T09:06:57.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525151+0000", 2024-09-14T09:06:57.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525152+0000", 2024-09-14T09:06:57.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5799999999999998e-07 2024-09-14T09:06:57.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525157+0000", 2024-09-14T09:06:57.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6380000000000003e-06 2024-09-14T09:06:57.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525167+0000", 2024-09-14T09:06:57.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0457e-05 2024-09-14T09:06:57.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525174+0000", 2024-09-14T09:06:57.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6220000000000003e-06 2024-09-14T09:06:57.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559237+0000", 2024-09-14T09:06:57.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034063361 2024-09-14T09:06:57.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559308+0000", 2024-09-14T09:06:57.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1124999999999994e-05 2024-09-14T09:06:57.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559716+0000", 2024-09-14T09:06:57.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000407759 2024-09-14T09:06:57.756 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.756 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.756 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.756 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49717 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:57.756 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525390+0000", 2024-09-14T09:06:57.756 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.201019647000003, 2024-09-14T09:06:57.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034552639000000003, 2024-09-14T09:06:57.756 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.756 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.756 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:57.756 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:57.756 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:57.756 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:57.757 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49717 2024-09-14T09:06:57.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.757 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525390+0000", 2024-09-14T09:06:57.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525390+0000", 2024-09-14T09:06:57.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525391+0000", 2024-09-14T09:06:57.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3099999999999997e-07 2024-09-14T09:06:57.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525395+0000", 2024-09-14T09:06:57.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.267e-06 2024-09-14T09:06:57.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525406+0000", 2024-09-14T09:06:57.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1224e-05 2024-09-14T09:06:57.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525413+0000", 2024-09-14T09:06:57.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5420000000000002e-06 2024-09-14T09:06:57.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559760+0000", 2024-09-14T09:06:57.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034347177999999999 2024-09-14T09:06:57.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559838+0000", 2024-09-14T09:06:57.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7708999999999997e-05 2024-09-14T09:06:57.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559943+0000", 2024-09-14T09:06:57.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010498800000000001 2024-09-14T09:06:57.760 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.760 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.760 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.760 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49718 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:57.760 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525514+0000", 2024-09-14T09:06:57.760 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.200895842000001, 2024-09-14T09:06:57.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034602766, 2024-09-14T09:06:57.760 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.761 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.761 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:57.761 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:57.761 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:57.761 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:57.761 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49718 2024-09-14T09:06:57.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.761 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525514+0000", 2024-09-14T09:06:57.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525514+0000", 2024-09-14T09:06:57.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525515+0000", 2024-09-14T09:06:57.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2099999999999996e-07 2024-09-14T09:06:57.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525518+0000", 2024-09-14T09:06:57.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7220000000000001e-06 2024-09-14T09:06:57.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525528+0000", 2024-09-14T09:06:57.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4639999999999995e-06 2024-09-14T09:06:57.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525534+0000", 2024-09-14T09:06:57.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2330000000000003e-06 2024-09-14T09:06:57.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.559974+0000", 2024-09-14T09:06:57.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034439866 2024-09-14T09:06:57.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560041+0000", 2024-09-14T09:06:57.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7343999999999998e-05 2024-09-14T09:06:57.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560117+0000", 2024-09-14T09:06:57.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5415999999999998e-05 2024-09-14T09:06:57.764 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.764 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.764 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.764 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49719 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:57.764 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:06:57.764 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.200776503999997, 2024-09-14T09:06:57.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034680563999999997, 2024-09-14T09:06:57.764 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.765 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.765 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:57.765 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:57.765 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:57.765 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:57.765 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49719 2024-09-14T09:06:57.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.765 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:06:57.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:06:57.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525634+0000", 2024-09-14T09:06:57.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4000000000000001e-07 2024-09-14T09:06:57.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525638+0000", 2024-09-14T09:06:57.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5899999999999999e-06 2024-09-14T09:06:57.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525647+0000", 2024-09-14T09:06:57.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2029999999999998e-06 2024-09-14T09:06:57.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525653+0000", 2024-09-14T09:06:57.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9850000000000003e-06 2024-09-14T09:06:57.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560150+0000", 2024-09-14T09:06:57.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034496845999999998 2024-09-14T09:06:57.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560221+0000", 2024-09-14T09:06:57.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1400000000000001e-05 2024-09-14T09:06:57.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560314+0000", 2024-09-14T09:06:57.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2800000000000006e-05 2024-09-14T09:06:57.768 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.768 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.768 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.768 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49720 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:57.769 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:06:57.769 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.200524880000003, 2024-09-14T09:06:57.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034919095999999997, 2024-09-14T09:06:57.769 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.769 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.769 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:57.769 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:57.769 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:57.769 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:57.769 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49720 2024-09-14T09:06:57.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.769 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:06:57.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:06:57.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525886+0000", 2024-09-14T09:06:57.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6000000000000002e-07 2024-09-14T09:06:57.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525890+0000", 2024-09-14T09:06:57.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4519999999999999e-06 2024-09-14T09:06:57.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525899+0000", 2024-09-14T09:06:57.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1090000000000004e-06 2024-09-14T09:06:57.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525906+0000", 2024-09-14T09:06:57.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0219999999999999e-06 2024-09-14T09:06:57.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560344+0000", 2024-09-14T09:06:57.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034437970999999998 2024-09-14T09:06:57.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560422+0000", 2024-09-14T09:06:57.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7857999999999999e-05 2024-09-14T09:06:57.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560804+0000", 2024-09-14T09:06:57.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00038182400000000001 2024-09-14T09:06:57.772 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.772 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.772 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.773 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49721 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:57.773 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:06:57.773 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.200477397999997, 2024-09-14T09:06:57.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035347380999999997, 2024-09-14T09:06:57.773 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.773 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.773 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:57.773 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:57.773 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:57.773 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:57.773 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49721 2024-09-14T09:06:57.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.773 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:06:57.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:06:57.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525933+0000", 2024-09-14T09:06:57.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T09:06:57.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525936+0000", 2024-09-14T09:06:57.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6759999999999999e-06 2024-09-14T09:06:57.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525943+0000", 2024-09-14T09:06:57.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2459999999999997e-06 2024-09-14T09:06:57.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525948+0000", 2024-09-14T09:06:57.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9429999999999999e-06 2024-09-14T09:06:57.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560845+0000", 2024-09-14T09:06:57.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034897354999999998 2024-09-14T09:06:57.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560925+0000", 2024-09-14T09:06:57.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9804999999999999e-05 2024-09-14T09:06:57.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561280+0000", 2024-09-14T09:06:57.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035480100000000002 2024-09-14T09:06:57.776 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.777 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.777 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.777 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49722 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3112960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:57.777 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:57.777 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.200453752000001, 2024-09-14T09:06:57.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035750491000000002, 2024-09-14T09:06:57.777 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.777 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.777 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:57.777 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:57.777 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:57.777 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:57.777 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49722 2024-09-14T09:06:57.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.778 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:57.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:57.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:06:57.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.96e-07 2024-09-14T09:06:57.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525958+0000", 2024-09-14T09:06:57.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.024e-06 2024-09-14T09:06:57.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525964+0000", 2024-09-14T09:06:57.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6699999999999999e-06 2024-09-14T09:06:57.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525969+0000", 2024-09-14T09:06:57.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3939999999999998e-06 2024-09-14T09:06:57.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561319+0000", 2024-09-14T09:06:57.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.03535013 2024-09-14T09:06:57.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561390+0000", 2024-09-14T09:06:57.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0920000000000005e-05 2024-09-14T09:06:57.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561707+0000", 2024-09-14T09:06:57.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031695699999999998 2024-09-14T09:06:57.781 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.781 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.781 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.781 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49723 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3506176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:57.781 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:57.781 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.200434768999997, 2024-09-14T09:06:57.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036113064, 2024-09-14T09:06:57.781 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.781 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.781 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:57.781 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:57.781 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:57.781 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:57.782 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49723 2024-09-14T09:06:57.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.782 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:57.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:57.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:06:57.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1699999999999999e-07 2024-09-14T09:06:57.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525977+0000", 2024-09-14T09:06:57.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9250000000000002e-06 2024-09-14T09:06:57.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525983+0000", 2024-09-14T09:06:57.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4620000000000004e-06 2024-09-14T09:06:57.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525987+0000", 2024-09-14T09:06:57.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.279e-06 2024-09-14T09:06:57.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561749+0000", 2024-09-14T09:06:57.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035762222000000003 2024-09-14T09:06:57.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561796+0000", 2024-09-14T09:06:57.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6959999999999998e-05 2024-09-14T09:06:57.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562088+0000", 2024-09-14T09:06:57.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029179899999999998 2024-09-14T09:06:57.785 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.785 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.785 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.785 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49724 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:06:57.785 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:06:57.785 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.200416613000002, 2024-09-14T09:06:57.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036449388999999999, 2024-09-14T09:06:57.785 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.785 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.785 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:57.785 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:57.785 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:57.786 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:57.786 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49724 2024-09-14T09:06:57.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.786 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:06:57.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:06:57.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525994+0000", 2024-09-14T09:06:57.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3099999999999998e-07 2024-09-14T09:06:57.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525996+0000", 2024-09-14T09:06:57.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.939e-06 2024-09-14T09:06:57.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526005+0000", 2024-09-14T09:06:57.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8430000000000005e-06 2024-09-14T09:06:57.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526010+0000", 2024-09-14T09:06:57.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9389999999999996e-06 2024-09-14T09:06:57.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562114+0000", 2024-09-14T09:06:57.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036103510999999998 2024-09-14T09:06:57.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562189+0000", 2024-09-14T09:06:57.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5162999999999998e-05 2024-09-14T09:06:57.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562443+0000", 2024-09-14T09:06:57.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000253563 2024-09-14T09:06:57.789 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.789 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.789 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.789 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:64935 3.12 3:4cd6d7cb:::rbd_data.115e21c31fcf.0000000000000049:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:06:57.789 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:06:57.789 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.975051264999998, 2024-09-14T09:06:57.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037353866999999999, 2024-09-14T09:06:57.789 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.789 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.789 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:57.789 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:57.789 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:57.790 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:57.790 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64935 2024-09-14T09:06:57.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.790 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:06:57.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:06:57.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751362+0000", 2024-09-14T09:06:57.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6899999999999998e-06 2024-09-14T09:06:57.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751378+0000", 2024-09-14T09:06:57.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6147e-05 2024-09-14T09:06:57.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751421+0000", 2024-09-14T09:06:57.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2691999999999997e-05 2024-09-14T09:06:57.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751444+0000", 2024-09-14T09:06:57.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3237000000000001e-05 2024-09-14T09:06:57.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751594+0000", 2024-09-14T09:06:57.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014947800000000001 2024-09-14T09:06:57.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751733+0000", 2024-09-14T09:06:57.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013945200000000001 2024-09-14T09:06:57.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:57.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.753315+0000", 2024-09-14T09:06:57.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0015815969999999999 2024-09-14T09:06:57.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:57.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788546+0000", 2024-09-14T09:06:57.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035231449999999997 2024-09-14T09:06:57.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:57.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788570+0000", 2024-09-14T09:06:57.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3991000000000001e-05 2024-09-14T09:06:57.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:57.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788645+0000", 2024-09-14T09:06:57.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4379000000000001e-05 2024-09-14T09:06:57.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788712+0000", 2024-09-14T09:06:57.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7754000000000002e-05 2024-09-14T09:06:57.794 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.794 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.794 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.794 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:71052 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:06:57.794 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:06:57.794 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.988842013999999, 2024-09-14T09:06:57.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14854635599999999, 2024-09-14T09:06:57.795 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.795 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.795 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:57.795 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:57.795 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:57.795 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:57.795 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 71052 2024-09-14T09:06:57.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.795 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:06:57.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:06:57.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737570+0000", 2024-09-14T09:06:57.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4770000000000002e-06 2024-09-14T09:06:57.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737584+0000", 2024-09-14T09:06:57.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3457e-05 2024-09-14T09:06:57.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737608+0000", 2024-09-14T09:06:57.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3893999999999999e-05 2024-09-14T09:06:57.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737620+0000", 2024-09-14T09:06:57.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2157000000000001e-05 2024-09-14T09:06:57.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737738+0000", 2024-09-14T09:06:57.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118409 2024-09-14T09:06:57.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737892+0000", 2024-09-14T09:06:57.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154222 2024-09-14T09:06:57.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:06:57.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.749622+0000", 2024-09-14T09:06:57.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011729537 2024-09-14T09:06:57.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:57.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885934+0000", 2024-09-14T09:06:57.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13631222200000001 2024-09-14T09:06:57.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:57.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885996+0000", 2024-09-14T09:06:57.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1555000000000003e-05 2024-09-14T09:06:57.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:57.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886060+0000", 2024-09-14T09:06:57.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3980999999999996e-05 2024-09-14T09:06:57.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886114+0000", 2024-09-14T09:06:57.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4444999999999998e-05 2024-09-14T09:06:57.799 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.799 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.800 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.800 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:71052 3.1 e71/34 ondisk, result = 0)", 2024-09-14T09:06:57.800 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:06:57.800 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.945938636000001, 2024-09-14T09:06:57.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10565334799999999, 2024-09-14T09:06:57.800 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.800 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.800 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:57.800 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:06:57.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:06:57.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780473+0000", 2024-09-14T09:06:57.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8980000000000001e-06 2024-09-14T09:06:57.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780482+0000", 2024-09-14T09:06:57.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9080000000000001e-06 2024-09-14T09:06:57.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780511+0000", 2024-09-14T09:06:57.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9490999999999999e-05 2024-09-14T09:06:57.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780527+0000", 2024-09-14T09:06:57.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.508e-05 2024-09-14T09:06:57.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885978+0000", 2024-09-14T09:06:57.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.105451328 2024-09-14T09:06:57.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885989+0000", 2024-09-14T09:06:57.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0875e-05 2024-09-14T09:06:57.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886125+0000", 2024-09-14T09:06:57.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013576800000000001 2024-09-14T09:06:57.803 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.803 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.803 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.804 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73871 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:1b [read 8339456~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:57.804 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:06:57.804 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 22.300453628, 2024-09-14T09:06:57.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034605047, 2024-09-14T09:06:57.804 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.804 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.804 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:57.804 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:57.804 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:57.804 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:57.804 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73871 2024-09-14T09:06:57.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.804 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:06:57.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:06:57.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425957+0000", 2024-09-14T09:06:57.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T09:06:57.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425961+0000", 2024-09-14T09:06:57.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8279999999999996e-06 2024-09-14T09:06:57.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425985+0000", 2024-09-14T09:06:57.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4182e-05 2024-09-14T09:06:57.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425997+0000", 2024-09-14T09:06:57.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.132e-05 2024-09-14T09:06:57.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460070+0000", 2024-09-14T09:06:57.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034072913000000003 2024-09-14T09:06:57.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460298+0000", 2024-09-14T09:06:57.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022819 2024-09-14T09:06:57.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460561+0000", 2024-09-14T09:06:57.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026341400000000001 2024-09-14T09:06:57.807 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.807 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.807 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.808 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:74276 3.12 3:49b683dd:::rbd_data.115ea8d131ff.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1376256~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:57.808 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:06:57.808 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 22.041306538000001, 2024-09-14T09:06:57.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036482939999999998, 2024-09-14T09:06:57.808 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.808 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.808 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:57.808 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:57.808 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:57.808 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:57.808 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74276 2024-09-14T09:06:57.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.809 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:06:57.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:06:57.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685105+0000", 2024-09-14T09:06:57.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9520000000000001e-06 2024-09-14T09:06:57.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686530+0000", 2024-09-14T09:06:57.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001424629 2024-09-14T09:06:57.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686550+0000", 2024-09-14T09:06:57.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0472e-05 2024-09-14T09:06:57.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686561+0000", 2024-09-14T09:06:57.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T09:06:57.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686671+0000", 2024-09-14T09:06:57.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110621 2024-09-14T09:06:57.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686919+0000", 2024-09-14T09:06:57.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024790199999999999 2024-09-14T09:06:57.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:57.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.687375+0000", 2024-09-14T09:06:57.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00045572100000000002 2024-09-14T09:06:57.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:57.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.710871+0000", 2024-09-14T09:06:57.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023496481999999999 2024-09-14T09:06:57.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:57.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721478+0000", 2024-09-14T09:06:57.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01060709 2024-09-14T09:06:57.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:57.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721534+0000", 2024-09-14T09:06:57.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5912999999999998e-05 2024-09-14T09:06:57.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721586+0000", 2024-09-14T09:06:57.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1792000000000002e-05 2024-09-14T09:06:57.813 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.813 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.813 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.813 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:84402 3.6 3:6123bacd:::rbd_data.115e77b799c3.0000000000000017:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:06:57.813 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:53.874733+0000", 2024-09-14T09:06:57.813 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.851676318, 2024-09-14T09:06:57.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040401678000000003, 2024-09-14T09:06:57.813 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:57.814 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:57.814 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:57.814 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:57.814 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:57.814 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:57.814 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 84402 2024-09-14T09:06:57.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.814 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:57.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:57.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874733+0000", 2024-09-14T09:06:57.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:57.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874733+0000", 2024-09-14T09:06:57.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:57.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:57.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874735+0000", 2024-09-14T09:06:57.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3170000000000001e-06 2024-09-14T09:06:57.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:57.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874742+0000", 2024-09-14T09:06:57.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0119999999999999e-06 2024-09-14T09:06:57.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:57.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874758+0000", 2024-09-14T09:06:57.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5971e-05 2024-09-14T09:06:57.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:57.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874768+0000", 2024-09-14T09:06:57.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0257e-05 2024-09-14T09:06:57.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:57.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874816+0000", 2024-09-14T09:06:57.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8133999999999997e-05 2024-09-14T09:06:57.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:57.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874862+0000", 2024-09-14T09:06:57.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6269e-05 2024-09-14T09:06:57.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:57.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.876731+0000", 2024-09-14T09:06:57.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001868722 2024-09-14T09:06:57.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:57.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.914723+0000", 2024-09-14T09:06:57.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037991912000000003 2024-09-14T09:06:57.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:57.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.914839+0000", 2024-09-14T09:06:57.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115901 2024-09-14T09:06:57.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:57.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.915013+0000", 2024-09-14T09:06:57.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017389 2024-09-14T09:06:57.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:57.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:57.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:57.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.915135+0000", 2024-09-14T09:06:57.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012229299999999999 2024-09-14T09:06:57.818 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.818 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.818 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.819 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:57.819 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:57.819 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:57.819 INFO:teuthology.orchestra.run.smithi176.stdout:4364 punch from 0x55c6a44 to 0x55d0cd3, (0xa28f bytes) 2024-09-14T09:06:57.820 INFO:teuthology.orchestra.run.smithi176.stdout:4365 punch from 0x4cd2072 to 0x4cd7267, (0x51f5 bytes) 2024-09-14T09:06:57.820 INFO:teuthology.orchestra.run.smithi176.stdout:4367 write 0x832d86d thru 0x833188f (0x4023 bytes) 2024-09-14T09:06:57.820 INFO:teuthology.orchestra.run.smithi176.stdout:4370 writesame 0xd114a6a thru 0xd12161d data_size 0x2(0xcbb4 bytes) 2024-09-14T09:06:57.821 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:06:57.821 INFO:teuthology.orchestra.run.smithi176.stdout:4371 read 0x2a24047 thru 0x2a31578 (0xd532 bytes) 2024-09-14T09:06:57.821 INFO:teuthology.orchestra.run.smithi176.stdout:4372 write 0x729b17d thru 0x72a3ecf (0x8d53 bytes) 2024-09-14T09:06:57.821 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:57.819+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:57.892 INFO:teuthology.orchestra.run.smithi176.stdout:4373 writesame 0x7981834 thru 0x7982560 data_size 0x1(0xd2d bytes) 2024-09-14T09:06:57.898 INFO:teuthology.orchestra.run.smithi176.stdout:4374 write 0x72da35 thru 0x72faca (0x2096 bytes) 2024-09-14T09:06:57.921 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:57.923+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:57.937 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is unset 2024-09-14T09:06:57.956 INFO:teuthology.orchestra.run.smithi176.stdout:4375 punch from 0xc4c575a to 0xc4d1e19, (0xc6bf bytes) 2024-09-14T09:06:57.956 INFO:teuthology.orchestra.run.smithi176.stdout:4376 write 0xce3abff thru 0xce3f612 (0x4a14 bytes) 2024-09-14T09:06:57.960 INFO:teuthology.orchestra.run.smithi176.stdout:4377 read 0xd15e052 thru 0xd16a3a3 (0xc352 bytes) 2024-09-14T09:06:57.965 INFO:teuthology.orchestra.run.smithi176.stdout:4378 read 0x45b9b55 thru 0x45c33e0 (0x988c bytes) 2024-09-14T09:06:57.965 INFO:teuthology.orchestra.run.smithi176.stdout:4379 writesame 0x7b40f7c thru 0x7b489e4 data_size 0x1(0x7a69 bytes) 2024-09-14T09:06:57.992 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:57.993 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:57.993 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:57.993 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:58.007 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:06:58.050 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:58.051+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:58.078 INFO:teuthology.orchestra.run.smithi176.stdout:4380 punch from 0x8bfae58 to 0x8c05cf1, (0xae99 bytes) 2024-09-14T09:06:58.078 INFO:teuthology.orchestra.run.smithi176.stdout:4381 write 0x4995faa thru 0x499b43c (0x5493 bytes) 2024-09-14T09:06:58.081 INFO:teuthology.orchestra.run.smithi176.stdout:4382 writesame 0xdd1325c thru 0xdd15ed5 data_size 0x1(0x2c7a bytes) 2024-09-14T09:06:58.088 INFO:teuthology.orchestra.run.smithi176.stdout:4384 write 0x78f9153 thru 0x7907577 (0xe425 bytes) 2024-09-14T09:06:58.095 INFO:teuthology.orchestra.run.smithi176.stdout:4386 writesame 0x9aa5055 thru 0x9ab0caf data_size 0x1(0xbc5b bytes) 2024-09-14T09:06:58.122 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:58.123+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:58.161 INFO:teuthology.orchestra.run.smithi176.stdout:4387 read 0x3c7329e thru 0x3c809f0 (0xd753 bytes) 2024-09-14T09:06:58.161 INFO:teuthology.orchestra.run.smithi176.stdout:4390 punch from 0xad8b8ce to 0xad99f44, (0xe676 bytes) 2024-09-14T09:06:58.162 INFO:teuthology.orchestra.run.smithi176.stdout:4392 read 0x695c5c7 thru 0x6968023 (0xba5d bytes) 2024-09-14T09:06:58.162 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:58.162 INFO:teuthology.orchestra.run.smithi176.stdout:4398 punch from 0x731980b to 0x7321a0a, (0x81ff bytes) 2024-09-14T09:06:58.169 INFO:teuthology.orchestra.run.smithi176.stdout:4399 trunc from 0xe873205 to 0x9d6251e 2024-09-14T09:06:58.181 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:58.182 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:58.182 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:58.182 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:58.182 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:58.186 INFO:teuthology.orchestra.run.smithi176.stdout:4401 writesame 0x74113dc thru 0x74117dc data_size 0x5(0x401 bytes) 2024-09-14T09:06:58.199 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:06:58.221 INFO:teuthology.orchestra.run.smithi176.stdout:4403 writesame 0x86492b8 thru 0x8651013 data_size 0x47(0x7d5c bytes) 2024-09-14T09:06:58.223 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:58.227+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:58.225 INFO:teuthology.orchestra.run.smithi176.stdout:4404 write 0x5eced0b thru 0x5ed242c (0x3722 bytes) 2024-09-14T09:06:58.271 INFO:teuthology.orchestra.run.smithi176.stdout:4405 read 0x86c2079 thru 0x86cc0bc (0xa044 bytes) 2024-09-14T09:06:58.271 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:58.272 INFO:teuthology.orchestra.run.smithi176.stdout:4407 read 0x9866747 thru 0x9873ad6 (0xd390 bytes) 2024-09-14T09:06:58.273 INFO:teuthology.orchestra.run.smithi176.stdout:4408 write 0x1df94ca thru 0x1dff2d7 (0x5e0e bytes) 2024-09-14T09:06:58.318 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:06:58.318 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:06:58.318 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:06:58.341 INFO:teuthology.orchestra.run.smithi176.stdout:4409 trunc from 0x9d6251e to 0xaf07918 2024-09-14T09:06:58.341 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:58.339+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:58.353 INFO:teuthology.orchestra.run.smithi176.stdout:4410 punch from 0x78adf66 to 0x78bafed, (0xd087 bytes) 2024-09-14T09:06:58.356 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73540 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15155200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:06:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.087498414999999, 2024-09-14T09:06:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.022930347, 2024-09-14T09:06:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.358 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73540 2024-09-14T09:06:58.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.358 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:06:58.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:06:58.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268665+0000", 2024-09-14T09:06:58.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6599999999999996e-07 2024-09-14T09:06:58.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268668+0000", 2024-09-14T09:06:58.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8760000000000001e-06 2024-09-14T09:06:58.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268676+0000", 2024-09-14T09:06:58.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7879999999999995e-06 2024-09-14T09:06:58.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268683+0000", 2024-09-14T09:06:58.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3180000000000003e-06 2024-09-14T09:06:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282017+0000", 2024-09-14T09:06:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01333409 2024-09-14T09:06:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282020+0000", 2024-09-14T09:06:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1609999999999999e-06 2024-09-14T09:06:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291158+0000", 2024-09-14T09:06:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0091376430000001 2024-09-14T09:06:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291383+0000", 2024-09-14T09:06:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022491000000000001 2024-09-14T09:06:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291595+0000", 2024-09-14T09:06:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021199500000000001 2024-09-14T09:06:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.362 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73541 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15237120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:06:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.087470995, 2024-09-14T09:06:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0232607330000001, 2024-09-14T09:06:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.363 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73541 2024-09-14T09:06:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:06:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:06:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:06:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5400000000000001e-07 2024-09-14T09:06:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268695+0000", 2024-09-14T09:06:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5799999999999999e-06 2024-09-14T09:06:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268703+0000", 2024-09-14T09:06:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5480000000000002e-06 2024-09-14T09:06:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268713+0000", 2024-09-14T09:06:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8640000000000008e-06 2024-09-14T09:06:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282032+0000", 2024-09-14T09:06:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013318831999999999 2024-09-14T09:06:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282035+0000", 2024-09-14T09:06:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4010000000000001e-06 2024-09-14T09:06:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291701+0000", 2024-09-14T09:06:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0096654199999999 2024-09-14T09:06:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291807+0000", 2024-09-14T09:06:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105852 2024-09-14T09:06:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291952+0000", 2024-09-14T09:06:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014568200000000001 2024-09-14T09:06:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73542 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15319040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:06:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.087401384, 2024-09-14T09:06:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0237091359999999, 2024-09-14T09:06:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73542 2024-09-14T09:06:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:06:58.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:06:58.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268762+0000", 2024-09-14T09:06:58.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:06:58.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268766+0000", 2024-09-14T09:06:58.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5880000000000002e-06 2024-09-14T09:06:58.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268776+0000", 2024-09-14T09:06:58.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0168e-05 2024-09-14T09:06:58.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268789+0000", 2024-09-14T09:06:58.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:06:58.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282053+0000", 2024-09-14T09:06:58.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013264496000000001 2024-09-14T09:06:58.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282057+0000", 2024-09-14T09:06:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6090000000000001e-06 2024-09-14T09:06:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292052+0000", 2024-09-14T09:06:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0099950070000001 2024-09-14T09:06:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292262+0000", 2024-09-14T09:06:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021009800000000001 2024-09-14T09:06:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292470+0000", 2024-09-14T09:06:58.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000208314 2024-09-14T09:06:58.373 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.373 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.373 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.373 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73543 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.373 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:06:58.373 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.087362593999998, 2024-09-14T09:06:58.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023985473, 2024-09-14T09:06:58.373 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73543 2024-09-14T09:06:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:06:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:06:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268801+0000", 2024-09-14T09:06:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:06:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268803+0000", 2024-09-14T09:06:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7439999999999999e-06 2024-09-14T09:06:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268812+0000", 2024-09-14T09:06:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7080000000000003e-06 2024-09-14T09:06:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268819+0000", 2024-09-14T09:06:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T09:06:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282068+0000", 2024-09-14T09:06:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013249540000000001 2024-09-14T09:06:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282072+0000", 2024-09-14T09:06:58.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0960000000000001e-06 2024-09-14T09:06:58.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292573+0000", 2024-09-14T09:06:58.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0105014560000001 2024-09-14T09:06:58.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292679+0000", 2024-09-14T09:06:58.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010545899999999999 2024-09-14T09:06:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292786+0000", 2024-09-14T09:06:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107052 2024-09-14T09:06:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73544 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:06:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.087334892000001, 2024-09-14T09:06:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0244830549999999, 2024-09-14T09:06:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.378 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73544 2024-09-14T09:06:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:06:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:06:58.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:06:58.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5599999999999995e-07 2024-09-14T09:06:58.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268831+0000", 2024-09-14T09:06:58.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.57e-06 2024-09-14T09:06:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268844+0000", 2024-09-14T09:06:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2811e-05 2024-09-14T09:06:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268851+0000", 2024-09-14T09:06:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9140000000000002e-06 2024-09-14T09:06:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282083+0000", 2024-09-14T09:06:58.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013232111 2024-09-14T09:06:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282086+0000", 2024-09-14T09:06:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2909999999999999e-06 2024-09-14T09:06:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292884+0000", 2024-09-14T09:06:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0107975069999999 2024-09-14T09:06:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293100+0000", 2024-09-14T09:06:58.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021639399999999999 2024-09-14T09:06:58.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293311+0000", 2024-09-14T09:06:58.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021090099999999999 2024-09-14T09:06:58.383 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.383 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.383 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73545 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15564800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:06:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.087283707000001, 2024-09-14T09:06:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0249746740000001, 2024-09-14T09:06:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73545 2024-09-14T09:06:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:06:58.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:06:58.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268880+0000", 2024-09-14T09:06:58.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5600000000000005e-07 2024-09-14T09:06:58.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268889+0000", 2024-09-14T09:06:58.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8559999999999996e-06 2024-09-14T09:06:58.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268913+0000", 2024-09-14T09:06:58.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3958e-05 2024-09-14T09:06:58.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268921+0000", 2024-09-14T09:06:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7479999999999999e-06 2024-09-14T09:06:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282098+0000", 2024-09-14T09:06:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013176297999999999 2024-09-14T09:06:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282101+0000", 2024-09-14T09:06:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2329999999999998e-06 2024-09-14T09:06:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293435+0000", 2024-09-14T09:06:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0113339459999999 2024-09-14T09:06:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293648+0000", 2024-09-14T09:06:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021307100000000001 2024-09-14T09:06:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293854+0000", 2024-09-14T09:06:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020580800000000001 2024-09-14T09:06:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.388 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73546 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15646720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:06:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.087229835999999, 2024-09-14T09:06:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0254846360000001, 2024-09-14T09:06:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73546 2024-09-14T09:06:58.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.390 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:06:58.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:06:58.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268934+0000", 2024-09-14T09:06:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7999999999999995e-07 2024-09-14T09:06:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268937+0000", 2024-09-14T09:06:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9840000000000001e-06 2024-09-14T09:06:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268947+0000", 2024-09-14T09:06:58.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0522e-05 2024-09-14T09:06:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268955+0000", 2024-09-14T09:06:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8059999999999995e-06 2024-09-14T09:06:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282112+0000", 2024-09-14T09:06:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013157189 2024-09-14T09:06:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282115+0000", 2024-09-14T09:06:58.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2949999999999998e-06 2024-09-14T09:06:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293968+0000", 2024-09-14T09:06:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0118530990000001 2024-09-14T09:06:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294197+0000", 2024-09-14T09:06:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228507 2024-09-14T09:06:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294417+0000", 2024-09-14T09:06:58.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000220554 2024-09-14T09:06:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73547 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:06:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.08719739, 2024-09-14T09:06:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.025970525, 2024-09-14T09:06:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73547 2024-09-14T09:06:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:06:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:06:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268966+0000", 2024-09-14T09:06:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3799999999999997e-07 2024-09-14T09:06:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268969+0000", 2024-09-14T09:06:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8739999999999999e-06 2024-09-14T09:06:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268979+0000", 2024-09-14T09:06:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9939999999999995e-06 2024-09-14T09:06:58.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268986+0000", 2024-09-14T09:06:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3459999999999996e-06 2024-09-14T09:06:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282127+0000", 2024-09-14T09:06:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01314039 2024-09-14T09:06:58.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282130+0000", 2024-09-14T09:06:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2219999999999998e-06 2024-09-14T09:06:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294540+0000", 2024-09-14T09:06:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124105880000001 2024-09-14T09:06:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294759+0000", 2024-09-14T09:06:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000218798 2024-09-14T09:06:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294936+0000", 2024-09-14T09:06:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000176675 2024-09-14T09:06:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73548 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15810560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.399 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:06:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.087166278000002, 2024-09-14T09:06:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0265045129999999, 2024-09-14T09:06:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73548 2024-09-14T09:06:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:06:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:06:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268997+0000", 2024-09-14T09:06:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7599999999999996e-07 2024-09-14T09:06:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269000+0000", 2024-09-14T09:06:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0580000000000002e-06 2024-09-14T09:06:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269010+0000", 2024-09-14T09:06:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124e-05 2024-09-14T09:06:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269018+0000", 2024-09-14T09:06:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5399999999999998e-06 2024-09-14T09:06:58.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282141+0000", 2024-09-14T09:06:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013123012 2024-09-14T09:06:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282144+0000", 2024-09-14T09:06:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.196e-06 2024-09-14T09:06:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295042+0000", 2024-09-14T09:06:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0128975689999999 2024-09-14T09:06:58.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295272+0000", 2024-09-14T09:06:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023048099999999999 2024-09-14T09:06:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295501+0000", 2024-09-14T09:06:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228657 2024-09-14T09:06:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73549 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15892480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:06:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.087120399, 2024-09-14T09:06:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0268354099999999, 2024-09-14T09:06:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73549 2024-09-14T09:06:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:06:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:06:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269043+0000", 2024-09-14T09:06:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6199999999999997e-07 2024-09-14T09:06:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269046+0000", 2024-09-14T09:06:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2480000000000001e-06 2024-09-14T09:06:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269057+0000", 2024-09-14T09:06:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0436e-05 2024-09-14T09:06:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269064+0000", 2024-09-14T09:06:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5179999999999998e-06 2024-09-14T09:06:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282155+0000", 2024-09-14T09:06:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013091190000000001 2024-09-14T09:06:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282159+0000", 2024-09-14T09:06:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.366e-06 2024-09-14T09:06:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295599+0000", 2024-09-14T09:06:58.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0134405559999999 2024-09-14T09:06:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295765+0000", 2024-09-14T09:06:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016575499999999999 2024-09-14T09:06:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295878+0000", 2024-09-14T09:06:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112679 2024-09-14T09:06:58.409 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73550 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15974400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:06:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.087087798999999, 2024-09-14T09:06:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027357657, 2024-09-14T09:06:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.410 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73550 2024-09-14T09:06:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:06:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:06:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269076+0000", 2024-09-14T09:06:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:06:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269079+0000", 2024-09-14T09:06:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:06:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269098+0000", 2024-09-14T09:06:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9698e-05 2024-09-14T09:06:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269106+0000", 2024-09-14T09:06:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6179999999999997e-06 2024-09-14T09:06:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282170+0000", 2024-09-14T09:06:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013063874 2024-09-14T09:06:58.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282173+0000", 2024-09-14T09:06:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1999999999999999e-06 2024-09-14T09:06:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295996+0000", 2024-09-14T09:06:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.013823151 2024-09-14T09:06:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296214+0000", 2024-09-14T09:06:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021817700000000001 2024-09-14T09:06:58.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296433+0000", 2024-09-14T09:06:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021829099999999999 2024-09-14T09:06:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73551 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:06:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.087046382, 2024-09-14T09:06:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027730276, 2024-09-14T09:06:58.415 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73551 2024-09-14T09:06:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:06:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:06:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269117+0000", 2024-09-14T09:06:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:06:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269120+0000", 2024-09-14T09:06:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.056e-06 2024-09-14T09:06:58.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269130+0000", 2024-09-14T09:06:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0084e-05 2024-09-14T09:06:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269137+0000", 2024-09-14T09:06:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1359999999999999e-06 2024-09-14T09:06:58.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282184+0000", 2024-09-14T09:06:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013046537 2024-09-14T09:06:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282188+0000", 2024-09-14T09:06:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3220000000000003e-06 2024-09-14T09:06:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296529+0000", 2024-09-14T09:06:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0143406109999999 2024-09-14T09:06:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296695+0000", 2024-09-14T09:06:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016604799999999999 2024-09-14T09:06:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296847+0000", 2024-09-14T09:06:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000151822 2024-09-14T09:06:58.422 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73552 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.423 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:06:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.087015802, 2024-09-14T09:06:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0281767500000001, 2024-09-14T09:06:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.424 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.425 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.425 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73552 2024-09-14T09:06:58.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.425 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:06:58.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:06:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:06:58.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:06:58.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269150+0000", 2024-09-14T09:06:58.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9119999999999998e-06 2024-09-14T09:06:58.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269160+0000", 2024-09-14T09:06:58.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6819999999999998e-06 2024-09-14T09:06:58.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269168+0000", 2024-09-14T09:06:58.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6079999999999998e-06 2024-09-14T09:06:58.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282205+0000", 2024-09-14T09:06:58.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013036982000000001 2024-09-14T09:06:58.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282208+0000", 2024-09-14T09:06:58.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4620000000000001e-06 2024-09-14T09:06:58.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296968+0000", 2024-09-14T09:06:58.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0147601209999999 2024-09-14T09:06:58.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297115+0000", 2024-09-14T09:06:58.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014673900000000001 2024-09-14T09:06:58.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297324+0000", 2024-09-14T09:06:58.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020866400000000001 2024-09-14T09:06:58.435 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.435 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.435 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.435 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73553 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.436 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:06:58.436 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.086985737999999, 2024-09-14T09:06:58.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0286556739999999, 2024-09-14T09:06:58.436 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.436 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.436 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.437 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73553 2024-09-14T09:06:58.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.437 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:06:58.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:06:58.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269178+0000", 2024-09-14T09:06:58.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:06:58.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269180+0000", 2024-09-14T09:06:58.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T09:06:58.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269189+0000", 2024-09-14T09:06:58.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2e-06 2024-09-14T09:06:58.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269197+0000", 2024-09-14T09:06:58.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2980000000000003e-06 2024-09-14T09:06:58.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282219+0000", 2024-09-14T09:06:58.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013022687 2024-09-14T09:06:58.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282223+0000", 2024-09-14T09:06:58.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.241e-06 2024-09-14T09:06:58.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297470+0000", 2024-09-14T09:06:58.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.015247778 2024-09-14T09:06:58.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297704+0000", 2024-09-14T09:06:58.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000233367 2024-09-14T09:06:58.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297833+0000", 2024-09-14T09:06:58.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012888900000000001 2024-09-14T09:06:58.447 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.447 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.447 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.447 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73554 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.448 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:06:58.448 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.086956421, 2024-09-14T09:06:58.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.029005113, 2024-09-14T09:06:58.448 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.448 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.448 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.449 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.449 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.449 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.449 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73554 2024-09-14T09:06:58.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.449 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:06:58.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:06:58.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269207+0000", 2024-09-14T09:06:58.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.68e-07 2024-09-14T09:06:58.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269210+0000", 2024-09-14T09:06:58.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8100000000000002e-06 2024-09-14T09:06:58.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269219+0000", 2024-09-14T09:06:58.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:06:58.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269227+0000", 2024-09-14T09:06:58.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3839999999999999e-06 2024-09-14T09:06:58.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282234+0000", 2024-09-14T09:06:58.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013007187 2024-09-14T09:06:58.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282237+0000", 2024-09-14T09:06:58.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1200000000000002e-06 2024-09-14T09:06:58.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297942+0000", 2024-09-14T09:06:58.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0157048040000001 2024-09-14T09:06:58.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298087+0000", 2024-09-14T09:06:58.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014499099999999999 2024-09-14T09:06:58.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298211+0000", 2024-09-14T09:06:58.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012461899999999999 2024-09-14T09:06:58.459 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.459 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.459 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.460 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73555 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.460 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:06:58.460 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.086926573, 2024-09-14T09:06:58.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.02941558, 2024-09-14T09:06:58.460 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.460 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.461 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.461 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.461 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.461 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.461 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73555 2024-09-14T09:06:58.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.461 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:06:58.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:06:58.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269237+0000", 2024-09-14T09:06:58.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3399999999999999e-07 2024-09-14T09:06:58.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269240+0000", 2024-09-14T09:06:58.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8480000000000001e-06 2024-09-14T09:06:58.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269249+0000", 2024-09-14T09:06:58.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1880000000000008e-06 2024-09-14T09:06:58.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269256+0000", 2024-09-14T09:06:58.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:06:58.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282248+0000", 2024-09-14T09:06:58.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012992327 2024-09-14T09:06:58.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282251+0000", 2024-09-14T09:06:58.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2160000000000002e-06 2024-09-14T09:06:58.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298299+0000", 2024-09-14T09:06:58.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0160479330000001 2024-09-14T09:06:58.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298446+0000", 2024-09-14T09:06:58.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000146525 2024-09-14T09:06:58.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298652+0000", 2024-09-14T09:06:58.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020584100000000001 2024-09-14T09:06:58.465 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.465 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.465 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.466 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73556 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.466 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:06:58.466 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.086896827, 2024-09-14T09:06:58.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0299854239999999, 2024-09-14T09:06:58.466 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.466 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.466 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.466 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.466 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.466 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.466 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73556 2024-09-14T09:06:58.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.466 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:06:58.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:06:58.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:06:58.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:06:58.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269270+0000", 2024-09-14T09:06:58.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0920000000000002e-06 2024-09-14T09:06:58.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269280+0000", 2024-09-14T09:06:58.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0013999999999999e-05 2024-09-14T09:06:58.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269288+0000", 2024-09-14T09:06:58.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:06:58.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282263+0000", 2024-09-14T09:06:58.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012974703000000001 2024-09-14T09:06:58.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282266+0000", 2024-09-14T09:06:58.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1520000000000001e-06 2024-09-14T09:06:58.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298729+0000", 2024-09-14T09:06:58.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.016463506 2024-09-14T09:06:58.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298884+0000", 2024-09-14T09:06:58.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015450800000000001 2024-09-14T09:06:58.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299251+0000", 2024-09-14T09:06:58.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036759899999999997 2024-09-14T09:06:58.470 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.470 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.470 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.471 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73557 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16547840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.471 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:06:58.471 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.086850176999999, 2024-09-14T09:06:58.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0307969480000001, 2024-09-14T09:06:58.471 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.471 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.471 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.471 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.471 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.471 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.471 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73557 2024-09-14T09:06:58.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.471 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:06:58.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:06:58.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269313+0000", 2024-09-14T09:06:58.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.92e-07 2024-09-14T09:06:58.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269336+0000", 2024-09-14T09:06:58.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3118000000000001e-05 2024-09-14T09:06:58.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269346+0000", 2024-09-14T09:06:58.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4069999999999999e-06 2024-09-14T09:06:58.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269352+0000", 2024-09-14T09:06:58.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5810000000000002e-06 2024-09-14T09:06:58.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282277+0000", 2024-09-14T09:06:58.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012924658 2024-09-14T09:06:58.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282280+0000", 2024-09-14T09:06:58.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2880000000000001e-06 2024-09-14T09:06:58.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299343+0000", 2024-09-14T09:06:58.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0170630190000001 2024-09-14T09:06:58.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299554+0000", 2024-09-14T09:06:58.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021085099999999999 2024-09-14T09:06:58.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300109+0000", 2024-09-14T09:06:58.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00055533399999999999 2024-09-14T09:06:58.475 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.475 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.475 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.475 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73558 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.476 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:06:58.476 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.086798554000001, 2024-09-14T09:06:58.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0314849340000001, 2024-09-14T09:06:58.476 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.476 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.476 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.476 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73558 2024-09-14T09:06:58.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.476 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:06:58.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:06:58.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269365+0000", 2024-09-14T09:06:58.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7400000000000003e-07 2024-09-14T09:06:58.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269367+0000", 2024-09-14T09:06:58.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4930000000000001e-06 2024-09-14T09:06:58.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269374+0000", 2024-09-14T09:06:58.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9519999999999997e-06 2024-09-14T09:06:58.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269379+0000", 2024-09-14T09:06:58.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9710000000000003e-06 2024-09-14T09:06:58.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282292+0000", 2024-09-14T09:06:58.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012912704000000001 2024-09-14T09:06:58.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282295+0000", 2024-09-14T09:06:58.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.179e-06 2024-09-14T09:06:58.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300195+0000", 2024-09-14T09:06:58.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.017899975 2024-09-14T09:06:58.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300345+0000", 2024-09-14T09:06:58.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014957499999999999 2024-09-14T09:06:58.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300849+0000", 2024-09-14T09:06:58.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050451100000000002 2024-09-14T09:06:58.480 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.480 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.480 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.480 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73559 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:06:58.480 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:06:58.480 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.086776145000002, 2024-09-14T09:06:58.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.031916343, 2024-09-14T09:06:58.481 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.481 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.481 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.481 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.481 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.481 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.481 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73559 2024-09-14T09:06:58.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.481 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:06:58.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:06:58.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:06:58.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3500000000000002e-07 2024-09-14T09:06:58.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269390+0000", 2024-09-14T09:06:58.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5890000000000001e-06 2024-09-14T09:06:58.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269396+0000", 2024-09-14T09:06:58.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3300000000000004e-06 2024-09-14T09:06:58.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269401+0000", 2024-09-14T09:06:58.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0340000000000004e-06 2024-09-14T09:06:58.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282311+0000", 2024-09-14T09:06:58.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012910204 2024-09-14T09:06:58.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:58.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282317+0000", 2024-09-14T09:06:58.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2639999999999997e-06 2024-09-14T09:06:58.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300933+0000", 2024-09-14T09:06:58.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0186160710000001 2024-09-14T09:06:58.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301158+0000", 2024-09-14T09:06:58.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225021 2024-09-14T09:06:58.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301303+0000", 2024-09-14T09:06:58.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014439499999999999 2024-09-14T09:06:58.485 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.485 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.485 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.485 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.485 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.485 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:58.486 INFO:teuthology.orchestra.run.smithi176.stdout:4411 trunc from 0xaf07918 to 0xe5ed9d0 2024-09-14T09:06:58.486 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:58.486 INFO:teuthology.orchestra.run.smithi176.stdout:4414 read 0x3800a32 thru 0x380f998 (0xef67 bytes) 2024-09-14T09:06:58.486 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:58.486 INFO:teuthology.orchestra.run.smithi176.stdout:4416 write 0x7f1b507 thru 0x7f22ca7 (0x77a1 bytes) 2024-09-14T09:06:58.487 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:06:58.488 INFO:teuthology.orchestra.run.smithi176.stdout:4417 read 0x9aed8b7 thru 0x9af33a0 (0x5aea bytes) 2024-09-14T09:06:58.488 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:58.488 INFO:teuthology.orchestra.run.smithi176.stdout:4419 write 0x1d8720f thru 0x1d8d81d (0x660f bytes) 2024-09-14T09:06:58.488 INFO:teuthology.orchestra.run.smithi176.stdout:4420 flatten 2024-09-14T09:06:58.488 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:58.487+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:58.566 INFO:teuthology.orchestra.run.smithi176.stdout:4422 punch from 0x434ec9 to 0x43b8db, (0x6a12 bytes) 2024-09-14T09:06:58.566 INFO:teuthology.orchestra.run.smithi176.stdout:4423 writesame 0x67027db thru 0x670f62a data_size 0x8(0xce50 bytes) 2024-09-14T09:06:58.582 INFO:teuthology.orchestra.run.smithi176.stdout:4424 read 0x568afbd thru 0x568b210 (0x254 bytes) 2024-09-14T09:06:58.583 INFO:teuthology.orchestra.run.smithi176.stdout:4426 write 0x1e2f879 thru 0x1e2fc41 (0x3c9 bytes) 2024-09-14T09:06:58.585 INFO:teuthology.orchestra.run.smithi176.stdout:4427 read 0xa95880b thru 0xa962b54 (0xa34a bytes) 2024-09-14T09:06:58.586 INFO:teuthology.orchestra.run.smithi176.stdout:4428 write 0x2475655 thru 0x248360f (0xdfbb bytes) 2024-09-14T09:06:58.586 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:58.587+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:58.593 INFO:teuthology.orchestra.run.smithi176.stdout:4429 write 0xaafefc3 thru 0xab03efb (0x4f39 bytes) 2024-09-14T09:06:58.596 INFO:teuthology.orchestra.run.smithi176.stdout:4430 writesame 0x6f02b21 thru 0x6f0d141 data_size 0x1(0xa621 bytes) 2024-09-14T09:06:58.653 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:58.653 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:58.653 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:58.653 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:58.669 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:06:58.681 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:06:58.682 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:06:58.687 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:58.691+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:58.687 INFO:teuthology.orchestra.run.smithi176.stdout:4431 read 0x9bde268 thru 0x9bdfdd7 (0x1b70 bytes) 2024-09-14T09:06:58.688 INFO:teuthology.orchestra.run.smithi176.stdout:4432 writesame 0x27195ab thru 0x2722e87 data_size 0x1(0x98dd bytes) 2024-09-14T09:06:58.703 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-14T09:06:57.986686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1121072198,"num_objects":1545,"num_object_clones":462,"num_object_copies":3090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1545,"num_whiteouts":558,"num_read":121607,"num_read_kb":637695,"num_write":16009,"num_write_kb":266473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1216,"num_bytes_recovered":72785688,"num_keys_recovered":252,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12544,"ondisk_log_size":12544,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":365320,"kb_used_data":50560,"kb_used_omap":12,"kb_used_meta":314547,"kb_avail":754609400,"statfs":{"total":773094113280,"available":772720025600,"internally_reserved":0,"allocated":51773440,"data_stored":374011439,"data_compressed":11162510,"data_compressed_allocated":31547392,"data_compressed_original":362697138,"omap_allocated":12751,"internal_metadata":322096689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51856032,"num_objects":202,"num_object_clones":58,"num_object_copies":404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":202,"num_whiteouts":70,"num_read":13548,"num_read_kb":63676,"num_write":2454,"num_write_kb":27583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001572"},"pg_stats":[{"pgid":"3.1f","version":"94'1435","reported_seq":4818,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-14T09:06:57.191219+0000","last_change":"2024-09-14T09:06:41.348384+0000","last_active":"2024-09-14T09:06:57.191219+0000","last_peered":"2024-09-14T09:06:57.191219+0000","last_clean":"2024-09-14T09:06:57.191219+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:06:57.191219+0000","last_undegraded":"2024-09-14T09:06:57.191219+0000","last_fullsized":"2024-09-14T09:06:57.191219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1435,"log_dups_size":0,"ondisk_log_size":1435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:47:09.929008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001387512,"stat_sum":{"num_bytes":18655428,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":16,"num_read":10267,"num_read_kb":19840,"num_write":2424,"num_write_kb":6013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"93'252","reported_seq":2959,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:56.055022+0000","last_change":"2024-09-14T09:05:29.039503+0000","last_active":"2024-09-14T09:06:56.055022+0000","last_peered":"2024-09-14T09:06:56.055022+0000","last_clean":"2024-09-14T09:06:56.055022+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:06:56.055022+0000","last_undegraded":"2024-09-14T09:06:56.055022+0000","last_fullsized":"2024-09-14T09:06:56.055022+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:38:41.933374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001657176,"stat_sum":{"num_bytes":34800867,"num_objects":65,"num_object_clones":23,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":25,"num_read":2631,"num_read_kb":25989,"num_write":287,"num_write_kb":10527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"93'416","reported_seq":4043,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-14T09:06:57.164444+0000","last_change":"2024-09-14T09:05:28.835298+0000","last_active":"2024-09-14T09:06:57.164444+0000","last_peered":"2024-09-14T09:06:57.164444+0000","last_clean":"2024-09-14T09:06:57.164444+0000","last_became_active":"2024-09-14T09:05:26.835372+0000","last_became_peered":"2024-09-14T09:05:26.835372+0000","last_unstale":"2024-09-14T09:06:57.164444+0000","last_undegraded":"2024-09-14T09:06:57.164444+0000","last_fullsized":"2024-09-14T09:06:57.164444+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:32:21.193788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014149329999999999,"stat_sum":{"num_bytes":62443453,"num_objects":54,"num_object_clones":15,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":17,"num_read":4241,"num_read_kb":53400,"num_write":553,"num_write_kb":10023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"94'579","reported_seq":3619,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-14T09:06:57.188652+0000","last_change":"2024-09-14T09:06:44.867658+0000","last_active":"2024-09-14T09:06:57.188652+0000","last_peered":"2024-09-14T09:06:57.188652+0000","last_clean":"2024-09-14T09:06:57.188652+0000","last_became_active":"2024-09-14T09:06:44.862945+0000","last_became_peered":"2024-09-14T09:06:44.862945+0000","last_unstale":"2024-09-14T09:06:57.188652+0000","last_undegraded":"2024-09-14T09:06:57.188652+0000","last_fullsized":"2024-09-14T09:06:57.188652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:59:46.026562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014661419999999999,"stat_sum":{"num_bytes":42433077,"num_objects":53,"num_object_clones":18,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":20,"num_read":5800,"num_read_kb":23707,"num_write":794,"num_write_kb":8444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"94'465","reported_seq":3560,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-14T09:06:57.228714+0000","last_change":"2024-09-14T09:06:41.347478+0000","last_active":"2024-09-14T09:06:57.228714+0000","last_peered":"2024-09-14T09:06:57.228714+0000","last_clean":"2024-09-14T09:06:57.228714+0000","last_became_active":"2024-09-14T09:06:41.346397+0000","last_became_peered":"2024-09-14T09:06:41.346397+0000","last_unstale":"2024-09-14T09:06:57.228714+0000","last_undegraded":"2024-09-14T09:06:57.228714+0000","last_fullsized":"2024-09-14T09:06:57.228714+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:18:13.759057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060373399999999998,"stat_sum":{"num_bytes":34171832,"num_objects":53,"num_object_clones":18,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":16,"num_read":4001,"num_read_kb":18168,"num_write":655,"num_write_kb":14233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":6073202,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"93'522","reported_seq":4059,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:56.138445+0000","last_change":"2024-09-14T09:06:44.872988+0000","last_active":"2024-09-14T09:06:56.138445+0000","last_peered":"2024-09-14T09:06:56.138445+0000","last_clean":"2024-09-14T09:06:56.138445+0000","last_became_active":"2024-09-14T09:06:44.869103+0000","last_became_peered":"2024-09-14T09:06:44.869103+0000","last_unstale":"2024-09-14T09:06:56.138445+0000","last_undegraded":"2024-09-14T09:06:56.138445+0000","last_fullsized":"2024-09-14T09:06:56.138445+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:13:31.616157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020845550000000001,"stat_sum":{"num_bytes":36228296,"num_objects":45,"num_object_clones":13,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":15,"num_read":5030,"num_read_kb":23348,"num_write":816,"num_write_kb":10907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"93'233","reported_seq":2224,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:56.145048+0000","last_change":"2024-09-14T09:05:28.822565+0000","last_active":"2024-09-14T09:06:56.145048+0000","last_peered":"2024-09-14T09:06:56.145048+0000","last_clean":"2024-09-14T09:06:56.145048+0000","last_became_active":"2024-09-14T09:05:26.816132+0000","last_became_peered":"2024-09-14T09:05:26.816132+0000","last_unstale":"2024-09-14T09:06:56.145048+0000","last_undegraded":"2024-09-14T09:06:56.145048+0000","last_fullsized":"2024-09-14T09:06:56.145048+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:32:28.659509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092420299999999996,"stat_sum":{"num_bytes":23450810,"num_objects":40,"num_object_clones":13,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":15,"num_read":1900,"num_read_kb":6826,"num_write":270,"num_write_kb":5329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"93'321","reported_seq":3625,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:55.840124+0000","last_change":"2024-09-14T09:06:41.342724+0000","last_active":"2024-09-14T09:06:55.840124+0000","last_peered":"2024-09-14T09:06:55.840124+0000","last_clean":"2024-09-14T09:06:55.840124+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:06:55.840124+0000","last_undegraded":"2024-09-14T09:06:55.840124+0000","last_fullsized":"2024-09-14T09:06:55.840124+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:13:46.847959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081579899999999997,"stat_sum":{"num_bytes":51815986,"num_objects":58,"num_object_clones":19,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":18,"num_read":3895,"num_read_kb":9805,"num_write":310,"num_write_kb":4471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"94'666","reported_seq":5064,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-14T09:06:57.250515+0000","last_change":"2024-09-14T09:05:28.839928+0000","last_active":"2024-09-14T09:06:57.250515+0000","last_peered":"2024-09-14T09:06:57.250515+0000","last_clean":"2024-09-14T09:06:57.250515+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:06:57.250515+0000","last_undegraded":"2024-09-14T09:06:57.250515+0000","last_fullsized":"2024-09-14T09:06:57.250515+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:46:59.106801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014343089999999999,"stat_sum":{"num_bytes":39364207,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":13,"num_read":6773,"num_read_kb":25100,"num_write":948,"num_write_kb":6553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"93'422","reported_seq":2335,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-14T09:06:56.938432+0000","last_change":"2024-09-14T09:05:28.836370+0000","last_active":"2024-09-14T09:06:56.938432+0000","last_peered":"2024-09-14T09:06:56.938432+0000","last_clean":"2024-09-14T09:06:56.938432+0000","last_became_active":"2024-09-14T09:05:26.836744+0000","last_became_peered":"2024-09-14T09:05:26.836744+0000","last_unstale":"2024-09-14T09:06:56.938432+0000","last_undegraded":"2024-09-14T09:06:56.938432+0000","last_fullsized":"2024-09-14T09:06:56.938432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:51.187792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015277190000000001,"stat_sum":{"num_bytes":19074982,"num_objects":46,"num_object_clones":9,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":12,"num_read":3841,"num_read_kb":12719,"num_write":553,"num_write_kb":5695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"93'541","reported_seq":1739,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:55.847984+0000","last_change":"2024-09-14T09:05:28.851587+0000","last_active":"2024-09-14T09:06:55.847984+0000","last_peered":"2024-09-14T09:06:55.847984+0000","last_clean":"2024-09-14T09:06:55.847984+0000","last_became_active":"2024-09-14T09:05:26.840893+0000","last_became_peered":"2024-09-14T09:05:26.840893+0000","last_unstale":"2024-09-14T09:06:55.847984+0000","last_undegraded":"2024-09-14T09:06:55.847984+0000","last_fullsized":"2024-09-14T09:06:55.847984+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:05:10.828259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098594000000000004,"stat_sum":{"num_bytes":10108839,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":3961,"num_read_kb":4460,"num_write":808,"num_write_kb":3633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"93'368","reported_seq":1893,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:56.066834+0000","last_change":"2024-09-14T09:06:44.867267+0000","last_active":"2024-09-14T09:06:56.066834+0000","last_peered":"2024-09-14T09:06:56.066834+0000","last_clean":"2024-09-14T09:06:56.066834+0000","last_became_active":"2024-09-14T09:06:44.865413+0000","last_became_peered":"2024-09-14T09:06:44.865413+0000","last_unstale":"2024-09-14T09:06:56.066834+0000","last_undegraded":"2024-09-14T09:06:56.066834+0000","last_fullsized":"2024-09-14T09:06:56.066834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:41:41.223281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014079679999999999,"stat_sum":{"num_bytes":22692095,"num_objects":50,"num_object_clones":16,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":20,"num_read":2395,"num_read_kb":4594,"num_write":411,"num_write_kb":1598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":253148,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":164,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-14T09:06:57.030603+0000","last_change":"2024-09-14T09:05:19.793674+0000","last_active":"2024-09-14T09:06:57.030603+0000","last_peered":"2024-09-14T09:06:57.030603+0000","last_clean":"2024-09-14T09:06:57.030603+0000","last_became_active":"2024-09-14T09:05:17.284977+0000","last_became_peered":"2024-09-14T09:05:17.284977+0000","last_unstale":"2024-09-14T09:06:57.030603+0000","last_undegraded":"2024-09-14T09:06:57.030603+0000","last_fullsized":"2024-09-14T09:06:57.030603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:30:38.832027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"93'353","reported_seq":4318,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-14T09:06:57.004766+0000","last_change":"2024-09-14T09:05:28.839614+0000","last_active":"2024-09-14T09:06:57.004766+0000","last_peered":"2024-09-14T09:06:57.004766+0000","last_clean":"2024-09-14T09:06:57.004766+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:06:57.004766+0000","last_undegraded":"2024-09-14T09:06:57.004766+0000","last_fullsized":"2024-09-14T09:06:57.004766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:51:03.903452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014245869999999999,"stat_sum":{"num_bytes":42518450,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":17,"num_read":5820,"num_read_kb":40808,"num_write":368,"num_write_kb":6964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:54.935758+0000","last_change":"2024-09-14T09:05:19.314029+0000","last_active":"2024-09-14T09:06:54.935758+0000","last_peered":"2024-09-14T09:06:54.935758+0000","last_clean":"2024-09-14T09:06:54.935758+0000","last_became_active":"2024-09-14T09:05:17.282890+0000","last_became_peered":"2024-09-14T09:05:17.282890+0000","last_unstale":"2024-09-14T09:06:54.935758+0000","last_undegraded":"2024-09-14T09:06:54.935758+0000","last_fullsized":"2024-09-14T09:06:54.935758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:25.304105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"93'397","reported_seq":2672,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:55.649417+0000","last_change":"2024-09-14T09:05:29.039154+0000","last_active":"2024-09-14T09:06:55.649417+0000","last_peered":"2024-09-14T09:06:55.649417+0000","last_clean":"2024-09-14T09:06:55.649417+0000","last_became_active":"2024-09-14T09:05:26.829855+0000","last_became_peered":"2024-09-14T09:05:26.829855+0000","last_unstale":"2024-09-14T09:06:55.649417+0000","last_undegraded":"2024-09-14T09:06:55.649417+0000","last_fullsized":"2024-09-14T09:06:55.649417+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:44:49.188614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019213749999999999,"stat_sum":{"num_bytes":38042154,"num_objects":30,"num_object_clones":10,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":3535,"num_read_kb":13187,"num_write":519,"num_write_kb":4728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:54.991294+0000","last_change":"2024-09-14T09:05:19.794165+0000","last_active":"2024-09-14T09:06:54.991294+0000","last_peered":"2024-09-14T09:06:54.991294+0000","last_clean":"2024-09-14T09:06:54.991294+0000","last_became_active":"2024-09-14T09:05:17.295864+0000","last_became_peered":"2024-09-14T09:05:17.295864+0000","last_unstale":"2024-09-14T09:06:54.991294+0000","last_undegraded":"2024-09-14T09:06:54.991294+0000","last_fullsized":"2024-09-14T09:06:54.991294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:53.901865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011399719999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"93'362","reported_seq":3198,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:55.886226+0000","last_change":"2024-09-14T09:06:35.286509+0000","last_active":"2024-09-14T09:06:55.886226+0000","last_peered":"2024-09-14T09:06:55.886226+0000","last_clean":"2024-09-14T09:06:55.886226+0000","last_became_active":"2024-09-14T09:06:35.282469+0000","last_became_peered":"2024-09-14T09:06:35.282469+0000","last_unstale":"2024-09-14T09:06:55.886226+0000","last_undegraded":"2024-09-14T09:06:55.886226+0000","last_fullsized":"2024-09-14T09:06:55.886226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:29:19.570430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014087959999999999,"stat_sum":{"num_bytes":22984834,"num_objects":59,"num_object_clones":20,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":26,"num_read":4249,"num_read_kb":17537,"num_write":360,"num_write_kb":7063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":1142719,"num_keys_recovered":23,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:54.935783+0000","last_change":"2024-09-14T09:05:19.314129+0000","last_active":"2024-09-14T09:06:54.935783+0000","last_peered":"2024-09-14T09:06:54.935783+0000","last_clean":"2024-09-14T09:06:54.935783+0000","last_became_active":"2024-09-14T09:05:17.295196+0000","last_became_peered":"2024-09-14T09:05:17.295196+0000","last_unstale":"2024-09-14T09:06:54.935783+0000","last_undegraded":"2024-09-14T09:06:54.935783+0000","last_fullsized":"2024-09-14T09:06:54.935783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:35.269115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011613800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"94'461","reported_seq":1664,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-14T09:06:57.077715+0000","last_change":"2024-09-14T09:05:28.835976+0000","last_active":"2024-09-14T09:06:57.077715+0000","last_peered":"2024-09-14T09:06:57.077715+0000","last_clean":"2024-09-14T09:06:57.077715+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:06:57.077715+0000","last_undegraded":"2024-09-14T09:06:57.077715+0000","last_fullsized":"2024-09-14T09:06:57.077715+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:44:28.573808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017480779999999999,"stat_sum":{"num_bytes":57000307,"num_objects":64,"num_object_clones":20,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":27,"num_read":2379,"num_read_kb":13867,"num_write":647,"num_write_kb":11959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"93'412","reported_seq":2274,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:55.927330+0000","last_change":"2024-09-14T09:06:46.882194+0000","last_active":"2024-09-14T09:06:55.927330+0000","last_peered":"2024-09-14T09:06:55.927330+0000","last_clean":"2024-09-14T09:06:55.927330+0000","last_became_active":"2024-09-14T09:06:46.880786+0000","last_became_peered":"2024-09-14T09:06:46.880786+0000","last_unstale":"2024-09-14T09:06:55.927330+0000","last_undegraded":"2024-09-14T09:06:55.927330+0000","last_fullsized":"2024-09-14T09:06:55.927330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:38:24.264269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014128,"stat_sum":{"num_bytes":27728523,"num_objects":47,"num_object_clones":16,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":16,"num_read":3125,"num_read_kb":17258,"num_write":498,"num_write_kb":10712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":166,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-14T09:06:56.938442+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:06:56.938442+0000","last_peered":"2024-09-14T09:06:56.938442+0000","last_clean":"2024-09-14T09:06:56.938442+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:06:56.938442+0000","last_undegraded":"2024-09-14T09:06:56.938442+0000","last_fullsized":"2024-09-14T09:06:56.938442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"93'206","reported_seq":1176,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:55.305996+0000","last_change":"2024-09-14T09:05:28.822791+0000","last_active":"2024-09-14T09:06:55.305996+0000","last_peered":"2024-09-14T09:06:55.305996+0000","last_clean":"2024-09-14T09:06:55.305996+0000","last_became_active":"2024-09-14T09:05:26.817109+0000","last_became_peered":"2024-09-14T09:05:26.817109+0000","last_unstale":"2024-09-14T09:06:55.305996+0000","last_undegraded":"2024-09-14T09:06:55.305996+0000","last_fullsized":"2024-09-14T09:06:55.305996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:57:58.709654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011728439999999999,"stat_sum":{"num_bytes":16627969,"num_objects":48,"num_object_clones":15,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":20,"num_read":1010,"num_read_kb":5934,"num_write":211,"num_write_kb":2747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":164,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-14T09:06:56.952481+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:06:56.952481+0000","last_peered":"2024-09-14T09:06:56.952481+0000","last_clean":"2024-09-14T09:06:56.952481+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:06:56.952481+0000","last_undegraded":"2024-09-14T09:06:56.952481+0000","last_fullsized":"2024-09-14T09:06:56.952481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"94'251","reported_seq":2072,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-14T09:06:56.944078+0000","last_change":"2024-09-14T09:06:45.868939+0000","last_active":"2024-09-14T09:06:56.944078+0000","last_peered":"2024-09-14T09:06:56.944078+0000","last_clean":"2024-09-14T09:06:56.944078+0000","last_became_active":"2024-09-14T09:06:45.868194+0000","last_became_peered":"2024-09-14T09:06:45.868194+0000","last_unstale":"2024-09-14T09:06:56.944078+0000","last_undegraded":"2024-09-14T09:06:56.944078+0000","last_fullsized":"2024-09-14T09:06:56.944078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:16:32.926546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074815999999999999,"stat_sum":{"num_bytes":31180200,"num_objects":41,"num_object_clones":15,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":18,"num_read":1538,"num_read_kb":15769,"num_write":246,"num_write_kb":9271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:54.990518+0000","last_change":"2024-09-14T09:05:19.793999+0000","last_active":"2024-09-14T09:06:54.990518+0000","last_peered":"2024-09-14T09:06:54.990518+0000","last_clean":"2024-09-14T09:06:54.990518+0000","last_became_active":"2024-09-14T09:05:17.284122+0000","last_became_peered":"2024-09-14T09:05:17.284122+0000","last_unstale":"2024-09-14T09:06:54.990518+0000","last_undegraded":"2024-09-14T09:06:54.990518+0000","last_fullsized":"2024-09-14T09:06:54.990518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:12:01.133215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010234269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'231","reported_seq":1322,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:56.125943+0000","last_change":"2024-09-14T09:05:29.038389+0000","last_active":"2024-09-14T09:06:56.125943+0000","last_peered":"2024-09-14T09:06:56.125943+0000","last_clean":"2024-09-14T09:06:56.125943+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:06:56.125943+0000","last_undegraded":"2024-09-14T09:06:56.125943+0000","last_fullsized":"2024-09-14T09:06:56.125943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:18:33.298357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001479076,"stat_sum":{"num_bytes":23655986,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":17,"num_read":1229,"num_read_kb":5951,"num_write":282,"num_write_kb":3953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":65,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:54.932648+0000","last_change":"2024-09-14T09:06:25.227644+0000","last_active":"2024-09-14T09:06:54.932648+0000","last_peered":"2024-09-14T09:06:54.932648+0000","last_clean":"2024-09-14T09:06:54.932648+0000","last_became_active":"2024-09-14T09:06:25.222344+0000","last_became_peered":"2024-09-14T09:06:25.222344+0000","last_unstale":"2024-09-14T09:06:54.932648+0000","last_undegraded":"2024-09-14T09:06:54.932648+0000","last_fullsized":"2024-09-14T09:06:54.932648+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:23:33.414859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":253,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:54.990826+0000","last_change":"2024-09-14T09:05:13.239808+0000","last_active":"2024-09-14T09:06:54.990826+0000","last_peered":"2024-09-14T09:06:54.990826+0000","last_clean":"2024-09-14T09:06:54.990826+0000","last_became_active":"2024-09-14T09:05:13.239064+0000","last_became_peered":"2024-09-14T09:05:13.239064+0000","last_unstale":"2024-09-14T09:06:54.990826+0000","last_undegraded":"2024-09-14T09:06:54.990826+0000","last_fullsized":"2024-09-14T09:06:54.990826+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:40:11.559161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"93'285","reported_seq":3610,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:56.053055+0000","last_change":"2024-09-14T09:05:29.037923+0000","last_active":"2024-09-14T09:06:56.053055+0000","last_peered":"2024-09-14T09:06:56.053055+0000","last_clean":"2024-09-14T09:06:56.053055+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:06:56.053055+0000","last_undegraded":"2024-09-14T09:06:56.053055+0000","last_fullsized":"2024-09-14T09:06:56.053055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:58:18.279469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099199800000000001,"stat_sum":{"num_bytes":44261474,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":16,"num_read":3973,"num_read_kb":30965,"num_write":300,"num_write_kb":8911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"93'228","reported_seq":1818,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:56.065094+0000","last_change":"2024-09-14T09:06:44.868332+0000","last_active":"2024-09-14T09:06:56.065094+0000","last_peered":"2024-09-14T09:06:56.065094+0000","last_clean":"2024-09-14T09:06:56.065094+0000","last_became_active":"2024-09-14T09:06:44.866346+0000","last_became_peered":"2024-09-14T09:06:44.866346+0000","last_unstale":"2024-09-14T09:06:56.065094+0000","last_undegraded":"2024-09-14T09:06:56.065094+0000","last_fullsized":"2024-09-14T09:06:56.065094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:02:14.264461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001266028,"stat_sum":{"num_bytes":28771029,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":15,"num_read":1325,"num_read_kb":8427,"num_write":212,"num_write_kb":14886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":4589732,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"94'308","reported_seq":3708,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-14T09:06:57.147010+0000","last_change":"2024-09-14T09:06:45.870752+0000","last_active":"2024-09-14T09:06:57.147010+0000","last_peered":"2024-09-14T09:06:57.147010+0000","last_clean":"2024-09-14T09:06:57.147010+0000","last_became_active":"2024-09-14T09:06:45.869266+0000","last_became_peered":"2024-09-14T09:06:45.869266+0000","last_unstale":"2024-09-14T09:06:57.147010+0000","last_undegraded":"2024-09-14T09:06:57.147010+0000","last_fullsized":"2024-09-14T09:06:57.147010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:12:01.815512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185731,"stat_sum":{"num_bytes":53985411,"num_objects":55,"num_object_clones":16,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":21,"num_read":3716,"num_read_kb":35684,"num_write":341,"num_write_kb":18236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"93'341","reported_seq":8248,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:55.797323+0000","last_change":"2024-09-14T09:06:02.837388+0000","last_active":"2024-09-14T09:06:55.797323+0000","last_peered":"2024-09-14T09:06:55.797323+0000","last_clean":"2024-09-14T09:06:55.797323+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:06:55.797323+0000","last_undegraded":"2024-09-14T09:06:55.797323+0000","last_fullsized":"2024-09-14T09:06:55.797323+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:10:34.292088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001585295,"stat_sum":{"num_bytes":40618849,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":16,"num_read":9079,"num_read_kb":55101,"num_write":338,"num_write_kb":8281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"93'258","reported_seq":1530,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-14T09:06:57.170114+0000","last_change":"2024-09-14T09:05:28.834647+0000","last_active":"2024-09-14T09:06:57.170114+0000","last_peered":"2024-09-14T09:06:57.170114+0000","last_clean":"2024-09-14T09:06:57.170114+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:06:57.170114+0000","last_undegraded":"2024-09-14T09:06:57.170114+0000","last_fullsized":"2024-09-14T09:06:57.170114+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:11:47.060089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244290000000001,"stat_sum":{"num_bytes":30291007,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":17,"num_read":1650,"num_read_kb":8329,"num_write":275,"num_write_kb":8855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"93'284","reported_seq":3425,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:56.012833+0000","last_change":"2024-09-14T09:05:28.851561+0000","last_active":"2024-09-14T09:06:56.012833+0000","last_peered":"2024-09-14T09:06:56.012833+0000","last_clean":"2024-09-14T09:06:56.012833+0000","last_became_active":"2024-09-14T09:05:26.843780+0000","last_became_peered":"2024-09-14T09:05:26.843780+0000","last_unstale":"2024-09-14T09:06:56.012833+0000","last_undegraded":"2024-09-14T09:06:56.012833+0000","last_fullsized":"2024-09-14T09:06:56.012833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:22:01.936962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072987200000000001,"stat_sum":{"num_bytes":30057193,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":14,"num_read":3846,"num_read_kb":27227,"num_write":333,"num_write_kb":14779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"93'208","reported_seq":1659,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:55.838010+0000","last_change":"2024-09-14T09:06:45.872735+0000","last_active":"2024-09-14T09:06:55.838010+0000","last_peered":"2024-09-14T09:06:55.838010+0000","last_clean":"2024-09-14T09:06:55.838010+0000","last_became_active":"2024-09-14T09:06:45.868989+0000","last_became_peered":"2024-09-14T09:06:45.868989+0000","last_unstale":"2024-09-14T09:06:55.838010+0000","last_undegraded":"2024-09-14T09:06:55.838010+0000","last_fullsized":"2024-09-14T09:06:55.838010+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:06:50.411134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057596100000000005,"stat_sum":{"num_bytes":36900119,"num_objects":43,"num_object_clones":12,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":16,"num_read":1151,"num_read_kb":11885,"num_write":193,"num_write_kb":5532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":3897102,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"93'282","reported_seq":1600,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:56.138615+0000","last_change":"2024-09-14T09:06:46.884414+0000","last_active":"2024-09-14T09:06:56.138615+0000","last_peered":"2024-09-14T09:06:56.138615+0000","last_clean":"2024-09-14T09:06:56.138615+0000","last_became_active":"2024-09-14T09:06:46.881561+0000","last_became_peered":"2024-09-14T09:06:46.881561+0000","last_unstale":"2024-09-14T09:06:56.138615+0000","last_undegraded":"2024-09-14T09:06:56.138615+0000","last_fullsized":"2024-09-14T09:06:56.138615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:25:19.792796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000909463,"stat_sum":{"num_bytes":31434996,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":17,"num_read":1546,"num_read_kb":8155,"num_write":300,"num_write_kb":9034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":3341023,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"94'301","reported_seq":4543,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-14T09:06:57.030854+0000","last_change":"2024-09-14T09:06:00.166357+0000","last_active":"2024-09-14T09:06:57.030854+0000","last_peered":"2024-09-14T09:06:57.030854+0000","last_clean":"2024-09-14T09:06:57.030854+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:06:57.030854+0000","last_undegraded":"2024-09-14T09:06:57.030854+0000","last_fullsized":"2024-09-14T09:06:57.030854+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:25.618645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011381609999999999,"stat_sum":{"num_bytes":86866987,"num_objects":50,"num_object_clones":14,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":18,"num_read":4302,"num_read_kb":52227,"num_write":393,"num_write_kb":15898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"93'370","reported_seq":2819,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:55.845989+0000","last_change":"2024-09-14T09:06:46.882214+0000","last_active":"2024-09-14T09:06:55.845989+0000","last_peered":"2024-09-14T09:06:55.845989+0000","last_clean":"2024-09-14T09:06:55.845989+0000","last_became_active":"2024-09-14T09:06:46.880416+0000","last_became_peered":"2024-09-14T09:06:46.880416+0000","last_unstale":"2024-09-14T09:06:55.845989+0000","last_undegraded":"2024-09-14T09:06:55.845989+0000","last_fullsized":"2024-09-14T09:06:55.845989+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:38.926784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092949299999999999,"stat_sum":{"num_bytes":7908592,"num_objects":48,"num_object_clones":15,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":20,"num_read":3629,"num_read_kb":4668,"num_write":473,"num_write_kb":3148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":30867,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"93'402","reported_seq":2868,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-14T09:06:56.943792+0000","last_change":"2024-09-14T09:05:28.839774+0000","last_active":"2024-09-14T09:06:56.943792+0000","last_peered":"2024-09-14T09:06:56.943792+0000","last_clean":"2024-09-14T09:06:56.943792+0000","last_became_active":"2024-09-14T09:05:26.826600+0000","last_became_peered":"2024-09-14T09:05:26.826600+0000","last_unstale":"2024-09-14T09:06:56.943792+0000","last_undegraded":"2024-09-14T09:06:56.943792+0000","last_fullsized":"2024-09-14T09:06:56.943792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:42:39.166603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018357099999999999,"stat_sum":{"num_bytes":47904210,"num_objects":51,"num_object_clones":17,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":21,"num_read":3898,"num_read_kb":8738,"num_write":497,"num_write_kb":4158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"93'309","reported_seq":4889,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-14T09:06:55.935801+0000","last_change":"2024-09-14T09:06:44.869874+0000","last_active":"2024-09-14T09:06:55.935801+0000","last_peered":"2024-09-14T09:06:55.935801+0000","last_clean":"2024-09-14T09:06:55.935801+0000","last_became_active":"2024-09-14T09:06:44.865881+0000","last_became_peered":"2024-09-14T09:06:44.865881+0000","last_unstale":"2024-09-14T09:06:55.935801+0000","last_undegraded":"2024-09-14T09:06:55.935801+0000","last_fullsized":"2024-09-14T09:06:55.935801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:19:50.326359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061978900000000004,"stat_sum":{"num_bytes":26503649,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":5806,"num_read_kb":27967,"num_write":291,"num_write_kb":8302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":6761444,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1120481811,"num_objects":1541,"num_object_clones":462,"num_object_copies":3082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1541,"num_whiteouts":558,"num_read":121541,"num_read_kb":637640,"num_write":15906,"num_write_kb":264843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1215,"num_bytes_recovered":72785688,"num_keys_recovered":252,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":43565056,"data_stored":371349921,"data_compressed":11166192,"data_compressed_allocated":31674368,"data_compressed_original":363780369,"omap_allocated":0,"internal_metadata":0},"log_size":12469,"ondisk_log_size":12469,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12424,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40124,"kb_used_data":4776,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331716,"statfs":{"total":96636764160,"available":96595677184,"internally_reserved":0,"allocated":4890624,"data_stored":35406425,"data_compressed":938700,"data_compressed_allocated":2949120,"data_compressed_original":34385688,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509465,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74708,"kb_used_data":7428,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94297132,"statfs":{"total":96636764160,"available":96560263168,"internally_reserved":0,"allocated":7606272,"data_stored":56138022,"data_compressed":1576891,"data_compressed_allocated":4730880,"data_compressed_original":54558572,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509466,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41540,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330300,"statfs":{"total":96636764160,"available":96594227200,"internally_reserved":0,"allocated":2080768,"data_stored":8063698,"data_compressed":290724,"data_compressed_allocated":749568,"data_compressed_original":7422752,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34768,"kb_used_data":7696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337072,"statfs":{"total":96636764160,"available":96601161728,"internally_reserved":0,"allocated":7880704,"data_stored":59594548,"data_compressed":1750601,"data_compressed_allocated":4829184,"data_compressed_original":57899881,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542168,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33964,"kb_used_data":6936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337876,"statfs":{"total":96636764160,"available":96601985024,"internally_reserved":0,"allocated":7102464,"data_stored":50879552,"data_compressed":1325237,"data_compressed_allocated":4251648,"data_compressed_original":49138366,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542169,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49628,"kb_used_data":10120,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322212,"statfs":{"total":96636764160,"available":96585945088,"internally_reserved":0,"allocated":10362880,"data_stored":82622406,"data_compressed":2454950,"data_compressed_allocated":7069696,"data_compressed_original":80696385,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542169,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46404,"kb_used_data":6896,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325436,"statfs":{"total":96636764160,"available":96589246464,"internally_reserved":0,"allocated":7061504,"data_stored":52086510,"data_compressed":1722115,"data_compressed_allocated":4304896,"data_compressed_original":50548355,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542170,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44184,"kb_used_data":4676,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327656,"statfs":{"total":96636764160,"available":96591519744,"internally_reserved":0,"allocated":4788224,"data_stored":29220278,"data_compressed":1103292,"data_compressed_allocated":2662400,"data_compressed_original":28047139,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6758400,"data_stored":58524076,"data_compressed":1744389,"data_compressed_allocated":4792320,"data_compressed_original":57326441,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3919872,"data_stored":29919877,"data_compressed":1111326,"data_compressed_allocated":2760704,"data_compressed_original":29167395,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":974848,"data_stored":7555188,"data_compressed":290724,"data_compressed_allocated":749568,"data_compressed_original":7422752,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6213632,"data_stored":52812231,"data_compressed":1730187,"data_compressed_allocated":4407296,"data_compressed_original":51678851,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":9428992,"data_stored":82200152,"data_compressed":2454950,"data_compressed_allocated":7069696,"data_compressed_original":80675744,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6496256,"data_stored":55629493,"data_compressed":1576891,"data_compressed_allocated":4730880,"data_compressed_original":54558572,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3784704,"data_stored":34897915,"data_compressed":938700,"data_compressed_allocated":2949120,"data_compressed_original":34385688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5988352,"data_stored":49810989,"data_compressed":1319025,"data_compressed_allocated":4214784,"data_compressed_original":48564926,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:06:58.756 INFO:teuthology.orchestra.run.smithi176.stdout:4433 write 0x2a87729 thru 0x2a955b8 (0xde90 bytes) 2024-09-14T09:06:58.765 INFO:teuthology.orchestra.run.smithi176.stdout:4434 writesame 0xe30595c thru 0xe3078f6 data_size 0x9(0x1f9b bytes) 2024-09-14T09:06:58.769 INFO:teuthology.orchestra.run.smithi176.stdout:4435 read 0x59fcff8 thru 0x5a089cf (0xb9d8 bytes) 2024-09-14T09:06:58.770 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:58.770 INFO:teuthology.orchestra.run.smithi176.stdout:4437 read 0xc985060 thru 0xc98818a (0x312b bytes) 2024-09-14T09:06:58.787 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:58.787+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:58.793 INFO:teuthology.orchestra.run.smithi176.stdout:4438 trunc from 0xe5ed9d0 to 0xb9a37b 2024-09-14T09:06:58.821 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:58.821 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:58.821 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:58.821 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:58.821 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:58.835 INFO:teuthology.orchestra.run.smithi176.stdout:4439 trunc from 0xb9a37b to 0x62ad708 2024-09-14T09:06:58.846 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:06:58.855 INFO:teuthology.orchestra.run.smithi176.stdout:4440 trunc from 0x62ad708 to 0xce44b59 2024-09-14T09:06:58.869 INFO:teuthology.orchestra.run.smithi176.stdout:4441 write 0x4bd611b thru 0x4bddae6 (0x79cc bytes) 2024-09-14T09:06:58.875 INFO:teuthology.orchestra.run.smithi176.stdout:4444 write 0x6f804c5 thru 0x6f8dbc6 (0xd702 bytes) 2024-09-14T09:06:58.884 INFO:teuthology.orchestra.run.smithi176.stdout:4446 read 0x4b7468c thru 0x4b7d7a1 (0x9116 bytes) 2024-09-14T09:06:58.884 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:58.884 INFO:teuthology.orchestra.run.smithi176.stdout:4449 write 0xa255180 thru 0xa25e49e (0x931f bytes) 2024-09-14T09:06:58.889 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:06:58.891+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:06:58.895 INFO:teuthology.orchestra.run.smithi176.stdout:4451 punch from 0x4119f17 to 0x411ca7b, (0x2b64 bytes) 2024-09-14T09:06:58.896 INFO:teuthology.orchestra.run.smithi176.stdout:4452 write 0x1abe7df thru 0x1acd8fd (0xf11f bytes) 2024-09-14T09:06:58.904 INFO:teuthology.orchestra.run.smithi176.stdout:4453 punch from 0x17af739 to 0x17b14eb, (0x1db2 bytes) 2024-09-14T09:06:58.909 INFO:teuthology.orchestra.run.smithi176.stdout:4455 trunc from 0xce44b59 to 0x9e41856 2024-09-14T09:06:58.925 INFO:teuthology.orchestra.run.smithi176.stdout:4456 trunc from 0x9e41856 to 0x9c7b573 2024-09-14T09:06:58.942 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:58.942 INFO:teuthology.orchestra.run.smithi176.stdout:4458 read 0x57245ad thru 0x572edc5 (0xa819 bytes) 2024-09-14T09:06:58.944 INFO:teuthology.orchestra.run.smithi176.stdout:4459 write 0x858ab31 thru 0x8596396 (0xb866 bytes) 2024-09-14T09:06:58.957 INFO:teuthology.orchestra.run.smithi176.stdout:4460 write 0xff8209 thru 0x1002eab (0xaca3 bytes) 2024-09-14T09:06:58.962 INFO:teuthology.orchestra.run.smithi176.stdout:4461 punch from 0x88b725 to 0x89384c, (0x8127 bytes) 2024-09-14T09:06:58.962 INFO:teuthology.orchestra.run.smithi176.stdout:4462 write 0x32d6694 thru 0x32e1aec (0xb459 bytes) 2024-09-14T09:06:58.983 INFO:teuthology.orchestra.run.smithi176.stdout:4463 trunc from 0x9c7b573 to 0xea2978 2024-09-14T09:06:58.989 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:58.990 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:58.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:58.990 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:58.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.990 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18383 3.9 3:9252f7c9:::rbd_data.115e3abd3b2c.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:06:58.990 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:58.990 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.831249344, 2024-09-14T09:06:58.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056433830999999997, 2024-09-14T09:06:58.990 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.990 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.990 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:58.990 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:58.991 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:58.991 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:58.991 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18383 2024-09-14T09:06:58.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.991 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:58.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:06:58.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158423+0000", 2024-09-14T09:06:58.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T09:06:58.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158429+0000", 2024-09-14T09:06:58.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0179999999999998e-06 2024-09-14T09:06:58.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158440+0000", 2024-09-14T09:06:58.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0769e-05 2024-09-14T09:06:58.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158445+0000", 2024-09-14T09:06:58.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5840000000000002e-06 2024-09-14T09:06:58.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158495+0000", 2024-09-14T09:06:58.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9129999999999999e-05 2024-09-14T09:06:58.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158598+0000", 2024-09-14T09:06:58.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102938 2024-09-14T09:06:58.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:58.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.161048+0000", 2024-09-14T09:06:58.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002450336 2024-09-14T09:06:58.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:58.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214605+0000", 2024-09-14T09:06:58.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053557526000000001 2024-09-14T09:06:58.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:58.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214634+0000", 2024-09-14T09:06:58.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8483000000000002e-05 2024-09-14T09:06:58.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:58.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214731+0000", 2024-09-14T09:06:58.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7026000000000002e-05 2024-09-14T09:06:58.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:58.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214856+0000", 2024-09-14T09:06:58.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012474799999999999 2024-09-14T09:06:58.996 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.996 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:58.996 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:58.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.996 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18383 3.9 e38/34 ondisk, result = 0)", 2024-09-14T09:06:58.996 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:58.996 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.820932862999996, 2024-09-14T09:06:58.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045880575999999999, 2024-09-14T09:06:58.996 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:58.996 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:58.996 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:58.996 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:58.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:58.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:58.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:58.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:06:58.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:58.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:58.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168739+0000", 2024-09-14T09:06:58.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3900000000000004e-07 2024-09-14T09:06:58.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:58.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168744+0000", 2024-09-14T09:06:58.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5929999999999999e-06 2024-09-14T09:06:58.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:58.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168754+0000", 2024-09-14T09:06:58.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0686999999999999e-05 2024-09-14T09:06:58.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:58.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168761+0000", 2024-09-14T09:06:58.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6340000000000003e-06 2024-09-14T09:06:58.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:58.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214578+0000", 2024-09-14T09:06:58.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045817443999999999 2024-09-14T09:06:58.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:58.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:58.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214596+0000", 2024-09-14T09:06:58.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7159e-05 2024-09-14T09:06:58.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:58.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214619+0000", 2024-09-14T09:06:59.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.332e-05 2024-09-14T09:06:59.000 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.000 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.000 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.000 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19152 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14159058~8940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:06:59.000 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:59.000 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 74.667020406999995, 2024-09-14T09:06:59.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12891994500000001, 2024-09-14T09:06:59.000 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.000 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.001 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:59.001 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.001 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.001 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.001 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19152 2024-09-14T09:06:59.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.001 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:59.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:06:59.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322653+0000", 2024-09-14T09:06:59.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3750000000000001e-06 2024-09-14T09:06:59.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329647+0000", 2024-09-14T09:06:59.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069941960000000003 2024-09-14T09:06:59.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329673+0000", 2024-09-14T09:06:59.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5760000000000001e-05 2024-09-14T09:06:59.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329684+0000", 2024-09-14T09:06:59.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0474e-05 2024-09-14T09:06:59.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329776+0000", 2024-09-14T09:06:59.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2467999999999996e-05 2024-09-14T09:06:59.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329987+0000", 2024-09-14T09:06:59.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021133 2024-09-14T09:06:59.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:59.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.331303+0000", 2024-09-14T09:06:59.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001315629 2024-09-14T09:06:59.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:59.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.411090+0000", 2024-09-14T09:06:59.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079787477999999995 2024-09-14T09:06:59.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:59.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451463+0000", 2024-09-14T09:06:59.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040372312 2024-09-14T09:06:59.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:59.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451523+0000", 2024-09-14T09:06:59.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0420000000000001e-05 2024-09-14T09:06:59.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451571+0000", 2024-09-14T09:06:59.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7503000000000002e-05 2024-09-14T09:06:59.006 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.006 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.006 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.006 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20926 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:59.006 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:59.006 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.317688407000006, 2024-09-14T09:06:59.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018572207, 2024-09-14T09:06:59.006 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.006 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.006 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.006 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.006 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.007 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.007 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20926 2024-09-14T09:06:59.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.007 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:59.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:59.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:06:59.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:06:59.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671990+0000", 2024-09-14T09:06:59.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5769999999999999e-06 2024-09-14T09:06:59.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672001+0000", 2024-09-14T09:06:59.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1293e-05 2024-09-14T09:06:59.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672008+0000", 2024-09-14T09:06:59.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9099999999999999e-06 2024-09-14T09:06:59.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689147+0000", 2024-09-14T09:06:59.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017139060000000001 2024-09-14T09:06:59.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689324+0000", 2024-09-14T09:06:59.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017640200000000001 2024-09-14T09:06:59.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690555+0000", 2024-09-14T09:06:59.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001231212 2024-09-14T09:06:59.010 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.010 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.010 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.011 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20927 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2621440~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:59.011 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:59.011 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.317651440999995, 2024-09-14T09:06:59.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020002227000000001, 2024-09-14T09:06:59.011 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.011 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.011 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.011 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20927 2024-09-14T09:06:59.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.011 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:59.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:59.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:06:59.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9699999999999995e-07 2024-09-14T09:06:59.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672024+0000", 2024-09-14T09:06:59.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:06:59.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672036+0000", 2024-09-14T09:06:59.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.119e-05 2024-09-14T09:06:59.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672044+0000", 2024-09-14T09:06:59.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8370000000000003e-06 2024-09-14T09:06:59.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690602+0000", 2024-09-14T09:06:59.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018557648 2024-09-14T09:06:59.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690805+0000", 2024-09-14T09:06:59.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020282899999999999 2024-09-14T09:06:59.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692022+0000", 2024-09-14T09:06:59.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012170429999999999 2024-09-14T09:06:59.014 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.015 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.015 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.015 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20928 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 917504~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:59.015 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:59.015 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.317615301000004, 2024-09-14T09:06:59.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021623738, 2024-09-14T09:06:59.015 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.015 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.015 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.015 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.015 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.015 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.015 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20928 2024-09-14T09:06:59.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.016 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:59.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:06:59.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672057+0000", 2024-09-14T09:06:59.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:06:59.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672059+0000", 2024-09-14T09:06:59.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9270000000000001e-06 2024-09-14T09:06:59.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672070+0000", 2024-09-14T09:06:59.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0773e-05 2024-09-14T09:06:59.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672077+0000", 2024-09-14T09:06:59.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9600000000000003e-06 2024-09-14T09:06:59.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692154+0000", 2024-09-14T09:06:59.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020076482 2024-09-14T09:06:59.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692419+0000", 2024-09-14T09:06:59.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026517400000000002 2024-09-14T09:06:59.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693680+0000", 2024-09-14T09:06:59.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012607390000000001 2024-09-14T09:06:59.019 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.019 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.019 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.019 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20929 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2752512~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:59.019 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:59.019 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.317582259000005, 2024-09-14T09:06:59.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022898728, 2024-09-14T09:06:59.019 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.019 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.020 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.020 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.020 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.020 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.020 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20929 2024-09-14T09:06:59.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.020 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:59.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.021 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:06:59.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.021 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.021 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672090+0000", 2024-09-14T09:06:59.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:06:59.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.021 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.021 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672092+0000", 2024-09-14T09:06:59.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8940000000000002e-06 2024-09-14T09:06:59.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.021 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672103+0000", 2024-09-14T09:06:59.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0815999999999999e-05 2024-09-14T09:06:59.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672110+0000", 2024-09-14T09:06:59.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9469999999999997e-06 2024-09-14T09:06:59.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693737+0000", 2024-09-14T09:06:59.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021626467999999999 2024-09-14T09:06:59.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693962+0000", 2024-09-14T09:06:59.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225474 2024-09-14T09:06:59.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.694988+0000", 2024-09-14T09:06:59.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001025469 2024-09-14T09:06:59.023 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.023 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.023 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.023 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20930 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:59.023 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:59.023 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.317549165000003, 2024-09-14T09:06:59.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024303286, 2024-09-14T09:06:59.024 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.024 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.024 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.024 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.024 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.024 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.024 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20930 2024-09-14T09:06:59.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.024 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:59.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:06:59.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672123+0000", 2024-09-14T09:06:59.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:06:59.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672125+0000", 2024-09-14T09:06:59.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7300000000000001e-06 2024-09-14T09:06:59.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672134+0000", 2024-09-14T09:06:59.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7930000000000003e-06 2024-09-14T09:06:59.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672141+0000", 2024-09-14T09:06:59.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9029999999999998e-06 2024-09-14T09:06:59.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695087+0000", 2024-09-14T09:06:59.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022946013000000001 2024-09-14T09:06:59.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695350+0000", 2024-09-14T09:06:59.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026266000000000001 2024-09-14T09:06:59.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696425+0000", 2024-09-14T09:06:59.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001075527 2024-09-14T09:06:59.028 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.028 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.028 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.028 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20931 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:59.028 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:59.028 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.317515936000007, 2024-09-14T09:06:59.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025331519, 2024-09-14T09:06:59.029 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.029 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.029 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.029 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.029 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.029 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.029 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20931 2024-09-14T09:06:59.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.029 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:59.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:06:59.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672156+0000", 2024-09-14T09:06:59.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4300000000000002e-07 2024-09-14T09:06:59.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672159+0000", 2024-09-14T09:06:59.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T09:06:59.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672170+0000", 2024-09-14T09:06:59.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1192999999999999e-05 2024-09-14T09:06:59.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672177+0000", 2024-09-14T09:06:59.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9059999999999996e-06 2024-09-14T09:06:59.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696532+0000", 2024-09-14T09:06:59.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024355291000000001 2024-09-14T09:06:59.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696755+0000", 2024-09-14T09:06:59.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022297699999999999 2024-09-14T09:06:59.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697487+0000", 2024-09-14T09:06:59.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00073171499999999997 2024-09-14T09:06:59.033 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.033 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.033 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.033 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20932 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1179648~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:59.033 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:59.033 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.317482493, 2024-09-14T09:06:59.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026104083, 2024-09-14T09:06:59.034 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.034 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.034 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.034 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.034 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.034 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.034 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20932 2024-09-14T09:06:59.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.034 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.034 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:59.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:59.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:06:59.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0699999999999996e-07 2024-09-14T09:06:59.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672192+0000", 2024-09-14T09:06:59.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7700000000000002e-06 2024-09-14T09:06:59.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672203+0000", 2024-09-14T09:06:59.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0956e-05 2024-09-14T09:06:59.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672211+0000", 2024-09-14T09:06:59.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3769999999999996e-06 2024-09-14T09:06:59.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697530+0000", 2024-09-14T09:06:59.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025318411999999998 2024-09-14T09:06:59.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697640+0000", 2024-09-14T09:06:59.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011045299999999999 2024-09-14T09:06:59.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698293+0000", 2024-09-14T09:06:59.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065240800000000005 2024-09-14T09:06:59.038 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.038 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.038 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.038 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20933 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3014656~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:59.038 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:59.038 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.317448290000002, 2024-09-14T09:06:59.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026558875999999999, 2024-09-14T09:06:59.039 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.039 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.039 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.039 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.039 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.039 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.039 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20933 2024-09-14T09:06:59.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.039 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:59.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:06:59.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672224+0000", 2024-09-14T09:06:59.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2399999999999997e-07 2024-09-14T09:06:59.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672234+0000", 2024-09-14T09:06:59.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0793e-05 2024-09-14T09:06:59.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.041 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.041 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672246+0000", 2024-09-14T09:06:59.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659e-05 2024-09-14T09:06:59.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.041 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.041 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672255+0000", 2024-09-14T09:06:59.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5730000000000008e-06 2024-09-14T09:06:59.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.041 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698326+0000", 2024-09-14T09:06:59.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026071414000000001 2024-09-14T09:06:59.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698374+0000", 2024-09-14T09:06:59.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8032999999999999e-05 2024-09-14T09:06:59.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698782+0000", 2024-09-14T09:06:59.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040768 2024-09-14T09:06:59.043 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.043 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.043 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.043 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20934 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1310720~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:59.043 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:59.043 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.317406414000004, 2024-09-14T09:06:59.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027292067999999999, 2024-09-14T09:06:59.043 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.043 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.043 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.044 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.044 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.044 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.044 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20934 2024-09-14T09:06:59.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.044 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:59.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:06:59.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672266+0000", 2024-09-14T09:06:59.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6299999999999993e-07 2024-09-14T09:06:59.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672269+0000", 2024-09-14T09:06:59.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8569999999999999e-06 2024-09-14T09:06:59.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672280+0000", 2024-09-14T09:06:59.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1236999999999999e-05 2024-09-14T09:06:59.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672287+0000", 2024-09-14T09:06:59.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8260000000000004e-06 2024-09-14T09:06:59.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698820+0000", 2024-09-14T09:06:59.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026533055999999999 2024-09-14T09:06:59.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698933+0000", 2024-09-14T09:06:59.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011291 2024-09-14T09:06:59.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699557+0000", 2024-09-14T09:06:59.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062421899999999997 2024-09-14T09:06:59.047 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.047 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.047 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.048 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20935 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:59.048 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:59.048 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.317372653999996, 2024-09-14T09:06:59.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028108610999999999, 2024-09-14T09:06:59.048 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.048 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.048 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.048 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.048 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.048 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.048 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20935 2024-09-14T09:06:59.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.048 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:59.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:06:59.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672299+0000", 2024-09-14T09:06:59.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T09:06:59.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672302+0000", 2024-09-14T09:06:59.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7640000000000001e-06 2024-09-14T09:06:59.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672313+0000", 2024-09-14T09:06:59.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1080000000000001e-05 2024-09-14T09:06:59.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672321+0000", 2024-09-14T09:06:59.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6490000000000008e-06 2024-09-14T09:06:59.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699587+0000", 2024-09-14T09:06:59.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027265884000000001 2024-09-14T09:06:59.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699631+0000", 2024-09-14T09:06:59.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4947999999999997e-05 2024-09-14T09:06:59.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700407+0000", 2024-09-14T09:06:59.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00077559299999999999 2024-09-14T09:06:59.051 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.052 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.052 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.052 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20936 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1441792~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:59.052 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:59.052 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.317338477999996, 2024-09-14T09:06:59.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028784750000000001, 2024-09-14T09:06:59.052 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.052 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.052 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.052 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.052 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.052 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.052 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20936 2024-09-14T09:06:59.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.053 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:59.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:59.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:06:59.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4700000000000001e-07 2024-09-14T09:06:59.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672336+0000", 2024-09-14T09:06:59.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6730000000000001e-06 2024-09-14T09:06:59.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672347+0000", 2024-09-14T09:06:59.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0710000000000001e-05 2024-09-14T09:06:59.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672354+0000", 2024-09-14T09:06:59.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4630000000000004e-06 2024-09-14T09:06:59.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700442+0000", 2024-09-14T09:06:59.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028087951 2024-09-14T09:06:59.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700557+0000", 2024-09-14T09:06:59.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011461800000000001 2024-09-14T09:06:59.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701117+0000", 2024-09-14T09:06:59.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056058799999999999 2024-09-14T09:06:59.056 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.056 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.056 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.056 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20937 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:59.056 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:59.056 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.317305352000005, 2024-09-14T09:06:59.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029493028000000001, 2024-09-14T09:06:59.056 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.056 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.056 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.056 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.057 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.057 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.057 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20937 2024-09-14T09:06:59.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.057 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:59.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:59.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:06:59.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T09:06:59.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672384+0000", 2024-09-14T09:06:59.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7127000000000001e-05 2024-09-14T09:06:59.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672395+0000", 2024-09-14T09:06:59.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759999999999999e-05 2024-09-14T09:06:59.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672402+0000", 2024-09-14T09:06:59.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0960000000000003e-06 2024-09-14T09:06:59.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701147+0000", 2024-09-14T09:06:59.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028744241 2024-09-14T09:06:59.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701246+0000", 2024-09-14T09:06:59.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9031999999999996e-05 2024-09-14T09:06:59.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701859+0000", 2024-09-14T09:06:59.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061310499999999997 2024-09-14T09:06:59.060 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.060 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.060 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.060 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20938 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:59.060 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:59.060 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.317258191999997, 2024-09-14T09:06:59.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030109219999999999, 2024-09-14T09:06:59.061 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.061 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.061 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.061 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.061 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.061 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.061 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20938 2024-09-14T09:06:59.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.061 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:59.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:06:59.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672414+0000", 2024-09-14T09:06:59.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-07 2024-09-14T09:06:59.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672417+0000", 2024-09-14T09:06:59.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:06:59.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672428+0000", 2024-09-14T09:06:59.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0873000000000001e-05 2024-09-14T09:06:59.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672435+0000", 2024-09-14T09:06:59.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:06:59.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701884+0000", 2024-09-14T09:06:59.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029449811999999999 2024-09-14T09:06:59.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701934+0000", 2024-09-14T09:06:59.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9703000000000002e-05 2024-09-14T09:06:59.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702522+0000", 2024-09-14T09:06:59.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00058811200000000001 2024-09-14T09:06:59.065 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.065 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.065 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.065 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20939 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:06:59.065 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:59.065 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.317226059000006, 2024-09-14T09:06:59.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030779259, 2024-09-14T09:06:59.065 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.065 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.066 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.066 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.066 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.066 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.066 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20939 2024-09-14T09:06:59.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.066 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:59.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:06:59.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672446+0000", 2024-09-14T09:06:59.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:06:59.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672449+0000", 2024-09-14T09:06:59.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7930000000000002e-06 2024-09-14T09:06:59.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672461+0000", 2024-09-14T09:06:59.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T09:06:59.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672471+0000", 2024-09-14T09:06:59.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0807e-05 2024-09-14T09:06:59.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702551+0000", 2024-09-14T09:06:59.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030079228 2024-09-14T09:06:59.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702664+0000", 2024-09-14T09:06:59.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011351800000000001 2024-09-14T09:06:59.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.703224+0000", 2024-09-14T09:06:59.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056026999999999997 2024-09-14T09:06:59.069 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.069 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.069 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.069 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:74298 3.9 3:95c8282a:::rbd_data.115ea8d131ff.0000000000000041:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:06:59.070 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:06:59.070 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.191916046999999, 2024-09-14T09:06:59.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042668064999999998, 2024-09-14T09:06:59.070 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.070 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.070 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:59.070 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.070 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.070 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.070 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74298 2024-09-14T09:06:59.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.070 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:06:59.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:06:59.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797758+0000", 2024-09-14T09:06:59.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9799999999999998e-06 2024-09-14T09:06:59.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797770+0000", 2024-09-14T09:06:59.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1667e-05 2024-09-14T09:06:59.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797797+0000", 2024-09-14T09:06:59.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7212999999999998e-05 2024-09-14T09:06:59.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797811+0000", 2024-09-14T09:06:59.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4256e-05 2024-09-14T09:06:59.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797882+0000", 2024-09-14T09:06:59.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0622000000000002e-05 2024-09-14T09:06:59.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797943+0000", 2024-09-14T09:06:59.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0893000000000001e-05 2024-09-14T09:06:59.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:59.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.799676+0000", 2024-09-14T09:06:59.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0017331619999999999 2024-09-14T09:06:59.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:59.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840109+0000", 2024-09-14T09:06:59.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040432994 2024-09-14T09:06:59.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:59.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840204+0000", 2024-09-14T09:06:59.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5247000000000005e-05 2024-09-14T09:06:59.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:59.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840346+0000", 2024-09-14T09:06:59.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014156499999999999 2024-09-14T09:06:59.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840423+0000", 2024-09-14T09:06:59.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7466000000000007e-05 2024-09-14T09:06:59.079 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.079 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.079 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.080 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:74298 3.9 e76/34 ondisk, result = 0)", 2024-09-14T09:06:59.080 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:06:59.080 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.185521403999999, 2024-09-14T09:06:59.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036289318000000001, 2024-09-14T09:06:59.080 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.080 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.080 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.080 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:06:59.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:06:59.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804151+0000", 2024-09-14T09:06:59.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6700000000000001e-06 2024-09-14T09:06:59.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804163+0000", 2024-09-14T09:06:59.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.148e-05 2024-09-14T09:06:59.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804197+0000", 2024-09-14T09:06:59.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3909999999999999e-05 2024-09-14T09:06:59.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804217+0000", 2024-09-14T09:06:59.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0287000000000001e-05 2024-09-14T09:06:59.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840176+0000", 2024-09-14T09:06:59.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035958641 2024-09-14T09:06:59.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840193+0000", 2024-09-14T09:06:59.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6986e-05 2024-09-14T09:06:59.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840439+0000", 2024-09-14T09:06:59.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024634400000000001 2024-09-14T09:06:59.083 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.083 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.084 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.084 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:93074 3.9 e94/84 3:925bb5bd:::rbd_data.115e7770ddc5.0000000000000001:head v 94'255, mlcod=94'252)", 2024-09-14T09:06:59.084 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:57.863727+0000", 2024-09-14T09:06:59.084 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.125944144, 2024-09-14T09:06:59.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019329799000000002, 2024-09-14T09:06:59.084 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.084 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.084 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:59.084 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.863727+0000", 2024-09-14T09:06:59.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.863727+0000", 2024-09-14T09:06:59.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.863729+0000", 2024-09-14T09:06:59.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3980000000000002e-06 2024-09-14T09:06:59.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866073+0000", 2024-09-14T09:06:59.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0023435109999999999 2024-09-14T09:06:59.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866094+0000", 2024-09-14T09:06:59.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1444e-05 2024-09-14T09:06:59.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866105+0000", 2024-09-14T09:06:59.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0949e-05 2024-09-14T09:06:59.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866216+0000", 2024-09-14T09:06:59.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110961 2024-09-14T09:06:59.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866258+0000", 2024-09-14T09:06:59.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1903000000000002e-05 2024-09-14T09:06:59.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:59.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.882981+0000", 2024-09-14T09:06:59.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016723198000000002 2024-09-14T09:06:59.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.883057+0000", 2024-09-14T09:06:59.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5434999999999999e-05 2024-09-14T09:06:59.088 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.088 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.088 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.088 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.088 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.088 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:59.088 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:58.991+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:59.089 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:06:59.090 INFO:teuthology.orchestra.run.smithi176.stdout:4465 writesame 0xd1440e7 thru 0xd153a16 data_size 0x48(0xf930 bytes) 2024-09-14T09:06:59.090 INFO:teuthology.orchestra.run.smithi176.stdout:4466 write 0x368f473 thru 0x369152d (0x20bb bytes) 2024-09-14T09:06:59.090 INFO:teuthology.orchestra.run.smithi176.stdout:4467 punch from 0x515e305 to 0x5165045, (0x6d40 bytes) 2024-09-14T09:06:59.091 INFO:teuthology.orchestra.run.smithi176.stdout:4468 write 0x8be78b3 thru 0x8befffb (0x8749 bytes) 2024-09-14T09:06:59.091 INFO:teuthology.orchestra.run.smithi176.stdout:4469 write 0x761e9a8 thru 0x7627cc1 (0x931a bytes) 2024-09-14T09:06:59.091 INFO:teuthology.orchestra.run.smithi176.stdout:4470 writesame 0xd6ebb72 thru 0xd6ebf82 data_size 0x1(0x411 bytes) 2024-09-14T09:06:59.091 INFO:teuthology.orchestra.run.smithi176.stdout:4471 writesame 0xaef439a thru 0xaf03a3b data_size 0x33(0xf6a2 bytes) 2024-09-14T09:06:59.094 INFO:teuthology.orchestra.run.smithi176.stdout:4472 writesame 0x64c3c93 thru 0x64c8519 data_size 0x3(0x4887 bytes) 2024-09-14T09:06:59.094 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:59.095+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:59.109 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:59.109 INFO:teuthology.orchestra.run.smithi176.stdout:4476 read 0xcc46f0e thru 0xcc4727d (0x370 bytes) 2024-09-14T09:06:59.127 INFO:teuthology.orchestra.run.smithi176.stdout:4478 trunc from 0xd6ebf83 to 0x377f7be 2024-09-14T09:06:59.156 INFO:teuthology.orchestra.run.smithi176.stdout:4479 punch from 0x393166 to 0x394a10, (0x18aa bytes) 2024-09-14T09:06:59.173 INFO:teuthology.orchestra.run.smithi176.stdout:4480 write 0x6665c6e thru 0x66724a7 (0xc83a bytes) 2024-09-14T09:06:59.177 INFO:teuthology.orchestra.run.smithi176.stdout:4482 punch from 0xc500eb to 0xc516a6, (0x15bb bytes) 2024-09-14T09:06:59.177 INFO:teuthology.orchestra.run.smithi176.stdout:4484 writesame 0x4266b5d thru 0x4273de2 data_size 0x1(0xd286 bytes) 2024-09-14T09:06:59.193 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:59.195+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:59.227 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:59.227 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:59.227 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:06:59.227 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:59.242 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:06:59.294 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:06:59.295+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:06:59.364 INFO:teuthology.orchestra.run.smithi176.stdout:4485 write 0x39ff372 thru 0x3a0ce85 (0xdb14 bytes) 2024-09-14T09:06:59.368 INFO:teuthology.orchestra.run.smithi176.stdout:4486 read 0x5d019b0 thru 0x5d07008 (0x5659 bytes) 2024-09-14T09:06:59.369 INFO:teuthology.orchestra.run.smithi176.stdout:4487 writesame 0x216cb1a thru 0x217a019 data_size 0x6(0xd500 bytes) 2024-09-14T09:06:59.381 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:06:59.394 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:06:59.395+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:06:59.400 INFO:teuthology.orchestra.run.smithi176.stdout:4488 trunc from 0x66724a8 to 0x2778083 2024-09-14T09:06:59.425 INFO:teuthology.orchestra.run.smithi176.stdout:4489 punch from 0x2648447 to 0x26541e7, (0xbda0 bytes) 2024-09-14T09:06:59.425 INFO:teuthology.orchestra.run.smithi176.stdout:4490 writesame 0x14c32a0 thru 0x14c60ee data_size 0x1(0x2e4f bytes) 2024-09-14T09:06:59.428 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:59.428 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:06:59.428 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:06:59.428 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:06:59.428 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:59.444 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:06:59.448 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:59.448 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:59.448 INFO:teuthology.orchestra.run.smithi176.stdout:4493 punch from 0x17a0267 to 0x17a8c07, (0x89a0 bytes) 2024-09-14T09:06:59.448 INFO:teuthology.orchestra.run.smithi176.stdout:4494 write 0x9406d6 thru 0x94ec7e (0xe5a9 bytes) 2024-09-14T09:06:59.456 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:59.476 INFO:teuthology.orchestra.run.smithi176.stdout:4496 write 0x69c805d thru 0x69cf01f (0x6fc3 bytes) 2024-09-14T09:06:59.479 INFO:teuthology.orchestra.run.smithi176.stdout:4497 read 0x675f398 thru 0x676b187 (0xbdf0 bytes) 2024-09-14T09:06:59.480 INFO:teuthology.orchestra.run.smithi176.stdout:4499 read 0x2490a53 thru 0x2499e2d (0x93db bytes) 2024-09-14T09:06:59.480 INFO:teuthology.orchestra.run.smithi176.stdout:4500 read 0xe5b69a thru 0xe5c682 (0xfe9 bytes) 2024-09-14T09:06:59.495 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:06:59.499+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:06:59.500 INFO:teuthology.orchestra.run.smithi176.stdout:4501 write 0xbaea710 thru 0xbaf1ae6 (0x73d7 bytes) 2024-09-14T09:06:59.507 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:59.521 INFO:teuthology.orchestra.run.smithi176.stdout:4503 writesame 0xd06449c thru 0xd06c1be data_size 0x5(0x7d23 bytes) 2024-09-14T09:06:59.521 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:06:59.530 INFO:teuthology.orchestra.run.smithi176.stdout:4507 trunc from 0xd06c1bf to 0x68751ff 2024-09-14T09:06:59.533 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:06:59.533 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:06:59.553 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:06:59.553 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:06:59.557 INFO:teuthology.orchestra.run.smithi176.stdout:4508 read 0x4534626 thru 0x45366c3 (0x209e bytes) 2024-09-14T09:06:59.558 INFO:teuthology.orchestra.run.smithi176.stdout:4509 write 0x4266d1f thru 0x42752c4 (0xe5a6 bytes) 2024-09-14T09:06:59.568 INFO:teuthology.orchestra.run.smithi176.stdout:4510 punch from 0x4d9f68a to 0x4dae01b, (0xe991 bytes) 2024-09-14T09:06:59.568 INFO:teuthology.orchestra.run.smithi176.stdout:4512 read 0x526c0ed thru 0x526f0c1 (0x2fd5 bytes) 2024-09-14T09:06:59.568 INFO:teuthology.orchestra.run.smithi176.stdout:4513 read 0x350f182 thru 0x3516f9a (0x7e19 bytes) 2024-09-14T09:06:59.569 INFO:teuthology.orchestra.run.smithi176.stdout:4515 write 0x5b313b8 thru 0x5b3b2c1 (0x9f0a bytes) 2024-09-14T09:06:59.571 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:06:59.572 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:06:59.576 INFO:teuthology.orchestra.run.smithi176.stdout:4516 read 0x675eec5 thru 0x675f7a3 (0x8df bytes) 2024-09-14T09:06:59.577 INFO:teuthology.orchestra.run.smithi176.stdout:4517 clone 32 order 18 su 131072 sc 12 2024-09-14T09:06:59.585 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:06:59.586 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:06:59.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:06:59.586 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:06:59.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.586 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:59.586 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:59.586 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 86.502351989999994, 2024-09-14T09:06:59.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:06:59.586 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.586 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.586 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.586 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.586 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.587 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.587 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:06:59.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.587 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:59.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:06:59.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:06:59.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:06:59.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:06:59.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:06:59.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:06:59.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:06:59.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:06:59.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:06:59.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:06:59.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:06:59.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:59.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:06:59.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:06:59.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:06:59.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:06:59.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:06:59.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:06:59.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:06:59.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:06:59.591 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.591 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.591 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.591 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:59.591 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:59.591 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 86.502297331999998, 2024-09-14T09:06:59.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:06:59.591 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.591 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.591 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.591 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.592 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.592 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.592 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:06:59.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.592 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:59.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:59.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:06:59.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:06:59.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:06:59.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:06:59.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:06:59.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:06:59.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:06:59.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:06:59.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:06:59.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:06:59.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:59.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:06:59.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:06:59.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:06:59.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:06:59.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:06:59.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:06:59.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:06:59.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:06:59.596 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.596 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.596 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.596 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:59.596 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:59.596 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 86.502191957999997, 2024-09-14T09:06:59.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:06:59.596 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.596 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.596 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.596 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.596 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.597 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.597 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:06:59.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.597 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:59.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:06:59.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:06:59.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:06:59.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:06:59.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:06:59.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:06:59.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:06:59.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:06:59.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:06:59.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:06:59.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:06:59.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:59.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:06:59.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:06:59.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:06:59.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:06:59.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:06:59.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:06:59.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:06:59.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:06:59.601 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.601 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.601 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.601 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:59.601 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:59.601 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 86.501835193999995, 2024-09-14T09:06:59.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:06:59.602 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.602 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.602 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.602 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.602 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.602 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.602 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:06:59.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.602 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:59.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:06:59.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:06:59.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:06:59.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:06:59.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:06:59.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:06:59.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:06:59.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:06:59.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:06:59.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:06:59.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:06:59.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:59.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:06:59.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:06:59.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:06:59.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:06:59.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:06:59.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:06:59.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:06:59.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:06:59.606 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.606 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.606 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.607 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:59.607 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:59.607 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 86.501750455999996, 2024-09-14T09:06:59.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:06:59.607 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.607 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.607 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.607 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.607 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.607 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.607 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:06:59.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.607 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:59.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:06:59.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:06:59.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:06:59.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:06:59.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:06:59.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:06:59.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:06:59.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:06:59.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:06:59.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:06:59.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:06:59.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:59.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:06:59.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:06:59.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:06:59.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:06:59.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:06:59.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:06:59.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:06:59.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:06:59.611 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.611 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.611 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.612 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:06:59.612 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:59.612 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 86.501662769999996, 2024-09-14T09:06:59.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:06:59.612 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.612 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.612 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.612 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.612 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.612 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.612 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:06:59.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.612 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:59.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:06:59.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:06:59.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:06:59.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:06:59.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:06:59.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:06:59.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:06:59.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:06:59.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:06:59.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:06:59.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:06:59.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:06:59.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:06:59.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:06:59.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:06:59.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:06:59.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:06:59.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:06:59.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:06:59.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:06:59.616 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.616 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.617 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.617 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18388 3.18 3:19e2be55:::rbd_data.115e3abd3b2c.0000000000000021:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:06:59.617 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:59.617 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.365749459, 2024-09-14T09:06:59.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042293648000000003, 2024-09-14T09:06:59.617 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.617 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.617 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:59.617 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.617 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.617 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.618 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18388 2024-09-14T09:06:59.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.618 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:59.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:06:59.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220236+0000", 2024-09-14T09:06:59.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1339999999999998e-06 2024-09-14T09:06:59.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220244+0000", 2024-09-14T09:06:59.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1950000000000003e-06 2024-09-14T09:06:59.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220262+0000", 2024-09-14T09:06:59.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8017e-05 2024-09-14T09:06:59.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220270+0000", 2024-09-14T09:06:59.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9480000000000008e-06 2024-09-14T09:06:59.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220357+0000", 2024-09-14T09:06:59.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7185e-05 2024-09-14T09:06:59.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220576+0000", 2024-09-14T09:06:59.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021912200000000001 2024-09-14T09:06:59.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:59.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.222435+0000", 2024-09-14T09:06:59.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018592350000000001 2024-09-14T09:06:59.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:59.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262439+0000", 2024-09-14T09:06:59.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040003368999999997 2024-09-14T09:06:59.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:59.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262452+0000", 2024-09-14T09:06:59.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2941e-05 2024-09-14T09:06:59.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:59.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262490+0000", 2024-09-14T09:06:59.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8478e-05 2024-09-14T09:06:59.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262527+0000", 2024-09-14T09:06:59.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7024000000000003e-05 2024-09-14T09:06:59.622 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.622 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.622 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.623 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38002 3.1e 3:7df715aa:::rbd_data.115eb07a8abd.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:59.623 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:59.623 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.196231552999997, 2024-09-14T09:06:59.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.065148304000000004, 2024-09-14T09:06:59.623 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.623 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.623 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:59.623 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.623 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.623 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.623 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38002 2024-09-14T09:06:59.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.624 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:59.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:06:59.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389752+0000", 2024-09-14T09:06:59.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1829999999999999e-06 2024-09-14T09:06:59.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389758+0000", 2024-09-14T09:06:59.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0919999999999997e-06 2024-09-14T09:06:59.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389769+0000", 2024-09-14T09:06:59.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759e-05 2024-09-14T09:06:59.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389775+0000", 2024-09-14T09:06:59.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8819999999999998e-06 2024-09-14T09:06:59.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389824+0000", 2024-09-14T09:06:59.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8573999999999999e-05 2024-09-14T09:06:59.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389876+0000", 2024-09-14T09:06:59.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2643999999999999e-05 2024-09-14T09:06:59.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:06:59.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.392640+0000", 2024-09-14T09:06:59.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0027636409999999998 2024-09-14T09:06:59.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:59.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454672+0000", 2024-09-14T09:06:59.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062031703000000001 2024-09-14T09:06:59.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:59.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454765+0000", 2024-09-14T09:06:59.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2862000000000006e-05 2024-09-14T09:06:59.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:59.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454841+0000", 2024-09-14T09:06:59.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5857000000000002e-05 2024-09-14T09:06:59.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454900+0000", 2024-09-14T09:06:59.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9107000000000002e-05 2024-09-14T09:06:59.628 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.629 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.629 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.629 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:38002 3.1e e48/23 ondisk, result = 0)", 2024-09-14T09:06:59.629 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:59.629 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.185914963999998, 2024-09-14T09:06:59.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054841741999999999, 2024-09-14T09:06:59.629 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.629 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.629 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.629 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:59.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:06:59.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400069+0000", 2024-09-14T09:06:59.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.339e-06 2024-09-14T09:06:59.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400076+0000", 2024-09-14T09:06:59.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7800000000000003e-06 2024-09-14T09:06:59.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400093+0000", 2024-09-14T09:06:59.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6915999999999999e-05 2024-09-14T09:06:59.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400106+0000", 2024-09-14T09:06:59.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2785e-05 2024-09-14T09:06:59.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454713+0000", 2024-09-14T09:06:59.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054607041000000002 2024-09-14T09:06:59.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454724+0000", 2024-09-14T09:06:59.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1202e-05 2024-09-14T09:06:59.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454910+0000", 2024-09-14T09:06:59.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018567900000000001 2024-09-14T09:06:59.633 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.633 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.633 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.633 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38100 3.b 3:d0f67c24:::rbd_data.115eb07a8abd.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:06:59.633 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:59.633 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 61.940061303999997, 2024-09-14T09:06:59.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053701964999999997, 2024-09-14T09:06:59.634 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.634 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.634 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:59.634 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.634 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.634 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.634 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38100 2024-09-14T09:06:59.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.634 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:59.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:06:59.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645923+0000", 2024-09-14T09:06:59.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:06:59.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645928+0000", 2024-09-14T09:06:59.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0699999999999997e-06 2024-09-14T09:06:59.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645942+0000", 2024-09-14T09:06:59.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4069e-05 2024-09-14T09:06:59.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645951+0000", 2024-09-14T09:06:59.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3230000000000004e-06 2024-09-14T09:06:59.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646008+0000", 2024-09-14T09:06:59.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6583999999999999e-05 2024-09-14T09:06:59.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646089+0000", 2024-09-14T09:06:59.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1557999999999994e-05 2024-09-14T09:06:59.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:06:59.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.650355+0000", 2024-09-14T09:06:59.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004265525 2024-09-14T09:06:59.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:59.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699439+0000", 2024-09-14T09:06:59.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049083904999999997 2024-09-14T09:06:59.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:59.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699467+0000", 2024-09-14T09:06:59.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8198999999999999e-05 2024-09-14T09:06:59.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:59.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699552+0000", 2024-09-14T09:06:59.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T09:06:59.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699624+0000", 2024-09-14T09:06:59.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1725999999999997e-05 2024-09-14T09:06:59.638 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.638 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.638 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.639 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:06:59.639 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:59.639 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.425370782999998, 2024-09-14T09:06:59.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:06:59.639 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.639 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.639 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:59.639 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.639 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.639 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.640 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:06:59.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.640 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:59.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:06:59.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:06:59.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:06:59.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:06:59.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:06:59.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:06:59.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:06:59.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:06:59.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:06:59.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:06:59.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:06:59.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:06:59.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:06:59.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:06:59.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:06:59.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:06:59.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:59.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:06:59.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:06:59.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:59.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:06:59.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:06:59.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:59.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:06:59.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:06:59.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:06:59.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:06:59.644 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.644 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.644 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.644 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:39401 3.b e56/53)", 2024-09-14T09:06:59.645 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:59.645 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.399474716999997, 2024-09-14T09:06:59.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10443659600000001, 2024-09-14T09:06:59.645 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.645 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.645 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.645 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:59.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:06:59.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.646 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186509+0000", 2024-09-14T09:06:59.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2640000000000001e-06 2024-09-14T09:06:59.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.646 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186516+0000", 2024-09-14T09:06:59.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1070000000000001e-06 2024-09-14T09:06:59.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186533+0000", 2024-09-14T09:06:59.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7818000000000001e-05 2024-09-14T09:06:59.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186543+0000", 2024-09-14T09:06:59.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5170000000000005e-06 2024-09-14T09:06:59.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290885+0000", 2024-09-14T09:06:59.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.104342221 2024-09-14T09:06:59.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290893+0000", 2024-09-14T09:06:59.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2859999999999999e-06 2024-09-14T09:06:59.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290945+0000", 2024-09-14T09:06:59.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1382999999999999e-05 2024-09-14T09:06:59.648 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.648 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.648 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.648 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:84137 3.7 3:e15b49d9:::rbd_data.115e77b799c3.00000000000000e0:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:06:59.648 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:06:59.648 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.2626233850000004, 2024-09-14T09:06:59.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11205707500000001, 2024-09-14T09:06:59.649 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.649 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.649 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:59.649 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.649 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.649 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.649 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 84137 2024-09-14T09:06:59.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.649 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:06:59.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:06:59.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323362+0000", 2024-09-14T09:06:59.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0839999999999999e-06 2024-09-14T09:06:59.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323370+0000", 2024-09-14T09:06:59.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0180000000000001e-06 2024-09-14T09:06:59.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323388+0000", 2024-09-14T09:06:59.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8326999999999999e-05 2024-09-14T09:06:59.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323397+0000", 2024-09-14T09:06:59.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1940000000000004e-06 2024-09-14T09:06:59.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323452+0000", 2024-09-14T09:06:59.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5043000000000001e-05 2024-09-14T09:06:59.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323550+0000", 2024-09-14T09:06:59.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8201000000000004e-05 2024-09-14T09:06:59.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:06:59.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.328273+0000", 2024-09-14T09:06:59.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0047227079999999999 2024-09-14T09:06:59.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:59.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435232+0000", 2024-09-14T09:06:59.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.106959339 2024-09-14T09:06:59.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:59.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435256+0000", 2024-09-14T09:06:59.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3774e-05 2024-09-14T09:06:59.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:59.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435314+0000", 2024-09-14T09:06:59.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7636999999999997e-05 2024-09-14T09:06:59.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435417+0000", 2024-09-14T09:06:59.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010275 2024-09-14T09:06:59.654 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.654 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.654 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.654 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:84137 3.7 e92/23 ondisk, result = 0)", 2024-09-14T09:06:59.654 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:53.341180+0000", 2024-09-14T09:06:59.654 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.2448028009999996, 2024-09-14T09:06:59.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.094062506000000004, 2024-09-14T09:06:59.655 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.655 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.655 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.655 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341180+0000", 2024-09-14T09:06:59.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341180+0000", 2024-09-14T09:06:59.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341181+0000", 2024-09-14T09:06:59.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.192e-06 2024-09-14T09:06:59.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341187+0000", 2024-09-14T09:06:59.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0070000000000003e-06 2024-09-14T09:06:59.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341208+0000", 2024-09-14T09:06:59.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0321000000000001e-05 2024-09-14T09:06:59.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341219+0000", 2024-09-14T09:06:59.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1764e-05 2024-09-14T09:06:59.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435214+0000", 2024-09-14T09:06:59.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.093994438999999999 2024-09-14T09:06:59.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435225+0000", 2024-09-14T09:06:59.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1123e-05 2024-09-14T09:06:59.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435243+0000", 2024-09-14T09:06:59.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.766e-05 2024-09-14T09:06:59.658 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.658 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.658 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.658 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:92675 3.8 3:105e0099:::rbd_data.115e7770ddc5.0000000000000016:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:06:59.659 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:57.736053+0000", 2024-09-14T09:06:59.659 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.849929556, 2024-09-14T09:06:59.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073115555999999998, 2024-09-14T09:06:59.659 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.659 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.659 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:59.659 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.659 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.659 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.659 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 92675 2024-09-14T09:06:59.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.659 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.659 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736053+0000", 2024-09-14T09:06:59.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736053+0000", 2024-09-14T09:06:59.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736055+0000", 2024-09-14T09:06:59.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.33e-06 2024-09-14T09:06:59.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736062+0000", 2024-09-14T09:06:59.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.08e-06 2024-09-14T09:06:59.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736077+0000", 2024-09-14T09:06:59.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5605000000000001e-05 2024-09-14T09:06:59.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736088+0000", 2024-09-14T09:06:59.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0705e-05 2024-09-14T09:06:59.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736175+0000", 2024-09-14T09:06:59.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6926000000000001e-05 2024-09-14T09:06:59.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736325+0000", 2024-09-14T09:06:59.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149739 2024-09-14T09:06:59.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:59.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.740035+0000", 2024-09-14T09:06:59.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0037099250000000002 2024-09-14T09:06:59.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:59.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809001+0000", 2024-09-14T09:06:59.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.068966240999999998 2024-09-14T09:06:59.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:59.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809049+0000", 2024-09-14T09:06:59.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8572999999999997e-05 2024-09-14T09:06:59.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:59.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809102+0000", 2024-09-14T09:06:59.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2564e-05 2024-09-14T09:06:59.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809169+0000", 2024-09-14T09:06:59.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6867999999999997e-05 2024-09-14T09:06:59.664 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.664 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.664 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.664 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:92675 3.8 e94/23 ondisk, result = 0)", 2024-09-14T09:06:59.664 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:57.748881+0000", 2024-09-14T09:06:59.664 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.837101506, 2024-09-14T09:06:59.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060300578000000001, 2024-09-14T09:06:59.665 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.665 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.665 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.665 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748881+0000", 2024-09-14T09:06:59.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748881+0000", 2024-09-14T09:06:59.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748883+0000", 2024-09-14T09:06:59.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.491e-06 2024-09-14T09:06:59.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748887+0000", 2024-09-14T09:06:59.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5310000000000003e-06 2024-09-14T09:06:59.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748904+0000", 2024-09-14T09:06:59.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.662e-05 2024-09-14T09:06:59.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748912+0000", 2024-09-14T09:06:59.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3869999999999995e-06 2024-09-14T09:06:59.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809034+0000", 2024-09-14T09:06:59.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060121757999999997 2024-09-14T09:06:59.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809044+0000", 2024-09-14T09:06:59.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7699999999999996e-06 2024-09-14T09:06:59.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809182+0000", 2024-09-14T09:06:59.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000138021 2024-09-14T09:06:59.668 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.668 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.668 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.668 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:93277 3.8 3:105e0099:::rbd_data.115e7770ddc5.0000000000000016:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:06:59.668 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.097058+0000", 2024-09-14T09:06:59.668 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.4889252449999999, 2024-09-14T09:06:59.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.065618911000000002, 2024-09-14T09:06:59.668 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.668 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.669 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:59.669 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.669 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.669 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.669 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 93277 2024-09-14T09:06:59.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.669 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097058+0000", 2024-09-14T09:06:59.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097058+0000", 2024-09-14T09:06:59.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097060+0000", 2024-09-14T09:06:59.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.497e-06 2024-09-14T09:06:59.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097072+0000", 2024-09-14T09:06:59.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1600000000000001e-05 2024-09-14T09:06:59.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097102+0000", 2024-09-14T09:06:59.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0552999999999997e-05 2024-09-14T09:06:59.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097124+0000", 2024-09-14T09:06:59.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1477000000000001e-05 2024-09-14T09:06:59.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097201+0000", 2024-09-14T09:06:59.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7065999999999997e-05 2024-09-14T09:06:59.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097277+0000", 2024-09-14T09:06:59.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5722000000000004e-05 2024-09-14T09:06:59.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:06:59.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.101895+0000", 2024-09-14T09:06:59.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004618212 2024-09-14T09:06:59.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:59.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162452+0000", 2024-09-14T09:06:59.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060557603000000002 2024-09-14T09:06:59.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:59.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162504+0000", 2024-09-14T09:06:59.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.147e-05 2024-09-14T09:06:59.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:59.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162643+0000", 2024-09-14T09:06:59.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000139245 2024-09-14T09:06:59.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162677+0000", 2024-09-14T09:06:59.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3466000000000001e-05 2024-09-14T09:06:59.673 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.673 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.673 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.673 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:93277 3.8 e95/23 ondisk, result = 0)", 2024-09-14T09:06:59.674 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.113855+0000", 2024-09-14T09:06:59.674 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 1.4721276759999999, 2024-09-14T09:06:59.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048828219999999999, 2024-09-14T09:06:59.674 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.674 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.674 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.674 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.113855+0000", 2024-09-14T09:06:59.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.113855+0000", 2024-09-14T09:06:59.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.113857+0000", 2024-09-14T09:06:59.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3400000000000001e-06 2024-09-14T09:06:59.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.113863+0000", 2024-09-14T09:06:59.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4740000000000002e-06 2024-09-14T09:06:59.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.113883+0000", 2024-09-14T09:06:59.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9738999999999999e-05 2024-09-14T09:06:59.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.113893+0000", 2024-09-14T09:06:59.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0634e-05 2024-09-14T09:06:59.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162478+0000", 2024-09-14T09:06:59.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048584862999999999 2024-09-14T09:06:59.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162484+0000", 2024-09-14T09:06:59.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1829999999999999e-06 2024-09-14T09:06:59.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162683+0000", 2024-09-14T09:06:59.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000198987 2024-09-14T09:06:59.677 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.677 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.677 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.677 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:95495 3.18 3:1d9f7d98:::rbd_data.115e7770ddc5.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:06:59.677 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.598437+0000", 2024-09-14T09:06:59.677 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.98754619600000004, 2024-09-14T09:06:59.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.090910543999999996, 2024-09-14T09:06:59.677 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.678 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.678 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:06:59.678 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:06:59.678 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:06:59.678 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:06:59.678 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 95495 2024-09-14T09:06:59.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.678 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598437+0000", 2024-09-14T09:06:59.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598437+0000", 2024-09-14T09:06:59.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598438+0000", 2024-09-14T09:06:59.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1149999999999999e-06 2024-09-14T09:06:59.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598443+0000", 2024-09-14T09:06:59.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4589999999999997e-06 2024-09-14T09:06:59.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598455+0000", 2024-09-14T09:06:59.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1793e-05 2024-09-14T09:06:59.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598463+0000", 2024-09-14T09:06:59.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9759999999999995e-06 2024-09-14T09:06:59.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598523+0000", 2024-09-14T09:06:59.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9926000000000002e-05 2024-09-14T09:06:59.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598581+0000", 2024-09-14T09:06:59.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8285000000000002e-05 2024-09-14T09:06:59.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:06:59.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.602809+0000", 2024-09-14T09:06:59.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0042279079999999998 2024-09-14T09:06:59.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:06:59.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689202+0000", 2024-09-14T09:06:59.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.086392847999999994 2024-09-14T09:06:59.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:06:59.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689230+0000", 2024-09-14T09:06:59.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8501999999999999e-05 2024-09-14T09:06:59.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:06:59.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689290+0000", 2024-09-14T09:06:59.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9524000000000001e-05 2024-09-14T09:06:59.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689347+0000", 2024-09-14T09:06:59.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7207999999999998e-05 2024-09-14T09:06:59.682 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.682 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.682 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.683 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:95495 3.18 e95/80 ondisk, result = 0)", 2024-09-14T09:06:59.683 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.610054+0000", 2024-09-14T09:06:59.683 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.97592873099999999, 2024-09-14T09:06:59.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079165141999999994, 2024-09-14T09:06:59.683 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:06:59.683 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:06:59.683 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:06:59.683 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:06:59.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:06:59.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610054+0000", 2024-09-14T09:06:59.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:06:59.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610054+0000", 2024-09-14T09:06:59.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:06:59.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:06:59.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610055+0000", 2024-09-14T09:06:59.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2669999999999999e-06 2024-09-14T09:06:59.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:06:59.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610061+0000", 2024-09-14T09:06:59.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:06:59.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:06:59.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610078+0000", 2024-09-14T09:06:59.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6965000000000001e-05 2024-09-14T09:06:59.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:06:59.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610087+0000", 2024-09-14T09:06:59.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.611e-06 2024-09-14T09:06:59.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:06:59.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689008+0000", 2024-09-14T09:06:59.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.078921186000000004 2024-09-14T09:06:59.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:06:59.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689020+0000", 2024-09-14T09:06:59.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.132e-05 2024-09-14T09:06:59.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:06:59.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:06:59.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:06:59.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689219+0000", 2024-09-14T09:06:59.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000199469 2024-09-14T09:06:59.686 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.686 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.686 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.686 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:06:59.686 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:06:59.686 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:06:59.687 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:06:59.687 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:06:59.688 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:06:59.689 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:06:59.689 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:06:59.689 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:06:59.690 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:06:59.690 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:06:59.690 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:06:59.690 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:06:59.691 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:06:59.691 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:06:59.687+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:06:59.788 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:06:59.787+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:06:59.821 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:59.821 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:59.835 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:06:59.889 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:06:59.887+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:06:59.956 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:06:59.957 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:06:59.961 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T09:06:59.972 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:07:00.006 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:00.007+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:00.091 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:00.095+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:00.096 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:00.096 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:00.110 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:07:00.181 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-14T09:07:00.188 INFO:teuthology.orchestra.run.smithi176.stdout:4518 trunc from 0x68751ff to 0x3be45af 2024-09-14T09:07:00.199 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:00.203+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:00.207 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:00.235 INFO:teuthology.orchestra.run.smithi176.stdout:4520 write 0xd181206 thru 0xd184b3b (0x3936 bytes) 2024-09-14T09:07:00.248 INFO:teuthology.orchestra.run.smithi176.stdout:4521 write 0x19097b3 thru 0x190ff86 (0x67d4 bytes) 2024-09-14T09:07:00.257 INFO:teuthology.orchestra.run.smithi176.stdout:4524 write 0x7a5565f thru 0x7a60a8c (0xb42e bytes) 2024-09-14T09:07:00.283 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:00.283 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:00.291 INFO:teuthology.orchestra.run.smithi176.stdout:4525 trunc from 0xd184b3c to 0x390695e 2024-09-14T09:07:00.292 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:00.291+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:00.297 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:07:00.341 INFO:teuthology.orchestra.run.smithi176.stdout:4527 trunc from 0x390695e to 0xbaca93a 2024-09-14T09:07:00.351 INFO:teuthology.orchestra.run.smithi176.stdout:4528 write 0x842487d thru 0x842b9ed (0x7171 bytes) 2024-09-14T09:07:00.358 INFO:teuthology.orchestra.run.smithi176.stdout:4530 read 0x5f61232 thru 0x5f684fd (0x72cc bytes) 2024-09-14T09:07:00.358 INFO:teuthology.orchestra.run.smithi176.stdout:4531 read 0xd2aa5a thru 0xd38189 (0xd730 bytes) 2024-09-14T09:07:00.359 INFO:teuthology.orchestra.run.smithi176.stdout:4532 read 0xaafe8c1 thru 0xab0cec7 (0xe607 bytes) 2024-09-14T09:07:00.359 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:00.359 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:00.359 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:00.359 INFO:teuthology.orchestra.run.smithi176.stdout:4538 write 0x1adbe16 thru 0x1ae0c3f (0x4e2a bytes) 2024-09-14T09:07:00.371 INFO:teuthology.orchestra.run.smithi176.stdout:4539 trunc from 0xbaca93a to 0xdc32255 2024-09-14T09:07:00.383 INFO:teuthology.orchestra.run.smithi176.stdout:4541 punch from 0xbc1e9b8 to 0xbc27bfc, (0x9244 bytes) 2024-09-14T09:07:00.393 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:00.395+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:00.406 INFO:teuthology.orchestra.run.smithi176.stdout:4542 trunc from 0xdc32255 to 0x48af142 2024-09-14T09:07:00.429 INFO:teuthology.orchestra.run.smithi176.stdout:4543 read 0x4a95a3 thru 0x4ab1bc (0x1c1a bytes) 2024-09-14T09:07:00.434 INFO:teuthology.orchestra.run.smithi176.stdout:4544 trunc from 0x48af142 to 0x3262eda 2024-09-14T09:07:00.466 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:00.466 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:00.474 INFO:teuthology.orchestra.run.smithi176.stdout:4545 writesame 0x905c76 thru 0x907d12 data_size 0x21(0x209d bytes) 2024-09-14T09:07:00.481 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:07:00.484 INFO:teuthology.orchestra.run.smithi176.stdout:4546 punch from 0x199d760 to 0x199f572, (0x1e12 bytes) 2024-09-14T09:07:00.484 INFO:teuthology.orchestra.run.smithi176.stdout:4548 punch from 0x9869b9 to 0x992936, (0xbf7d bytes) 2024-09-14T09:07:00.507 INFO:teuthology.orchestra.run.smithi176.stdout:4549 trunc from 0x3262eda to 0xc194ef8 2024-09-14T09:07:00.519 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:00.520 INFO:teuthology.orchestra.run.smithi176.stdout:4551 write 0x9a4fefb thru 0x9a52cfb (0x2e01 bytes) 2024-09-14T09:07:00.523 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:00.527+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:00.527 INFO:teuthology.orchestra.run.smithi176.stdout:4552 trunc from 0xc194ef8 to 0xcc983b9 2024-09-14T09:07:00.539 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:00.539 INFO:teuthology.orchestra.run.smithi176.stdout:4555 writesame 0x3a103a9 thru 0x3a1adcf data_size 0x1(0xaa27 bytes) 2024-09-14T09:07:00.594 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:00.595+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:00.649 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:00.650 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:00.654 INFO:teuthology.orchestra.run.smithi176.stdout:4557 trunc from 0xcc983b9 to 0x73a6627 2024-09-14T09:07:00.665 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:07:00.676 INFO:teuthology.orchestra.run.smithi176.stdout:4561 write 0x4a32bd7 thru 0x4a40809 (0xdc33 bytes) 2024-09-14T09:07:00.689 INFO:teuthology.orchestra.run.smithi176.stdout:4562 read 0x583015e thru 0x5835cbb (0x5b5e bytes) 2024-09-14T09:07:00.689 INFO:teuthology.orchestra.run.smithi176.stdout:4563 punch from 0x5c3e753 to 0x5c425b2, (0x3e5f bytes) 2024-09-14T09:07:00.695 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:00.695+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:00.698 INFO:teuthology.orchestra.run.smithi176.stdout:4564 trunc from 0x73a6627 to 0x3d9ba27 2024-09-14T09:07:00.721 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:00.753 INFO:teuthology.orchestra.run.smithi176.stdout:4570 writesame 0x9156f31 thru 0x9160d00 data_size 0x65(0x9dd0 bytes) 2024-09-14T09:07:00.753 INFO:teuthology.orchestra.run.smithi176.stdout:4571 read 0x7056e7b thru 0x7058d6c (0x1ef2 bytes) 2024-09-14T09:07:00.776 INFO:teuthology.orchestra.run.smithi176.stdout:4572 writesame 0xc117ed1 thru 0xc118508 data_size 0x4(0x638 bytes) 2024-09-14T09:07:00.777 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:00.777 INFO:teuthology.orchestra.run.smithi176.stdout:4574 write 0x8de4b74 thru 0x8df2fc5 (0xe452 bytes) 2024-09-14T09:07:00.790 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:00.791 INFO:teuthology.orchestra.run.smithi176.stdout:4576 writesame 0x8b8ee0a thru 0x8b9a526 data_size 0x1(0xb71d bytes) 2024-09-14T09:07:00.796 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:00.799+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:00.832 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:00.833 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:00.847 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:07:00.897 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:00.899+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:00.952 INFO:teuthology.orchestra.run.smithi176.stdout:4577 writesame 0x8661005 thru 0x8669a77 data_size 0x17(0x8a73 bytes) 2024-09-14T09:07:00.961 INFO:teuthology.orchestra.run.smithi176.stdout:4579 write 0x2a7c1c3 thru 0x2a7fc7d (0x3abb bytes) 2024-09-14T09:07:00.967 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is set 2024-09-14T09:07:00.970 INFO:teuthology.orchestra.run.smithi176.stdout:4580 punch from 0x7da996d to 0x7db6246, (0xc8d9 bytes) 2024-09-14T09:07:00.970 INFO:teuthology.orchestra.run.smithi176.stdout:4581 write 0x3616c50 thru 0x3620d05 (0xa0b6 bytes) 2024-09-14T09:07:00.991 INFO:teuthology.orchestra.run.smithi176.stdout:4582 write 0xc1b5ab9 thru 0xc1bd1d1 (0x7719 bytes) 2024-09-14T09:07:00.997 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:00.999+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:01.002 INFO:teuthology.orchestra.run.smithi176.stdout:4583 trunc from 0xc1bd1d2 to 0x9095e93 2024-09-14T09:07:01.019 INFO:teuthology.orchestra.run.smithi176.stdout:4584 write 0x89edbb7 thru 0x89f0bca (0x3014 bytes) 2024-09-14T09:07:01.025 INFO:teuthology.orchestra.run.smithi176.stdout:4585 read 0x3f4690 thru 0x401311 (0xcc82 bytes) 2024-09-14T09:07:01.029 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:01.030 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:01.043 INFO:teuthology.orchestra.run.smithi176.stdout:4587 write 0xcd954fc thru 0xcd9a8e8 (0x53ed bytes) 2024-09-14T09:07:01.045 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:07:01.050 INFO:teuthology.orchestra.run.smithi176.stdout:4588 read 0x2e2dcbd thru 0x2e3395f (0x5ca3 bytes) 2024-09-14T09:07:01.050 INFO:teuthology.orchestra.run.smithi176.stdout:4589 write 0x28f85d1 thru 0x28fe857 (0x6287 bytes) 2024-09-14T09:07:01.057 INFO:teuthology.orchestra.run.smithi176.stdout:4591 writesame 0x7a1be28 thru 0x7a27bcb data_size 0x6a(0xbda4 bytes) 2024-09-14T09:07:01.066 INFO:teuthology.orchestra.run.smithi176.stdout:4592 write 0xcaa38d4 thru 0xcab1550 (0xdc7d bytes) 2024-09-14T09:07:01.076 INFO:teuthology.orchestra.run.smithi176.stdout:4593 punch from 0x4a77ab1 to 0x4a7ccfb, (0x524a bytes) 2024-09-14T09:07:01.076 INFO:teuthology.orchestra.run.smithi176.stdout:4594 read 0x6f46bae thru 0x6f536ff (0xcb52 bytes) 2024-09-14T09:07:01.076 INFO:teuthology.orchestra.run.smithi176.stdout:4595 write 0xbad9a43 thru 0xbadafdc (0x159a bytes) 2024-09-14T09:07:01.082 INFO:teuthology.orchestra.run.smithi176.stdout:4596 writesame 0x77db894 thru 0x77df287 data_size 0x2(0x39f4 bytes) 2024-09-14T09:07:01.098 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:01.099+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:01.103 INFO:teuthology.orchestra.run.smithi176.stdout:4599 read 0x51df725 thru 0x51ec925 (0xd201 bytes) 2024-09-14T09:07:01.103 INFO:teuthology.orchestra.run.smithi176.stdout:4600 read 0x1db17ad thru 0x1dc163f (0xfe93 bytes) 2024-09-14T09:07:01.105 INFO:teuthology.orchestra.run.smithi176.stdout:4601 write 0x82759ec thru 0x828151d (0xbb32 bytes) 2024-09-14T09:07:01.113 INFO:teuthology.orchestra.run.smithi176.stdout:4602 writesame 0x94b3f69 thru 0x94c02a7 data_size 0x1(0xc33f bytes) 2024-09-14T09:07:01.199 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:01.199+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:01.212 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:01.212 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:01.226 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:07:01.284 INFO:teuthology.orchestra.run.smithi176.stdout:4603 read 0x507b150 thru 0x507f127 (0x3fd8 bytes) 2024-09-14T09:07:01.300 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:01.299+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:01.310 INFO:teuthology.orchestra.run.smithi176.stdout:4604 trunc from 0xcd9a8e9 to 0x1ffca2a 2024-09-14T09:07:01.360 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:01.360 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:01.374 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:07:01.397 INFO:teuthology.orchestra.run.smithi176.stdout:4605 write 0xb235bd6 thru 0xb2385e2 (0x2a0d bytes) 2024-09-14T09:07:01.400 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:01.399+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:01.402 INFO:teuthology.orchestra.run.smithi176.stdout:4606 writesame 0x2c615e7 thru 0x2c63255 data_size 0x1(0x1c6f bytes) 2024-09-14T09:07:01.412 INFO:teuthology.orchestra.run.smithi176.stdout:4607 writesame 0x4649282 thru 0x464a1c7 data_size 0xa(0xf46 bytes) 2024-09-14T09:07:01.419 INFO:teuthology.orchestra.run.smithi176.stdout:4608 read 0x625a5de thru 0x625e739 (0x415c bytes) 2024-09-14T09:07:01.419 INFO:teuthology.orchestra.run.smithi176.stdout:4609 read 0x3b0d156 thru 0x3b15bc1 (0x8a6c bytes) 2024-09-14T09:07:01.433 INFO:teuthology.orchestra.run.smithi176.stdout:4610 trunc from 0xb2385e3 to 0x5fb4cb8 2024-09-14T09:07:01.471 INFO:teuthology.orchestra.run.smithi176.stdout:4611 trunc from 0x5fb4cb8 to 0xada2e70 2024-09-14T09:07:01.484 INFO:teuthology.orchestra.run.smithi176.stdout:4612 read 0x770deb2 thru 0x7710bdf (0x2d2e bytes) 2024-09-14T09:07:01.484 INFO:teuthology.orchestra.run.smithi176.stdout:4613 read 0x3957dd7 thru 0x396587a (0xdaa4 bytes) 2024-09-14T09:07:01.485 INFO:teuthology.orchestra.run.smithi176.stdout:4615 write 0x9f77b15 thru 0x9f844b5 (0xc9a1 bytes) 2024-09-14T09:07:01.496 INFO:teuthology.orchestra.run.smithi176.stdout:4616 punch from 0x7f6b653 to 0x7f70746, (0x50f3 bytes) 2024-09-14T09:07:01.500 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:01.501 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:01.501 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:01.499+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:01.515 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:07:01.523 INFO:teuthology.orchestra.run.smithi176.stdout:4617 trunc from 0xada2e70 to 0x7f9d6d 2024-09-14T09:07:01.565 INFO:teuthology.orchestra.run.smithi176.stdout:4619 read 0x28862f thru 0x28ba82 (0x3454 bytes) 2024-09-14T09:07:01.597 INFO:teuthology.orchestra.run.smithi176.stdout:4620 writesame 0x9ae93f2 thru 0x9ae9d74 data_size 0x1(0x983 bytes) 2024-09-14T09:07:01.602 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:01.603+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:01.615 INFO:teuthology.orchestra.run.smithi176.stdout:4621 writesame 0xc11f898 thru 0xc127e2a data_size 0x7(0x8593 bytes) 2024-09-14T09:07:01.622 INFO:teuthology.orchestra.run.smithi176.stdout:4622 trunc from 0xc127e2b to 0x94e5f64 2024-09-14T09:07:01.635 INFO:teuthology.orchestra.run.smithi176.stdout:4623 read 0x678128 thru 0x68528e (0xd167 bytes) 2024-09-14T09:07:01.637 INFO:teuthology.orchestra.run.smithi176.stdout:4624 trunc from 0x94e5f64 to 0x9981973 2024-09-14T09:07:01.648 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:01.648 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:01.649 INFO:teuthology.orchestra.run.smithi176.stdout:4625 punch from 0xb92a75 to 0xb9ec20, (0xc1ab bytes) 2024-09-14T09:07:01.649 INFO:teuthology.orchestra.run.smithi176.stdout:4626 read 0x57aec68 thru 0x57b4089 (0x5422 bytes) 2024-09-14T09:07:01.651 INFO:teuthology.orchestra.run.smithi176.stdout:4627 trunc from 0x9981973 to 0xa346766 2024-09-14T09:07:01.663 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:07:01.665 INFO:teuthology.orchestra.run.smithi176.stdout:4628 punch from 0x73eb116 to 0x73fa864, (0xf74e bytes) 2024-09-14T09:07:01.665 INFO:teuthology.orchestra.run.smithi176.stdout:4629 punch from 0x84ab751 to 0x84b2c5b, (0x750a bytes) 2024-09-14T09:07:01.684 INFO:teuthology.orchestra.run.smithi176.stdout:4630 writesame 0xd542b56 thru 0xd5438f0 data_size 0x1b(0xd9b bytes) 2024-09-14T09:07:01.684 INFO:teuthology.orchestra.run.smithi176.stdout:4631 write 0xaf1a84 thru 0xaf2435 (0x9b2 bytes) 2024-09-14T09:07:01.689 INFO:teuthology.orchestra.run.smithi176.stdout:4632 write 0x95dc76f thru 0x95ec715 (0xffa7 bytes) 2024-09-14T09:07:01.692 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:07:01.701 INFO:teuthology.orchestra.run.smithi176.stdout:4633 punch from 0x22d3a9e to 0x22db027, (0x7589 bytes) 2024-09-14T09:07:01.701 INFO:teuthology.orchestra.run.smithi176.stdout:4634 read 0x7360201 thru 0x736e3ce (0xe1ce bytes) 2024-09-14T09:07:01.701 INFO:teuthology.orchestra.run.smithi176.stdout:4635 punch from 0x1c0c7ff to 0x1c1b782, (0xef83 bytes) 2024-09-14T09:07:01.701 INFO:teuthology.orchestra.run.smithi176.stdout:4636 punch from 0xb855d00 to 0xb85a504, (0x4804 bytes) 2024-09-14T09:07:01.701 INFO:teuthology.orchestra.run.smithi176.stdout:4639 writesame 0x8f1b0fc thru 0x8f1b36a data_size 0x1(0x26f bytes) 2024-09-14T09:07:01.702 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:01.703+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:01.708 INFO:teuthology.orchestra.run.smithi176.stdout:4640 punch from 0xce6e01c to 0xce7c2ac, (0xe290 bytes) 2024-09-14T09:07:01.708 INFO:teuthology.orchestra.run.smithi176.stdout:4641 punch from 0xc0f9910 to 0xc10192a, (0x801a bytes) 2024-09-14T09:07:01.708 INFO:teuthology.orchestra.run.smithi176.stdout:4643 write 0x6b2856d thru 0x6b300ef (0x7b83 bytes) 2024-09-14T09:07:01.727 INFO:teuthology.orchestra.run.smithi176.stdout:4644 write 0xe3add6d thru 0xe3adf0a (0x19e bytes) 2024-09-14T09:07:01.732 INFO:teuthology.orchestra.run.smithi176.stdout:4645 write 0x6e94503 thru 0x6e979a6 (0x34a4 bytes) 2024-09-14T09:07:01.740 INFO:teuthology.orchestra.run.smithi176.stdout:4646 write 0x1bf281b thru 0x1bfd507 (0xaced bytes) 2024-09-14T09:07:01.751 INFO:teuthology.orchestra.run.smithi176.stdout:4647 read 0x1114e46 thru 0x111abf7 (0x5db2 bytes) 2024-09-14T09:07:01.751 INFO:teuthology.orchestra.run.smithi176.stdout:4651 punch from 0xa3776b to 0xa389f2, (0x1287 bytes) 2024-09-14T09:07:01.751 INFO:teuthology.orchestra.run.smithi176.stdout:4652 write 0x459447f thru 0x459af93 (0x6b15 bytes) 2024-09-14T09:07:01.758 INFO:teuthology.orchestra.run.smithi176.stdout:4653 writesame 0x615ff30 thru 0x61672be data_size 0x39(0x738f bytes) 2024-09-14T09:07:01.766 INFO:teuthology.orchestra.run.smithi176.stdout:4655 read 0xab0cb26 thru 0xab110c5 (0x45a0 bytes) 2024-09-14T09:07:01.766 INFO:teuthology.orchestra.run.smithi176.stdout:4658 write 0xd3992db thru 0xd3a1602 (0x8328 bytes) 2024-09-14T09:07:01.777 INFO:teuthology.orchestra.run.smithi176.stdout:4659 read 0x4a5bc93 thru 0x4a6a810 (0xeb7e bytes) 2024-09-14T09:07:01.777 INFO:teuthology.orchestra.run.smithi176.stdout:4660 read 0xdb2689d thru 0xdb35418 (0xeb7c bytes) 2024-09-14T09:07:01.777 INFO:teuthology.orchestra.run.smithi176.stdout:4661 read 0x3416bbd thru 0x341f337 (0x877b bytes) 2024-09-14T09:07:01.777 INFO:teuthology.orchestra.run.smithi176.stdout:4662 write 0x153bbb9 thru 0x153ff66 (0x43ae bytes) 2024-09-14T09:07:01.784 INFO:teuthology.orchestra.run.smithi176.stdout:4663 read 0x507b33f thru 0x508af1a (0xfbdc bytes) 2024-09-14T09:07:01.784 INFO:teuthology.orchestra.run.smithi176.stdout:4664 write 0xd989175 thru 0xd990bcc (0x7a58 bytes) 2024-09-14T09:07:01.794 INFO:teuthology.orchestra.run.smithi176.stdout:4665 writesame 0x852735c thru 0x853016a data_size 0x1(0x8e0f bytes) 2024-09-14T09:07:01.803 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:01.803+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:01.808 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:01.808 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:01.834 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:07:01.842 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:01.842 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:01.860 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:01.861 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:01.878 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:01.878 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:01.885 INFO:teuthology.orchestra.run.smithi176.stdout:4666 punch from 0x81e144a to 0x81efbbe, (0xe774 bytes) 2024-09-14T09:07:01.890 INFO:teuthology.orchestra.run.smithi176.stdout:4667 trunc from 0xe3adf0b to 0xa710665 2024-09-14T09:07:01.896 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:01.896 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:01.904 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:01.907+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:01.918 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:07:01.918 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:07:01.920 INFO:teuthology.orchestra.run.smithi176.stdout:4668 trunc from 0xa710665 to 0x270ebdd 2024-09-14T09:07:01.941 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:01.941 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:01.955 INFO:teuthology.orchestra.run.smithi176.stdout:4669 writesame 0x76288be thru 0x762eb60 data_size 0x13(0x62a3 bytes) 2024-09-14T09:07:01.955 INFO:teuthology.orchestra.run.smithi176.stdout:4670 punch from 0x71991f4 to 0x719a3c7, (0x11d3 bytes) 2024-09-14T09:07:01.955 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:01.955 INFO:teuthology.orchestra.run.smithi176.stdout:truncating to largest ever: 0xee653c5 2024-09-14T09:07:01.963 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:01.963 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:01.965 INFO:teuthology.orchestra.run.smithi176.stdout:4672 trunc from 0x762eb61 to 0xee653c5 2024-09-14T09:07:01.972 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:01.973 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:01.973 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:01.981 INFO:teuthology.orchestra.run.smithi176.stdout:4673 trunc from 0xee653c5 to 0xa0132ff 2024-09-14T09:07:01.987 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:01.987 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:01.991 INFO:teuthology.orchestra.run.smithi176.stdout:4675 read 0x38c00c6 thru 0x38cc8c4 (0xc7ff bytes) 2024-09-14T09:07:01.991 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:01.991 INFO:teuthology.orchestra.run.smithi176.stdout:4678 write 0x641d8be thru 0x642b466 (0xdba9 bytes) 2024-09-14T09:07:01.994 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:07:02.009 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:02.007+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:02.017 INFO:teuthology.orchestra.run.smithi176.stdout:4680 write 0xda60e2b thru 0xda63677 (0x284d bytes) 2024-09-14T09:07:02.023 INFO:teuthology.orchestra.run.smithi176.stdout:4681 read 0x45c8dd7 thru 0x45d3e55 (0xb07f bytes) 2024-09-14T09:07:02.023 INFO:teuthology.orchestra.run.smithi176.stdout:4682 punch from 0x53d6f91 to 0x53d99b7, (0x2a26 bytes) 2024-09-14T09:07:02.023 INFO:teuthology.orchestra.run.smithi176.stdout:4683 read 0xaf209de thru 0xaf2f73f (0xed62 bytes) 2024-09-14T09:07:02.025 INFO:teuthology.orchestra.run.smithi176.stdout:4684 trunc from 0xda63678 to 0xc3b36b7 2024-09-14T09:07:02.042 INFO:teuthology.orchestra.run.smithi176.stdout:4686 read 0x9ee8046 thru 0x9eec29f (0x425a bytes) 2024-09-14T09:07:02.046 INFO:teuthology.orchestra.run.smithi176.stdout:4687 trunc from 0xc3b36b7 to 0x9a36c81 2024-09-14T09:07:02.054 INFO:teuthology.orchestra.run.smithi176.stdout:4688 clone 33 order 18 su 131072 sc 9 2024-09-14T09:07:02.106 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:02.107+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:02.137 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:02.137 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:02.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:02.137 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:02.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.137 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20858 3.17 3:ed5052d1:::rbd_data.115e889d4809.000000000000015c:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:02.137 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:07:02.137 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 74.706096942000002, 2024-09-14T09:07:02.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053381794000000003, 2024-09-14T09:07:02.137 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.137 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.137 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:02.137 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.137 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.138 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.138 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20858 2024-09-14T09:07:02.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.138 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:07:02.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:07:02.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429351+0000", 2024-09-14T09:07:02.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1149999999999998e-06 2024-09-14T09:07:02.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429363+0000", 2024-09-14T09:07:02.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1819e-05 2024-09-14T09:07:02.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429389+0000", 2024-09-14T09:07:02.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5605e-05 2024-09-14T09:07:02.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429402+0000", 2024-09-14T09:07:02.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3175e-05 2024-09-14T09:07:02.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429492+0000", 2024-09-14T09:07:02.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.988e-05 2024-09-14T09:07:02.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429597+0000", 2024-09-14T09:07:02.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010564 2024-09-14T09:07:02.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:07:02.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.433465+0000", 2024-09-14T09:07:02.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0038674249999999999 2024-09-14T09:07:02.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:02.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482483+0000", 2024-09-14T09:07:02.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049017999999999999 2024-09-14T09:07:02.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:02.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482625+0000", 2024-09-14T09:07:02.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000142447 2024-09-14T09:07:02.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:02.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482684+0000", 2024-09-14T09:07:02.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8522e-05 2024-09-14T09:07:02.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482730+0000", 2024-09-14T09:07:02.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6165999999999998e-05 2024-09-14T09:07:02.143 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.143 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.143 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.143 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:20858 3.17 e41/23 ondisk, result = 0)", 2024-09-14T09:07:02.143 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:07:02.143 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 74.691337498999999, 2024-09-14T09:07:02.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038631402000000002, 2024-09-14T09:07:02.144 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.144 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.144 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.144 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:07:02.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:07:02.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444109+0000", 2024-09-14T09:07:02.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.331e-06 2024-09-14T09:07:02.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444114+0000", 2024-09-14T09:07:02.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3709999999999999e-06 2024-09-14T09:07:02.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444130+0000", 2024-09-14T09:07:02.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6079e-05 2024-09-14T09:07:02.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444144+0000", 2024-09-14T09:07:02.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3331e-05 2024-09-14T09:07:02.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482538+0000", 2024-09-14T09:07:02.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038394878 2024-09-14T09:07:02.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482548+0000", 2024-09-14T09:07:02.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0093e-05 2024-09-14T09:07:02.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482739+0000", 2024-09-14T09:07:02.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019031899999999999 2024-09-14T09:07:02.147 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.147 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.147 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.148 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:37800 3.1 3:830c4670:::rbd_data.115eb07a8abd.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:07:02.148 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:07:02.148 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 65.104889532000001, 2024-09-14T09:07:02.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.068040724999999996, 2024-09-14T09:07:02.148 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.148 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.148 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:02.148 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.148 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.149 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.149 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 37800 2024-09-14T09:07:02.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.149 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:07:02.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:07:02.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030557+0000", 2024-09-14T09:07:02.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.888e-06 2024-09-14T09:07:02.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030566+0000", 2024-09-14T09:07:02.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8540000000000003e-06 2024-09-14T09:07:02.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030585+0000", 2024-09-14T09:07:02.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8878000000000001e-05 2024-09-14T09:07:02.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030593+0000", 2024-09-14T09:07:02.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5870000000000004e-06 2024-09-14T09:07:02.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030667+0000", 2024-09-14T09:07:02.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4207000000000003e-05 2024-09-14T09:07:02.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030761+0000", 2024-09-14T09:07:02.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4223999999999998e-05 2024-09-14T09:07:02.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:07:02.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.034400+0000", 2024-09-14T09:07:02.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0036387759999999998 2024-09-14T09:07:02.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:02.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098457+0000", 2024-09-14T09:07:02.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.064057501000000003 2024-09-14T09:07:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098518+0000", 2024-09-14T09:07:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0730999999999998e-05 2024-09-14T09:07:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098560+0000", 2024-09-14T09:07:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2267e-05 2024-09-14T09:07:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098596+0000", 2024-09-14T09:07:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5812000000000002e-05 2024-09-14T09:07:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:37800 3.1 e48/34 ondisk, result = 0)", 2024-09-14T09:07:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:07:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 65.093897916000003, 2024-09-14T09:07:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057056260999999997, 2024-09-14T09:07:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:07:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:07:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041548+0000", 2024-09-14T09:07:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8100000000000002e-07 2024-09-14T09:07:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041552+0000", 2024-09-14T09:07:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7419999999999999e-06 2024-09-14T09:07:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041563+0000", 2024-09-14T09:07:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1926000000000001e-05 2024-09-14T09:07:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041572+0000", 2024-09-14T09:07:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8610000000000004e-06 2024-09-14T09:07:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098503+0000", 2024-09-14T09:07:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056930827000000003 2024-09-14T09:07:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098512+0000", 2024-09-14T09:07:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5019999999999992e-06 2024-09-14T09:07:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098603+0000", 2024-09-14T09:07:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1621999999999998e-05 2024-09-14T09:07:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38523 3.3 3:c62e0486:::rbd_data.115e9e133b92.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 4194304~28499] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T09:07:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:07:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 63.681470218000001, 2024-09-14T09:07:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035654406999999999, 2024-09-14T09:07:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38523 2024-09-14T09:07:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:07:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:07:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453977+0000", 2024-09-14T09:07:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6299999999999998e-06 2024-09-14T09:07:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455684+0000", 2024-09-14T09:07:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001706843 2024-09-14T09:07:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455710+0000", 2024-09-14T09:07:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6302e-05 2024-09-14T09:07:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455722+0000", 2024-09-14T09:07:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1798e-05 2024-09-14T09:07:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455825+0000", 2024-09-14T09:07:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102822 2024-09-14T09:07:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.456032+0000", 2024-09-14T09:07:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020704400000000001 2024-09-14T09:07:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:07:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.456759+0000", 2024-09-14T09:07:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00072735699999999998 2024-09-14T09:07:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.477856+0000", 2024-09-14T09:07:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021096011000000001 2024-09-14T09:07:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489526+0000", 2024-09-14T09:07:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011670313 2024-09-14T09:07:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489572+0000", 2024-09-14T09:07:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6408e-05 2024-09-14T09:07:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489629+0000", 2024-09-14T09:07:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6879000000000002e-05 2024-09-14T09:07:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49719 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:07:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:07:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.609811616999998, 2024-09-14T09:07:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034680563999999997, 2024-09-14T09:07:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49719 2024-09-14T09:07:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:07:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:07:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525634+0000", 2024-09-14T09:07:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4000000000000001e-07 2024-09-14T09:07:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525638+0000", 2024-09-14T09:07:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5899999999999999e-06 2024-09-14T09:07:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525647+0000", 2024-09-14T09:07:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2029999999999998e-06 2024-09-14T09:07:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525653+0000", 2024-09-14T09:07:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9850000000000003e-06 2024-09-14T09:07:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560150+0000", 2024-09-14T09:07:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034496845999999998 2024-09-14T09:07:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560221+0000", 2024-09-14T09:07:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1400000000000001e-05 2024-09-14T09:07:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560314+0000", 2024-09-14T09:07:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2800000000000006e-05 2024-09-14T09:07:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49720 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:07:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:07:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.609559992999998, 2024-09-14T09:07:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034919095999999997, 2024-09-14T09:07:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49720 2024-09-14T09:07:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:07:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:07:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525886+0000", 2024-09-14T09:07:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6000000000000002e-07 2024-09-14T09:07:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525890+0000", 2024-09-14T09:07:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4519999999999999e-06 2024-09-14T09:07:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525899+0000", 2024-09-14T09:07:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1090000000000004e-06 2024-09-14T09:07:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525906+0000", 2024-09-14T09:07:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0219999999999999e-06 2024-09-14T09:07:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560344+0000", 2024-09-14T09:07:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034437970999999998 2024-09-14T09:07:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560422+0000", 2024-09-14T09:07:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7857999999999999e-05 2024-09-14T09:07:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560804+0000", 2024-09-14T09:07:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00038182400000000001 2024-09-14T09:07:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49721 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:07:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:07:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.609512510999998, 2024-09-14T09:07:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035347380999999997, 2024-09-14T09:07:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49721 2024-09-14T09:07:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:07:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:07:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525933+0000", 2024-09-14T09:07:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T09:07:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525936+0000", 2024-09-14T09:07:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6759999999999999e-06 2024-09-14T09:07:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525943+0000", 2024-09-14T09:07:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2459999999999997e-06 2024-09-14T09:07:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525948+0000", 2024-09-14T09:07:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9429999999999999e-06 2024-09-14T09:07:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560845+0000", 2024-09-14T09:07:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034897354999999998 2024-09-14T09:07:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560925+0000", 2024-09-14T09:07:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9804999999999999e-05 2024-09-14T09:07:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561280+0000", 2024-09-14T09:07:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035480100000000002 2024-09-14T09:07:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49722 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3112960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:07:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:07:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.609488865000003, 2024-09-14T09:07:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035750491000000002, 2024-09-14T09:07:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49722 2024-09-14T09:07:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:07:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:07:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:07:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.96e-07 2024-09-14T09:07:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525958+0000", 2024-09-14T09:07:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.024e-06 2024-09-14T09:07:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525964+0000", 2024-09-14T09:07:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6699999999999999e-06 2024-09-14T09:07:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525969+0000", 2024-09-14T09:07:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3939999999999998e-06 2024-09-14T09:07:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561319+0000", 2024-09-14T09:07:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.03535013 2024-09-14T09:07:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561390+0000", 2024-09-14T09:07:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0920000000000005e-05 2024-09-14T09:07:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561707+0000", 2024-09-14T09:07:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031695699999999998 2024-09-14T09:07:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49723 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3506176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:07:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:07:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.609469881999999, 2024-09-14T09:07:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036113064, 2024-09-14T09:07:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49723 2024-09-14T09:07:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:07:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:07:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:07:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1699999999999999e-07 2024-09-14T09:07:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525977+0000", 2024-09-14T09:07:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9250000000000002e-06 2024-09-14T09:07:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525983+0000", 2024-09-14T09:07:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4620000000000004e-06 2024-09-14T09:07:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525987+0000", 2024-09-14T09:07:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.279e-06 2024-09-14T09:07:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561749+0000", 2024-09-14T09:07:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035762222000000003 2024-09-14T09:07:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561796+0000", 2024-09-14T09:07:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6959999999999998e-05 2024-09-14T09:07:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562088+0000", 2024-09-14T09:07:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029179899999999998 2024-09-14T09:07:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49724 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:07:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:07:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 51.609451726000003, 2024-09-14T09:07:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036449388999999999, 2024-09-14T09:07:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49724 2024-09-14T09:07:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:07:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:07:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525994+0000", 2024-09-14T09:07:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3099999999999998e-07 2024-09-14T09:07:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525996+0000", 2024-09-14T09:07:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.939e-06 2024-09-14T09:07:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526005+0000", 2024-09-14T09:07:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8430000000000005e-06 2024-09-14T09:07:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526010+0000", 2024-09-14T09:07:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9389999999999996e-06 2024-09-14T09:07:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562114+0000", 2024-09-14T09:07:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036103510999999998 2024-09-14T09:07:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562189+0000", 2024-09-14T09:07:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5162999999999998e-05 2024-09-14T09:07:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562443+0000", 2024-09-14T09:07:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000253563 2024-09-14T09:07:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:64935 3.12 3:4cd6d7cb:::rbd_data.115e21c31fcf.0000000000000049:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:07:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:07:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.384086377999999, 2024-09-14T09:07:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037353866999999999, 2024-09-14T09:07:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64935 2024-09-14T09:07:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:07:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:07:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751362+0000", 2024-09-14T09:07:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6899999999999998e-06 2024-09-14T09:07:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751378+0000", 2024-09-14T09:07:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6147e-05 2024-09-14T09:07:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751421+0000", 2024-09-14T09:07:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2691999999999997e-05 2024-09-14T09:07:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751444+0000", 2024-09-14T09:07:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3237000000000001e-05 2024-09-14T09:07:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751594+0000", 2024-09-14T09:07:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014947800000000001 2024-09-14T09:07:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751733+0000", 2024-09-14T09:07:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013945200000000001 2024-09-14T09:07:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.753315+0000", 2024-09-14T09:07:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0015815969999999999 2024-09-14T09:07:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788546+0000", 2024-09-14T09:07:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035231449999999997 2024-09-14T09:07:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788570+0000", 2024-09-14T09:07:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3991000000000001e-05 2024-09-14T09:07:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788645+0000", 2024-09-14T09:07:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4379000000000001e-05 2024-09-14T09:07:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788712+0000", 2024-09-14T09:07:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7754000000000002e-05 2024-09-14T09:07:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:71052 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:07:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:07:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.397877127000001, 2024-09-14T09:07:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14854635599999999, 2024-09-14T09:07:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 71052 2024-09-14T09:07:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:07:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:07:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737570+0000", 2024-09-14T09:07:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4770000000000002e-06 2024-09-14T09:07:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737584+0000", 2024-09-14T09:07:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3457e-05 2024-09-14T09:07:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737608+0000", 2024-09-14T09:07:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3893999999999999e-05 2024-09-14T09:07:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737620+0000", 2024-09-14T09:07:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2157000000000001e-05 2024-09-14T09:07:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737738+0000", 2024-09-14T09:07:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118409 2024-09-14T09:07:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737892+0000", 2024-09-14T09:07:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154222 2024-09-14T09:07:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:07:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.749622+0000", 2024-09-14T09:07:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011729537 2024-09-14T09:07:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885934+0000", 2024-09-14T09:07:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13631222200000001 2024-09-14T09:07:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885996+0000", 2024-09-14T09:07:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1555000000000003e-05 2024-09-14T09:07:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886060+0000", 2024-09-14T09:07:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3980999999999996e-05 2024-09-14T09:07:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886114+0000", 2024-09-14T09:07:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4444999999999998e-05 2024-09-14T09:07:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:71052 3.1 e71/34 ondisk, result = 0)", 2024-09-14T09:07:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:07:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.354973749000003, 2024-09-14T09:07:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10565334799999999, 2024-09-14T09:07:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:07:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:07:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780473+0000", 2024-09-14T09:07:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8980000000000001e-06 2024-09-14T09:07:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780482+0000", 2024-09-14T09:07:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9080000000000001e-06 2024-09-14T09:07:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780511+0000", 2024-09-14T09:07:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9490999999999999e-05 2024-09-14T09:07:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780527+0000", 2024-09-14T09:07:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.508e-05 2024-09-14T09:07:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885978+0000", 2024-09-14T09:07:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.105451328 2024-09-14T09:07:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885989+0000", 2024-09-14T09:07:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0875e-05 2024-09-14T09:07:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886125+0000", 2024-09-14T09:07:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013576800000000001 2024-09-14T09:07:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73871 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:1b [read 8339456~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:07:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:07:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.709488741000001, 2024-09-14T09:07:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034605047, 2024-09-14T09:07:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73871 2024-09-14T09:07:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:07:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:07:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425957+0000", 2024-09-14T09:07:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T09:07:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425961+0000", 2024-09-14T09:07:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8279999999999996e-06 2024-09-14T09:07:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425985+0000", 2024-09-14T09:07:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4182e-05 2024-09-14T09:07:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425997+0000", 2024-09-14T09:07:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.132e-05 2024-09-14T09:07:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460070+0000", 2024-09-14T09:07:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034072913000000003 2024-09-14T09:07:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460298+0000", 2024-09-14T09:07:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022819 2024-09-14T09:07:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460561+0000", 2024-09-14T09:07:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026341400000000001 2024-09-14T09:07:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:74276 3.12 3:49b683dd:::rbd_data.115ea8d131ff.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1376256~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:07:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:07:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.450341650999999, 2024-09-14T09:07:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036482939999999998, 2024-09-14T09:07:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74276 2024-09-14T09:07:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:07:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:07:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685105+0000", 2024-09-14T09:07:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9520000000000001e-06 2024-09-14T09:07:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686530+0000", 2024-09-14T09:07:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001424629 2024-09-14T09:07:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686550+0000", 2024-09-14T09:07:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0472e-05 2024-09-14T09:07:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686561+0000", 2024-09-14T09:07:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T09:07:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686671+0000", 2024-09-14T09:07:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110621 2024-09-14T09:07:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686919+0000", 2024-09-14T09:07:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024790199999999999 2024-09-14T09:07:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.687375+0000", 2024-09-14T09:07:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00045572100000000002 2024-09-14T09:07:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.710871+0000", 2024-09-14T09:07:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023496481999999999 2024-09-14T09:07:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721478+0000", 2024-09-14T09:07:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01060709 2024-09-14T09:07:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721534+0000", 2024-09-14T09:07:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5912999999999998e-05 2024-09-14T09:07:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721586+0000", 2024-09-14T09:07:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1792000000000002e-05 2024-09-14T09:07:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:84402 3.6 3:6123bacd:::rbd_data.115e77b799c3.0000000000000017:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:07:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:53.874733+0000", 2024-09-14T09:07:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 8.2607114310000007, 2024-09-14T09:07:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040401678000000003, 2024-09-14T09:07:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 84402 2024-09-14T09:07:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874733+0000", 2024-09-14T09:07:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874733+0000", 2024-09-14T09:07:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874735+0000", 2024-09-14T09:07:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3170000000000001e-06 2024-09-14T09:07:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874742+0000", 2024-09-14T09:07:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0119999999999999e-06 2024-09-14T09:07:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874758+0000", 2024-09-14T09:07:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5971e-05 2024-09-14T09:07:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874768+0000", 2024-09-14T09:07:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0257e-05 2024-09-14T09:07:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874816+0000", 2024-09-14T09:07:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8133999999999997e-05 2024-09-14T09:07:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874862+0000", 2024-09-14T09:07:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6269e-05 2024-09-14T09:07:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.876731+0000", 2024-09-14T09:07:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001868722 2024-09-14T09:07:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.914723+0000", 2024-09-14T09:07:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037991912000000003 2024-09-14T09:07:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.914839+0000", 2024-09-14T09:07:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115901 2024-09-14T09:07:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.915013+0000", 2024-09-14T09:07:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017389 2024-09-14T09:07:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.915135+0000", 2024-09-14T09:07:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012229299999999999 2024-09-14T09:07:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:95497 3.1 3:845f2eaa:::rbd_data.115e7770ddc5.000000000000000c:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:07:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.690856+0000", 2024-09-14T09:07:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.4445892379999998, 2024-09-14T09:07:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.066534455000000006, 2024-09-14T09:07:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 95497 2024-09-14T09:07:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.690856+0000", 2024-09-14T09:07:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.690856+0000", 2024-09-14T09:07:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.690860+0000", 2024-09-14T09:07:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0929999999999996e-06 2024-09-14T09:07:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.690876+0000", 2024-09-14T09:07:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6066000000000002e-05 2024-09-14T09:07:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.690932+0000", 2024-09-14T09:07:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6273000000000001e-05 2024-09-14T09:07:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.690955+0000", 2024-09-14T09:07:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.266e-05 2024-09-14T09:07:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.691057+0000", 2024-09-14T09:07:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010218899999999999 2024-09-14T09:07:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.691173+0000", 2024-09-14T09:07:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115617 2024-09-14T09:07:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:07:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.694376+0000", 2024-09-14T09:07:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032031960000000002 2024-09-14T09:07:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757283+0000", 2024-09-14T09:07:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062907413999999995 2024-09-14T09:07:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757294+0000", 2024-09-14T09:07:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0577999999999999e-05 2024-09-14T09:07:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757332+0000", 2024-09-14T09:07:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8457999999999997e-05 2024-09-14T09:07:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757390+0000", 2024-09-14T09:07:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7911000000000002e-05 2024-09-14T09:07:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:95498 3.1 3:8579c379:::rbd_data.115e7770ddc5.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:07:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.691002+0000", 2024-09-14T09:07:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.4444428230000002, 2024-09-14T09:07:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.066695790000000005, 2024-09-14T09:07:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 95498 2024-09-14T09:07:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.691002+0000", 2024-09-14T09:07:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.691002+0000", 2024-09-14T09:07:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.691003+0000", 2024-09-14T09:07:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0440000000000001e-06 2024-09-14T09:07:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.691010+0000", 2024-09-14T09:07:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7299999999999999e-06 2024-09-14T09:07:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.691024+0000", 2024-09-14T09:07:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3896e-05 2024-09-14T09:07:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.691038+0000", 2024-09-14T09:07:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4177e-05 2024-09-14T09:07:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.747580+0000", 2024-09-14T09:07:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056541705999999997 2024-09-14T09:07:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.747666+0000", 2024-09-14T09:07:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5884999999999995e-05 2024-09-14T09:07:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:07:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.748789+0000", 2024-09-14T09:07:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0011238979999999999 2024-09-14T09:07:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757403+0000", 2024-09-14T09:07:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0086136579999999997 2024-09-14T09:07:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757532+0000", 2024-09-14T09:07:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000128527 2024-09-14T09:07:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757644+0000", 2024-09-14T09:07:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011266599999999999 2024-09-14T09:07:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757698+0000", 2024-09-14T09:07:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3603000000000002e-05 2024-09-14T09:07:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:95497 3.1 e95/85 ondisk, result = 0)", 2024-09-14T09:07:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.701592+0000", 2024-09-14T09:07:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.4338530700000001, 2024-09-14T09:07:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055812796999999997, 2024-09-14T09:07:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.701592+0000", 2024-09-14T09:07:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.701592+0000", 2024-09-14T09:07:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.701593+0000", 2024-09-14T09:07:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.37e-06 2024-09-14T09:07:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.701599+0000", 2024-09-14T09:07:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8699999999999997e-06 2024-09-14T09:07:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.701623+0000", 2024-09-14T09:07:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3442e-05 2024-09-14T09:07:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.701634+0000", 2024-09-14T09:07:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1287000000000001e-05 2024-09-14T09:07:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757271+0000", 2024-09-14T09:07:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055636891000000001 2024-09-14T09:07:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757278+0000", 2024-09-14T09:07:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4259999999999997e-06 2024-09-14T09:07:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757405+0000", 2024-09-14T09:07:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000126511 2024-09-14T09:07:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.235 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:02.237 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:07:02.237 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:02.239+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:02.338 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:02.343+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:02.407 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:02.407 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:02.407 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:02.407 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:02.421 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:07:02.466 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:02.467+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:02.540 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:02.539+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:02.542 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:02.543 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:02.543 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:02.543 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:02.543 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:02.557 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:07:02.642 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:02.643+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:02.726 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73540 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15155200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:07:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.455751477, 2024-09-14T09:07:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.022930347, 2024-09-14T09:07:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73540 2024-09-14T09:07:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:07:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:07:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268665+0000", 2024-09-14T09:07:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6599999999999996e-07 2024-09-14T09:07:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268668+0000", 2024-09-14T09:07:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8760000000000001e-06 2024-09-14T09:07:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268676+0000", 2024-09-14T09:07:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7879999999999995e-06 2024-09-14T09:07:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268683+0000", 2024-09-14T09:07:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3180000000000003e-06 2024-09-14T09:07:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282017+0000", 2024-09-14T09:07:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01333409 2024-09-14T09:07:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282020+0000", 2024-09-14T09:07:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1609999999999999e-06 2024-09-14T09:07:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291158+0000", 2024-09-14T09:07:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0091376430000001 2024-09-14T09:07:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291383+0000", 2024-09-14T09:07:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022491000000000001 2024-09-14T09:07:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291595+0000", 2024-09-14T09:07:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021199500000000001 2024-09-14T09:07:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73541 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15237120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:07:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.455724057000001, 2024-09-14T09:07:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0232607330000001, 2024-09-14T09:07:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73541 2024-09-14T09:07:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:07:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:07:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:07:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5400000000000001e-07 2024-09-14T09:07:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268695+0000", 2024-09-14T09:07:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5799999999999999e-06 2024-09-14T09:07:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268703+0000", 2024-09-14T09:07:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5480000000000002e-06 2024-09-14T09:07:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268713+0000", 2024-09-14T09:07:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8640000000000008e-06 2024-09-14T09:07:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282032+0000", 2024-09-14T09:07:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013318831999999999 2024-09-14T09:07:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282035+0000", 2024-09-14T09:07:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4010000000000001e-06 2024-09-14T09:07:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291701+0000", 2024-09-14T09:07:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0096654199999999 2024-09-14T09:07:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291807+0000", 2024-09-14T09:07:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105852 2024-09-14T09:07:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291952+0000", 2024-09-14T09:07:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014568200000000001 2024-09-14T09:07:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73542 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15319040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.455654446, 2024-09-14T09:07:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0237091359999999, 2024-09-14T09:07:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73542 2024-09-14T09:07:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268762+0000", 2024-09-14T09:07:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268766+0000", 2024-09-14T09:07:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5880000000000002e-06 2024-09-14T09:07:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268776+0000", 2024-09-14T09:07:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0168e-05 2024-09-14T09:07:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268789+0000", 2024-09-14T09:07:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:07:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282053+0000", 2024-09-14T09:07:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013264496000000001 2024-09-14T09:07:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282057+0000", 2024-09-14T09:07:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6090000000000001e-06 2024-09-14T09:07:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292052+0000", 2024-09-14T09:07:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0099950070000001 2024-09-14T09:07:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292262+0000", 2024-09-14T09:07:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021009800000000001 2024-09-14T09:07:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292470+0000", 2024-09-14T09:07:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000208314 2024-09-14T09:07:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73543 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.455615655999999, 2024-09-14T09:07:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023985473, 2024-09-14T09:07:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73543 2024-09-14T09:07:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268801+0000", 2024-09-14T09:07:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268803+0000", 2024-09-14T09:07:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7439999999999999e-06 2024-09-14T09:07:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268812+0000", 2024-09-14T09:07:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7080000000000003e-06 2024-09-14T09:07:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268819+0000", 2024-09-14T09:07:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T09:07:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282068+0000", 2024-09-14T09:07:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013249540000000001 2024-09-14T09:07:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282072+0000", 2024-09-14T09:07:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0960000000000001e-06 2024-09-14T09:07:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292573+0000", 2024-09-14T09:07:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0105014560000001 2024-09-14T09:07:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292679+0000", 2024-09-14T09:07:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010545899999999999 2024-09-14T09:07:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292786+0000", 2024-09-14T09:07:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107052 2024-09-14T09:07:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73544 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.455587953999999, 2024-09-14T09:07:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0244830549999999, 2024-09-14T09:07:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73544 2024-09-14T09:07:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5599999999999995e-07 2024-09-14T09:07:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268831+0000", 2024-09-14T09:07:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.57e-06 2024-09-14T09:07:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268844+0000", 2024-09-14T09:07:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2811e-05 2024-09-14T09:07:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268851+0000", 2024-09-14T09:07:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9140000000000002e-06 2024-09-14T09:07:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282083+0000", 2024-09-14T09:07:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013232111 2024-09-14T09:07:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282086+0000", 2024-09-14T09:07:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2909999999999999e-06 2024-09-14T09:07:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292884+0000", 2024-09-14T09:07:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0107975069999999 2024-09-14T09:07:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293100+0000", 2024-09-14T09:07:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021639399999999999 2024-09-14T09:07:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293311+0000", 2024-09-14T09:07:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021090099999999999 2024-09-14T09:07:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73545 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15564800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.455536768999998, 2024-09-14T09:07:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0249746740000001, 2024-09-14T09:07:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73545 2024-09-14T09:07:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268880+0000", 2024-09-14T09:07:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5600000000000005e-07 2024-09-14T09:07:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268889+0000", 2024-09-14T09:07:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8559999999999996e-06 2024-09-14T09:07:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268913+0000", 2024-09-14T09:07:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3958e-05 2024-09-14T09:07:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268921+0000", 2024-09-14T09:07:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7479999999999999e-06 2024-09-14T09:07:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282098+0000", 2024-09-14T09:07:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013176297999999999 2024-09-14T09:07:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282101+0000", 2024-09-14T09:07:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2329999999999998e-06 2024-09-14T09:07:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293435+0000", 2024-09-14T09:07:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0113339459999999 2024-09-14T09:07:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293648+0000", 2024-09-14T09:07:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021307100000000001 2024-09-14T09:07:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293854+0000", 2024-09-14T09:07:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020580800000000001 2024-09-14T09:07:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73546 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15646720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.455482898, 2024-09-14T09:07:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0254846360000001, 2024-09-14T09:07:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73546 2024-09-14T09:07:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268934+0000", 2024-09-14T09:07:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7999999999999995e-07 2024-09-14T09:07:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268937+0000", 2024-09-14T09:07:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9840000000000001e-06 2024-09-14T09:07:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268947+0000", 2024-09-14T09:07:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0522e-05 2024-09-14T09:07:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268955+0000", 2024-09-14T09:07:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8059999999999995e-06 2024-09-14T09:07:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282112+0000", 2024-09-14T09:07:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013157189 2024-09-14T09:07:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282115+0000", 2024-09-14T09:07:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2949999999999998e-06 2024-09-14T09:07:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293968+0000", 2024-09-14T09:07:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0118530990000001 2024-09-14T09:07:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294197+0000", 2024-09-14T09:07:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228507 2024-09-14T09:07:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294417+0000", 2024-09-14T09:07:02.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000220554 2024-09-14T09:07:02.763 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.763 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.763 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.763 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73547 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.763 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:02.763 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.455450452000001, 2024-09-14T09:07:02.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.025970525, 2024-09-14T09:07:02.763 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.763 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73547 2024-09-14T09:07:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268966+0000", 2024-09-14T09:07:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3799999999999997e-07 2024-09-14T09:07:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268969+0000", 2024-09-14T09:07:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8739999999999999e-06 2024-09-14T09:07:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268979+0000", 2024-09-14T09:07:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9939999999999995e-06 2024-09-14T09:07:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268986+0000", 2024-09-14T09:07:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3459999999999996e-06 2024-09-14T09:07:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282127+0000", 2024-09-14T09:07:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01314039 2024-09-14T09:07:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282130+0000", 2024-09-14T09:07:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2219999999999998e-06 2024-09-14T09:07:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294540+0000", 2024-09-14T09:07:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124105880000001 2024-09-14T09:07:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294759+0000", 2024-09-14T09:07:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000218798 2024-09-14T09:07:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294936+0000", 2024-09-14T09:07:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000176675 2024-09-14T09:07:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73548 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15810560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.455419339999999, 2024-09-14T09:07:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0265045129999999, 2024-09-14T09:07:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73548 2024-09-14T09:07:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268997+0000", 2024-09-14T09:07:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7599999999999996e-07 2024-09-14T09:07:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269000+0000", 2024-09-14T09:07:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0580000000000002e-06 2024-09-14T09:07:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269010+0000", 2024-09-14T09:07:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124e-05 2024-09-14T09:07:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269018+0000", 2024-09-14T09:07:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5399999999999998e-06 2024-09-14T09:07:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282141+0000", 2024-09-14T09:07:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013123012 2024-09-14T09:07:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282144+0000", 2024-09-14T09:07:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.196e-06 2024-09-14T09:07:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295042+0000", 2024-09-14T09:07:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0128975689999999 2024-09-14T09:07:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295272+0000", 2024-09-14T09:07:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023048099999999999 2024-09-14T09:07:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295501+0000", 2024-09-14T09:07:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228657 2024-09-14T09:07:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73549 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15892480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.455373461000001, 2024-09-14T09:07:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0268354099999999, 2024-09-14T09:07:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73549 2024-09-14T09:07:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269043+0000", 2024-09-14T09:07:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6199999999999997e-07 2024-09-14T09:07:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269046+0000", 2024-09-14T09:07:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2480000000000001e-06 2024-09-14T09:07:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269057+0000", 2024-09-14T09:07:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0436e-05 2024-09-14T09:07:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269064+0000", 2024-09-14T09:07:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5179999999999998e-06 2024-09-14T09:07:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282155+0000", 2024-09-14T09:07:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013091190000000001 2024-09-14T09:07:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282159+0000", 2024-09-14T09:07:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.366e-06 2024-09-14T09:07:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295599+0000", 2024-09-14T09:07:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0134405559999999 2024-09-14T09:07:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295765+0000", 2024-09-14T09:07:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016575499999999999 2024-09-14T09:07:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295878+0000", 2024-09-14T09:07:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112679 2024-09-14T09:07:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73550 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15974400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.455340861, 2024-09-14T09:07:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027357657, 2024-09-14T09:07:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73550 2024-09-14T09:07:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269076+0000", 2024-09-14T09:07:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:07:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269079+0000", 2024-09-14T09:07:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:07:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269098+0000", 2024-09-14T09:07:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9698e-05 2024-09-14T09:07:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269106+0000", 2024-09-14T09:07:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6179999999999997e-06 2024-09-14T09:07:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282170+0000", 2024-09-14T09:07:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013063874 2024-09-14T09:07:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282173+0000", 2024-09-14T09:07:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1999999999999999e-06 2024-09-14T09:07:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295996+0000", 2024-09-14T09:07:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.013823151 2024-09-14T09:07:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296214+0000", 2024-09-14T09:07:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021817700000000001 2024-09-14T09:07:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296433+0000", 2024-09-14T09:07:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021829099999999999 2024-09-14T09:07:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73551 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.455299444000001, 2024-09-14T09:07:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027730276, 2024-09-14T09:07:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73551 2024-09-14T09:07:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269117+0000", 2024-09-14T09:07:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:07:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269120+0000", 2024-09-14T09:07:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.056e-06 2024-09-14T09:07:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269130+0000", 2024-09-14T09:07:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0084e-05 2024-09-14T09:07:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269137+0000", 2024-09-14T09:07:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1359999999999999e-06 2024-09-14T09:07:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282184+0000", 2024-09-14T09:07:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013046537 2024-09-14T09:07:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282188+0000", 2024-09-14T09:07:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3220000000000003e-06 2024-09-14T09:07:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296529+0000", 2024-09-14T09:07:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0143406109999999 2024-09-14T09:07:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296695+0000", 2024-09-14T09:07:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016604799999999999 2024-09-14T09:07:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296847+0000", 2024-09-14T09:07:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000151822 2024-09-14T09:07:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73552 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.455268864000001, 2024-09-14T09:07:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0281767500000001, 2024-09-14T09:07:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73552 2024-09-14T09:07:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:07:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269150+0000", 2024-09-14T09:07:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9119999999999998e-06 2024-09-14T09:07:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269160+0000", 2024-09-14T09:07:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6819999999999998e-06 2024-09-14T09:07:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269168+0000", 2024-09-14T09:07:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6079999999999998e-06 2024-09-14T09:07:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282205+0000", 2024-09-14T09:07:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013036982000000001 2024-09-14T09:07:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282208+0000", 2024-09-14T09:07:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4620000000000001e-06 2024-09-14T09:07:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296968+0000", 2024-09-14T09:07:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0147601209999999 2024-09-14T09:07:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297115+0000", 2024-09-14T09:07:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014673900000000001 2024-09-14T09:07:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297324+0000", 2024-09-14T09:07:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020866400000000001 2024-09-14T09:07:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73553 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.4552388, 2024-09-14T09:07:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0286556739999999, 2024-09-14T09:07:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.795 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.795 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.795 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.795 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73553 2024-09-14T09:07:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269178+0000", 2024-09-14T09:07:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:07:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269180+0000", 2024-09-14T09:07:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T09:07:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269189+0000", 2024-09-14T09:07:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2e-06 2024-09-14T09:07:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269197+0000", 2024-09-14T09:07:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2980000000000003e-06 2024-09-14T09:07:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282219+0000", 2024-09-14T09:07:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013022687 2024-09-14T09:07:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282223+0000", 2024-09-14T09:07:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.241e-06 2024-09-14T09:07:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297470+0000", 2024-09-14T09:07:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.015247778 2024-09-14T09:07:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297704+0000", 2024-09-14T09:07:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000233367 2024-09-14T09:07:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297833+0000", 2024-09-14T09:07:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012888900000000001 2024-09-14T09:07:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73554 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.455209483000001, 2024-09-14T09:07:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.029005113, 2024-09-14T09:07:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73554 2024-09-14T09:07:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269207+0000", 2024-09-14T09:07:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.68e-07 2024-09-14T09:07:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269210+0000", 2024-09-14T09:07:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8100000000000002e-06 2024-09-14T09:07:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269219+0000", 2024-09-14T09:07:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:07:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269227+0000", 2024-09-14T09:07:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3839999999999999e-06 2024-09-14T09:07:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282234+0000", 2024-09-14T09:07:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013007187 2024-09-14T09:07:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282237+0000", 2024-09-14T09:07:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1200000000000002e-06 2024-09-14T09:07:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297942+0000", 2024-09-14T09:07:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0157048040000001 2024-09-14T09:07:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298087+0000", 2024-09-14T09:07:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014499099999999999 2024-09-14T09:07:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298211+0000", 2024-09-14T09:07:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012461899999999999 2024-09-14T09:07:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73555 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.455179635, 2024-09-14T09:07:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.02941558, 2024-09-14T09:07:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73555 2024-09-14T09:07:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269237+0000", 2024-09-14T09:07:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3399999999999999e-07 2024-09-14T09:07:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269240+0000", 2024-09-14T09:07:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8480000000000001e-06 2024-09-14T09:07:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269249+0000", 2024-09-14T09:07:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1880000000000008e-06 2024-09-14T09:07:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269256+0000", 2024-09-14T09:07:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:07:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282248+0000", 2024-09-14T09:07:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012992327 2024-09-14T09:07:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282251+0000", 2024-09-14T09:07:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2160000000000002e-06 2024-09-14T09:07:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298299+0000", 2024-09-14T09:07:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0160479330000001 2024-09-14T09:07:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298446+0000", 2024-09-14T09:07:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000146525 2024-09-14T09:07:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298652+0000", 2024-09-14T09:07:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020584100000000001 2024-09-14T09:07:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73556 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.455149889000001, 2024-09-14T09:07:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0299854239999999, 2024-09-14T09:07:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.812 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.812 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73556 2024-09-14T09:07:02.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.812 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:02.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:02.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:02.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:02.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269270+0000", 2024-09-14T09:07:02.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0920000000000002e-06 2024-09-14T09:07:02.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269280+0000", 2024-09-14T09:07:02.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0013999999999999e-05 2024-09-14T09:07:02.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269288+0000", 2024-09-14T09:07:02.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:07:02.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282263+0000", 2024-09-14T09:07:02.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012974703000000001 2024-09-14T09:07:02.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282266+0000", 2024-09-14T09:07:02.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1520000000000001e-06 2024-09-14T09:07:02.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298729+0000", 2024-09-14T09:07:02.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.016463506 2024-09-14T09:07:02.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298884+0000", 2024-09-14T09:07:02.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015450800000000001 2024-09-14T09:07:02.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299251+0000", 2024-09-14T09:07:02.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036759899999999997 2024-09-14T09:07:02.816 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.816 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.816 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.816 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73557 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16547840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.816 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:02.816 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.455103239, 2024-09-14T09:07:02.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0307969480000001, 2024-09-14T09:07:02.816 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.816 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.817 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.817 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.817 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.817 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.817 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73557 2024-09-14T09:07:02.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.817 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:02.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:02.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269313+0000", 2024-09-14T09:07:02.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.92e-07 2024-09-14T09:07:02.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269336+0000", 2024-09-14T09:07:02.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3118000000000001e-05 2024-09-14T09:07:02.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269346+0000", 2024-09-14T09:07:02.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4069999999999999e-06 2024-09-14T09:07:02.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269352+0000", 2024-09-14T09:07:02.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5810000000000002e-06 2024-09-14T09:07:02.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282277+0000", 2024-09-14T09:07:02.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012924658 2024-09-14T09:07:02.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282280+0000", 2024-09-14T09:07:02.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2880000000000001e-06 2024-09-14T09:07:02.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299343+0000", 2024-09-14T09:07:02.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0170630190000001 2024-09-14T09:07:02.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299554+0000", 2024-09-14T09:07:02.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021085099999999999 2024-09-14T09:07:02.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300109+0000", 2024-09-14T09:07:02.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00055533399999999999 2024-09-14T09:07:02.821 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.821 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.821 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.821 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73558 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.821 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:02.821 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.455051615999999, 2024-09-14T09:07:02.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0314849340000001, 2024-09-14T09:07:02.821 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.821 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.821 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.821 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.822 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.822 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.822 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73558 2024-09-14T09:07:02.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.822 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:02.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:02.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269365+0000", 2024-09-14T09:07:02.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7400000000000003e-07 2024-09-14T09:07:02.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269367+0000", 2024-09-14T09:07:02.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4930000000000001e-06 2024-09-14T09:07:02.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269374+0000", 2024-09-14T09:07:02.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9519999999999997e-06 2024-09-14T09:07:02.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269379+0000", 2024-09-14T09:07:02.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9710000000000003e-06 2024-09-14T09:07:02.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282292+0000", 2024-09-14T09:07:02.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012912704000000001 2024-09-14T09:07:02.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282295+0000", 2024-09-14T09:07:02.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.179e-06 2024-09-14T09:07:02.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300195+0000", 2024-09-14T09:07:02.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.017899975 2024-09-14T09:07:02.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300345+0000", 2024-09-14T09:07:02.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014957499999999999 2024-09-14T09:07:02.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300849+0000", 2024-09-14T09:07:02.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050451100000000002 2024-09-14T09:07:02.826 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.826 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.826 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.826 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73559 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:02.826 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:02.826 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.455029206999999, 2024-09-14T09:07:02.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.031916343, 2024-09-14T09:07:02.826 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:02.826 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:02.826 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:02.826 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:02.826 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:02.827 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:02.827 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73559 2024-09-14T09:07:02.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.827 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:02.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:02.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:02.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:02.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:02.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:02.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:02.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:02.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3500000000000002e-07 2024-09-14T09:07:02.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:02.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269390+0000", 2024-09-14T09:07:02.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5890000000000001e-06 2024-09-14T09:07:02.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:02.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269396+0000", 2024-09-14T09:07:02.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3300000000000004e-06 2024-09-14T09:07:02.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:02.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269401+0000", 2024-09-14T09:07:02.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0340000000000004e-06 2024-09-14T09:07:02.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282311+0000", 2024-09-14T09:07:02.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012910204 2024-09-14T09:07:02.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:02.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282317+0000", 2024-09-14T09:07:02.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2639999999999997e-06 2024-09-14T09:07:02.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:02.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300933+0000", 2024-09-14T09:07:02.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0186160710000001 2024-09-14T09:07:02.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:02.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301158+0000", 2024-09-14T09:07:02.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225021 2024-09-14T09:07:02.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:02.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:02.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:02.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301303+0000", 2024-09-14T09:07:02.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014439499999999999 2024-09-14T09:07:02.831 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.831 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.831 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.831 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:02.831 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:02.831 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:02.832 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:07:02.833 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:02.831+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:02.933 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:02.935+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:02.993 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T09:07:03.008 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:03.008 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:03.008 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:03.008 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:03.023 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:07:03.033 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:03.035+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:03.134 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:03.135+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:03.189 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:03.189 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:03.189 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:03.189 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:03.189 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:03.204 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:07:03.246 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:03.247+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:03.336 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:03.335+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:03.371 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:03.371 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:03.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:03.371 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:03.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.371 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18383 3.9 3:9252f7c9:::rbd_data.115e3abd3b2c.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:07:03.372 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:07:03.372 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.211948352999997, 2024-09-14T09:07:03.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056433830999999997, 2024-09-14T09:07:03.372 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.372 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.372 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:03.372 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.372 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.372 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.372 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18383 2024-09-14T09:07:03.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.372 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:07:03.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:07:03.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158423+0000", 2024-09-14T09:07:03.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T09:07:03.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158429+0000", 2024-09-14T09:07:03.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0179999999999998e-06 2024-09-14T09:07:03.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158440+0000", 2024-09-14T09:07:03.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0769e-05 2024-09-14T09:07:03.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158445+0000", 2024-09-14T09:07:03.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5840000000000002e-06 2024-09-14T09:07:03.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158495+0000", 2024-09-14T09:07:03.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9129999999999999e-05 2024-09-14T09:07:03.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158598+0000", 2024-09-14T09:07:03.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102938 2024-09-14T09:07:03.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:03.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.161048+0000", 2024-09-14T09:07:03.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002450336 2024-09-14T09:07:03.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:03.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214605+0000", 2024-09-14T09:07:03.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053557526000000001 2024-09-14T09:07:03.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:03.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214634+0000", 2024-09-14T09:07:03.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8483000000000002e-05 2024-09-14T09:07:03.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:03.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214731+0000", 2024-09-14T09:07:03.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7026000000000002e-05 2024-09-14T09:07:03.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214856+0000", 2024-09-14T09:07:03.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012474799999999999 2024-09-14T09:07:03.378 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.378 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.378 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.378 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18383 3.9 e38/34 ondisk, result = 0)", 2024-09-14T09:07:03.378 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:07:03.378 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.201631871999993, 2024-09-14T09:07:03.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045880575999999999, 2024-09-14T09:07:03.378 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.378 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.379 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.379 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:07:03.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:07:03.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168739+0000", 2024-09-14T09:07:03.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3900000000000004e-07 2024-09-14T09:07:03.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168744+0000", 2024-09-14T09:07:03.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5929999999999999e-06 2024-09-14T09:07:03.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168754+0000", 2024-09-14T09:07:03.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0686999999999999e-05 2024-09-14T09:07:03.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168761+0000", 2024-09-14T09:07:03.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6340000000000003e-06 2024-09-14T09:07:03.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214578+0000", 2024-09-14T09:07:03.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045817443999999999 2024-09-14T09:07:03.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214596+0000", 2024-09-14T09:07:03.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7159e-05 2024-09-14T09:07:03.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214619+0000", 2024-09-14T09:07:03.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.332e-05 2024-09-14T09:07:03.382 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.382 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.382 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.383 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19152 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14159058~8940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:07:03.383 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:07:03.383 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 79.047719416000007, 2024-09-14T09:07:03.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12891994500000001, 2024-09-14T09:07:03.383 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.383 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.383 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:03.383 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.383 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.383 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.384 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19152 2024-09-14T09:07:03.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.384 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:07:03.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:07:03.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322653+0000", 2024-09-14T09:07:03.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3750000000000001e-06 2024-09-14T09:07:03.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329647+0000", 2024-09-14T09:07:03.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069941960000000003 2024-09-14T09:07:03.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329673+0000", 2024-09-14T09:07:03.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5760000000000001e-05 2024-09-14T09:07:03.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329684+0000", 2024-09-14T09:07:03.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0474e-05 2024-09-14T09:07:03.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329776+0000", 2024-09-14T09:07:03.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2467999999999996e-05 2024-09-14T09:07:03.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329987+0000", 2024-09-14T09:07:03.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021133 2024-09-14T09:07:03.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:03.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.331303+0000", 2024-09-14T09:07:03.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001315629 2024-09-14T09:07:03.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:03.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.411090+0000", 2024-09-14T09:07:03.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079787477999999995 2024-09-14T09:07:03.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:03.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451463+0000", 2024-09-14T09:07:03.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040372312 2024-09-14T09:07:03.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:03.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451523+0000", 2024-09-14T09:07:03.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0420000000000001e-05 2024-09-14T09:07:03.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451571+0000", 2024-09-14T09:07:03.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7503000000000002e-05 2024-09-14T09:07:03.389 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.389 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.389 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.389 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20926 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:03.389 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:03.389 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.698387416000003, 2024-09-14T09:07:03.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018572207, 2024-09-14T09:07:03.389 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.389 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.390 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.390 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.390 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.390 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.390 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20926 2024-09-14T09:07:03.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.390 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:03.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:03.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:03.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:07:03.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671990+0000", 2024-09-14T09:07:03.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5769999999999999e-06 2024-09-14T09:07:03.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672001+0000", 2024-09-14T09:07:03.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1293e-05 2024-09-14T09:07:03.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672008+0000", 2024-09-14T09:07:03.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9099999999999999e-06 2024-09-14T09:07:03.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689147+0000", 2024-09-14T09:07:03.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017139060000000001 2024-09-14T09:07:03.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689324+0000", 2024-09-14T09:07:03.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017640200000000001 2024-09-14T09:07:03.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690555+0000", 2024-09-14T09:07:03.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001231212 2024-09-14T09:07:03.393 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.394 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.394 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.394 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20927 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2621440~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:03.394 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:03.394 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.698350450000007, 2024-09-14T09:07:03.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020002227000000001, 2024-09-14T09:07:03.394 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.394 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.394 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.395 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20927 2024-09-14T09:07:03.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.395 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:03.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:03.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:03.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9699999999999995e-07 2024-09-14T09:07:03.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672024+0000", 2024-09-14T09:07:03.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:07:03.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672036+0000", 2024-09-14T09:07:03.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.119e-05 2024-09-14T09:07:03.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672044+0000", 2024-09-14T09:07:03.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8370000000000003e-06 2024-09-14T09:07:03.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690602+0000", 2024-09-14T09:07:03.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018557648 2024-09-14T09:07:03.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690805+0000", 2024-09-14T09:07:03.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020282899999999999 2024-09-14T09:07:03.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692022+0000", 2024-09-14T09:07:03.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012170429999999999 2024-09-14T09:07:03.398 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.398 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.398 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.398 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20928 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 917504~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:03.398 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:07:03.399 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.698314310000001, 2024-09-14T09:07:03.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021623738, 2024-09-14T09:07:03.399 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.399 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.399 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.399 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.399 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.399 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.399 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20928 2024-09-14T09:07:03.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.399 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:07:03.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:07:03.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672057+0000", 2024-09-14T09:07:03.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:07:03.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672059+0000", 2024-09-14T09:07:03.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9270000000000001e-06 2024-09-14T09:07:03.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672070+0000", 2024-09-14T09:07:03.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0773e-05 2024-09-14T09:07:03.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672077+0000", 2024-09-14T09:07:03.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9600000000000003e-06 2024-09-14T09:07:03.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692154+0000", 2024-09-14T09:07:03.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020076482 2024-09-14T09:07:03.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692419+0000", 2024-09-14T09:07:03.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026517400000000002 2024-09-14T09:07:03.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693680+0000", 2024-09-14T09:07:03.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012607390000000001 2024-09-14T09:07:03.403 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.403 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.403 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.403 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20929 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2752512~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:03.403 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:07:03.403 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.698281268000002, 2024-09-14T09:07:03.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022898728, 2024-09-14T09:07:03.403 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.403 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.403 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.404 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.404 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.404 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.404 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20929 2024-09-14T09:07:03.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.404 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:07:03.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:07:03.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672090+0000", 2024-09-14T09:07:03.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:07:03.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672092+0000", 2024-09-14T09:07:03.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8940000000000002e-06 2024-09-14T09:07:03.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672103+0000", 2024-09-14T09:07:03.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0815999999999999e-05 2024-09-14T09:07:03.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672110+0000", 2024-09-14T09:07:03.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9469999999999997e-06 2024-09-14T09:07:03.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693737+0000", 2024-09-14T09:07:03.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021626467999999999 2024-09-14T09:07:03.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693962+0000", 2024-09-14T09:07:03.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225474 2024-09-14T09:07:03.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.694988+0000", 2024-09-14T09:07:03.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001025469 2024-09-14T09:07:03.409 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.409 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.409 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.409 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20930 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:03.409 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:07:03.409 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.698248174, 2024-09-14T09:07:03.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024303286, 2024-09-14T09:07:03.409 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.409 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.409 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.409 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.410 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.410 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.410 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20930 2024-09-14T09:07:03.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.410 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:07:03.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:07:03.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672123+0000", 2024-09-14T09:07:03.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:07:03.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672125+0000", 2024-09-14T09:07:03.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7300000000000001e-06 2024-09-14T09:07:03.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672134+0000", 2024-09-14T09:07:03.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7930000000000003e-06 2024-09-14T09:07:03.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672141+0000", 2024-09-14T09:07:03.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9029999999999998e-06 2024-09-14T09:07:03.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695087+0000", 2024-09-14T09:07:03.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022946013000000001 2024-09-14T09:07:03.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695350+0000", 2024-09-14T09:07:03.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026266000000000001 2024-09-14T09:07:03.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696425+0000", 2024-09-14T09:07:03.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001075527 2024-09-14T09:07:03.413 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.413 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.413 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.414 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20931 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:03.414 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:07:03.414 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.698214945000004, 2024-09-14T09:07:03.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025331519, 2024-09-14T09:07:03.414 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.414 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.414 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.414 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.414 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.414 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.414 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20931 2024-09-14T09:07:03.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.414 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:07:03.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:07:03.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672156+0000", 2024-09-14T09:07:03.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4300000000000002e-07 2024-09-14T09:07:03.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672159+0000", 2024-09-14T09:07:03.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T09:07:03.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672170+0000", 2024-09-14T09:07:03.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1192999999999999e-05 2024-09-14T09:07:03.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672177+0000", 2024-09-14T09:07:03.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9059999999999996e-06 2024-09-14T09:07:03.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696532+0000", 2024-09-14T09:07:03.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024355291000000001 2024-09-14T09:07:03.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696755+0000", 2024-09-14T09:07:03.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022297699999999999 2024-09-14T09:07:03.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697487+0000", 2024-09-14T09:07:03.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00073171499999999997 2024-09-14T09:07:03.418 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.418 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.418 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.418 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20932 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1179648~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:03.418 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:03.418 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.698181501999997, 2024-09-14T09:07:03.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026104083, 2024-09-14T09:07:03.419 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.419 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.419 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.419 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.419 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.419 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.419 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20932 2024-09-14T09:07:03.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.419 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:03.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:03.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:03.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0699999999999996e-07 2024-09-14T09:07:03.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672192+0000", 2024-09-14T09:07:03.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7700000000000002e-06 2024-09-14T09:07:03.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672203+0000", 2024-09-14T09:07:03.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0956e-05 2024-09-14T09:07:03.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672211+0000", 2024-09-14T09:07:03.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3769999999999996e-06 2024-09-14T09:07:03.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697530+0000", 2024-09-14T09:07:03.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025318411999999998 2024-09-14T09:07:03.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697640+0000", 2024-09-14T09:07:03.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011045299999999999 2024-09-14T09:07:03.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698293+0000", 2024-09-14T09:07:03.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065240800000000005 2024-09-14T09:07:03.422 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.422 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.423 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.423 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20933 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3014656~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:03.423 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:07:03.423 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.698147298999999, 2024-09-14T09:07:03.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026558875999999999, 2024-09-14T09:07:03.423 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.423 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.423 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.423 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.423 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.423 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.423 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20933 2024-09-14T09:07:03.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.424 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:07:03.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:07:03.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672224+0000", 2024-09-14T09:07:03.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2399999999999997e-07 2024-09-14T09:07:03.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672234+0000", 2024-09-14T09:07:03.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0793e-05 2024-09-14T09:07:03.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672246+0000", 2024-09-14T09:07:03.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659e-05 2024-09-14T09:07:03.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672255+0000", 2024-09-14T09:07:03.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5730000000000008e-06 2024-09-14T09:07:03.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698326+0000", 2024-09-14T09:07:03.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026071414000000001 2024-09-14T09:07:03.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698374+0000", 2024-09-14T09:07:03.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8032999999999999e-05 2024-09-14T09:07:03.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698782+0000", 2024-09-14T09:07:03.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040768 2024-09-14T09:07:03.427 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.427 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.427 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.427 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20934 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1310720~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:03.427 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:07:03.427 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.698105423000001, 2024-09-14T09:07:03.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027292067999999999, 2024-09-14T09:07:03.428 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.428 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.428 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.428 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20934 2024-09-14T09:07:03.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.428 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:07:03.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:07:03.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672266+0000", 2024-09-14T09:07:03.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6299999999999993e-07 2024-09-14T09:07:03.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672269+0000", 2024-09-14T09:07:03.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8569999999999999e-06 2024-09-14T09:07:03.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672280+0000", 2024-09-14T09:07:03.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1236999999999999e-05 2024-09-14T09:07:03.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672287+0000", 2024-09-14T09:07:03.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8260000000000004e-06 2024-09-14T09:07:03.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698820+0000", 2024-09-14T09:07:03.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026533055999999999 2024-09-14T09:07:03.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698933+0000", 2024-09-14T09:07:03.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011291 2024-09-14T09:07:03.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699557+0000", 2024-09-14T09:07:03.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062421899999999997 2024-09-14T09:07:03.431 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.431 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.431 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.431 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20935 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:03.432 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:07:03.432 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.698071662999993, 2024-09-14T09:07:03.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028108610999999999, 2024-09-14T09:07:03.432 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.432 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.432 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.432 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.432 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.432 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.432 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20935 2024-09-14T09:07:03.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.432 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:07:03.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:07:03.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672299+0000", 2024-09-14T09:07:03.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T09:07:03.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672302+0000", 2024-09-14T09:07:03.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7640000000000001e-06 2024-09-14T09:07:03.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672313+0000", 2024-09-14T09:07:03.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1080000000000001e-05 2024-09-14T09:07:03.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672321+0000", 2024-09-14T09:07:03.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6490000000000008e-06 2024-09-14T09:07:03.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699587+0000", 2024-09-14T09:07:03.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027265884000000001 2024-09-14T09:07:03.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699631+0000", 2024-09-14T09:07:03.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4947999999999997e-05 2024-09-14T09:07:03.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700407+0000", 2024-09-14T09:07:03.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00077559299999999999 2024-09-14T09:07:03.436 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.436 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.436 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.436 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20936 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1441792~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:03.436 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:03.436 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.698037486999993, 2024-09-14T09:07:03.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028784750000000001, 2024-09-14T09:07:03.436 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.436 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.436 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.437 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20936 2024-09-14T09:07:03.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.437 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:03.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:03.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:03.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4700000000000001e-07 2024-09-14T09:07:03.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672336+0000", 2024-09-14T09:07:03.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6730000000000001e-06 2024-09-14T09:07:03.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672347+0000", 2024-09-14T09:07:03.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0710000000000001e-05 2024-09-14T09:07:03.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672354+0000", 2024-09-14T09:07:03.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4630000000000004e-06 2024-09-14T09:07:03.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700442+0000", 2024-09-14T09:07:03.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028087951 2024-09-14T09:07:03.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700557+0000", 2024-09-14T09:07:03.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011461800000000001 2024-09-14T09:07:03.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701117+0000", 2024-09-14T09:07:03.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056058799999999999 2024-09-14T09:07:03.440 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.440 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.441 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.441 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20937 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:03.441 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:03.441 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.698004361000002, 2024-09-14T09:07:03.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029493028000000001, 2024-09-14T09:07:03.441 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.441 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.441 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.441 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.441 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.441 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.442 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20937 2024-09-14T09:07:03.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.442 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:03.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:03.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:03.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T09:07:03.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672384+0000", 2024-09-14T09:07:03.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7127000000000001e-05 2024-09-14T09:07:03.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672395+0000", 2024-09-14T09:07:03.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759999999999999e-05 2024-09-14T09:07:03.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672402+0000", 2024-09-14T09:07:03.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0960000000000003e-06 2024-09-14T09:07:03.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701147+0000", 2024-09-14T09:07:03.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028744241 2024-09-14T09:07:03.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701246+0000", 2024-09-14T09:07:03.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9031999999999996e-05 2024-09-14T09:07:03.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701859+0000", 2024-09-14T09:07:03.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061310499999999997 2024-09-14T09:07:03.445 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.445 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.445 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.445 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20938 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:03.446 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:07:03.446 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.697957200999994, 2024-09-14T09:07:03.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030109219999999999, 2024-09-14T09:07:03.446 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.446 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.446 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.446 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.446 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.446 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.446 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20938 2024-09-14T09:07:03.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.446 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:07:03.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:07:03.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672414+0000", 2024-09-14T09:07:03.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-07 2024-09-14T09:07:03.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672417+0000", 2024-09-14T09:07:03.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:07:03.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672428+0000", 2024-09-14T09:07:03.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0873000000000001e-05 2024-09-14T09:07:03.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672435+0000", 2024-09-14T09:07:03.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:07:03.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701884+0000", 2024-09-14T09:07:03.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029449811999999999 2024-09-14T09:07:03.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701934+0000", 2024-09-14T09:07:03.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9703000000000002e-05 2024-09-14T09:07:03.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702522+0000", 2024-09-14T09:07:03.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00058811200000000001 2024-09-14T09:07:03.450 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.450 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.450 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.450 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20939 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:03.450 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:07:03.450 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.697925068000004, 2024-09-14T09:07:03.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030779259, 2024-09-14T09:07:03.450 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.450 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.450 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.451 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.451 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20939 2024-09-14T09:07:03.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.451 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:07:03.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:07:03.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672446+0000", 2024-09-14T09:07:03.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:07:03.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672449+0000", 2024-09-14T09:07:03.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7930000000000002e-06 2024-09-14T09:07:03.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672461+0000", 2024-09-14T09:07:03.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T09:07:03.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672471+0000", 2024-09-14T09:07:03.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0807e-05 2024-09-14T09:07:03.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702551+0000", 2024-09-14T09:07:03.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030079228 2024-09-14T09:07:03.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702664+0000", 2024-09-14T09:07:03.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011351800000000001 2024-09-14T09:07:03.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.703224+0000", 2024-09-14T09:07:03.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056026999999999997 2024-09-14T09:07:03.454 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.454 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.454 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.455 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:74298 3.9 3:95c8282a:::rbd_data.115ea8d131ff.0000000000000041:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:07:03.455 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:07:03.455 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.572615056, 2024-09-14T09:07:03.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042668064999999998, 2024-09-14T09:07:03.455 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.455 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.455 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:03.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.455 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74298 2024-09-14T09:07:03.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.456 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:07:03.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:07:03.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797758+0000", 2024-09-14T09:07:03.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9799999999999998e-06 2024-09-14T09:07:03.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797770+0000", 2024-09-14T09:07:03.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1667e-05 2024-09-14T09:07:03.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797797+0000", 2024-09-14T09:07:03.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7212999999999998e-05 2024-09-14T09:07:03.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797811+0000", 2024-09-14T09:07:03.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4256e-05 2024-09-14T09:07:03.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797882+0000", 2024-09-14T09:07:03.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0622000000000002e-05 2024-09-14T09:07:03.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797943+0000", 2024-09-14T09:07:03.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0893000000000001e-05 2024-09-14T09:07:03.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:03.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.799676+0000", 2024-09-14T09:07:03.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0017331619999999999 2024-09-14T09:07:03.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:03.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840109+0000", 2024-09-14T09:07:03.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040432994 2024-09-14T09:07:03.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:03.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840204+0000", 2024-09-14T09:07:03.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5247000000000005e-05 2024-09-14T09:07:03.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:03.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840346+0000", 2024-09-14T09:07:03.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014156499999999999 2024-09-14T09:07:03.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840423+0000", 2024-09-14T09:07:03.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7466000000000007e-05 2024-09-14T09:07:03.460 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.460 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.460 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.461 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:74298 3.9 e76/34 ondisk, result = 0)", 2024-09-14T09:07:03.461 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:07:03.461 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 27.566220413, 2024-09-14T09:07:03.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036289318000000001, 2024-09-14T09:07:03.461 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.461 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.461 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.461 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:07:03.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:07:03.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804151+0000", 2024-09-14T09:07:03.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6700000000000001e-06 2024-09-14T09:07:03.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804163+0000", 2024-09-14T09:07:03.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.148e-05 2024-09-14T09:07:03.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804197+0000", 2024-09-14T09:07:03.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3909999999999999e-05 2024-09-14T09:07:03.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804217+0000", 2024-09-14T09:07:03.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0287000000000001e-05 2024-09-14T09:07:03.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840176+0000", 2024-09-14T09:07:03.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035958641 2024-09-14T09:07:03.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840193+0000", 2024-09-14T09:07:03.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6986e-05 2024-09-14T09:07:03.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840439+0000", 2024-09-14T09:07:03.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024634400000000001 2024-09-14T09:07:03.465 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.465 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.465 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.465 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:93074 3.9 e94/84 3:925bb5bd:::rbd_data.115e7770ddc5.0000000000000001:head v 94'255, mlcod=94'252)", 2024-09-14T09:07:03.465 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:57.863727+0000", 2024-09-14T09:07:03.465 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.5066431529999997, 2024-09-14T09:07:03.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019329799000000002, 2024-09-14T09:07:03.465 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.465 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.465 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:03.465 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.863727+0000", 2024-09-14T09:07:03.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.863727+0000", 2024-09-14T09:07:03.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.863729+0000", 2024-09-14T09:07:03.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3980000000000002e-06 2024-09-14T09:07:03.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866073+0000", 2024-09-14T09:07:03.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0023435109999999999 2024-09-14T09:07:03.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866094+0000", 2024-09-14T09:07:03.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1444e-05 2024-09-14T09:07:03.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866105+0000", 2024-09-14T09:07:03.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0949e-05 2024-09-14T09:07:03.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866216+0000", 2024-09-14T09:07:03.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110961 2024-09-14T09:07:03.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866258+0000", 2024-09-14T09:07:03.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1903000000000002e-05 2024-09-14T09:07:03.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:03.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.882981+0000", 2024-09-14T09:07:03.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016723198000000002 2024-09-14T09:07:03.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.883057+0000", 2024-09-14T09:07:03.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5434999999999999e-05 2024-09-14T09:07:03.469 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.469 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.469 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.469 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.469 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.469 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:03.470 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:07:03.471 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:03.471+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:03.571 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:03.575+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:03.637 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:03.637 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:03.637 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:03.637 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:03.651 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:07:03.672 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:03.675+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:03.706 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:07:03.706 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:07:03.706 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:07:03.706 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 7 2024-09-14T09:07:03.771 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:03.771 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:03.771 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:03.771 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:03.771 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:03.772 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:03.775+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:03.785 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:07:03.872 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:03.871+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:03.906 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:03.906 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:03.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:03.906 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:03.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.906 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:03.906 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:03.906 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.820872187000006, 2024-09-14T09:07:03.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:07:03.906 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.906 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.907 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.907 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.907 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.907 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.907 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:07:03.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.907 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:03.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:03.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:07:03.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:07:03.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:07:03.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:07:03.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:07:03.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:07:03.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:07:03.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:07:03.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:07:03.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:07:03.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:03.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:07:03.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:07:03.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:07:03.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:07:03.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:07:03.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:07:03.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:07:03.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:07:03.911 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.911 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.911 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.911 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:03.911 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:03.911 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.820817528999996, 2024-09-14T09:07:03.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:07:03.912 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.912 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.912 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.912 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.912 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.912 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.912 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:07:03.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.912 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:03.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:03.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:03.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:07:03.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:07:03.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:07:03.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:07:03.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:07:03.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:07:03.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:07:03.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:07:03.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:07:03.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:03.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:07:03.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:07:03.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:07:03.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:07:03.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:07:03.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:07:03.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:07:03.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:07:03.916 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.916 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.916 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.916 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:03.916 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:03.916 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.820712154999995, 2024-09-14T09:07:03.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:07:03.917 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.917 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.917 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.917 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.917 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.917 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.917 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:07:03.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.917 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:03.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:03.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:07:03.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:07:03.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:07:03.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:07:03.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:07:03.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:07:03.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:07:03.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:07:03.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:07:03.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:07:03.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:03.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:07:03.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:07:03.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:07:03.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:07:03.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:07:03.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:07:03.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:07:03.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:07:03.921 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.921 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.921 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.921 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:03.921 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:03.921 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.820355391000007, 2024-09-14T09:07:03.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:07:03.922 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.922 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.922 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.922 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:07:03.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.922 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:03.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:03.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:07:03.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:07:03.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:07:03.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:07:03.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:07:03.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:07:03.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:07:03.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:07:03.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:07:03.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:07:03.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:03.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:07:03.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:07:03.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:07:03.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:07:03.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:07:03.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:07:03.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:07:03.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:07:03.926 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.926 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.926 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.926 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:03.926 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:03.926 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.820270652999994, 2024-09-14T09:07:03.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:07:03.927 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.927 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.927 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.927 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.927 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.927 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.927 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:07:03.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.927 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:03.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:03.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:07:03.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:07:03.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:07:03.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:07:03.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:07:03.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:07:03.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:07:03.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:07:03.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:07:03.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:07:03.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:03.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:07:03.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:07:03.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:07:03.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:07:03.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:07:03.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:07:03.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:07:03.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:07:03.931 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.931 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.931 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.931 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:03.931 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:03.931 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 90.820182966999994, 2024-09-14T09:07:03.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:07:03.931 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.931 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.931 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.931 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.931 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.931 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.931 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:07:03.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.932 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:03.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:03.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:07:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:07:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:07:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:07:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:07:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:07:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:07:03.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:07:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:07:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:07:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:07:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:07:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:07:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:07:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:07:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:07:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:07:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:07:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.935 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18388 3.18 3:19e2be55:::rbd_data.115e3abd3b2c.0000000000000021:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:07:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:07:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.684269655999998, 2024-09-14T09:07:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042293648000000003, 2024-09-14T09:07:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18388 2024-09-14T09:07:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.936 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:07:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:07:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220236+0000", 2024-09-14T09:07:03.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1339999999999998e-06 2024-09-14T09:07:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220244+0000", 2024-09-14T09:07:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1950000000000003e-06 2024-09-14T09:07:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220262+0000", 2024-09-14T09:07:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8017e-05 2024-09-14T09:07:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220270+0000", 2024-09-14T09:07:03.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9480000000000008e-06 2024-09-14T09:07:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220357+0000", 2024-09-14T09:07:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7185e-05 2024-09-14T09:07:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220576+0000", 2024-09-14T09:07:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021912200000000001 2024-09-14T09:07:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:03.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.222435+0000", 2024-09-14T09:07:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018592350000000001 2024-09-14T09:07:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262439+0000", 2024-09-14T09:07:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040003368999999997 2024-09-14T09:07:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262452+0000", 2024-09-14T09:07:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2941e-05 2024-09-14T09:07:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:03.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262490+0000", 2024-09-14T09:07:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8478e-05 2024-09-14T09:07:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262527+0000", 2024-09-14T09:07:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7024000000000003e-05 2024-09-14T09:07:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38002 3.1e 3:7df715aa:::rbd_data.115eb07a8abd.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:07:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:07:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.514751750000002, 2024-09-14T09:07:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.065148304000000004, 2024-09-14T09:07:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38002 2024-09-14T09:07:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:07:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:07:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389752+0000", 2024-09-14T09:07:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1829999999999999e-06 2024-09-14T09:07:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389758+0000", 2024-09-14T09:07:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0919999999999997e-06 2024-09-14T09:07:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389769+0000", 2024-09-14T09:07:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759e-05 2024-09-14T09:07:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389775+0000", 2024-09-14T09:07:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8819999999999998e-06 2024-09-14T09:07:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389824+0000", 2024-09-14T09:07:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8573999999999999e-05 2024-09-14T09:07:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389876+0000", 2024-09-14T09:07:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2643999999999999e-05 2024-09-14T09:07:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:07:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.392640+0000", 2024-09-14T09:07:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0027636409999999998 2024-09-14T09:07:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454672+0000", 2024-09-14T09:07:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062031703000000001 2024-09-14T09:07:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454765+0000", 2024-09-14T09:07:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2862000000000006e-05 2024-09-14T09:07:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454841+0000", 2024-09-14T09:07:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5857000000000002e-05 2024-09-14T09:07:03.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454900+0000", 2024-09-14T09:07:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9107000000000002e-05 2024-09-14T09:07:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:38002 3.1e e48/23 ondisk, result = 0)", 2024-09-14T09:07:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:07:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.504435161000004, 2024-09-14T09:07:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054841741999999999, 2024-09-14T09:07:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.947 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:07:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:07:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400069+0000", 2024-09-14T09:07:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.339e-06 2024-09-14T09:07:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400076+0000", 2024-09-14T09:07:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7800000000000003e-06 2024-09-14T09:07:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400093+0000", 2024-09-14T09:07:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6915999999999999e-05 2024-09-14T09:07:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400106+0000", 2024-09-14T09:07:03.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2785e-05 2024-09-14T09:07:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454713+0000", 2024-09-14T09:07:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054607041000000002 2024-09-14T09:07:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454724+0000", 2024-09-14T09:07:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1202e-05 2024-09-14T09:07:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454910+0000", 2024-09-14T09:07:03.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018567900000000001 2024-09-14T09:07:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38100 3.b 3:d0f67c24:::rbd_data.115eb07a8abd.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:07:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:07:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.258581500999995, 2024-09-14T09:07:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053701964999999997, 2024-09-14T09:07:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.951 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38100 2024-09-14T09:07:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:07:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:07:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645923+0000", 2024-09-14T09:07:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:07:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645928+0000", 2024-09-14T09:07:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0699999999999997e-06 2024-09-14T09:07:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645942+0000", 2024-09-14T09:07:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4069e-05 2024-09-14T09:07:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645951+0000", 2024-09-14T09:07:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3230000000000004e-06 2024-09-14T09:07:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646008+0000", 2024-09-14T09:07:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6583999999999999e-05 2024-09-14T09:07:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646089+0000", 2024-09-14T09:07:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1557999999999994e-05 2024-09-14T09:07:03.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:07:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.650355+0000", 2024-09-14T09:07:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004265525 2024-09-14T09:07:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699439+0000", 2024-09-14T09:07:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049083904999999997 2024-09-14T09:07:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699467+0000", 2024-09-14T09:07:03.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8198999999999999e-05 2024-09-14T09:07:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699552+0000", 2024-09-14T09:07:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T09:07:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699624+0000", 2024-09-14T09:07:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1725999999999997e-05 2024-09-14T09:07:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.956 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:07:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.743890980000003, 2024-09-14T09:07:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:07:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:07:03.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:07:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:07:03.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:07:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:07:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:07:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:07:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:07:03.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:07:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:07:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:07:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:07:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:07:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:07:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:07:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:07:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:07:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:07:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:07:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:07:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:07:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:07:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:07:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:39401 3.b e56/53)", 2024-09-14T09:07:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:07:03.962 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 57.717994914000002, 2024-09-14T09:07:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10443659600000001, 2024-09-14T09:07:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:07:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:07:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186509+0000", 2024-09-14T09:07:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2640000000000001e-06 2024-09-14T09:07:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186516+0000", 2024-09-14T09:07:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1070000000000001e-06 2024-09-14T09:07:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186533+0000", 2024-09-14T09:07:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7818000000000001e-05 2024-09-14T09:07:03.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186543+0000", 2024-09-14T09:07:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5170000000000005e-06 2024-09-14T09:07:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290885+0000", 2024-09-14T09:07:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.104342221 2024-09-14T09:07:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290893+0000", 2024-09-14T09:07:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2859999999999999e-06 2024-09-14T09:07:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290945+0000", 2024-09-14T09:07:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1382999999999999e-05 2024-09-14T09:07:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:84137 3.7 3:e15b49d9:::rbd_data.115e77b799c3.00000000000000e0:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:07:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.581143581999999, 2024-09-14T09:07:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11205707500000001, 2024-09-14T09:07:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:03.966 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 84137 2024-09-14T09:07:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323362+0000", 2024-09-14T09:07:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0839999999999999e-06 2024-09-14T09:07:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323370+0000", 2024-09-14T09:07:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0180000000000001e-06 2024-09-14T09:07:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323388+0000", 2024-09-14T09:07:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8326999999999999e-05 2024-09-14T09:07:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323397+0000", 2024-09-14T09:07:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1940000000000004e-06 2024-09-14T09:07:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323452+0000", 2024-09-14T09:07:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5043000000000001e-05 2024-09-14T09:07:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323550+0000", 2024-09-14T09:07:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8201000000000004e-05 2024-09-14T09:07:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.328273+0000", 2024-09-14T09:07:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0047227079999999999 2024-09-14T09:07:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435232+0000", 2024-09-14T09:07:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.106959339 2024-09-14T09:07:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435256+0000", 2024-09-14T09:07:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3774e-05 2024-09-14T09:07:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435314+0000", 2024-09-14T09:07:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7636999999999997e-05 2024-09-14T09:07:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435417+0000", 2024-09-14T09:07:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010275 2024-09-14T09:07:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:84137 3.7 e92/23 ondisk, result = 0)", 2024-09-14T09:07:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:53.341180+0000", 2024-09-14T09:07:03.971 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.563322998, 2024-09-14T09:07:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.094062506000000004, 2024-09-14T09:07:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341180+0000", 2024-09-14T09:07:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341180+0000", 2024-09-14T09:07:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341181+0000", 2024-09-14T09:07:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.192e-06 2024-09-14T09:07:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341187+0000", 2024-09-14T09:07:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0070000000000003e-06 2024-09-14T09:07:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341208+0000", 2024-09-14T09:07:03.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0321000000000001e-05 2024-09-14T09:07:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341219+0000", 2024-09-14T09:07:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1764e-05 2024-09-14T09:07:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435214+0000", 2024-09-14T09:07:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.093994438999999999 2024-09-14T09:07:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435225+0000", 2024-09-14T09:07:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1123e-05 2024-09-14T09:07:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435243+0000", 2024-09-14T09:07:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.766e-05 2024-09-14T09:07:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:92675 3.8 3:105e0099:::rbd_data.115e7770ddc5.0000000000000016:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:03.975 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:57.736053+0000", 2024-09-14T09:07:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.168449753, 2024-09-14T09:07:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073115555999999998, 2024-09-14T09:07:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 92675 2024-09-14T09:07:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736053+0000", 2024-09-14T09:07:03.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736053+0000", 2024-09-14T09:07:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736055+0000", 2024-09-14T09:07:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.33e-06 2024-09-14T09:07:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736062+0000", 2024-09-14T09:07:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.08e-06 2024-09-14T09:07:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736077+0000", 2024-09-14T09:07:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5605000000000001e-05 2024-09-14T09:07:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736088+0000", 2024-09-14T09:07:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0705e-05 2024-09-14T09:07:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736175+0000", 2024-09-14T09:07:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6926000000000001e-05 2024-09-14T09:07:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736325+0000", 2024-09-14T09:07:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149739 2024-09-14T09:07:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.740035+0000", 2024-09-14T09:07:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0037099250000000002 2024-09-14T09:07:03.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809001+0000", 2024-09-14T09:07:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.068966240999999998 2024-09-14T09:07:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809049+0000", 2024-09-14T09:07:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8572999999999997e-05 2024-09-14T09:07:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:03.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809102+0000", 2024-09-14T09:07:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2564e-05 2024-09-14T09:07:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809169+0000", 2024-09-14T09:07:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6867999999999997e-05 2024-09-14T09:07:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:92675 3.8 e94/23 ondisk, result = 0)", 2024-09-14T09:07:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:57.748881+0000", 2024-09-14T09:07:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.1556217029999996, 2024-09-14T09:07:03.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060300578000000001, 2024-09-14T09:07:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748881+0000", 2024-09-14T09:07:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748881+0000", 2024-09-14T09:07:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748883+0000", 2024-09-14T09:07:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.491e-06 2024-09-14T09:07:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748887+0000", 2024-09-14T09:07:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5310000000000003e-06 2024-09-14T09:07:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748904+0000", 2024-09-14T09:07:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.662e-05 2024-09-14T09:07:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748912+0000", 2024-09-14T09:07:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3869999999999995e-06 2024-09-14T09:07:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809034+0000", 2024-09-14T09:07:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060121757999999997 2024-09-14T09:07:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809044+0000", 2024-09-14T09:07:03.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7699999999999996e-06 2024-09-14T09:07:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809182+0000", 2024-09-14T09:07:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000138021 2024-09-14T09:07:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:93277 3.8 3:105e0099:::rbd_data.115e7770ddc5.0000000000000016:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:07:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.097058+0000", 2024-09-14T09:07:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.8074454419999997, 2024-09-14T09:07:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.065618911000000002, 2024-09-14T09:07:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.985 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 93277 2024-09-14T09:07:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097058+0000", 2024-09-14T09:07:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097058+0000", 2024-09-14T09:07:03.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097060+0000", 2024-09-14T09:07:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.497e-06 2024-09-14T09:07:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097072+0000", 2024-09-14T09:07:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1600000000000001e-05 2024-09-14T09:07:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097102+0000", 2024-09-14T09:07:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0552999999999997e-05 2024-09-14T09:07:03.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097124+0000", 2024-09-14T09:07:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1477000000000001e-05 2024-09-14T09:07:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097201+0000", 2024-09-14T09:07:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7065999999999997e-05 2024-09-14T09:07:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097277+0000", 2024-09-14T09:07:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5722000000000004e-05 2024-09-14T09:07:03.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.101895+0000", 2024-09-14T09:07:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004618212 2024-09-14T09:07:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162452+0000", 2024-09-14T09:07:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060557603000000002 2024-09-14T09:07:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162504+0000", 2024-09-14T09:07:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.147e-05 2024-09-14T09:07:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162643+0000", 2024-09-14T09:07:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000139245 2024-09-14T09:07:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162677+0000", 2024-09-14T09:07:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3466000000000001e-05 2024-09-14T09:07:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:93277 3.8 e95/23 ondisk, result = 0)", 2024-09-14T09:07:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.113855+0000", 2024-09-14T09:07:03.990 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.7906478730000002, 2024-09-14T09:07:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048828219999999999, 2024-09-14T09:07:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.113855+0000", 2024-09-14T09:07:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.113855+0000", 2024-09-14T09:07:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.113857+0000", 2024-09-14T09:07:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3400000000000001e-06 2024-09-14T09:07:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.113863+0000", 2024-09-14T09:07:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4740000000000002e-06 2024-09-14T09:07:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.113883+0000", 2024-09-14T09:07:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9738999999999999e-05 2024-09-14T09:07:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.113893+0000", 2024-09-14T09:07:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0634e-05 2024-09-14T09:07:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162478+0000", 2024-09-14T09:07:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048584862999999999 2024-09-14T09:07:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162484+0000", 2024-09-14T09:07:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1829999999999999e-06 2024-09-14T09:07:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162683+0000", 2024-09-14T09:07:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000198987 2024-09-14T09:07:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:95495 3.18 3:1d9f7d98:::rbd_data.115e7770ddc5.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:07:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.598437+0000", 2024-09-14T09:07:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.306066393, 2024-09-14T09:07:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.090910543999999996, 2024-09-14T09:07:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:03.994 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 95495 2024-09-14T09:07:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598437+0000", 2024-09-14T09:07:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598437+0000", 2024-09-14T09:07:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598438+0000", 2024-09-14T09:07:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1149999999999999e-06 2024-09-14T09:07:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598443+0000", 2024-09-14T09:07:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4589999999999997e-06 2024-09-14T09:07:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598455+0000", 2024-09-14T09:07:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1793e-05 2024-09-14T09:07:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598463+0000", 2024-09-14T09:07:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9759999999999995e-06 2024-09-14T09:07:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598523+0000", 2024-09-14T09:07:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9926000000000002e-05 2024-09-14T09:07:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598581+0000", 2024-09-14T09:07:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8285000000000002e-05 2024-09-14T09:07:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:03.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.602809+0000", 2024-09-14T09:07:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0042279079999999998 2024-09-14T09:07:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689202+0000", 2024-09-14T09:07:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.086392847999999994 2024-09-14T09:07:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689230+0000", 2024-09-14T09:07:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8501999999999999e-05 2024-09-14T09:07:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689290+0000", 2024-09-14T09:07:03.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9524000000000001e-05 2024-09-14T09:07:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689347+0000", 2024-09-14T09:07:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7207999999999998e-05 2024-09-14T09:07:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:95495 3.18 e95/80 ondisk, result = 0)", 2024-09-14T09:07:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.610054+0000", 2024-09-14T09:07:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 5.2944489279999996, 2024-09-14T09:07:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079165141999999994, 2024-09-14T09:07:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:03.999 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610054+0000", 2024-09-14T09:07:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610054+0000", 2024-09-14T09:07:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:04.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610055+0000", 2024-09-14T09:07:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2669999999999999e-06 2024-09-14T09:07:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610061+0000", 2024-09-14T09:07:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:07:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610078+0000", 2024-09-14T09:07:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6965000000000001e-05 2024-09-14T09:07:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:04.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610087+0000", 2024-09-14T09:07:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.611e-06 2024-09-14T09:07:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689008+0000", 2024-09-14T09:07:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.078921186000000004 2024-09-14T09:07:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689020+0000", 2024-09-14T09:07:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.132e-05 2024-09-14T09:07:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:04.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689219+0000", 2024-09-14T09:07:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000199469 2024-09-14T09:07:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:04.003 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:04.003 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:04.004 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:07:04.005 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is set 2024-09-14T09:07:04.005 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:04.003+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:04.006 INFO:tasks.ceph.mon.a.smithi176.stderr:2024-09-14T09:07:04.007+0000 7f9a2f8af640 -1 mon.a@0(leader).osd e99 definitely_dead 0 2024-09-14T09:07:04.010 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:07:04.105 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:04.107+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:04.136 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:04.136 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:04.136 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:07:04.136 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:04.150 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:07:04.188 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:04.188 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:04.204 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:04.204 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:04.220 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:04.220 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:04.236 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:04.236 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:04.239 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:04.243+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:04.254 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:07:04.254 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:07:04.273 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:04.274 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:04.293 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:04.293 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:04.306 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:04.307+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:04.313 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:04.314 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:04.319 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:04.319 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:04.333 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:07:04.423 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:04.427+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:04.504 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:04.504 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:04.509 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:04.511+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:04.518 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:07:04.610 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:04.611+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:04.687 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:04.687 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:04.702 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:07:04.710 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:04.711+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:04.811 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:04.811+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:04.870 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:04.870 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:04.885 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:07:04.931 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:04.935+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:04.981 INFO:teuthology.orchestra.run.smithi176.stderr:marked out osd.7. 2024-09-14T09:07:05.012 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:05.011+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:05.021 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:05.022 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:05.038 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:07:05.123 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:05.127+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:05.204 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:05.204 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:05.214 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:05.215+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:05.221 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:07:05.316 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:05.315+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:05.389 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:05.389 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:05.404 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:07:05.451 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:05.455+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:05.517 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:05.515+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:05.539 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:05.539 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:05.555 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:07:05.647 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:05.651+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:05.719 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:05.723+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:05.722 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:05.722 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:05.737 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:07:05.821 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:05.819+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:05.904 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:05.904 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:05.918 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:07:05.921 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:05.919+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:06.006 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T09:07:06.021 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:06.023+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:06.055 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:06.055 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:06.071 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:07:06.122 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:06.123+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:06.196 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:06.196 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:06.211 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:07:06.258 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:06.259+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:06.323 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:06.323+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:06.328 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:06.328 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:06.332 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:07:06.343 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:07:06.424 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:06.427+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:06.471 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:06.472 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:06.488 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:06.488 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:06.504 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:06.504 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:06.506 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20858 3.17 3:ed5052d1:::rbd_data.115e889d4809.000000000000015c:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:07:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 79.077517857000004, 2024-09-14T09:07:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053381794000000003, 2024-09-14T09:07:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20858 2024-09-14T09:07:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:07:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429348+0000", 2024-09-14T09:07:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429351+0000", 2024-09-14T09:07:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1149999999999998e-06 2024-09-14T09:07:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429363+0000", 2024-09-14T09:07:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1819e-05 2024-09-14T09:07:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429389+0000", 2024-09-14T09:07:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5605e-05 2024-09-14T09:07:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429402+0000", 2024-09-14T09:07:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3175e-05 2024-09-14T09:07:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429492+0000", 2024-09-14T09:07:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.988e-05 2024-09-14T09:07:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.429597+0000", 2024-09-14T09:07:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010564 2024-09-14T09:07:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:07:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.433465+0000", 2024-09-14T09:07:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0038674249999999999 2024-09-14T09:07:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482483+0000", 2024-09-14T09:07:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049017999999999999 2024-09-14T09:07:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482625+0000", 2024-09-14T09:07:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000142447 2024-09-14T09:07:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482684+0000", 2024-09-14T09:07:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8522e-05 2024-09-14T09:07:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482730+0000", 2024-09-14T09:07:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6165999999999998e-05 2024-09-14T09:07:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:20858 3.17 e41/23 ondisk, result = 0)", 2024-09-14T09:07:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:07:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 79.062758414000001, 2024-09-14T09:07:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038631402000000002, 2024-09-14T09:07:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:07:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444107+0000", 2024-09-14T09:07:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444109+0000", 2024-09-14T09:07:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.331e-06 2024-09-14T09:07:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444114+0000", 2024-09-14T09:07:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3709999999999999e-06 2024-09-14T09:07:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444130+0000", 2024-09-14T09:07:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6079e-05 2024-09-14T09:07:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.444144+0000", 2024-09-14T09:07:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3331e-05 2024-09-14T09:07:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482538+0000", 2024-09-14T09:07:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.038394878 2024-09-14T09:07:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482548+0000", 2024-09-14T09:07:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0093e-05 2024-09-14T09:07:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.482739+0000", 2024-09-14T09:07:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019031899999999999 2024-09-14T09:07:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:37800 3.1 3:830c4670:::rbd_data.115eb07a8abd.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:07:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:07:06.517 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 69.476310447000003, 2024-09-14T09:07:06.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.068040724999999996, 2024-09-14T09:07:06.517 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.517 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.517 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:06.517 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:06.517 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:06.517 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:06.517 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 37800 2024-09-14T09:07:06.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.517 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:07:06.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030555+0000", 2024-09-14T09:07:06.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030557+0000", 2024-09-14T09:07:06.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.888e-06 2024-09-14T09:07:06.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030566+0000", 2024-09-14T09:07:06.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8540000000000003e-06 2024-09-14T09:07:06.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030585+0000", 2024-09-14T09:07:06.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8878000000000001e-05 2024-09-14T09:07:06.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030593+0000", 2024-09-14T09:07:06.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5870000000000004e-06 2024-09-14T09:07:06.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030667+0000", 2024-09-14T09:07:06.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4207000000000003e-05 2024-09-14T09:07:06.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.030761+0000", 2024-09-14T09:07:06.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4223999999999998e-05 2024-09-14T09:07:06.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:07:06.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.034400+0000", 2024-09-14T09:07:06.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0036387759999999998 2024-09-14T09:07:06.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:06.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098457+0000", 2024-09-14T09:07:06.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.064057501000000003 2024-09-14T09:07:06.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:06.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098518+0000", 2024-09-14T09:07:06.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0730999999999998e-05 2024-09-14T09:07:06.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:06.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098560+0000", 2024-09-14T09:07:06.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2267e-05 2024-09-14T09:07:06.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098596+0000", 2024-09-14T09:07:06.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5812000000000002e-05 2024-09-14T09:07:06.522 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.522 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.522 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.522 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:37800 3.1 e48/34 ondisk, result = 0)", 2024-09-14T09:07:06.522 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:07:06.522 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 69.465318831000005, 2024-09-14T09:07:06.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.057056260999999997, 2024-09-14T09:07:06.522 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.522 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.522 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:06.522 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:07:06.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041547+0000", 2024-09-14T09:07:06.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041548+0000", 2024-09-14T09:07:06.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8100000000000002e-07 2024-09-14T09:07:06.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041552+0000", 2024-09-14T09:07:06.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7419999999999999e-06 2024-09-14T09:07:06.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041563+0000", 2024-09-14T09:07:06.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1926000000000001e-05 2024-09-14T09:07:06.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.041572+0000", 2024-09-14T09:07:06.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8610000000000004e-06 2024-09-14T09:07:06.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098503+0000", 2024-09-14T09:07:06.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056930827000000003 2024-09-14T09:07:06.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098512+0000", 2024-09-14T09:07:06.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5019999999999992e-06 2024-09-14T09:07:06.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.098603+0000", 2024-09-14T09:07:06.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1621999999999998e-05 2024-09-14T09:07:06.525 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.526 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.526 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.526 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38523 3.3 3:c62e0486:::rbd_data.115e9e133b92.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 4194304~28499] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T09:07:06.526 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:07:06.526 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.052891133000003, 2024-09-14T09:07:06.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035654406999999999, 2024-09-14T09:07:06.526 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.526 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.526 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:06.526 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:06.526 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:06.526 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:06.526 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38523 2024-09-14T09:07:06.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.527 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:07:06.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453975+0000", 2024-09-14T09:07:06.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.453977+0000", 2024-09-14T09:07:06.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6299999999999998e-06 2024-09-14T09:07:06.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455684+0000", 2024-09-14T09:07:06.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001706843 2024-09-14T09:07:06.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455710+0000", 2024-09-14T09:07:06.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6302e-05 2024-09-14T09:07:06.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455722+0000", 2024-09-14T09:07:06.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1798e-05 2024-09-14T09:07:06.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.455825+0000", 2024-09-14T09:07:06.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102822 2024-09-14T09:07:06.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.456032+0000", 2024-09-14T09:07:06.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020704400000000001 2024-09-14T09:07:06.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:07:06.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.456759+0000", 2024-09-14T09:07:06.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00072735699999999998 2024-09-14T09:07:06.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:06.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.477856+0000", 2024-09-14T09:07:06.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021096011000000001 2024-09-14T09:07:06.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:06.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489526+0000", 2024-09-14T09:07:06.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011670313 2024-09-14T09:07:06.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:06.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489572+0000", 2024-09-14T09:07:06.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6408e-05 2024-09-14T09:07:06.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:58.489629+0000", 2024-09-14T09:07:06.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6879000000000002e-05 2024-09-14T09:07:06.531 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.531 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.531 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.532 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49719 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:07:06.532 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:07:06.532 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 55.981232532, 2024-09-14T09:07:06.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034680563999999997, 2024-09-14T09:07:06.532 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.532 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.532 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:06.532 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:06.532 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:06.532 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:06.532 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49719 2024-09-14T09:07:06.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.532 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:07:06.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525633+0000", 2024-09-14T09:07:06.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525634+0000", 2024-09-14T09:07:06.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4000000000000001e-07 2024-09-14T09:07:06.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525638+0000", 2024-09-14T09:07:06.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5899999999999999e-06 2024-09-14T09:07:06.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525647+0000", 2024-09-14T09:07:06.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2029999999999998e-06 2024-09-14T09:07:06.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525653+0000", 2024-09-14T09:07:06.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9850000000000003e-06 2024-09-14T09:07:06.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560150+0000", 2024-09-14T09:07:06.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034496845999999998 2024-09-14T09:07:06.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560221+0000", 2024-09-14T09:07:06.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1400000000000001e-05 2024-09-14T09:07:06.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560314+0000", 2024-09-14T09:07:06.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2800000000000006e-05 2024-09-14T09:07:06.536 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.536 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.536 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.536 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49720 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:07:06.536 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:07:06.536 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 55.980980907999999, 2024-09-14T09:07:06.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034919095999999997, 2024-09-14T09:07:06.536 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.536 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.536 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:06.537 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:06.537 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:06.537 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:06.537 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49720 2024-09-14T09:07:06.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.537 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:07:06.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525885+0000", 2024-09-14T09:07:06.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525886+0000", 2024-09-14T09:07:06.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6000000000000002e-07 2024-09-14T09:07:06.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525890+0000", 2024-09-14T09:07:06.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4519999999999999e-06 2024-09-14T09:07:06.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525899+0000", 2024-09-14T09:07:06.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1090000000000004e-06 2024-09-14T09:07:06.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525906+0000", 2024-09-14T09:07:06.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0219999999999999e-06 2024-09-14T09:07:06.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560344+0000", 2024-09-14T09:07:06.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034437970999999998 2024-09-14T09:07:06.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560422+0000", 2024-09-14T09:07:06.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7857999999999999e-05 2024-09-14T09:07:06.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560804+0000", 2024-09-14T09:07:06.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00038182400000000001 2024-09-14T09:07:06.540 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.540 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.540 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.540 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49721 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:07:06.540 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:07:06.540 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 55.980933426, 2024-09-14T09:07:06.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035347380999999997, 2024-09-14T09:07:06.541 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.541 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.541 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:06.541 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:06.541 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:06.541 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:06.541 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49721 2024-09-14T09:07:06.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.541 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:07:06.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525932+0000", 2024-09-14T09:07:06.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525933+0000", 2024-09-14T09:07:06.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5499999999999998e-07 2024-09-14T09:07:06.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525936+0000", 2024-09-14T09:07:06.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6759999999999999e-06 2024-09-14T09:07:06.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525943+0000", 2024-09-14T09:07:06.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2459999999999997e-06 2024-09-14T09:07:06.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525948+0000", 2024-09-14T09:07:06.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9429999999999999e-06 2024-09-14T09:07:06.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560845+0000", 2024-09-14T09:07:06.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034897354999999998 2024-09-14T09:07:06.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.560925+0000", 2024-09-14T09:07:06.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9804999999999999e-05 2024-09-14T09:07:06.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561280+0000", 2024-09-14T09:07:06.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035480100000000002 2024-09-14T09:07:06.544 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.544 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.544 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.545 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49722 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3112960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:07:06.545 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:07:06.545 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 55.980909779999998, 2024-09-14T09:07:06.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035750491000000002, 2024-09-14T09:07:06.545 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.545 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.545 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:06.545 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:06.545 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:06.545 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:06.545 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49722 2024-09-14T09:07:06.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.546 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:07:06.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:07:06.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525956+0000", 2024-09-14T09:07:06.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.96e-07 2024-09-14T09:07:06.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525958+0000", 2024-09-14T09:07:06.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.024e-06 2024-09-14T09:07:06.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525964+0000", 2024-09-14T09:07:06.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6699999999999999e-06 2024-09-14T09:07:06.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525969+0000", 2024-09-14T09:07:06.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3939999999999998e-06 2024-09-14T09:07:06.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561319+0000", 2024-09-14T09:07:06.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.03535013 2024-09-14T09:07:06.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561390+0000", 2024-09-14T09:07:06.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0920000000000005e-05 2024-09-14T09:07:06.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561707+0000", 2024-09-14T09:07:06.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031695699999999998 2024-09-14T09:07:06.549 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.549 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.549 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.549 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49723 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3506176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:07:06.549 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:07:06.549 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 55.980890797000001, 2024-09-14T09:07:06.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036113064, 2024-09-14T09:07:06.549 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.549 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.549 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:06.550 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:06.550 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:06.550 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:06.550 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49723 2024-09-14T09:07:06.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.550 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:07:06.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:07:06.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525975+0000", 2024-09-14T09:07:06.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1699999999999999e-07 2024-09-14T09:07:06.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525977+0000", 2024-09-14T09:07:06.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9250000000000002e-06 2024-09-14T09:07:06.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525983+0000", 2024-09-14T09:07:06.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4620000000000004e-06 2024-09-14T09:07:06.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525987+0000", 2024-09-14T09:07:06.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.279e-06 2024-09-14T09:07:06.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561749+0000", 2024-09-14T09:07:06.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035762222000000003 2024-09-14T09:07:06.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.561796+0000", 2024-09-14T09:07:06.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6959999999999998e-05 2024-09-14T09:07:06.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562088+0000", 2024-09-14T09:07:06.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029179899999999998 2024-09-14T09:07:06.553 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.553 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.553 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.553 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:49724 3.6 3:66c8f347:::rbd_data.115e51d92ed2.0000000000000001:12 [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-14T09:07:06.554 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:07:06.554 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 55.980872640999998, 2024-09-14T09:07:06.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036449388999999999, 2024-09-14T09:07:06.554 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.554 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.554 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:06.554 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:06.554 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:06.554 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:06.554 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 49724 2024-09-14T09:07:06.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.555 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:07:06.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525993+0000", 2024-09-14T09:07:06.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525994+0000", 2024-09-14T09:07:06.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3099999999999998e-07 2024-09-14T09:07:06.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.525996+0000", 2024-09-14T09:07:06.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.939e-06 2024-09-14T09:07:06.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526005+0000", 2024-09-14T09:07:06.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8430000000000005e-06 2024-09-14T09:07:06.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.526010+0000", 2024-09-14T09:07:06.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9389999999999996e-06 2024-09-14T09:07:06.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562114+0000", 2024-09-14T09:07:06.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036103510999999998 2024-09-14T09:07:06.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562189+0000", 2024-09-14T09:07:06.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5162999999999998e-05 2024-09-14T09:07:06.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:10.562443+0000", 2024-09-14T09:07:06.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000253563 2024-09-14T09:07:06.558 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.558 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.558 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.558 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:64935 3.12 3:4cd6d7cb:::rbd_data.115e21c31fcf.0000000000000049:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:07:06.558 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:07:06.558 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.755507293000001, 2024-09-14T09:07:06.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037353866999999999, 2024-09-14T09:07:06.558 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.558 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.559 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:06.559 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:06.559 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:06.559 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:06.559 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 64935 2024-09-14T09:07:06.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.559 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:07:06.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751359+0000", 2024-09-14T09:07:06.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751362+0000", 2024-09-14T09:07:06.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6899999999999998e-06 2024-09-14T09:07:06.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751378+0000", 2024-09-14T09:07:06.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6147e-05 2024-09-14T09:07:06.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751421+0000", 2024-09-14T09:07:06.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2691999999999997e-05 2024-09-14T09:07:06.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751444+0000", 2024-09-14T09:07:06.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3237000000000001e-05 2024-09-14T09:07:06.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751594+0000", 2024-09-14T09:07:06.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014947800000000001 2024-09-14T09:07:06.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.751733+0000", 2024-09-14T09:07:06.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013945200000000001 2024-09-14T09:07:06.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:06.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.753315+0000", 2024-09-14T09:07:06.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0015815969999999999 2024-09-14T09:07:06.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:06.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788546+0000", 2024-09-14T09:07:06.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035231449999999997 2024-09-14T09:07:06.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:06.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788570+0000", 2024-09-14T09:07:06.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3991000000000001e-05 2024-09-14T09:07:06.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:06.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788645+0000", 2024-09-14T09:07:06.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4379000000000001e-05 2024-09-14T09:07:06.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:20.788712+0000", 2024-09-14T09:07:06.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7754000000000002e-05 2024-09-14T09:07:06.564 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.564 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.564 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.564 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:71052 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:07:06.564 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:07:06.564 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.769298042000003, 2024-09-14T09:07:06.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14854635599999999, 2024-09-14T09:07:06.564 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.564 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.564 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:06.564 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:06.565 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:06.565 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:06.565 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 71052 2024-09-14T09:07:06.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.565 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:07:06.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.566 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:07:06.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.566 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737570+0000", 2024-09-14T09:07:06.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4770000000000002e-06 2024-09-14T09:07:06.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.566 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737584+0000", 2024-09-14T09:07:06.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3457e-05 2024-09-14T09:07:06.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737608+0000", 2024-09-14T09:07:06.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3893999999999999e-05 2024-09-14T09:07:06.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737620+0000", 2024-09-14T09:07:06.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2157000000000001e-05 2024-09-14T09:07:06.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737738+0000", 2024-09-14T09:07:06.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118409 2024-09-14T09:07:06.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737892+0000", 2024-09-14T09:07:06.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154222 2024-09-14T09:07:06.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:07:06.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.749622+0000", 2024-09-14T09:07:06.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011729537 2024-09-14T09:07:06.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:06.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885934+0000", 2024-09-14T09:07:06.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13631222200000001 2024-09-14T09:07:06.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:06.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885996+0000", 2024-09-14T09:07:06.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1555000000000003e-05 2024-09-14T09:07:06.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:06.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886060+0000", 2024-09-14T09:07:06.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3980999999999996e-05 2024-09-14T09:07:06.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886114+0000", 2024-09-14T09:07:06.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4444999999999998e-05 2024-09-14T09:07:06.570 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.570 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.570 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.570 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:71052 3.1 e71/34 ondisk, result = 0)", 2024-09-14T09:07:06.570 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:07:06.570 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.726394663999997, 2024-09-14T09:07:06.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10565334799999999, 2024-09-14T09:07:06.570 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.571 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.571 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:06.571 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:07:06.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:07:06.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780473+0000", 2024-09-14T09:07:06.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8980000000000001e-06 2024-09-14T09:07:06.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780482+0000", 2024-09-14T09:07:06.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9080000000000001e-06 2024-09-14T09:07:06.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.572 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.572 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780511+0000", 2024-09-14T09:07:06.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9490999999999999e-05 2024-09-14T09:07:06.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780527+0000", 2024-09-14T09:07:06.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.508e-05 2024-09-14T09:07:06.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885978+0000", 2024-09-14T09:07:06.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.105451328 2024-09-14T09:07:06.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885989+0000", 2024-09-14T09:07:06.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0875e-05 2024-09-14T09:07:06.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886125+0000", 2024-09-14T09:07:06.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013576800000000001 2024-09-14T09:07:06.574 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.574 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.574 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.574 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73871 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:1b [read 8339456~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:07:06.574 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:07:06.574 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 31.080909655999999, 2024-09-14T09:07:06.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034605047, 2024-09-14T09:07:06.574 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.575 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.575 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:06.575 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:06.575 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:06.575 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:06.575 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73871 2024-09-14T09:07:06.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.575 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:07:06.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425956+0000", 2024-09-14T09:07:06.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425957+0000", 2024-09-14T09:07:06.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1999999999999999e-06 2024-09-14T09:07:06.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425961+0000", 2024-09-14T09:07:06.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8279999999999996e-06 2024-09-14T09:07:06.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425985+0000", 2024-09-14T09:07:06.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4182e-05 2024-09-14T09:07:06.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.577 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.425997+0000", 2024-09-14T09:07:06.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.132e-05 2024-09-14T09:07:06.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.577 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460070+0000", 2024-09-14T09:07:06.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.034072913000000003 2024-09-14T09:07:06.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460298+0000", 2024-09-14T09:07:06.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022819 2024-09-14T09:07:06.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.460561+0000", 2024-09-14T09:07:06.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026341400000000001 2024-09-14T09:07:06.578 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.578 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.578 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.579 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:74276 3.12 3:49b683dd:::rbd_data.115ea8d131ff.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 1376256~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:07:06.579 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:07:06.579 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.821762566, 2024-09-14T09:07:06.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036482939999999998, 2024-09-14T09:07:06.579 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.579 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.579 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:06.579 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:06.579 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:06.579 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:06.579 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74276 2024-09-14T09:07:06.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.580 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:07:06.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685103+0000", 2024-09-14T09:07:06.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.685105+0000", 2024-09-14T09:07:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9520000000000001e-06 2024-09-14T09:07:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686530+0000", 2024-09-14T09:07:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001424629 2024-09-14T09:07:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686550+0000", 2024-09-14T09:07:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0472e-05 2024-09-14T09:07:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686561+0000", 2024-09-14T09:07:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0366e-05 2024-09-14T09:07:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686671+0000", 2024-09-14T09:07:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110621 2024-09-14T09:07:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.686919+0000", 2024-09-14T09:07:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024790199999999999 2024-09-14T09:07:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.687375+0000", 2024-09-14T09:07:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00045572100000000002 2024-09-14T09:07:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.710871+0000", 2024-09-14T09:07:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.023496481999999999 2024-09-14T09:07:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721478+0000", 2024-09-14T09:07:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01060709 2024-09-14T09:07:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721534+0000", 2024-09-14T09:07:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5912999999999998e-05 2024-09-14T09:07:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.721586+0000", 2024-09-14T09:07:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1792000000000002e-05 2024-09-14T09:07:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.584 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:84402 3.6 3:6123bacd:::rbd_data.115e77b799c3.0000000000000017:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:07:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:53.874733+0000", 2024-09-14T09:07:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 12.632132346000001, 2024-09-14T09:07:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040401678000000003, 2024-09-14T09:07:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 84402 2024-09-14T09:07:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874733+0000", 2024-09-14T09:07:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874733+0000", 2024-09-14T09:07:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874735+0000", 2024-09-14T09:07:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3170000000000001e-06 2024-09-14T09:07:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874742+0000", 2024-09-14T09:07:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0119999999999999e-06 2024-09-14T09:07:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874758+0000", 2024-09-14T09:07:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5971e-05 2024-09-14T09:07:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874768+0000", 2024-09-14T09:07:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0257e-05 2024-09-14T09:07:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874816+0000", 2024-09-14T09:07:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8133999999999997e-05 2024-09-14T09:07:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.874862+0000", 2024-09-14T09:07:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6269e-05 2024-09-14T09:07:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.876731+0000", 2024-09-14T09:07:06.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001868722 2024-09-14T09:07:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.914723+0000", 2024-09-14T09:07:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.037991912000000003 2024-09-14T09:07:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.914839+0000", 2024-09-14T09:07:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115901 2024-09-14T09:07:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.915013+0000", 2024-09-14T09:07:06.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017389 2024-09-14T09:07:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.915135+0000", 2024-09-14T09:07:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012229299999999999 2024-09-14T09:07:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:95497 3.1 3:845f2eaa:::rbd_data.115e7770ddc5.000000000000000c:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:07:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.690856+0000", 2024-09-14T09:07:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.8160101529999997, 2024-09-14T09:07:06.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.066534455000000006, 2024-09-14T09:07:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 95497 2024-09-14T09:07:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.690856+0000", 2024-09-14T09:07:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.690856+0000", 2024-09-14T09:07:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.690860+0000", 2024-09-14T09:07:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0929999999999996e-06 2024-09-14T09:07:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.690876+0000", 2024-09-14T09:07:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6066000000000002e-05 2024-09-14T09:07:06.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.690932+0000", 2024-09-14T09:07:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6273000000000001e-05 2024-09-14T09:07:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.690955+0000", 2024-09-14T09:07:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.266e-05 2024-09-14T09:07:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.691057+0000", 2024-09-14T09:07:06.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010218899999999999 2024-09-14T09:07:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.691173+0000", 2024-09-14T09:07:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000115617 2024-09-14T09:07:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:07:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.694376+0000", 2024-09-14T09:07:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0032031960000000002 2024-09-14T09:07:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:06.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757283+0000", 2024-09-14T09:07:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062907413999999995 2024-09-14T09:07:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757294+0000", 2024-09-14T09:07:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0577999999999999e-05 2024-09-14T09:07:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757332+0000", 2024-09-14T09:07:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8457999999999997e-05 2024-09-14T09:07:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757390+0000", 2024-09-14T09:07:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7911000000000002e-05 2024-09-14T09:07:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:95498 3.1 3:8579c379:::rbd_data.115e7770ddc5.000000000000000d:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:07:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.691002+0000", 2024-09-14T09:07:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.815863738, 2024-09-14T09:07:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.066695790000000005, 2024-09-14T09:07:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.596 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 95498 2024-09-14T09:07:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.691002+0000", 2024-09-14T09:07:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.597 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.691002+0000", 2024-09-14T09:07:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.691003+0000", 2024-09-14T09:07:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0440000000000001e-06 2024-09-14T09:07:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.691010+0000", 2024-09-14T09:07:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7299999999999999e-06 2024-09-14T09:07:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.691024+0000", 2024-09-14T09:07:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3896e-05 2024-09-14T09:07:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.691038+0000", 2024-09-14T09:07:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4177e-05 2024-09-14T09:07:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.747580+0000", 2024-09-14T09:07:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056541705999999997 2024-09-14T09:07:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.747666+0000", 2024-09-14T09:07:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5884999999999995e-05 2024-09-14T09:07:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:07:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.748789+0000", 2024-09-14T09:07:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0011238979999999999 2024-09-14T09:07:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757403+0000", 2024-09-14T09:07:06.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0086136579999999997 2024-09-14T09:07:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757532+0000", 2024-09-14T09:07:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000128527 2024-09-14T09:07:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757644+0000", 2024-09-14T09:07:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011266599999999999 2024-09-14T09:07:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757698+0000", 2024-09-14T09:07:06.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3603000000000002e-05 2024-09-14T09:07:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:95497 3.1 e95/85 ondisk, result = 0)", 2024-09-14T09:07:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.701592+0000", 2024-09-14T09:07:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.8052739850000004, 2024-09-14T09:07:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055812796999999997, 2024-09-14T09:07:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:06.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.701592+0000", 2024-09-14T09:07:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.701592+0000", 2024-09-14T09:07:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.701593+0000", 2024-09-14T09:07:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.37e-06 2024-09-14T09:07:06.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.701599+0000", 2024-09-14T09:07:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8699999999999997e-06 2024-09-14T09:07:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.701623+0000", 2024-09-14T09:07:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3442e-05 2024-09-14T09:07:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.701634+0000", 2024-09-14T09:07:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1287000000000001e-05 2024-09-14T09:07:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757271+0000", 2024-09-14T09:07:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.055636891000000001 2024-09-14T09:07:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757278+0000", 2024-09-14T09:07:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4259999999999997e-06 2024-09-14T09:07:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.757405+0000", 2024-09-14T09:07:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000126511 2024-09-14T09:07:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:06.606 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:06.606 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:06.607 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:07:06.608 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:06.608 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:06.608 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:07:06.609 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:07:06.609 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:06.609 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:06.609 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:06.610 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:06.610 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:06.610 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:06.610 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:06.607+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:06.708 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:06.711+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:06.789 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:06.790 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:06.790 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:06.790 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:06.804 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:07:06.809 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:06.811+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:06.827 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is unset 2024-09-14T09:07:06.918 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:06.919+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:06.975 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone32 from 0x9a36c81 (overlap 0x7f9d6d) to 0x72da7b 2024-09-14T09:07:06.989 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:06.989 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:06.989 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:06.990 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:06.990 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:07.011 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:07.011+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:07.016 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:07:07.111 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:07.115+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:07.196 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:07.196 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:07.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:07.196 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:07.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.196 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73540 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15155200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.197 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:07:07.197 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.92724467, 2024-09-14T09:07:07.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.022930347, 2024-09-14T09:07:07.197 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.197 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.197 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.197 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.197 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.197 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.197 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73540 2024-09-14T09:07:07.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.197 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:07:07.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:07:07.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268665+0000", 2024-09-14T09:07:07.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6599999999999996e-07 2024-09-14T09:07:07.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268668+0000", 2024-09-14T09:07:07.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8760000000000001e-06 2024-09-14T09:07:07.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268676+0000", 2024-09-14T09:07:07.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7879999999999995e-06 2024-09-14T09:07:07.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268683+0000", 2024-09-14T09:07:07.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3180000000000003e-06 2024-09-14T09:07:07.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282017+0000", 2024-09-14T09:07:07.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01333409 2024-09-14T09:07:07.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282020+0000", 2024-09-14T09:07:07.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1609999999999999e-06 2024-09-14T09:07:07.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291158+0000", 2024-09-14T09:07:07.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0091376430000001 2024-09-14T09:07:07.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291383+0000", 2024-09-14T09:07:07.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022491000000000001 2024-09-14T09:07:07.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291595+0000", 2024-09-14T09:07:07.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021199500000000001 2024-09-14T09:07:07.201 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.201 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.201 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.202 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73541 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15237120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.202 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:07:07.202 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.927217249999998, 2024-09-14T09:07:07.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0232607330000001, 2024-09-14T09:07:07.202 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.202 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.202 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.202 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.202 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.202 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.202 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73541 2024-09-14T09:07:07.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.203 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:07:07.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:07:07.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:07:07.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5400000000000001e-07 2024-09-14T09:07:07.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268695+0000", 2024-09-14T09:07:07.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5799999999999999e-06 2024-09-14T09:07:07.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268703+0000", 2024-09-14T09:07:07.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5480000000000002e-06 2024-09-14T09:07:07.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268713+0000", 2024-09-14T09:07:07.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8640000000000008e-06 2024-09-14T09:07:07.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282032+0000", 2024-09-14T09:07:07.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013318831999999999 2024-09-14T09:07:07.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282035+0000", 2024-09-14T09:07:07.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4010000000000001e-06 2024-09-14T09:07:07.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291701+0000", 2024-09-14T09:07:07.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0096654199999999 2024-09-14T09:07:07.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291807+0000", 2024-09-14T09:07:07.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105852 2024-09-14T09:07:07.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291952+0000", 2024-09-14T09:07:07.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014568200000000001 2024-09-14T09:07:07.206 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.206 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.207 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73542 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15319040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.207 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:07.207 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.927147638999998, 2024-09-14T09:07:07.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0237091359999999, 2024-09-14T09:07:07.207 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.207 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.207 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.207 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.207 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.207 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.207 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73542 2024-09-14T09:07:07.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.208 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:07.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:07.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268762+0000", 2024-09-14T09:07:07.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:07.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268766+0000", 2024-09-14T09:07:07.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5880000000000002e-06 2024-09-14T09:07:07.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268776+0000", 2024-09-14T09:07:07.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0168e-05 2024-09-14T09:07:07.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268789+0000", 2024-09-14T09:07:07.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:07:07.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282053+0000", 2024-09-14T09:07:07.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013264496000000001 2024-09-14T09:07:07.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282057+0000", 2024-09-14T09:07:07.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6090000000000001e-06 2024-09-14T09:07:07.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292052+0000", 2024-09-14T09:07:07.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0099950070000001 2024-09-14T09:07:07.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292262+0000", 2024-09-14T09:07:07.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021009800000000001 2024-09-14T09:07:07.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292470+0000", 2024-09-14T09:07:07.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000208314 2024-09-14T09:07:07.212 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.212 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.212 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.212 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73543 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.212 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:07.212 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.927108849, 2024-09-14T09:07:07.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023985473, 2024-09-14T09:07:07.212 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.212 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.212 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.213 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.213 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.213 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73543 2024-09-14T09:07:07.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.213 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:07.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:07.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268801+0000", 2024-09-14T09:07:07.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:07.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268803+0000", 2024-09-14T09:07:07.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7439999999999999e-06 2024-09-14T09:07:07.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268812+0000", 2024-09-14T09:07:07.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7080000000000003e-06 2024-09-14T09:07:07.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268819+0000", 2024-09-14T09:07:07.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T09:07:07.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282068+0000", 2024-09-14T09:07:07.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013249540000000001 2024-09-14T09:07:07.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282072+0000", 2024-09-14T09:07:07.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0960000000000001e-06 2024-09-14T09:07:07.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292573+0000", 2024-09-14T09:07:07.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0105014560000001 2024-09-14T09:07:07.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292679+0000", 2024-09-14T09:07:07.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010545899999999999 2024-09-14T09:07:07.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292786+0000", 2024-09-14T09:07:07.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107052 2024-09-14T09:07:07.217 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.217 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.217 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.217 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73544 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.217 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:07.217 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.927081147000003, 2024-09-14T09:07:07.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0244830549999999, 2024-09-14T09:07:07.218 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.218 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.218 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.218 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.218 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.218 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.218 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73544 2024-09-14T09:07:07.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.218 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:07.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:07.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:07.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5599999999999995e-07 2024-09-14T09:07:07.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268831+0000", 2024-09-14T09:07:07.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.57e-06 2024-09-14T09:07:07.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268844+0000", 2024-09-14T09:07:07.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2811e-05 2024-09-14T09:07:07.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268851+0000", 2024-09-14T09:07:07.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9140000000000002e-06 2024-09-14T09:07:07.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282083+0000", 2024-09-14T09:07:07.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013232111 2024-09-14T09:07:07.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282086+0000", 2024-09-14T09:07:07.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2909999999999999e-06 2024-09-14T09:07:07.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292884+0000", 2024-09-14T09:07:07.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0107975069999999 2024-09-14T09:07:07.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293100+0000", 2024-09-14T09:07:07.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021639399999999999 2024-09-14T09:07:07.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293311+0000", 2024-09-14T09:07:07.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021090099999999999 2024-09-14T09:07:07.222 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.222 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.222 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.222 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73545 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15564800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.223 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:07.223 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.927029961999999, 2024-09-14T09:07:07.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0249746740000001, 2024-09-14T09:07:07.223 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.223 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.223 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.223 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73545 2024-09-14T09:07:07.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.223 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:07.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:07.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268880+0000", 2024-09-14T09:07:07.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5600000000000005e-07 2024-09-14T09:07:07.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268889+0000", 2024-09-14T09:07:07.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8559999999999996e-06 2024-09-14T09:07:07.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268913+0000", 2024-09-14T09:07:07.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3958e-05 2024-09-14T09:07:07.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268921+0000", 2024-09-14T09:07:07.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7479999999999999e-06 2024-09-14T09:07:07.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282098+0000", 2024-09-14T09:07:07.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013176297999999999 2024-09-14T09:07:07.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282101+0000", 2024-09-14T09:07:07.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2329999999999998e-06 2024-09-14T09:07:07.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293435+0000", 2024-09-14T09:07:07.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0113339459999999 2024-09-14T09:07:07.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293648+0000", 2024-09-14T09:07:07.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021307100000000001 2024-09-14T09:07:07.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293854+0000", 2024-09-14T09:07:07.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020580800000000001 2024-09-14T09:07:07.227 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.227 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.227 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.228 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73546 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15646720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.228 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:07.228 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.926976091, 2024-09-14T09:07:07.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0254846360000001, 2024-09-14T09:07:07.228 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.228 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.228 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.228 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.228 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.228 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.228 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73546 2024-09-14T09:07:07.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.228 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:07.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:07.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268934+0000", 2024-09-14T09:07:07.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7999999999999995e-07 2024-09-14T09:07:07.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268937+0000", 2024-09-14T09:07:07.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9840000000000001e-06 2024-09-14T09:07:07.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268947+0000", 2024-09-14T09:07:07.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0522e-05 2024-09-14T09:07:07.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268955+0000", 2024-09-14T09:07:07.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8059999999999995e-06 2024-09-14T09:07:07.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282112+0000", 2024-09-14T09:07:07.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013157189 2024-09-14T09:07:07.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282115+0000", 2024-09-14T09:07:07.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2949999999999998e-06 2024-09-14T09:07:07.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293968+0000", 2024-09-14T09:07:07.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0118530990000001 2024-09-14T09:07:07.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294197+0000", 2024-09-14T09:07:07.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228507 2024-09-14T09:07:07.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294417+0000", 2024-09-14T09:07:07.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000220554 2024-09-14T09:07:07.232 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.232 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.232 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.233 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73547 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.233 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:07.233 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.926943645000001, 2024-09-14T09:07:07.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.025970525, 2024-09-14T09:07:07.233 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.233 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.233 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.233 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.233 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.233 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.233 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73547 2024-09-14T09:07:07.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.234 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:07.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:07.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268966+0000", 2024-09-14T09:07:07.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3799999999999997e-07 2024-09-14T09:07:07.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268969+0000", 2024-09-14T09:07:07.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8739999999999999e-06 2024-09-14T09:07:07.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268979+0000", 2024-09-14T09:07:07.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9939999999999995e-06 2024-09-14T09:07:07.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268986+0000", 2024-09-14T09:07:07.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3459999999999996e-06 2024-09-14T09:07:07.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282127+0000", 2024-09-14T09:07:07.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01314039 2024-09-14T09:07:07.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282130+0000", 2024-09-14T09:07:07.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2219999999999998e-06 2024-09-14T09:07:07.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294540+0000", 2024-09-14T09:07:07.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124105880000001 2024-09-14T09:07:07.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294759+0000", 2024-09-14T09:07:07.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000218798 2024-09-14T09:07:07.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294936+0000", 2024-09-14T09:07:07.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000176675 2024-09-14T09:07:07.237 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.238 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.238 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.238 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73548 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15810560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.238 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:07.238 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.926912532999999, 2024-09-14T09:07:07.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0265045129999999, 2024-09-14T09:07:07.238 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.238 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.238 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.238 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.238 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.238 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.239 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73548 2024-09-14T09:07:07.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.239 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:07.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:07.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268997+0000", 2024-09-14T09:07:07.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7599999999999996e-07 2024-09-14T09:07:07.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269000+0000", 2024-09-14T09:07:07.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0580000000000002e-06 2024-09-14T09:07:07.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269010+0000", 2024-09-14T09:07:07.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124e-05 2024-09-14T09:07:07.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269018+0000", 2024-09-14T09:07:07.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5399999999999998e-06 2024-09-14T09:07:07.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282141+0000", 2024-09-14T09:07:07.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013123012 2024-09-14T09:07:07.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282144+0000", 2024-09-14T09:07:07.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.196e-06 2024-09-14T09:07:07.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295042+0000", 2024-09-14T09:07:07.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0128975689999999 2024-09-14T09:07:07.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295272+0000", 2024-09-14T09:07:07.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023048099999999999 2024-09-14T09:07:07.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295501+0000", 2024-09-14T09:07:07.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228657 2024-09-14T09:07:07.243 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.243 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.243 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.243 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73549 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15892480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.243 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:07.243 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.926866654000001, 2024-09-14T09:07:07.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0268354099999999, 2024-09-14T09:07:07.243 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.243 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.243 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.243 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.243 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.243 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.244 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73549 2024-09-14T09:07:07.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.244 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:07.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:07.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269043+0000", 2024-09-14T09:07:07.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6199999999999997e-07 2024-09-14T09:07:07.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269046+0000", 2024-09-14T09:07:07.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2480000000000001e-06 2024-09-14T09:07:07.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269057+0000", 2024-09-14T09:07:07.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0436e-05 2024-09-14T09:07:07.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269064+0000", 2024-09-14T09:07:07.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5179999999999998e-06 2024-09-14T09:07:07.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282155+0000", 2024-09-14T09:07:07.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013091190000000001 2024-09-14T09:07:07.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282159+0000", 2024-09-14T09:07:07.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.366e-06 2024-09-14T09:07:07.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295599+0000", 2024-09-14T09:07:07.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0134405559999999 2024-09-14T09:07:07.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295765+0000", 2024-09-14T09:07:07.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016575499999999999 2024-09-14T09:07:07.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295878+0000", 2024-09-14T09:07:07.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112679 2024-09-14T09:07:07.248 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.248 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.248 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.248 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73550 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15974400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.248 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:07.248 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.926834053999997, 2024-09-14T09:07:07.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027357657, 2024-09-14T09:07:07.248 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.248 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.248 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.248 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.248 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.249 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.249 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73550 2024-09-14T09:07:07.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.249 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:07.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:07.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269076+0000", 2024-09-14T09:07:07.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:07:07.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269079+0000", 2024-09-14T09:07:07.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:07:07.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269098+0000", 2024-09-14T09:07:07.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9698e-05 2024-09-14T09:07:07.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269106+0000", 2024-09-14T09:07:07.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6179999999999997e-06 2024-09-14T09:07:07.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282170+0000", 2024-09-14T09:07:07.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013063874 2024-09-14T09:07:07.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282173+0000", 2024-09-14T09:07:07.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1999999999999999e-06 2024-09-14T09:07:07.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295996+0000", 2024-09-14T09:07:07.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.013823151 2024-09-14T09:07:07.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296214+0000", 2024-09-14T09:07:07.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021817700000000001 2024-09-14T09:07:07.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296433+0000", 2024-09-14T09:07:07.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021829099999999999 2024-09-14T09:07:07.253 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.253 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.253 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.253 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73551 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.253 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:07.253 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.926792636999998, 2024-09-14T09:07:07.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027730276, 2024-09-14T09:07:07.253 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.254 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.254 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.254 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.254 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.254 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.254 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73551 2024-09-14T09:07:07.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.254 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:07.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:07.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269117+0000", 2024-09-14T09:07:07.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:07:07.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269120+0000", 2024-09-14T09:07:07.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.056e-06 2024-09-14T09:07:07.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269130+0000", 2024-09-14T09:07:07.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0084e-05 2024-09-14T09:07:07.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269137+0000", 2024-09-14T09:07:07.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1359999999999999e-06 2024-09-14T09:07:07.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282184+0000", 2024-09-14T09:07:07.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013046537 2024-09-14T09:07:07.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282188+0000", 2024-09-14T09:07:07.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3220000000000003e-06 2024-09-14T09:07:07.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296529+0000", 2024-09-14T09:07:07.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0143406109999999 2024-09-14T09:07:07.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296695+0000", 2024-09-14T09:07:07.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016604799999999999 2024-09-14T09:07:07.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296847+0000", 2024-09-14T09:07:07.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000151822 2024-09-14T09:07:07.258 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.258 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.258 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.258 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73552 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.258 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:07.258 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.926762056999998, 2024-09-14T09:07:07.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0281767500000001, 2024-09-14T09:07:07.259 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.259 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.259 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.259 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.259 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.259 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.259 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73552 2024-09-14T09:07:07.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.259 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:07.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:07.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:07.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:07:07.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269150+0000", 2024-09-14T09:07:07.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9119999999999998e-06 2024-09-14T09:07:07.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269160+0000", 2024-09-14T09:07:07.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6819999999999998e-06 2024-09-14T09:07:07.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269168+0000", 2024-09-14T09:07:07.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6079999999999998e-06 2024-09-14T09:07:07.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282205+0000", 2024-09-14T09:07:07.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013036982000000001 2024-09-14T09:07:07.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282208+0000", 2024-09-14T09:07:07.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4620000000000001e-06 2024-09-14T09:07:07.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296968+0000", 2024-09-14T09:07:07.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0147601209999999 2024-09-14T09:07:07.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297115+0000", 2024-09-14T09:07:07.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014673900000000001 2024-09-14T09:07:07.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297324+0000", 2024-09-14T09:07:07.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020866400000000001 2024-09-14T09:07:07.263 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.263 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.263 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.263 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73553 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.264 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:07.264 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.926731992999997, 2024-09-14T09:07:07.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0286556739999999, 2024-09-14T09:07:07.264 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.264 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.264 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.264 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.264 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.264 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.264 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73553 2024-09-14T09:07:07.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.264 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:07.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:07.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269178+0000", 2024-09-14T09:07:07.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:07:07.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269180+0000", 2024-09-14T09:07:07.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T09:07:07.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269189+0000", 2024-09-14T09:07:07.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2e-06 2024-09-14T09:07:07.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269197+0000", 2024-09-14T09:07:07.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2980000000000003e-06 2024-09-14T09:07:07.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282219+0000", 2024-09-14T09:07:07.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013022687 2024-09-14T09:07:07.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282223+0000", 2024-09-14T09:07:07.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.241e-06 2024-09-14T09:07:07.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.267 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.267 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.267 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297470+0000", 2024-09-14T09:07:07.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.015247778 2024-09-14T09:07:07.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297704+0000", 2024-09-14T09:07:07.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000233367 2024-09-14T09:07:07.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297833+0000", 2024-09-14T09:07:07.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012888900000000001 2024-09-14T09:07:07.268 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.268 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.268 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.269 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73554 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.269 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:07.269 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.926702675999998, 2024-09-14T09:07:07.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.029005113, 2024-09-14T09:07:07.269 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.269 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.269 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.269 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.269 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.269 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.269 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73554 2024-09-14T09:07:07.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.269 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:07.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:07.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269207+0000", 2024-09-14T09:07:07.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.68e-07 2024-09-14T09:07:07.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269210+0000", 2024-09-14T09:07:07.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8100000000000002e-06 2024-09-14T09:07:07.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269219+0000", 2024-09-14T09:07:07.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:07:07.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269227+0000", 2024-09-14T09:07:07.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3839999999999999e-06 2024-09-14T09:07:07.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282234+0000", 2024-09-14T09:07:07.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013007187 2024-09-14T09:07:07.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.272 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282237+0000", 2024-09-14T09:07:07.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1200000000000002e-06 2024-09-14T09:07:07.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.272 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297942+0000", 2024-09-14T09:07:07.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0157048040000001 2024-09-14T09:07:07.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298087+0000", 2024-09-14T09:07:07.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014499099999999999 2024-09-14T09:07:07.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298211+0000", 2024-09-14T09:07:07.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012461899999999999 2024-09-14T09:07:07.273 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.273 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.274 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.274 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73555 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.274 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:07.274 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.926672828000001, 2024-09-14T09:07:07.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.02941558, 2024-09-14T09:07:07.274 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.274 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.274 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.274 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.274 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.274 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.275 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73555 2024-09-14T09:07:07.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.275 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:07.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:07.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269237+0000", 2024-09-14T09:07:07.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3399999999999999e-07 2024-09-14T09:07:07.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269240+0000", 2024-09-14T09:07:07.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8480000000000001e-06 2024-09-14T09:07:07.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269249+0000", 2024-09-14T09:07:07.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1880000000000008e-06 2024-09-14T09:07:07.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269256+0000", 2024-09-14T09:07:07.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:07:07.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282248+0000", 2024-09-14T09:07:07.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012992327 2024-09-14T09:07:07.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282251+0000", 2024-09-14T09:07:07.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2160000000000002e-06 2024-09-14T09:07:07.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298299+0000", 2024-09-14T09:07:07.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0160479330000001 2024-09-14T09:07:07.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298446+0000", 2024-09-14T09:07:07.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000146525 2024-09-14T09:07:07.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298652+0000", 2024-09-14T09:07:07.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020584100000000001 2024-09-14T09:07:07.279 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.279 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.279 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.279 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73556 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.279 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:07.279 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.926643081999998, 2024-09-14T09:07:07.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0299854239999999, 2024-09-14T09:07:07.279 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.279 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.279 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.280 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.280 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.280 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.280 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73556 2024-09-14T09:07:07.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.280 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:07.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:07.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:07.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:07.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269270+0000", 2024-09-14T09:07:07.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0920000000000002e-06 2024-09-14T09:07:07.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269280+0000", 2024-09-14T09:07:07.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0013999999999999e-05 2024-09-14T09:07:07.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269288+0000", 2024-09-14T09:07:07.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:07:07.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282263+0000", 2024-09-14T09:07:07.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012974703000000001 2024-09-14T09:07:07.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282266+0000", 2024-09-14T09:07:07.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1520000000000001e-06 2024-09-14T09:07:07.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298729+0000", 2024-09-14T09:07:07.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.016463506 2024-09-14T09:07:07.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298884+0000", 2024-09-14T09:07:07.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015450800000000001 2024-09-14T09:07:07.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299251+0000", 2024-09-14T09:07:07.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036759899999999997 2024-09-14T09:07:07.284 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.284 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.284 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.284 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73557 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16547840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.284 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:07.284 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.926596431999997, 2024-09-14T09:07:07.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0307969480000001, 2024-09-14T09:07:07.284 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.284 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.284 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.285 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.285 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.285 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.285 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73557 2024-09-14T09:07:07.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.285 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:07.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:07.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269313+0000", 2024-09-14T09:07:07.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.92e-07 2024-09-14T09:07:07.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269336+0000", 2024-09-14T09:07:07.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3118000000000001e-05 2024-09-14T09:07:07.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269346+0000", 2024-09-14T09:07:07.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4069999999999999e-06 2024-09-14T09:07:07.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269352+0000", 2024-09-14T09:07:07.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5810000000000002e-06 2024-09-14T09:07:07.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282277+0000", 2024-09-14T09:07:07.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012924658 2024-09-14T09:07:07.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282280+0000", 2024-09-14T09:07:07.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2880000000000001e-06 2024-09-14T09:07:07.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299343+0000", 2024-09-14T09:07:07.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0170630190000001 2024-09-14T09:07:07.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299554+0000", 2024-09-14T09:07:07.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021085099999999999 2024-09-14T09:07:07.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300109+0000", 2024-09-14T09:07:07.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00055533399999999999 2024-09-14T09:07:07.289 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.289 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.289 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.289 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73558 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.289 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:07.289 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.926544808999999, 2024-09-14T09:07:07.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0314849340000001, 2024-09-14T09:07:07.289 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.289 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.289 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.289 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.290 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.290 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.290 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73558 2024-09-14T09:07:07.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.290 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:07.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:07.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269365+0000", 2024-09-14T09:07:07.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7400000000000003e-07 2024-09-14T09:07:07.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269367+0000", 2024-09-14T09:07:07.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4930000000000001e-06 2024-09-14T09:07:07.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269374+0000", 2024-09-14T09:07:07.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9519999999999997e-06 2024-09-14T09:07:07.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269379+0000", 2024-09-14T09:07:07.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9710000000000003e-06 2024-09-14T09:07:07.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282292+0000", 2024-09-14T09:07:07.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012912704000000001 2024-09-14T09:07:07.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282295+0000", 2024-09-14T09:07:07.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.179e-06 2024-09-14T09:07:07.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300195+0000", 2024-09-14T09:07:07.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.017899975 2024-09-14T09:07:07.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300345+0000", 2024-09-14T09:07:07.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014957499999999999 2024-09-14T09:07:07.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300849+0000", 2024-09-14T09:07:07.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050451100000000002 2024-09-14T09:07:07.293 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.294 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.294 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.294 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73559 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:07.294 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:07.294 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.926522400000003, 2024-09-14T09:07:07.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.031916343, 2024-09-14T09:07:07.294 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.294 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.294 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.294 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.294 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.294 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.295 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73559 2024-09-14T09:07:07.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.295 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:07.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:07.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:07.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3500000000000002e-07 2024-09-14T09:07:07.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269390+0000", 2024-09-14T09:07:07.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5890000000000001e-06 2024-09-14T09:07:07.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.296 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269396+0000", 2024-09-14T09:07:07.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3300000000000004e-06 2024-09-14T09:07:07.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.296 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269401+0000", 2024-09-14T09:07:07.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0340000000000004e-06 2024-09-14T09:07:07.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282311+0000", 2024-09-14T09:07:07.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012910204 2024-09-14T09:07:07.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:07.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282317+0000", 2024-09-14T09:07:07.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2639999999999997e-06 2024-09-14T09:07:07.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300933+0000", 2024-09-14T09:07:07.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0186160710000001 2024-09-14T09:07:07.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301158+0000", 2024-09-14T09:07:07.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225021 2024-09-14T09:07:07.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301303+0000", 2024-09-14T09:07:07.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014439499999999999 2024-09-14T09:07:07.299 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.299 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.299 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.299 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.299 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.299 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:07.301 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:07:07.301 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:07.299+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:07.401 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:07.399+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:07.473 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:07.473 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:07.473 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:07.473 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:07.488 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:07:07.502 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:07.503+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:07.603 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:07.603+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:07.664 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:07.664 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:07.664 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:07.664 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:07.664 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:07.678 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:07:07.722 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:07.723+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:07.806 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:07.807+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:07.858 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:07.858 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:07.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:07.858 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:07.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.859 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18383 3.9 3:9252f7c9:::rbd_data.115e3abd3b2c.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:07:07.859 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:07:07.859 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 84.697892684999999, 2024-09-14T09:07:07.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056433830999999997, 2024-09-14T09:07:07.859 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.859 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.859 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:07.859 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.859 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.859 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.859 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18383 2024-09-14T09:07:07.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.860 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:07:07.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:07:07.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158423+0000", 2024-09-14T09:07:07.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T09:07:07.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158429+0000", 2024-09-14T09:07:07.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0179999999999998e-06 2024-09-14T09:07:07.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158440+0000", 2024-09-14T09:07:07.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0769e-05 2024-09-14T09:07:07.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158445+0000", 2024-09-14T09:07:07.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5840000000000002e-06 2024-09-14T09:07:07.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158495+0000", 2024-09-14T09:07:07.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9129999999999999e-05 2024-09-14T09:07:07.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158598+0000", 2024-09-14T09:07:07.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102938 2024-09-14T09:07:07.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:07.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.161048+0000", 2024-09-14T09:07:07.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002450336 2024-09-14T09:07:07.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:07.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214605+0000", 2024-09-14T09:07:07.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053557526000000001 2024-09-14T09:07:07.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:07.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214634+0000", 2024-09-14T09:07:07.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8483000000000002e-05 2024-09-14T09:07:07.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:07.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214731+0000", 2024-09-14T09:07:07.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7026000000000002e-05 2024-09-14T09:07:07.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214856+0000", 2024-09-14T09:07:07.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012474799999999999 2024-09-14T09:07:07.864 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.864 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.864 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.864 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18383 3.9 e38/34 ondisk, result = 0)", 2024-09-14T09:07:07.865 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:07:07.865 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 84.687576203999996, 2024-09-14T09:07:07.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045880575999999999, 2024-09-14T09:07:07.865 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.865 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.865 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.865 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:07:07.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:07:07.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168739+0000", 2024-09-14T09:07:07.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3900000000000004e-07 2024-09-14T09:07:07.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168744+0000", 2024-09-14T09:07:07.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5929999999999999e-06 2024-09-14T09:07:07.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168754+0000", 2024-09-14T09:07:07.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0686999999999999e-05 2024-09-14T09:07:07.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168761+0000", 2024-09-14T09:07:07.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6340000000000003e-06 2024-09-14T09:07:07.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214578+0000", 2024-09-14T09:07:07.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045817443999999999 2024-09-14T09:07:07.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214596+0000", 2024-09-14T09:07:07.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7159e-05 2024-09-14T09:07:07.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214619+0000", 2024-09-14T09:07:07.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.332e-05 2024-09-14T09:07:07.868 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.868 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.868 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.868 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19152 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14159058~8940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:07:07.868 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:07:07.868 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 83.533663747999995, 2024-09-14T09:07:07.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12891994500000001, 2024-09-14T09:07:07.869 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.869 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.869 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:07.869 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.869 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.869 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.869 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19152 2024-09-14T09:07:07.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.869 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:07:07.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:07:07.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322653+0000", 2024-09-14T09:07:07.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3750000000000001e-06 2024-09-14T09:07:07.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329647+0000", 2024-09-14T09:07:07.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069941960000000003 2024-09-14T09:07:07.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329673+0000", 2024-09-14T09:07:07.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5760000000000001e-05 2024-09-14T09:07:07.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329684+0000", 2024-09-14T09:07:07.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0474e-05 2024-09-14T09:07:07.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329776+0000", 2024-09-14T09:07:07.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2467999999999996e-05 2024-09-14T09:07:07.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329987+0000", 2024-09-14T09:07:07.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021133 2024-09-14T09:07:07.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:07.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.331303+0000", 2024-09-14T09:07:07.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001315629 2024-09-14T09:07:07.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:07.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.411090+0000", 2024-09-14T09:07:07.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079787477999999995 2024-09-14T09:07:07.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:07.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451463+0000", 2024-09-14T09:07:07.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040372312 2024-09-14T09:07:07.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:07.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451523+0000", 2024-09-14T09:07:07.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0420000000000001e-05 2024-09-14T09:07:07.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451571+0000", 2024-09-14T09:07:07.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7503000000000002e-05 2024-09-14T09:07:07.874 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.874 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.874 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.874 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20926 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:07.874 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:07.874 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.184331748000005, 2024-09-14T09:07:07.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018572207, 2024-09-14T09:07:07.874 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.875 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.875 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.875 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20926 2024-09-14T09:07:07.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.875 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:07.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:07.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:07.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:07:07.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671990+0000", 2024-09-14T09:07:07.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5769999999999999e-06 2024-09-14T09:07:07.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672001+0000", 2024-09-14T09:07:07.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1293e-05 2024-09-14T09:07:07.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672008+0000", 2024-09-14T09:07:07.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9099999999999999e-06 2024-09-14T09:07:07.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689147+0000", 2024-09-14T09:07:07.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017139060000000001 2024-09-14T09:07:07.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689324+0000", 2024-09-14T09:07:07.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017640200000000001 2024-09-14T09:07:07.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690555+0000", 2024-09-14T09:07:07.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001231212 2024-09-14T09:07:07.878 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.878 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.878 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.878 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20927 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2621440~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:07.878 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:07.878 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.184294781999995, 2024-09-14T09:07:07.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020002227000000001, 2024-09-14T09:07:07.879 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.879 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.879 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.879 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.879 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.879 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.879 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20927 2024-09-14T09:07:07.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.879 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:07.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:07.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:07.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9699999999999995e-07 2024-09-14T09:07:07.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672024+0000", 2024-09-14T09:07:07.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:07:07.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672036+0000", 2024-09-14T09:07:07.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.119e-05 2024-09-14T09:07:07.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672044+0000", 2024-09-14T09:07:07.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8370000000000003e-06 2024-09-14T09:07:07.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690602+0000", 2024-09-14T09:07:07.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018557648 2024-09-14T09:07:07.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690805+0000", 2024-09-14T09:07:07.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020282899999999999 2024-09-14T09:07:07.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692022+0000", 2024-09-14T09:07:07.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012170429999999999 2024-09-14T09:07:07.882 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.882 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.882 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.882 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20928 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 917504~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:07.882 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:07:07.882 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.184258642000003, 2024-09-14T09:07:07.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021623738, 2024-09-14T09:07:07.883 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.883 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.883 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.883 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.883 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.883 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.883 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20928 2024-09-14T09:07:07.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.883 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:07:07.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:07:07.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672057+0000", 2024-09-14T09:07:07.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:07:07.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672059+0000", 2024-09-14T09:07:07.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9270000000000001e-06 2024-09-14T09:07:07.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672070+0000", 2024-09-14T09:07:07.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0773e-05 2024-09-14T09:07:07.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672077+0000", 2024-09-14T09:07:07.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9600000000000003e-06 2024-09-14T09:07:07.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692154+0000", 2024-09-14T09:07:07.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020076482 2024-09-14T09:07:07.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692419+0000", 2024-09-14T09:07:07.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026517400000000002 2024-09-14T09:07:07.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693680+0000", 2024-09-14T09:07:07.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012607390000000001 2024-09-14T09:07:07.886 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.886 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.886 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.886 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20929 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2752512~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:07.886 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:07:07.886 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.184225600000005, 2024-09-14T09:07:07.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022898728, 2024-09-14T09:07:07.887 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.887 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.887 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.887 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.887 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.887 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.887 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20929 2024-09-14T09:07:07.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.887 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:07:07.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:07:07.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672090+0000", 2024-09-14T09:07:07.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:07:07.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672092+0000", 2024-09-14T09:07:07.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8940000000000002e-06 2024-09-14T09:07:07.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672103+0000", 2024-09-14T09:07:07.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0815999999999999e-05 2024-09-14T09:07:07.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672110+0000", 2024-09-14T09:07:07.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9469999999999997e-06 2024-09-14T09:07:07.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693737+0000", 2024-09-14T09:07:07.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021626467999999999 2024-09-14T09:07:07.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693962+0000", 2024-09-14T09:07:07.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225474 2024-09-14T09:07:07.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.694988+0000", 2024-09-14T09:07:07.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001025469 2024-09-14T09:07:07.890 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.890 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.890 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.890 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20930 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:07.890 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:07:07.890 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.184192506000002, 2024-09-14T09:07:07.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024303286, 2024-09-14T09:07:07.891 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.891 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.891 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.891 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.891 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.891 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.891 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20930 2024-09-14T09:07:07.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.891 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:07:07.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:07:07.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672123+0000", 2024-09-14T09:07:07.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:07:07.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672125+0000", 2024-09-14T09:07:07.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7300000000000001e-06 2024-09-14T09:07:07.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672134+0000", 2024-09-14T09:07:07.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7930000000000003e-06 2024-09-14T09:07:07.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672141+0000", 2024-09-14T09:07:07.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9029999999999998e-06 2024-09-14T09:07:07.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695087+0000", 2024-09-14T09:07:07.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022946013000000001 2024-09-14T09:07:07.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695350+0000", 2024-09-14T09:07:07.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026266000000000001 2024-09-14T09:07:07.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696425+0000", 2024-09-14T09:07:07.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001075527 2024-09-14T09:07:07.894 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.894 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.894 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.894 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20931 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:07.894 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:07:07.895 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.184159277000006, 2024-09-14T09:07:07.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025331519, 2024-09-14T09:07:07.895 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.895 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.895 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.895 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.895 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.895 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.895 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20931 2024-09-14T09:07:07.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.895 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:07:07.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:07:07.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672156+0000", 2024-09-14T09:07:07.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4300000000000002e-07 2024-09-14T09:07:07.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672159+0000", 2024-09-14T09:07:07.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T09:07:07.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672170+0000", 2024-09-14T09:07:07.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1192999999999999e-05 2024-09-14T09:07:07.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672177+0000", 2024-09-14T09:07:07.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9059999999999996e-06 2024-09-14T09:07:07.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696532+0000", 2024-09-14T09:07:07.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024355291000000001 2024-09-14T09:07:07.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696755+0000", 2024-09-14T09:07:07.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022297699999999999 2024-09-14T09:07:07.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697487+0000", 2024-09-14T09:07:07.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00073171499999999997 2024-09-14T09:07:07.898 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.898 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.899 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.899 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20932 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1179648~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:07.899 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:07.899 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.184125834, 2024-09-14T09:07:07.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026104083, 2024-09-14T09:07:07.899 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.899 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.899 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.899 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.899 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.899 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.899 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20932 2024-09-14T09:07:07.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.900 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:07.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:07.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:07.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0699999999999996e-07 2024-09-14T09:07:07.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672192+0000", 2024-09-14T09:07:07.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7700000000000002e-06 2024-09-14T09:07:07.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672203+0000", 2024-09-14T09:07:07.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0956e-05 2024-09-14T09:07:07.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672211+0000", 2024-09-14T09:07:07.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3769999999999996e-06 2024-09-14T09:07:07.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697530+0000", 2024-09-14T09:07:07.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025318411999999998 2024-09-14T09:07:07.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697640+0000", 2024-09-14T09:07:07.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011045299999999999 2024-09-14T09:07:07.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698293+0000", 2024-09-14T09:07:07.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065240800000000005 2024-09-14T09:07:07.903 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.903 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.903 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.903 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20933 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3014656~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:07.903 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:07:07.903 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.184091631000001, 2024-09-14T09:07:07.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026558875999999999, 2024-09-14T09:07:07.904 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.904 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.904 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.904 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.904 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.904 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.904 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20933 2024-09-14T09:07:07.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.904 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:07:07.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:07:07.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672224+0000", 2024-09-14T09:07:07.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2399999999999997e-07 2024-09-14T09:07:07.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672234+0000", 2024-09-14T09:07:07.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0793e-05 2024-09-14T09:07:07.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672246+0000", 2024-09-14T09:07:07.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659e-05 2024-09-14T09:07:07.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672255+0000", 2024-09-14T09:07:07.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5730000000000008e-06 2024-09-14T09:07:07.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698326+0000", 2024-09-14T09:07:07.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026071414000000001 2024-09-14T09:07:07.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698374+0000", 2024-09-14T09:07:07.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8032999999999999e-05 2024-09-14T09:07:07.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698782+0000", 2024-09-14T09:07:07.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040768 2024-09-14T09:07:07.907 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.907 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.907 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.908 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20934 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1310720~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:07.908 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:07:07.908 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.184049755000004, 2024-09-14T09:07:07.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027292067999999999, 2024-09-14T09:07:07.908 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.908 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.908 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.908 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.908 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.908 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.908 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20934 2024-09-14T09:07:07.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.908 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:07:07.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:07:07.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672266+0000", 2024-09-14T09:07:07.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6299999999999993e-07 2024-09-14T09:07:07.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672269+0000", 2024-09-14T09:07:07.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8569999999999999e-06 2024-09-14T09:07:07.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672280+0000", 2024-09-14T09:07:07.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1236999999999999e-05 2024-09-14T09:07:07.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672287+0000", 2024-09-14T09:07:07.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8260000000000004e-06 2024-09-14T09:07:07.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698820+0000", 2024-09-14T09:07:07.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026533055999999999 2024-09-14T09:07:07.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698933+0000", 2024-09-14T09:07:07.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011291 2024-09-14T09:07:07.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699557+0000", 2024-09-14T09:07:07.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062421899999999997 2024-09-14T09:07:07.911 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.911 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.911 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.912 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20935 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:07.912 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:07:07.912 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.184015994999996, 2024-09-14T09:07:07.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028108610999999999, 2024-09-14T09:07:07.912 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.912 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.912 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.912 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.912 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.912 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.912 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20935 2024-09-14T09:07:07.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.913 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:07:07.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:07:07.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672299+0000", 2024-09-14T09:07:07.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T09:07:07.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672302+0000", 2024-09-14T09:07:07.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7640000000000001e-06 2024-09-14T09:07:07.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672313+0000", 2024-09-14T09:07:07.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1080000000000001e-05 2024-09-14T09:07:07.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672321+0000", 2024-09-14T09:07:07.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6490000000000008e-06 2024-09-14T09:07:07.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699587+0000", 2024-09-14T09:07:07.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027265884000000001 2024-09-14T09:07:07.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699631+0000", 2024-09-14T09:07:07.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4947999999999997e-05 2024-09-14T09:07:07.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700407+0000", 2024-09-14T09:07:07.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00077559299999999999 2024-09-14T09:07:07.916 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.916 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.916 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.916 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20936 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1441792~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:07.916 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:07.916 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.183981818999996, 2024-09-14T09:07:07.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028784750000000001, 2024-09-14T09:07:07.917 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.917 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.917 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.917 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.917 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.917 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.917 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20936 2024-09-14T09:07:07.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.917 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:07.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:07.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:07.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4700000000000001e-07 2024-09-14T09:07:07.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672336+0000", 2024-09-14T09:07:07.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6730000000000001e-06 2024-09-14T09:07:07.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672347+0000", 2024-09-14T09:07:07.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0710000000000001e-05 2024-09-14T09:07:07.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672354+0000", 2024-09-14T09:07:07.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4630000000000004e-06 2024-09-14T09:07:07.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700442+0000", 2024-09-14T09:07:07.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028087951 2024-09-14T09:07:07.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700557+0000", 2024-09-14T09:07:07.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011461800000000001 2024-09-14T09:07:07.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701117+0000", 2024-09-14T09:07:07.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056058799999999999 2024-09-14T09:07:07.921 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.921 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.921 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.921 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20937 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:07.921 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:07.921 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.183948693000005, 2024-09-14T09:07:07.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029493028000000001, 2024-09-14T09:07:07.921 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.921 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.921 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.921 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.922 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20937 2024-09-14T09:07:07.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.922 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:07.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:07.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:07.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T09:07:07.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672384+0000", 2024-09-14T09:07:07.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7127000000000001e-05 2024-09-14T09:07:07.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672395+0000", 2024-09-14T09:07:07.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759999999999999e-05 2024-09-14T09:07:07.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672402+0000", 2024-09-14T09:07:07.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0960000000000003e-06 2024-09-14T09:07:07.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701147+0000", 2024-09-14T09:07:07.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028744241 2024-09-14T09:07:07.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701246+0000", 2024-09-14T09:07:07.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9031999999999996e-05 2024-09-14T09:07:07.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701859+0000", 2024-09-14T09:07:07.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061310499999999997 2024-09-14T09:07:07.925 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.925 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.925 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.925 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20938 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:07.925 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:07:07.925 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.183901532999997, 2024-09-14T09:07:07.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030109219999999999, 2024-09-14T09:07:07.926 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.926 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.926 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.926 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.926 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.926 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.926 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20938 2024-09-14T09:07:07.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.926 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:07:07.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:07:07.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672414+0000", 2024-09-14T09:07:07.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-07 2024-09-14T09:07:07.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672417+0000", 2024-09-14T09:07:07.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:07:07.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672428+0000", 2024-09-14T09:07:07.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0873000000000001e-05 2024-09-14T09:07:07.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672435+0000", 2024-09-14T09:07:07.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:07:07.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701884+0000", 2024-09-14T09:07:07.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029449811999999999 2024-09-14T09:07:07.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701934+0000", 2024-09-14T09:07:07.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9703000000000002e-05 2024-09-14T09:07:07.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702522+0000", 2024-09-14T09:07:07.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00058811200000000001 2024-09-14T09:07:07.929 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.929 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.929 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.929 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20939 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:07.930 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:07:07.930 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.183869400000006, 2024-09-14T09:07:07.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030779259, 2024-09-14T09:07:07.930 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.930 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.930 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.930 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.930 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.930 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.930 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20939 2024-09-14T09:07:07.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.930 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:07:07.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:07:07.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672446+0000", 2024-09-14T09:07:07.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:07:07.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672449+0000", 2024-09-14T09:07:07.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7930000000000002e-06 2024-09-14T09:07:07.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672461+0000", 2024-09-14T09:07:07.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T09:07:07.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672471+0000", 2024-09-14T09:07:07.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0807e-05 2024-09-14T09:07:07.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702551+0000", 2024-09-14T09:07:07.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030079228 2024-09-14T09:07:07.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702664+0000", 2024-09-14T09:07:07.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011351800000000001 2024-09-14T09:07:07.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.703224+0000", 2024-09-14T09:07:07.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056026999999999997 2024-09-14T09:07:07.934 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.934 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.934 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.934 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:74298 3.9 3:95c8282a:::rbd_data.115ea8d131ff.0000000000000041:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:07:07.934 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:07:07.934 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.058559387999999, 2024-09-14T09:07:07.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042668064999999998, 2024-09-14T09:07:07.934 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.934 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.934 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:07.934 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:07.934 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:07.935 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:07.935 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74298 2024-09-14T09:07:07.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.935 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:07:07.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:07:07.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797758+0000", 2024-09-14T09:07:07.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9799999999999998e-06 2024-09-14T09:07:07.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797770+0000", 2024-09-14T09:07:07.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1667e-05 2024-09-14T09:07:07.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797797+0000", 2024-09-14T09:07:07.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7212999999999998e-05 2024-09-14T09:07:07.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797811+0000", 2024-09-14T09:07:07.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4256e-05 2024-09-14T09:07:07.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797882+0000", 2024-09-14T09:07:07.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0622000000000002e-05 2024-09-14T09:07:07.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797943+0000", 2024-09-14T09:07:07.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0893000000000001e-05 2024-09-14T09:07:07.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:07.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.799676+0000", 2024-09-14T09:07:07.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0017331619999999999 2024-09-14T09:07:07.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:07.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840109+0000", 2024-09-14T09:07:07.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040432994 2024-09-14T09:07:07.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:07.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840204+0000", 2024-09-14T09:07:07.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5247000000000005e-05 2024-09-14T09:07:07.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:07.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840346+0000", 2024-09-14T09:07:07.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014156499999999999 2024-09-14T09:07:07.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840423+0000", 2024-09-14T09:07:07.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7466000000000007e-05 2024-09-14T09:07:07.940 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.940 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.940 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.940 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:74298 3.9 e76/34 ondisk, result = 0)", 2024-09-14T09:07:07.940 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:07:07.940 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 32.052164744999999, 2024-09-14T09:07:07.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036289318000000001, 2024-09-14T09:07:07.940 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.941 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.941 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:07.941 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:07:07.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:07:07.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804151+0000", 2024-09-14T09:07:07.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6700000000000001e-06 2024-09-14T09:07:07.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804163+0000", 2024-09-14T09:07:07.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.148e-05 2024-09-14T09:07:07.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804197+0000", 2024-09-14T09:07:07.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3909999999999999e-05 2024-09-14T09:07:07.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804217+0000", 2024-09-14T09:07:07.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0287000000000001e-05 2024-09-14T09:07:07.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840176+0000", 2024-09-14T09:07:07.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035958641 2024-09-14T09:07:07.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840193+0000", 2024-09-14T09:07:07.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6986e-05 2024-09-14T09:07:07.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840439+0000", 2024-09-14T09:07:07.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024634400000000001 2024-09-14T09:07:07.944 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.944 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.944 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.944 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:93074 3.9 e94/84 3:925bb5bd:::rbd_data.115e7770ddc5.0000000000000001:head v 94'255, mlcod=94'252)", 2024-09-14T09:07:07.944 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:57.863727+0000", 2024-09-14T09:07:07.944 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.9925874849999996, 2024-09-14T09:07:07.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019329799000000002, 2024-09-14T09:07:07.945 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:07.945 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:07.945 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:07.945 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:07.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:07.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.863727+0000", 2024-09-14T09:07:07.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:07.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.863727+0000", 2024-09-14T09:07:07.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:07.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:07.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.863729+0000", 2024-09-14T09:07:07.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3980000000000002e-06 2024-09-14T09:07:07.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:07.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866073+0000", 2024-09-14T09:07:07.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0023435109999999999 2024-09-14T09:07:07.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:07.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866094+0000", 2024-09-14T09:07:07.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1444e-05 2024-09-14T09:07:07.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:07.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866105+0000", 2024-09-14T09:07:07.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0949e-05 2024-09-14T09:07:07.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:07.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866216+0000", 2024-09-14T09:07:07.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110961 2024-09-14T09:07:07.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:07.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866258+0000", 2024-09-14T09:07:07.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1903000000000002e-05 2024-09-14T09:07:07.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:07.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.882981+0000", 2024-09-14T09:07:07.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016723198000000002 2024-09-14T09:07:07.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:07.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:07.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:07.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.883057+0000", 2024-09-14T09:07:07.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5434999999999999e-05 2024-09-14T09:07:07.948 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.948 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.948 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.948 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:07.948 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:07.948 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:07.950 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:07:07.950 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:07.951+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:08.051 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:08.055+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:08.129 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:08.129 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:08.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.129 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:96708 3.15 3.3f1ef5b5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-14T09:07:08.129 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:07.856873+0000", 2024-09-14T09:07:08.129 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.27340101999999999, 2024-09-14T09:07:08.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.27344154300000001, 2024-09-14T09:07:08.129 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.129 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.129 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:08.129 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:08.129 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:08.129 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:08.130 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 96708 2024-09-14T09:07:08.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.130 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:07.856873+0000", 2024-09-14T09:07:08.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:07.856873+0000", 2024-09-14T09:07:08.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:07.856876+0000", 2024-09-14T09:07:08.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5659999999999998e-06 2024-09-14T09:07:08.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:07.856892+0000", 2024-09-14T09:07:08.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5860000000000001e-05 2024-09-14T09:07:08.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:07.856933+0000", 2024-09-14T09:07:08.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0689999999999998e-05 2024-09-14T09:07:08.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:07.856955+0000", 2024-09-14T09:07:08.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2027000000000001e-05 2024-09-14T09:07:08.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:07.857065+0000", 2024-09-14T09:07:08.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000109843 2024-09-14T09:07:08.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:08.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:07.857081+0000", 2024-09-14T09:07:08.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6605000000000001e-05 2024-09-14T09:07:08.132 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.132 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.132 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.133 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.133 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:08.133 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 1 2024-09-14T09:07:08.133 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:08.143 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:07:08.190 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:08.191+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:08.253 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:08.251+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:08.265 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:08.265 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:08.265 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:08.265 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:08.265 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:08.279 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:07:08.354 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:08.355+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:08.449 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:08.449 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:08.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:08.449 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:08.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.449 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:08.449 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:08.450 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.363793944999998, 2024-09-14T09:07:08.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:07:08.450 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.450 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.450 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:08.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:08.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:08.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:08.450 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:07:08.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.450 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:08.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:08.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:07:08.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:07:08.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:07:08.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:07:08.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:07:08.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:07:08.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:07:08.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:07:08.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:07:08.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:07:08.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:08.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:07:08.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:07:08.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:07:08.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:07:08.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:07:08.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:07:08.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:07:08.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:07:08.454 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.454 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.454 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.454 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:08.454 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:08.455 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.363739287000001, 2024-09-14T09:07:08.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:07:08.455 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.455 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.455 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:08.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:08.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:08.455 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:08.455 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:07:08.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.455 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:08.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:08.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:08.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:07:08.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:07:08.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:07:08.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:07:08.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:07:08.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:07:08.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:07:08.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:07:08.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:07:08.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:08.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:07:08.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:07:08.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:07:08.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:07:08.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:07:08.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:07:08.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:07:08.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:07:08.459 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.459 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.459 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.459 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:08.459 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:08.460 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.363633913000001, 2024-09-14T09:07:08.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:07:08.460 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.460 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.460 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:08.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:08.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:08.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:08.460 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:07:08.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.460 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:08.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:08.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:07:08.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:07:08.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:07:08.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:07:08.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:07:08.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:07:08.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:07:08.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:07:08.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:07:08.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:07:08.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:08.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:07:08.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:07:08.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:07:08.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:07:08.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:07:08.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:07:08.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:07:08.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:07:08.464 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.464 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.465 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.465 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:08.465 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:08.465 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.363277148999998, 2024-09-14T09:07:08.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:07:08.465 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.465 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.465 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:08.465 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:08.465 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:08.465 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:08.465 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:07:08.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.466 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:08.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:08.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:07:08.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:07:08.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:07:08.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:07:08.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:07:08.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:07:08.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:07:08.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:07:08.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:07:08.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:07:08.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:08.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:07:08.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:07:08.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:07:08.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:07:08.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:07:08.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:07:08.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:07:08.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:07:08.469 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.470 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.470 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.470 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:08.470 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:08.470 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.363192411, 2024-09-14T09:07:08.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:07:08.470 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.470 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.470 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:08.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:08.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:08.471 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:08.471 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:07:08.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.471 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:08.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:08.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:07:08.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:07:08.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:07:08.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:07:08.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:07:08.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:07:08.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:07:08.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:07:08.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:07:08.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:07:08.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:08.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:07:08.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:07:08.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:07:08.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:07:08.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:07:08.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:07:08.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:07:08.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:07:08.475 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.475 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.475 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.475 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:08.475 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:08.475 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.363104724999999, 2024-09-14T09:07:08.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:07:08.475 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.476 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.476 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:08.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:08.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:08.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:08.476 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:07:08.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.476 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:08.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:08.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:07:08.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:07:08.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:07:08.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:07:08.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:07:08.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:07:08.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:07:08.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:07:08.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:07:08.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:07:08.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:08.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:07:08.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:07:08.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:07:08.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:07:08.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:07:08.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:07:08.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:07:08.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:07:08.480 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.480 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.480 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.480 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18388 3.18 3:19e2be55:::rbd_data.115e3abd3b2c.0000000000000021:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:07:08.480 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:07:08.480 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 85.227191414000004, 2024-09-14T09:07:08.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042293648000000003, 2024-09-14T09:07:08.481 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.481 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.481 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:08.481 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:08.481 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:08.481 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:08.481 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18388 2024-09-14T09:07:08.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.481 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:07:08.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220233+0000", 2024-09-14T09:07:08.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220236+0000", 2024-09-14T09:07:08.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1339999999999998e-06 2024-09-14T09:07:08.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220244+0000", 2024-09-14T09:07:08.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1950000000000003e-06 2024-09-14T09:07:08.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220262+0000", 2024-09-14T09:07:08.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8017e-05 2024-09-14T09:07:08.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220270+0000", 2024-09-14T09:07:08.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9480000000000008e-06 2024-09-14T09:07:08.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220357+0000", 2024-09-14T09:07:08.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7185e-05 2024-09-14T09:07:08.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.220576+0000", 2024-09-14T09:07:08.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021912200000000001 2024-09-14T09:07:08.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:08.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.222435+0000", 2024-09-14T09:07:08.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0018592350000000001 2024-09-14T09:07:08.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:08.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262439+0000", 2024-09-14T09:07:08.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040003368999999997 2024-09-14T09:07:08.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:08.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262452+0000", 2024-09-14T09:07:08.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2941e-05 2024-09-14T09:07:08.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:08.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262490+0000", 2024-09-14T09:07:08.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8478e-05 2024-09-14T09:07:08.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.262527+0000", 2024-09-14T09:07:08.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7024000000000003e-05 2024-09-14T09:07:08.486 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.486 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.486 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.486 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38002 3.1e 3:7df715aa:::rbd_data.115eb07a8abd.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:07:08.486 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:07:08.486 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.057673507999993, 2024-09-14T09:07:08.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.065148304000000004, 2024-09-14T09:07:08.486 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.486 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.486 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:08.486 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:08.486 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:08.486 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:08.487 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38002 2024-09-14T09:07:08.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.487 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:07:08.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:07:08.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389752+0000", 2024-09-14T09:07:08.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1829999999999999e-06 2024-09-14T09:07:08.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389758+0000", 2024-09-14T09:07:08.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0919999999999997e-06 2024-09-14T09:07:08.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389769+0000", 2024-09-14T09:07:08.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759e-05 2024-09-14T09:07:08.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389775+0000", 2024-09-14T09:07:08.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8819999999999998e-06 2024-09-14T09:07:08.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389824+0000", 2024-09-14T09:07:08.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8573999999999999e-05 2024-09-14T09:07:08.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389876+0000", 2024-09-14T09:07:08.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2643999999999999e-05 2024-09-14T09:07:08.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:07:08.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.392640+0000", 2024-09-14T09:07:08.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0027636409999999998 2024-09-14T09:07:08.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:08.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454672+0000", 2024-09-14T09:07:08.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062031703000000001 2024-09-14T09:07:08.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:08.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454765+0000", 2024-09-14T09:07:08.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2862000000000006e-05 2024-09-14T09:07:08.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:08.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454841+0000", 2024-09-14T09:07:08.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5857000000000002e-05 2024-09-14T09:07:08.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454900+0000", 2024-09-14T09:07:08.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9107000000000002e-05 2024-09-14T09:07:08.491 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.491 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.491 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.491 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:38002 3.1e e48/23 ondisk, result = 0)", 2024-09-14T09:07:08.491 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:07:08.491 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.047356918999995, 2024-09-14T09:07:08.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054841741999999999, 2024-09-14T09:07:08.491 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.491 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.492 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:08.492 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:07:08.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:07:08.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400069+0000", 2024-09-14T09:07:08.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.339e-06 2024-09-14T09:07:08.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400076+0000", 2024-09-14T09:07:08.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7800000000000003e-06 2024-09-14T09:07:08.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400093+0000", 2024-09-14T09:07:08.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6915999999999999e-05 2024-09-14T09:07:08.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400106+0000", 2024-09-14T09:07:08.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2785e-05 2024-09-14T09:07:08.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454713+0000", 2024-09-14T09:07:08.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054607041000000002 2024-09-14T09:07:08.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454724+0000", 2024-09-14T09:07:08.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1202e-05 2024-09-14T09:07:08.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454910+0000", 2024-09-14T09:07:08.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018567900000000001 2024-09-14T09:07:08.495 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.495 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.495 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.495 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38100 3.b 3:d0f67c24:::rbd_data.115eb07a8abd.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:07:08.495 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:07:08.495 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 70.801503259, 2024-09-14T09:07:08.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053701964999999997, 2024-09-14T09:07:08.495 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.495 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.495 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:08.496 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:08.496 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:08.496 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:08.496 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38100 2024-09-14T09:07:08.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.496 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:07:08.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:07:08.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645923+0000", 2024-09-14T09:07:08.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:07:08.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645928+0000", 2024-09-14T09:07:08.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0699999999999997e-06 2024-09-14T09:07:08.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645942+0000", 2024-09-14T09:07:08.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4069e-05 2024-09-14T09:07:08.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645951+0000", 2024-09-14T09:07:08.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3230000000000004e-06 2024-09-14T09:07:08.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646008+0000", 2024-09-14T09:07:08.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6583999999999999e-05 2024-09-14T09:07:08.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646089+0000", 2024-09-14T09:07:08.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1557999999999994e-05 2024-09-14T09:07:08.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:07:08.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.650355+0000", 2024-09-14T09:07:08.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004265525 2024-09-14T09:07:08.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:08.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699439+0000", 2024-09-14T09:07:08.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049083904999999997 2024-09-14T09:07:08.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:08.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699467+0000", 2024-09-14T09:07:08.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8198999999999999e-05 2024-09-14T09:07:08.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:08.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699552+0000", 2024-09-14T09:07:08.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T09:07:08.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699624+0000", 2024-09-14T09:07:08.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1725999999999997e-05 2024-09-14T09:07:08.501 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.501 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.501 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.501 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:07:08.501 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:08.501 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.286812738000002, 2024-09-14T09:07:08.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:07:08.501 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.501 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.501 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:08.501 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:08.501 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:08.502 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:08.502 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:07:08.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.502 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:08.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:08.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:07:08.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:07:08.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:07:08.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:07:08.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:07:08.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:07:08.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:07:08.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:07:08.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:07:08.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:07:08.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:07:08.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:07:08.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:08.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:07:08.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:07:08.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:08.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:07:08.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:07:08.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:08.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:07:08.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:07:08.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:08.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:07:08.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:07:08.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:07:08.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:07:08.506 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.506 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.506 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.507 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:39401 3.b e56/53)", 2024-09-14T09:07:08.507 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:07:08.507 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.260916672, 2024-09-14T09:07:08.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10443659600000001, 2024-09-14T09:07:08.507 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.507 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.507 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:08.507 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:07:08.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:07:08.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186509+0000", 2024-09-14T09:07:08.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2640000000000001e-06 2024-09-14T09:07:08.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186516+0000", 2024-09-14T09:07:08.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1070000000000001e-06 2024-09-14T09:07:08.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186533+0000", 2024-09-14T09:07:08.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7818000000000001e-05 2024-09-14T09:07:08.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186543+0000", 2024-09-14T09:07:08.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5170000000000005e-06 2024-09-14T09:07:08.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290885+0000", 2024-09-14T09:07:08.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.104342221 2024-09-14T09:07:08.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290893+0000", 2024-09-14T09:07:08.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2859999999999999e-06 2024-09-14T09:07:08.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290945+0000", 2024-09-14T09:07:08.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1382999999999999e-05 2024-09-14T09:07:08.510 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.510 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.510 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.510 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:84137 3.7 3:e15b49d9:::rbd_data.115e77b799c3.00000000000000e0:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:07:08.511 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:08.511 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.12406534, 2024-09-14T09:07:08.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11205707500000001, 2024-09-14T09:07:08.511 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.511 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.511 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:08.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:08.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:08.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:08.511 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 84137 2024-09-14T09:07:08.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.511 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:08.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:08.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323362+0000", 2024-09-14T09:07:08.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0839999999999999e-06 2024-09-14T09:07:08.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323370+0000", 2024-09-14T09:07:08.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0180000000000001e-06 2024-09-14T09:07:08.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323388+0000", 2024-09-14T09:07:08.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8326999999999999e-05 2024-09-14T09:07:08.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323397+0000", 2024-09-14T09:07:08.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1940000000000004e-06 2024-09-14T09:07:08.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323452+0000", 2024-09-14T09:07:08.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5043000000000001e-05 2024-09-14T09:07:08.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323550+0000", 2024-09-14T09:07:08.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8201000000000004e-05 2024-09-14T09:07:08.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:08.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.328273+0000", 2024-09-14T09:07:08.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0047227079999999999 2024-09-14T09:07:08.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:08.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435232+0000", 2024-09-14T09:07:08.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.106959339 2024-09-14T09:07:08.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:08.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435256+0000", 2024-09-14T09:07:08.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3774e-05 2024-09-14T09:07:08.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:08.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435314+0000", 2024-09-14T09:07:08.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7636999999999997e-05 2024-09-14T09:07:08.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435417+0000", 2024-09-14T09:07:08.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010275 2024-09-14T09:07:08.515 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.516 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.516 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.516 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:84137 3.7 e92/23 ondisk, result = 0)", 2024-09-14T09:07:08.516 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:53.341180+0000", 2024-09-14T09:07:08.516 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 15.106244756000001, 2024-09-14T09:07:08.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.094062506000000004, 2024-09-14T09:07:08.516 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.516 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.516 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:08.516 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341180+0000", 2024-09-14T09:07:08.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341180+0000", 2024-09-14T09:07:08.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341181+0000", 2024-09-14T09:07:08.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.192e-06 2024-09-14T09:07:08.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341187+0000", 2024-09-14T09:07:08.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0070000000000003e-06 2024-09-14T09:07:08.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341208+0000", 2024-09-14T09:07:08.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0321000000000001e-05 2024-09-14T09:07:08.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341219+0000", 2024-09-14T09:07:08.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1764e-05 2024-09-14T09:07:08.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435214+0000", 2024-09-14T09:07:08.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.093994438999999999 2024-09-14T09:07:08.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435225+0000", 2024-09-14T09:07:08.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1123e-05 2024-09-14T09:07:08.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435243+0000", 2024-09-14T09:07:08.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.766e-05 2024-09-14T09:07:08.519 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.519 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.519 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.520 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:92675 3.8 3:105e0099:::rbd_data.115e7770ddc5.0000000000000016:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:08.520 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:57.736053+0000", 2024-09-14T09:07:08.520 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.711371510999999, 2024-09-14T09:07:08.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073115555999999998, 2024-09-14T09:07:08.520 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.520 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.520 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:08.520 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:08.520 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:08.520 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:08.520 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 92675 2024-09-14T09:07:08.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.520 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736053+0000", 2024-09-14T09:07:08.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736053+0000", 2024-09-14T09:07:08.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736055+0000", 2024-09-14T09:07:08.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.33e-06 2024-09-14T09:07:08.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736062+0000", 2024-09-14T09:07:08.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.08e-06 2024-09-14T09:07:08.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736077+0000", 2024-09-14T09:07:08.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5605000000000001e-05 2024-09-14T09:07:08.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736088+0000", 2024-09-14T09:07:08.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0705e-05 2024-09-14T09:07:08.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736175+0000", 2024-09-14T09:07:08.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6926000000000001e-05 2024-09-14T09:07:08.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736325+0000", 2024-09-14T09:07:08.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149739 2024-09-14T09:07:08.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:08.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.740035+0000", 2024-09-14T09:07:08.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0037099250000000002 2024-09-14T09:07:08.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:08.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809001+0000", 2024-09-14T09:07:08.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.068966240999999998 2024-09-14T09:07:08.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:08.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809049+0000", 2024-09-14T09:07:08.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8572999999999997e-05 2024-09-14T09:07:08.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:08.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809102+0000", 2024-09-14T09:07:08.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2564e-05 2024-09-14T09:07:08.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809169+0000", 2024-09-14T09:07:08.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6867999999999997e-05 2024-09-14T09:07:08.525 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.525 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.525 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.525 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:92675 3.8 e94/23 ondisk, result = 0)", 2024-09-14T09:07:08.525 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:57.748881+0000", 2024-09-14T09:07:08.525 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.698543461, 2024-09-14T09:07:08.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060300578000000001, 2024-09-14T09:07:08.525 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.525 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.525 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:08.525 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748881+0000", 2024-09-14T09:07:08.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748881+0000", 2024-09-14T09:07:08.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748883+0000", 2024-09-14T09:07:08.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.491e-06 2024-09-14T09:07:08.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748887+0000", 2024-09-14T09:07:08.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5310000000000003e-06 2024-09-14T09:07:08.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748904+0000", 2024-09-14T09:07:08.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.662e-05 2024-09-14T09:07:08.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748912+0000", 2024-09-14T09:07:08.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3869999999999995e-06 2024-09-14T09:07:08.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809034+0000", 2024-09-14T09:07:08.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060121757999999997 2024-09-14T09:07:08.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809044+0000", 2024-09-14T09:07:08.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7699999999999996e-06 2024-09-14T09:07:08.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809182+0000", 2024-09-14T09:07:08.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000138021 2024-09-14T09:07:08.528 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.528 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.528 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.529 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:93277 3.8 3:105e0099:::rbd_data.115e7770ddc5.0000000000000016:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:07:08.529 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.097058+0000", 2024-09-14T09:07:08.529 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.350367200000001, 2024-09-14T09:07:08.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.065618911000000002, 2024-09-14T09:07:08.529 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.529 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.529 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:08.529 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:08.529 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:08.529 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:08.529 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 93277 2024-09-14T09:07:08.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.530 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097058+0000", 2024-09-14T09:07:08.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097058+0000", 2024-09-14T09:07:08.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097060+0000", 2024-09-14T09:07:08.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.497e-06 2024-09-14T09:07:08.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097072+0000", 2024-09-14T09:07:08.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1600000000000001e-05 2024-09-14T09:07:08.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097102+0000", 2024-09-14T09:07:08.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0552999999999997e-05 2024-09-14T09:07:08.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097124+0000", 2024-09-14T09:07:08.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1477000000000001e-05 2024-09-14T09:07:08.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097201+0000", 2024-09-14T09:07:08.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7065999999999997e-05 2024-09-14T09:07:08.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097277+0000", 2024-09-14T09:07:08.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5722000000000004e-05 2024-09-14T09:07:08.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:08.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.101895+0000", 2024-09-14T09:07:08.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004618212 2024-09-14T09:07:08.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:08.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162452+0000", 2024-09-14T09:07:08.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060557603000000002 2024-09-14T09:07:08.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:08.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162504+0000", 2024-09-14T09:07:08.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.147e-05 2024-09-14T09:07:08.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:08.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162643+0000", 2024-09-14T09:07:08.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000139245 2024-09-14T09:07:08.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162677+0000", 2024-09-14T09:07:08.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3466000000000001e-05 2024-09-14T09:07:08.534 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.534 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.534 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.535 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:93277 3.8 e95/23 ondisk, result = 0)", 2024-09-14T09:07:08.535 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.113855+0000", 2024-09-14T09:07:08.535 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.333569631, 2024-09-14T09:07:08.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048828219999999999, 2024-09-14T09:07:08.535 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.535 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.535 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:08.535 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.113855+0000", 2024-09-14T09:07:08.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.113855+0000", 2024-09-14T09:07:08.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.113857+0000", 2024-09-14T09:07:08.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3400000000000001e-06 2024-09-14T09:07:08.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.113863+0000", 2024-09-14T09:07:08.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4740000000000002e-06 2024-09-14T09:07:08.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.113883+0000", 2024-09-14T09:07:08.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9738999999999999e-05 2024-09-14T09:07:08.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.113893+0000", 2024-09-14T09:07:08.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0634e-05 2024-09-14T09:07:08.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162478+0000", 2024-09-14T09:07:08.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.048584862999999999 2024-09-14T09:07:08.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162484+0000", 2024-09-14T09:07:08.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1829999999999999e-06 2024-09-14T09:07:08.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162683+0000", 2024-09-14T09:07:08.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000198987 2024-09-14T09:07:08.538 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.538 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.538 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.538 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:95495 3.18 3:1d9f7d98:::rbd_data.115e7770ddc5.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:07:08.538 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.598437+0000", 2024-09-14T09:07:08.538 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8489881510000004, 2024-09-14T09:07:08.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.090910543999999996, 2024-09-14T09:07:08.539 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.539 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.539 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:08.539 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:08.539 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:08.539 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:08.539 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 95495 2024-09-14T09:07:08.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.539 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598437+0000", 2024-09-14T09:07:08.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598437+0000", 2024-09-14T09:07:08.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598438+0000", 2024-09-14T09:07:08.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1149999999999999e-06 2024-09-14T09:07:08.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598443+0000", 2024-09-14T09:07:08.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4589999999999997e-06 2024-09-14T09:07:08.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598455+0000", 2024-09-14T09:07:08.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1793e-05 2024-09-14T09:07:08.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598463+0000", 2024-09-14T09:07:08.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9759999999999995e-06 2024-09-14T09:07:08.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598523+0000", 2024-09-14T09:07:08.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9926000000000002e-05 2024-09-14T09:07:08.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598581+0000", 2024-09-14T09:07:08.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8285000000000002e-05 2024-09-14T09:07:08.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:08.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.602809+0000", 2024-09-14T09:07:08.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0042279079999999998 2024-09-14T09:07:08.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:08.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689202+0000", 2024-09-14T09:07:08.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.086392847999999994 2024-09-14T09:07:08.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:08.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689230+0000", 2024-09-14T09:07:08.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8501999999999999e-05 2024-09-14T09:07:08.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:08.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689290+0000", 2024-09-14T09:07:08.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9524000000000001e-05 2024-09-14T09:07:08.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689347+0000", 2024-09-14T09:07:08.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7207999999999998e-05 2024-09-14T09:07:08.544 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.544 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.544 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.544 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:95495 3.18 e95/80 ondisk, result = 0)", 2024-09-14T09:07:08.544 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.610054+0000", 2024-09-14T09:07:08.544 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.8373706859999999, 2024-09-14T09:07:08.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079165141999999994, 2024-09-14T09:07:08.544 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:08.544 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:08.544 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:08.544 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:08.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:08.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610054+0000", 2024-09-14T09:07:08.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:08.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610054+0000", 2024-09-14T09:07:08.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:08.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:08.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610055+0000", 2024-09-14T09:07:08.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2669999999999999e-06 2024-09-14T09:07:08.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:08.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610061+0000", 2024-09-14T09:07:08.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:07:08.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:08.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610078+0000", 2024-09-14T09:07:08.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6965000000000001e-05 2024-09-14T09:07:08.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:08.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610087+0000", 2024-09-14T09:07:08.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.611e-06 2024-09-14T09:07:08.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:08.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689008+0000", 2024-09-14T09:07:08.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.078921186000000004 2024-09-14T09:07:08.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:08.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689020+0000", 2024-09-14T09:07:08.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.132e-05 2024-09-14T09:07:08.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:08.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:08.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:08.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689219+0000", 2024-09-14T09:07:08.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000199469 2024-09-14T09:07:08.548 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.548 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.548 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.548 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:08.548 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:08.548 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:08.549 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:07:08.550 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:08.547+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:08.624 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:07:08.650 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:08.651+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:08.683 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:08.683 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:08.683 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:07:08.683 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:08.699 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:07:08.779 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:08.779 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:08.787 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:08.791+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:08.795 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:08.795 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:08.813 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:08.813 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:08.831 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:08.831 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:08.852 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:08.855+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:08.857 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:07:08.857 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:07:08.872 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T09:07:08.883 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:08.883 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:08.892 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:08.892 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:08.906 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:08.906 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:08.910 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:07:08.926 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:08.926 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:08.986 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:08.987+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:09.053 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:09.051+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:09.069 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:09.069 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:09.084 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:07:09.154 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:09.155+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:09.207 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:09.207 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:09.222 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:07:09.255 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:09.255+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:09.356 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:09.355+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:09.384 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:09.384 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:09.398 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:07:09.457 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:09.455+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:09.520 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:09.520 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:09.534 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:07:09.558 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:09.559+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:09.654 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:09.654 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:09.659 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:09.663+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:09.668 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:07:09.759 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:09.759+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:09.833 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:09.833 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:09.847 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:07:09.853 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is unset 2024-09-14T09:07:09.860 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:09.859+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:09.882 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-14T09:07:09.961 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:09.963+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:09.968 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:09.968 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:09.982 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:07:10.013 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6] out_osds: [7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:07:10.013 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:07:10.014 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-14T09:07:10.014 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T09:07:10.014 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T09:07:10.014 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-14T09:07:10.067 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:10.071+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:10.116 INFO:teuthology.orchestra.run.smithi176.stdout:4689 trunc from 0x9a36c81 to 0xec84fe0 2024-09-14T09:07:10.134 INFO:teuthology.orchestra.run.smithi176.stdout:4690 read 0x546432d thru 0x546463d (0x311 bytes) 2024-09-14T09:07:10.135 INFO:teuthology.orchestra.run.smithi176.stdout:4692 read 0xba760be thru 0xba843d0 (0xe313 bytes) 2024-09-14T09:07:10.135 INFO:teuthology.orchestra.run.smithi176.stdout:4693 write 0xd7b97e9 thru 0xd7c2b3d (0x9355 bytes) 2024-09-14T09:07:10.138 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:10.138 INFO:teuthology.orchestra.run.smithi183.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T09:07:10.138 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:10.140 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:10.140 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:10.145 INFO:teuthology.orchestra.run.smithi176.stdout:4694 writesame 0x17d45ab thru 0x17d6c66 data_size 0x25(0x26bc bytes) 2024-09-14T09:07:10.152 INFO:teuthology.orchestra.run.smithi176.stdout:4695 writesame 0x3a8229d thru 0x3a8eff6 data_size 0x23(0xcd5a bytes) 2024-09-14T09:07:10.158 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:07:10.161 INFO:teuthology.orchestra.run.smithi176.stdout:4696 punch from 0xac967e2 to 0xac97ba2, (0x13c0 bytes) 2024-09-14T09:07:10.162 INFO:teuthology.orchestra.run.smithi176.stdout:4698 read 0x5fbcb35 thru 0x5fcb1bd (0xe689 bytes) 2024-09-14T09:07:10.162 INFO:teuthology.orchestra.run.smithi176.stdout:4699 read 0x2eb2641 thru 0x2ebb2e2 (0x8ca2 bytes) 2024-09-14T09:07:10.162 INFO:teuthology.orchestra.run.smithi176.stdout:4700 writesame 0x9d6cdba thru 0x9d76a49 data_size 0x1e(0x9c90 bytes) 2024-09-14T09:07:10.172 INFO:teuthology.orchestra.run.smithi176.stdout:4702 write 0xa02b0db thru 0xa037ead (0xcdd3 bytes) 2024-09-14T09:07:10.180 INFO:teuthology.orchestra.run.smithi176.stdout:4705 write 0x65674a6 thru 0x6570915 (0x9470 bytes) 2024-09-14T09:07:10.191 INFO:teuthology.orchestra.run.smithi176.stdout:4706 write 0x3aef8aa thru 0x3af18db (0x2032 bytes) 2024-09-14T09:07:10.199 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:10.203+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:10.218 INFO:teuthology.orchestra.run.smithi176.stdout:4708 trunc from 0xec84fe0 to 0x3b81c2e 2024-09-14T09:07:10.255 INFO:teuthology.orchestra.run.smithi176.stdout:4710 punch from 0x2938b32 to 0x2942247, (0x9715 bytes) 2024-09-14T09:07:10.255 INFO:teuthology.orchestra.run.smithi176.stdout:4712 read 0x10a3536 thru 0x10b29a4 (0xf46f bytes) 2024-09-14T09:07:10.255 INFO:teuthology.orchestra.run.smithi176.stdout:4713 read 0x148a060 thru 0x148c7cf (0x2770 bytes) 2024-09-14T09:07:10.255 INFO:teuthology.orchestra.run.smithi176.stdout:4714 writesame 0x10c5437 thru 0x10c7fc7 data_size 0x1(0x2b91 bytes) 2024-09-14T09:07:10.263 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:10.263+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:10.288 INFO:teuthology.orchestra.run.smithi176.stdout:4715 write 0x77d83f2 thru 0x77e3687 (0xb296 bytes) 2024-09-14T09:07:10.308 INFO:teuthology.orchestra.run.smithi176.stdout:4717 trunc from 0x77e3688 to 0x8c47ad 2024-09-14T09:07:10.320 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:10.321 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:10.337 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:07:10.341 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:10.346 INFO:teuthology.orchestra.run.smithi176.stdout:4719 trunc from 0x8c47ad to 0x2b3f942 2024-09-14T09:07:10.356 INFO:teuthology.orchestra.run.smithi176.stdout:4720 punch from 0xf08f07 to 0xf10321, (0x741a bytes) 2024-09-14T09:07:10.356 INFO:teuthology.orchestra.run.smithi176.stdout:4721 punch from 0x2199850 to 0x21a33db, (0x9b8b bytes) 2024-09-14T09:07:10.356 INFO:teuthology.orchestra.run.smithi176.stdout:4722 read 0x9619cf thru 0x9633cb (0x19fd bytes) 2024-09-14T09:07:10.356 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:10.357 INFO:teuthology.orchestra.run.smithi176.stdout:4724 punch from 0x19f6153 to 0x1a009ee, (0xa89b bytes) 2024-09-14T09:07:10.357 INFO:teuthology.orchestra.run.smithi176.stdout:4726 write 0xca0e6c thru 0xca12cd (0x462 bytes) 2024-09-14T09:07:10.364 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:10.363+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:10.378 INFO:teuthology.orchestra.run.smithi176.stdout:4728 write 0x3b360a0 thru 0x3b3d91f (0x7880 bytes) 2024-09-14T09:07:10.416 INFO:teuthology.orchestra.run.smithi176.stdout:4730 write 0xb44f0ab thru 0xb456596 (0x74ec bytes) 2024-09-14T09:07:10.423 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:10.423 INFO:teuthology.orchestra.run.smithi176.stdout:4732 punch from 0x5b640ce to 0x5b6bdc2, (0x7cf4 bytes) 2024-09-14T09:07:10.424 INFO:teuthology.orchestra.run.smithi176.stdout:4733 read 0x635f3e thru 0x63da12 (0x7ad5 bytes) 2024-09-14T09:07:10.425 INFO:teuthology.orchestra.run.smithi176.stdout:4735 read 0x54ca5cc thru 0x54ca730 (0x165 bytes) 2024-09-14T09:07:10.425 INFO:teuthology.orchestra.run.smithi176.stdout:4736 read 0x3c0e1f9 thru 0x3c0e34e (0x156 bytes) 2024-09-14T09:07:10.425 INFO:teuthology.orchestra.run.smithi176.stdout:4737 writesame 0x9becae1 thru 0x9bf95a1 data_size 0x7(0xcac1 bytes) 2024-09-14T09:07:10.461 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:10.461 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:10.465 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:10.463+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:10.465 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:10.465 INFO:teuthology.orchestra.run.smithi176.stdout:4740 writesame 0x9988aba thru 0x9990be3 data_size 0x21(0x812a bytes) 2024-09-14T09:07:10.475 INFO:teuthology.orchestra.run.smithi176.stdout:4741 write 0xa6cf819 thru 0xa6d166c (0x1e54 bytes) 2024-09-14T09:07:10.478 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:07:10.482 INFO:teuthology.orchestra.run.smithi176.stdout:4742 read 0x66bd149 thru 0x66bfd0b (0x2bc3 bytes) 2024-09-14T09:07:10.483 INFO:teuthology.orchestra.run.smithi176.stdout:4744 write 0x33b920a thru 0x33c5482 (0xc279 bytes) 2024-09-14T09:07:10.492 INFO:teuthology.orchestra.run.smithi176.stdout:4745 read 0xa57d2b6 thru 0xa5891c8 (0xbf13 bytes) 2024-09-14T09:07:10.492 INFO:teuthology.orchestra.run.smithi176.stdout:4747 writesame 0x3dd3e18 thru 0x3ddf4b3 data_size 0x34(0xb69c bytes) 2024-09-14T09:07:10.508 INFO:teuthology.orchestra.run.smithi176.stdout:4748 trunc from 0xb456597 to 0x7c50617 2024-09-14T09:07:10.527 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:10.527 INFO:teuthology.orchestra.run.smithi176.stdout:4751 write 0x4890772 thru 0x4895e28 (0x56b7 bytes) 2024-09-14T09:07:10.534 INFO:teuthology.orchestra.run.smithi176.stdout:4752 writesame 0x602f147 thru 0x603bae9 data_size 0x1(0xc9a3 bytes) 2024-09-14T09:07:10.565 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:10.563+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:10.615 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:10.615 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:10.629 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:07:10.666 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:10.671+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:10.694 INFO:teuthology.orchestra.run.smithi176.stdout:4753 writesame 0x3f34bee thru 0x3f3515b data_size 0x5(0x56e bytes) 2024-09-14T09:07:10.700 INFO:teuthology.orchestra.run.smithi176.stdout:4754 read 0x2a5ad5 thru 0x2b038f (0xa8bb bytes) 2024-09-14T09:07:10.715 INFO:teuthology.orchestra.run.smithi176.stdout:4756 write 0xa904fb5 thru 0xa914bb8 (0xfc04 bytes) 2024-09-14T09:07:10.725 INFO:teuthology.orchestra.run.smithi176.stdout:4758 trunc from 0xa914bb9 to 0x90c7ec4 2024-09-14T09:07:10.741 INFO:teuthology.orchestra.run.smithi176.stdout:4759 read 0x86e6c3f thru 0x86e9716 (0x2ad8 bytes) 2024-09-14T09:07:10.741 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:10.741 INFO:teuthology.orchestra.run.smithi176.stdout:4764 read 0x2d7490a thru 0x2d7d114 (0x880b bytes) 2024-09-14T09:07:10.741 INFO:teuthology.orchestra.run.smithi176.stdout:4765 punch from 0x31265e to 0x31c3e0, (0x9d82 bytes) 2024-09-14T09:07:10.741 INFO:teuthology.orchestra.run.smithi176.stdout:4767 write 0x8d2ccf8 thru 0x8d31fe6 (0x52ef bytes) 2024-09-14T09:07:10.746 INFO:teuthology.orchestra.run.smithi176.stdout:4772 read 0x2b258c thru 0x2bda37 (0xb4ac bytes) 2024-09-14T09:07:10.768 INFO:teuthology.orchestra.run.smithi176.stdout:4773 writesame 0xd3079a1 thru 0xd31079d data_size 0x1(0x8dfd bytes) 2024-09-14T09:07:10.769 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:10.771+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:10.770 INFO:teuthology.orchestra.run.smithi176.stdout:4774 trunc from 0xd31079e to 0xe753ab3 2024-09-14T09:07:10.782 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:10.789 INFO:teuthology.orchestra.run.smithi176.stdout:4776 trunc from 0xe753ab3 to 0xb254028 2024-09-14T09:07:10.804 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:10.804 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:10.818 INFO:teuthology.orchestra.run.smithi176.stdout:4777 write 0xd7a5a74 thru 0xd7aa216 (0x47a3 bytes) 2024-09-14T09:07:10.822 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:07:10.827 INFO:teuthology.orchestra.run.smithi176.stdout:4779 punch from 0x9935841 to 0x9944b86, (0xf345 bytes) 2024-09-14T09:07:10.827 INFO:teuthology.orchestra.run.smithi176.stdout:4780 punch from 0xadbb09b to 0xadc9774, (0xe6d9 bytes) 2024-09-14T09:07:10.827 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:10.827 INFO:teuthology.orchestra.run.smithi176.stdout:4782 punch from 0x401ad8 to 0x401d65, (0x28d bytes) 2024-09-14T09:07:10.827 INFO:teuthology.orchestra.run.smithi176.stdout:4783 write 0xcbc8af4 thru 0xcbc91bc (0x6c9 bytes) 2024-09-14T09:07:10.831 INFO:teuthology.orchestra.run.smithi176.stdout:4784 writesame 0x1d218b thru 0x1d302e data_size 0x2(0xea4 bytes) 2024-09-14T09:07:10.844 INFO:teuthology.orchestra.run.smithi176.stdout:4785 clone 34 order 20 su 524288 sc 9 2024-09-14T09:07:10.870 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:10.871+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:10.945 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:07:10.957 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:10.957 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:10.970 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone33 from 0xd7aa217 (overlap 0x8c47ad) to 0x0 2024-09-14T09:07:10.970 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:10.971+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:10.974 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:07:11.071 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:11.075+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:11.117 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:11.118 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:11.136 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:11.136 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:11.144 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-14T09:07:11.154 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:11.154 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:11.155 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:11.155 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:11.157 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:11.158 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:11.163 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:11.170 INFO:teuthology.orchestra.run.smithi176.stdout:4787 trunc from 0xd7aa217 to 0x8d5d0b3 2024-09-14T09:07:11.173 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:07:11.173 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:11.175+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:11.175 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:11.176 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:11.201 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:07:11.201 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:07:11.207 INFO:teuthology.orchestra.run.smithi176.stdout:4788 punch from 0x437af82 to 0x4382118, (0x7196 bytes) 2024-09-14T09:07:11.207 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:11.223 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:11.223 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:11.227 INFO:teuthology.orchestra.run.smithi176.stdout:4790 write 0xe50cdce thru 0xe514b55 (0x7d88 bytes) 2024-09-14T09:07:11.232 INFO:teuthology.orchestra.run.smithi176.stdout:4791 read 0x97484c1 thru 0x974f60a (0x714a bytes) 2024-09-14T09:07:11.232 INFO:teuthology.orchestra.run.smithi176.stdout:4792 write 0x218d339 thru 0x218ddb0 (0xa78 bytes) 2024-09-14T09:07:11.238 INFO:teuthology.orchestra.run.smithi176.stdout:4793 read 0x40874a1 thru 0x408ccc3 (0x5823 bytes) 2024-09-14T09:07:11.246 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:11.246 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:11.255 INFO:teuthology.orchestra.run.smithi176.stdout:4794 trunc from 0xe514b56 to 0x2db49e3 2024-09-14T09:07:11.270 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:11.270 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:11.273 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:11.275+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:11.298 INFO:teuthology.orchestra.run.smithi176.stdout:4795 write 0x42bfb84 thru 0x42c406c (0x44e9 bytes) 2024-09-14T09:07:11.303 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:11.303 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:11.303 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:11.303 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:11.303 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:11.305 INFO:teuthology.orchestra.run.smithi176.stdout:4796 write 0x3dda65c thru 0x3de7949 (0xd2ee bytes) 2024-09-14T09:07:11.313 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:11.314 INFO:teuthology.orchestra.run.smithi176.stdout:4799 punch from 0xb0522f to 0xb106c9, (0xb49a bytes) 2024-09-14T09:07:11.314 INFO:teuthology.orchestra.run.smithi176.stdout:4800 punch from 0x30323ea to 0x303f792, (0xd3a8 bytes) 2024-09-14T09:07:11.322 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:07:11.335 INFO:teuthology.orchestra.run.smithi176.stdout:4801 write 0xa30aade thru 0xa30adf2 (0x315 bytes) 2024-09-14T09:07:11.342 INFO:teuthology.orchestra.run.smithi176.stdout:4802 read 0x942f2e6 thru 0x943e75a (0xf475 bytes) 2024-09-14T09:07:11.342 INFO:teuthology.orchestra.run.smithi176.stdout:4803 read 0x2890310 thru 0x28949b5 (0x46a6 bytes) 2024-09-14T09:07:11.342 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:11.360 INFO:teuthology.orchestra.run.smithi176.stdout:4806 write 0xe776193 thru 0xe781d99 (0xbc07 bytes) 2024-09-14T09:07:11.369 INFO:teuthology.orchestra.run.smithi176.stdout:4808 write 0x8c40cca thru 0x8c457ae (0x4ae5 bytes) 2024-09-14T09:07:11.374 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:11.375+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:11.377 INFO:teuthology.orchestra.run.smithi176.stdout:4810 read 0x5733444 thru 0x573f2c6 (0xbe83 bytes) 2024-09-14T09:07:11.377 INFO:teuthology.orchestra.run.smithi176.stdout:4811 punch from 0x274dc3c to 0x275c6cb, (0xea8f bytes) 2024-09-14T09:07:11.377 INFO:teuthology.orchestra.run.smithi176.stdout:4812 write 0xbc61a91 thru 0xbc70088 (0xe5f8 bytes) 2024-09-14T09:07:11.391 INFO:teuthology.orchestra.run.smithi176.stdout:4813 writesame 0xe4890ec thru 0xe497f34 data_size 0x1(0xee49 bytes) 2024-09-14T09:07:11.459 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:11.459 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:11.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73540 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15155200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:07:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.189306545000001, 2024-09-14T09:07:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.022930347, 2024-09-14T09:07:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73540 2024-09-14T09:07:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:07:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268664+0000", 2024-09-14T09:07:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268665+0000", 2024-09-14T09:07:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6599999999999996e-07 2024-09-14T09:07:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268668+0000", 2024-09-14T09:07:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8760000000000001e-06 2024-09-14T09:07:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268676+0000", 2024-09-14T09:07:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7879999999999995e-06 2024-09-14T09:07:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268683+0000", 2024-09-14T09:07:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3180000000000003e-06 2024-09-14T09:07:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282017+0000", 2024-09-14T09:07:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01333409 2024-09-14T09:07:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282020+0000", 2024-09-14T09:07:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1609999999999999e-06 2024-09-14T09:07:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291158+0000", 2024-09-14T09:07:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0091376430000001 2024-09-14T09:07:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291383+0000", 2024-09-14T09:07:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022491000000000001 2024-09-14T09:07:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291595+0000", 2024-09-14T09:07:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021199500000000001 2024-09-14T09:07:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73541 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15237120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:07:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.189279124999999, 2024-09-14T09:07:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0232607330000001, 2024-09-14T09:07:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73541 2024-09-14T09:07:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:07:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:07:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268692+0000", 2024-09-14T09:07:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5400000000000001e-07 2024-09-14T09:07:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268695+0000", 2024-09-14T09:07:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5799999999999999e-06 2024-09-14T09:07:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268703+0000", 2024-09-14T09:07:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5480000000000002e-06 2024-09-14T09:07:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268713+0000", 2024-09-14T09:07:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8640000000000008e-06 2024-09-14T09:07:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282032+0000", 2024-09-14T09:07:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013318831999999999 2024-09-14T09:07:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282035+0000", 2024-09-14T09:07:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4010000000000001e-06 2024-09-14T09:07:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291701+0000", 2024-09-14T09:07:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0096654199999999 2024-09-14T09:07:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291807+0000", 2024-09-14T09:07:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000105852 2024-09-14T09:07:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.291952+0000", 2024-09-14T09:07:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014568200000000001 2024-09-14T09:07:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73542 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15319040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.189209513999998, 2024-09-14T09:07:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0237091359999999, 2024-09-14T09:07:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73542 2024-09-14T09:07:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268762+0000", 2024-09-14T09:07:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268766+0000", 2024-09-14T09:07:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5880000000000002e-06 2024-09-14T09:07:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268776+0000", 2024-09-14T09:07:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0168e-05 2024-09-14T09:07:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268789+0000", 2024-09-14T09:07:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:07:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282053+0000", 2024-09-14T09:07:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013264496000000001 2024-09-14T09:07:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282057+0000", 2024-09-14T09:07:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6090000000000001e-06 2024-09-14T09:07:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292052+0000", 2024-09-14T09:07:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0099950070000001 2024-09-14T09:07:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292262+0000", 2024-09-14T09:07:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021009800000000001 2024-09-14T09:07:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292470+0000", 2024-09-14T09:07:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000208314 2024-09-14T09:07:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73543 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.189170724, 2024-09-14T09:07:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023985473, 2024-09-14T09:07:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73543 2024-09-14T09:07:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268801+0000", 2024-09-14T09:07:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268803+0000", 2024-09-14T09:07:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7439999999999999e-06 2024-09-14T09:07:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268812+0000", 2024-09-14T09:07:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7080000000000003e-06 2024-09-14T09:07:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268819+0000", 2024-09-14T09:07:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T09:07:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282068+0000", 2024-09-14T09:07:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013249540000000001 2024-09-14T09:07:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282072+0000", 2024-09-14T09:07:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0960000000000001e-06 2024-09-14T09:07:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292573+0000", 2024-09-14T09:07:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0105014560000001 2024-09-14T09:07:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292679+0000", 2024-09-14T09:07:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010545899999999999 2024-09-14T09:07:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292786+0000", 2024-09-14T09:07:11.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107052 2024-09-14T09:07:11.480 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.480 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.480 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.480 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73544 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.480 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:11.480 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.189143022000003, 2024-09-14T09:07:11.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0244830549999999, 2024-09-14T09:07:11.480 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.481 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.481 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.481 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.481 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.481 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.481 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73544 2024-09-14T09:07:11.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.481 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:11.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:11.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:11.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5599999999999995e-07 2024-09-14T09:07:11.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268831+0000", 2024-09-14T09:07:11.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.57e-06 2024-09-14T09:07:11.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268844+0000", 2024-09-14T09:07:11.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2811e-05 2024-09-14T09:07:11.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268851+0000", 2024-09-14T09:07:11.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9140000000000002e-06 2024-09-14T09:07:11.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282083+0000", 2024-09-14T09:07:11.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013232111 2024-09-14T09:07:11.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282086+0000", 2024-09-14T09:07:11.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2909999999999999e-06 2024-09-14T09:07:11.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292884+0000", 2024-09-14T09:07:11.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0107975069999999 2024-09-14T09:07:11.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293100+0000", 2024-09-14T09:07:11.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021639399999999999 2024-09-14T09:07:11.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293311+0000", 2024-09-14T09:07:11.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021090099999999999 2024-09-14T09:07:11.485 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.485 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.485 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.485 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73545 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15564800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.486 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:11.486 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.189091836999999, 2024-09-14T09:07:11.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0249746740000001, 2024-09-14T09:07:11.486 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.486 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.486 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.486 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.486 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.486 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.486 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73545 2024-09-14T09:07:11.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.486 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:11.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:11.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268880+0000", 2024-09-14T09:07:11.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5600000000000005e-07 2024-09-14T09:07:11.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268889+0000", 2024-09-14T09:07:11.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8559999999999996e-06 2024-09-14T09:07:11.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268913+0000", 2024-09-14T09:07:11.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3958e-05 2024-09-14T09:07:11.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268921+0000", 2024-09-14T09:07:11.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7479999999999999e-06 2024-09-14T09:07:11.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282098+0000", 2024-09-14T09:07:11.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013176297999999999 2024-09-14T09:07:11.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282101+0000", 2024-09-14T09:07:11.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2329999999999998e-06 2024-09-14T09:07:11.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293435+0000", 2024-09-14T09:07:11.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0113339459999999 2024-09-14T09:07:11.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293648+0000", 2024-09-14T09:07:11.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021307100000000001 2024-09-14T09:07:11.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293854+0000", 2024-09-14T09:07:11.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020580800000000001 2024-09-14T09:07:11.490 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.490 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.490 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.490 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73546 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15646720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.491 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:11.491 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.189037966000001, 2024-09-14T09:07:11.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0254846360000001, 2024-09-14T09:07:11.491 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.491 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.491 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.491 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.491 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.491 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.491 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73546 2024-09-14T09:07:11.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.491 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:11.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:11.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268934+0000", 2024-09-14T09:07:11.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7999999999999995e-07 2024-09-14T09:07:11.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268937+0000", 2024-09-14T09:07:11.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9840000000000001e-06 2024-09-14T09:07:11.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268947+0000", 2024-09-14T09:07:11.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0522e-05 2024-09-14T09:07:11.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268955+0000", 2024-09-14T09:07:11.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8059999999999995e-06 2024-09-14T09:07:11.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282112+0000", 2024-09-14T09:07:11.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013157189 2024-09-14T09:07:11.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282115+0000", 2024-09-14T09:07:11.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2949999999999998e-06 2024-09-14T09:07:11.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293968+0000", 2024-09-14T09:07:11.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0118530990000001 2024-09-14T09:07:11.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294197+0000", 2024-09-14T09:07:11.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228507 2024-09-14T09:07:11.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294417+0000", 2024-09-14T09:07:11.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000220554 2024-09-14T09:07:11.495 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.495 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.495 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.496 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73547 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.496 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:11.496 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.189005520000002, 2024-09-14T09:07:11.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.025970525, 2024-09-14T09:07:11.496 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.496 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.496 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.496 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.496 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.496 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.496 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73547 2024-09-14T09:07:11.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.496 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:11.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:11.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268966+0000", 2024-09-14T09:07:11.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3799999999999997e-07 2024-09-14T09:07:11.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268969+0000", 2024-09-14T09:07:11.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8739999999999999e-06 2024-09-14T09:07:11.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268979+0000", 2024-09-14T09:07:11.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9939999999999995e-06 2024-09-14T09:07:11.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268986+0000", 2024-09-14T09:07:11.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3459999999999996e-06 2024-09-14T09:07:11.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282127+0000", 2024-09-14T09:07:11.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01314039 2024-09-14T09:07:11.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282130+0000", 2024-09-14T09:07:11.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2219999999999998e-06 2024-09-14T09:07:11.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294540+0000", 2024-09-14T09:07:11.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124105880000001 2024-09-14T09:07:11.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294759+0000", 2024-09-14T09:07:11.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000218798 2024-09-14T09:07:11.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294936+0000", 2024-09-14T09:07:11.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000176675 2024-09-14T09:07:11.500 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.500 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.500 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.500 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73548 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15810560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.500 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:11.501 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.188974408, 2024-09-14T09:07:11.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0265045129999999, 2024-09-14T09:07:11.501 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.501 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.501 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.501 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.501 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.501 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.501 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73548 2024-09-14T09:07:11.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.501 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:11.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:11.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268997+0000", 2024-09-14T09:07:11.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7599999999999996e-07 2024-09-14T09:07:11.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269000+0000", 2024-09-14T09:07:11.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0580000000000002e-06 2024-09-14T09:07:11.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269010+0000", 2024-09-14T09:07:11.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124e-05 2024-09-14T09:07:11.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269018+0000", 2024-09-14T09:07:11.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5399999999999998e-06 2024-09-14T09:07:11.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282141+0000", 2024-09-14T09:07:11.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013123012 2024-09-14T09:07:11.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282144+0000", 2024-09-14T09:07:11.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.196e-06 2024-09-14T09:07:11.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295042+0000", 2024-09-14T09:07:11.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0128975689999999 2024-09-14T09:07:11.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295272+0000", 2024-09-14T09:07:11.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023048099999999999 2024-09-14T09:07:11.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295501+0000", 2024-09-14T09:07:11.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228657 2024-09-14T09:07:11.505 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.505 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.505 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.506 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73549 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15892480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.506 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:11.506 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.188928529000002, 2024-09-14T09:07:11.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0268354099999999, 2024-09-14T09:07:11.506 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.506 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.506 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.506 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.506 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.506 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.506 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73549 2024-09-14T09:07:11.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.506 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:11.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:11.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269043+0000", 2024-09-14T09:07:11.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6199999999999997e-07 2024-09-14T09:07:11.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269046+0000", 2024-09-14T09:07:11.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2480000000000001e-06 2024-09-14T09:07:11.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269057+0000", 2024-09-14T09:07:11.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0436e-05 2024-09-14T09:07:11.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269064+0000", 2024-09-14T09:07:11.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5179999999999998e-06 2024-09-14T09:07:11.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282155+0000", 2024-09-14T09:07:11.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013091190000000001 2024-09-14T09:07:11.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282159+0000", 2024-09-14T09:07:11.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.366e-06 2024-09-14T09:07:11.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295599+0000", 2024-09-14T09:07:11.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0134405559999999 2024-09-14T09:07:11.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295765+0000", 2024-09-14T09:07:11.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016575499999999999 2024-09-14T09:07:11.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295878+0000", 2024-09-14T09:07:11.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112679 2024-09-14T09:07:11.510 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.510 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.510 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.511 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73550 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15974400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.511 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:11.511 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.188895928999997, 2024-09-14T09:07:11.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027357657, 2024-09-14T09:07:11.511 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.511 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.511 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.511 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.511 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73550 2024-09-14T09:07:11.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.512 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:11.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:11.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269076+0000", 2024-09-14T09:07:11.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:07:11.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269079+0000", 2024-09-14T09:07:11.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:07:11.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269098+0000", 2024-09-14T09:07:11.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9698e-05 2024-09-14T09:07:11.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269106+0000", 2024-09-14T09:07:11.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6179999999999997e-06 2024-09-14T09:07:11.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282170+0000", 2024-09-14T09:07:11.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013063874 2024-09-14T09:07:11.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282173+0000", 2024-09-14T09:07:11.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1999999999999999e-06 2024-09-14T09:07:11.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295996+0000", 2024-09-14T09:07:11.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.013823151 2024-09-14T09:07:11.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296214+0000", 2024-09-14T09:07:11.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021817700000000001 2024-09-14T09:07:11.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296433+0000", 2024-09-14T09:07:11.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021829099999999999 2024-09-14T09:07:11.515 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.515 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.516 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.516 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73551 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.516 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:11.516 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.188854511999999, 2024-09-14T09:07:11.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027730276, 2024-09-14T09:07:11.516 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.516 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.516 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.516 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.516 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.516 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.517 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73551 2024-09-14T09:07:11.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.517 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:11.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:11.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269117+0000", 2024-09-14T09:07:11.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:07:11.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269120+0000", 2024-09-14T09:07:11.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.056e-06 2024-09-14T09:07:11.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269130+0000", 2024-09-14T09:07:11.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0084e-05 2024-09-14T09:07:11.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269137+0000", 2024-09-14T09:07:11.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1359999999999999e-06 2024-09-14T09:07:11.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282184+0000", 2024-09-14T09:07:11.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013046537 2024-09-14T09:07:11.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282188+0000", 2024-09-14T09:07:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3220000000000003e-06 2024-09-14T09:07:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296529+0000", 2024-09-14T09:07:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0143406109999999 2024-09-14T09:07:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296695+0000", 2024-09-14T09:07:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016604799999999999 2024-09-14T09:07:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296847+0000", 2024-09-14T09:07:11.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000151822 2024-09-14T09:07:11.521 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.521 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.521 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.521 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73552 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.521 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:11.521 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.188823931999998, 2024-09-14T09:07:11.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0281767500000001, 2024-09-14T09:07:11.521 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.521 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.521 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.521 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.522 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.522 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.522 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73552 2024-09-14T09:07:11.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.522 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:11.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:11.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:11.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:07:11.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269150+0000", 2024-09-14T09:07:11.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9119999999999998e-06 2024-09-14T09:07:11.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269160+0000", 2024-09-14T09:07:11.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6819999999999998e-06 2024-09-14T09:07:11.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269168+0000", 2024-09-14T09:07:11.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6079999999999998e-06 2024-09-14T09:07:11.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282205+0000", 2024-09-14T09:07:11.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013036982000000001 2024-09-14T09:07:11.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282208+0000", 2024-09-14T09:07:11.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4620000000000001e-06 2024-09-14T09:07:11.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296968+0000", 2024-09-14T09:07:11.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0147601209999999 2024-09-14T09:07:11.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297115+0000", 2024-09-14T09:07:11.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014673900000000001 2024-09-14T09:07:11.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297324+0000", 2024-09-14T09:07:11.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020866400000000001 2024-09-14T09:07:11.526 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.526 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.526 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.526 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73553 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.526 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:11.526 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.188793867999998, 2024-09-14T09:07:11.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0286556739999999, 2024-09-14T09:07:11.526 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.526 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.526 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.526 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.526 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.526 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.526 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73553 2024-09-14T09:07:11.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.527 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:11.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:11.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269178+0000", 2024-09-14T09:07:11.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:07:11.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269180+0000", 2024-09-14T09:07:11.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T09:07:11.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269189+0000", 2024-09-14T09:07:11.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2e-06 2024-09-14T09:07:11.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269197+0000", 2024-09-14T09:07:11.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2980000000000003e-06 2024-09-14T09:07:11.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282219+0000", 2024-09-14T09:07:11.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013022687 2024-09-14T09:07:11.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282223+0000", 2024-09-14T09:07:11.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.241e-06 2024-09-14T09:07:11.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297470+0000", 2024-09-14T09:07:11.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.015247778 2024-09-14T09:07:11.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297704+0000", 2024-09-14T09:07:11.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000233367 2024-09-14T09:07:11.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297833+0000", 2024-09-14T09:07:11.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012888900000000001 2024-09-14T09:07:11.531 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.531 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.531 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.531 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73554 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.531 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:11.531 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.188764550999998, 2024-09-14T09:07:11.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.029005113, 2024-09-14T09:07:11.531 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.531 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.531 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.532 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.532 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.532 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.532 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73554 2024-09-14T09:07:11.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.532 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:11.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:11.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269207+0000", 2024-09-14T09:07:11.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.68e-07 2024-09-14T09:07:11.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269210+0000", 2024-09-14T09:07:11.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8100000000000002e-06 2024-09-14T09:07:11.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269219+0000", 2024-09-14T09:07:11.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:07:11.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269227+0000", 2024-09-14T09:07:11.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3839999999999999e-06 2024-09-14T09:07:11.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282234+0000", 2024-09-14T09:07:11.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013007187 2024-09-14T09:07:11.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282237+0000", 2024-09-14T09:07:11.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1200000000000002e-06 2024-09-14T09:07:11.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297942+0000", 2024-09-14T09:07:11.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0157048040000001 2024-09-14T09:07:11.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298087+0000", 2024-09-14T09:07:11.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014499099999999999 2024-09-14T09:07:11.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298211+0000", 2024-09-14T09:07:11.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012461899999999999 2024-09-14T09:07:11.536 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.536 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.536 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.536 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73555 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.536 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:11.536 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.188734703000002, 2024-09-14T09:07:11.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.02941558, 2024-09-14T09:07:11.536 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.537 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.537 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.537 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.537 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.537 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.537 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73555 2024-09-14T09:07:11.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.537 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:11.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:11.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269237+0000", 2024-09-14T09:07:11.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3399999999999999e-07 2024-09-14T09:07:11.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269240+0000", 2024-09-14T09:07:11.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8480000000000001e-06 2024-09-14T09:07:11.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269249+0000", 2024-09-14T09:07:11.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1880000000000008e-06 2024-09-14T09:07:11.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269256+0000", 2024-09-14T09:07:11.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:07:11.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282248+0000", 2024-09-14T09:07:11.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012992327 2024-09-14T09:07:11.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282251+0000", 2024-09-14T09:07:11.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2160000000000002e-06 2024-09-14T09:07:11.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298299+0000", 2024-09-14T09:07:11.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0160479330000001 2024-09-14T09:07:11.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298446+0000", 2024-09-14T09:07:11.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000146525 2024-09-14T09:07:11.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298652+0000", 2024-09-14T09:07:11.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020584100000000001 2024-09-14T09:07:11.541 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.541 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.541 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.541 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73556 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.541 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:11.541 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.188704956999999, 2024-09-14T09:07:11.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0299854239999999, 2024-09-14T09:07:11.541 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.542 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.542 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.542 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.542 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.542 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.542 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73556 2024-09-14T09:07:11.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.542 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:11.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:11.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:11.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:11.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269270+0000", 2024-09-14T09:07:11.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0920000000000002e-06 2024-09-14T09:07:11.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269280+0000", 2024-09-14T09:07:11.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0013999999999999e-05 2024-09-14T09:07:11.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269288+0000", 2024-09-14T09:07:11.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:07:11.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282263+0000", 2024-09-14T09:07:11.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012974703000000001 2024-09-14T09:07:11.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282266+0000", 2024-09-14T09:07:11.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1520000000000001e-06 2024-09-14T09:07:11.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298729+0000", 2024-09-14T09:07:11.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.016463506 2024-09-14T09:07:11.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298884+0000", 2024-09-14T09:07:11.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015450800000000001 2024-09-14T09:07:11.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299251+0000", 2024-09-14T09:07:11.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036759899999999997 2024-09-14T09:07:11.546 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.546 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.546 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.546 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73557 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16547840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.546 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:11.546 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.188658306999997, 2024-09-14T09:07:11.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0307969480000001, 2024-09-14T09:07:11.547 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.547 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.547 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.547 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.547 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.547 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.547 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73557 2024-09-14T09:07:11.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.547 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:11.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:11.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269313+0000", 2024-09-14T09:07:11.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.92e-07 2024-09-14T09:07:11.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269336+0000", 2024-09-14T09:07:11.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3118000000000001e-05 2024-09-14T09:07:11.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269346+0000", 2024-09-14T09:07:11.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4069999999999999e-06 2024-09-14T09:07:11.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269352+0000", 2024-09-14T09:07:11.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5810000000000002e-06 2024-09-14T09:07:11.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282277+0000", 2024-09-14T09:07:11.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012924658 2024-09-14T09:07:11.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282280+0000", 2024-09-14T09:07:11.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2880000000000001e-06 2024-09-14T09:07:11.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299343+0000", 2024-09-14T09:07:11.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0170630190000001 2024-09-14T09:07:11.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299554+0000", 2024-09-14T09:07:11.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021085099999999999 2024-09-14T09:07:11.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300109+0000", 2024-09-14T09:07:11.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00055533399999999999 2024-09-14T09:07:11.551 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.551 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.551 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.551 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73558 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.551 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:11.551 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.188606684, 2024-09-14T09:07:11.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0314849340000001, 2024-09-14T09:07:11.551 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.551 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.552 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.552 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.552 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.552 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.552 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73558 2024-09-14T09:07:11.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.552 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:11.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:11.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269365+0000", 2024-09-14T09:07:11.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7400000000000003e-07 2024-09-14T09:07:11.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269367+0000", 2024-09-14T09:07:11.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4930000000000001e-06 2024-09-14T09:07:11.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269374+0000", 2024-09-14T09:07:11.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9519999999999997e-06 2024-09-14T09:07:11.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269379+0000", 2024-09-14T09:07:11.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9710000000000003e-06 2024-09-14T09:07:11.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282292+0000", 2024-09-14T09:07:11.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012912704000000001 2024-09-14T09:07:11.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282295+0000", 2024-09-14T09:07:11.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.179e-06 2024-09-14T09:07:11.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300195+0000", 2024-09-14T09:07:11.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.017899975 2024-09-14T09:07:11.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.555 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.555 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300345+0000", 2024-09-14T09:07:11.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014957499999999999 2024-09-14T09:07:11.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300849+0000", 2024-09-14T09:07:11.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050451100000000002 2024-09-14T09:07:11.556 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.556 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.556 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.556 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73559 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:11.556 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:11.556 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.188584274999997, 2024-09-14T09:07:11.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.031916343, 2024-09-14T09:07:11.557 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:11.557 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:11.557 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:11.557 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:11.557 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:11.557 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:11.557 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73559 2024-09-14T09:07:11.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.557 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:11.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:11.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:11.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:11.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:11.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:11.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:11.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:11.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3500000000000002e-07 2024-09-14T09:07:11.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:11.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269390+0000", 2024-09-14T09:07:11.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5890000000000001e-06 2024-09-14T09:07:11.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:11.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269396+0000", 2024-09-14T09:07:11.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3300000000000004e-06 2024-09-14T09:07:11.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:11.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269401+0000", 2024-09-14T09:07:11.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0340000000000004e-06 2024-09-14T09:07:11.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282311+0000", 2024-09-14T09:07:11.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012910204 2024-09-14T09:07:11.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:11.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282317+0000", 2024-09-14T09:07:11.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2639999999999997e-06 2024-09-14T09:07:11.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:11.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300933+0000", 2024-09-14T09:07:11.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0186160710000001 2024-09-14T09:07:11.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:11.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301158+0000", 2024-09-14T09:07:11.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225021 2024-09-14T09:07:11.561 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:11.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:11.561 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:11.561 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301303+0000", 2024-09-14T09:07:11.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014439499999999999 2024-09-14T09:07:11.561 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.561 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.561 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.561 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:11.561 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:11.561 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:11.563 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:07:11.563 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:11.563+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:11.593 INFO:teuthology.orchestra.run.smithi176.stdout:4815 writesame 0xed9f84c thru 0xeda446e data_size 0x49(0x4c23 bytes) 2024-09-14T09:07:11.593 INFO:teuthology.orchestra.run.smithi176.stdout:4816 write 0x6c5884c thru 0x6c5daf4 (0x52a9 bytes) 2024-09-14T09:07:11.602 INFO:teuthology.orchestra.run.smithi176.stdout:4817 write 0x422e802 thru 0x42349aa (0x61a9 bytes) 2024-09-14T09:07:11.611 INFO:teuthology.orchestra.run.smithi176.stdout:4818 write 0x4f9953d thru 0x4fa2132 (0x8bf6 bytes) 2024-09-14T09:07:11.621 INFO:teuthology.orchestra.run.smithi176.stdout:4819 punch from 0x60bb1f7 to 0x60bc096, (0xe9f bytes) 2024-09-14T09:07:11.621 INFO:teuthology.orchestra.run.smithi176.stdout:4820 read 0x584ee3b thru 0x585c55e (0xd724 bytes) 2024-09-14T09:07:11.621 INFO:teuthology.orchestra.run.smithi176.stdout:4821 write 0x19af9f8 thru 0x19bb743 (0xbd4c bytes) 2024-09-14T09:07:11.649 INFO:teuthology.orchestra.run.smithi176.stdout:4822 trunc from 0xeda446f to 0x6564280 2024-09-14T09:07:11.663 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:11.663+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:11.675 INFO:teuthology.orchestra.run.smithi176.stdout:4823 trunc from 0x6564280 to 0x4a597b3 2024-09-14T09:07:11.695 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:11.726 INFO:teuthology.orchestra.run.smithi176.stdout:4825 writesame 0xd7eacd1 thru 0xd7fabc3 data_size 0x1(0xfef3 bytes) 2024-09-14T09:07:11.726 INFO:teuthology.orchestra.run.smithi176.stdout:4828 write 0x49c7695 thru 0x49d22ad (0xac19 bytes) 2024-09-14T09:07:11.726 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:11.726 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:11.726 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:11.726 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:11.736 INFO:teuthology.orchestra.run.smithi176.stdout:4829 write 0x2aa7faf thru 0x2aaa1c4 (0x2216 bytes) 2024-09-14T09:07:11.743 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:07:11.744 INFO:teuthology.orchestra.run.smithi176.stdout:4831 write 0x19a0863 thru 0x19a5336 (0x4ad4 bytes) 2024-09-14T09:07:11.750 INFO:teuthology.orchestra.run.smithi176.stdout:4833 writesame 0x1f0c7d6 thru 0x1f19351 data_size 0x12(0xcb7c bytes) 2024-09-14T09:07:11.762 INFO:teuthology.orchestra.run.smithi176.stdout:4834 read 0x9ed861e thru 0x9edec14 (0x65f7 bytes) 2024-09-14T09:07:11.764 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:11.767+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:11.767 INFO:teuthology.orchestra.run.smithi176.stdout:4835 trunc from 0xd7fabc4 to 0xa8006a3 2024-09-14T09:07:11.778 INFO:teuthology.orchestra.run.smithi176.stdout:4836 write 0x80ef556 thru 0x80f24fe (0x2fa9 bytes) 2024-09-14T09:07:11.783 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:11.801 INFO:teuthology.orchestra.run.smithi176.stdout:4841 writesame 0xe3bc921 thru 0xe3c50d6 data_size 0x2(0x87b6 bytes) 2024-09-14T09:07:11.801 INFO:teuthology.orchestra.run.smithi176.stdout:4842 write 0xc44e114 thru 0xc4584da (0xa3c7 bytes) 2024-09-14T09:07:11.826 INFO:teuthology.orchestra.run.smithi176.stdout:4843 trunc from 0xe3c50d7 to 0x49636f3 2024-09-14T09:07:11.858 INFO:teuthology.orchestra.run.smithi176.stdout:4845 read 0x1853612 thru 0x1860601 (0xcff0 bytes) 2024-09-14T09:07:11.858 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:11.865 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:11.863+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:11.876 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:11.876 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:11.876 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:11.876 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:11.876 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:11.881 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T09:07:11.885 INFO:teuthology.orchestra.run.smithi176.stdout:4847 write 0xa4a40f4 thru 0xa4a9a06 (0x5913 bytes) 2024-09-14T09:07:11.892 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:07:11.893 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:11.893 INFO:teuthology.orchestra.run.smithi176.stdout:4849 writesame 0x848265d thru 0x84920c4 data_size 0xc(0xfa68 bytes) 2024-09-14T09:07:11.910 INFO:teuthology.orchestra.run.smithi176.stdout:4850 write 0x511d3db thru 0x512a1fb (0xce21 bytes) 2024-09-14T09:07:11.934 INFO:teuthology.orchestra.run.smithi176.stdout:4852 write 0xbe31ebd thru 0xbe38f5b (0x709f bytes) 2024-09-14T09:07:11.942 INFO:teuthology.orchestra.run.smithi176.stdout:4853 write 0x4eb2c9d thru 0x4ebd061 (0xa3c5 bytes) 2024-09-14T09:07:11.953 INFO:teuthology.orchestra.run.smithi176.stdout:4854 writesame 0x9578883 thru 0x9578b0a data_size 0x18(0x288 bytes) 2024-09-14T09:07:11.961 INFO:teuthology.orchestra.run.smithi176.stdout:4855 read 0x995e1f thru 0x99a0a5 (0x4287 bytes) 2024-09-14T09:07:11.961 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:11.961 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:11.961 INFO:teuthology.orchestra.run.smithi176.stdout:4858 punch from 0x8c1a9e7 to 0x8c1bc30, (0x1249 bytes) 2024-09-14T09:07:11.961 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:11.961 INFO:teuthology.orchestra.run.smithi176.stdout:4864 writesame 0x7711b38 thru 0x7720b82 data_size 0xf(0xf04b bytes) 2024-09-14T09:07:11.966 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:11.967+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:11.970 INFO:teuthology.orchestra.run.smithi176.stdout:4865 write 0x69f34c0 thru 0x69f74d4 (0x4015 bytes) 2024-09-14T09:07:11.977 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:11.977 INFO:teuthology.orchestra.run.smithi176.stdout:4868 writesame 0x8a044f7 thru 0x8a138ee data_size 0x94(0xf3f8 bytes) 2024-09-14T09:07:11.986 INFO:teuthology.orchestra.run.smithi176.stdout:4869 read 0x285400f thru 0x2860032 (0xc024 bytes) 2024-09-14T09:07:11.986 INFO:teuthology.orchestra.run.smithi176.stdout:4870 punch from 0x79e0a8c to 0x79ea23f, (0x97b3 bytes) 2024-09-14T09:07:11.986 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:11.986 INFO:teuthology.orchestra.run.smithi176.stdout:4872 read 0x3ecb769 thru 0x3ecd982 (0x221a bytes) 2024-09-14T09:07:11.986 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:11.986 INFO:teuthology.orchestra.run.smithi176.stdout:4874 writesame 0x12aa309 thru 0x12aafcf data_size 0x1(0xcc7 bytes) 2024-09-14T09:07:12.007 INFO:teuthology.orchestra.run.smithi176.stdout:4875 write 0xd0e7bf3 thru 0xd0efb11 (0x7f1f bytes) 2024-09-14T09:07:12.015 INFO:teuthology.orchestra.run.smithi176.stdout:4877 write 0x80bef7 thru 0x816d51 (0xae5b bytes) 2024-09-14T09:07:12.023 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:12.023 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:12.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:12.024 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:12.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.024 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18383 3.9 3:9252f7c9:::rbd_data.115e3abd3b2c.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:07:12.024 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:07:12.024 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.864971788999995, 2024-09-14T09:07:12.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056433830999999997, 2024-09-14T09:07:12.024 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.024 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.024 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:12.024 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.024 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.024 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.024 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18383 2024-09-14T09:07:12.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.025 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:07:12.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:07:12.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158423+0000", 2024-09-14T09:07:12.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T09:07:12.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158429+0000", 2024-09-14T09:07:12.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0179999999999998e-06 2024-09-14T09:07:12.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158440+0000", 2024-09-14T09:07:12.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0769e-05 2024-09-14T09:07:12.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158445+0000", 2024-09-14T09:07:12.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5840000000000002e-06 2024-09-14T09:07:12.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158495+0000", 2024-09-14T09:07:12.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9129999999999999e-05 2024-09-14T09:07:12.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158598+0000", 2024-09-14T09:07:12.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102938 2024-09-14T09:07:12.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:12.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.161048+0000", 2024-09-14T09:07:12.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002450336 2024-09-14T09:07:12.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:12.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214605+0000", 2024-09-14T09:07:12.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053557526000000001 2024-09-14T09:07:12.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:12.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214634+0000", 2024-09-14T09:07:12.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8483000000000002e-05 2024-09-14T09:07:12.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:12.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214731+0000", 2024-09-14T09:07:12.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7026000000000002e-05 2024-09-14T09:07:12.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214856+0000", 2024-09-14T09:07:12.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012474799999999999 2024-09-14T09:07:12.030 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.030 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.030 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.030 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18383 3.9 e38/34 ondisk, result = 0)", 2024-09-14T09:07:12.030 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:07:12.030 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.854655308000005, 2024-09-14T09:07:12.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045880575999999999, 2024-09-14T09:07:12.031 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.031 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.031 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.031 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:07:12.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:07:12.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168739+0000", 2024-09-14T09:07:12.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3900000000000004e-07 2024-09-14T09:07:12.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168744+0000", 2024-09-14T09:07:12.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5929999999999999e-06 2024-09-14T09:07:12.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168754+0000", 2024-09-14T09:07:12.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0686999999999999e-05 2024-09-14T09:07:12.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168761+0000", 2024-09-14T09:07:12.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6340000000000003e-06 2024-09-14T09:07:12.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214578+0000", 2024-09-14T09:07:12.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045817443999999999 2024-09-14T09:07:12.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214596+0000", 2024-09-14T09:07:12.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7159e-05 2024-09-14T09:07:12.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.034 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214619+0000", 2024-09-14T09:07:12.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.332e-05 2024-09-14T09:07:12.034 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.034 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.034 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.034 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19152 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14159058~8940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:07:12.034 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:07:12.034 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 87.700742852000005, 2024-09-14T09:07:12.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12891994500000001, 2024-09-14T09:07:12.035 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.035 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.035 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:12.035 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.035 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.035 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.035 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19152 2024-09-14T09:07:12.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.035 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:07:12.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:07:12.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322653+0000", 2024-09-14T09:07:12.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3750000000000001e-06 2024-09-14T09:07:12.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329647+0000", 2024-09-14T09:07:12.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069941960000000003 2024-09-14T09:07:12.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329673+0000", 2024-09-14T09:07:12.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5760000000000001e-05 2024-09-14T09:07:12.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329684+0000", 2024-09-14T09:07:12.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0474e-05 2024-09-14T09:07:12.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329776+0000", 2024-09-14T09:07:12.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2467999999999996e-05 2024-09-14T09:07:12.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329987+0000", 2024-09-14T09:07:12.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021133 2024-09-14T09:07:12.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:12.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.331303+0000", 2024-09-14T09:07:12.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001315629 2024-09-14T09:07:12.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:12.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.411090+0000", 2024-09-14T09:07:12.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079787477999999995 2024-09-14T09:07:12.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:12.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451463+0000", 2024-09-14T09:07:12.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040372312 2024-09-14T09:07:12.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:12.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451523+0000", 2024-09-14T09:07:12.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0420000000000001e-05 2024-09-14T09:07:12.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451571+0000", 2024-09-14T09:07:12.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7503000000000002e-05 2024-09-14T09:07:12.040 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.040 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.040 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.040 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20926 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:12.040 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:12.040 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 84.351410852000001, 2024-09-14T09:07:12.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018572207, 2024-09-14T09:07:12.041 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.041 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.041 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.041 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.041 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.041 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.041 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20926 2024-09-14T09:07:12.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.041 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.041 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.041 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:12.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:12.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:12.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:07:12.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671990+0000", 2024-09-14T09:07:12.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5769999999999999e-06 2024-09-14T09:07:12.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672001+0000", 2024-09-14T09:07:12.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1293e-05 2024-09-14T09:07:12.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672008+0000", 2024-09-14T09:07:12.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9099999999999999e-06 2024-09-14T09:07:12.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689147+0000", 2024-09-14T09:07:12.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017139060000000001 2024-09-14T09:07:12.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689324+0000", 2024-09-14T09:07:12.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017640200000000001 2024-09-14T09:07:12.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690555+0000", 2024-09-14T09:07:12.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001231212 2024-09-14T09:07:12.044 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.045 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.045 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.045 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20927 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2621440~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:12.045 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:12.045 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 84.351373886000005, 2024-09-14T09:07:12.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020002227000000001, 2024-09-14T09:07:12.045 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.045 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.045 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.045 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.045 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.045 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.046 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20927 2024-09-14T09:07:12.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.046 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:12.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:12.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:12.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9699999999999995e-07 2024-09-14T09:07:12.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672024+0000", 2024-09-14T09:07:12.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:07:12.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672036+0000", 2024-09-14T09:07:12.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.119e-05 2024-09-14T09:07:12.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672044+0000", 2024-09-14T09:07:12.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8370000000000003e-06 2024-09-14T09:07:12.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690602+0000", 2024-09-14T09:07:12.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018557648 2024-09-14T09:07:12.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690805+0000", 2024-09-14T09:07:12.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020282899999999999 2024-09-14T09:07:12.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692022+0000", 2024-09-14T09:07:12.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012170429999999999 2024-09-14T09:07:12.049 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.049 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.049 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.050 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20928 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 917504~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:12.050 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:07:12.050 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 84.351337745999999, 2024-09-14T09:07:12.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021623738, 2024-09-14T09:07:12.050 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.050 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.050 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.050 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.050 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.050 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.050 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20928 2024-09-14T09:07:12.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.051 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:07:12.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:07:12.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672057+0000", 2024-09-14T09:07:12.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:07:12.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672059+0000", 2024-09-14T09:07:12.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9270000000000001e-06 2024-09-14T09:07:12.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672070+0000", 2024-09-14T09:07:12.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0773e-05 2024-09-14T09:07:12.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672077+0000", 2024-09-14T09:07:12.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9600000000000003e-06 2024-09-14T09:07:12.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692154+0000", 2024-09-14T09:07:12.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020076482 2024-09-14T09:07:12.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692419+0000", 2024-09-14T09:07:12.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026517400000000002 2024-09-14T09:07:12.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693680+0000", 2024-09-14T09:07:12.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012607390000000001 2024-09-14T09:07:12.054 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.054 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.054 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.054 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20929 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2752512~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:12.054 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:07:12.054 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 84.351304704, 2024-09-14T09:07:12.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022898728, 2024-09-14T09:07:12.055 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.055 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.055 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.055 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.055 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.055 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.055 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20929 2024-09-14T09:07:12.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.055 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:07:12.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:07:12.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672090+0000", 2024-09-14T09:07:12.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:07:12.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672092+0000", 2024-09-14T09:07:12.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8940000000000002e-06 2024-09-14T09:07:12.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672103+0000", 2024-09-14T09:07:12.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0815999999999999e-05 2024-09-14T09:07:12.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672110+0000", 2024-09-14T09:07:12.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9469999999999997e-06 2024-09-14T09:07:12.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693737+0000", 2024-09-14T09:07:12.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021626467999999999 2024-09-14T09:07:12.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693962+0000", 2024-09-14T09:07:12.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225474 2024-09-14T09:07:12.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.694988+0000", 2024-09-14T09:07:12.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001025469 2024-09-14T09:07:12.058 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.058 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.058 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.059 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20930 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:12.059 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:07:12.059 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 84.351271609999998, 2024-09-14T09:07:12.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024303286, 2024-09-14T09:07:12.059 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.059 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.059 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.059 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20930 2024-09-14T09:07:12.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.059 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:07:12.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:07:12.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672123+0000", 2024-09-14T09:07:12.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:07:12.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672125+0000", 2024-09-14T09:07:12.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7300000000000001e-06 2024-09-14T09:07:12.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672134+0000", 2024-09-14T09:07:12.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7930000000000003e-06 2024-09-14T09:07:12.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672141+0000", 2024-09-14T09:07:12.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9029999999999998e-06 2024-09-14T09:07:12.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695087+0000", 2024-09-14T09:07:12.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022946013000000001 2024-09-14T09:07:12.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695350+0000", 2024-09-14T09:07:12.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026266000000000001 2024-09-14T09:07:12.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696425+0000", 2024-09-14T09:07:12.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001075527 2024-09-14T09:07:12.063 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.063 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.063 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.063 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20931 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:12.063 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:07:12.063 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 84.351238381000002, 2024-09-14T09:07:12.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025331519, 2024-09-14T09:07:12.063 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.063 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.063 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.063 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.064 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.064 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.064 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20931 2024-09-14T09:07:12.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.064 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:07:12.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:07:12.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672156+0000", 2024-09-14T09:07:12.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4300000000000002e-07 2024-09-14T09:07:12.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672159+0000", 2024-09-14T09:07:12.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T09:07:12.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672170+0000", 2024-09-14T09:07:12.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1192999999999999e-05 2024-09-14T09:07:12.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672177+0000", 2024-09-14T09:07:12.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9059999999999996e-06 2024-09-14T09:07:12.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696532+0000", 2024-09-14T09:07:12.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024355291000000001 2024-09-14T09:07:12.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696755+0000", 2024-09-14T09:07:12.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022297699999999999 2024-09-14T09:07:12.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697487+0000", 2024-09-14T09:07:12.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00073171499999999997 2024-09-14T09:07:12.067 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.067 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.067 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.067 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20932 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1179648~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:12.067 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:12.068 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 84.351204937999995, 2024-09-14T09:07:12.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026104083, 2024-09-14T09:07:12.068 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.068 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.068 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.068 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.068 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.068 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.068 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20932 2024-09-14T09:07:12.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.068 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:12.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:12.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:12.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0699999999999996e-07 2024-09-14T09:07:12.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672192+0000", 2024-09-14T09:07:12.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7700000000000002e-06 2024-09-14T09:07:12.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672203+0000", 2024-09-14T09:07:12.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0956e-05 2024-09-14T09:07:12.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672211+0000", 2024-09-14T09:07:12.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3769999999999996e-06 2024-09-14T09:07:12.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697530+0000", 2024-09-14T09:07:12.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025318411999999998 2024-09-14T09:07:12.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697640+0000", 2024-09-14T09:07:12.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011045299999999999 2024-09-14T09:07:12.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698293+0000", 2024-09-14T09:07:12.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065240800000000005 2024-09-14T09:07:12.071 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.072 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.072 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.072 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20933 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3014656~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:12.072 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:07:12.072 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 84.351170734999997, 2024-09-14T09:07:12.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026558875999999999, 2024-09-14T09:07:12.072 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.072 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.072 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.072 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.072 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.072 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.073 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20933 2024-09-14T09:07:12.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.073 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:07:12.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:07:12.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672224+0000", 2024-09-14T09:07:12.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2399999999999997e-07 2024-09-14T09:07:12.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672234+0000", 2024-09-14T09:07:12.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0793e-05 2024-09-14T09:07:12.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672246+0000", 2024-09-14T09:07:12.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659e-05 2024-09-14T09:07:12.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672255+0000", 2024-09-14T09:07:12.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5730000000000008e-06 2024-09-14T09:07:12.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698326+0000", 2024-09-14T09:07:12.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026071414000000001 2024-09-14T09:07:12.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698374+0000", 2024-09-14T09:07:12.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8032999999999999e-05 2024-09-14T09:07:12.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698782+0000", 2024-09-14T09:07:12.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040768 2024-09-14T09:07:12.076 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.076 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.076 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.077 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20934 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1310720~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:12.077 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:07:12.077 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 84.351128858999999, 2024-09-14T09:07:12.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027292067999999999, 2024-09-14T09:07:12.077 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.077 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.077 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.077 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.077 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.077 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.077 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20934 2024-09-14T09:07:12.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.078 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:07:12.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:07:12.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672266+0000", 2024-09-14T09:07:12.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6299999999999993e-07 2024-09-14T09:07:12.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672269+0000", 2024-09-14T09:07:12.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8569999999999999e-06 2024-09-14T09:07:12.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672280+0000", 2024-09-14T09:07:12.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1236999999999999e-05 2024-09-14T09:07:12.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672287+0000", 2024-09-14T09:07:12.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8260000000000004e-06 2024-09-14T09:07:12.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698820+0000", 2024-09-14T09:07:12.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026533055999999999 2024-09-14T09:07:12.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698933+0000", 2024-09-14T09:07:12.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011291 2024-09-14T09:07:12.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699557+0000", 2024-09-14T09:07:12.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062421899999999997 2024-09-14T09:07:12.081 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.081 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.081 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.081 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20935 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:12.081 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:07:12.081 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 84.351095099000005, 2024-09-14T09:07:12.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028108610999999999, 2024-09-14T09:07:12.082 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.082 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.082 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.082 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.082 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.082 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.082 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20935 2024-09-14T09:07:12.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.082 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:07:12.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:07:12.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672299+0000", 2024-09-14T09:07:12.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T09:07:12.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672302+0000", 2024-09-14T09:07:12.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7640000000000001e-06 2024-09-14T09:07:12.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672313+0000", 2024-09-14T09:07:12.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1080000000000001e-05 2024-09-14T09:07:12.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672321+0000", 2024-09-14T09:07:12.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6490000000000008e-06 2024-09-14T09:07:12.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699587+0000", 2024-09-14T09:07:12.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027265884000000001 2024-09-14T09:07:12.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699631+0000", 2024-09-14T09:07:12.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4947999999999997e-05 2024-09-14T09:07:12.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700407+0000", 2024-09-14T09:07:12.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00077559299999999999 2024-09-14T09:07:12.085 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.085 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.086 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.086 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20936 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1441792~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:12.086 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:12.086 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 84.351060923000006, 2024-09-14T09:07:12.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028784750000000001, 2024-09-14T09:07:12.086 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.086 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.086 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.086 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.086 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.086 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.086 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20936 2024-09-14T09:07:12.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.087 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:12.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:12.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:12.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4700000000000001e-07 2024-09-14T09:07:12.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672336+0000", 2024-09-14T09:07:12.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6730000000000001e-06 2024-09-14T09:07:12.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672347+0000", 2024-09-14T09:07:12.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0710000000000001e-05 2024-09-14T09:07:12.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.089 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672354+0000", 2024-09-14T09:07:12.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4630000000000004e-06 2024-09-14T09:07:12.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.089 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.089 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700442+0000", 2024-09-14T09:07:12.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028087951 2024-09-14T09:07:12.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.089 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.089 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700557+0000", 2024-09-14T09:07:12.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011461800000000001 2024-09-14T09:07:12.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.090 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701117+0000", 2024-09-14T09:07:12.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056058799999999999 2024-09-14T09:07:12.090 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.090 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.090 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.090 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20937 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:12.090 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:12.090 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 84.351027797, 2024-09-14T09:07:12.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029493028000000001, 2024-09-14T09:07:12.090 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.090 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.091 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.091 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.091 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.091 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.091 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20937 2024-09-14T09:07:12.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.091 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:12.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:12.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:12.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T09:07:12.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672384+0000", 2024-09-14T09:07:12.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7127000000000001e-05 2024-09-14T09:07:12.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672395+0000", 2024-09-14T09:07:12.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759999999999999e-05 2024-09-14T09:07:12.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672402+0000", 2024-09-14T09:07:12.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0960000000000003e-06 2024-09-14T09:07:12.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701147+0000", 2024-09-14T09:07:12.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028744241 2024-09-14T09:07:12.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.094 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.094 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701246+0000", 2024-09-14T09:07:12.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9031999999999996e-05 2024-09-14T09:07:12.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.094 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.094 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701859+0000", 2024-09-14T09:07:12.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061310499999999997 2024-09-14T09:07:12.094 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.094 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.094 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.095 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20938 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:12.095 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:07:12.095 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 84.350980637000006, 2024-09-14T09:07:12.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030109219999999999, 2024-09-14T09:07:12.095 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.095 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.095 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.095 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.095 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.095 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.095 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20938 2024-09-14T09:07:12.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.095 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:07:12.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:07:12.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672414+0000", 2024-09-14T09:07:12.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-07 2024-09-14T09:07:12.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672417+0000", 2024-09-14T09:07:12.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:07:12.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672428+0000", 2024-09-14T09:07:12.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0873000000000001e-05 2024-09-14T09:07:12.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672435+0000", 2024-09-14T09:07:12.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:07:12.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701884+0000", 2024-09-14T09:07:12.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029449811999999999 2024-09-14T09:07:12.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701934+0000", 2024-09-14T09:07:12.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9703000000000002e-05 2024-09-14T09:07:12.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702522+0000", 2024-09-14T09:07:12.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00058811200000000001 2024-09-14T09:07:12.099 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.099 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.099 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.099 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20939 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:12.099 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:07:12.099 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 84.350948504000002, 2024-09-14T09:07:12.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030779259, 2024-09-14T09:07:12.099 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.099 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.099 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.100 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.100 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.100 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20939 2024-09-14T09:07:12.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.100 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:07:12.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:07:12.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672446+0000", 2024-09-14T09:07:12.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:07:12.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672449+0000", 2024-09-14T09:07:12.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7930000000000002e-06 2024-09-14T09:07:12.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672461+0000", 2024-09-14T09:07:12.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T09:07:12.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672471+0000", 2024-09-14T09:07:12.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0807e-05 2024-09-14T09:07:12.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702551+0000", 2024-09-14T09:07:12.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030079228 2024-09-14T09:07:12.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702664+0000", 2024-09-14T09:07:12.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011351800000000001 2024-09-14T09:07:12.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.703224+0000", 2024-09-14T09:07:12.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056026999999999997 2024-09-14T09:07:12.103 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.103 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.103 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.103 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:74298 3.9 3:95c8282a:::rbd_data.115ea8d131ff.0000000000000041:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:07:12.104 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:07:12.104 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.225638492000002, 2024-09-14T09:07:12.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042668064999999998, 2024-09-14T09:07:12.104 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.104 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.104 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:12.104 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.104 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.104 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.104 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74298 2024-09-14T09:07:12.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.104 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:07:12.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:07:12.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797758+0000", 2024-09-14T09:07:12.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9799999999999998e-06 2024-09-14T09:07:12.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797770+0000", 2024-09-14T09:07:12.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1667e-05 2024-09-14T09:07:12.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797797+0000", 2024-09-14T09:07:12.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7212999999999998e-05 2024-09-14T09:07:12.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797811+0000", 2024-09-14T09:07:12.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4256e-05 2024-09-14T09:07:12.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797882+0000", 2024-09-14T09:07:12.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0622000000000002e-05 2024-09-14T09:07:12.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797943+0000", 2024-09-14T09:07:12.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0893000000000001e-05 2024-09-14T09:07:12.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:12.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.799676+0000", 2024-09-14T09:07:12.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0017331619999999999 2024-09-14T09:07:12.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:12.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840109+0000", 2024-09-14T09:07:12.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040432994 2024-09-14T09:07:12.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:12.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840204+0000", 2024-09-14T09:07:12.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5247000000000005e-05 2024-09-14T09:07:12.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:12.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840346+0000", 2024-09-14T09:07:12.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014156499999999999 2024-09-14T09:07:12.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840423+0000", 2024-09-14T09:07:12.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7466000000000007e-05 2024-09-14T09:07:12.109 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.109 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.109 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.109 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:74298 3.9 e76/34 ondisk, result = 0)", 2024-09-14T09:07:12.109 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:07:12.109 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.219243849000001, 2024-09-14T09:07:12.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036289318000000001, 2024-09-14T09:07:12.109 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.110 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.110 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.110 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:07:12.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:07:12.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804151+0000", 2024-09-14T09:07:12.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6700000000000001e-06 2024-09-14T09:07:12.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804163+0000", 2024-09-14T09:07:12.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.148e-05 2024-09-14T09:07:12.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804197+0000", 2024-09-14T09:07:12.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3909999999999999e-05 2024-09-14T09:07:12.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804217+0000", 2024-09-14T09:07:12.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0287000000000001e-05 2024-09-14T09:07:12.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840176+0000", 2024-09-14T09:07:12.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035958641 2024-09-14T09:07:12.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840193+0000", 2024-09-14T09:07:12.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6986e-05 2024-09-14T09:07:12.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840439+0000", 2024-09-14T09:07:12.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024634400000000001 2024-09-14T09:07:12.113 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.113 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.113 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.113 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:93074 3.9 e94/84 3:925bb5bd:::rbd_data.115e7770ddc5.0000000000000001:head v 94'255, mlcod=94'252)", 2024-09-14T09:07:12.113 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:57.863727+0000", 2024-09-14T09:07:12.113 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.159666589, 2024-09-14T09:07:12.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019329799000000002, 2024-09-14T09:07:12.113 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.113 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.114 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:12.114 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.863727+0000", 2024-09-14T09:07:12.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.863727+0000", 2024-09-14T09:07:12.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.863729+0000", 2024-09-14T09:07:12.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3980000000000002e-06 2024-09-14T09:07:12.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866073+0000", 2024-09-14T09:07:12.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0023435109999999999 2024-09-14T09:07:12.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866094+0000", 2024-09-14T09:07:12.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1444e-05 2024-09-14T09:07:12.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866105+0000", 2024-09-14T09:07:12.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0949e-05 2024-09-14T09:07:12.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866216+0000", 2024-09-14T09:07:12.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110961 2024-09-14T09:07:12.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866258+0000", 2024-09-14T09:07:12.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1903000000000002e-05 2024-09-14T09:07:12.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:12.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.882981+0000", 2024-09-14T09:07:12.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016723198000000002 2024-09-14T09:07:12.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.883057+0000", 2024-09-14T09:07:12.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5434999999999999e-05 2024-09-14T09:07:12.117 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.117 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.117 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.117 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.117 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.118 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:12.118 INFO:teuthology.orchestra.run.smithi176.stdout:4878 write 0x90cd5ba thru 0x90ce9af (0x13f6 bytes) 2024-09-14T09:07:12.119 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:07:12.120 INFO:teuthology.orchestra.run.smithi176.stdout:4880 trunc from 0xd0efb12 to 0x66d7824 2024-09-14T09:07:12.120 INFO:teuthology.orchestra.run.smithi176.stdout:4881 write 0x4e45864 thru 0x4e52ff3 (0xd790 bytes) 2024-09-14T09:07:12.120 INFO:teuthology.orchestra.run.smithi176.stdout:4882 read 0x3cfce2d thru 0x3cfeff2 (0x21c6 bytes) 2024-09-14T09:07:12.120 INFO:teuthology.orchestra.run.smithi176.stdout:4883 writesame 0x26beebd thru 0x26ccbad data_size 0x1(0xdcf1 bytes) 2024-09-14T09:07:12.121 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:12.123+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:12.220 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:12.219+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:12.254 INFO:teuthology.orchestra.run.smithi176.stdout:4884 writesame 0x9d2bbf4 thru 0x9d2dc24 data_size 0x29(0x2031 bytes) 2024-09-14T09:07:12.266 INFO:teuthology.orchestra.run.smithi176.stdout:4885 write 0xa7053a4 thru 0xa7078dc (0x2539 bytes) 2024-09-14T09:07:12.272 INFO:teuthology.orchestra.run.smithi176.stdout:4888 punch from 0x5578539 to 0x5583967, (0xb42e bytes) 2024-09-14T09:07:12.273 INFO:teuthology.orchestra.run.smithi176.stdout:4889 punch from 0x43720df to 0x4379027, (0x6f48 bytes) 2024-09-14T09:07:12.276 INFO:teuthology.orchestra.run.smithi176.stdout:4890 trunc from 0xa7078dd to 0x8573135 2024-09-14T09:07:12.297 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:12.300 INFO:teuthology.orchestra.run.smithi176.stdout:4894 trunc from 0x8573135 to 0x67d95cd 2024-09-14T09:07:12.301 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:12.301 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:12.301 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:12.301 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:12.315 INFO:teuthology.orchestra.run.smithi176.stdout:4897 punch from 0x1ad14f6 to 0x1ad9162, (0x7c6c bytes) 2024-09-14T09:07:12.315 INFO:teuthology.orchestra.run.smithi176.stdout:4899 punch from 0x28fcd4e to 0x29071a3, (0xa455 bytes) 2024-09-14T09:07:12.315 INFO:teuthology.orchestra.run.smithi176.stdout:4900 read 0x5131a59 thru 0x514065e (0xec06 bytes) 2024-09-14T09:07:12.316 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:07:12.321 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:12.323+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:12.339 INFO:teuthology.orchestra.run.smithi176.stdout:4901 write 0xd7f79eb thru 0xd7fa9ec (0x3002 bytes) 2024-09-14T09:07:12.354 INFO:teuthology.orchestra.run.smithi176.stdout:4902 trunc from 0xd7fa9ed to 0x7d7731b 2024-09-14T09:07:12.396 INFO:teuthology.orchestra.run.smithi176.stdout:4903 write 0xe729282 thru 0xe737970 (0xe6ef bytes) 2024-09-14T09:07:12.404 INFO:teuthology.orchestra.run.smithi176.stdout:4904 writesame 0x6240b6c thru 0x6244a27 data_size 0x4(0x3ebc bytes) 2024-09-14T09:07:12.422 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:12.419+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:12.424 INFO:teuthology.orchestra.run.smithi176.stdout:4905 write 0x6c825ee thru 0x6c8ea0f (0xc422 bytes) 2024-09-14T09:07:12.459 INFO:teuthology.orchestra.run.smithi176.stdout:4906 trunc from 0xe737971 to 0x106fc4b 2024-09-14T09:07:12.463 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:12.463 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:12.463 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:12.463 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:12.463 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:12.481 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:07:12.526 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:12.527+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:12.546 INFO:teuthology.orchestra.run.smithi176.stdout:4908 trunc from 0x106fc4b to 0xc5ad1f8 2024-09-14T09:07:12.558 INFO:teuthology.orchestra.run.smithi176.stdout:4909 write 0x9d0b7bd thru 0x9d130e4 (0x7928 bytes) 2024-09-14T09:07:12.568 INFO:teuthology.orchestra.run.smithi176.stdout:4910 write 0x67d3ca9 thru 0x67dee37 (0xb18f bytes) 2024-09-14T09:07:12.593 INFO:teuthology.orchestra.run.smithi176.stdout:4911 writesame 0xe7e5ac3 thru 0xe7e9c1c data_size 0xe(0x415a bytes) 2024-09-14T09:07:12.593 INFO:teuthology.orchestra.run.smithi176.stdout:4912 punch from 0x498fa9d to 0x499e5a6, (0xeb09 bytes) 2024-09-14T09:07:12.593 INFO:teuthology.orchestra.run.smithi176.stdout:4913 punch from 0x4e54247 to 0x4e5eea7, (0xac60 bytes) 2024-09-14T09:07:12.593 INFO:teuthology.orchestra.run.smithi176.stdout:4914 punch from 0x2bb3813 to 0x2bbc74e, (0x8f3b bytes) 2024-09-14T09:07:12.593 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:12.593 INFO:teuthology.orchestra.run.smithi176.stdout:4916 read 0x1d7acc3 thru 0x1d7c6db (0x1a19 bytes) 2024-09-14T09:07:12.593 INFO:teuthology.orchestra.run.smithi176.stdout:4917 writesame 0x489d7fa thru 0x48a465a data_size 0x1(0x6e61 bytes) 2024-09-14T09:07:12.624 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:12.627+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:12.659 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:12.659 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:12.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:12.659 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:12.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.659 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:12.659 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:12.659 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 99.573763855999999, 2024-09-14T09:07:12.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:07:12.659 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.659 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.659 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.660 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.660 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.660 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.660 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:07:12.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.660 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:12.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:12.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:07:12.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:07:12.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:07:12.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:07:12.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:07:12.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:07:12.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:07:12.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:07:12.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:07:12.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:07:12.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:12.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:07:12.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:07:12.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:07:12.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:07:12.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:07:12.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:07:12.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:07:12.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:07:12.664 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.664 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.664 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.664 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:12.664 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:12.664 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 99.573709198000003, 2024-09-14T09:07:12.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:07:12.664 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.665 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.665 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.665 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.665 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.665 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.665 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:07:12.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.665 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:12.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:12.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:12.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:07:12.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:07:12.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:07:12.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:07:12.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:07:12.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:07:12.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:07:12.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:07:12.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:07:12.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:12.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:07:12.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:07:12.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:07:12.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:07:12.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:07:12.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:07:12.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:07:12.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:07:12.669 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.669 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.669 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.669 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:12.669 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:12.669 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 99.573603824000003, 2024-09-14T09:07:12.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:07:12.670 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.670 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.670 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.670 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.670 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.670 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.670 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:07:12.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.670 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:12.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:12.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:07:12.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:07:12.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:07:12.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:07:12.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:07:12.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:07:12.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:07:12.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:07:12.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:07:12.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:07:12.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:12.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:07:12.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:07:12.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:07:12.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:07:12.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:07:12.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:07:12.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:07:12.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:07:12.674 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.674 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.674 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.674 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:12.675 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:12.675 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 99.57324706, 2024-09-14T09:07:12.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:07:12.675 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.675 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.675 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.675 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.675 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.675 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.676 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:07:12.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.676 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:12.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:12.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:07:12.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:07:12.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:07:12.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:07:12.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:07:12.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:07:12.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:07:12.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:07:12.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:07:12.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:07:12.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:12.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:07:12.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:07:12.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:07:12.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:07:12.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:07:12.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:07:12.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:07:12.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:07:12.679 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.680 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.680 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.680 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:12.680 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:12.680 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 99.573162322000002, 2024-09-14T09:07:12.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:07:12.680 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.680 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.680 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.680 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.680 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.680 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.681 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:07:12.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.681 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:12.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:12.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:07:12.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:07:12.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:07:12.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:07:12.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:07:12.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:07:12.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:07:12.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:07:12.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:07:12.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:07:12.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:12.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:07:12.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:07:12.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:07:12.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:07:12.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:07:12.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:07:12.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:07:12.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:07:12.685 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.685 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.685 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.685 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:12.685 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:12.685 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 99.573074636000001, 2024-09-14T09:07:12.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:07:12.685 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.686 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.686 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.686 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.686 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.686 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.686 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:07:12.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.686 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:12.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:12.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:07:12.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:07:12.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:07:12.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:07:12.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:07:12.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:07:12.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:07:12.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:07:12.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:07:12.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:07:12.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:12.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:07:12.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:07:12.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:07:12.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:07:12.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:07:12.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:07:12.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:07:12.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:07:12.690 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.690 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.690 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.690 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38002 3.1e 3:7df715aa:::rbd_data.115eb07a8abd.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:07:12.691 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:07:12.691 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.267643418999995, 2024-09-14T09:07:12.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.065148304000000004, 2024-09-14T09:07:12.691 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.691 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.691 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:12.691 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.691 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.691 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.691 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38002 2024-09-14T09:07:12.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.691 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:07:12.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389751+0000", 2024-09-14T09:07:12.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389752+0000", 2024-09-14T09:07:12.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1829999999999999e-06 2024-09-14T09:07:12.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389758+0000", 2024-09-14T09:07:12.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0919999999999997e-06 2024-09-14T09:07:12.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389769+0000", 2024-09-14T09:07:12.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759e-05 2024-09-14T09:07:12.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389775+0000", 2024-09-14T09:07:12.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8819999999999998e-06 2024-09-14T09:07:12.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389824+0000", 2024-09-14T09:07:12.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8573999999999999e-05 2024-09-14T09:07:12.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.389876+0000", 2024-09-14T09:07:12.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2643999999999999e-05 2024-09-14T09:07:12.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 6", 2024-09-14T09:07:12.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.392640+0000", 2024-09-14T09:07:12.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0027636409999999998 2024-09-14T09:07:12.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:12.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454672+0000", 2024-09-14T09:07:12.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.062031703000000001 2024-09-14T09:07:12.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:12.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454765+0000", 2024-09-14T09:07:12.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2862000000000006e-05 2024-09-14T09:07:12.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:12.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454841+0000", 2024-09-14T09:07:12.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5857000000000002e-05 2024-09-14T09:07:12.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454900+0000", 2024-09-14T09:07:12.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9107000000000002e-05 2024-09-14T09:07:12.696 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.696 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.696 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.696 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:38002 3.1e e48/23 ondisk, result = 0)", 2024-09-14T09:07:12.696 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:07:12.696 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.257326829999997, 2024-09-14T09:07:12.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054841741999999999, 2024-09-14T09:07:12.696 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.696 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.697 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.697 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:07:12.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400068+0000", 2024-09-14T09:07:12.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400069+0000", 2024-09-14T09:07:12.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.339e-06 2024-09-14T09:07:12.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400076+0000", 2024-09-14T09:07:12.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7800000000000003e-06 2024-09-14T09:07:12.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400093+0000", 2024-09-14T09:07:12.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6915999999999999e-05 2024-09-14T09:07:12.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.400106+0000", 2024-09-14T09:07:12.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2785e-05 2024-09-14T09:07:12.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454713+0000", 2024-09-14T09:07:12.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.054607041000000002 2024-09-14T09:07:12.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454724+0000", 2024-09-14T09:07:12.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1202e-05 2024-09-14T09:07:12.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.454910+0000", 2024-09-14T09:07:12.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018567900000000001 2024-09-14T09:07:12.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.700 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.700 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:38100 3.b 3:d0f67c24:::rbd_data.115eb07a8abd.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-14T09:07:12.700 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:07:12.700 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.011473170000002, 2024-09-14T09:07:12.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053701964999999997, 2024-09-14T09:07:12.700 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.700 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.701 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:12.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.701 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 38100 2024-09-14T09:07:12.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.701 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:07:12.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645922+0000", 2024-09-14T09:07:12.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645923+0000", 2024-09-14T09:07:12.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:07:12.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645928+0000", 2024-09-14T09:07:12.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0699999999999997e-06 2024-09-14T09:07:12.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645942+0000", 2024-09-14T09:07:12.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4069e-05 2024-09-14T09:07:12.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.645951+0000", 2024-09-14T09:07:12.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3230000000000004e-06 2024-09-14T09:07:12.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646008+0000", 2024-09-14T09:07:12.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6583999999999999e-05 2024-09-14T09:07:12.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.646089+0000", 2024-09-14T09:07:12.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1557999999999994e-05 2024-09-14T09:07:12.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:07:12.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.650355+0000", 2024-09-14T09:07:12.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004265525 2024-09-14T09:07:12.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:12.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699439+0000", 2024-09-14T09:07:12.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.049083904999999997 2024-09-14T09:07:12.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:12.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699467+0000", 2024-09-14T09:07:12.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8198999999999999e-05 2024-09-14T09:07:12.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:12.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699552+0000", 2024-09-14T09:07:12.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4815999999999996e-05 2024-09-14T09:07:12.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:57.699624+0000", 2024-09-14T09:07:12.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1725999999999997e-05 2024-09-14T09:07:12.705 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.705 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.706 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.706 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:07:12.706 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:12.706 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.496782648999996, 2024-09-14T09:07:12.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:07:12.706 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.706 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.706 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:12.706 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.706 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.706 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.706 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:07:12.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.707 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:12.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:12.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:07:12.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:07:12.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:07:12.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:07:12.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:07:12.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:07:12.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:07:12.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:07:12.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:07:12.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:07:12.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:07:12.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:07:12.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:12.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:07:12.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:07:12.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:12.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:07:12.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:07:12.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:12.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:07:12.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:07:12.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:12.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:07:12.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:07:12.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:07:12.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:07:12.711 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.711 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.711 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.711 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:39401 3.b e56/53)", 2024-09-14T09:07:12.712 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:07:12.712 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.470886582999995, 2024-09-14T09:07:12.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10443659600000001, 2024-09-14T09:07:12.712 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.712 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.712 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.712 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:07:12.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:07:12.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186509+0000", 2024-09-14T09:07:12.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2640000000000001e-06 2024-09-14T09:07:12.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186516+0000", 2024-09-14T09:07:12.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1070000000000001e-06 2024-09-14T09:07:12.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186533+0000", 2024-09-14T09:07:12.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7818000000000001e-05 2024-09-14T09:07:12.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186543+0000", 2024-09-14T09:07:12.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5170000000000005e-06 2024-09-14T09:07:12.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290885+0000", 2024-09-14T09:07:12.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.104342221 2024-09-14T09:07:12.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290893+0000", 2024-09-14T09:07:12.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2859999999999999e-06 2024-09-14T09:07:12.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290945+0000", 2024-09-14T09:07:12.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1382999999999999e-05 2024-09-14T09:07:12.715 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.715 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.715 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.715 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:84137 3.7 3:e15b49d9:::rbd_data.115e77b799c3.00000000000000e0:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:07:12.716 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:12.716 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.334035251, 2024-09-14T09:07:12.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11205707500000001, 2024-09-14T09:07:12.716 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.716 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.716 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:12.716 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.716 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.716 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.716 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 84137 2024-09-14T09:07:12.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.716 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:12.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:12.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323362+0000", 2024-09-14T09:07:12.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0839999999999999e-06 2024-09-14T09:07:12.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323370+0000", 2024-09-14T09:07:12.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0180000000000001e-06 2024-09-14T09:07:12.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323388+0000", 2024-09-14T09:07:12.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8326999999999999e-05 2024-09-14T09:07:12.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323397+0000", 2024-09-14T09:07:12.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1940000000000004e-06 2024-09-14T09:07:12.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323452+0000", 2024-09-14T09:07:12.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5043000000000001e-05 2024-09-14T09:07:12.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323550+0000", 2024-09-14T09:07:12.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8201000000000004e-05 2024-09-14T09:07:12.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:12.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.328273+0000", 2024-09-14T09:07:12.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0047227079999999999 2024-09-14T09:07:12.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:12.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435232+0000", 2024-09-14T09:07:12.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.106959339 2024-09-14T09:07:12.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:12.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435256+0000", 2024-09-14T09:07:12.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3774e-05 2024-09-14T09:07:12.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:12.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435314+0000", 2024-09-14T09:07:12.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7636999999999997e-05 2024-09-14T09:07:12.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435417+0000", 2024-09-14T09:07:12.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010275 2024-09-14T09:07:12.721 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.721 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.721 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.721 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:84137 3.7 e92/23 ondisk, result = 0)", 2024-09-14T09:07:12.721 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:53.341180+0000", 2024-09-14T09:07:12.721 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.316214667000001, 2024-09-14T09:07:12.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.094062506000000004, 2024-09-14T09:07:12.722 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.722 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.722 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.722 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341180+0000", 2024-09-14T09:07:12.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341180+0000", 2024-09-14T09:07:12.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341181+0000", 2024-09-14T09:07:12.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.192e-06 2024-09-14T09:07:12.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341187+0000", 2024-09-14T09:07:12.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0070000000000003e-06 2024-09-14T09:07:12.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341208+0000", 2024-09-14T09:07:12.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0321000000000001e-05 2024-09-14T09:07:12.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341219+0000", 2024-09-14T09:07:12.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1764e-05 2024-09-14T09:07:12.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435214+0000", 2024-09-14T09:07:12.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.093994438999999999 2024-09-14T09:07:12.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435225+0000", 2024-09-14T09:07:12.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1123e-05 2024-09-14T09:07:12.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435243+0000", 2024-09-14T09:07:12.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.766e-05 2024-09-14T09:07:12.725 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.725 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.725 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.725 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:92675 3.8 3:105e0099:::rbd_data.115e7770ddc5.0000000000000016:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:12.725 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:57.736053+0000", 2024-09-14T09:07:12.725 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.921341421999999, 2024-09-14T09:07:12.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073115555999999998, 2024-09-14T09:07:12.725 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.725 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.726 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:12.726 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.726 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.726 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.726 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 92675 2024-09-14T09:07:12.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.726 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736053+0000", 2024-09-14T09:07:12.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736053+0000", 2024-09-14T09:07:12.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736055+0000", 2024-09-14T09:07:12.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.33e-06 2024-09-14T09:07:12.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736062+0000", 2024-09-14T09:07:12.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.08e-06 2024-09-14T09:07:12.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736077+0000", 2024-09-14T09:07:12.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5605000000000001e-05 2024-09-14T09:07:12.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736088+0000", 2024-09-14T09:07:12.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0705e-05 2024-09-14T09:07:12.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736175+0000", 2024-09-14T09:07:12.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6926000000000001e-05 2024-09-14T09:07:12.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736325+0000", 2024-09-14T09:07:12.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149739 2024-09-14T09:07:12.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:12.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.740035+0000", 2024-09-14T09:07:12.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0037099250000000002 2024-09-14T09:07:12.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:12.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809001+0000", 2024-09-14T09:07:12.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.068966240999999998 2024-09-14T09:07:12.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:12.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809049+0000", 2024-09-14T09:07:12.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8572999999999997e-05 2024-09-14T09:07:12.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:12.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809102+0000", 2024-09-14T09:07:12.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2564e-05 2024-09-14T09:07:12.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809169+0000", 2024-09-14T09:07:12.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6867999999999997e-05 2024-09-14T09:07:12.731 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.731 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.731 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.731 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:92675 3.8 e94/23 ondisk, result = 0)", 2024-09-14T09:07:12.731 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:57.748881+0000", 2024-09-14T09:07:12.731 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.908513372, 2024-09-14T09:07:12.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060300578000000001, 2024-09-14T09:07:12.731 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.731 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.731 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.731 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748881+0000", 2024-09-14T09:07:12.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748881+0000", 2024-09-14T09:07:12.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748883+0000", 2024-09-14T09:07:12.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.491e-06 2024-09-14T09:07:12.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748887+0000", 2024-09-14T09:07:12.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5310000000000003e-06 2024-09-14T09:07:12.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748904+0000", 2024-09-14T09:07:12.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.662e-05 2024-09-14T09:07:12.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.748912+0000", 2024-09-14T09:07:12.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3869999999999995e-06 2024-09-14T09:07:12.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809034+0000", 2024-09-14T09:07:12.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060121757999999997 2024-09-14T09:07:12.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809044+0000", 2024-09-14T09:07:12.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7699999999999996e-06 2024-09-14T09:07:12.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809182+0000", 2024-09-14T09:07:12.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000138021 2024-09-14T09:07:12.735 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.735 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.735 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.735 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:93277 3.8 3:105e0099:::rbd_data.115e7770ddc5.0000000000000016:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:07:12.735 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.097058+0000", 2024-09-14T09:07:12.735 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.560337111000001, 2024-09-14T09:07:12.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.065618911000000002, 2024-09-14T09:07:12.735 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.735 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.735 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:12.736 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.736 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.736 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.736 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 93277 2024-09-14T09:07:12.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.736 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097058+0000", 2024-09-14T09:07:12.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097058+0000", 2024-09-14T09:07:12.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097060+0000", 2024-09-14T09:07:12.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.497e-06 2024-09-14T09:07:12.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097072+0000", 2024-09-14T09:07:12.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1600000000000001e-05 2024-09-14T09:07:12.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097102+0000", 2024-09-14T09:07:12.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0552999999999997e-05 2024-09-14T09:07:12.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097124+0000", 2024-09-14T09:07:12.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1477000000000001e-05 2024-09-14T09:07:12.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097201+0000", 2024-09-14T09:07:12.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7065999999999997e-05 2024-09-14T09:07:12.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097277+0000", 2024-09-14T09:07:12.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5722000000000004e-05 2024-09-14T09:07:12.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:12.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.101895+0000", 2024-09-14T09:07:12.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004618212 2024-09-14T09:07:12.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:12.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162452+0000", 2024-09-14T09:07:12.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060557603000000002 2024-09-14T09:07:12.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:12.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162504+0000", 2024-09-14T09:07:12.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.147e-05 2024-09-14T09:07:12.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:12.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162643+0000", 2024-09-14T09:07:12.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000139245 2024-09-14T09:07:12.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162677+0000", 2024-09-14T09:07:12.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3466000000000001e-05 2024-09-14T09:07:12.740 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.740 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.740 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.741 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:95495 3.18 3:1d9f7d98:::rbd_data.115e7770ddc5.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:07:12.741 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.598437+0000", 2024-09-14T09:07:12.741 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.058958062, 2024-09-14T09:07:12.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.090910543999999996, 2024-09-14T09:07:12.741 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.741 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.741 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:12.741 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.741 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.741 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.741 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 95495 2024-09-14T09:07:12.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.741 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598437+0000", 2024-09-14T09:07:12.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598437+0000", 2024-09-14T09:07:12.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598438+0000", 2024-09-14T09:07:12.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1149999999999999e-06 2024-09-14T09:07:12.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598443+0000", 2024-09-14T09:07:12.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4589999999999997e-06 2024-09-14T09:07:12.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598455+0000", 2024-09-14T09:07:12.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1793e-05 2024-09-14T09:07:12.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598463+0000", 2024-09-14T09:07:12.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9759999999999995e-06 2024-09-14T09:07:12.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598523+0000", 2024-09-14T09:07:12.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9926000000000002e-05 2024-09-14T09:07:12.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598581+0000", 2024-09-14T09:07:12.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8285000000000002e-05 2024-09-14T09:07:12.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:12.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.602809+0000", 2024-09-14T09:07:12.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0042279079999999998 2024-09-14T09:07:12.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:12.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689202+0000", 2024-09-14T09:07:12.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.086392847999999994 2024-09-14T09:07:12.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:12.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689230+0000", 2024-09-14T09:07:12.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8501999999999999e-05 2024-09-14T09:07:12.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:12.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689290+0000", 2024-09-14T09:07:12.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9524000000000001e-05 2024-09-14T09:07:12.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689347+0000", 2024-09-14T09:07:12.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7207999999999998e-05 2024-09-14T09:07:12.746 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.746 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.746 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.746 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:95495 3.18 e95/80 ondisk, result = 0)", 2024-09-14T09:07:12.746 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.610054+0000", 2024-09-14T09:07:12.747 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.047340597, 2024-09-14T09:07:12.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079165141999999994, 2024-09-14T09:07:12.747 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.747 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.747 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.747 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610054+0000", 2024-09-14T09:07:12.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610054+0000", 2024-09-14T09:07:12.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610055+0000", 2024-09-14T09:07:12.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2669999999999999e-06 2024-09-14T09:07:12.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610061+0000", 2024-09-14T09:07:12.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:07:12.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610078+0000", 2024-09-14T09:07:12.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6965000000000001e-05 2024-09-14T09:07:12.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610087+0000", 2024-09-14T09:07:12.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.611e-06 2024-09-14T09:07:12.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689008+0000", 2024-09-14T09:07:12.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.078921186000000004 2024-09-14T09:07:12.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689020+0000", 2024-09-14T09:07:12.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.132e-05 2024-09-14T09:07:12.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689219+0000", 2024-09-14T09:07:12.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000199469 2024-09-14T09:07:12.750 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.750 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.750 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.751 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:97796 3.15 3:aaaa5f7f:::rbd_data.115e113dc8ca.0000000000000029:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:07:12.751 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:12.751 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.56291377899999995, 2024-09-14T09:07:12.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14537731300000001, 2024-09-14T09:07:12.751 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.751 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.751 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:12.751 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:12.751 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:12.751 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:12.751 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97796 2024-09-14T09:07:12.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.751 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:12.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:12.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094482+0000", 2024-09-14T09:07:12.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6400000000000001e-07 2024-09-14T09:07:12.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094485+0000", 2024-09-14T09:07:12.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.693e-06 2024-09-14T09:07:12.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094494+0000", 2024-09-14T09:07:12.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9590000000000001e-06 2024-09-14T09:07:12.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094502+0000", 2024-09-14T09:07:12.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3180000000000001e-06 2024-09-14T09:07:12.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094567+0000", 2024-09-14T09:07:12.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5526999999999998e-05 2024-09-14T09:07:12.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094726+0000", 2024-09-14T09:07:12.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015842499999999999 2024-09-14T09:07:12.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:07:12.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.101940+0000", 2024-09-14T09:07:12.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0072146559999999998 2024-09-14T09:07:12.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:12.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239665+0000", 2024-09-14T09:07:12.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.137724501 2024-09-14T09:07:12.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:12.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239726+0000", 2024-09-14T09:07:12.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1550999999999994e-05 2024-09-14T09:07:12.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:12.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239800+0000", 2024-09-14T09:07:12.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3318000000000005e-05 2024-09-14T09:07:12.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239858+0000", 2024-09-14T09:07:12.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8600999999999997e-05 2024-09-14T09:07:12.756 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.756 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.756 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.756 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:97796 3.15 e106/103 ondisk, result = 0)", 2024-09-14T09:07:12.757 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:12.757 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.54056791000000004, 2024-09-14T09:07:12.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123043465, 2024-09-14T09:07:12.757 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:12.757 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:12.757 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:12.757 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:12.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:12.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:12.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:12.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:12.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:12.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:12.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116828+0000", 2024-09-14T09:07:12.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8100000000000002e-07 2024-09-14T09:07:12.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:12.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116831+0000", 2024-09-14T09:07:12.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3009999999999998e-06 2024-09-14T09:07:12.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:12.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116841+0000", 2024-09-14T09:07:12.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7829999999999993e-06 2024-09-14T09:07:12.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:12.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116851+0000", 2024-09-14T09:07:12.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0135999999999999e-05 2024-09-14T09:07:12.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:12.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239707+0000", 2024-09-14T09:07:12.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12285581499999999 2024-09-14T09:07:12.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:12.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239718+0000", 2024-09-14T09:07:12.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1804000000000001e-05 2024-09-14T09:07:12.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:12.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:12.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:12.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239870+0000", 2024-09-14T09:07:12.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015194500000000001 2024-09-14T09:07:12.760 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.760 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.760 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.760 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:12.760 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:12.761 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:12.762 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:07:12.763 INFO:teuthology.orchestra.run.smithi176.stdout:4918 trunc from 0xe7e9c1d to 0xd1383e2 2024-09-14T09:07:12.763 INFO:teuthology.orchestra.run.smithi176.stdout:4919 read 0x2b608bd thru 0x2b6fcf1 (0xf435 bytes) 2024-09-14T09:07:12.763 INFO:teuthology.orchestra.run.smithi176.stdout:4920 trunc from 0xd1383e2 to 0xc3f4830 2024-09-14T09:07:12.763 INFO:teuthology.orchestra.run.smithi176.stdout:4921 punch from 0x9e95bcf to 0x9e96a74, (0xea5 bytes) 2024-09-14T09:07:12.764 INFO:teuthology.orchestra.run.smithi176.stdout:4922 write 0xb7f9510 thru 0xb7fa192 (0xc83 bytes) 2024-09-14T09:07:12.764 INFO:teuthology.orchestra.run.smithi176.stdout:4923 read 0x447d237 thru 0x448c2fc (0xf0c6 bytes) 2024-09-14T09:07:12.764 INFO:teuthology.orchestra.run.smithi176.stdout:4924 writesame 0x7e875b6 thru 0x7e91f3d data_size 0xaf(0xa988 bytes) 2024-09-14T09:07:12.764 INFO:teuthology.orchestra.run.smithi176.stdout:4925 write 0xeb45b67 thru 0xeb4afea (0x5484 bytes) 2024-09-14T09:07:12.764 INFO:teuthology.orchestra.run.smithi176.stdout:4926 write 0xc826c85 thru 0xc82b04f (0x43cb bytes) 2024-09-14T09:07:12.764 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:12.763+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:12.764 INFO:teuthology.orchestra.run.smithi176.stdout:4927 write 0x50184f5 thru 0x5021d12 (0x981e bytes) 2024-09-14T09:07:12.776 INFO:teuthology.orchestra.run.smithi176.stdout:4929 write 0x8f1566f thru 0x8f17092 (0x1a24 bytes) 2024-09-14T09:07:12.783 INFO:teuthology.orchestra.run.smithi176.stdout:4930 write 0x4479f1e thru 0x44836fc (0x97df bytes) 2024-09-14T09:07:12.792 INFO:teuthology.orchestra.run.smithi176.stdout:4932 punch from 0x5075e0f to 0x507cbb4, (0x6da5 bytes) 2024-09-14T09:07:12.792 INFO:teuthology.orchestra.run.smithi176.stdout:4937 write 0xbabb7f2 thru 0xbac3154 (0x7963 bytes) 2024-09-14T09:07:12.819 INFO:teuthology.orchestra.run.smithi176.stdout:4938 trunc from 0xeb4afeb to 0x73b70e6 2024-09-14T09:07:12.847 INFO:teuthology.orchestra.run.smithi176.stdout:4939 punch from 0x1355e58 to 0x1358038, (0x21e0 bytes) 2024-09-14T09:07:12.847 INFO:teuthology.orchestra.run.smithi176.stdout:4940 read 0x88fbcb thru 0x895f8f (0x63c5 bytes) 2024-09-14T09:07:12.848 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:12.848 INFO:teuthology.orchestra.run.smithi176.stdout:4942 write 0x1ab48c4 thru 0x1ab6c95 (0x23d2 bytes) 2024-09-14T09:07:12.855 INFO:teuthology.orchestra.run.smithi176.stdout:4943 writesame 0x4be5e2c thru 0x4bf4e40 data_size 0x3(0xf015 bytes) 2024-09-14T09:07:12.863 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:12.863+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:12.865 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is set 2024-09-14T09:07:12.902 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:12.902 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:12.903 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:07:12.903 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:12.917 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:07:12.954 INFO:teuthology.orchestra.run.smithi176.stdout:4945 writesame 0x14e2c33 thru 0x14eafb8 data_size 0x82(0x8386 bytes) 2024-09-14T09:07:12.959 INFO:teuthology.orchestra.run.smithi176.stdout:4947 punch from 0x1b688de to 0x1b75566, (0xcc88 bytes) 2024-09-14T09:07:12.959 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:12.959 INFO:teuthology.orchestra.run.smithi176.stdout:4950 write 0x48e0f49 thru 0x48f0b25 (0xfbdd bytes) 2024-09-14T09:07:12.964 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:12.967+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:12.974 INFO:teuthology.orchestra.run.smithi176.stdout:4951 trunc from 0x73b70e6 to 0x53ab35b 2024-09-14T09:07:12.995 INFO:teuthology.orchestra.run.smithi176.stdout:4952 punch from 0x2c82f9 to 0x2cea4c, (0x6753 bytes) 2024-09-14T09:07:13.013 INFO:teuthology.orchestra.run.smithi176.stdout:4954 write 0x726a74f thru 0x7276f12 (0xc7c4 bytes) 2024-09-14T09:07:13.022 INFO:teuthology.orchestra.run.smithi176.stdout:4955 punch from 0x33882f to 0x347583, (0xed54 bytes) 2024-09-14T09:07:13.028 INFO:teuthology.orchestra.run.smithi176.stdout:4956 trunc from 0x7276f13 to 0x96271e9 2024-09-14T09:07:13.040 INFO:teuthology.orchestra.run.smithi176.stdout:4957 read 0x8228e13 thru 0x82332c4 (0xa4b2 bytes) 2024-09-14T09:07:13.040 INFO:teuthology.orchestra.run.smithi176.stdout:4958 read 0x77e5f9 thru 0x7867ba (0x81c2 bytes) 2024-09-14T09:07:13.042 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:13.042 INFO:teuthology.orchestra.run.smithi176.stdout:4960 write 0x34196b4 thru 0x342706f (0xd9bc bytes) 2024-09-14T09:07:13.046 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:13.046 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:13.047 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:07:13.047 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:07:13.047 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:13.055 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:13.055 INFO:teuthology.orchestra.run.smithi176.stdout:4964 writesame 0x2ed98b7 thru 0x2edbe2c data_size 0x23(0x2576 bytes) 2024-09-14T09:07:13.061 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:07:13.062 INFO:teuthology.orchestra.run.smithi176.stdout:4966 write 0x23bcac0 thru 0x23cc6b5 (0xfbf6 bytes) 2024-09-14T09:07:13.077 INFO:teuthology.orchestra.run.smithi176.stdout:4968 punch from 0x5fa0764 to 0x5fa0cb4, (0x550 bytes) 2024-09-14T09:07:13.094 INFO:teuthology.orchestra.run.smithi176.stdout:4969 write 0xb79f4f8 thru 0xb79f718 (0x221 bytes) 2024-09-14T09:07:13.101 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:13.101 INFO:teuthology.orchestra.run.smithi176.stdout:4972 write 0x3a750b3 thru 0x3a820f7 (0xd045 bytes) 2024-09-14T09:07:13.103 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:13.107+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:13.115 INFO:teuthology.orchestra.run.smithi176.stdout:4973 read 0x847b24a thru 0x848692f (0xb6e6 bytes) 2024-09-14T09:07:13.115 INFO:teuthology.orchestra.run.smithi176.stdout:4974 write 0xa7fd5e thru 0xa8101c (0x12bf bytes) 2024-09-14T09:07:13.125 INFO:teuthology.orchestra.run.smithi176.stdout:4975 punch from 0x7b9b4b9 to 0x7ba5878, (0xa3bf bytes) 2024-09-14T09:07:13.126 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:13.126 INFO:teuthology.orchestra.run.smithi176.stdout:4978 write 0xb5a239a thru 0xb5a5f3a (0x3ba1 bytes) 2024-09-14T09:07:13.131 INFO:teuthology.orchestra.run.smithi176.stdout:4980 read 0x9ad33f3 thru 0x9ade0c5 (0xacd3 bytes) 2024-09-14T09:07:13.131 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:13.132 INFO:teuthology.orchestra.run.smithi176.stdout:4983 writesame 0xa462aae thru 0xa463cda data_size 0xb(0x122d bytes) 2024-09-14T09:07:13.137 INFO:teuthology.orchestra.run.smithi176.stdout:4984 read 0x7c2c86c thru 0x7c3325b (0x69f0 bytes) 2024-09-14T09:07:13.153 INFO:teuthology.orchestra.run.smithi176.stdout:4985 write 0xd2e3fa0 thru 0xd2ebb75 (0x7bd6 bytes) 2024-09-14T09:07:13.165 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:13.167+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:13.183 INFO:teuthology.orchestra.run.smithi176.stdout:4986 trunc from 0xd2ebb76 to 0x4156f57 2024-09-14T09:07:13.216 INFO:teuthology.orchestra.run.smithi176.stdout:4989 writesame 0x33522d thru 0x339f19 data_size 0x1(0x4ced bytes) 2024-09-14T09:07:13.236 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:13.236 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:07:13.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:07:13.237 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:07:13.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.237 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:9982 3.1a 3:5af3e476:::rbd_data.115eadec874.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:07:13.237 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:07:13.237 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 95.639807465999993, 2024-09-14T09:07:13.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042276341000000002, 2024-09-14T09:07:13.238 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.238 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.238 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:13.238 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:13.238 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:13.239 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:13.239 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 9982 2024-09-14T09:07:13.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.239 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:07:13.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:07:13.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597970+0000", 2024-09-14T09:07:13.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0599999999999999e-07 2024-09-14T09:07:13.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597974+0000", 2024-09-14T09:07:13.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2649999999999998e-06 2024-09-14T09:07:13.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597986+0000", 2024-09-14T09:07:13.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1019000000000001e-05 2024-09-14T09:07:13.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597991+0000", 2024-09-14T09:07:13.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5049999999999998e-06 2024-09-14T09:07:13.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.598083+0000", 2024-09-14T09:07:13.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2062e-05 2024-09-14T09:07:13.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.598227+0000", 2024-09-14T09:07:13.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000143609 2024-09-14T09:07:13.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:13.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.603107+0000", 2024-09-14T09:07:13.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048805819999999996 2024-09-14T09:07:13.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:13.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640002+0000", 2024-09-14T09:07:13.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.036894641999999998 2024-09-14T09:07:13.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:13.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640077+0000", 2024-09-14T09:07:13.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4927999999999998e-05 2024-09-14T09:07:13.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:13.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640174+0000", 2024-09-14T09:07:13.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7417999999999995e-05 2024-09-14T09:07:13.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640246+0000", 2024-09-14T09:07:13.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1505000000000001e-05 2024-09-14T09:07:13.250 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.250 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.251 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.251 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:76803 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-14T09:07:13.251 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:43.858877+0000", 2024-09-14T09:07:13.251 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.378900036000001, 2024-09-14T09:07:13.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0137577900000001, 2024-09-14T09:07:13.252 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.252 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.252 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:13.252 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:13.252 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:13.253 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:13.253 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 76803 2024-09-14T09:07:13.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.253 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858877+0000", 2024-09-14T09:07:13.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858877+0000", 2024-09-14T09:07:13.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858879+0000", 2024-09-14T09:07:13.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.097e-06 2024-09-14T09:07:13.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858887+0000", 2024-09-14T09:07:13.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6510000000000001e-06 2024-09-14T09:07:13.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858903+0000", 2024-09-14T09:07:13.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6560999999999999e-05 2024-09-14T09:07:13.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858912+0000", 2024-09-14T09:07:13.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2019999999999993e-06 2024-09-14T09:07:13.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858964+0000", 2024-09-14T09:07:13.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2046999999999998e-05 2024-09-14T09:07:13.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:13.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858972+0000", 2024-09-14T09:07:13.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6029999999999999e-06 2024-09-14T09:07:13.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:44.871832+0000", 2024-09-14T09:07:13.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.012860109 2024-09-14T09:07:13.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:44.872248+0000", 2024-09-14T09:07:13.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00041549300000000001 2024-09-14T09:07:13.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:44.872635+0000", 2024-09-14T09:07:13.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038702700000000002 2024-09-14T09:07:13.263 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.263 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.263 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.264 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:80293 3.14 3:2933aeec:::rbd_data.115e2eebe49a.0000000000000004:1d [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-14T09:07:13.264 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:49.353426+0000", 2024-09-14T09:07:13.264 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.884350991000002, 2024-09-14T09:07:13.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.044484119000000003, 2024-09-14T09:07:13.264 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.265 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.265 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:13.265 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:13.265 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:13.265 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:13.265 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 80293 2024-09-14T09:07:13.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.266 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353426+0000", 2024-09-14T09:07:13.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353426+0000", 2024-09-14T09:07:13.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353427+0000", 2024-09-14T09:07:13.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8299999999999996e-07 2024-09-14T09:07:13.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353430+0000", 2024-09-14T09:07:13.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2739999999999999e-06 2024-09-14T09:07:13.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353439+0000", 2024-09-14T09:07:13.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8359999999999998e-06 2024-09-14T09:07:13.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353446+0000", 2024-09-14T09:07:13.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3599999999999998e-06 2024-09-14T09:07:13.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.389701+0000", 2024-09-14T09:07:13.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.036255250000000003 2024-09-14T09:07:13.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.390249+0000", 2024-09-14T09:07:13.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000547381 2024-09-14T09:07:13.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.397910+0000", 2024-09-14T09:07:13.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076612349999999997 2024-09-14T09:07:13.269 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.270 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.270 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.270 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:80322 3.14 3:2933aeec:::rbd_data.115e2eebe49a.0000000000000004:1d [read 16384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-14T09:07:13.270 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:49.353456+0000", 2024-09-14T09:07:13.270 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.884320584000001, 2024-09-14T09:07:13.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.045427997999999997, 2024-09-14T09:07:13.270 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.270 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.270 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:13.270 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:13.270 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:13.270 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:13.270 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 80322 2024-09-14T09:07:13.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.271 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353456+0000", 2024-09-14T09:07:13.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353456+0000", 2024-09-14T09:07:13.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353457+0000", 2024-09-14T09:07:13.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3300000000000001e-07 2024-09-14T09:07:13.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353460+0000", 2024-09-14T09:07:13.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0000000000000001e-06 2024-09-14T09:07:13.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353469+0000", 2024-09-14T09:07:13.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.636e-06 2024-09-14T09:07:13.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353477+0000", 2024-09-14T09:07:13.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0099999999999995e-06 2024-09-14T09:07:13.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.397975+0000", 2024-09-14T09:07:13.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.044498217999999999 2024-09-14T09:07:13.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.398131+0000", 2024-09-14T09:07:13.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015567099999999999 2024-09-14T09:07:13.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.398884+0000", 2024-09-14T09:07:13.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00075372999999999998 2024-09-14T09:07:13.275 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.275 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.275 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.275 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:80323 3.14 3:2933aeec:::rbd_data.115e2eebe49a.0000000000000004:1d [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-14T09:07:13.275 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:49.353490+0000", 2024-09-14T09:07:13.275 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.884286455000002, 2024-09-14T09:07:13.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.046828271999999997, 2024-09-14T09:07:13.275 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.275 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.275 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:13.276 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:13.276 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:13.276 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:13.276 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 80323 2024-09-14T09:07:13.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.276 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353490+0000", 2024-09-14T09:07:13.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353490+0000", 2024-09-14T09:07:13.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353491+0000", 2024-09-14T09:07:13.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5000000000000001e-07 2024-09-14T09:07:13.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353497+0000", 2024-09-14T09:07:13.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.947e-06 2024-09-14T09:07:13.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353513+0000", 2024-09-14T09:07:13.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5769000000000001e-05 2024-09-14T09:07:13.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353525+0000", 2024-09-14T09:07:13.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2094e-05 2024-09-14T09:07:13.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.398947+0000", 2024-09-14T09:07:13.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.045421802999999997 2024-09-14T09:07:13.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.399130+0000", 2024-09-14T09:07:13.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000182776 2024-09-14T09:07:13.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.400319+0000", 2024-09-14T09:07:13.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011889330000000001 2024-09-14T09:07:13.279 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.279 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.279 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.279 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:80324 3.14 3:2933aeec:::rbd_data.115e2eebe49a.0000000000000004:1d [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-14T09:07:13.279 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:49.353544+0000", 2024-09-14T09:07:13.279 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.884232484999998, 2024-09-14T09:07:13.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.047536104000000003, 2024-09-14T09:07:13.279 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.279 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.280 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:13.280 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:13.280 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:13.280 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:13.280 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 80324 2024-09-14T09:07:13.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.280 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353544+0000", 2024-09-14T09:07:13.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353544+0000", 2024-09-14T09:07:13.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353545+0000", 2024-09-14T09:07:13.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8599999999999997e-07 2024-09-14T09:07:13.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353549+0000", 2024-09-14T09:07:13.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7840000000000001e-06 2024-09-14T09:07:13.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353566+0000", 2024-09-14T09:07:13.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7212999999999999e-05 2024-09-14T09:07:13.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353579+0000", 2024-09-14T09:07:13.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2537e-05 2024-09-14T09:07:13.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.400380+0000", 2024-09-14T09:07:13.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.046800913 2024-09-14T09:07:13.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.400524+0000", 2024-09-14T09:07:13.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014461500000000001 2024-09-14T09:07:13.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.401081+0000", 2024-09-14T09:07:13.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055625599999999998 2024-09-14T09:07:13.283 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.283 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.283 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.283 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:80325 3.14 3:2933aeec:::rbd_data.115e2eebe49a.0000000000000004:1d [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-14T09:07:13.283 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:49.353599+0000", 2024-09-14T09:07:13.283 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.884177932, 2024-09-14T09:07:13.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.048432831000000003, 2024-09-14T09:07:13.283 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.284 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.284 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:13.284 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:13.284 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:13.284 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:13.284 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 80325 2024-09-14T09:07:13.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.284 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353599+0000", 2024-09-14T09:07:13.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353599+0000", 2024-09-14T09:07:13.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353600+0000", 2024-09-14T09:07:13.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.266e-06 2024-09-14T09:07:13.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353606+0000", 2024-09-14T09:07:13.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8900000000000004e-06 2024-09-14T09:07:13.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353623+0000", 2024-09-14T09:07:13.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6923e-05 2024-09-14T09:07:13.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353637+0000", 2024-09-14T09:07:13.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3943e-05 2024-09-14T09:07:13.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.401169+0000", 2024-09-14T09:07:13.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.047532079999999997 2024-09-14T09:07:13.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.401340+0000", 2024-09-14T09:07:13.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017054700000000001 2024-09-14T09:07:13.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.402032+0000", 2024-09-14T09:07:13.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069218199999999995 2024-09-14T09:07:13.287 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.287 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.287 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.287 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:80354 3.14 3:2933aeec:::rbd_data.115e2eebe49a.0000000000000004:1d [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-14T09:07:13.287 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:49.353658+0000", 2024-09-14T09:07:13.288 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.884119052999999, 2024-09-14T09:07:13.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.050321308000000002, 2024-09-14T09:07:13.288 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.288 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.288 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:13.288 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:13.288 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:13.288 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:13.288 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 80354 2024-09-14T09:07:13.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.288 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353658+0000", 2024-09-14T09:07:13.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353658+0000", 2024-09-14T09:07:13.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353659+0000", 2024-09-14T09:07:13.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0499999999999999e-06 2024-09-14T09:07:13.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353664+0000", 2024-09-14T09:07:13.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5330000000000002e-06 2024-09-14T09:07:13.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353682+0000", 2024-09-14T09:07:13.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-05 2024-09-14T09:07:13.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353696+0000", 2024-09-14T09:07:13.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3699999999999999e-05 2024-09-14T09:07:13.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.402990+0000", 2024-09-14T09:07:13.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.049294075 2024-09-14T09:07:13.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.403147+0000", 2024-09-14T09:07:13.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015756099999999999 2024-09-14T09:07:13.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.403979+0000", 2024-09-14T09:07:13.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00083194900000000001 2024-09-14T09:07:13.291 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.291 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.291 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.291 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:80355 3.14 3:2933aeec:::rbd_data.115e2eebe49a.0000000000000004:1d [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-14T09:07:13.291 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:49.353717+0000", 2024-09-14T09:07:13.291 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.884060216999998, 2024-09-14T09:07:13.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.051852537999999997, 2024-09-14T09:07:13.292 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.292 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.292 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:13.292 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:13.292 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:13.292 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:13.292 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 80355 2024-09-14T09:07:13.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.292 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353717+0000", 2024-09-14T09:07:13.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353717+0000", 2024-09-14T09:07:13.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353718+0000", 2024-09-14T09:07:13.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1769999999999999e-06 2024-09-14T09:07:13.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353725+0000", 2024-09-14T09:07:13.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0770000000000002e-06 2024-09-14T09:07:13.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353743+0000", 2024-09-14T09:07:13.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7943000000000001e-05 2024-09-14T09:07:13.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353753+0000", 2024-09-14T09:07:13.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0326000000000001e-05 2024-09-14T09:07:13.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.404040+0000", 2024-09-14T09:07:13.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.050286889000000001 2024-09-14T09:07:13.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.404614+0000", 2024-09-14T09:07:13.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00057370100000000003 2024-09-14T09:07:13.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.405569+0000", 2024-09-14T09:07:13.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00095542499999999998 2024-09-14T09:07:13.295 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.295 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.295 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.296 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:80356 3.14 3:2933aeec:::rbd_data.115e2eebe49a.0000000000000004:1d [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-14T09:07:13.296 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:49.353766+0000", 2024-09-14T09:07:13.296 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.884011257000001, 2024-09-14T09:07:13.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.052613749000000001, 2024-09-14T09:07:13.296 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.296 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.296 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:13.296 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:13.296 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:13.296 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:13.296 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 80356 2024-09-14T09:07:13.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.296 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353766+0000", 2024-09-14T09:07:13.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353766+0000", 2024-09-14T09:07:13.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353766+0000", 2024-09-14T09:07:13.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4300000000000002e-07 2024-09-14T09:07:13.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353770+0000", 2024-09-14T09:07:13.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6030000000000001e-06 2024-09-14T09:07:13.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353779+0000", 2024-09-14T09:07:13.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2070000000000001e-06 2024-09-14T09:07:13.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353786+0000", 2024-09-14T09:07:13.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.007e-06 2024-09-14T09:07:13.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.405698+0000", 2024-09-14T09:07:13.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.051911539 2024-09-14T09:07:13.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.405849+0000", 2024-09-14T09:07:13.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000151051 2024-09-14T09:07:13.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.406379+0000", 2024-09-14T09:07:13.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00053049900000000003 2024-09-14T09:07:13.299 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.299 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.299 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.300 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:80357 3.14 3:2933aeec:::rbd_data.115e2eebe49a.0000000000000004:1d [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-14T09:07:13.300 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:49.353797+0000", 2024-09-14T09:07:13.300 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.883979843999999, 2024-09-14T09:07:13.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.053313526, 2024-09-14T09:07:13.300 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.300 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.300 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:13.300 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:13.300 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:13.300 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:13.300 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 80357 2024-09-14T09:07:13.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.300 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353797+0000", 2024-09-14T09:07:13.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353797+0000", 2024-09-14T09:07:13.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353798+0000", 2024-09-14T09:07:13.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5999999999999991e-07 2024-09-14T09:07:13.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353803+0000", 2024-09-14T09:07:13.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0830000000000003e-06 2024-09-14T09:07:13.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353820+0000", 2024-09-14T09:07:13.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6816999999999998e-05 2024-09-14T09:07:13.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353827+0000", 2024-09-14T09:07:13.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3130000000000001e-06 2024-09-14T09:07:13.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.406443+0000", 2024-09-14T09:07:13.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.052616164999999999 2024-09-14T09:07:13.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.406584+0000", 2024-09-14T09:07:13.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014090899999999999 2024-09-14T09:07:13.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.407111+0000", 2024-09-14T09:07:13.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00052627900000000005 2024-09-14T09:07:13.303 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.303 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.303 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.304 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:83981 3.3 3:c015e742:::rbd_data.115e77b799c3.0000000000000073:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:07:13.304 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:53.107172+0000", 2024-09-14T09:07:13.304 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.130604791, 2024-09-14T09:07:13.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.038857021999999998, 2024-09-14T09:07:13.304 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.304 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.304 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:13.304 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:13.304 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:13.304 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:13.304 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 83981 2024-09-14T09:07:13.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.304 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.107172+0000", 2024-09-14T09:07:13.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.107172+0000", 2024-09-14T09:07:13.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.107175+0000", 2024-09-14T09:07:13.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7829999999999999e-06 2024-09-14T09:07:13.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.107189+0000", 2024-09-14T09:07:13.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.366e-05 2024-09-14T09:07:13.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.107222+0000", 2024-09-14T09:07:13.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3232999999999998e-05 2024-09-14T09:07:13.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.107237+0000", 2024-09-14T09:07:13.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5489999999999999e-05 2024-09-14T09:07:13.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.107345+0000", 2024-09-14T09:07:13.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010822199999999999 2024-09-14T09:07:13.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.107566+0000", 2024-09-14T09:07:13.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022014199999999999 2024-09-14T09:07:13.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:13.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.109326+0000", 2024-09-14T09:07:13.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017599359999999999 2024-09-14T09:07:13.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:13.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.145587+0000", 2024-09-14T09:07:13.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.036261104000000002 2024-09-14T09:07:13.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:13.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.145733+0000", 2024-09-14T09:07:13.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000146355 2024-09-14T09:07:13.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:13.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.145905+0000", 2024-09-14T09:07:13.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017187800000000001 2024-09-14T09:07:13.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.146029+0000", 2024-09-14T09:07:13.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012421900000000001 2024-09-14T09:07:13.309 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.309 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.309 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.309 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:84403 3.14 3:2fc85a9a:::rbd_data.115e77b799c3.0000000000000018:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:07:13.309 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:53.878069+0000", 2024-09-14T09:07:13.309 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.359708167000001, 2024-09-14T09:07:13.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.128852405, 2024-09-14T09:07:13.310 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.310 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.310 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:13.310 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:13.310 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:13.310 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:13.310 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 84403 2024-09-14T09:07:13.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.310 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878069+0000", 2024-09-14T09:07:13.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878069+0000", 2024-09-14T09:07:13.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878071+0000", 2024-09-14T09:07:13.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0899999999999999e-06 2024-09-14T09:07:13.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878082+0000", 2024-09-14T09:07:13.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1226999999999999e-05 2024-09-14T09:07:13.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878108+0000", 2024-09-14T09:07:13.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5568999999999999e-05 2024-09-14T09:07:13.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878122+0000", 2024-09-14T09:07:13.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.488e-05 2024-09-14T09:07:13.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878247+0000", 2024-09-14T09:07:13.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000124039 2024-09-14T09:07:13.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878387+0000", 2024-09-14T09:07:13.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014041600000000001 2024-09-14T09:07:13.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:13.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.886261+0000", 2024-09-14T09:07:13.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0078738499999999999 2024-09-14T09:07:13.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:13.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006598+0000", 2024-09-14T09:07:13.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.120337316 2024-09-14T09:07:13.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:13.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006724+0000", 2024-09-14T09:07:13.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012601600000000001 2024-09-14T09:07:13.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:13.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006827+0000", 2024-09-14T09:07:13.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103299 2024-09-14T09:07:13.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006921+0000", 2024-09-14T09:07:13.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3702999999999994e-05 2024-09-14T09:07:13.315 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.315 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.315 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.315 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:84403 3.14 e92/84 ondisk, result = 0)", 2024-09-14T09:07:13.315 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:53.909978+0000", 2024-09-14T09:07:13.315 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.327798651999998, 2024-09-14T09:07:13.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.097025845999999999, 2024-09-14T09:07:13.315 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.315 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.315 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:13.315 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.909978+0000", 2024-09-14T09:07:13.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.909978+0000", 2024-09-14T09:07:13.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.909980+0000", 2024-09-14T09:07:13.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.714e-06 2024-09-14T09:07:13.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.909988+0000", 2024-09-14T09:07:13.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7859999999999997e-06 2024-09-14T09:07:13.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.910017+0000", 2024-09-14T09:07:13.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9737e-05 2024-09-14T09:07:13.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.910039+0000", 2024-09-14T09:07:13.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1653000000000001e-05 2024-09-14T09:07:13.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006697+0000", 2024-09-14T09:07:13.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.096657958000000002 2024-09-14T09:07:13.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006715+0000", 2024-09-14T09:07:13.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.751e-05 2024-09-14T09:07:13.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.007004+0000", 2024-09-14T09:07:13.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028948800000000002 2024-09-14T09:07:13.318 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.318 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.318 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.319 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:93272 3.3 3:c49b4fbc:::rbd_data.115e7770ddc5.000000000000001a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:07:13.319 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.971357+0000", 2024-09-14T09:07:13.319 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.266420108, 2024-09-14T09:07:13.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.112268252, 2024-09-14T09:07:13.319 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.319 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.319 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:13.319 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:13.319 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:13.319 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:13.319 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 93272 2024-09-14T09:07:13.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.319 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971357+0000", 2024-09-14T09:07:13.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971357+0000", 2024-09-14T09:07:13.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971360+0000", 2024-09-14T09:07:13.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8250000000000001e-06 2024-09-14T09:07:13.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971371+0000", 2024-09-14T09:07:13.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1537e-05 2024-09-14T09:07:13.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971399+0000", 2024-09-14T09:07:13.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8274999999999999e-05 2024-09-14T09:07:13.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971417+0000", 2024-09-14T09:07:13.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7241999999999999e-05 2024-09-14T09:07:13.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971537+0000", 2024-09-14T09:07:13.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012010600000000001 2024-09-14T09:07:13.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971685+0000", 2024-09-14T09:07:13.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014777599999999999 2024-09-14T09:07:13.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:13.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.979172+0000", 2024-09-14T09:07:13.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0074876810000000004 2024-09-14T09:07:13.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:13.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083438+0000", 2024-09-14T09:07:13.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.104265969 2024-09-14T09:07:13.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:13.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083466+0000", 2024-09-14T09:07:13.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7665999999999999e-05 2024-09-14T09:07:13.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:13.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083553+0000", 2024-09-14T09:07:13.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6809000000000002e-05 2024-09-14T09:07:13.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083625+0000", 2024-09-14T09:07:13.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2366000000000005e-05 2024-09-14T09:07:13.324 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.324 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.324 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.324 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:93272 3.3 e95/86 ondisk, result = 0)", 2024-09-14T09:07:13.324 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.997797+0000", 2024-09-14T09:07:13.324 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.239980127000001, 2024-09-14T09:07:13.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.085655196000000003, 2024-09-14T09:07:13.324 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.325 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.325 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:13.325 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.997797+0000", 2024-09-14T09:07:13.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.997797+0000", 2024-09-14T09:07:13.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.997799+0000", 2024-09-14T09:07:13.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7460000000000001e-06 2024-09-14T09:07:13.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.997807+0000", 2024-09-14T09:07:13.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1110000000000008e-06 2024-09-14T09:07:13.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.997836+0000", 2024-09-14T09:07:13.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9580000000000001e-05 2024-09-14T09:07:13.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.997851+0000", 2024-09-14T09:07:13.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4708e-05 2024-09-14T09:07:13.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083413+0000", 2024-09-14T09:07:13.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.085562568000000006 2024-09-14T09:07:13.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083428+0000", 2024-09-14T09:07:13.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4970000000000001e-05 2024-09-14T09:07:13.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083452+0000", 2024-09-14T09:07:13.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3513e-05 2024-09-14T09:07:13.328 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.328 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.328 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.328 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:95446 3.9 3:9150faf5:::rbd_data.115e9a87c1b2.000000000000003c:21 [read 358400~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:07:13.328 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:58.504395+0000", 2024-09-14T09:07:13.328 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.7333815, 2024-09-14T09:07:13.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.038354049000000001, 2024-09-14T09:07:13.329 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.329 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.329 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:13.329 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:13.329 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:13.329 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:13.329 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 95446 2024-09-14T09:07:13.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.329 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.504395+0000", 2024-09-14T09:07:13.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.504395+0000", 2024-09-14T09:07:13.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.504396+0000", 2024-09-14T09:07:13.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3200000000000005e-07 2024-09-14T09:07:13.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.504408+0000", 2024-09-14T09:07:13.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1819e-05 2024-09-14T09:07:13.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.504419+0000", 2024-09-14T09:07:13.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1252e-05 2024-09-14T09:07:13.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.504427+0000", 2024-09-14T09:07:13.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3909999999999998e-06 2024-09-14T09:07:13.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.542656+0000", 2024-09-14T09:07:13.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.038229015999999998 2024-09-14T09:07:13.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.542708+0000", 2024-09-14T09:07:13.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1102e-05 2024-09-14T09:07:13.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.542749+0000", 2024-09-14T09:07:13.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1836999999999999e-05 2024-09-14T09:07:13.332 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.332 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.332 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.333 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:96694 3.11 3:8a7e3c58:::rbd_data.115e63832e35.000000000000001c:head [stat,truncate 131072] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-14T09:07:13.333 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:07.004845+0000", 2024-09-14T09:07:13.333 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.2329320920000004, 2024-09-14T09:07:13.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.85224771499999996, 2024-09-14T09:07:13.333 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.333 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.333 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:13.333 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:13.333 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:13.333 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:13.333 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 96694 2024-09-14T09:07:13.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.334 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004845+0000", 2024-09-14T09:07:13.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004845+0000", 2024-09-14T09:07:13.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004848+0000", 2024-09-14T09:07:13.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:07:13.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004862+0000", 2024-09-14T09:07:13.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4426e-05 2024-09-14T09:07:13.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004903+0000", 2024-09-14T09:07:13.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0447000000000001e-05 2024-09-14T09:07:13.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004926+0000", 2024-09-14T09:07:13.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3260000000000001e-05 2024-09-14T09:07:13.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.005104+0000", 2024-09-14T09:07:13.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000178018 2024-09-14T09:07:13.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:13.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.005120+0000", 2024-09-14T09:07:13.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6422999999999998e-05 2024-09-14T09:07:13.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.851775+0000", 2024-09-14T09:07:13.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.84665435499999997 2024-09-14T09:07:13.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.852176+0000", 2024-09-14T09:07:13.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000400787 2024-09-14T09:07:13.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-09-14T09:07:13.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.852662+0000", 2024-09-14T09:07:13.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048621899999999997 2024-09-14T09:07:13.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:13.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.855261+0000", 2024-09-14T09:07:13.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025991479999999999 2024-09-14T09:07:13.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:13.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.856839+0000", 2024-09-14T09:07:13.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015779819999999999 2024-09-14T09:07:13.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:13.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.856993+0000", 2024-09-14T09:07:13.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015431900000000001 2024-09-14T09:07:13.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.857093+0000", 2024-09-14T09:07:13.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9314000000000005e-05 2024-09-14T09:07:13.339 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.339 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.339 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.340 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:97612 3.e 3:7298f82d:::rbd_data.115e113dc8ca.00000000000000e8:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:07:13.340 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:11.399529+0000", 2024-09-14T09:07:13.340 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.838248039, 2024-09-14T09:07:13.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.18555535000000001, 2024-09-14T09:07:13.340 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.340 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.340 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:13.340 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:13.340 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:13.340 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:13.340 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 97612 2024-09-14T09:07:13.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.340 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399529+0000", 2024-09-14T09:07:13.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399529+0000", 2024-09-14T09:07:13.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399532+0000", 2024-09-14T09:07:13.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6599999999999999e-06 2024-09-14T09:07:13.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399544+0000", 2024-09-14T09:07:13.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2527000000000001e-05 2024-09-14T09:07:13.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399574+0000", 2024-09-14T09:07:13.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.97e-05 2024-09-14T09:07:13.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399588+0000", 2024-09-14T09:07:13.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4653e-05 2024-09-14T09:07:13.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399740+0000", 2024-09-14T09:07:13.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015180900000000001 2024-09-14T09:07:13.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399952+0000", 2024-09-14T09:07:13.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000211435 2024-09-14T09:07:13.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:13.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.411686+0000", 2024-09-14T09:07:13.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011733883000000001 2024-09-14T09:07:13.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:13.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584791+0000", 2024-09-14T09:07:13.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.17310596 2024-09-14T09:07:13.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:13.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584915+0000", 2024-09-14T09:07:13.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012330800000000001 2024-09-14T09:07:13.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:13.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.585010+0000", 2024-09-14T09:07:13.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5216000000000005e-05 2024-09-14T09:07:13.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.585084+0000", 2024-09-14T09:07:13.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4198999999999999e-05 2024-09-14T09:07:13.345 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.345 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.345 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.345 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:97612 3.e e106/103 ondisk, result = 0)", 2024-09-14T09:07:13.345 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:11.451700+0000", 2024-09-14T09:07:13.345 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.7860767449999999, 2024-09-14T09:07:13.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13339934000000001, 2024-09-14T09:07:13.346 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:13.346 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:13.346 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:13.346 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:13.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:13.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451700+0000", 2024-09-14T09:07:13.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:13.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451700+0000", 2024-09-14T09:07:13.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:13.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:13.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451702+0000", 2024-09-14T09:07:13.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.34e-06 2024-09-14T09:07:13.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:13.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451713+0000", 2024-09-14T09:07:13.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1e-05 2024-09-14T09:07:13.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:13.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451751+0000", 2024-09-14T09:07:13.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7762999999999997e-05 2024-09-14T09:07:13.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:13.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451770+0000", 2024-09-14T09:07:13.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8626000000000001e-05 2024-09-14T09:07:13.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:13.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584888+0000", 2024-09-14T09:07:13.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13311764500000001 2024-09-14T09:07:13.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:13.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584903+0000", 2024-09-14T09:07:13.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.499e-05 2024-09-14T09:07:13.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:13.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:13.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:13.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.585099+0000", 2024-09-14T09:07:13.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019697599999999999 2024-09-14T09:07:13.349 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.349 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.349 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.349 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:13.349 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:13.349 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:13.350 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:07:13.351 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:07:13.352 INFO:teuthology.orchestra.run.smithi176.stdout:4991 read 0x2807b5a thru 0x28169b2 (0xee59 bytes) 2024-09-14T09:07:13.352 INFO:teuthology.orchestra.run.smithi176.stdout:4992 trunc from 0x4156f57 to 0xd30abdf 2024-09-14T09:07:13.352 INFO:teuthology.orchestra.run.smithi176.stdout:4994 writesame 0x8013d6a thru 0x80163d6 data_size 0x3(0x266d bytes) 2024-09-14T09:07:13.352 INFO:teuthology.orchestra.run.smithi176.stdout:4995 read 0x42ebc0d thru 0x42f9ead (0xe2a1 bytes) 2024-09-14T09:07:13.352 INFO:teuthology.orchestra.run.smithi176.stdout:4996 read 0xc39ce43 thru 0xc39e45b (0x1619 bytes) 2024-09-14T09:07:13.352 INFO:teuthology.orchestra.run.smithi176.stdout:4998 write 0xad3047d thru 0xad3d2d7 (0xce5b bytes) 2024-09-14T09:07:13.352 INFO:teuthology.orchestra.run.smithi176.stdout:4999 punch from 0x2b59ca0 to 0x2b5f81b, (0x5b7b bytes) 2024-09-14T09:07:13.352 INFO:teuthology.orchestra.run.smithi176.stdout:5000 trunc from 0xd30abdf to 0xdfc40a1 2024-09-14T09:07:13.353 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:13.351+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:13.363 INFO:teuthology.orchestra.run.smithi176.stdout:5001 read 0x75a5b1b thru 0x75ae429 (0x890f bytes) 2024-09-14T09:07:13.363 INFO:teuthology.orchestra.run.smithi176.stdout:5002 read 0x13817f8 thru 0x1384783 (0x2f8c bytes) 2024-09-14T09:07:13.363 INFO:teuthology.orchestra.run.smithi176.stdout:5003 punch from 0xaa51da2 to 0xaa5bf4e, (0xa1ac bytes) 2024-09-14T09:07:13.387 INFO:teuthology.orchestra.run.smithi176.stdout:5006 trunc from 0xdfc40a1 to 0x528efeb 2024-09-14T09:07:13.407 INFO:teuthology.orchestra.run.smithi176.stdout:5008 read 0x92008 thru 0x9412f (0x2128 bytes) 2024-09-14T09:07:13.409 INFO:teuthology.orchestra.run.smithi176.stdout:5009 read 0x437e51b thru 0x4384231 (0x5d17 bytes) 2024-09-14T09:07:13.414 INFO:teuthology.orchestra.run.smithi176.stdout:5011 trunc from 0x528efeb to 0x75807c7 2024-09-14T09:07:13.424 INFO:teuthology.orchestra.run.smithi176.stdout:5016 writesame 0x1e307ae thru 0x1e3ed1a data_size 0x1(0xe56d bytes) 2024-09-14T09:07:13.451 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:13.455+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:13.506 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:13.506 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:13.523 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:13.523 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:13.527 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:13.527 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:13.527 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:07:13.527 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:13.541 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:07:13.543 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:13.544 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:13.552 INFO:teuthology.orchestra.run.smithi176.stdout:5017 write 0xcab11ab thru 0xcab5789 (0x45df bytes) 2024-09-14T09:07:13.558 INFO:teuthology.orchestra.run.smithi176.stdout:5018 trunc from 0xcab578a to 0xb5c6a77 2024-09-14T09:07:13.561 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:13.562 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:13.570 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:13.570 INFO:teuthology.orchestra.run.smithi176.stdout:5021 write 0x8e06e86 thru 0x8e072c3 (0x43e bytes) 2024-09-14T09:07:13.574 INFO:teuthology.orchestra.run.smithi176.stdout:5022 write 0x2deb9af thru 0x2df3278 (0x78ca bytes) 2024-09-14T09:07:13.580 INFO:teuthology.orchestra.run.smithi176.stdout:5023 write 0x978586d thru 0x9791aa5 (0xc239 bytes) 2024-09-14T09:07:13.580 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:07:13.581 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:07:13.583 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:13.587+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:13.585 INFO:teuthology.orchestra.run.smithi176.stdout:5024 punch from 0x8ad868d to 0x8adbd27, (0x369a bytes) 2024-09-14T09:07:13.585 INFO:teuthology.orchestra.run.smithi176.stdout:5025 write 0x2a97701 thru 0x2a99ed3 (0x27d3 bytes) 2024-09-14T09:07:13.588 INFO:teuthology.orchestra.run.smithi176.stdout:5027 clone 35 order 19 su 16384 sc 2 2024-09-14T09:07:13.601 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:13.601 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:13.620 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:13.620 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:13.638 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:13.638 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:13.653 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:13.651+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:13.703 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:13.703 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:13.718 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:07:13.754 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:13.755+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:13.838 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:13.838 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:13.852 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:07:13.854 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:13.859+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:13.955 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:13.955+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:13.976 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:13.976 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:13.992 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:07:14.031 INFO:teuthology.orchestra.run.smithi176.stdout:leaving image image_client.0-clone34 intact 2024-09-14T09:07:14.055 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:14.059+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:14.119 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:14.119 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:14.133 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:07:14.156 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:14.155+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:14.257 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:14.255+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:14.302 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:14.302 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:14.307 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-14T09:07:14.311 INFO:teuthology.orchestra.run.smithi176.stdout:5029 punch from 0x14e46a2 to 0x14f2cec, (0xe64a bytes) 2024-09-14T09:07:14.317 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:07:14.332 INFO:teuthology.orchestra.run.smithi176.stdout:5030 trunc from 0xb5c6a77 to 0x5b26433 2024-09-14T09:07:14.350 INFO:teuthology.orchestra.run.smithi176.stdout:5031 read 0x4cff851 thru 0x4d04c3e (0x53ee bytes) 2024-09-14T09:07:14.350 INFO:teuthology.orchestra.run.smithi176.stdout:5032 writesame 0x2e78313 thru 0x2e829de data_size 0x7a(0xa6cc bytes) 2024-09-14T09:07:14.358 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:14.359+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:14.365 INFO:teuthology.orchestra.run.smithi176.stdout:5033 read 0x56a2b4e thru 0x56aebe3 (0xc096 bytes) 2024-09-14T09:07:14.379 INFO:teuthology.orchestra.run.smithi176.stdout:5034 trunc from 0x5b26433 to 0xb0b66e1 2024-09-14T09:07:14.392 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:14.392 INFO:teuthology.orchestra.run.smithi176.stdout:5036 punch from 0x76cbb3d to 0x76d798b, (0xbe4e bytes) 2024-09-14T09:07:14.392 INFO:teuthology.orchestra.run.smithi176.stdout:5037 writesame 0x9df14d1 thru 0x9dfd0af data_size 0x1(0xbbdf bytes) 2024-09-14T09:07:14.459 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:14.459+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:14.469 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:14.469 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:14.476 INFO:teuthology.orchestra.run.smithi176.stdout:5040 read 0xabe258a thru 0xabee818 (0xc28f bytes) 2024-09-14T09:07:14.484 INFO:teuthology.orchestra.run.smithi176.stdout:5042 trunc from 0xb0b66e1 to 0xdab90fc 2024-09-14T09:07:14.485 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:07:14.497 INFO:teuthology.orchestra.run.smithi176.stdout:5044 writesame 0x7a76e50 thru 0x7a83079 data_size 0x1d(0xc22a bytes) 2024-09-14T09:07:14.509 INFO:teuthology.orchestra.run.smithi176.stdout:5045 read 0x5665a21 thru 0x566a3d7 (0x49b7 bytes) 2024-09-14T09:07:14.509 INFO:teuthology.orchestra.run.smithi176.stdout:5046 read 0xd8611f1 thru 0xd86c450 (0xb260 bytes) 2024-09-14T09:07:14.509 INFO:teuthology.orchestra.run.smithi176.stdout:5047 punch from 0xd26a6ef to 0xd276128, (0xba39 bytes) 2024-09-14T09:07:14.509 INFO:teuthology.orchestra.run.smithi176.stdout:5048 punch from 0x9258b6d to 0x925fea4, (0x7337 bytes) 2024-09-14T09:07:14.509 INFO:teuthology.orchestra.run.smithi176.stdout:5049 write 0xb92c38c thru 0xb92faa3 (0x3718 bytes) 2024-09-14T09:07:14.516 INFO:teuthology.orchestra.run.smithi176.stdout:5050 read 0xb7ed9fc thru 0xb7fc780 (0xed85 bytes) 2024-09-14T09:07:14.517 INFO:teuthology.orchestra.run.smithi176.stdout:5051 read 0x33ec087 thru 0x33f2f80 (0x6efa bytes) 2024-09-14T09:07:14.517 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:14.517 INFO:teuthology.orchestra.run.smithi176.stdout:5053 read 0x4686860 thru 0x4687992 (0x1133 bytes) 2024-09-14T09:07:14.523 INFO:teuthology.orchestra.run.smithi176.stdout:5056 trunc from 0xdab90fc to 0xb5df5ca 2024-09-14T09:07:14.550 INFO:teuthology.orchestra.run.smithi176.stdout:5057 punch from 0x181aaf2 to 0x1828ab4, (0xdfc2 bytes) 2024-09-14T09:07:14.550 INFO:teuthology.orchestra.run.smithi176.stdout:5058 punch from 0x9834433 to 0x9842b87, (0xe754 bytes) 2024-09-14T09:07:14.551 INFO:teuthology.orchestra.run.smithi176.stdout:5059 read 0x50f0081 thru 0x50f18f2 (0x1872 bytes) 2024-09-14T09:07:14.551 INFO:teuthology.orchestra.run.smithi176.stdout:5060 punch from 0x8c644ed to 0x8c68293, (0x3da6 bytes) 2024-09-14T09:07:14.551 INFO:teuthology.orchestra.run.smithi176.stdout:5061 punch from 0x871430e to 0x87172db, (0x2fcd bytes) 2024-09-14T09:07:14.560 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:14.559+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:14.572 INFO:teuthology.orchestra.run.smithi176.stdout:5062 write 0xde8afa6 thru 0xde8f6f7 (0x4752 bytes) 2024-09-14T09:07:14.582 INFO:teuthology.orchestra.run.smithi176.stdout:5063 punch from 0x27a649 to 0x27b74c, (0x1103 bytes) 2024-09-14T09:07:14.582 INFO:teuthology.orchestra.run.smithi176.stdout:5064 write 0x8d0e744 thru 0x8d12f1a (0x47d7 bytes) 2024-09-14T09:07:14.607 INFO:teuthology.orchestra.run.smithi176.stdout:5065 trunc from 0xde8f6f8 to 0x878641f 2024-09-14T09:07:14.609 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:14.609 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:14.624 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:07:14.632 INFO:teuthology.orchestra.run.smithi176.stdout:5066 write 0x6899ee0 thru 0x68a0d64 (0x6e85 bytes) 2024-09-14T09:07:14.641 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:14.641 INFO:teuthology.orchestra.run.smithi176.stdout:5068 punch from 0x4a5b13a to 0x4a6acc3, (0xfb89 bytes) 2024-09-14T09:07:14.641 INFO:teuthology.orchestra.run.smithi176.stdout:5070 punch from 0x77d44a4 to 0x77da0a0, (0x5bfc bytes) 2024-09-14T09:07:14.641 INFO:teuthology.orchestra.run.smithi176.stdout:5072 punch from 0x1145aaf to 0x114f78f, (0x9ce0 bytes) 2024-09-14T09:07:14.641 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:14.641 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:14.658 INFO:teuthology.orchestra.run.smithi176.stdout:5075 write 0xa6344f7 thru 0xa6364ca (0x1fd4 bytes) 2024-09-14T09:07:14.671 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:14.675+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:14.688 INFO:teuthology.orchestra.run.smithi176.stdout:5076 write 0xe153096 thru 0xe1603f9 (0xd364 bytes) 2024-09-14T09:07:14.698 INFO:teuthology.orchestra.run.smithi176.stdout:5078 punch from 0x18dd403 to 0x18eaed4, (0xdad1 bytes) 2024-09-14T09:07:14.698 INFO:teuthology.orchestra.run.smithi176.stdout:5079 write 0xd01b4f1 thru 0xd01dba8 (0x26b8 bytes) 2024-09-14T09:07:14.707 INFO:teuthology.orchestra.run.smithi176.stdout:5081 read 0xe01dd47 thru 0xe02c913 (0xebcd bytes) 2024-09-14T09:07:14.712 INFO:teuthology.orchestra.run.smithi176.stdout:5082 trunc from 0xe1603fa to 0xc73cb00 2024-09-14T09:07:14.738 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:14.738 INFO:teuthology.orchestra.run.smithi176.stdout:5086 writesame 0xb40023d thru 0xb40d877 data_size 0x65(0xd63b bytes) 2024-09-14T09:07:14.749 INFO:teuthology.orchestra.run.smithi176.stdout:5087 write 0x54476d5 thru 0x5455734 (0xe060 bytes) 2024-09-14T09:07:14.749 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:14.749 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:14.762 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:14.763+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:14.763 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:14.763 INFO:teuthology.orchestra.run.smithi176.stdout:5089 read 0x76da5c5 thru 0x76dcd2c (0x2768 bytes) 2024-09-14T09:07:14.763 INFO:teuthology.orchestra.run.smithi176.stdout:5093 write 0x124da71 thru 0x124eceb (0x127b bytes) 2024-09-14T09:07:14.765 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:07:14.770 INFO:teuthology.orchestra.run.smithi176.stdout:5094 write 0xa0cb0a thru 0xa1944b (0xc942 bytes) 2024-09-14T09:07:14.792 INFO:teuthology.orchestra.run.smithi176.stdout:5095 read 0xbb545be thru 0xbb5b5af (0x6ff2 bytes) 2024-09-14T09:07:14.792 INFO:teuthology.orchestra.run.smithi176.stdout:5096 write 0x9903264 thru 0x9906e90 (0x3c2d bytes) 2024-09-14T09:07:14.803 INFO:teuthology.orchestra.run.smithi176.stdout:5098 trunc from 0xc73cb00 to 0xb44fe69 2024-09-14T09:07:14.837 INFO:teuthology.orchestra.run.smithi176.stdout:5100 trunc from 0xb44fe69 to 0x12d163b 2024-09-14T09:07:14.865 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:14.863+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:14.874 INFO:teuthology.orchestra.run.smithi176.stdout:5101 trunc from 0x12d163b to 0x66ba745 2024-09-14T09:07:14.890 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T09:07:14.898 INFO:teuthology.orchestra.run.smithi176.stdout:5102 write 0xa3504cd thru 0xa35e660 (0xe194 bytes) 2024-09-14T09:07:14.904 INFO:teuthology.orchestra.run.smithi176.stdout:5103 write 0x4fad752 thru 0x4fb51fb (0x7aaa bytes) 2024-09-14T09:07:14.911 INFO:teuthology.orchestra.run.smithi176.stdout:5104 read 0x84e6ffc thru 0x84f2b87 (0xbb8c bytes) 2024-09-14T09:07:14.911 INFO:teuthology.orchestra.run.smithi176.stdout:5105 writesame 0x575ada1 thru 0x575b21e data_size 0x17(0x47e bytes) 2024-09-14T09:07:14.915 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:14.916 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:14.916 INFO:teuthology.orchestra.run.smithi176.stdout:5106 read 0x62ca5a2 thru 0x62cca48 (0x24a7 bytes) 2024-09-14T09:07:14.916 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:14.930 INFO:teuthology.orchestra.run.smithi176.stdout:5108 write 0xe318582 thru 0xe31c111 (0x3b90 bytes) 2024-09-14T09:07:14.936 INFO:teuthology.orchestra.run.smithi176.stdout:5109 writesame 0x75a4c48 thru 0x75aa2ec data_size 0x1(0x56a5 bytes) 2024-09-14T09:07:14.937 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:07:14.966 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:14.967+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:14.983 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:15.005 INFO:teuthology.orchestra.run.smithi176.stdout:5111 trunc from 0xe31c112 to 0xf1cd25 2024-09-14T09:07:15.064 INFO:teuthology.orchestra.run.smithi176.stdout:5112 writesame 0xde097c9 thru 0xde0d861 data_size 0x1(0x4099 bytes) 2024-09-14T09:07:15.069 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:15.067+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:15.069 INFO:teuthology.orchestra.run.smithi176.stdout:5116 trunc from 0xde0d862 to 0xa9e0c34 2024-09-14T09:07:15.077 INFO:teuthology.orchestra.run.smithi176.stdout:5117 write 0x1830cc7 thru 0x18363bf (0x56f9 bytes) 2024-09-14T09:07:15.089 INFO:teuthology.orchestra.run.smithi176.stdout:5118 trunc from 0xa9e0c34 to 0x7897ec9 2024-09-14T09:07:15.104 INFO:teuthology.orchestra.run.smithi176.stdout:5119 trunc from 0x7897ec9 to 0x2439097 2024-09-14T09:07:15.111 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:15.132 INFO:teuthology.orchestra.run.smithi176.stdout:5122 writesame 0xb8296b9 thru 0xb82da4d data_size 0x3(0x4395 bytes) 2024-09-14T09:07:15.132 INFO:teuthology.orchestra.run.smithi176.stdout:5127 punch from 0x510db21 to 0x5112964, (0x4e43 bytes) 2024-09-14T09:07:15.132 INFO:teuthology.orchestra.run.smithi176.stdout:5128 punch from 0x8b0ea8f to 0x8b13db0, (0x5321 bytes) 2024-09-14T09:07:15.132 INFO:teuthology.orchestra.run.smithi176.stdout:5130 write 0x6ef5773 thru 0x6efbba6 (0x6434 bytes) 2024-09-14T09:07:15.136 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:15.136 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:15.139 INFO:teuthology.orchestra.run.smithi176.stdout:5132 read 0xb5475c9 thru 0xb55740c (0xfe44 bytes) 2024-09-14T09:07:15.139 INFO:teuthology.orchestra.run.smithi176.stdout:5133 write 0x424166 thru 0x4257ba (0x1655 bytes) 2024-09-14T09:07:15.154 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:15.159 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6] out_osds: [7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:07:15.159 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:07:15.159 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-14T09:07:15.159 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T09:07:15.159 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T09:07:15.159 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-14T09:07:15.161 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:07:15.164 INFO:teuthology.orchestra.run.smithi176.stdout:5135 trunc from 0xb82da4e to 0x54c0835 2024-09-14T09:07:15.192 INFO:teuthology.orchestra.run.smithi176.stdout:5136 punch from 0xbd81d5 to 0xbda3a4, (0x21cf bytes) 2024-09-14T09:07:15.192 INFO:teuthology.orchestra.run.smithi176.stdout:5137 read 0x445d3ef thru 0x446bdde (0xe9f0 bytes) 2024-09-14T09:07:15.192 INFO:teuthology.orchestra.run.smithi176.stdout:5138 punch from 0x443e227 to 0x443e54b, (0x324 bytes) 2024-09-14T09:07:15.192 INFO:teuthology.orchestra.run.smithi176.stdout:5139 read 0x1079195 thru 0x107c298 (0x3104 bytes) 2024-09-14T09:07:15.193 INFO:teuthology.orchestra.run.smithi176.stdout:5141 trunc from 0x54c0835 to 0x640a97f 2024-09-14T09:07:15.200 INFO:teuthology.orchestra.run.smithi176.stdout:5142 punch from 0x2cfc51c to 0x2d0b041, (0xeb25 bytes) 2024-09-14T09:07:15.200 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:15.207 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:15.211+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:15.217 INFO:teuthology.orchestra.run.smithi176.stdout:5144 writesame 0xcee308e thru 0xcee51a2 data_size 0x3(0x2115 bytes) 2024-09-14T09:07:15.217 INFO:teuthology.orchestra.run.smithi176.stdout:5145 writesame 0x3604f08 thru 0x3605bfa data_size 0x11(0xcf3 bytes) 2024-09-14T09:07:15.220 INFO:teuthology.orchestra.run.smithi176.stdout:5148 read 0x1e98650 thru 0x1e9c7b0 (0x4161 bytes) 2024-09-14T09:07:15.220 INFO:teuthology.orchestra.run.smithi176.stdout:5150 punch from 0x394db64 to 0x395a714, (0xcbb0 bytes) 2024-09-14T09:07:15.220 INFO:teuthology.orchestra.run.smithi176.stdout:5152 read 0xba6ab58 thru 0xba6f3ef (0x4898 bytes) 2024-09-14T09:07:15.229 INFO:teuthology.orchestra.run.smithi176.stdout:5153 writesame 0xe4f60c3 thru 0xe4fe60a data_size 0x14(0x8548 bytes) 2024-09-14T09:07:15.237 INFO:teuthology.orchestra.run.smithi176.stdout:5154 trunc from 0xe4fe60b to 0x934bf49 2024-09-14T09:07:15.244 INFO:teuthology.orchestra.run.smithi176.stdout:5155 writesame 0x60d36a5 thru 0x60e2429 data_size 0x1(0xed85 bytes) 2024-09-14T09:07:15.270 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:15.271+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:15.290 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:15.290 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:15.301 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:15.301 INFO:teuthology.orchestra.run.smithi183.stdout: "success": "" 2024-09-14T09:07:15.301 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:15.307 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:07:15.329 INFO:teuthology.orchestra.run.smithi176.stdout:5156 read 0x20a4d5d thru 0x20b3b8f (0xee33 bytes) 2024-09-14T09:07:15.337 INFO:teuthology.orchestra.run.smithi176.stdout:5157 write 0x9387638 thru 0x9393009 (0xb9d2 bytes) 2024-09-14T09:07:15.343 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:15.352 INFO:teuthology.orchestra.run.smithi176.stdout:5159 trunc from 0x939300a to 0xe80b857 2024-09-14T09:07:15.359 INFO:teuthology.orchestra.run.smithi176.stdout:5160 write 0x1e35c69 thru 0x1e3f0f5 (0x948d bytes) 2024-09-14T09:07:15.365 INFO:teuthology.orchestra.run.smithi176.stdout:5162 punch from 0x844d221 to 0x8457f68, (0xad47 bytes) 2024-09-14T09:07:15.365 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:15.365 INFO:teuthology.orchestra.run.smithi176.stdout:5166 writesame 0x5b04aed thru 0x5b07686 data_size 0x1(0x2b9a bytes) 2024-09-14T09:07:15.371 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:15.375+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:15.415 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:15.415 INFO:teuthology.orchestra.run.smithi176.stdout:5168 write 0xbbf6521 thru 0xbc0125d (0xad3d bytes) 2024-09-14T09:07:15.431 INFO:teuthology.orchestra.run.smithi176.stdout:5169 punch from 0x8a74cb to 0x8af2a2, (0x7dd7 bytes) 2024-09-14T09:07:15.431 INFO:teuthology.orchestra.run.smithi176.stdout:5170 punch from 0xcf8ad7e to 0xcf99fe5, (0xf267 bytes) 2024-09-14T09:07:15.431 INFO:teuthology.orchestra.run.smithi176.stdout:5171 read 0x35f21e5 thru 0x3601334 (0xf150 bytes) 2024-09-14T09:07:15.431 INFO:teuthology.orchestra.run.smithi176.stdout:5172 write 0xe9417e thru 0xe95b2f (0x19b2 bytes) 2024-09-14T09:07:15.441 INFO:teuthology.orchestra.run.smithi176.stdout:5174 write 0xd23a29b thru 0xd247b06 (0xd86c bytes) 2024-09-14T09:07:15.453 INFO:teuthology.orchestra.run.smithi176.stdout:5175 read 0x5f249b2 thru 0x5f323ce (0xda1d bytes) 2024-09-14T09:07:15.453 INFO:teuthology.orchestra.run.smithi176.stdout:5176 punch from 0x55b38d9 to 0x55be3ed, (0xab14 bytes) 2024-09-14T09:07:15.454 INFO:teuthology.orchestra.run.smithi176.stdout:5177 write 0x66538a5 thru 0x6661c50 (0xe3ac bytes) 2024-09-14T09:07:15.462 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:15.462 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:15.467 INFO:teuthology.orchestra.run.smithi176.stdout:5178 read 0x214e52c thru 0x215cfb6 (0xea8b bytes) 2024-09-14T09:07:15.467 INFO:teuthology.orchestra.run.smithi176.stdout:5179 writesame 0x550bf47 thru 0x5514ccc data_size 0x5(0x8d86 bytes) 2024-09-14T09:07:15.472 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:15.471+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:15.479 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:07:15.479 INFO:teuthology.orchestra.run.smithi176.stdout:5180 write 0x1433dae thru 0x1434433 (0x686 bytes) 2024-09-14T09:07:15.509 INFO:teuthology.orchestra.run.smithi176.stdout:5181 trunc from 0xe80b857 to 0x8830436 2024-09-14T09:07:15.551 INFO:teuthology.orchestra.run.smithi176.stdout:5182 write 0xe6bdb81 thru 0xe6c0274 (0x26f4 bytes) 2024-09-14T09:07:15.560 INFO:teuthology.orchestra.run.smithi176.stdout:5183 write 0xb464837 thru 0xb46e460 (0x9c2a bytes) 2024-09-14T09:07:15.573 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:15.575+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:15.576 INFO:teuthology.orchestra.run.smithi176.stdout:5184 write 0xedecd8d thru 0xedef9e8 (0x2c5c bytes) 2024-09-14T09:07:15.581 INFO:teuthology.orchestra.run.smithi176.stdout:5186 writesame 0x560ae6f thru 0x560d315 data_size 0x1(0x24a7 bytes) 2024-09-14T09:07:15.592 INFO:teuthology.orchestra.run.smithi176.stdout:5187 writesame 0xd037b54 thru 0xd03d16e data_size 0x1(0x561b bytes) 2024-09-14T09:07:15.639 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:15.640 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:15.654 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:07:15.655 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:07:15.675 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:15.679+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:15.676 INFO:teuthology.orchestra.run.smithi176.stdout:5188 read 0x395a5c6 thru 0x395e130 (0x3b6b bytes) 2024-09-14T09:07:15.677 INFO:teuthology.orchestra.run.smithi176.stdout:5189 write 0x1191ea3 thru 0x1194a05 (0x2b63 bytes) 2024-09-14T09:07:15.685 INFO:teuthology.orchestra.run.smithi176.stdout:5191 write 0x2e1c383 thru 0x2e2a91b (0xe599 bytes) 2024-09-14T09:07:15.697 INFO:teuthology.orchestra.run.smithi176.stdout:5192 read 0x16d4940 thru 0x16ddd70 (0x9431 bytes) 2024-09-14T09:07:15.697 INFO:teuthology.orchestra.run.smithi176.stdout:5193 write 0x87ea01d thru 0x87f3d94 (0x9d78 bytes) 2024-09-14T09:07:15.705 INFO:teuthology.orchestra.run.smithi176.stdout:5194 punch from 0x7db5379 to 0x7db8355, (0x2fdc bytes) 2024-09-14T09:07:15.705 INFO:teuthology.orchestra.run.smithi176.stdout:5196 read 0xc01d66 thru 0xc0cecb (0xb166 bytes) 2024-09-14T09:07:15.707 INFO:teuthology.orchestra.run.smithi176.stdout:5197 read 0x55cb773 thru 0x55cda00 (0x228e bytes) 2024-09-14T09:07:15.729 INFO:teuthology.orchestra.run.smithi176.stdout:5198 trunc from 0xedef9e9 to 0x1680621 2024-09-14T09:07:15.755 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:15.756 INFO:teuthology.orchestra.run.smithi176.stdout:5200 clone 36 order 19 su 4096 sc 4 2024-09-14T09:07:15.776 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:15.779+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:15.809 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:15.809 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:15.825 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:07:15.837 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:15.838 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:15.854 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:15.855 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:15.871 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:15.871 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:15.877 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:15.875+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:15.887 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is set 2024-09-14T09:07:15.892 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:15.892 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:15.914 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:07:15.914 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:07:15.936 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:15.936 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:15.959 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:15.959 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:15.969 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:15.969 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:15.969 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:15.969 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:15.977 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:15.979+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:15.981 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:15.981 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:15.985 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:07:16.005 INFO:teuthology.orchestra.run.smithi176.stdout:leaving image image_client.0-clone35 intact 2024-09-14T09:07:16.078 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:16.079+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:16.121 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:16.121 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:16.122 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:16.122 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:16.122 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:16.137 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:07:16.174 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-14T09:07:16.218 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:16.219+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:16.281 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:16.283+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:16.313 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:16.314 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:16.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:16.314 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:16.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.314 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18383 3.9 3:9252f7c9:::rbd_data.115e3abd3b2c.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:07:16.314 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:07:16.314 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 93.155374494, 2024-09-14T09:07:16.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056433830999999997, 2024-09-14T09:07:16.314 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.314 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.314 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:16.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.315 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.315 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18383 2024-09-14T09:07:16.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.315 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:07:16.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:07:16.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158423+0000", 2024-09-14T09:07:16.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T09:07:16.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158429+0000", 2024-09-14T09:07:16.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0179999999999998e-06 2024-09-14T09:07:16.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158440+0000", 2024-09-14T09:07:16.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0769e-05 2024-09-14T09:07:16.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158445+0000", 2024-09-14T09:07:16.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5840000000000002e-06 2024-09-14T09:07:16.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158495+0000", 2024-09-14T09:07:16.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9129999999999999e-05 2024-09-14T09:07:16.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158598+0000", 2024-09-14T09:07:16.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102938 2024-09-14T09:07:16.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:16.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.161048+0000", 2024-09-14T09:07:16.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002450336 2024-09-14T09:07:16.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:16.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214605+0000", 2024-09-14T09:07:16.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053557526000000001 2024-09-14T09:07:16.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:16.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214634+0000", 2024-09-14T09:07:16.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8483000000000002e-05 2024-09-14T09:07:16.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:16.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214731+0000", 2024-09-14T09:07:16.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7026000000000002e-05 2024-09-14T09:07:16.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.320 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214856+0000", 2024-09-14T09:07:16.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012474799999999999 2024-09-14T09:07:16.320 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.320 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.320 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.320 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18383 3.9 e38/34 ondisk, result = 0)", 2024-09-14T09:07:16.321 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:07:16.321 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 93.145058012999996, 2024-09-14T09:07:16.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045880575999999999, 2024-09-14T09:07:16.321 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.321 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.321 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.321 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:07:16.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:07:16.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168739+0000", 2024-09-14T09:07:16.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3900000000000004e-07 2024-09-14T09:07:16.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168744+0000", 2024-09-14T09:07:16.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5929999999999999e-06 2024-09-14T09:07:16.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168754+0000", 2024-09-14T09:07:16.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0686999999999999e-05 2024-09-14T09:07:16.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168761+0000", 2024-09-14T09:07:16.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6340000000000003e-06 2024-09-14T09:07:16.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214578+0000", 2024-09-14T09:07:16.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045817443999999999 2024-09-14T09:07:16.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214596+0000", 2024-09-14T09:07:16.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7159e-05 2024-09-14T09:07:16.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214619+0000", 2024-09-14T09:07:16.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.332e-05 2024-09-14T09:07:16.325 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.325 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.325 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.325 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19152 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14159058~8940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:07:16.326 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:07:16.326 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 91.991145556999996, 2024-09-14T09:07:16.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12891994500000001, 2024-09-14T09:07:16.326 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.326 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.326 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:16.326 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.326 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.326 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.326 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19152 2024-09-14T09:07:16.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.326 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:07:16.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:07:16.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322653+0000", 2024-09-14T09:07:16.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3750000000000001e-06 2024-09-14T09:07:16.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329647+0000", 2024-09-14T09:07:16.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069941960000000003 2024-09-14T09:07:16.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329673+0000", 2024-09-14T09:07:16.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5760000000000001e-05 2024-09-14T09:07:16.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329684+0000", 2024-09-14T09:07:16.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0474e-05 2024-09-14T09:07:16.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329776+0000", 2024-09-14T09:07:16.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2467999999999996e-05 2024-09-14T09:07:16.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329987+0000", 2024-09-14T09:07:16.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021133 2024-09-14T09:07:16.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:16.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.331303+0000", 2024-09-14T09:07:16.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001315629 2024-09-14T09:07:16.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:16.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.411090+0000", 2024-09-14T09:07:16.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079787477999999995 2024-09-14T09:07:16.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:16.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451463+0000", 2024-09-14T09:07:16.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040372312 2024-09-14T09:07:16.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:16.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451523+0000", 2024-09-14T09:07:16.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0420000000000001e-05 2024-09-14T09:07:16.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451571+0000", 2024-09-14T09:07:16.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7503000000000002e-05 2024-09-14T09:07:16.332 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.332 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.332 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.333 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20926 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:16.333 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:16.333 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.641813557000006, 2024-09-14T09:07:16.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018572207, 2024-09-14T09:07:16.333 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.333 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.333 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.333 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.333 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.333 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.333 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20926 2024-09-14T09:07:16.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.334 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:16.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:16.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:16.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:07:16.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671990+0000", 2024-09-14T09:07:16.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5769999999999999e-06 2024-09-14T09:07:16.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672001+0000", 2024-09-14T09:07:16.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1293e-05 2024-09-14T09:07:16.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672008+0000", 2024-09-14T09:07:16.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9099999999999999e-06 2024-09-14T09:07:16.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689147+0000", 2024-09-14T09:07:16.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017139060000000001 2024-09-14T09:07:16.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689324+0000", 2024-09-14T09:07:16.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017640200000000001 2024-09-14T09:07:16.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690555+0000", 2024-09-14T09:07:16.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001231212 2024-09-14T09:07:16.338 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.338 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.338 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.338 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20927 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2621440~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:16.338 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:16.338 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.641776590999996, 2024-09-14T09:07:16.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020002227000000001, 2024-09-14T09:07:16.338 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.338 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.339 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.339 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.339 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.339 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.339 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20927 2024-09-14T09:07:16.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.339 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:16.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:16.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:16.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9699999999999995e-07 2024-09-14T09:07:16.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672024+0000", 2024-09-14T09:07:16.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:07:16.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672036+0000", 2024-09-14T09:07:16.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.119e-05 2024-09-14T09:07:16.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672044+0000", 2024-09-14T09:07:16.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8370000000000003e-06 2024-09-14T09:07:16.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690602+0000", 2024-09-14T09:07:16.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018557648 2024-09-14T09:07:16.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690805+0000", 2024-09-14T09:07:16.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020282899999999999 2024-09-14T09:07:16.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692022+0000", 2024-09-14T09:07:16.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012170429999999999 2024-09-14T09:07:16.343 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.343 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.343 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.343 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20928 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 917504~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:16.343 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:07:16.343 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.641740451000004, 2024-09-14T09:07:16.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021623738, 2024-09-14T09:07:16.343 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.344 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.344 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.344 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.344 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.344 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.344 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20928 2024-09-14T09:07:16.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.344 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:07:16.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:07:16.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672057+0000", 2024-09-14T09:07:16.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:07:16.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672059+0000", 2024-09-14T09:07:16.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9270000000000001e-06 2024-09-14T09:07:16.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672070+0000", 2024-09-14T09:07:16.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0773e-05 2024-09-14T09:07:16.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672077+0000", 2024-09-14T09:07:16.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9600000000000003e-06 2024-09-14T09:07:16.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692154+0000", 2024-09-14T09:07:16.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020076482 2024-09-14T09:07:16.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692419+0000", 2024-09-14T09:07:16.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026517400000000002 2024-09-14T09:07:16.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693680+0000", 2024-09-14T09:07:16.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012607390000000001 2024-09-14T09:07:16.348 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.348 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.348 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.348 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20929 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2752512~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:16.348 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:07:16.348 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.641707409000006, 2024-09-14T09:07:16.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022898728, 2024-09-14T09:07:16.348 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.349 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.349 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.349 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.349 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.349 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.349 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20929 2024-09-14T09:07:16.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.349 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:07:16.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:07:16.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672090+0000", 2024-09-14T09:07:16.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:07:16.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672092+0000", 2024-09-14T09:07:16.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8940000000000002e-06 2024-09-14T09:07:16.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672103+0000", 2024-09-14T09:07:16.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0815999999999999e-05 2024-09-14T09:07:16.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672110+0000", 2024-09-14T09:07:16.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9469999999999997e-06 2024-09-14T09:07:16.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693737+0000", 2024-09-14T09:07:16.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021626467999999999 2024-09-14T09:07:16.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693962+0000", 2024-09-14T09:07:16.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225474 2024-09-14T09:07:16.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.694988+0000", 2024-09-14T09:07:16.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001025469 2024-09-14T09:07:16.353 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.353 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.353 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.353 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20930 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:16.353 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:07:16.354 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.641674315000003, 2024-09-14T09:07:16.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024303286, 2024-09-14T09:07:16.354 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.354 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.354 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.354 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.354 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.354 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.354 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20930 2024-09-14T09:07:16.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.354 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:07:16.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:07:16.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672123+0000", 2024-09-14T09:07:16.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:07:16.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672125+0000", 2024-09-14T09:07:16.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7300000000000001e-06 2024-09-14T09:07:16.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672134+0000", 2024-09-14T09:07:16.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7930000000000003e-06 2024-09-14T09:07:16.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672141+0000", 2024-09-14T09:07:16.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9029999999999998e-06 2024-09-14T09:07:16.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695087+0000", 2024-09-14T09:07:16.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022946013000000001 2024-09-14T09:07:16.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695350+0000", 2024-09-14T09:07:16.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026266000000000001 2024-09-14T09:07:16.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696425+0000", 2024-09-14T09:07:16.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001075527 2024-09-14T09:07:16.358 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.358 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.358 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.359 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20931 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:16.359 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:07:16.359 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.641641086000007, 2024-09-14T09:07:16.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025331519, 2024-09-14T09:07:16.359 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.359 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.359 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.359 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.359 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.359 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.359 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20931 2024-09-14T09:07:16.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.360 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:07:16.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:07:16.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672156+0000", 2024-09-14T09:07:16.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4300000000000002e-07 2024-09-14T09:07:16.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672159+0000", 2024-09-14T09:07:16.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T09:07:16.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672170+0000", 2024-09-14T09:07:16.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1192999999999999e-05 2024-09-14T09:07:16.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672177+0000", 2024-09-14T09:07:16.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9059999999999996e-06 2024-09-14T09:07:16.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696532+0000", 2024-09-14T09:07:16.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024355291000000001 2024-09-14T09:07:16.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696755+0000", 2024-09-14T09:07:16.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022297699999999999 2024-09-14T09:07:16.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697487+0000", 2024-09-14T09:07:16.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00073171499999999997 2024-09-14T09:07:16.364 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.364 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.364 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.364 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20932 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1179648~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:16.364 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:16.364 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.641607643, 2024-09-14T09:07:16.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026104083, 2024-09-14T09:07:16.364 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.364 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.364 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.365 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.365 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.365 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.365 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20932 2024-09-14T09:07:16.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.365 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:16.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:16.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:16.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0699999999999996e-07 2024-09-14T09:07:16.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672192+0000", 2024-09-14T09:07:16.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7700000000000002e-06 2024-09-14T09:07:16.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672203+0000", 2024-09-14T09:07:16.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0956e-05 2024-09-14T09:07:16.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672211+0000", 2024-09-14T09:07:16.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3769999999999996e-06 2024-09-14T09:07:16.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697530+0000", 2024-09-14T09:07:16.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025318411999999998 2024-09-14T09:07:16.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697640+0000", 2024-09-14T09:07:16.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011045299999999999 2024-09-14T09:07:16.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698293+0000", 2024-09-14T09:07:16.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065240800000000005 2024-09-14T09:07:16.369 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.369 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.369 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.369 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20933 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3014656~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:16.369 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:07:16.369 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.641573440000002, 2024-09-14T09:07:16.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026558875999999999, 2024-09-14T09:07:16.370 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.370 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.370 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.370 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20933 2024-09-14T09:07:16.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.370 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:07:16.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:07:16.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672224+0000", 2024-09-14T09:07:16.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2399999999999997e-07 2024-09-14T09:07:16.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672234+0000", 2024-09-14T09:07:16.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0793e-05 2024-09-14T09:07:16.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672246+0000", 2024-09-14T09:07:16.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659e-05 2024-09-14T09:07:16.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672255+0000", 2024-09-14T09:07:16.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5730000000000008e-06 2024-09-14T09:07:16.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698326+0000", 2024-09-14T09:07:16.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026071414000000001 2024-09-14T09:07:16.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698374+0000", 2024-09-14T09:07:16.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8032999999999999e-05 2024-09-14T09:07:16.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698782+0000", 2024-09-14T09:07:16.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040768 2024-09-14T09:07:16.374 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.374 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.374 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.375 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20934 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1310720~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:16.375 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:07:16.375 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.641531564000005, 2024-09-14T09:07:16.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027292067999999999, 2024-09-14T09:07:16.375 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.375 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.375 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.375 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.375 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.375 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.375 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20934 2024-09-14T09:07:16.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.376 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:07:16.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:07:16.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672266+0000", 2024-09-14T09:07:16.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6299999999999993e-07 2024-09-14T09:07:16.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672269+0000", 2024-09-14T09:07:16.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8569999999999999e-06 2024-09-14T09:07:16.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672280+0000", 2024-09-14T09:07:16.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1236999999999999e-05 2024-09-14T09:07:16.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672287+0000", 2024-09-14T09:07:16.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8260000000000004e-06 2024-09-14T09:07:16.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698820+0000", 2024-09-14T09:07:16.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026533055999999999 2024-09-14T09:07:16.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698933+0000", 2024-09-14T09:07:16.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011291 2024-09-14T09:07:16.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699557+0000", 2024-09-14T09:07:16.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062421899999999997 2024-09-14T09:07:16.379 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.379 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.379 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.380 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20935 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:16.380 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:07:16.380 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.641497803999997, 2024-09-14T09:07:16.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028108610999999999, 2024-09-14T09:07:16.380 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.380 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.380 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.381 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20935 2024-09-14T09:07:16.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.381 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:07:16.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:07:16.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672299+0000", 2024-09-14T09:07:16.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T09:07:16.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672302+0000", 2024-09-14T09:07:16.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7640000000000001e-06 2024-09-14T09:07:16.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672313+0000", 2024-09-14T09:07:16.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1080000000000001e-05 2024-09-14T09:07:16.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672321+0000", 2024-09-14T09:07:16.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6490000000000008e-06 2024-09-14T09:07:16.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699587+0000", 2024-09-14T09:07:16.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027265884000000001 2024-09-14T09:07:16.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699631+0000", 2024-09-14T09:07:16.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4947999999999997e-05 2024-09-14T09:07:16.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700407+0000", 2024-09-14T09:07:16.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00077559299999999999 2024-09-14T09:07:16.384 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.385 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.385 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.385 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20936 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1441792~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:16.385 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:16.385 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.641463627999997, 2024-09-14T09:07:16.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028784750000000001, 2024-09-14T09:07:16.385 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.385 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.385 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.386 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.386 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.386 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.386 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20936 2024-09-14T09:07:16.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.386 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:16.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:16.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:16.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4700000000000001e-07 2024-09-14T09:07:16.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672336+0000", 2024-09-14T09:07:16.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6730000000000001e-06 2024-09-14T09:07:16.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672347+0000", 2024-09-14T09:07:16.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0710000000000001e-05 2024-09-14T09:07:16.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672354+0000", 2024-09-14T09:07:16.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4630000000000004e-06 2024-09-14T09:07:16.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700442+0000", 2024-09-14T09:07:16.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028087951 2024-09-14T09:07:16.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700557+0000", 2024-09-14T09:07:16.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011461800000000001 2024-09-14T09:07:16.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701117+0000", 2024-09-14T09:07:16.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056058799999999999 2024-09-14T09:07:16.390 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.390 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.390 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.391 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20937 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:16.391 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:16.391 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.641430502000006, 2024-09-14T09:07:16.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029493028000000001, 2024-09-14T09:07:16.391 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.391 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.391 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.391 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.391 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.391 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.392 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20937 2024-09-14T09:07:16.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.392 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:16.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:16.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:16.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T09:07:16.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672384+0000", 2024-09-14T09:07:16.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7127000000000001e-05 2024-09-14T09:07:16.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672395+0000", 2024-09-14T09:07:16.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759999999999999e-05 2024-09-14T09:07:16.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672402+0000", 2024-09-14T09:07:16.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0960000000000003e-06 2024-09-14T09:07:16.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701147+0000", 2024-09-14T09:07:16.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028744241 2024-09-14T09:07:16.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701246+0000", 2024-09-14T09:07:16.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9031999999999996e-05 2024-09-14T09:07:16.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701859+0000", 2024-09-14T09:07:16.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061310499999999997 2024-09-14T09:07:16.395 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.395 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.396 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.396 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20938 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:16.396 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:07:16.396 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.641383341999997, 2024-09-14T09:07:16.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030109219999999999, 2024-09-14T09:07:16.396 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.396 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.396 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.396 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.396 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.396 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.397 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20938 2024-09-14T09:07:16.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.397 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:07:16.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:07:16.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672414+0000", 2024-09-14T09:07:16.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-07 2024-09-14T09:07:16.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672417+0000", 2024-09-14T09:07:16.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:07:16.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672428+0000", 2024-09-14T09:07:16.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0873000000000001e-05 2024-09-14T09:07:16.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672435+0000", 2024-09-14T09:07:16.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:07:16.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701884+0000", 2024-09-14T09:07:16.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029449811999999999 2024-09-14T09:07:16.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701934+0000", 2024-09-14T09:07:16.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9703000000000002e-05 2024-09-14T09:07:16.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702522+0000", 2024-09-14T09:07:16.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00058811200000000001 2024-09-14T09:07:16.400 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.401 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.401 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.401 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20939 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:16.401 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:07:16.401 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.641351209000007, 2024-09-14T09:07:16.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030779259, 2024-09-14T09:07:16.401 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.401 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.401 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.401 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.402 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.402 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.402 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20939 2024-09-14T09:07:16.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.402 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:07:16.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:07:16.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672446+0000", 2024-09-14T09:07:16.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:07:16.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672449+0000", 2024-09-14T09:07:16.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7930000000000002e-06 2024-09-14T09:07:16.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672461+0000", 2024-09-14T09:07:16.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T09:07:16.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672471+0000", 2024-09-14T09:07:16.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0807e-05 2024-09-14T09:07:16.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702551+0000", 2024-09-14T09:07:16.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030079228 2024-09-14T09:07:16.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702664+0000", 2024-09-14T09:07:16.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011351800000000001 2024-09-14T09:07:16.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.703224+0000", 2024-09-14T09:07:16.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056026999999999997 2024-09-14T09:07:16.406 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.406 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.406 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.406 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:74298 3.9 3:95c8282a:::rbd_data.115ea8d131ff.0000000000000041:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:07:16.406 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:07:16.406 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 40.516041197, 2024-09-14T09:07:16.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042668064999999998, 2024-09-14T09:07:16.406 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.406 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.406 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:16.407 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.407 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.407 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.407 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74298 2024-09-14T09:07:16.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.407 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:07:16.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:07:16.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797758+0000", 2024-09-14T09:07:16.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9799999999999998e-06 2024-09-14T09:07:16.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797770+0000", 2024-09-14T09:07:16.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1667e-05 2024-09-14T09:07:16.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797797+0000", 2024-09-14T09:07:16.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7212999999999998e-05 2024-09-14T09:07:16.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797811+0000", 2024-09-14T09:07:16.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4256e-05 2024-09-14T09:07:16.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797882+0000", 2024-09-14T09:07:16.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0622000000000002e-05 2024-09-14T09:07:16.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797943+0000", 2024-09-14T09:07:16.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0893000000000001e-05 2024-09-14T09:07:16.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:16.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.799676+0000", 2024-09-14T09:07:16.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0017331619999999999 2024-09-14T09:07:16.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:16.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840109+0000", 2024-09-14T09:07:16.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040432994 2024-09-14T09:07:16.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:16.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840204+0000", 2024-09-14T09:07:16.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5247000000000005e-05 2024-09-14T09:07:16.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:16.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840346+0000", 2024-09-14T09:07:16.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014156499999999999 2024-09-14T09:07:16.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840423+0000", 2024-09-14T09:07:16.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7466000000000007e-05 2024-09-14T09:07:16.412 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.412 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.412 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.412 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:74298 3.9 e76/34 ondisk, result = 0)", 2024-09-14T09:07:16.412 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:07:16.413 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 40.509646554, 2024-09-14T09:07:16.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036289318000000001, 2024-09-14T09:07:16.413 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.413 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.413 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.413 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:07:16.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:07:16.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804151+0000", 2024-09-14T09:07:16.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6700000000000001e-06 2024-09-14T09:07:16.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804163+0000", 2024-09-14T09:07:16.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.148e-05 2024-09-14T09:07:16.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804197+0000", 2024-09-14T09:07:16.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3909999999999999e-05 2024-09-14T09:07:16.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804217+0000", 2024-09-14T09:07:16.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0287000000000001e-05 2024-09-14T09:07:16.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840176+0000", 2024-09-14T09:07:16.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035958641 2024-09-14T09:07:16.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840193+0000", 2024-09-14T09:07:16.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6986e-05 2024-09-14T09:07:16.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840439+0000", 2024-09-14T09:07:16.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024634400000000001 2024-09-14T09:07:16.416 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.417 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.417 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.417 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:93074 3.9 e94/84 3:925bb5bd:::rbd_data.115e7770ddc5.0000000000000001:head v 94'255, mlcod=94'252)", 2024-09-14T09:07:16.417 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:57.863727+0000", 2024-09-14T09:07:16.417 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.450069293999999, 2024-09-14T09:07:16.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019329799000000002, 2024-09-14T09:07:16.417 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.417 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.417 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:16.417 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.863727+0000", 2024-09-14T09:07:16.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.863727+0000", 2024-09-14T09:07:16.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.863729+0000", 2024-09-14T09:07:16.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3980000000000002e-06 2024-09-14T09:07:16.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866073+0000", 2024-09-14T09:07:16.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0023435109999999999 2024-09-14T09:07:16.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866094+0000", 2024-09-14T09:07:16.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1444e-05 2024-09-14T09:07:16.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866105+0000", 2024-09-14T09:07:16.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0949e-05 2024-09-14T09:07:16.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866216+0000", 2024-09-14T09:07:16.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110961 2024-09-14T09:07:16.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866258+0000", 2024-09-14T09:07:16.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1903000000000002e-05 2024-09-14T09:07:16.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:16.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.882981+0000", 2024-09-14T09:07:16.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016723198000000002 2024-09-14T09:07:16.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.883057+0000", 2024-09-14T09:07:16.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5434999999999999e-05 2024-09-14T09:07:16.421 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.421 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.421 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.421 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.421 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.421 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:16.423 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:07:16.424 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:16.427+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:16.524 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:16.523+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:16.585 INFO:teuthology.orchestra.run.smithi176.stdout:5201 writesame 0xb0087ec thru 0xb018452 data_size 0x1(0xfc67 bytes) 2024-09-14T09:07:16.585 INFO:teuthology.orchestra.run.smithi176.stdout:5202 write 0x52d63a8 thru 0x52e2ceb (0xc944 bytes) 2024-09-14T09:07:16.595 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:16.596 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:16.596 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:16.596 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:16.613 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:07:16.619 INFO:teuthology.orchestra.run.smithi176.stdout:5203 writesame 0xe36a82f thru 0xe36f53b data_size 0x5(0x4d0d bytes) 2024-09-14T09:07:16.620 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:16.620 INFO:teuthology.orchestra.run.smithi176.stdout:5206 read 0xbd249d3 thru 0xbd31738 (0xcd66 bytes) 2024-09-14T09:07:16.620 INFO:teuthology.orchestra.run.smithi176.stdout:5207 read 0x62a9e12 thru 0x62b97ee (0xf9dd bytes) 2024-09-14T09:07:16.620 INFO:teuthology.orchestra.run.smithi176.stdout:5208 write 0xd8f89d1 thru 0xd907a01 (0xf031 bytes) 2024-09-14T09:07:16.625 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:16.627+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:16.636 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:16.638 INFO:teuthology.orchestra.run.smithi176.stdout:5210 trunc from 0xe36f53c to 0xcd04f1c 2024-09-14T09:07:16.671 INFO:teuthology.orchestra.run.smithi176.stdout:5211 trunc from 0xcd04f1c to 0x72821af 2024-09-14T09:07:16.701 INFO:teuthology.orchestra.run.smithi176.stdout:5213 write 0xb340e00 thru 0xb347de3 (0x6fe4 bytes) 2024-09-14T09:07:16.718 INFO:teuthology.orchestra.run.smithi176.stdout:5214 write 0x2dff084 thru 0x2e0094e (0x18cb bytes) 2024-09-14T09:07:16.725 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:16.723+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:16.732 INFO:teuthology.orchestra.run.smithi176.stdout:5215 trunc from 0xb347de4 to 0xdb25f89 2024-09-14T09:07:16.745 INFO:teuthology.orchestra.run.smithi176.stdout:5216 write 0x9dedc01 thru 0x9df5985 (0x7d85 bytes) 2024-09-14T09:07:16.760 INFO:teuthology.orchestra.run.smithi176.stdout:5218 write 0x76426b thru 0x76869a (0x4430 bytes) 2024-09-14T09:07:16.764 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:16.764 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:16.764 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:16.764 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:16.764 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:16.786 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:07:16.830 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:16.831+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:16.928 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:16.931+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:16.957 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:16.957 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:16.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:16.957 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:16.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.957 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:16.958 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:16.958 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 103.87190359900001, 2024-09-14T09:07:16.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:07:16.958 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.958 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.958 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.958 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.958 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.958 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.958 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:07:16.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.958 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:16.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:16.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:07:16.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:07:16.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:07:16.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:07:16.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:07:16.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:07:16.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:07:16.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:07:16.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:07:16.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:07:16.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:16.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:07:16.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:07:16.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:07:16.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:07:16.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:07:16.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:07:16.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:07:16.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:07:16.962 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.962 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.963 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.963 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:16.963 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:16.963 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 103.871848941, 2024-09-14T09:07:16.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:07:16.963 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.963 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.963 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.963 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:07:16.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.964 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:16.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:16.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:16.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:07:16.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:07:16.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:07:16.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:07:16.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:07:16.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:07:16.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:07:16.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:07:16.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:07:16.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:16.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:07:16.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:07:16.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:07:16.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:07:16.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:07:16.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:07:16.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:07:16.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:07:16.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.968 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.968 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.968 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:16.968 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:16.968 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 103.871743567, 2024-09-14T09:07:16.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:07:16.968 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.968 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.968 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.969 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:07:16.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.969 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:16.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:16.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:07:16.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:07:16.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:07:16.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:07:16.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:07:16.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:07:16.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:07:16.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:07:16.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:07:16.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:07:16.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:16.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:07:16.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:07:16.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:07:16.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:07:16.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:07:16.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:07:16.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:07:16.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:07:16.972 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.973 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.973 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.973 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:16.973 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:16.973 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 103.87138680300001, 2024-09-14T09:07:16.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:07:16.973 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.973 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.973 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.973 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.973 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.973 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.974 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:07:16.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.974 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:16.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:16.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:07:16.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:07:16.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:07:16.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:07:16.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:07:16.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:07:16.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:07:16.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:07:16.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:07:16.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:07:16.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:16.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:07:16.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:07:16.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:07:16.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:07:16.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:07:16.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:07:16.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:07:16.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:07:16.978 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.978 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.978 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.978 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:16.978 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:16.978 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 103.87130206499999, 2024-09-14T09:07:16.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:07:16.978 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.978 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.978 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.979 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.979 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.979 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.979 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:07:16.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.979 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:16.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:16.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:07:16.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:07:16.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:07:16.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:07:16.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:07:16.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:07:16.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:07:16.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:07:16.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:07:16.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:07:16.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:16.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:07:16.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:07:16.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:07:16.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:07:16.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:07:16.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:07:16.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:07:16.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:07:16.983 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.983 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.983 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.983 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:16.983 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:16.983 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 103.87121437899999, 2024-09-14T09:07:16.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:07:16.983 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.983 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.983 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.983 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.984 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.984 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.984 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:07:16.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.984 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:16.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:16.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:07:16.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:07:16.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:07:16.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:07:16.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:07:16.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:07:16.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:07:16.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:07:16.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:07:16.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:07:16.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:16.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:07:16.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:07:16.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:07:16.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:07:16.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:07:16.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:07:16.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:07:16.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:07:16.988 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.988 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.988 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.988 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:07:16.988 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:16.988 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 70.794922392000004, 2024-09-14T09:07:16.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:07:16.989 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.989 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.989 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:16.989 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.989 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.989 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.989 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:07:16.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.989 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:16.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:16.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:07:16.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:07:16.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:07:16.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:07:16.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:07:16.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:07:16.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:07:16.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:07:16.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:07:16.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:07:16.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:07:16.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:07:16.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:16.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:07:16.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:07:16.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:16.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:07:16.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:07:16.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:16.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:07:16.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:07:16.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:16.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:07:16.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:07:16.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:07:16.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:07:16.994 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.994 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.994 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.995 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:39401 3.b e56/53)", 2024-09-14T09:07:16.995 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:07:16.995 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 70.769026326000002, 2024-09-14T09:07:16.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10443659600000001, 2024-09-14T09:07:16.995 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.995 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.995 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:16.995 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:16.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:07:16.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:16.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:07:16.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:16.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:16.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186509+0000", 2024-09-14T09:07:16.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2640000000000001e-06 2024-09-14T09:07:16.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:16.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186516+0000", 2024-09-14T09:07:16.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1070000000000001e-06 2024-09-14T09:07:16.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:16.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186533+0000", 2024-09-14T09:07:16.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7818000000000001e-05 2024-09-14T09:07:16.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:16.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186543+0000", 2024-09-14T09:07:16.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5170000000000005e-06 2024-09-14T09:07:16.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:16.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290885+0000", 2024-09-14T09:07:16.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.104342221 2024-09-14T09:07:16.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:16.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290893+0000", 2024-09-14T09:07:16.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2859999999999999e-06 2024-09-14T09:07:16.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:16.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290945+0000", 2024-09-14T09:07:16.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1382999999999999e-05 2024-09-14T09:07:16.998 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.998 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:16.998 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:16.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:16.999 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:84137 3.7 3:e15b49d9:::rbd_data.115e77b799c3.00000000000000e0:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:07:16.999 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:16.999 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.632174994, 2024-09-14T09:07:16.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11205707500000001, 2024-09-14T09:07:16.999 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:16.999 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:16.999 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:16.999 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:16.999 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:16.999 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:16.999 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 84137 2024-09-14T09:07:16.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:16.999 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:16.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:17.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:17.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:17.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:17.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:17.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323362+0000", 2024-09-14T09:07:17.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0839999999999999e-06 2024-09-14T09:07:17.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:17.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323370+0000", 2024-09-14T09:07:17.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0180000000000001e-06 2024-09-14T09:07:17.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:17.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323388+0000", 2024-09-14T09:07:17.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8326999999999999e-05 2024-09-14T09:07:17.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323397+0000", 2024-09-14T09:07:17.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1940000000000004e-06 2024-09-14T09:07:17.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:17.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323452+0000", 2024-09-14T09:07:17.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5043000000000001e-05 2024-09-14T09:07:17.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:17.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323550+0000", 2024-09-14T09:07:17.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8201000000000004e-05 2024-09-14T09:07:17.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:17.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.328273+0000", 2024-09-14T09:07:17.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0047227079999999999 2024-09-14T09:07:17.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:17.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435232+0000", 2024-09-14T09:07:17.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.106959339 2024-09-14T09:07:17.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:17.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435256+0000", 2024-09-14T09:07:17.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3774e-05 2024-09-14T09:07:17.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:17.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435314+0000", 2024-09-14T09:07:17.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7636999999999997e-05 2024-09-14T09:07:17.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:17.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435417+0000", 2024-09-14T09:07:17.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010275 2024-09-14T09:07:17.004 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.004 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:17.004 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.005 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:84137 3.7 e92/23 ondisk, result = 0)", 2024-09-14T09:07:17.005 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:53.341180+0000", 2024-09-14T09:07:17.005 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 23.614354410000001, 2024-09-14T09:07:17.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.094062506000000004, 2024-09-14T09:07:17.005 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:17.005 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:17.005 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:17.005 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:17.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:17.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341180+0000", 2024-09-14T09:07:17.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:17.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341180+0000", 2024-09-14T09:07:17.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:17.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341181+0000", 2024-09-14T09:07:17.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.192e-06 2024-09-14T09:07:17.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:17.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341187+0000", 2024-09-14T09:07:17.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0070000000000003e-06 2024-09-14T09:07:17.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:17.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341208+0000", 2024-09-14T09:07:17.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0321000000000001e-05 2024-09-14T09:07:17.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.341219+0000", 2024-09-14T09:07:17.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1764e-05 2024-09-14T09:07:17.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:17.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435214+0000", 2024-09-14T09:07:17.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.093994438999999999 2024-09-14T09:07:17.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:17.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435225+0000", 2024-09-14T09:07:17.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1123e-05 2024-09-14T09:07:17.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:17.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435243+0000", 2024-09-14T09:07:17.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.766e-05 2024-09-14T09:07:17.008 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.008 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:17.009 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.009 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:92675 3.8 3:105e0099:::rbd_data.115e7770ddc5.0000000000000016:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:17.009 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:57.736053+0000", 2024-09-14T09:07:17.009 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.219481165000001, 2024-09-14T09:07:17.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.073115555999999998, 2024-09-14T09:07:17.009 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:17.009 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:17.009 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:17.009 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:17.009 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:17.009 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.009 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 92675 2024-09-14T09:07:17.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.010 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:17.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:17.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736053+0000", 2024-09-14T09:07:17.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:17.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736053+0000", 2024-09-14T09:07:17.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:17.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736055+0000", 2024-09-14T09:07:17.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.33e-06 2024-09-14T09:07:17.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:17.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736062+0000", 2024-09-14T09:07:17.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.08e-06 2024-09-14T09:07:17.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:17.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736077+0000", 2024-09-14T09:07:17.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5605000000000001e-05 2024-09-14T09:07:17.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736088+0000", 2024-09-14T09:07:17.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0705e-05 2024-09-14T09:07:17.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:17.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736175+0000", 2024-09-14T09:07:17.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6926000000000001e-05 2024-09-14T09:07:17.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:17.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.736325+0000", 2024-09-14T09:07:17.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149739 2024-09-14T09:07:17.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:17.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.740035+0000", 2024-09-14T09:07:17.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0037099250000000002 2024-09-14T09:07:17.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:17.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809001+0000", 2024-09-14T09:07:17.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.068966240999999998 2024-09-14T09:07:17.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:17.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809049+0000", 2024-09-14T09:07:17.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8572999999999997e-05 2024-09-14T09:07:17.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:17.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809102+0000", 2024-09-14T09:07:17.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2564e-05 2024-09-14T09:07:17.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:17.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.809169+0000", 2024-09-14T09:07:17.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6867999999999997e-05 2024-09-14T09:07:17.014 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.014 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:17.014 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.014 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:93277 3.8 3:105e0099:::rbd_data.115e7770ddc5.0000000000000016:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:07:17.015 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.097058+0000", 2024-09-14T09:07:17.015 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.858476853999999, 2024-09-14T09:07:17.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.065618911000000002, 2024-09-14T09:07:17.015 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:17.015 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:17.015 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:17.015 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:17.015 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:17.015 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.015 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 93277 2024-09-14T09:07:17.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.015 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:17.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:17.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097058+0000", 2024-09-14T09:07:17.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:17.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097058+0000", 2024-09-14T09:07:17.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:17.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097060+0000", 2024-09-14T09:07:17.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.497e-06 2024-09-14T09:07:17.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:17.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097072+0000", 2024-09-14T09:07:17.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1600000000000001e-05 2024-09-14T09:07:17.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:17.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097102+0000", 2024-09-14T09:07:17.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0552999999999997e-05 2024-09-14T09:07:17.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097124+0000", 2024-09-14T09:07:17.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1477000000000001e-05 2024-09-14T09:07:17.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:17.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097201+0000", 2024-09-14T09:07:17.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7065999999999997e-05 2024-09-14T09:07:17.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:17.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.097277+0000", 2024-09-14T09:07:17.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5722000000000004e-05 2024-09-14T09:07:17.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:17.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.101895+0000", 2024-09-14T09:07:17.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004618212 2024-09-14T09:07:17.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:17.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162452+0000", 2024-09-14T09:07:17.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.060557603000000002 2024-09-14T09:07:17.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:17.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162504+0000", 2024-09-14T09:07:17.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.147e-05 2024-09-14T09:07:17.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:17.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162643+0000", 2024-09-14T09:07:17.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000139245 2024-09-14T09:07:17.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:17.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.162677+0000", 2024-09-14T09:07:17.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3466000000000001e-05 2024-09-14T09:07:17.020 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.020 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:17.020 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.020 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:95495 3.18 3:1d9f7d98:::rbd_data.115e7770ddc5.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:07:17.020 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.598437+0000", 2024-09-14T09:07:17.020 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.357097804999999, 2024-09-14T09:07:17.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.090910543999999996, 2024-09-14T09:07:17.020 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:17.020 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:17.021 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:17.021 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:17.021 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:17.021 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.021 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 95495 2024-09-14T09:07:17.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.021 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:17.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.021 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:17.021 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598437+0000", 2024-09-14T09:07:17.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.021 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:17.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598437+0000", 2024-09-14T09:07:17.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:17.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598438+0000", 2024-09-14T09:07:17.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1149999999999999e-06 2024-09-14T09:07:17.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:17.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598443+0000", 2024-09-14T09:07:17.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4589999999999997e-06 2024-09-14T09:07:17.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:17.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598455+0000", 2024-09-14T09:07:17.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1793e-05 2024-09-14T09:07:17.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598463+0000", 2024-09-14T09:07:17.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9759999999999995e-06 2024-09-14T09:07:17.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:17.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598523+0000", 2024-09-14T09:07:17.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9926000000000002e-05 2024-09-14T09:07:17.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:17.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.598581+0000", 2024-09-14T09:07:17.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8285000000000002e-05 2024-09-14T09:07:17.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:17.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.602809+0000", 2024-09-14T09:07:17.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0042279079999999998 2024-09-14T09:07:17.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:17.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689202+0000", 2024-09-14T09:07:17.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.086392847999999994 2024-09-14T09:07:17.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:17.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689230+0000", 2024-09-14T09:07:17.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8501999999999999e-05 2024-09-14T09:07:17.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:17.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689290+0000", 2024-09-14T09:07:17.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9524000000000001e-05 2024-09-14T09:07:17.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:17.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689347+0000", 2024-09-14T09:07:17.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7207999999999998e-05 2024-09-14T09:07:17.025 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.025 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:17.026 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.026 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:95495 3.18 e95/80 ondisk, result = 0)", 2024-09-14T09:07:17.026 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:58.610054+0000", 2024-09-14T09:07:17.026 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.345480340000002, 2024-09-14T09:07:17.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079165141999999994, 2024-09-14T09:07:17.026 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:17.026 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:17.026 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:17.026 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:17.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:17.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610054+0000", 2024-09-14T09:07:17.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:17.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610054+0000", 2024-09-14T09:07:17.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:17.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610055+0000", 2024-09-14T09:07:17.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2669999999999999e-06 2024-09-14T09:07:17.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:17.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610061+0000", 2024-09-14T09:07:17.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:07:17.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:17.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610078+0000", 2024-09-14T09:07:17.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6965000000000001e-05 2024-09-14T09:07:17.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.610087+0000", 2024-09-14T09:07:17.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.611e-06 2024-09-14T09:07:17.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:17.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689008+0000", 2024-09-14T09:07:17.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.078921186000000004 2024-09-14T09:07:17.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:17.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689020+0000", 2024-09-14T09:07:17.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.132e-05 2024-09-14T09:07:17.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:17.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:58.689219+0000", 2024-09-14T09:07:17.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000199469 2024-09-14T09:07:17.030 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.030 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:17.030 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.030 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:97796 3.15 3:aaaa5f7f:::rbd_data.115e113dc8ca.0000000000000029:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:07:17.030 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:17.030 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8610535219999997, 2024-09-14T09:07:17.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14537731300000001, 2024-09-14T09:07:17.030 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:17.030 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:17.030 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:17.030 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:17.030 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:17.031 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.031 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97796 2024-09-14T09:07:17.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.031 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:17.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:17.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:17.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:17.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:17.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:17.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094482+0000", 2024-09-14T09:07:17.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6400000000000001e-07 2024-09-14T09:07:17.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:17.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094485+0000", 2024-09-14T09:07:17.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.693e-06 2024-09-14T09:07:17.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:17.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094494+0000", 2024-09-14T09:07:17.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9590000000000001e-06 2024-09-14T09:07:17.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094502+0000", 2024-09-14T09:07:17.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3180000000000001e-06 2024-09-14T09:07:17.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:17.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094567+0000", 2024-09-14T09:07:17.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5526999999999998e-05 2024-09-14T09:07:17.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:17.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094726+0000", 2024-09-14T09:07:17.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015842499999999999 2024-09-14T09:07:17.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.034 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:07:17.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.101940+0000", 2024-09-14T09:07:17.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0072146559999999998 2024-09-14T09:07:17.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.034 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:17.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239665+0000", 2024-09-14T09:07:17.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.137724501 2024-09-14T09:07:17.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:17.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239726+0000", 2024-09-14T09:07:17.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1550999999999994e-05 2024-09-14T09:07:17.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:17.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239800+0000", 2024-09-14T09:07:17.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3318000000000005e-05 2024-09-14T09:07:17.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:17.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239858+0000", 2024-09-14T09:07:17.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8600999999999997e-05 2024-09-14T09:07:17.035 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.035 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:17.036 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.036 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:97796 3.15 e106/103 ondisk, result = 0)", 2024-09-14T09:07:17.036 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:17.036 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.8387076530000002, 2024-09-14T09:07:17.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123043465, 2024-09-14T09:07:17.036 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:17.036 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:17.036 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:17.036 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:17.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:17.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:17.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:17.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:17.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:17.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116828+0000", 2024-09-14T09:07:17.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8100000000000002e-07 2024-09-14T09:07:17.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:17.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116831+0000", 2024-09-14T09:07:17.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3009999999999998e-06 2024-09-14T09:07:17.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:17.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116841+0000", 2024-09-14T09:07:17.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7829999999999993e-06 2024-09-14T09:07:17.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116851+0000", 2024-09-14T09:07:17.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0135999999999999e-05 2024-09-14T09:07:17.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:17.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239707+0000", 2024-09-14T09:07:17.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12285581499999999 2024-09-14T09:07:17.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:17.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239718+0000", 2024-09-14T09:07:17.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1804000000000001e-05 2024-09-14T09:07:17.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:17.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239870+0000", 2024-09-14T09:07:17.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015194500000000001 2024-09-14T09:07:17.039 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.040 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:17.040 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.040 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:97940 3.7 3:e1066577:::rbd_data.115e113dc8ca.0000000000000049:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:07:17.040 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.598424+0000", 2024-09-14T09:07:17.040 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.3571109159999999, 2024-09-14T09:07:17.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.088410237000000003, 2024-09-14T09:07:17.040 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:17.040 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:17.040 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:17.040 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:17.041 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:17.041 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.041 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97940 2024-09-14T09:07:17.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.041 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:17.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.041 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:17.041 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.598424+0000", 2024-09-14T09:07:17.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.041 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:17.041 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.598424+0000", 2024-09-14T09:07:17.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:17.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.598424+0000", 2024-09-14T09:07:17.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4399999999999999e-07 2024-09-14T09:07:17.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:17.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.598428+0000", 2024-09-14T09:07:17.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8330000000000004e-06 2024-09-14T09:07:17.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:17.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.598438+0000", 2024-09-14T09:07:17.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5270000000000004e-06 2024-09-14T09:07:17.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.598445+0000", 2024-09-14T09:07:17.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3900000000000004e-06 2024-09-14T09:07:17.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:17.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.598540+0000", 2024-09-14T09:07:17.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5361999999999999e-05 2024-09-14T09:07:17.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:17.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.598772+0000", 2024-09-14T09:07:17.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000231096 2024-09-14T09:07:17.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:17.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.604987+0000", 2024-09-14T09:07:17.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0062157990000000002 2024-09-14T09:07:17.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:17.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.686715+0000", 2024-09-14T09:07:17.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.081727223000000002 2024-09-14T09:07:17.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:17.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.686734+0000", 2024-09-14T09:07:17.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9409999999999999e-05 2024-09-14T09:07:17.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:17.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.686787+0000", 2024-09-14T09:07:17.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3270999999999999e-05 2024-09-14T09:07:17.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:17.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.686834+0000", 2024-09-14T09:07:17.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6582000000000002e-05 2024-09-14T09:07:17.045 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.045 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:17.046 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.046 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:97940 3.7 e106/103 ondisk, result = 0)", 2024-09-14T09:07:17.046 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.618121+0000", 2024-09-14T09:07:17.046 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.3374132589999999, 2024-09-14T09:07:17.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.068602811999999999, 2024-09-14T09:07:17.046 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:17.046 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:17.046 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:17.046 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:17.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:17.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.618121+0000", 2024-09-14T09:07:17.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:17.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.618121+0000", 2024-09-14T09:07:17.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:17.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.618123+0000", 2024-09-14T09:07:17.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.657e-06 2024-09-14T09:07:17.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:17.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.618131+0000", 2024-09-14T09:07:17.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4100000000000008e-06 2024-09-14T09:07:17.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:17.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.618169+0000", 2024-09-14T09:07:17.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.7857e-05 2024-09-14T09:07:17.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.618190+0000", 2024-09-14T09:07:17.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.103e-05 2024-09-14T09:07:17.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:17.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.686697+0000", 2024-09-14T09:07:17.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.068507048000000001 2024-09-14T09:07:17.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:17.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.686708+0000", 2024-09-14T09:07:17.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0951000000000001e-05 2024-09-14T09:07:17.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:17.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.686724+0000", 2024-09-14T09:07:17.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5858999999999998e-05 2024-09-14T09:07:17.049 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.049 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:17.049 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.050 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:98161 3.15 3:ac845c17:::rbd_data.115e113dc8ca.0000000000000021:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-14T09:07:17.050 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:13.429237+0000", 2024-09-14T09:07:17.050 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.526297665, 2024-09-14T09:07:17.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.108985944, 2024-09-14T09:07:17.050 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:17.050 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:17.050 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:17.050 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:17.050 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:17.050 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.050 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 98161 2024-09-14T09:07:17.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.050 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:17.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:17.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.429237+0000", 2024-09-14T09:07:17.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:17.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.429237+0000", 2024-09-14T09:07:17.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:17.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.429238+0000", 2024-09-14T09:07:17.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.328e-06 2024-09-14T09:07:17.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:17.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.429245+0000", 2024-09-14T09:07:17.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4799999999999998e-06 2024-09-14T09:07:17.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:17.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.429261+0000", 2024-09-14T09:07:17.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6623e-05 2024-09-14T09:07:17.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.429270+0000", 2024-09-14T09:07:17.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9989999999999997e-06 2024-09-14T09:07:17.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:17.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.429332+0000", 2024-09-14T09:07:17.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1890000000000005e-05 2024-09-14T09:07:17.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:17.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.429449+0000", 2024-09-14T09:07:17.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011655 2024-09-14T09:07:17.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:07:17.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.436399+0000", 2024-09-14T09:07:17.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069505249999999999 2024-09-14T09:07:17.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:17.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.538131+0000", 2024-09-14T09:07:17.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.101731998 2024-09-14T09:07:17.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:17.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.538147+0000", 2024-09-14T09:07:17.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5449999999999999e-05 2024-09-14T09:07:17.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:17.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.538186+0000", 2024-09-14T09:07:17.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9280999999999999e-05 2024-09-14T09:07:17.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:17.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.538223+0000", 2024-09-14T09:07:17.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6820000000000003e-05 2024-09-14T09:07:17.055 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.055 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:17.055 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.055 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:98161 3.15 e107/103 ondisk, result = 0)", 2024-09-14T09:07:17.055 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:13.453737+0000", 2024-09-14T09:07:17.055 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.5017979390000002, 2024-09-14T09:07:17.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.084402934999999998, 2024-09-14T09:07:17.056 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:17.056 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:17.056 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:17.056 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:17.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:17.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.453737+0000", 2024-09-14T09:07:17.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:17.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.453737+0000", 2024-09-14T09:07:17.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:17.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:17.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.453738+0000", 2024-09-14T09:07:17.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.482e-06 2024-09-14T09:07:17.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:17.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.453744+0000", 2024-09-14T09:07:17.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0970000000000002e-06 2024-09-14T09:07:17.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:17.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.453767+0000", 2024-09-14T09:07:17.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2475000000000001e-05 2024-09-14T09:07:17.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.453778+0000", 2024-09-14T09:07:17.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1398e-05 2024-09-14T09:07:17.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:17.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.537977+0000", 2024-09-14T09:07:17.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.084198577999999996 2024-09-14T09:07:17.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:17.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.537984+0000", 2024-09-14T09:07:17.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3170000000000004e-06 2024-09-14T09:07:17.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:17.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:17.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:17.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.538139+0000", 2024-09-14T09:07:17.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015558799999999999 2024-09-14T09:07:17.059 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.059 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:17.059 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.059 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:17.059 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:17.059 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:17.061 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:07:17.061 INFO:teuthology.orchestra.run.smithi176.stdout:5219 writesame 0x87d5f53 thru 0x87d6510 data_size 0x7(0x5be bytes) 2024-09-14T09:07:17.061 INFO:teuthology.orchestra.run.smithi176.stdout:5220 trunc from 0xdb25f89 to 0xaefbc89 2024-09-14T09:07:17.062 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:17.063+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:17.080 INFO:teuthology.orchestra.run.smithi176.stdout:5221 write 0x87112e0 thru 0x87143e1 (0x3102 bytes) 2024-09-14T09:07:17.090 INFO:teuthology.orchestra.run.smithi176.stdout:5223 punch from 0x7f4e624 to 0x7f4efd6, (0x9b2 bytes) 2024-09-14T09:07:17.090 INFO:teuthology.orchestra.run.smithi176.stdout:5224 punch from 0xa6ccf17 to 0xa6cdb8a, (0xc73 bytes) 2024-09-14T09:07:17.090 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:17.091 INFO:teuthology.orchestra.run.smithi176.stdout:5227 read 0x63bf0c5 thru 0x63cd84c (0xe788 bytes) 2024-09-14T09:07:17.091 INFO:teuthology.orchestra.run.smithi176.stdout:5228 punch from 0x6e17566 to 0x6e22b5a, (0xb5f4 bytes) 2024-09-14T09:07:17.110 INFO:teuthology.orchestra.run.smithi176.stdout:5229 trunc from 0xaefbc89 to 0x3e2843a 2024-09-14T09:07:17.135 INFO:teuthology.orchestra.run.smithi176.stdout:5232 trunc from 0x3e2843a to 0x3e73031 2024-09-14T09:07:17.147 INFO:teuthology.orchestra.run.smithi176.stdout:5233 trunc from 0x3e73031 to 0x424c5c7 2024-09-14T09:07:17.159 INFO:teuthology.orchestra.run.smithi176.stdout:5234 punch from 0x1012485 to 0x1019c50, (0x77cb bytes) 2024-09-14T09:07:17.161 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:17.159+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:17.191 INFO:teuthology.orchestra.run.smithi176.stdout:5236 write 0x9dd2e0a thru 0x9ddaff4 (0x81eb bytes) 2024-09-14T09:07:17.205 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:17.205 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:17.206 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:07:17.206 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:17.206 INFO:teuthology.orchestra.run.smithi176.stdout:5237 write 0x9fe973 thru 0x9ffffc (0x168a bytes) 2024-09-14T09:07:17.223 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:07:17.246 INFO:teuthology.orchestra.run.smithi176.stdout:5238 write 0x9b9c537 thru 0x9ba9890 (0xd35a bytes) 2024-09-14T09:07:17.259 INFO:teuthology.orchestra.run.smithi176.stdout:5240 read 0x363d9ce thru 0x364d3f1 (0xfa24 bytes) 2024-09-14T09:07:17.260 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:17.260 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:17.260 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:17.260 INFO:teuthology.orchestra.run.smithi176.stdout:5247 read 0x580e865 thru 0x581e781 (0xff1d bytes) 2024-09-14T09:07:17.260 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:17.260 INFO:teuthology.orchestra.run.smithi176.stdout:5249 write 0x11c1798 thru 0x11c7d2e (0x6597 bytes) 2024-09-14T09:07:17.270 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:17.275+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:17.327 INFO:teuthology.orchestra.run.smithi176.stdout:5250 read 0x820ffe5 thru 0x821fb5b (0xfb77 bytes) 2024-09-14T09:07:17.327 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:17.336 INFO:teuthology.orchestra.run.smithi176.stdout:5252 trunc from 0x9ddaff5 to 0xe0c98c7 2024-09-14T09:07:17.346 INFO:teuthology.orchestra.run.smithi176.stdout:5254 read 0x9015850 thru 0x901be10 (0x65c1 bytes) 2024-09-14T09:07:17.346 INFO:teuthology.orchestra.run.smithi176.stdout:5257 read 0xb2b68b9 thru 0xb2bac2b (0x4373 bytes) 2024-09-14T09:07:17.346 INFO:teuthology.orchestra.run.smithi176.stdout:5259 read 0x2be49e7 thru 0x2be5d6c (0x1386 bytes) 2024-09-14T09:07:17.346 INFO:teuthology.orchestra.run.smithi176.stdout:5260 writesame 0x30e3bb6 thru 0x30ea5b8 data_size 0x1(0x6a03 bytes) 2024-09-14T09:07:17.351 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:17.351 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:17.351 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:07:17.352 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:07:17.352 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:17.362 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:17.363+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:17.371 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:07:17.392 INFO:teuthology.orchestra.run.smithi176.stdout:5261 write 0x7e2fe7e thru 0x7e38998 (0x8b1b bytes) 2024-09-14T09:07:17.411 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:17.411 INFO:teuthology.orchestra.run.smithi176.stdout:5263 writesame 0xbc8fe21 thru 0xbc91e77 data_size 0x1(0x2057 bytes) 2024-09-14T09:07:17.420 INFO:teuthology.orchestra.run.smithi176.stdout:5264 read 0x26195f7 thru 0x2620ecc (0x78d6 bytes) 2024-09-14T09:07:17.420 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:17.420 INFO:teuthology.orchestra.run.smithi176.stdout:5267 write 0xa6c1aff thru 0xa6c4514 (0x2a16 bytes) 2024-09-14T09:07:17.461 INFO:teuthology.orchestra.run.smithi176.stdout:5270 trunc from 0xe0c98c7 to 0x277f8c0 2024-09-14T09:07:17.463 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:17.463+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:17.493 INFO:teuthology.orchestra.run.smithi176.stdout:5272 punch from 0x18da1bf to 0x18e948f, (0xf2d0 bytes) 2024-09-14T09:07:17.517 INFO:teuthology.orchestra.run.smithi176.stdout:5273 write 0x5c93dcd thru 0x5c975b5 (0x37e9 bytes) 2024-09-14T09:07:17.532 INFO:teuthology.orchestra.run.smithi176.stdout:5275 punch from 0x2d5952e to 0x2d65589, (0xc05b bytes) 2024-09-14T09:07:17.532 INFO:teuthology.orchestra.run.smithi176.stdout:5276 read 0x50fd282 thru 0x50ffd76 (0x2af5 bytes) 2024-09-14T09:07:17.532 INFO:teuthology.orchestra.run.smithi176.stdout:5277 read 0x4c0cde thru 0x4c8c66 (0x7f89 bytes) 2024-09-14T09:07:17.537 INFO:teuthology.orchestra.run.smithi176.stdout:5278 punch from 0x2a1faaa to 0x2a23245, (0x379b bytes) 2024-09-14T09:07:17.537 INFO:teuthology.orchestra.run.smithi176.stdout:5279 punch from 0x4c43342 to 0x4c4ea59, (0xb717 bytes) 2024-09-14T09:07:17.540 INFO:teuthology.orchestra.run.smithi176.stdout:5280 trunc from 0x5c975b6 to 0x7552566 2024-09-14T09:07:17.552 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:17.553 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:07:17.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:07:17.553 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:07:17.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.553 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:9982 3.1a 3:5af3e476:::rbd_data.115eadec874.000000000000000c:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T09:07:17.553 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:07:17.553 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 99.956730413000002, 2024-09-14T09:07:17.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.042276341000000002, 2024-09-14T09:07:17.553 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.553 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.554 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:17.554 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:17.554 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:17.554 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.554 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 9982 2024-09-14T09:07:17.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.554 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:07:17.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597969+0000", 2024-09-14T09:07:17.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597970+0000", 2024-09-14T09:07:17.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0599999999999999e-07 2024-09-14T09:07:17.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597974+0000", 2024-09-14T09:07:17.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2649999999999998e-06 2024-09-14T09:07:17.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597986+0000", 2024-09-14T09:07:17.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1019000000000001e-05 2024-09-14T09:07:17.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.597991+0000", 2024-09-14T09:07:17.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5049999999999998e-06 2024-09-14T09:07:17.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.598083+0000", 2024-09-14T09:07:17.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2062e-05 2024-09-14T09:07:17.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.598227+0000", 2024-09-14T09:07:17.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000143609 2024-09-14T09:07:17.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:17.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.603107+0000", 2024-09-14T09:07:17.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048805819999999996 2024-09-14T09:07:17.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:17.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640002+0000", 2024-09-14T09:07:17.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.036894641999999998 2024-09-14T09:07:17.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:17.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640077+0000", 2024-09-14T09:07:17.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4927999999999998e-05 2024-09-14T09:07:17.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:17.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640174+0000", 2024-09-14T09:07:17.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7417999999999995e-05 2024-09-14T09:07:17.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:05:37.640246+0000", 2024-09-14T09:07:17.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1505000000000001e-05 2024-09-14T09:07:17.560 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.560 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.560 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.560 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:76803 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-14T09:07:17.560 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:43.858877+0000", 2024-09-14T09:07:17.560 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.695822982999999, 2024-09-14T09:07:17.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0137577900000001, 2024-09-14T09:07:17.560 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.560 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.560 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:17.560 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:17.560 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:17.561 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.561 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 76803 2024-09-14T09:07:17.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.561 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858877+0000", 2024-09-14T09:07:17.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858877+0000", 2024-09-14T09:07:17.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858879+0000", 2024-09-14T09:07:17.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.097e-06 2024-09-14T09:07:17.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858887+0000", 2024-09-14T09:07:17.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6510000000000001e-06 2024-09-14T09:07:17.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858903+0000", 2024-09-14T09:07:17.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6560999999999999e-05 2024-09-14T09:07:17.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858912+0000", 2024-09-14T09:07:17.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2019999999999993e-06 2024-09-14T09:07:17.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858964+0000", 2024-09-14T09:07:17.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2046999999999998e-05 2024-09-14T09:07:17.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:17.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858972+0000", 2024-09-14T09:07:17.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6029999999999999e-06 2024-09-14T09:07:17.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:44.871832+0000", 2024-09-14T09:07:17.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.012860109 2024-09-14T09:07:17.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:44.872248+0000", 2024-09-14T09:07:17.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00041549300000000001 2024-09-14T09:07:17.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:44.872635+0000", 2024-09-14T09:07:17.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038702700000000002 2024-09-14T09:07:17.565 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.565 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.565 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.565 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:80293 3.14 3:2933aeec:::rbd_data.115e2eebe49a.0000000000000004:1d [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-14T09:07:17.565 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:49.353426+0000", 2024-09-14T09:07:17.566 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 28.201273938, 2024-09-14T09:07:17.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.044484119000000003, 2024-09-14T09:07:17.566 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.566 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.566 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:17.566 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:17.566 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:17.566 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.566 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 80293 2024-09-14T09:07:17.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.566 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353426+0000", 2024-09-14T09:07:17.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353426+0000", 2024-09-14T09:07:17.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353427+0000", 2024-09-14T09:07:17.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8299999999999996e-07 2024-09-14T09:07:17.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353430+0000", 2024-09-14T09:07:17.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2739999999999999e-06 2024-09-14T09:07:17.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353439+0000", 2024-09-14T09:07:17.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8359999999999998e-06 2024-09-14T09:07:17.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353446+0000", 2024-09-14T09:07:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3599999999999998e-06 2024-09-14T09:07:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.389701+0000", 2024-09-14T09:07:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.036255250000000003 2024-09-14T09:07:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.390249+0000", 2024-09-14T09:07:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000547381 2024-09-14T09:07:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.397910+0000", 2024-09-14T09:07:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076612349999999997 2024-09-14T09:07:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:80322 3.14 3:2933aeec:::rbd_data.115e2eebe49a.0000000000000004:1d [read 16384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-14T09:07:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:49.353456+0000", 2024-09-14T09:07:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 28.201243530999999, 2024-09-14T09:07:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.045427997999999997, 2024-09-14T09:07:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 80322 2024-09-14T09:07:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353456+0000", 2024-09-14T09:07:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353456+0000", 2024-09-14T09:07:17.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353457+0000", 2024-09-14T09:07:17.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3300000000000001e-07 2024-09-14T09:07:17.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353460+0000", 2024-09-14T09:07:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0000000000000001e-06 2024-09-14T09:07:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353469+0000", 2024-09-14T09:07:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.636e-06 2024-09-14T09:07:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353477+0000", 2024-09-14T09:07:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0099999999999995e-06 2024-09-14T09:07:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.397975+0000", 2024-09-14T09:07:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.044498217999999999 2024-09-14T09:07:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.398131+0000", 2024-09-14T09:07:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015567099999999999 2024-09-14T09:07:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.398884+0000", 2024-09-14T09:07:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00075372999999999998 2024-09-14T09:07:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:80323 3.14 3:2933aeec:::rbd_data.115e2eebe49a.0000000000000004:1d [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-14T09:07:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:49.353490+0000", 2024-09-14T09:07:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 28.201209402, 2024-09-14T09:07:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.046828271999999997, 2024-09-14T09:07:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 80323 2024-09-14T09:07:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353490+0000", 2024-09-14T09:07:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353490+0000", 2024-09-14T09:07:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353491+0000", 2024-09-14T09:07:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5000000000000001e-07 2024-09-14T09:07:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353497+0000", 2024-09-14T09:07:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.947e-06 2024-09-14T09:07:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353513+0000", 2024-09-14T09:07:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5769000000000001e-05 2024-09-14T09:07:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353525+0000", 2024-09-14T09:07:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2094e-05 2024-09-14T09:07:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.398947+0000", 2024-09-14T09:07:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.045421802999999997 2024-09-14T09:07:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.399130+0000", 2024-09-14T09:07:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000182776 2024-09-14T09:07:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.400319+0000", 2024-09-14T09:07:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011889330000000001 2024-09-14T09:07:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:80324 3.14 3:2933aeec:::rbd_data.115e2eebe49a.0000000000000004:1d [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-14T09:07:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:49.353544+0000", 2024-09-14T09:07:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 28.201155432, 2024-09-14T09:07:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.047536104000000003, 2024-09-14T09:07:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 80324 2024-09-14T09:07:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353544+0000", 2024-09-14T09:07:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353544+0000", 2024-09-14T09:07:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353545+0000", 2024-09-14T09:07:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8599999999999997e-07 2024-09-14T09:07:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353549+0000", 2024-09-14T09:07:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7840000000000001e-06 2024-09-14T09:07:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353566+0000", 2024-09-14T09:07:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7212999999999999e-05 2024-09-14T09:07:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353579+0000", 2024-09-14T09:07:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2537e-05 2024-09-14T09:07:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.400380+0000", 2024-09-14T09:07:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.046800913 2024-09-14T09:07:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.400524+0000", 2024-09-14T09:07:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014461500000000001 2024-09-14T09:07:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.401081+0000", 2024-09-14T09:07:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055625599999999998 2024-09-14T09:07:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:80325 3.14 3:2933aeec:::rbd_data.115e2eebe49a.0000000000000004:1d [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-14T09:07:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:49.353599+0000", 2024-09-14T09:07:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 28.201100878999998, 2024-09-14T09:07:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.048432831000000003, 2024-09-14T09:07:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 80325 2024-09-14T09:07:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353599+0000", 2024-09-14T09:07:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353599+0000", 2024-09-14T09:07:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353600+0000", 2024-09-14T09:07:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.266e-06 2024-09-14T09:07:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353606+0000", 2024-09-14T09:07:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8900000000000004e-06 2024-09-14T09:07:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353623+0000", 2024-09-14T09:07:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6923e-05 2024-09-14T09:07:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353637+0000", 2024-09-14T09:07:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3943e-05 2024-09-14T09:07:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.401169+0000", 2024-09-14T09:07:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.047532079999999997 2024-09-14T09:07:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.401340+0000", 2024-09-14T09:07:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017054700000000001 2024-09-14T09:07:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.402032+0000", 2024-09-14T09:07:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069218199999999995 2024-09-14T09:07:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:80354 3.14 3:2933aeec:::rbd_data.115e2eebe49a.0000000000000004:1d [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-14T09:07:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:49.353658+0000", 2024-09-14T09:07:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 28.201042000000001, 2024-09-14T09:07:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.050321308000000002, 2024-09-14T09:07:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 80354 2024-09-14T09:07:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353658+0000", 2024-09-14T09:07:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353658+0000", 2024-09-14T09:07:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353659+0000", 2024-09-14T09:07:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0499999999999999e-06 2024-09-14T09:07:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353664+0000", 2024-09-14T09:07:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5330000000000002e-06 2024-09-14T09:07:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353682+0000", 2024-09-14T09:07:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-05 2024-09-14T09:07:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353696+0000", 2024-09-14T09:07:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3699999999999999e-05 2024-09-14T09:07:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.402990+0000", 2024-09-14T09:07:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.049294075 2024-09-14T09:07:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.403147+0000", 2024-09-14T09:07:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015756099999999999 2024-09-14T09:07:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.403979+0000", 2024-09-14T09:07:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00083194900000000001 2024-09-14T09:07:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:80355 3.14 3:2933aeec:::rbd_data.115e2eebe49a.0000000000000004:1d [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-14T09:07:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:49.353717+0000", 2024-09-14T09:07:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 28.200983164, 2024-09-14T09:07:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.051852537999999997, 2024-09-14T09:07:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 80355 2024-09-14T09:07:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353717+0000", 2024-09-14T09:07:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353717+0000", 2024-09-14T09:07:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353718+0000", 2024-09-14T09:07:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1769999999999999e-06 2024-09-14T09:07:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353725+0000", 2024-09-14T09:07:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0770000000000002e-06 2024-09-14T09:07:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353743+0000", 2024-09-14T09:07:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7943000000000001e-05 2024-09-14T09:07:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353753+0000", 2024-09-14T09:07:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0326000000000001e-05 2024-09-14T09:07:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.404040+0000", 2024-09-14T09:07:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.050286889000000001 2024-09-14T09:07:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.404614+0000", 2024-09-14T09:07:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00057370100000000003 2024-09-14T09:07:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.405569+0000", 2024-09-14T09:07:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00095542499999999998 2024-09-14T09:07:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:80356 3.14 3:2933aeec:::rbd_data.115e2eebe49a.0000000000000004:1d [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-14T09:07:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:49.353766+0000", 2024-09-14T09:07:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 28.200934203999999, 2024-09-14T09:07:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.052613749000000001, 2024-09-14T09:07:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 80356 2024-09-14T09:07:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353766+0000", 2024-09-14T09:07:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353766+0000", 2024-09-14T09:07:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353766+0000", 2024-09-14T09:07:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4300000000000002e-07 2024-09-14T09:07:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353770+0000", 2024-09-14T09:07:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6030000000000001e-06 2024-09-14T09:07:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353779+0000", 2024-09-14T09:07:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2070000000000001e-06 2024-09-14T09:07:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353786+0000", 2024-09-14T09:07:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.007e-06 2024-09-14T09:07:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.405698+0000", 2024-09-14T09:07:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.051911539 2024-09-14T09:07:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.405849+0000", 2024-09-14T09:07:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000151051 2024-09-14T09:07:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.406379+0000", 2024-09-14T09:07:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00053049900000000003 2024-09-14T09:07:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.602 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:80357 3.14 3:2933aeec:::rbd_data.115e2eebe49a.0000000000000004:1d [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-14T09:07:17.602 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:49.353797+0000", 2024-09-14T09:07:17.602 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 28.200902791000001, 2024-09-14T09:07:17.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.053313526, 2024-09-14T09:07:17.602 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.602 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.602 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:17.602 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:17.602 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:17.602 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.602 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 80357 2024-09-14T09:07:17.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.602 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353797+0000", 2024-09-14T09:07:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353797+0000", 2024-09-14T09:07:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353798+0000", 2024-09-14T09:07:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5999999999999991e-07 2024-09-14T09:07:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353803+0000", 2024-09-14T09:07:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0830000000000003e-06 2024-09-14T09:07:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353820+0000", 2024-09-14T09:07:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6816999999999998e-05 2024-09-14T09:07:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.353827+0000", 2024-09-14T09:07:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3130000000000001e-06 2024-09-14T09:07:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.406443+0000", 2024-09-14T09:07:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.052616164999999999 2024-09-14T09:07:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.406584+0000", 2024-09-14T09:07:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014090899999999999 2024-09-14T09:07:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:49.407111+0000", 2024-09-14T09:07:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00052627900000000005 2024-09-14T09:07:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:83981 3.3 3:c015e742:::rbd_data.115e77b799c3.0000000000000073:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:07:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:53.107172+0000", 2024-09-14T09:07:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.447527738000002, 2024-09-14T09:07:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.038857021999999998, 2024-09-14T09:07:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 83981 2024-09-14T09:07:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.107172+0000", 2024-09-14T09:07:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.107172+0000", 2024-09-14T09:07:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.107175+0000", 2024-09-14T09:07:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7829999999999999e-06 2024-09-14T09:07:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.107189+0000", 2024-09-14T09:07:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.366e-05 2024-09-14T09:07:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.107222+0000", 2024-09-14T09:07:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3232999999999998e-05 2024-09-14T09:07:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.107237+0000", 2024-09-14T09:07:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5489999999999999e-05 2024-09-14T09:07:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.107345+0000", 2024-09-14T09:07:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010822199999999999 2024-09-14T09:07:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.107566+0000", 2024-09-14T09:07:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022014199999999999 2024-09-14T09:07:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.109326+0000", 2024-09-14T09:07:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017599359999999999 2024-09-14T09:07:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.145587+0000", 2024-09-14T09:07:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.036261104000000002 2024-09-14T09:07:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.145733+0000", 2024-09-14T09:07:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000146355 2024-09-14T09:07:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.145905+0000", 2024-09-14T09:07:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017187800000000001 2024-09-14T09:07:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.146029+0000", 2024-09-14T09:07:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012421900000000001 2024-09-14T09:07:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:84403 3.14 3:2fc85a9a:::rbd_data.115e77b799c3.0000000000000018:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:07:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:53.878069+0000", 2024-09-14T09:07:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.676631113999999, 2024-09-14T09:07:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.128852405, 2024-09-14T09:07:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 84403 2024-09-14T09:07:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878069+0000", 2024-09-14T09:07:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878069+0000", 2024-09-14T09:07:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878071+0000", 2024-09-14T09:07:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0899999999999999e-06 2024-09-14T09:07:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878082+0000", 2024-09-14T09:07:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1226999999999999e-05 2024-09-14T09:07:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878108+0000", 2024-09-14T09:07:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5568999999999999e-05 2024-09-14T09:07:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878122+0000", 2024-09-14T09:07:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.488e-05 2024-09-14T09:07:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878247+0000", 2024-09-14T09:07:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000124039 2024-09-14T09:07:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878387+0000", 2024-09-14T09:07:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014041600000000001 2024-09-14T09:07:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.886261+0000", 2024-09-14T09:07:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0078738499999999999 2024-09-14T09:07:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006598+0000", 2024-09-14T09:07:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.120337316 2024-09-14T09:07:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006724+0000", 2024-09-14T09:07:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012601600000000001 2024-09-14T09:07:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006827+0000", 2024-09-14T09:07:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103299 2024-09-14T09:07:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006921+0000", 2024-09-14T09:07:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3702999999999994e-05 2024-09-14T09:07:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:84403 3.14 e92/84 ondisk, result = 0)", 2024-09-14T09:07:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:53.909978+0000", 2024-09-14T09:07:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.644721599, 2024-09-14T09:07:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.097025845999999999, 2024-09-14T09:07:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.909978+0000", 2024-09-14T09:07:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.909978+0000", 2024-09-14T09:07:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.909980+0000", 2024-09-14T09:07:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.714e-06 2024-09-14T09:07:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.909988+0000", 2024-09-14T09:07:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7859999999999997e-06 2024-09-14T09:07:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.910017+0000", 2024-09-14T09:07:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9737e-05 2024-09-14T09:07:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.910039+0000", 2024-09-14T09:07:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1653000000000001e-05 2024-09-14T09:07:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006697+0000", 2024-09-14T09:07:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.096657958000000002 2024-09-14T09:07:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006715+0000", 2024-09-14T09:07:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.751e-05 2024-09-14T09:07:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.007004+0000", 2024-09-14T09:07:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028948800000000002 2024-09-14T09:07:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:93272 3.3 3:c49b4fbc:::rbd_data.115e7770ddc5.000000000000001a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:07:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.971357+0000", 2024-09-14T09:07:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.583343055, 2024-09-14T09:07:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.112268252, 2024-09-14T09:07:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 93272 2024-09-14T09:07:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971357+0000", 2024-09-14T09:07:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971357+0000", 2024-09-14T09:07:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971360+0000", 2024-09-14T09:07:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8250000000000001e-06 2024-09-14T09:07:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971371+0000", 2024-09-14T09:07:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1537e-05 2024-09-14T09:07:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971399+0000", 2024-09-14T09:07:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8274999999999999e-05 2024-09-14T09:07:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971417+0000", 2024-09-14T09:07:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7241999999999999e-05 2024-09-14T09:07:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971537+0000", 2024-09-14T09:07:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012010600000000001 2024-09-14T09:07:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971685+0000", 2024-09-14T09:07:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014777599999999999 2024-09-14T09:07:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.979172+0000", 2024-09-14T09:07:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0074876810000000004 2024-09-14T09:07:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083438+0000", 2024-09-14T09:07:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.104265969 2024-09-14T09:07:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083466+0000", 2024-09-14T09:07:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7665999999999999e-05 2024-09-14T09:07:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083553+0000", 2024-09-14T09:07:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6809000000000002e-05 2024-09-14T09:07:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083625+0000", 2024-09-14T09:07:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2366000000000005e-05 2024-09-14T09:07:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:93272 3.3 e95/86 ondisk, result = 0)", 2024-09-14T09:07:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.997797+0000", 2024-09-14T09:07:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.556903074000001, 2024-09-14T09:07:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.085655196000000003, 2024-09-14T09:07:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.997797+0000", 2024-09-14T09:07:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.997797+0000", 2024-09-14T09:07:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.997799+0000", 2024-09-14T09:07:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7460000000000001e-06 2024-09-14T09:07:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.997807+0000", 2024-09-14T09:07:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1110000000000008e-06 2024-09-14T09:07:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.997836+0000", 2024-09-14T09:07:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9580000000000001e-05 2024-09-14T09:07:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.997851+0000", 2024-09-14T09:07:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4708e-05 2024-09-14T09:07:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083413+0000", 2024-09-14T09:07:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.085562568000000006 2024-09-14T09:07:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083428+0000", 2024-09-14T09:07:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4970000000000001e-05 2024-09-14T09:07:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083452+0000", 2024-09-14T09:07:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3513e-05 2024-09-14T09:07:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:95446 3.9 3:9150faf5:::rbd_data.115e9a87c1b2.000000000000003c:21 [read 358400~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:07:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:58.504395+0000", 2024-09-14T09:07:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 19.050304446999998, 2024-09-14T09:07:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.038354049000000001, 2024-09-14T09:07:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 95446 2024-09-14T09:07:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.504395+0000", 2024-09-14T09:07:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.504395+0000", 2024-09-14T09:07:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.504396+0000", 2024-09-14T09:07:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3200000000000005e-07 2024-09-14T09:07:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.504408+0000", 2024-09-14T09:07:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1819e-05 2024-09-14T09:07:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.504419+0000", 2024-09-14T09:07:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1252e-05 2024-09-14T09:07:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.504427+0000", 2024-09-14T09:07:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3909999999999998e-06 2024-09-14T09:07:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.542656+0000", 2024-09-14T09:07:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.038229015999999998 2024-09-14T09:07:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.542708+0000", 2024-09-14T09:07:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1102e-05 2024-09-14T09:07:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.542749+0000", 2024-09-14T09:07:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1836999999999999e-05 2024-09-14T09:07:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:96694 3.11 3:8a7e3c58:::rbd_data.115e63832e35.000000000000001c:head [stat,truncate 131072] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-14T09:07:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:07.004845+0000", 2024-09-14T09:07:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.549855039000001, 2024-09-14T09:07:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.85224771499999996, 2024-09-14T09:07:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 96694 2024-09-14T09:07:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004845+0000", 2024-09-14T09:07:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004845+0000", 2024-09-14T09:07:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004848+0000", 2024-09-14T09:07:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:07:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004862+0000", 2024-09-14T09:07:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4426e-05 2024-09-14T09:07:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004903+0000", 2024-09-14T09:07:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0447000000000001e-05 2024-09-14T09:07:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004926+0000", 2024-09-14T09:07:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3260000000000001e-05 2024-09-14T09:07:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.005104+0000", 2024-09-14T09:07:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000178018 2024-09-14T09:07:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.005120+0000", 2024-09-14T09:07:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6422999999999998e-05 2024-09-14T09:07:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.851775+0000", 2024-09-14T09:07:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.84665435499999997 2024-09-14T09:07:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.852176+0000", 2024-09-14T09:07:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000400787 2024-09-14T09:07:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-09-14T09:07:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.852662+0000", 2024-09-14T09:07:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048621899999999997 2024-09-14T09:07:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.855261+0000", 2024-09-14T09:07:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025991479999999999 2024-09-14T09:07:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.856839+0000", 2024-09-14T09:07:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015779819999999999 2024-09-14T09:07:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.856993+0000", 2024-09-14T09:07:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015431900000000001 2024-09-14T09:07:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.857093+0000", 2024-09-14T09:07:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9314000000000005e-05 2024-09-14T09:07:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:97612 3.e 3:7298f82d:::rbd_data.115e113dc8ca.00000000000000e8:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:07:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:11.399529+0000", 2024-09-14T09:07:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.1551709859999999, 2024-09-14T09:07:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.18555535000000001, 2024-09-14T09:07:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 97612 2024-09-14T09:07:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399529+0000", 2024-09-14T09:07:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399529+0000", 2024-09-14T09:07:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399532+0000", 2024-09-14T09:07:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6599999999999999e-06 2024-09-14T09:07:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399544+0000", 2024-09-14T09:07:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2527000000000001e-05 2024-09-14T09:07:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399574+0000", 2024-09-14T09:07:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.97e-05 2024-09-14T09:07:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399588+0000", 2024-09-14T09:07:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4653e-05 2024-09-14T09:07:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399740+0000", 2024-09-14T09:07:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015180900000000001 2024-09-14T09:07:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399952+0000", 2024-09-14T09:07:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000211435 2024-09-14T09:07:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.411686+0000", 2024-09-14T09:07:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011733883000000001 2024-09-14T09:07:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584791+0000", 2024-09-14T09:07:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.17310596 2024-09-14T09:07:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584915+0000", 2024-09-14T09:07:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012330800000000001 2024-09-14T09:07:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.585010+0000", 2024-09-14T09:07:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5216000000000005e-05 2024-09-14T09:07:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.585084+0000", 2024-09-14T09:07:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4198999999999999e-05 2024-09-14T09:07:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:97612 3.e e106/103 ondisk, result = 0)", 2024-09-14T09:07:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:11.451700+0000", 2024-09-14T09:07:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.102999692, 2024-09-14T09:07:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13339934000000001, 2024-09-14T09:07:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451700+0000", 2024-09-14T09:07:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451700+0000", 2024-09-14T09:07:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451702+0000", 2024-09-14T09:07:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.34e-06 2024-09-14T09:07:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451713+0000", 2024-09-14T09:07:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1e-05 2024-09-14T09:07:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451751+0000", 2024-09-14T09:07:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7762999999999997e-05 2024-09-14T09:07:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451770+0000", 2024-09-14T09:07:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8626000000000001e-05 2024-09-14T09:07:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584888+0000", 2024-09-14T09:07:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13311764500000001 2024-09-14T09:07:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584903+0000", 2024-09-14T09:07:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.499e-05 2024-09-14T09:07:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.585099+0000", 2024-09-14T09:07:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019697599999999999 2024-09-14T09:07:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:17.653 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:17.654 INFO:teuthology.orchestra.run.smithi176.stdout:5281 write 0xd702f23 thru 0xd703036 (0x114 bytes) 2024-09-14T09:07:17.654 INFO:teuthology.orchestra.run.smithi176.stdout:5282 writesame 0xcfbc326 thru 0xcfc512e data_size 0x1(0x8e09 bytes) 2024-09-14T09:07:17.655 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:07:17.655 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:17.655+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:17.692 INFO:teuthology.orchestra.run.smithi176.stdout:5284 trunc from 0xd703037 to 0x25d80ef 2024-09-14T09:07:17.744 INFO:teuthology.orchestra.run.smithi176.stdout:5285 trunc from 0x25d80ef to 0xcb91932 2024-09-14T09:07:17.756 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:17.755+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:17.758 INFO:teuthology.orchestra.run.smithi176.stdout:5286 writesame 0x2d3e805 thru 0x2d40eba data_size 0x5(0x26b6 bytes) 2024-09-14T09:07:17.773 INFO:teuthology.orchestra.run.smithi176.stdout:5289 writesame 0x49237df thru 0x492dff2 data_size 0x3e(0xa814 bytes) 2024-09-14T09:07:17.787 INFO:teuthology.orchestra.run.smithi176.stdout:5290 read 0x8a6d8fc thru 0x8a757ff (0x7f04 bytes) 2024-09-14T09:07:17.787 INFO:teuthology.orchestra.run.smithi176.stdout:5291 read 0xc9680c2 thru 0xc971731 (0x9670 bytes) 2024-09-14T09:07:17.791 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:17.791 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:17.791 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:07:17.791 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:17.805 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:07:17.818 INFO:teuthology.orchestra.run.smithi176.stdout:5292 trunc from 0xcb91932 to 0x6679db 2024-09-14T09:07:17.856 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:17.855+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:17.882 INFO:teuthology.orchestra.run.smithi176.stdout:5293 writesame 0xa94fb5d thru 0xa950c0e data_size 0x1(0x10b2 bytes) 2024-09-14T09:07:17.887 INFO:teuthology.orchestra.run.smithi176.stdout:5294 trunc from 0xa950c0f to 0xcb3ce85 2024-09-14T09:07:17.897 INFO:teuthology.orchestra.run.smithi176.stdout:5295 read 0x1088fd2 thru 0x108de57 (0x4e86 bytes) 2024-09-14T09:07:17.910 INFO:teuthology.orchestra.run.smithi176.stdout:5296 write 0xd893907 thru 0xd8a00d2 (0xc7cc bytes) 2024-09-14T09:07:17.919 INFO:teuthology.orchestra.run.smithi176.stdout:5297 read 0x59a5da0 thru 0x59ab13e (0x539f bytes) 2024-09-14T09:07:17.919 INFO:teuthology.orchestra.run.smithi176.stdout:5298 write 0xd53abc thru 0xd5e00d (0xa552 bytes) 2024-09-14T09:07:17.919 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T09:07:17.932 INFO:teuthology.orchestra.run.smithi176.stdout:5299 punch from 0x8485b2d to 0x8486b79, (0x104c bytes) 2024-09-14T09:07:17.932 INFO:teuthology.orchestra.run.smithi176.stdout:5300 write 0x83c4000 thru 0x83c6949 (0x294a bytes) 2024-09-14T09:07:17.936 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:17.936 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:17.936 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:07:17.936 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:07:17.936 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:17.942 INFO:teuthology.orchestra.run.smithi176.stdout:5301 read 0x43fc4f0 thru 0x4404d57 (0x8868 bytes) 2024-09-14T09:07:17.943 INFO:teuthology.orchestra.run.smithi176.stdout:5302 read 0x7fbddf9 thru 0x7fbec86 (0xe8e bytes) 2024-09-14T09:07:17.943 INFO:teuthology.orchestra.run.smithi176.stdout:5303 writesame 0xd0e9720 thru 0xd0f5ffe data_size 0x3(0xc8df bytes) 2024-09-14T09:07:17.954 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:07:17.957 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:17.955+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:17.960 INFO:teuthology.orchestra.run.smithi176.stdout:5304 writesame 0x56287fd thru 0x562d3a1 data_size 0x5(0x4ba5 bytes) 2024-09-14T09:07:17.971 INFO:teuthology.orchestra.run.smithi176.stdout:5305 punch from 0x90c0ac8 to 0x90cb3e7, (0xa91f bytes) 2024-09-14T09:07:17.972 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:17.972 INFO:teuthology.orchestra.run.smithi176.stdout:5307 writesame 0xa275aed thru 0xa282041 data_size 0x9(0xc555 bytes) 2024-09-14T09:07:18.002 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:07:18.010 INFO:teuthology.orchestra.run.smithi176.stdout:5308 trunc from 0xd8a00d3 to 0x22e09c5 2024-09-14T09:07:18.058 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:18.059+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:18.058 INFO:teuthology.orchestra.run.smithi176.stdout:5309 write 0x9b73397 thru 0x9b7e238 (0xaea2 bytes) 2024-09-14T09:07:18.082 INFO:teuthology.orchestra.run.smithi176.stdout:5310 write 0xbe9e84f thru 0xbea11ab (0x295d bytes) 2024-09-14T09:07:18.087 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:18.087 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:07:18.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:07:18.087 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:07:18.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.087 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:71067 3.1d 3:b9bb9b2f:::rbd_data.115ebd4a9f36.0000000000000006:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:07:18.087 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:29.920442+0000", 2024-09-14T09:07:18.087 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.170277796999997, 2024-09-14T09:07:18.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13139427400000001, 2024-09-14T09:07:18.087 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.087 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.088 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:18.088 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.088 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.088 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.088 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 71067 2024-09-14T09:07:18.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.088 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920442+0000", 2024-09-14T09:07:18.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920442+0000", 2024-09-14T09:07:18.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920445+0000", 2024-09-14T09:07:18.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7099999999999999e-06 2024-09-14T09:07:18.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920461+0000", 2024-09-14T09:07:18.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6436e-05 2024-09-14T09:07:18.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920492+0000", 2024-09-14T09:07:18.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1069999999999999e-05 2024-09-14T09:07:18.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920508+0000", 2024-09-14T09:07:18.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5463000000000001e-05 2024-09-14T09:07:18.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920625+0000", 2024-09-14T09:07:18.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116932 2024-09-14T09:07:18.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.920766+0000", 2024-09-14T09:07:18.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000141092 2024-09-14T09:07:18.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-09-14T09:07:18.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:29.929373+0000", 2024-09-14T09:07:18.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086074510000000003 2024-09-14T09:07:18.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:18.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:30.051654+0000", 2024-09-14T09:07:18.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.12228082999999999 2024-09-14T09:07:18.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:18.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:30.051711+0000", 2024-09-14T09:07:18.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7108000000000003e-05 2024-09-14T09:07:18.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:18.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:30.051779+0000", 2024-09-14T09:07:18.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7696999999999999e-05 2024-09-14T09:07:18.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:30.051836+0000", 2024-09-14T09:07:18.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7485000000000003e-05 2024-09-14T09:07:18.093 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.093 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.093 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.093 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:84446 3.c 3:32273729:::rbd_data.115e77b799c3.0000000000000138:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:07:18.093 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:54.107345+0000", 2024-09-14T09:07:18.093 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.983374957999999, 2024-09-14T09:07:18.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14020856500000001, 2024-09-14T09:07:18.093 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.093 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.094 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:18.094 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.094 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.094 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.094 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 84446 2024-09-14T09:07:18.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.094 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.107345+0000", 2024-09-14T09:07:18.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.107345+0000", 2024-09-14T09:07:18.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.107347+0000", 2024-09-14T09:07:18.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1940000000000001e-06 2024-09-14T09:07:18.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.107358+0000", 2024-09-14T09:07:18.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0743e-05 2024-09-14T09:07:18.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.107384+0000", 2024-09-14T09:07:18.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5789999999999999e-05 2024-09-14T09:07:18.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.107399+0000", 2024-09-14T09:07:18.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5546999999999999e-05 2024-09-14T09:07:18.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.107557+0000", 2024-09-14T09:07:18.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015800800000000001 2024-09-14T09:07:18.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.107783+0000", 2024-09-14T09:07:18.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00022566200000000001 2024-09-14T09:07:18.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-09-14T09:07:18.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.117026+0000", 2024-09-14T09:07:18.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0092434270000000002 2024-09-14T09:07:18.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:18.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.247287+0000", 2024-09-14T09:07:18.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13026119899999999 2024-09-14T09:07:18.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:18.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.247387+0000", 2024-09-14T09:07:18.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9293e-05 2024-09-14T09:07:18.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:18.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.247478+0000", 2024-09-14T09:07:18.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1586e-05 2024-09-14T09:07:18.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.247553+0000", 2024-09-14T09:07:18.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5116000000000004e-05 2024-09-14T09:07:18.099 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.099 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.100 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.100 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:91994 3.1c 3:3a22cc55:::rbd_data.115e77b799c3.0000000000000000:20 [read 329728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:18.100 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.328950+0000", 2024-09-14T09:07:18.100 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.761770326000001, 2024-09-14T09:07:18.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.118722383, 2024-09-14T09:07:18.100 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.100 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.100 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:18.100 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.100 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.100 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.101 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 91994 2024-09-14T09:07:18.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.101 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.328950+0000", 2024-09-14T09:07:18.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.328950+0000", 2024-09-14T09:07:18.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336244+0000", 2024-09-14T09:07:18.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007294638 2024-09-14T09:07:18.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336250+0000", 2024-09-14T09:07:18.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4709999999999998e-06 2024-09-14T09:07:18.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336259+0000", 2024-09-14T09:07:18.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8320000000000001e-06 2024-09-14T09:07:18.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336266+0000", 2024-09-14T09:07:18.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.117e-06 2024-09-14T09:07:18.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.446742+0000", 2024-09-14T09:07:18.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11047644500000001 2024-09-14T09:07:18.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.446898+0000", 2024-09-14T09:07:18.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000156436 2024-09-14T09:07:18.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.447672+0000", 2024-09-14T09:07:18.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077344399999999998 2024-09-14T09:07:18.104 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.104 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.104 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.105 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:92029 3.c 3:36a17670:::rbd_data.115e77b799c3.0000000000000002:20 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:18.105 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.336432+0000", 2024-09-14T09:07:18.105 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.754288241000001, 2024-09-14T09:07:18.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.118542886, 2024-09-14T09:07:18.105 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.105 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.105 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:18.105 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.105 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.105 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.105 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 92029 2024-09-14T09:07:18.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.105 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336432+0000", 2024-09-14T09:07:18.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336432+0000", 2024-09-14T09:07:18.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336432+0000", 2024-09-14T09:07:18.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2800000000000003e-07 2024-09-14T09:07:18.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336433+0000", 2024-09-14T09:07:18.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.314e-06 2024-09-14T09:07:18.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336438+0000", 2024-09-14T09:07:18.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2630000000000003e-06 2024-09-14T09:07:18.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336442+0000", 2024-09-14T09:07:18.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1829999999999998e-06 2024-09-14T09:07:18.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.454074+0000", 2024-09-14T09:07:18.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.117632532 2024-09-14T09:07:18.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.454184+0000", 2024-09-14T09:07:18.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109894 2024-09-14T09:07:18.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.454974+0000", 2024-09-14T09:07:18.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00079037200000000001 2024-09-14T09:07:18.109 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.109 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.109 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.109 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:92030 3.c 3:36a17670:::rbd_data.115e77b799c3.0000000000000002:20 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:18.109 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.336446+0000", 2024-09-14T09:07:18.109 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.754274235, 2024-09-14T09:07:18.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11960451599999999, 2024-09-14T09:07:18.109 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.110 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.110 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:18.110 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.110 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.110 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.110 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 92030 2024-09-14T09:07:18.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.110 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336446+0000", 2024-09-14T09:07:18.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336446+0000", 2024-09-14T09:07:18.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336446+0000", 2024-09-14T09:07:18.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3299999999999998e-07 2024-09-14T09:07:18.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336451+0000", 2024-09-14T09:07:18.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4e-06 2024-09-14T09:07:18.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336457+0000", 2024-09-14T09:07:18.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6899999999999997e-06 2024-09-14T09:07:18.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336461+0000", 2024-09-14T09:07:18.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5810000000000001e-06 2024-09-14T09:07:18.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.455008+0000", 2024-09-14T09:07:18.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.118546914 2024-09-14T09:07:18.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.455114+0000", 2024-09-14T09:07:18.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106586 2024-09-14T09:07:18.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.456050+0000", 2024-09-14T09:07:18.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093601199999999996 2024-09-14T09:07:18.113 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.113 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.113 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.114 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:92031 3.c 3:36a17670:::rbd_data.115e77b799c3.0000000000000002:20 [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:18.114 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.336465+0000", 2024-09-14T09:07:18.114 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.754254821, 2024-09-14T09:07:18.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.12057048500000001, 2024-09-14T09:07:18.114 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.114 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.114 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:18.114 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.114 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.114 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.114 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 92031 2024-09-14T09:07:18.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.114 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336465+0000", 2024-09-14T09:07:18.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336465+0000", 2024-09-14T09:07:18.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336465+0000", 2024-09-14T09:07:18.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2800000000000003e-07 2024-09-14T09:07:18.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336467+0000", 2024-09-14T09:07:18.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.451e-06 2024-09-14T09:07:18.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336471+0000", 2024-09-14T09:07:18.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8e-06 2024-09-14T09:07:18.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336474+0000", 2024-09-14T09:07:18.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2329999999999998e-06 2024-09-14T09:07:18.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.456133+0000", 2024-09-14T09:07:18.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.119659486 2024-09-14T09:07:18.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.456242+0000", 2024-09-14T09:07:18.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010828300000000001 2024-09-14T09:07:18.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.457035+0000", 2024-09-14T09:07:18.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00079390400000000001 2024-09-14T09:07:18.118 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.118 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.118 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.118 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:92032 3.c 3:36a17670:::rbd_data.115e77b799c3.0000000000000002:20 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:18.118 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.336478+0000", 2024-09-14T09:07:18.118 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.754242292000001, 2024-09-14T09:07:18.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.121204889, 2024-09-14T09:07:18.118 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.118 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.118 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:18.118 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.119 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.119 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.119 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 92032 2024-09-14T09:07:18.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.119 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336478+0000", 2024-09-14T09:07:18.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336478+0000", 2024-09-14T09:07:18.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336478+0000", 2024-09-14T09:07:18.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1199999999999999e-07 2024-09-14T09:07:18.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336479+0000", 2024-09-14T09:07:18.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.279e-06 2024-09-14T09:07:18.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336483+0000", 2024-09-14T09:07:18.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5640000000000001e-06 2024-09-14T09:07:18.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336486+0000", 2024-09-14T09:07:18.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.269e-06 2024-09-14T09:07:18.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.457079+0000", 2024-09-14T09:07:18.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.12059325899999999 2024-09-14T09:07:18.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.457159+0000", 2024-09-14T09:07:18.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0016000000000001e-05 2024-09-14T09:07:18.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.457682+0000", 2024-09-14T09:07:18.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00052318999999999998 2024-09-14T09:07:18.122 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.122 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.122 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.122 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:92033 3.c 3:36a17670:::rbd_data.115e77b799c3.0000000000000002:20 [read 161792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:18.122 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.336490+0000", 2024-09-14T09:07:18.122 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.754230186000001, 2024-09-14T09:07:18.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.121973129, 2024-09-14T09:07:18.123 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.123 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.123 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:18.123 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.123 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.123 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.123 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 92033 2024-09-14T09:07:18.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.123 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336490+0000", 2024-09-14T09:07:18.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336490+0000", 2024-09-14T09:07:18.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336490+0000", 2024-09-14T09:07:18.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1699999999999999e-07 2024-09-14T09:07:18.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336491+0000", 2024-09-14T09:07:18.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.358e-06 2024-09-14T09:07:18.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336495+0000", 2024-09-14T09:07:18.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6109999999999998e-06 2024-09-14T09:07:18.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336498+0000", 2024-09-14T09:07:18.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1920000000000001e-06 2024-09-14T09:07:18.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.457733+0000", 2024-09-14T09:07:18.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.121234457 2024-09-14T09:07:18.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.457835+0000", 2024-09-14T09:07:18.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010231299999999999 2024-09-14T09:07:18.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.458463+0000", 2024-09-14T09:07:18.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00062788099999999995 2024-09-14T09:07:18.126 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.126 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:92034 3.c 3:36a17670:::rbd_data.115e77b799c3.0000000000000002:20 [read 184320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.336513+0000", 2024-09-14T09:07:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.754206921000002, 2024-09-14T09:07:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.122616791, 2024-09-14T09:07:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 92034 2024-09-14T09:07:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336513+0000", 2024-09-14T09:07:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336513+0000", 2024-09-14T09:07:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336513+0000", 2024-09-14T09:07:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2399999999999999e-07 2024-09-14T09:07:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336515+0000", 2024-09-14T09:07:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7060000000000001e-06 2024-09-14T09:07:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336520+0000", 2024-09-14T09:07:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4770000000000002e-06 2024-09-14T09:07:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336524+0000", 2024-09-14T09:07:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5049999999999999e-06 2024-09-14T09:07:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.458513+0000", 2024-09-14T09:07:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.12198932899999999 2024-09-14T09:07:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.458618+0000", 2024-09-14T09:07:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000104314 2024-09-14T09:07:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.459130+0000", 2024-09-14T09:07:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00051213599999999995 2024-09-14T09:07:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:92035 3.c 3:36a17670:::rbd_data.115e77b799c3.0000000000000002:20 [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.336528+0000", 2024-09-14T09:07:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.754191339999998, 2024-09-14T09:07:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.123053973, 2024-09-14T09:07:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 92035 2024-09-14T09:07:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336528+0000", 2024-09-14T09:07:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.336528+0000", 2024-09-14T09:07:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344139+0000", 2024-09-14T09:07:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076106000000000004 2024-09-14T09:07:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344144+0000", 2024-09-14T09:07:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2109999999999997e-06 2024-09-14T09:07:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344157+0000", 2024-09-14T09:07:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2315e-05 2024-09-14T09:07:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344163+0000", 2024-09-14T09:07:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.674e-06 2024-09-14T09:07:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.459187+0000", 2024-09-14T09:07:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11502417400000001 2024-09-14T09:07:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.459287+0000", 2024-09-14T09:07:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9233000000000003e-05 2024-09-14T09:07:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.459582+0000", 2024-09-14T09:07:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029576599999999999 2024-09-14T09:07:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:92039 3.c 3:36a17670:::rbd_data.115e77b799c3.0000000000000002:20 [read 299008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.344226+0000", 2024-09-14T09:07:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.746493866000002, 2024-09-14T09:07:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.118415198, 2024-09-14T09:07:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 92039 2024-09-14T09:07:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344226+0000", 2024-09-14T09:07:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344226+0000", 2024-09-14T09:07:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344226+0000", 2024-09-14T09:07:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0100000000000001e-07 2024-09-14T09:07:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344228+0000", 2024-09-14T09:07:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7519999999999999e-06 2024-09-14T09:07:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344232+0000", 2024-09-14T09:07:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7479999999999999e-06 2024-09-14T09:07:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344235+0000", 2024-09-14T09:07:18.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3419999999999999e-06 2024-09-14T09:07:18.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.462185+0000", 2024-09-14T09:07:18.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11795006 2024-09-14T09:07:18.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.462275+0000", 2024-09-14T09:07:18.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9727000000000003e-05 2024-09-14T09:07:18.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.462641+0000", 2024-09-14T09:07:18.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000366268 2024-09-14T09:07:18.141 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.141 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.141 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.141 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:92040 3.c 3:36a17670:::rbd_data.115e77b799c3.0000000000000002:20 [read 321536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:18.141 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.344239+0000", 2024-09-14T09:07:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.746480609999999, 2024-09-14T09:07:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.118924688, 2024-09-14T09:07:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 92040 2024-09-14T09:07:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344239+0000", 2024-09-14T09:07:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344239+0000", 2024-09-14T09:07:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344240+0000", 2024-09-14T09:07:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9700000000000003e-07 2024-09-14T09:07:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344241+0000", 2024-09-14T09:07:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.485e-06 2024-09-14T09:07:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344245+0000", 2024-09-14T09:07:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8160000000000004e-06 2024-09-14T09:07:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344254+0000", 2024-09-14T09:07:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1630000000000002e-06 2024-09-14T09:07:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.462704+0000", 2024-09-14T09:07:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.118449582 2024-09-14T09:07:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.462796+0000", 2024-09-14T09:07:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2207999999999995e-05 2024-09-14T09:07:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.463164+0000", 2024-09-14T09:07:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036813699999999999 2024-09-14T09:07:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:92041 3.c 3:36a17670:::rbd_data.115e77b799c3.0000000000000002:20 [read 333824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.344258+0000", 2024-09-14T09:07:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.746461707000002, 2024-09-14T09:07:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11931754, 2024-09-14T09:07:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 92041 2024-09-14T09:07:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344258+0000", 2024-09-14T09:07:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344258+0000", 2024-09-14T09:07:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344258+0000", 2024-09-14T09:07:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0699999999999998e-07 2024-09-14T09:07:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344260+0000", 2024-09-14T09:07:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4130000000000001e-06 2024-09-14T09:07:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344264+0000", 2024-09-14T09:07:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4739999999999999e-06 2024-09-14T09:07:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344268+0000", 2024-09-14T09:07:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6109999999999998e-06 2024-09-14T09:07:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.463185+0000", 2024-09-14T09:07:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.118917412 2024-09-14T09:07:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.463252+0000", 2024-09-14T09:07:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6903999999999995e-05 2024-09-14T09:07:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.463576+0000", 2024-09-14T09:07:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032341899999999997 2024-09-14T09:07:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:92042 3.c 3:36a17670:::rbd_data.115e77b799c3.0000000000000002:20 [read 356352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.344272+0000", 2024-09-14T09:07:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.746447913000001, 2024-09-14T09:07:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.119796246, 2024-09-14T09:07:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 92042 2024-09-14T09:07:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344272+0000", 2024-09-14T09:07:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344272+0000", 2024-09-14T09:07:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344272+0000", 2024-09-14T09:07:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2399999999999999e-07 2024-09-14T09:07:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344281+0000", 2024-09-14T09:07:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3599999999999996e-06 2024-09-14T09:07:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344285+0000", 2024-09-14T09:07:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1389999999999997e-06 2024-09-14T09:07:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344289+0000", 2024-09-14T09:07:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5199999999999999e-06 2024-09-14T09:07:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.463619+0000", 2024-09-14T09:07:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.119329888 2024-09-14T09:07:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.463727+0000", 2024-09-14T09:07:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000108306 2024-09-14T09:07:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.464068+0000", 2024-09-14T09:07:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034070900000000002 2024-09-14T09:07:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:92043 3.c 3:36a17670:::rbd_data.115e77b799c3.0000000000000002:20 [read 378880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.344294+0000", 2024-09-14T09:07:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.746425762000001, 2024-09-14T09:07:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.12026036800000001, 2024-09-14T09:07:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 92043 2024-09-14T09:07:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344294+0000", 2024-09-14T09:07:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344294+0000", 2024-09-14T09:07:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344294+0000", 2024-09-14T09:07:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2899999999999999e-07 2024-09-14T09:07:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344296+0000", 2024-09-14T09:07:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.505e-06 2024-09-14T09:07:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344300+0000", 2024-09-14T09:07:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7069999999999998e-06 2024-09-14T09:07:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344303+0000", 2024-09-14T09:07:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3790000000000002e-06 2024-09-14T09:07:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.464116+0000", 2024-09-14T09:07:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.119812871 2024-09-14T09:07:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.464212+0000", 2024-09-14T09:07:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6303000000000003e-05 2024-09-14T09:07:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.464554+0000", 2024-09-14T09:07:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000342274 2024-09-14T09:07:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:92044 3.c 3:36a17670:::rbd_data.115e77b799c3.0000000000000002:20 [read 413696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.344307+0000", 2024-09-14T09:07:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.746412930999998, 2024-09-14T09:07:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.120749328, 2024-09-14T09:07:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 92044 2024-09-14T09:07:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344307+0000", 2024-09-14T09:07:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344307+0000", 2024-09-14T09:07:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344307+0000", 2024-09-14T09:07:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1e-07 2024-09-14T09:07:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344309+0000", 2024-09-14T09:07:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.48e-06 2024-09-14T09:07:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344312+0000", 2024-09-14T09:07:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5590000000000001e-06 2024-09-14T09:07:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344317+0000", 2024-09-14T09:07:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3669999999999999e-06 2024-09-14T09:07:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.464593+0000", 2024-09-14T09:07:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.120276482 2024-09-14T09:07:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.464684+0000", 2024-09-14T09:07:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0619999999999996e-05 2024-09-14T09:07:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.465056+0000", 2024-09-14T09:07:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037251 2024-09-14T09:07:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:92045 3.c 3:36a17670:::rbd_data.115e77b799c3.0000000000000002:20 [read 436224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.344320+0000", 2024-09-14T09:07:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.746399342, 2024-09-14T09:07:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.121241711, 2024-09-14T09:07:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 92045 2024-09-14T09:07:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344320+0000", 2024-09-14T09:07:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344320+0000", 2024-09-14T09:07:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344321+0000", 2024-09-14T09:07:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0800000000000001e-07 2024-09-14T09:07:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344322+0000", 2024-09-14T09:07:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.48e-06 2024-09-14T09:07:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344326+0000", 2024-09-14T09:07:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5659999999999998e-06 2024-09-14T09:07:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344329+0000", 2024-09-14T09:07:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2260000000000001e-06 2024-09-14T09:07:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.465113+0000", 2024-09-14T09:07:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.12078362600000001 2024-09-14T09:07:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.465219+0000", 2024-09-14T09:07:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010594000000000001 2024-09-14T09:07:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.465562+0000", 2024-09-14T09:07:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034356499999999999 2024-09-14T09:07:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:92046 3.c 3:36a17670:::rbd_data.115e77b799c3.0000000000000002:20 [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.344333+0000", 2024-09-14T09:07:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.746386973, 2024-09-14T09:07:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.12173684999999999, 2024-09-14T09:07:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 92046 2024-09-14T09:07:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344333+0000", 2024-09-14T09:07:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344333+0000", 2024-09-14T09:07:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344333+0000", 2024-09-14T09:07:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1e-07 2024-09-14T09:07:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344335+0000", 2024-09-14T09:07:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.483e-06 2024-09-14T09:07:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344338+0000", 2024-09-14T09:07:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7110000000000001e-06 2024-09-14T09:07:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344342+0000", 2024-09-14T09:07:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2610000000000002e-06 2024-09-14T09:07:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.465612+0000", 2024-09-14T09:07:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.12127030499999999 2024-09-14T09:07:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.465708+0000", 2024-09-14T09:07:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5731999999999998e-05 2024-09-14T09:07:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.466070+0000", 2024-09-14T09:07:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036204800000000001 2024-09-14T09:07:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:92047 3.c 3:36a17670:::rbd_data.115e77b799c3.0000000000000002:20 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.344346+0000", 2024-09-14T09:07:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.74637426, 2024-09-14T09:07:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.12216389499999999, 2024-09-14T09:07:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 92047 2024-09-14T09:07:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344346+0000", 2024-09-14T09:07:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344346+0000", 2024-09-14T09:07:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344346+0000", 2024-09-14T09:07:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1100000000000002e-07 2024-09-14T09:07:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344347+0000", 2024-09-14T09:07:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4789999999999999e-06 2024-09-14T09:07:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344351+0000", 2024-09-14T09:07:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6150000000000001e-06 2024-09-14T09:07:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344354+0000", 2024-09-14T09:07:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2009999999999999e-06 2024-09-14T09:07:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.466124+0000", 2024-09-14T09:07:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.121769496 2024-09-14T09:07:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.466181+0000", 2024-09-14T09:07:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7077999999999998e-05 2024-09-14T09:07:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.466509+0000", 2024-09-14T09:07:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032871499999999999 2024-09-14T09:07:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:92048 3.c 3:36a17670:::rbd_data.115e77b799c3.0000000000000002:20 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T09:07:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.344358+0000", 2024-09-14T09:07:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.746361785000001, 2024-09-14T09:07:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.122674888, 2024-09-14T09:07:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 92048 2024-09-14T09:07:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344358+0000", 2024-09-14T09:07:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344358+0000", 2024-09-14T09:07:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344358+0000", 2024-09-14T09:07:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3200000000000001e-07 2024-09-14T09:07:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344360+0000", 2024-09-14T09:07:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3909999999999999e-06 2024-09-14T09:07:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344363+0000", 2024-09-14T09:07:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.631e-06 2024-09-14T09:07:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.344367+0000", 2024-09-14T09:07:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3589999999999999e-06 2024-09-14T09:07:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.466562+0000", 2024-09-14T09:07:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.122195342 2024-09-14T09:07:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.466684+0000", 2024-09-14T09:07:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121981 2024-09-14T09:07:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.467033+0000", 2024-09-14T09:07:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034885200000000003 2024-09-14T09:07:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:18.181 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:18.183 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:07:18.184 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:18.187+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:18.184 INFO:teuthology.orchestra.run.smithi176.stdout:5311 write 0x402775c thru 0x40289c5 (0x126a bytes) 2024-09-14T09:07:18.184 INFO:teuthology.orchestra.run.smithi176.stdout:5312 punch from 0x2397d8d to 0x239f15e, (0x73d1 bytes) 2024-09-14T09:07:18.185 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:18.185 INFO:teuthology.orchestra.run.smithi176.stdout:5316 writesame 0xa239710 thru 0xa23a665 data_size 0xd(0xf56 bytes) 2024-09-14T09:07:18.185 INFO:teuthology.orchestra.run.smithi176.stdout:5317 read 0x8f17c9 thru 0x8f4849 (0x3081 bytes) 2024-09-14T09:07:18.185 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:18.185 INFO:teuthology.orchestra.run.smithi176.stdout:5321 write 0x3180734 thru 0x318fd43 (0xf610 bytes) 2024-09-14T09:07:18.185 INFO:teuthology.orchestra.run.smithi176.stdout:5322 read 0x7d4ec27 thru 0x7d50ab1 (0x1e8b bytes) 2024-09-14T09:07:18.185 INFO:teuthology.orchestra.run.smithi176.stdout:5323 writesame 0x409efa1 thru 0x40a712a data_size 0x1(0x818a bytes) 2024-09-14T09:07:18.185 INFO:teuthology.orchestra.run.smithi176.stdout:5328 write 0x9145ef9 thru 0x914d0cd (0x71d5 bytes) 2024-09-14T09:07:18.185 INFO:teuthology.orchestra.run.smithi176.stdout:5329 punch from 0x3a5bda4 to 0x3a61897, (0x5af3 bytes) 2024-09-14T09:07:18.185 INFO:teuthology.orchestra.run.smithi176.stdout:5330 punch from 0x1461e9b to 0x146394e, (0x1ab3 bytes) 2024-09-14T09:07:18.185 INFO:teuthology.orchestra.run.smithi176.stdout:5331 write 0xb57c7b3 thru 0xb583751 (0x6f9f bytes) 2024-09-14T09:07:18.186 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:18.186 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:18.186 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:18.188 INFO:teuthology.orchestra.run.smithi176.stdout:5333 writesame 0xe809a85 thru 0xe813fd9 data_size 0x5(0xa555 bytes) 2024-09-14T09:07:18.188 INFO:teuthology.orchestra.run.smithi176.stdout:5334 read 0x280cee9 thru 0x280e420 (0x1538 bytes) 2024-09-14T09:07:18.188 INFO:teuthology.orchestra.run.smithi176.stdout:5335 write 0xdafca5d thru 0xdb09440 (0xc9e4 bytes) 2024-09-14T09:07:18.200 INFO:teuthology.orchestra.run.smithi176.stdout:5336 read 0x3c5889 thru 0x3c5dbb (0x533 bytes) 2024-09-14T09:07:18.201 INFO:teuthology.orchestra.run.smithi176.stdout:5340 read 0xdb74453 thru 0xdb7ca5b (0x8609 bytes) 2024-09-14T09:07:18.202 INFO:teuthology.orchestra.run.smithi176.stdout:5342 punch from 0x762a633 to 0x76392ca, (0xec97 bytes) 2024-09-14T09:07:18.202 INFO:teuthology.orchestra.run.smithi176.stdout:5344 write 0x6c9d2b5 thru 0x6ca224c (0x4f98 bytes) 2024-09-14T09:07:18.208 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:18.209 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:18.212 INFO:teuthology.orchestra.run.smithi176.stdout:5345 read 0x70e8d9d thru 0x70f65a7 (0xd80b bytes) 2024-09-14T09:07:18.212 INFO:teuthology.orchestra.run.smithi176.stdout:5346 punch from 0xcda6107 to 0xcda8dea, (0x2ce3 bytes) 2024-09-14T09:07:18.213 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:18.213 INFO:teuthology.orchestra.run.smithi176.stdout:5349 read 0x9d65479 thru 0x9d6bb56 (0x66de bytes) 2024-09-14T09:07:18.227 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:18.227 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:18.234 INFO:teuthology.orchestra.run.smithi176.stdout:5351 trunc from 0xe813fda to 0x152d935 2024-09-14T09:07:18.247 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:18.247 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:18.267 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:07:18.267 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:07:18.284 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:18.287+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:18.286 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:18.286 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:18.290 INFO:teuthology.orchestra.run.smithi176.stdout:5353 write 0xecf8051 thru 0xed00f16 (0x8ec6 bytes) 2024-09-14T09:07:18.305 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:18.305 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:18.307 INFO:teuthology.orchestra.run.smithi176.stdout:5354 write 0xed9b78e thru 0xeda163e (0x5eb1 bytes) 2024-09-14T09:07:18.310 INFO:teuthology.orchestra.run.smithi176.stdout:5355 punch from 0xc2b0e6a to 0xc2bd84d, (0xc9e3 bytes) 2024-09-14T09:07:18.310 INFO:teuthology.orchestra.run.smithi176.stdout:5357 write 0x1e4e306 thru 0x1e51910 (0x360b bytes) 2024-09-14T09:07:18.319 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:18.319 INFO:teuthology.orchestra.run.smithi176.stdout:5359 read 0xbfdc5e1 thru 0xbfe97a6 (0xd1c6 bytes) 2024-09-14T09:07:18.319 INFO:teuthology.orchestra.run.smithi176.stdout:5360 writesame 0xde77f25 thru 0xde806f7 data_size 0x1d(0x87d3 bytes) 2024-09-14T09:07:18.329 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:18.330 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:18.331 INFO:teuthology.orchestra.run.smithi176.stdout:5361 punch from 0xe7d5162 to 0xe7e3cef, (0xeb8d bytes) 2024-09-14T09:07:18.331 INFO:teuthology.orchestra.run.smithi176.stdout:5362 read 0x37ccf61 thru 0x37dc199 (0xf239 bytes) 2024-09-14T09:07:18.331 INFO:teuthology.orchestra.run.smithi176.stdout:5364 read 0xb71bbb2 thru 0xb720f2e (0x537d bytes) 2024-09-14T09:07:18.331 INFO:teuthology.orchestra.run.smithi176.stdout:5365 clone 37 order 24 su 1048576 sc 2 2024-09-14T09:07:18.360 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:18.360 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:18.374 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:07:18.390 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:18.391+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:18.486 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:18.487+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:18.541 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:18.542 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:18.556 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:07:18.587 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:18.587+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:18.674 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:18.674 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:18.688 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:07:18.689 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:18.691+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:18.789 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:18.791+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:18.852 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:18.852 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:18.867 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:07:18.890 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:18.891+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:18.991 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:18.991+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:19.038 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:19.038 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:19.043 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is unset 2024-09-14T09:07:19.054 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:07:19.114 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:19.115+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:19.178 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:19.179 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:19.192 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:19.195+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:19.193 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:07:19.208 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone36 from 0xeda163f (overlap 0x6679db) to 0x0 2024-09-14T09:07:19.294 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:19.295+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:19.374 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:19.374 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:19.390 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:07:19.438 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:19.439+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:19.495 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:19.495+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:19.532 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-14T09:07:19.579 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:19.579 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:19.595 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:07:19.596 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:19.595+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:19.620 INFO:teuthology.orchestra.run.smithi176.stdout:5366 read 0x468ede5 thru 0x4698898 (0x9ab4 bytes) 2024-09-14T09:07:19.621 INFO:teuthology.orchestra.run.smithi176.stdout:5367 clone 38 order 18 su 1024 sc 8 2024-09-14T09:07:19.697 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:19.699+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:19.737 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:19.737 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:19.751 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:07:19.798 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:19.799+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:19.870 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:19.871 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:19.884 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:07:19.926 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:19.927+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:19.999 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:20.003+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:20.046 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:20.046 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:20.063 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:07:20.114 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:20.115+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:20.195 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:20.196 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:20.200 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:20.199+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:20.211 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:07:20.261 INFO:teuthology.orchestra.run.smithi176.stdout:leaving image image_client.0-clone37 intact 2024-09-14T09:07:20.301 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:20.303+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:20.318 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6] out_osds: [7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:07:20.319 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:07:20.319 INFO:tasks.thrashosds.thrasher:Adding osd 7 2024-09-14T09:07:20.319 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 7 2024-09-14T09:07:20.347 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:07:20.380 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:20.380 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:20.400 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:07:20.401 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:20.403+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:20.502 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:20.503+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:20.522 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:20.522 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:20.541 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-14T09:07:20.545 INFO:teuthology.orchestra.run.smithi176.stdout:5368 write 0x973c8d3 thru 0x9747e9b (0xb5c9 bytes) 2024-09-14T09:07:20.555 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:20.555 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:20.573 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:20.573 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:20.573 INFO:teuthology.orchestra.run.smithi176.stdout:5372 read 0x437760c thru 0x43869e5 (0xf3da bytes) 2024-09-14T09:07:20.574 INFO:teuthology.orchestra.run.smithi176.stdout:5374 read 0xdd36b1 thru 0xdd5bff (0x254f bytes) 2024-09-14T09:07:20.576 INFO:teuthology.orchestra.run.smithi176.stdout:5375 writesame 0xa16d60 thru 0xa25a35 data_size 0x5e(0xecd6 bytes) 2024-09-14T09:07:20.589 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:20.589 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:20.600 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:20.600 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:20.600 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:20.600 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:20.600 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:20.601 INFO:teuthology.orchestra.run.smithi176.stdout:5377 write 0xd47da4d thru 0xd48a27e (0xc832 bytes) 2024-09-14T09:07:20.603 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:20.603+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:20.615 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:20.615 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:20.620 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:20.620 INFO:teuthology.orchestra.run.smithi176.stdout:5382 read 0x72bef6c thru 0x72c7fc8 (0x905d bytes) 2024-09-14T09:07:20.620 INFO:teuthology.orchestra.run.smithi176.stdout:5383 punch from 0xccf3178 to 0xccfd076, (0x9efe bytes) 2024-09-14T09:07:20.620 INFO:teuthology.orchestra.run.smithi176.stdout:5384 read 0xaf1fd54 thru 0xaf239e7 (0x3c94 bytes) 2024-09-14T09:07:20.621 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:07:20.621 INFO:teuthology.orchestra.run.smithi176.stdout:5385 write 0xa5f3646 thru 0xa5f4c07 (0x15c2 bytes) 2024-09-14T09:07:20.637 INFO:teuthology.orchestra.run.smithi176.stdout:5388 read 0xd63a6eb thru 0xd641aaf (0x73c5 bytes) 2024-09-14T09:07:20.637 INFO:teuthology.orchestra.run.smithi176.stdout:5389 write 0x20f78d6 thru 0x21066f7 (0xee22 bytes) 2024-09-14T09:07:20.641 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:07:20.641 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:07:20.657 INFO:teuthology.orchestra.run.smithi176.stdout:5390 writesame 0x2acfe2 thru 0x2ad3a0 data_size 0x1(0x3bf bytes) 2024-09-14T09:07:20.677 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:20.677 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:20.704 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:20.703+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:20.705 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:20.706 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:20.722 INFO:tasks.ceph.mon.a.smithi176.stderr:2024-09-14T09:07:20.723+0000 7f9a2f8af640 -1 mon.a@0(leader).osd e111 definitely_dead 0 2024-09-14T09:07:20.732 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:20.732 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:20.787 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:18383 3.9 3:9252f7c9:::rbd_data.115e3abd3b2c.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T09:07:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:07:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 97.629585571000007, 2024-09-14T09:07:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.056433830999999997, 2024-09-14T09:07:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.788 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 18383 2024-09-14T09:07:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:07:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158422+0000", 2024-09-14T09:07:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158423+0000", 2024-09-14T09:07:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2729999999999999e-06 2024-09-14T09:07:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158429+0000", 2024-09-14T09:07:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0179999999999998e-06 2024-09-14T09:07:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158440+0000", 2024-09-14T09:07:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0769e-05 2024-09-14T09:07:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158445+0000", 2024-09-14T09:07:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5840000000000002e-06 2024-09-14T09:07:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158495+0000", 2024-09-14T09:07:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9129999999999999e-05 2024-09-14T09:07:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.158598+0000", 2024-09-14T09:07:20.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000102938 2024-09-14T09:07:20.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:20.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.161048+0000", 2024-09-14T09:07:20.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002450336 2024-09-14T09:07:20.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:20.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214605+0000", 2024-09-14T09:07:20.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.053557526000000001 2024-09-14T09:07:20.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:20.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214634+0000", 2024-09-14T09:07:20.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8483000000000002e-05 2024-09-14T09:07:20.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:20.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214731+0000", 2024-09-14T09:07:20.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7026000000000002e-05 2024-09-14T09:07:20.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214856+0000", 2024-09-14T09:07:20.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012474799999999999 2024-09-14T09:07:20.794 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.794 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.794 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.794 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:18383 3.9 e38/34 ondisk, result = 0)", 2024-09-14T09:07:20.794 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:07:20.794 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 97.619269090000003, 2024-09-14T09:07:20.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045880575999999999, 2024-09-14T09:07:20.794 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.795 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.795 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:20.795 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:07:20.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168738+0000", 2024-09-14T09:07:20.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168739+0000", 2024-09-14T09:07:20.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3900000000000004e-07 2024-09-14T09:07:20.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168744+0000", 2024-09-14T09:07:20.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5929999999999999e-06 2024-09-14T09:07:20.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168754+0000", 2024-09-14T09:07:20.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0686999999999999e-05 2024-09-14T09:07:20.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.168761+0000", 2024-09-14T09:07:20.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6340000000000003e-06 2024-09-14T09:07:20.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214578+0000", 2024-09-14T09:07:20.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.045817443999999999 2024-09-14T09:07:20.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214596+0000", 2024-09-14T09:07:20.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7159e-05 2024-09-14T09:07:20.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:43.214619+0000", 2024-09-14T09:07:20.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.332e-05 2024-09-14T09:07:20.798 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.798 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.798 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.798 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:19152 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14159058~8940] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T09:07:20.799 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:07:20.799 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 96.465356634000003, 2024-09-14T09:07:20.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12891994500000001, 2024-09-14T09:07:20.799 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.799 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.799 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:20.799 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:20.799 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:20.799 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:20.799 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 19152 2024-09-14T09:07:20.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.799 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:07:20.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322651+0000", 2024-09-14T09:07:20.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.322653+0000", 2024-09-14T09:07:20.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3750000000000001e-06 2024-09-14T09:07:20.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329647+0000", 2024-09-14T09:07:20.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069941960000000003 2024-09-14T09:07:20.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329673+0000", 2024-09-14T09:07:20.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5760000000000001e-05 2024-09-14T09:07:20.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329684+0000", 2024-09-14T09:07:20.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0474e-05 2024-09-14T09:07:20.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329776+0000", 2024-09-14T09:07:20.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2467999999999996e-05 2024-09-14T09:07:20.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.329987+0000", 2024-09-14T09:07:20.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021133 2024-09-14T09:07:20.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:20.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.331303+0000", 2024-09-14T09:07:20.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001315629 2024-09-14T09:07:20.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:20.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.411090+0000", 2024-09-14T09:07:20.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.079787477999999995 2024-09-14T09:07:20.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:20.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451463+0000", 2024-09-14T09:07:20.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040372312 2024-09-14T09:07:20.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:20.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451523+0000", 2024-09-14T09:07:20.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0420000000000001e-05 2024-09-14T09:07:20.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:44.451571+0000", 2024-09-14T09:07:20.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7503000000000002e-05 2024-09-14T09:07:20.805 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.805 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.805 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.805 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20926 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 786432~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:20.805 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:20.805 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 93.116024633999999, 2024-09-14T09:07:20.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018572207, 2024-09-14T09:07:20.806 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.806 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.806 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:20.806 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:20.806 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:20.806 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:20.806 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20926 2024-09-14T09:07:20.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.806 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:20.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:20.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671983+0000", 2024-09-14T09:07:20.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:07:20.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.671990+0000", 2024-09-14T09:07:20.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5769999999999999e-06 2024-09-14T09:07:20.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672001+0000", 2024-09-14T09:07:20.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1293e-05 2024-09-14T09:07:20.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672008+0000", 2024-09-14T09:07:20.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9099999999999999e-06 2024-09-14T09:07:20.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689147+0000", 2024-09-14T09:07:20.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.017139060000000001 2024-09-14T09:07:20.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.689324+0000", 2024-09-14T09:07:20.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017640200000000001 2024-09-14T09:07:20.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690555+0000", 2024-09-14T09:07:20.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001231212 2024-09-14T09:07:20.810 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.810 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.810 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.810 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20927 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2621440~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:20.810 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:20.810 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 93.115987668000002, 2024-09-14T09:07:20.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020002227000000001, 2024-09-14T09:07:20.810 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.810 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.810 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:20.811 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:20.811 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:20.811 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:20.811 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20927 2024-09-14T09:07:20.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.811 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:20.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:20.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672020+0000", 2024-09-14T09:07:20.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9699999999999995e-07 2024-09-14T09:07:20.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672024+0000", 2024-09-14T09:07:20.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:07:20.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672036+0000", 2024-09-14T09:07:20.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.119e-05 2024-09-14T09:07:20.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672044+0000", 2024-09-14T09:07:20.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8370000000000003e-06 2024-09-14T09:07:20.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690602+0000", 2024-09-14T09:07:20.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.018557648 2024-09-14T09:07:20.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.690805+0000", 2024-09-14T09:07:20.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020282899999999999 2024-09-14T09:07:20.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692022+0000", 2024-09-14T09:07:20.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012170429999999999 2024-09-14T09:07:20.814 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.814 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.814 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.815 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20928 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 917504~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:20.815 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:07:20.815 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 93.115951527999997, 2024-09-14T09:07:20.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021623738, 2024-09-14T09:07:20.815 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.815 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.815 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:20.815 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:20.815 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:20.815 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:20.815 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20928 2024-09-14T09:07:20.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.815 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:07:20.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672056+0000", 2024-09-14T09:07:20.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672057+0000", 2024-09-14T09:07:20.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:07:20.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672059+0000", 2024-09-14T09:07:20.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9270000000000001e-06 2024-09-14T09:07:20.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672070+0000", 2024-09-14T09:07:20.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0773e-05 2024-09-14T09:07:20.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672077+0000", 2024-09-14T09:07:20.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9600000000000003e-06 2024-09-14T09:07:20.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692154+0000", 2024-09-14T09:07:20.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.020076482 2024-09-14T09:07:20.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.692419+0000", 2024-09-14T09:07:20.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026517400000000002 2024-09-14T09:07:20.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693680+0000", 2024-09-14T09:07:20.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0012607390000000001 2024-09-14T09:07:20.819 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.820 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.820 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.820 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20929 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2752512~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:20.820 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:07:20.820 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 93.115918485999998, 2024-09-14T09:07:20.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022898728, 2024-09-14T09:07:20.820 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.821 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.821 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:20.821 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:20.821 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:20.821 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:20.821 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20929 2024-09-14T09:07:20.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.821 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:07:20.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672089+0000", 2024-09-14T09:07:20.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672090+0000", 2024-09-14T09:07:20.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:07:20.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672092+0000", 2024-09-14T09:07:20.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8940000000000002e-06 2024-09-14T09:07:20.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672103+0000", 2024-09-14T09:07:20.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0815999999999999e-05 2024-09-14T09:07:20.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672110+0000", 2024-09-14T09:07:20.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9469999999999997e-06 2024-09-14T09:07:20.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693737+0000", 2024-09-14T09:07:20.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.021626467999999999 2024-09-14T09:07:20.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.693962+0000", 2024-09-14T09:07:20.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225474 2024-09-14T09:07:20.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.694988+0000", 2024-09-14T09:07:20.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001025469 2024-09-14T09:07:20.825 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.825 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.825 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.825 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20930 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1048576~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:20.826 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:07:20.826 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 93.115885391999996, 2024-09-14T09:07:20.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024303286, 2024-09-14T09:07:20.826 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.826 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.826 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:20.826 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:20.826 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:20.826 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:20.826 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20930 2024-09-14T09:07:20.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.826 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:07:20.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672122+0000", 2024-09-14T09:07:20.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672123+0000", 2024-09-14T09:07:20.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:07:20.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672125+0000", 2024-09-14T09:07:20.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7300000000000001e-06 2024-09-14T09:07:20.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672134+0000", 2024-09-14T09:07:20.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7930000000000003e-06 2024-09-14T09:07:20.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672141+0000", 2024-09-14T09:07:20.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9029999999999998e-06 2024-09-14T09:07:20.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695087+0000", 2024-09-14T09:07:20.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022946013000000001 2024-09-14T09:07:20.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.695350+0000", 2024-09-14T09:07:20.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026266000000000001 2024-09-14T09:07:20.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696425+0000", 2024-09-14T09:07:20.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001075527 2024-09-14T09:07:20.830 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.830 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.830 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.830 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20931 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 2883584~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:20.830 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:07:20.830 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 93.115852163, 2024-09-14T09:07:20.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025331519, 2024-09-14T09:07:20.830 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.830 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.831 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:20.831 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:20.831 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:20.831 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:20.831 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20931 2024-09-14T09:07:20.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.831 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:07:20.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672155+0000", 2024-09-14T09:07:20.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672156+0000", 2024-09-14T09:07:20.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4300000000000002e-07 2024-09-14T09:07:20.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672159+0000", 2024-09-14T09:07:20.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.694e-06 2024-09-14T09:07:20.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672170+0000", 2024-09-14T09:07:20.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1192999999999999e-05 2024-09-14T09:07:20.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672177+0000", 2024-09-14T09:07:20.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9059999999999996e-06 2024-09-14T09:07:20.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696532+0000", 2024-09-14T09:07:20.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.024355291000000001 2024-09-14T09:07:20.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.696755+0000", 2024-09-14T09:07:20.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022297699999999999 2024-09-14T09:07:20.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697487+0000", 2024-09-14T09:07:20.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00073171499999999997 2024-09-14T09:07:20.834 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.835 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.835 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.835 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20932 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1179648~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:20.835 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:20.835 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 93.115818719999993, 2024-09-14T09:07:20.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026104083, 2024-09-14T09:07:20.835 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.835 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.835 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:20.835 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:20.835 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:20.835 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:20.835 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20932 2024-09-14T09:07:20.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.836 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:20.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:20.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672189+0000", 2024-09-14T09:07:20.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0699999999999996e-07 2024-09-14T09:07:20.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672192+0000", 2024-09-14T09:07:20.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7700000000000002e-06 2024-09-14T09:07:20.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672203+0000", 2024-09-14T09:07:20.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0956e-05 2024-09-14T09:07:20.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672211+0000", 2024-09-14T09:07:20.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3769999999999996e-06 2024-09-14T09:07:20.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697530+0000", 2024-09-14T09:07:20.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.025318411999999998 2024-09-14T09:07:20.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.697640+0000", 2024-09-14T09:07:20.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011045299999999999 2024-09-14T09:07:20.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698293+0000", 2024-09-14T09:07:20.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00065240800000000005 2024-09-14T09:07:20.839 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.839 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.839 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.839 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20933 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3014656~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:20.839 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:07:20.839 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 93.115784516999994, 2024-09-14T09:07:20.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026558875999999999, 2024-09-14T09:07:20.840 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.840 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.840 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:20.840 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:20.840 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:20.840 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:20.840 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20933 2024-09-14T09:07:20.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.840 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:07:20.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672223+0000", 2024-09-14T09:07:20.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672224+0000", 2024-09-14T09:07:20.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2399999999999997e-07 2024-09-14T09:07:20.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672234+0000", 2024-09-14T09:07:20.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0793e-05 2024-09-14T09:07:20.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672246+0000", 2024-09-14T09:07:20.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1659e-05 2024-09-14T09:07:20.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672255+0000", 2024-09-14T09:07:20.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5730000000000008e-06 2024-09-14T09:07:20.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698326+0000", 2024-09-14T09:07:20.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026071414000000001 2024-09-14T09:07:20.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698374+0000", 2024-09-14T09:07:20.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8032999999999999e-05 2024-09-14T09:07:20.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698782+0000", 2024-09-14T09:07:20.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040768 2024-09-14T09:07:20.843 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.843 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.844 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.844 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20934 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1310720~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:20.844 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:07:20.844 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 93.115742640999997, 2024-09-14T09:07:20.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027292067999999999, 2024-09-14T09:07:20.844 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.844 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.844 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:20.844 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:20.844 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:20.844 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:20.844 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20934 2024-09-14T09:07:20.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.845 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:07:20.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672265+0000", 2024-09-14T09:07:20.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672266+0000", 2024-09-14T09:07:20.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6299999999999993e-07 2024-09-14T09:07:20.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672269+0000", 2024-09-14T09:07:20.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8569999999999999e-06 2024-09-14T09:07:20.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672280+0000", 2024-09-14T09:07:20.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1236999999999999e-05 2024-09-14T09:07:20.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672287+0000", 2024-09-14T09:07:20.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8260000000000004e-06 2024-09-14T09:07:20.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698820+0000", 2024-09-14T09:07:20.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026533055999999999 2024-09-14T09:07:20.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.698933+0000", 2024-09-14T09:07:20.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011291 2024-09-14T09:07:20.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699557+0000", 2024-09-14T09:07:20.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062421899999999997 2024-09-14T09:07:20.848 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.848 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.848 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.849 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20935 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3145728~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:20.849 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:07:20.849 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 93.115708881000003, 2024-09-14T09:07:20.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028108610999999999, 2024-09-14T09:07:20.849 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.849 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.849 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:20.849 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:20.849 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:20.849 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:20.849 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20935 2024-09-14T09:07:20.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.849 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:07:20.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672298+0000", 2024-09-14T09:07:20.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672299+0000", 2024-09-14T09:07:20.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9299999999999997e-07 2024-09-14T09:07:20.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672302+0000", 2024-09-14T09:07:20.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7640000000000001e-06 2024-09-14T09:07:20.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672313+0000", 2024-09-14T09:07:20.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1080000000000001e-05 2024-09-14T09:07:20.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672321+0000", 2024-09-14T09:07:20.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6490000000000008e-06 2024-09-14T09:07:20.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699587+0000", 2024-09-14T09:07:20.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.027265884000000001 2024-09-14T09:07:20.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.699631+0000", 2024-09-14T09:07:20.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4947999999999997e-05 2024-09-14T09:07:20.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700407+0000", 2024-09-14T09:07:20.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00077559299999999999 2024-09-14T09:07:20.853 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.853 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.853 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.853 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20936 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1441792~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:20.853 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:20.853 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 93.115674705000004, 2024-09-14T09:07:20.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028784750000000001, 2024-09-14T09:07:20.854 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.854 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.854 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:20.854 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:20.854 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:20.854 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:20.854 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20936 2024-09-14T09:07:20.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.854 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:20.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:20.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672333+0000", 2024-09-14T09:07:20.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4700000000000001e-07 2024-09-14T09:07:20.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672336+0000", 2024-09-14T09:07:20.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6730000000000001e-06 2024-09-14T09:07:20.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672347+0000", 2024-09-14T09:07:20.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0710000000000001e-05 2024-09-14T09:07:20.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672354+0000", 2024-09-14T09:07:20.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4630000000000004e-06 2024-09-14T09:07:20.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700442+0000", 2024-09-14T09:07:20.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028087951 2024-09-14T09:07:20.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.700557+0000", 2024-09-14T09:07:20.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011461800000000001 2024-09-14T09:07:20.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701117+0000", 2024-09-14T09:07:20.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056058799999999999 2024-09-14T09:07:20.857 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.858 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.858 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.858 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20937 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3276800~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:20.858 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:20.858 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 93.115641578999998, 2024-09-14T09:07:20.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029493028000000001, 2024-09-14T09:07:20.858 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.858 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.858 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:20.859 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:20.859 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:20.859 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:20.859 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20937 2024-09-14T09:07:20.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.859 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:20.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:20.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672366+0000", 2024-09-14T09:07:20.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6700000000000003e-07 2024-09-14T09:07:20.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672384+0000", 2024-09-14T09:07:20.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7127000000000001e-05 2024-09-14T09:07:20.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672395+0000", 2024-09-14T09:07:20.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1759999999999999e-05 2024-09-14T09:07:20.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672402+0000", 2024-09-14T09:07:20.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0960000000000003e-06 2024-09-14T09:07:20.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701147+0000", 2024-09-14T09:07:20.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.028744241 2024-09-14T09:07:20.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701246+0000", 2024-09-14T09:07:20.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9031999999999996e-05 2024-09-14T09:07:20.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701859+0000", 2024-09-14T09:07:20.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061310499999999997 2024-09-14T09:07:20.862 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.862 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.862 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.862 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20938 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 1572864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:20.862 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:07:20.862 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 93.115594419000004, 2024-09-14T09:07:20.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030109219999999999, 2024-09-14T09:07:20.863 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.863 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.863 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:20.863 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:20.863 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:20.863 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:20.863 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20938 2024-09-14T09:07:20.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.863 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:07:20.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672413+0000", 2024-09-14T09:07:20.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672414+0000", 2024-09-14T09:07:20.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-07 2024-09-14T09:07:20.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672417+0000", 2024-09-14T09:07:20.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:07:20.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672428+0000", 2024-09-14T09:07:20.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0873000000000001e-05 2024-09-14T09:07:20.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672435+0000", 2024-09-14T09:07:20.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:07:20.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701884+0000", 2024-09-14T09:07:20.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.029449811999999999 2024-09-14T09:07:20.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.701934+0000", 2024-09-14T09:07:20.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9703000000000002e-05 2024-09-14T09:07:20.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702522+0000", 2024-09-14T09:07:20.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00058811200000000001 2024-09-14T09:07:20.866 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.866 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.866 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.867 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:20939 3.9 3:979bccc1:::rbd_data.115e4c71d6e8.0000000000000000:9 [sparse-read 3407872~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-14T09:07:20.867 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:07:20.867 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 93.115562285999999, 2024-09-14T09:07:20.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030779259, 2024-09-14T09:07:20.867 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.867 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.867 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:20.867 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:20.867 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:20.867 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:20.867 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 20939 2024-09-14T09:07:20.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.867 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:07:20.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672445+0000", 2024-09-14T09:07:20.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672446+0000", 2024-09-14T09:07:20.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:07:20.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672449+0000", 2024-09-14T09:07:20.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7930000000000002e-06 2024-09-14T09:07:20.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672461+0000", 2024-09-14T09:07:20.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1960000000000001e-05 2024-09-14T09:07:20.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.672471+0000", 2024-09-14T09:07:20.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0807e-05 2024-09-14T09:07:20.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702551+0000", 2024-09-14T09:07:20.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.030079228 2024-09-14T09:07:20.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.702664+0000", 2024-09-14T09:07:20.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011351800000000001 2024-09-14T09:07:20.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:47.703224+0000", 2024-09-14T09:07:20.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00056026999999999997 2024-09-14T09:07:20.870 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.871 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.871 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.871 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:74298 3.9 3:95c8282a:::rbd_data.115ea8d131ff.0000000000000041:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-14T09:07:20.871 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:07:20.871 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 44.990252273999999, 2024-09-14T09:07:20.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.042668064999999998, 2024-09-14T09:07:20.871 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.871 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.871 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:20.871 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:20.871 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:20.871 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:20.871 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 74298 2024-09-14T09:07:20.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.872 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:07:20.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797755+0000", 2024-09-14T09:07:20.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797758+0000", 2024-09-14T09:07:20.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9799999999999998e-06 2024-09-14T09:07:20.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797770+0000", 2024-09-14T09:07:20.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1667e-05 2024-09-14T09:07:20.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797797+0000", 2024-09-14T09:07:20.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7212999999999998e-05 2024-09-14T09:07:20.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797811+0000", 2024-09-14T09:07:20.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4256e-05 2024-09-14T09:07:20.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797882+0000", 2024-09-14T09:07:20.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0622000000000002e-05 2024-09-14T09:07:20.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.797943+0000", 2024-09-14T09:07:20.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0893000000000001e-05 2024-09-14T09:07:20.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:20.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.799676+0000", 2024-09-14T09:07:20.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0017331619999999999 2024-09-14T09:07:20.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:20.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840109+0000", 2024-09-14T09:07:20.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.040432994 2024-09-14T09:07:20.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:20.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840204+0000", 2024-09-14T09:07:20.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5247000000000005e-05 2024-09-14T09:07:20.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:20.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840346+0000", 2024-09-14T09:07:20.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014156499999999999 2024-09-14T09:07:20.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840423+0000", 2024-09-14T09:07:20.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7466000000000007e-05 2024-09-14T09:07:20.876 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.877 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.877 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.877 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:74298 3.9 e76/34 ondisk, result = 0)", 2024-09-14T09:07:20.877 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:07:20.877 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 44.983857630999999, 2024-09-14T09:07:20.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.036289318000000001, 2024-09-14T09:07:20.877 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.877 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.877 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:20.877 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:07:20.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804150+0000", 2024-09-14T09:07:20.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804151+0000", 2024-09-14T09:07:20.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6700000000000001e-06 2024-09-14T09:07:20.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804163+0000", 2024-09-14T09:07:20.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.148e-05 2024-09-14T09:07:20.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804197+0000", 2024-09-14T09:07:20.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3909999999999999e-05 2024-09-14T09:07:20.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.804217+0000", 2024-09-14T09:07:20.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0287000000000001e-05 2024-09-14T09:07:20.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840176+0000", 2024-09-14T09:07:20.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.035958641 2024-09-14T09:07:20.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840193+0000", 2024-09-14T09:07:20.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6986e-05 2024-09-14T09:07:20.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.840439+0000", 2024-09-14T09:07:20.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024634400000000001 2024-09-14T09:07:20.880 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.880 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.881 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.881 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:93074 3.9 e94/84 3:925bb5bd:::rbd_data.115e7770ddc5.0000000000000001:head v 94'255, mlcod=94'252)", 2024-09-14T09:07:20.881 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:57.863727+0000", 2024-09-14T09:07:20.881 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 22.924280371000002, 2024-09-14T09:07:20.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.019329799000000002, 2024-09-14T09:07:20.881 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:20.881 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:20.881 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:20.881 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:20.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:20.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.863727+0000", 2024-09-14T09:07:20.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:20.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.863727+0000", 2024-09-14T09:07:20.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:20.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:20.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.863729+0000", 2024-09-14T09:07:20.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3980000000000002e-06 2024-09-14T09:07:20.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:20.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866073+0000", 2024-09-14T09:07:20.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0023435109999999999 2024-09-14T09:07:20.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:20.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866094+0000", 2024-09-14T09:07:20.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1444e-05 2024-09-14T09:07:20.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:20.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866105+0000", 2024-09-14T09:07:20.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0949e-05 2024-09-14T09:07:20.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:20.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866216+0000", 2024-09-14T09:07:20.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000110961 2024-09-14T09:07:20.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:20.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.866258+0000", 2024-09-14T09:07:20.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1903000000000002e-05 2024-09-14T09:07:20.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:20.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.882981+0000", 2024-09-14T09:07:20.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.016723198000000002 2024-09-14T09:07:20.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:20.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:20.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:20.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:57.883057+0000", 2024-09-14T09:07:20.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5434999999999999e-05 2024-09-14T09:07:20.885 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.885 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.885 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.885 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:20.885 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:20.885 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:20.886 INFO:teuthology.orchestra.run.smithi176.stdout:5391 write 0xa2837f7 thru 0xa28e6a9 (0xaeb3 bytes) 2024-09-14T09:07:20.887 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:07:20.888 INFO:teuthology.orchestra.run.smithi176.stdout:5392 write 0x22a8df3 thru 0x22b7352 (0xe560 bytes) 2024-09-14T09:07:20.888 INFO:teuthology.orchestra.run.smithi176.stdout:5395 trunc from 0xeda163f to 0x39cb2d4 2024-09-14T09:07:20.888 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:20.891+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:20.894 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:20.909 INFO:teuthology.orchestra.run.smithi176.stdout:5398 write 0x5ad6ce1 thru 0x5adf7c9 (0x8ae9 bytes) 2024-09-14T09:07:20.939 INFO:teuthology.orchestra.run.smithi176.stdout:5399 write 0x8c497e4 thru 0x8c501b3 (0x69d0 bytes) 2024-09-14T09:07:20.957 INFO:teuthology.orchestra.run.smithi176.stdout:5400 trunc from 0x8c501b4 to 0xc2f7ee2 2024-09-14T09:07:20.969 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:20.969 INFO:teuthology.orchestra.run.smithi176.stdout:5403 write 0x6a0825 thru 0x6af1bf (0xe99b bytes) 2024-09-14T09:07:20.987 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:20.991+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:21.037 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:21.037 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:21.037 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:21.037 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:21.053 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:07:21.070 INFO:teuthology.orchestra.run.smithi176.stderr:marked in osd.7. 2024-09-14T09:07:21.071 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T09:07:21.102 INFO:tasks.thrashosds.thrasher:Added osd 7 2024-09-14T09:07:21.103 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:21.103+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:21.189 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:21.187+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:21.212 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:21.213 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:21.213 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:21.213 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:21.213 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:21.228 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:07:21.318 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:21.319+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:21.383 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:21.383 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:21.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:21.383 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:21.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.384 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:21.384 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:21.384 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 108.299089755, 2024-09-14T09:07:21.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:07:21.384 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.384 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.384 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:21.384 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:21.384 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:21.384 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.384 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:07:21.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.384 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:21.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:21.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:07:21.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:07:21.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:07:21.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:07:21.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:07:21.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:07:21.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:07:21.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:07:21.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:07:21.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:07:21.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:21.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:07:21.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:07:21.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:07:21.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:07:21.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:21.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:07:21.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:07:21.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:07:21.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:07:21.388 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.388 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.388 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.389 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:21.389 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:21.389 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 108.299035097, 2024-09-14T09:07:21.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:07:21.389 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.389 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.389 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:21.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:21.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:21.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.389 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:07:21.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.390 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:21.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:21.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:21.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:07:21.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:07:21.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:07:21.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:07:21.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:07:21.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:07:21.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:07:21.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:07:21.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:07:21.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:21.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:07:21.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:07:21.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:07:21.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:07:21.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:21.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:07:21.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:07:21.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:07:21.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:07:21.393 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.393 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.393 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.394 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:21.394 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:21.394 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 108.298929723, 2024-09-14T09:07:21.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:07:21.394 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.394 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.394 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:21.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:21.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:21.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.394 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:07:21.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.394 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:21.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:21.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:07:21.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:07:21.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:07:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:07:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:07:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:07:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:07:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:07:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:07:21.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:07:21.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:21.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:07:21.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:07:21.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:07:21.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:07:21.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:21.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:07:21.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:07:21.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:07:21.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:07:21.398 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.398 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.398 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.398 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:21.398 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:21.399 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 108.298572959, 2024-09-14T09:07:21.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:07:21.399 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.399 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.399 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:21.399 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:21.399 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:21.399 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.399 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:07:21.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.399 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:21.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:21.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:07:21.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:07:21.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:07:21.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:07:21.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:07:21.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:07:21.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:07:21.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:07:21.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:07:21.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:07:21.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:21.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:07:21.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:07:21.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:07:21.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:07:21.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:21.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:07:21.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:07:21.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:07:21.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:07:21.403 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.403 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.403 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.404 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:21.404 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:21.404 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 108.298488221, 2024-09-14T09:07:21.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:07:21.404 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.404 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.404 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:21.404 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:21.404 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:21.404 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.404 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:07:21.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.405 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:21.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:21.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:07:21.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:07:21.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:07:21.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:07:21.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:07:21.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:07:21.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:07:21.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:07:21.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:07:21.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:07:21.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:21.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:07:21.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:07:21.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:07:21.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:07:21.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:21.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:07:21.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:07:21.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:07:21.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:07:21.408 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.409 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.409 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.409 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:21.409 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:21.409 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 108.298400535, 2024-09-14T09:07:21.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:07:21.409 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.409 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.409 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:21.409 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:21.409 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:21.409 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.410 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:07:21.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.410 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:21.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:21.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:07:21.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:07:21.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:07:21.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:07:21.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:07:21.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:07:21.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:07:21.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:07:21.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:07:21.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:07:21.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:21.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:07:21.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:07:21.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:07:21.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:07:21.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:21.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:07:21.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:07:21.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:07:21.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:07:21.414 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.414 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.414 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.414 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:07:21.414 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:21.414 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.222108547999994, 2024-09-14T09:07:21.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:07:21.414 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.414 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.414 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:21.414 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:21.414 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:21.415 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.415 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:07:21.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.415 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:21.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:21.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:07:21.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:07:21.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:07:21.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:07:21.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:07:21.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:07:21.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:07:21.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:07:21.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:07:21.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:07:21.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:21.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:07:21.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:07:21.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:21.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:07:21.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:07:21.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:21.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:07:21.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:07:21.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:21.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:07:21.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:07:21.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:21.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:07:21.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:07:21.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:07:21.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:07:21.422 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.422 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.422 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.422 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:39401 3.b e56/53)", 2024-09-14T09:07:21.423 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:07:21.423 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.196212482000007, 2024-09-14T09:07:21.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10443659600000001, 2024-09-14T09:07:21.423 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.423 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.423 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:21.423 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:07:21.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186508+0000", 2024-09-14T09:07:21.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186509+0000", 2024-09-14T09:07:21.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2640000000000001e-06 2024-09-14T09:07:21.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186516+0000", 2024-09-14T09:07:21.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1070000000000001e-06 2024-09-14T09:07:21.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186533+0000", 2024-09-14T09:07:21.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7818000000000001e-05 2024-09-14T09:07:21.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.186543+0000", 2024-09-14T09:07:21.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5170000000000005e-06 2024-09-14T09:07:21.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290885+0000", 2024-09-14T09:07:21.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.104342221 2024-09-14T09:07:21.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:21.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290893+0000", 2024-09-14T09:07:21.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2859999999999999e-06 2024-09-14T09:07:21.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290945+0000", 2024-09-14T09:07:21.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1382999999999999e-05 2024-09-14T09:07:21.426 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.426 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.426 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.427 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:84137 3.7 3:e15b49d9:::rbd_data.115e77b799c3.00000000000000e0:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:07:21.427 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:21.427 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 28.059361150000001, 2024-09-14T09:07:21.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11205707500000001, 2024-09-14T09:07:21.427 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.427 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.427 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:21.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:21.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:21.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.427 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 84137 2024-09-14T09:07:21.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.427 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:21.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:21.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323362+0000", 2024-09-14T09:07:21.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0839999999999999e-06 2024-09-14T09:07:21.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323370+0000", 2024-09-14T09:07:21.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0180000000000001e-06 2024-09-14T09:07:21.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323388+0000", 2024-09-14T09:07:21.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8326999999999999e-05 2024-09-14T09:07:21.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323397+0000", 2024-09-14T09:07:21.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1940000000000004e-06 2024-09-14T09:07:21.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323452+0000", 2024-09-14T09:07:21.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5043000000000001e-05 2024-09-14T09:07:21.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:21.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323550+0000", 2024-09-14T09:07:21.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8201000000000004e-05 2024-09-14T09:07:21.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:21.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.328273+0000", 2024-09-14T09:07:21.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0047227079999999999 2024-09-14T09:07:21.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:21.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435232+0000", 2024-09-14T09:07:21.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.106959339 2024-09-14T09:07:21.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:21.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435256+0000", 2024-09-14T09:07:21.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3774e-05 2024-09-14T09:07:21.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:21.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435314+0000", 2024-09-14T09:07:21.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7636999999999997e-05 2024-09-14T09:07:21.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435417+0000", 2024-09-14T09:07:21.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010275 2024-09-14T09:07:21.432 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.432 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.432 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.433 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:97796 3.15 3:aaaa5f7f:::rbd_data.115e113dc8ca.0000000000000029:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:07:21.433 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:21.433 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.2882396780000001, 2024-09-14T09:07:21.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14537731300000001, 2024-09-14T09:07:21.433 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.433 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.433 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:21.433 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:21.433 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:21.433 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.433 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97796 2024-09-14T09:07:21.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.434 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:21.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:21.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094482+0000", 2024-09-14T09:07:21.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6400000000000001e-07 2024-09-14T09:07:21.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094485+0000", 2024-09-14T09:07:21.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.693e-06 2024-09-14T09:07:21.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094494+0000", 2024-09-14T09:07:21.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9590000000000001e-06 2024-09-14T09:07:21.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094502+0000", 2024-09-14T09:07:21.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3180000000000001e-06 2024-09-14T09:07:21.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094567+0000", 2024-09-14T09:07:21.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5526999999999998e-05 2024-09-14T09:07:21.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:21.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094726+0000", 2024-09-14T09:07:21.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015842499999999999 2024-09-14T09:07:21.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:07:21.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.101940+0000", 2024-09-14T09:07:21.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0072146559999999998 2024-09-14T09:07:21.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:21.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239665+0000", 2024-09-14T09:07:21.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.137724501 2024-09-14T09:07:21.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:21.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239726+0000", 2024-09-14T09:07:21.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1550999999999994e-05 2024-09-14T09:07:21.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:21.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239800+0000", 2024-09-14T09:07:21.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3318000000000005e-05 2024-09-14T09:07:21.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239858+0000", 2024-09-14T09:07:21.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8600999999999997e-05 2024-09-14T09:07:21.438 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.438 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.439 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.439 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:97796 3.15 e106/103 ondisk, result = 0)", 2024-09-14T09:07:21.439 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:21.439 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 9.2658938089999996, 2024-09-14T09:07:21.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123043465, 2024-09-14T09:07:21.439 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.439 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.439 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:21.439 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:21.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:21.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116828+0000", 2024-09-14T09:07:21.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8100000000000002e-07 2024-09-14T09:07:21.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116831+0000", 2024-09-14T09:07:21.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3009999999999998e-06 2024-09-14T09:07:21.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116841+0000", 2024-09-14T09:07:21.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7829999999999993e-06 2024-09-14T09:07:21.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116851+0000", 2024-09-14T09:07:21.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0135999999999999e-05 2024-09-14T09:07:21.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239707+0000", 2024-09-14T09:07:21.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12285581499999999 2024-09-14T09:07:21.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:21.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239718+0000", 2024-09-14T09:07:21.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1804000000000001e-05 2024-09-14T09:07:21.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239870+0000", 2024-09-14T09:07:21.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015194500000000001 2024-09-14T09:07:21.443 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.443 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.443 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.443 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:98161 3.15 3:ac845c17:::rbd_data.115e113dc8ca.0000000000000021:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-14T09:07:21.443 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:13.429237+0000", 2024-09-14T09:07:21.443 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.9534838209999998, 2024-09-14T09:07:21.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.108985944, 2024-09-14T09:07:21.443 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.443 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.444 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:21.444 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:21.444 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:21.444 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.444 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 98161 2024-09-14T09:07:21.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.444 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.429237+0000", 2024-09-14T09:07:21.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.429237+0000", 2024-09-14T09:07:21.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.429238+0000", 2024-09-14T09:07:21.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.328e-06 2024-09-14T09:07:21.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.429245+0000", 2024-09-14T09:07:21.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4799999999999998e-06 2024-09-14T09:07:21.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.429261+0000", 2024-09-14T09:07:21.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6623e-05 2024-09-14T09:07:21.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.429270+0000", 2024-09-14T09:07:21.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9989999999999997e-06 2024-09-14T09:07:21.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.429332+0000", 2024-09-14T09:07:21.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1890000000000005e-05 2024-09-14T09:07:21.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:21.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.429449+0000", 2024-09-14T09:07:21.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00011655 2024-09-14T09:07:21.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:07:21.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.436399+0000", 2024-09-14T09:07:21.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0069505249999999999 2024-09-14T09:07:21.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:21.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.538131+0000", 2024-09-14T09:07:21.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.101731998 2024-09-14T09:07:21.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:21.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.538147+0000", 2024-09-14T09:07:21.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5449999999999999e-05 2024-09-14T09:07:21.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:21.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.538186+0000", 2024-09-14T09:07:21.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9280999999999999e-05 2024-09-14T09:07:21.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:13.538223+0000", 2024-09-14T09:07:21.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6820000000000003e-05 2024-09-14T09:07:21.449 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.449 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.449 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.449 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:104747 3.18 3.1979ee18 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.449 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:20.667183+0000", 2024-09-14T09:07:21.449 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.71553751700000001, 2024-09-14T09:07:21.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095131387999999997, 2024-09-14T09:07:21.449 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.450 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.450 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:21.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:21.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:21.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.450 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104747 2024-09-14T09:07:21.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.450 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667183+0000", 2024-09-14T09:07:21.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667183+0000", 2024-09-14T09:07:21.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667183+0000", 2024-09-14T09:07:21.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.53e-07 2024-09-14T09:07:21.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667185+0000", 2024-09-14T09:07:21.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7850000000000001e-06 2024-09-14T09:07:21.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667192+0000", 2024-09-14T09:07:21.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0010000000000004e-06 2024-09-14T09:07:21.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667197+0000", 2024-09-14T09:07:21.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4739999999999999e-06 2024-09-14T09:07:21.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.762305+0000", 2024-09-14T09:07:21.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095107918999999999 2024-09-14T09:07:21.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.762315+0000", 2024-09-14T09:07:21.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8560000000000002e-06 2024-09-14T09:07:21.453 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.453 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.453 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.453 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:104749 3.18 3.1979ee18 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.453 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:20.667204+0000", 2024-09-14T09:07:21.454 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.71551622999999998, 2024-09-14T09:07:21.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095150767999999997, 2024-09-14T09:07:21.454 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.454 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.454 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:21.454 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:21.454 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:21.454 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.454 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104749 2024-09-14T09:07:21.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.454 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667204+0000", 2024-09-14T09:07:21.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667204+0000", 2024-09-14T09:07:21.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667205+0000", 2024-09-14T09:07:21.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2599999999999998e-07 2024-09-14T09:07:21.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667207+0000", 2024-09-14T09:07:21.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.863e-06 2024-09-14T09:07:21.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667214+0000", 2024-09-14T09:07:21.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3000000000000004e-06 2024-09-14T09:07:21.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667220+0000", 2024-09-14T09:07:21.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8769999999999998e-06 2024-09-14T09:07:21.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.762347+0000", 2024-09-14T09:07:21.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095126768 2024-09-14T09:07:21.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.762355+0000", 2024-09-14T09:07:21.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5339999999999999e-06 2024-09-14T09:07:21.457 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.457 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.457 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.458 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:104751 3.18 3.1979ee18 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.458 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:20.667235+0000", 2024-09-14T09:07:21.458 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.715486026, 2024-09-14T09:07:21.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095164886000000004, 2024-09-14T09:07:21.458 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.458 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.458 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:21.458 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:21.458 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:21.458 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.458 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104751 2024-09-14T09:07:21.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.458 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667235+0000", 2024-09-14T09:07:21.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667235+0000", 2024-09-14T09:07:21.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667235+0000", 2024-09-14T09:07:21.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3500000000000002e-07 2024-09-14T09:07:21.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667242+0000", 2024-09-14T09:07:21.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4120000000000004e-06 2024-09-14T09:07:21.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667251+0000", 2024-09-14T09:07:21.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9630000000000004e-06 2024-09-14T09:07:21.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667263+0000", 2024-09-14T09:07:21.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1557e-05 2024-09-14T09:07:21.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.762389+0000", 2024-09-14T09:07:21.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095126191999999998 2024-09-14T09:07:21.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.762399+0000", 2024-09-14T09:07:21.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0327e-05 2024-09-14T09:07:21.461 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.461 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.462 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.462 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:104753 3.18 3.1979ee18 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.462 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:20.667272+0000", 2024-09-14T09:07:21.462 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.71544823499999999, 2024-09-14T09:07:21.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095167333000000007, 2024-09-14T09:07:21.462 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.462 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.462 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:21.462 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:21.462 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:21.462 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.462 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104753 2024-09-14T09:07:21.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.463 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667272+0000", 2024-09-14T09:07:21.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667272+0000", 2024-09-14T09:07:21.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667273+0000", 2024-09-14T09:07:21.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7100000000000002e-07 2024-09-14T09:07:21.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667275+0000", 2024-09-14T09:07:21.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.526e-06 2024-09-14T09:07:21.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667284+0000", 2024-09-14T09:07:21.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1799999999999996e-06 2024-09-14T09:07:21.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667289+0000", 2024-09-14T09:07:21.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1680000000000003e-06 2024-09-14T09:07:21.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.762431+0000", 2024-09-14T09:07:21.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095142240000000003 2024-09-14T09:07:21.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.762440+0000", 2024-09-14T09:07:21.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7479999999999999e-06 2024-09-14T09:07:21.466 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.466 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.466 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.466 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:104755 3.18 3.1979ee18 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.466 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:20.667297+0000", 2024-09-14T09:07:21.466 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.71542387299999999, 2024-09-14T09:07:21.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095184426000000003, 2024-09-14T09:07:21.466 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.466 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.466 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:21.467 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:21.467 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:21.467 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.467 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104755 2024-09-14T09:07:21.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.467 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667297+0000", 2024-09-14T09:07:21.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667297+0000", 2024-09-14T09:07:21.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667297+0000", 2024-09-14T09:07:21.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5999999999999999e-07 2024-09-14T09:07:21.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667299+0000", 2024-09-14T09:07:21.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9649999999999998e-06 2024-09-14T09:07:21.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667307+0000", 2024-09-14T09:07:21.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.447e-06 2024-09-14T09:07:21.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667311+0000", 2024-09-14T09:07:21.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5079999999999999e-06 2024-09-14T09:07:21.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.762471+0000", 2024-09-14T09:07:21.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095160119000000001 2024-09-14T09:07:21.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.762481+0000", 2024-09-14T09:07:21.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.927e-06 2024-09-14T09:07:21.470 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.470 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.470 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.470 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:104756 3.18 3.1979ee18 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.470 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:20.667320+0000", 2024-09-14T09:07:21.470 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.71540109500000004, 2024-09-14T09:07:21.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095205128, 2024-09-14T09:07:21.470 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.470 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.470 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:21.471 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:21.471 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:21.471 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.471 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104756 2024-09-14T09:07:21.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.471 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667320+0000", 2024-09-14T09:07:21.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667320+0000", 2024-09-14T09:07:21.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667320+0000", 2024-09-14T09:07:21.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.08e-07 2024-09-14T09:07:21.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667322+0000", 2024-09-14T09:07:21.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.951e-06 2024-09-14T09:07:21.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667330+0000", 2024-09-14T09:07:21.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6070000000000002e-06 2024-09-14T09:07:21.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667334+0000", 2024-09-14T09:07:21.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8729999999999998e-06 2024-09-14T09:07:21.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.762516+0000", 2024-09-14T09:07:21.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095181630000000003 2024-09-14T09:07:21.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.762525+0000", 2024-09-14T09:07:21.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6589999999999996e-06 2024-09-14T09:07:21.474 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.474 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.474 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.474 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:104759 3.18 3.1979ee18 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.474 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:20.667343+0000", 2024-09-14T09:07:21.475 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.71537784000000004, 2024-09-14T09:07:21.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095228342999999993, 2024-09-14T09:07:21.475 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.475 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.475 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:21.475 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:21.475 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:21.475 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.475 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104759 2024-09-14T09:07:21.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.475 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667343+0000", 2024-09-14T09:07:21.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667343+0000", 2024-09-14T09:07:21.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667343+0000", 2024-09-14T09:07:21.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.58e-07 2024-09-14T09:07:21.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667345+0000", 2024-09-14T09:07:21.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.968e-06 2024-09-14T09:07:21.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667353+0000", 2024-09-14T09:07:21.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.435e-06 2024-09-14T09:07:21.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667358+0000", 2024-09-14T09:07:21.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8609999999999997e-06 2024-09-14T09:07:21.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.762560+0000", 2024-09-14T09:07:21.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095202568000000001 2024-09-14T09:07:21.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.762571+0000", 2024-09-14T09:07:21.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1053000000000001e-05 2024-09-14T09:07:21.478 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.478 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.478 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.479 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:104761 3.18 3.1979ee18 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.479 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:20.667366+0000", 2024-09-14T09:07:21.479 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.71535462900000002, 2024-09-14T09:07:21.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095249173000000006, 2024-09-14T09:07:21.479 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:21.479 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:21.479 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:21.479 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:21.479 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:21.479 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.479 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 104761 2024-09-14T09:07:21.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.479 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:21.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:21.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667366+0000", 2024-09-14T09:07:21.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:21.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667366+0000", 2024-09-14T09:07:21.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:21.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:21.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667366+0000", 2024-09-14T09:07:21.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4999999999999998e-07 2024-09-14T09:07:21.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:21.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667368+0000", 2024-09-14T09:07:21.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9489999999999999e-06 2024-09-14T09:07:21.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:21.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667376+0000", 2024-09-14T09:07:21.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.52e-06 2024-09-14T09:07:21.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.667381+0000", 2024-09-14T09:07:21.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7509999999999999e-06 2024-09-14T09:07:21.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:21.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.762605+0000", 2024-09-14T09:07:21.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095224759000000006 2024-09-14T09:07:21.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:21.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:21.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:21.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:20.762615+0000", 2024-09-14T09:07:21.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7440000000000002e-06 2024-09-14T09:07:21.482 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.482 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.483 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.483 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:21.483 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:21.483 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:21.484 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:07:21.485 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:21.487+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:21.586 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:21.587+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:21.616 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:21.617 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:21.617 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:07:21.617 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:21.631 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:07:21.687 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:21.687+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:21.747 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:21.747 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:21.747 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:07:21.747 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:07:21.748 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:21.761 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:07:21.811 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:21.815+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:21.832 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is unset 2024-09-14T09:07:21.888 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:21.891+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:21.904 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:21.904 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:07:21.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:07:21.905 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:07:21.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.905 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:76803 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-14T09:07:21.905 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:43.858877+0000", 2024-09-14T09:07:21.905 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.049636986000003, 2024-09-14T09:07:21.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0137577900000001, 2024-09-14T09:07:21.905 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:21.905 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:21.905 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:21.905 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:21.905 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:21.906 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.906 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 76803 2024-09-14T09:07:21.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.906 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:21.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:21.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858877+0000", 2024-09-14T09:07:21.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:21.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858877+0000", 2024-09-14T09:07:21.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:21.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858879+0000", 2024-09-14T09:07:21.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.097e-06 2024-09-14T09:07:21.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:21.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858887+0000", 2024-09-14T09:07:21.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6510000000000001e-06 2024-09-14T09:07:21.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:21.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858903+0000", 2024-09-14T09:07:21.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6560999999999999e-05 2024-09-14T09:07:21.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858912+0000", 2024-09-14T09:07:21.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2019999999999993e-06 2024-09-14T09:07:21.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858964+0000", 2024-09-14T09:07:21.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2046999999999998e-05 2024-09-14T09:07:21.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:21.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858972+0000", 2024-09-14T09:07:21.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6029999999999999e-06 2024-09-14T09:07:21.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:44.871832+0000", 2024-09-14T09:07:21.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.012860109 2024-09-14T09:07:21.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:21.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:44.872248+0000", 2024-09-14T09:07:21.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00041549300000000001 2024-09-14T09:07:21.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:21.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:44.872635+0000", 2024-09-14T09:07:21.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038702700000000002 2024-09-14T09:07:21.911 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.911 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:21.911 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.911 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:84403 3.14 3:2fc85a9a:::rbd_data.115e77b799c3.0000000000000018:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:07:21.911 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:53.878069+0000", 2024-09-14T09:07:21.911 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 28.030445116999999, 2024-09-14T09:07:21.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.128852405, 2024-09-14T09:07:21.911 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:21.911 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:21.911 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:21.912 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:21.912 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:21.912 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.912 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 84403 2024-09-14T09:07:21.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.912 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:21.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:21.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878069+0000", 2024-09-14T09:07:21.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:21.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878069+0000", 2024-09-14T09:07:21.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:21.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878071+0000", 2024-09-14T09:07:21.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0899999999999999e-06 2024-09-14T09:07:21.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:21.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878082+0000", 2024-09-14T09:07:21.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1226999999999999e-05 2024-09-14T09:07:21.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:21.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878108+0000", 2024-09-14T09:07:21.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5568999999999999e-05 2024-09-14T09:07:21.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878122+0000", 2024-09-14T09:07:21.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.488e-05 2024-09-14T09:07:21.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878247+0000", 2024-09-14T09:07:21.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000124039 2024-09-14T09:07:21.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:21.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878387+0000", 2024-09-14T09:07:21.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014041600000000001 2024-09-14T09:07:21.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:21.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.886261+0000", 2024-09-14T09:07:21.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0078738499999999999 2024-09-14T09:07:21.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:21.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006598+0000", 2024-09-14T09:07:21.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.120337316 2024-09-14T09:07:21.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:21.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006724+0000", 2024-09-14T09:07:21.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012601600000000001 2024-09-14T09:07:21.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:21.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006827+0000", 2024-09-14T09:07:21.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103299 2024-09-14T09:07:21.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:21.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006921+0000", 2024-09-14T09:07:21.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3702999999999994e-05 2024-09-14T09:07:21.918 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.918 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:21.918 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.918 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:93272 3.3 3:c49b4fbc:::rbd_data.115e7770ddc5.000000000000001a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:07:21.918 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.971357+0000", 2024-09-14T09:07:21.918 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 23.937157058, 2024-09-14T09:07:21.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.112268252, 2024-09-14T09:07:21.918 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:21.918 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:21.918 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:21.918 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:21.919 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:21.919 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.919 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 93272 2024-09-14T09:07:21.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.919 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:21.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:21.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971357+0000", 2024-09-14T09:07:21.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:21.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971357+0000", 2024-09-14T09:07:21.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:21.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971360+0000", 2024-09-14T09:07:21.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8250000000000001e-06 2024-09-14T09:07:21.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:21.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971371+0000", 2024-09-14T09:07:21.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1537e-05 2024-09-14T09:07:21.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:21.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971399+0000", 2024-09-14T09:07:21.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8274999999999999e-05 2024-09-14T09:07:21.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971417+0000", 2024-09-14T09:07:21.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7241999999999999e-05 2024-09-14T09:07:21.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971537+0000", 2024-09-14T09:07:21.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012010600000000001 2024-09-14T09:07:21.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:21.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971685+0000", 2024-09-14T09:07:21.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014777599999999999 2024-09-14T09:07:21.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:21.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.979172+0000", 2024-09-14T09:07:21.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0074876810000000004 2024-09-14T09:07:21.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:21.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083438+0000", 2024-09-14T09:07:21.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.104265969 2024-09-14T09:07:21.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:21.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083466+0000", 2024-09-14T09:07:21.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7665999999999999e-05 2024-09-14T09:07:21.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:21.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083553+0000", 2024-09-14T09:07:21.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6809000000000002e-05 2024-09-14T09:07:21.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:21.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083625+0000", 2024-09-14T09:07:21.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2366000000000005e-05 2024-09-14T09:07:21.924 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.924 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:21.924 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.925 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:96694 3.11 3:8a7e3c58:::rbd_data.115e63832e35.000000000000001c:head [stat,truncate 131072] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-14T09:07:21.925 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:07.004845+0000", 2024-09-14T09:07:21.925 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 14.903669042000001, 2024-09-14T09:07:21.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.85224771499999996, 2024-09-14T09:07:21.925 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:21.925 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:21.925 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:21.925 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:21.925 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:21.925 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.925 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 96694 2024-09-14T09:07:21.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.925 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:21.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:21.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004845+0000", 2024-09-14T09:07:21.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:21.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004845+0000", 2024-09-14T09:07:21.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:21.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004848+0000", 2024-09-14T09:07:21.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:07:21.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:21.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004862+0000", 2024-09-14T09:07:21.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4426e-05 2024-09-14T09:07:21.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:21.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004903+0000", 2024-09-14T09:07:21.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0447000000000001e-05 2024-09-14T09:07:21.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004926+0000", 2024-09-14T09:07:21.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3260000000000001e-05 2024-09-14T09:07:21.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.005104+0000", 2024-09-14T09:07:21.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000178018 2024-09-14T09:07:21.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:21.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.005120+0000", 2024-09-14T09:07:21.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6422999999999998e-05 2024-09-14T09:07:21.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.851775+0000", 2024-09-14T09:07:21.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.84665435499999997 2024-09-14T09:07:21.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:21.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.852176+0000", 2024-09-14T09:07:21.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000400787 2024-09-14T09:07:21.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-09-14T09:07:21.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.852662+0000", 2024-09-14T09:07:21.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048621899999999997 2024-09-14T09:07:21.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:21.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.855261+0000", 2024-09-14T09:07:21.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025991479999999999 2024-09-14T09:07:21.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:21.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.856839+0000", 2024-09-14T09:07:21.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015779819999999999 2024-09-14T09:07:21.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:21.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.856993+0000", 2024-09-14T09:07:21.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015431900000000001 2024-09-14T09:07:21.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:21.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.857093+0000", 2024-09-14T09:07:21.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9314000000000005e-05 2024-09-14T09:07:21.931 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.931 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:21.932 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.932 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:97612 3.e 3:7298f82d:::rbd_data.115e113dc8ca.00000000000000e8:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:07:21.932 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:11.399529+0000", 2024-09-14T09:07:21.932 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.508984989, 2024-09-14T09:07:21.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.18555535000000001, 2024-09-14T09:07:21.932 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:21.932 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:21.932 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:21.932 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:21.932 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:21.933 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.933 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 97612 2024-09-14T09:07:21.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.933 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:21.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:21.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399529+0000", 2024-09-14T09:07:21.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:21.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399529+0000", 2024-09-14T09:07:21.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:21.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399532+0000", 2024-09-14T09:07:21.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6599999999999999e-06 2024-09-14T09:07:21.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:21.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399544+0000", 2024-09-14T09:07:21.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2527000000000001e-05 2024-09-14T09:07:21.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:21.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399574+0000", 2024-09-14T09:07:21.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.97e-05 2024-09-14T09:07:21.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399588+0000", 2024-09-14T09:07:21.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4653e-05 2024-09-14T09:07:21.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399740+0000", 2024-09-14T09:07:21.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015180900000000001 2024-09-14T09:07:21.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:21.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399952+0000", 2024-09-14T09:07:21.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000211435 2024-09-14T09:07:21.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:21.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.411686+0000", 2024-09-14T09:07:21.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011733883000000001 2024-09-14T09:07:21.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:21.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584791+0000", 2024-09-14T09:07:21.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.17310596 2024-09-14T09:07:21.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:21.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584915+0000", 2024-09-14T09:07:21.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012330800000000001 2024-09-14T09:07:21.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:21.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.585010+0000", 2024-09-14T09:07:21.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5216000000000005e-05 2024-09-14T09:07:21.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:21.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.585084+0000", 2024-09-14T09:07:21.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4198999999999999e-05 2024-09-14T09:07:21.938 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.938 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:21.938 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.938 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:97612 3.e e106/103 ondisk, result = 0)", 2024-09-14T09:07:21.938 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:11.451700+0000", 2024-09-14T09:07:21.939 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.456813694999999, 2024-09-14T09:07:21.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13339934000000001, 2024-09-14T09:07:21.939 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:21.939 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:21.939 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:21.939 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:21.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:21.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451700+0000", 2024-09-14T09:07:21.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:21.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451700+0000", 2024-09-14T09:07:21.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:21.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451702+0000", 2024-09-14T09:07:21.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.34e-06 2024-09-14T09:07:21.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:21.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451713+0000", 2024-09-14T09:07:21.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1e-05 2024-09-14T09:07:21.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:21.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451751+0000", 2024-09-14T09:07:21.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7762999999999997e-05 2024-09-14T09:07:21.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451770+0000", 2024-09-14T09:07:21.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8626000000000001e-05 2024-09-14T09:07:21.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584888+0000", 2024-09-14T09:07:21.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13311764500000001 2024-09-14T09:07:21.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:21.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584903+0000", 2024-09-14T09:07:21.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.499e-05 2024-09-14T09:07:21.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:21.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.585099+0000", 2024-09-14T09:07:21.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019697599999999999 2024-09-14T09:07:21.942 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.942 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:21.943 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.943 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104478 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 470016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.943 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680713+0000", 2024-09-14T09:07:21.943 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.2278006880000001, 2024-09-14T09:07:21.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10242654800000001, 2024-09-14T09:07:21.943 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:21.943 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:21.943 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:21.943 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:21.943 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:21.943 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.944 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104478 2024-09-14T09:07:21.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.944 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:21.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:21.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680713+0000", 2024-09-14T09:07:21.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:21.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680713+0000", 2024-09-14T09:07:21.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:21.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680714+0000", 2024-09-14T09:07:21.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3300000000000001e-07 2024-09-14T09:07:21.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:21.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680717+0000", 2024-09-14T09:07:21.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:07:21.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:21.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680729+0000", 2024-09-14T09:07:21.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1643e-05 2024-09-14T09:07:21.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680736+0000", 2024-09-14T09:07:21.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8730000000000007e-06 2024-09-14T09:07:21.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.782300+0000", 2024-09-14T09:07:21.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10156356499999999 2024-09-14T09:07:21.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:21.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.782436+0000", 2024-09-14T09:07:21.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013629899999999999 2024-09-14T09:07:21.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:21.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.783140+0000", 2024-09-14T09:07:21.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070341800000000003 2024-09-14T09:07:21.947 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.947 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:21.947 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.947 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104480 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 474112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.947 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680747+0000", 2024-09-14T09:07:21.948 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.2277672150000001, 2024-09-14T09:07:21.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.103294211, 2024-09-14T09:07:21.948 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:21.948 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:21.948 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:21.948 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:21.948 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:21.948 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.948 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104480 2024-09-14T09:07:21.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.948 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:21.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:21.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680747+0000", 2024-09-14T09:07:21.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:21.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680747+0000", 2024-09-14T09:07:21.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:21.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680747+0000", 2024-09-14T09:07:21.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3300000000000001e-07 2024-09-14T09:07:21.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:21.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680750+0000", 2024-09-14T09:07:21.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9160000000000001e-06 2024-09-14T09:07:21.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:21.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680759+0000", 2024-09-14T09:07:21.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8699999999999998e-06 2024-09-14T09:07:21.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680766+0000", 2024-09-14T09:07:21.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1339999999999998e-06 2024-09-14T09:07:21.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.783218+0000", 2024-09-14T09:07:21.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.102451925 2024-09-14T09:07:21.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:21.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.783350+0000", 2024-09-14T09:07:21.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013207200000000001 2024-09-14T09:07:21.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:21.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.784041+0000", 2024-09-14T09:07:21.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069056100000000002 2024-09-14T09:07:21.952 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.952 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:21.952 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.952 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104482 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 478208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.952 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680777+0000", 2024-09-14T09:07:21.952 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.227737326, 2024-09-14T09:07:21.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.104207155, 2024-09-14T09:07:21.953 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:21.953 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:21.953 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:21.953 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:21.953 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:21.953 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.953 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104482 2024-09-14T09:07:21.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.953 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:21.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:21.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680777+0000", 2024-09-14T09:07:21.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:21.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680777+0000", 2024-09-14T09:07:21.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:21.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680777+0000", 2024-09-14T09:07:21.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8700000000000005e-07 2024-09-14T09:07:21.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:21.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680801+0000", 2024-09-14T09:07:21.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3963000000000001e-05 2024-09-14T09:07:21.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:21.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680832+0000", 2024-09-14T09:07:21.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1029999999999999e-05 2024-09-14T09:07:21.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680841+0000", 2024-09-14T09:07:21.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3030000000000005e-06 2024-09-14T09:07:21.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.784136+0000", 2024-09-14T09:07:21.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.103294752 2024-09-14T09:07:21.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:21.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.784275+0000", 2024-09-14T09:07:21.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013825900000000001 2024-09-14T09:07:21.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:21.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.784984+0000", 2024-09-14T09:07:21.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070916100000000004 2024-09-14T09:07:21.957 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.957 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:21.957 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.957 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104484 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 482304~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.957 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680855+0000", 2024-09-14T09:07:21.957 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.2276588399999999, 2024-09-14T09:07:21.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.105039178, 2024-09-14T09:07:21.957 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:21.957 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:21.957 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:21.958 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:21.958 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:21.958 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.958 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104484 2024-09-14T09:07:21.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.958 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:21.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:21.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680855+0000", 2024-09-14T09:07:21.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:21.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680855+0000", 2024-09-14T09:07:21.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:21.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680856+0000", 2024-09-14T09:07:21.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3399999999999997e-07 2024-09-14T09:07:21.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:21.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680859+0000", 2024-09-14T09:07:21.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4800000000000001e-06 2024-09-14T09:07:21.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:21.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680872+0000", 2024-09-14T09:07:21.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2918999999999999e-05 2024-09-14T09:07:21.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680880+0000", 2024-09-14T09:07:21.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9370000000000004e-06 2024-09-14T09:07:21.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.785063+0000", 2024-09-14T09:07:21.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.104182498 2024-09-14T09:07:21.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:21.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.785202+0000", 2024-09-14T09:07:21.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000139362 2024-09-14T09:07:21.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:21.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.785894+0000", 2024-09-14T09:07:21.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069204800000000001 2024-09-14T09:07:21.961 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.962 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:21.962 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.962 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104486 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 486400~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.962 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680891+0000", 2024-09-14T09:07:21.962 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.2276232, 2024-09-14T09:07:21.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.105913091, 2024-09-14T09:07:21.962 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:21.962 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:21.962 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:21.962 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:21.962 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:21.962 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.963 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104486 2024-09-14T09:07:21.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.963 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:21.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:21.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680891+0000", 2024-09-14T09:07:21.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:21.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680891+0000", 2024-09-14T09:07:21.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:21.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680891+0000", 2024-09-14T09:07:21.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1299999999999999e-07 2024-09-14T09:07:21.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:21.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680894+0000", 2024-09-14T09:07:21.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0199999999999999e-06 2024-09-14T09:07:21.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:21.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680904+0000", 2024-09-14T09:07:21.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2140000000000002e-06 2024-09-14T09:07:21.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680911+0000", 2024-09-14T09:07:21.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1130000000000003e-06 2024-09-14T09:07:21.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.785969+0000", 2024-09-14T09:07:21.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.105058734 2024-09-14T09:07:21.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:21.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.786104+0000", 2024-09-14T09:07:21.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013463900000000001 2024-09-14T09:07:21.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:21.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.786804+0000", 2024-09-14T09:07:21.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069965800000000003 2024-09-14T09:07:21.967 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.967 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:21.967 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.967 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104488 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 490496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.967 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680925+0000", 2024-09-14T09:07:21.967 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.227588997, 2024-09-14T09:07:21.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10678225700000001, 2024-09-14T09:07:21.967 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:21.968 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:21.968 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:21.968 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:21.968 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:21.968 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.968 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104488 2024-09-14T09:07:21.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.968 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:21.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.968 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:21.968 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680925+0000", 2024-09-14T09:07:21.968 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.969 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:21.969 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680925+0000", 2024-09-14T09:07:21.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.969 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:21.969 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680926+0000", 2024-09-14T09:07:21.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-07 2024-09-14T09:07:21.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.969 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:21.969 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680928+0000", 2024-09-14T09:07:21.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8870000000000001e-06 2024-09-14T09:07:21.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:21.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680940+0000", 2024-09-14T09:07:21.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1717e-05 2024-09-14T09:07:21.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680948+0000", 2024-09-14T09:07:21.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8429999999999993e-06 2024-09-14T09:07:21.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.786883+0000", 2024-09-14T09:07:21.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10593530399999999 2024-09-14T09:07:21.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:21.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.787015+0000", 2024-09-14T09:07:21.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013217200000000001 2024-09-14T09:07:21.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:21.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.787707+0000", 2024-09-14T09:07:21.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069162399999999995 2024-09-14T09:07:21.972 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.972 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:21.972 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.972 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104490 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 494592~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.972 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680960+0000", 2024-09-14T09:07:21.972 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.2275536599999999, 2024-09-14T09:07:21.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.107672767, 2024-09-14T09:07:21.972 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:21.972 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:21.973 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:21.973 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:21.973 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:21.973 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.973 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104490 2024-09-14T09:07:21.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.973 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:21.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:21.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680960+0000", 2024-09-14T09:07:21.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:21.974 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680960+0000", 2024-09-14T09:07:21.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:21.974 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680961+0000", 2024-09-14T09:07:21.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-07 2024-09-14T09:07:21.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:21.974 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680964+0000", 2024-09-14T09:07:21.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3859999999999999e-06 2024-09-14T09:07:21.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:21.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680974+0000", 2024-09-14T09:07:21.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3700000000000001e-06 2024-09-14T09:07:21.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680981+0000", 2024-09-14T09:07:21.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1230000000000002e-06 2024-09-14T09:07:21.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.787780+0000", 2024-09-14T09:07:21.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.106799291 2024-09-14T09:07:21.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:21.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.787914+0000", 2024-09-14T09:07:21.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013379200000000001 2024-09-14T09:07:21.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:21.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.788633+0000", 2024-09-14T09:07:21.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000719095 2024-09-14T09:07:21.976 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.977 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:21.977 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.977 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104493 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 498688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.977 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681058+0000", 2024-09-14T09:07:21.977 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.227456141, 2024-09-14T09:07:21.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.108480707, 2024-09-14T09:07:21.977 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:21.977 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:21.977 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:21.977 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:21.977 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:21.977 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.977 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104493 2024-09-14T09:07:21.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.978 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:21.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:21.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681058+0000", 2024-09-14T09:07:21.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:21.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681058+0000", 2024-09-14T09:07:21.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:21.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681059+0000", 2024-09-14T09:07:21.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1200000000000001e-06 2024-09-14T09:07:21.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:21.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681062+0000", 2024-09-14T09:07:21.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3100000000000001e-06 2024-09-14T09:07:21.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:21.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681078+0000", 2024-09-14T09:07:21.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5630000000000001e-05 2024-09-14T09:07:21.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.980 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681096+0000", 2024-09-14T09:07:21.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8036000000000001e-05 2024-09-14T09:07:21.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.980 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.980 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.788719+0000", 2024-09-14T09:07:21.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.107623651 2024-09-14T09:07:21.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.980 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:21.980 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.788851+0000", 2024-09-14T09:07:21.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000131402 2024-09-14T09:07:21.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:21.981 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.789538+0000", 2024-09-14T09:07:21.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068755800000000001 2024-09-14T09:07:21.981 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.981 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:21.981 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.981 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104494 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 502784~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.981 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681108+0000", 2024-09-14T09:07:21.982 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.227405378, 2024-09-14T09:07:21.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10933915700000001, 2024-09-14T09:07:21.982 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:21.982 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:21.982 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:21.982 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:21.982 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:21.982 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.982 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104494 2024-09-14T09:07:21.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.982 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:21.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:21.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681108+0000", 2024-09-14T09:07:21.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:21.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681108+0000", 2024-09-14T09:07:21.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:21.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681109+0000", 2024-09-14T09:07:21.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0999999999999997e-07 2024-09-14T09:07:21.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:21.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681113+0000", 2024-09-14T09:07:21.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4659999999999999e-06 2024-09-14T09:07:21.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:21.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681122+0000", 2024-09-14T09:07:21.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5140000000000007e-06 2024-09-14T09:07:21.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681129+0000", 2024-09-14T09:07:21.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1130000000000003e-06 2024-09-14T09:07:21.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.789610+0000", 2024-09-14T09:07:21.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10848097700000001 2024-09-14T09:07:21.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:21.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.789748+0000", 2024-09-14T09:07:21.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013734899999999999 2024-09-14T09:07:21.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:21.986 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.790448+0000", 2024-09-14T09:07:21.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069992800000000005 2024-09-14T09:07:21.986 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.986 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:21.986 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.986 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104496 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 506880~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.986 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681140+0000", 2024-09-14T09:07:21.986 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.2273742320000001, 2024-09-14T09:07:21.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.110205621, 2024-09-14T09:07:21.987 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:21.987 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:21.987 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:21.987 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:21.987 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:21.987 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.987 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104496 2024-09-14T09:07:21.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.987 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:21.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:21.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681140+0000", 2024-09-14T09:07:21.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:21.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681140+0000", 2024-09-14T09:07:21.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:21.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681141+0000", 2024-09-14T09:07:21.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8700000000000004e-07 2024-09-14T09:07:21.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:21.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681144+0000", 2024-09-14T09:07:21.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0259999999999999e-06 2024-09-14T09:07:21.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:21.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681153+0000", 2024-09-14T09:07:21.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9770000000000006e-06 2024-09-14T09:07:21.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681159+0000", 2024-09-14T09:07:21.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.72e-06 2024-09-14T09:07:21.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.790527+0000", 2024-09-14T09:07:21.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.109368144 2024-09-14T09:07:21.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:21.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.790660+0000", 2024-09-14T09:07:21.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013242500000000001 2024-09-14T09:07:21.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:21.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.791345+0000", 2024-09-14T09:07:21.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068534200000000003 2024-09-14T09:07:21.990 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.991 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:21.991 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.991 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104498 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 510976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.991 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681169+0000", 2024-09-14T09:07:21.991 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.2273444650000001, 2024-09-14T09:07:21.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.111099946, 2024-09-14T09:07:21.991 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:21.991 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:21.991 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:21.991 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:21.991 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:21.991 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.991 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104498 2024-09-14T09:07:21.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.992 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:21.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:21.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681169+0000", 2024-09-14T09:07:21.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:21.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681169+0000", 2024-09-14T09:07:21.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:21.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681170+0000", 2024-09-14T09:07:21.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9599999999999999e-07 2024-09-14T09:07:21.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:21.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681173+0000", 2024-09-14T09:07:21.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:07:21.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:21.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681182+0000", 2024-09-14T09:07:21.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8100000000000004e-06 2024-09-14T09:07:21.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681189+0000", 2024-09-14T09:07:21.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-06 2024-09-14T09:07:21.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.791421+0000", 2024-09-14T09:07:21.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11023223 2024-09-14T09:07:21.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:21.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.791553+0000", 2024-09-14T09:07:21.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013137899999999999 2024-09-14T09:07:21.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:21.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.792269+0000", 2024-09-14T09:07:21.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071678099999999999 2024-09-14T09:07:21.995 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.995 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:21.995 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:21.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.995 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104500 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 515072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:21.996 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681199+0000", 2024-09-14T09:07:21.996 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.2273149249999999, 2024-09-14T09:07:21.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.111976886, 2024-09-14T09:07:21.996 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:21.996 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:21.996 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:21.996 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:21.996 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:21.996 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:21.996 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104500 2024-09-14T09:07:21.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.996 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:21.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:21.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681199+0000", 2024-09-14T09:07:21.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:21.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681199+0000", 2024-09-14T09:07:21.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:21.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:21.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681200+0000", 2024-09-14T09:07:21.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3600000000000003e-07 2024-09-14T09:07:21.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:21.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681202+0000", 2024-09-14T09:07:21.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7870000000000002e-06 2024-09-14T09:07:21.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:21.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681211+0000", 2024-09-14T09:07:21.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7730000000000005e-06 2024-09-14T09:07:21.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:21.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681218+0000", 2024-09-14T09:07:21.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9670000000000004e-06 2024-09-14T09:07:21.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:21.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.792355+0000", 2024-09-14T09:07:21.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11113640599999999 2024-09-14T09:07:21.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:21.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:21.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:21.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.792486+0000", 2024-09-14T09:07:22.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013117900000000001 2024-09-14T09:07:22.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:22.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.793176+0000", 2024-09-14T09:07:22.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069003800000000002 2024-09-14T09:07:22.000 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.000 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.000 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.000 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104502 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 519168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:22.000 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681228+0000", 2024-09-14T09:07:22.000 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.227285706, 2024-09-14T09:07:22.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11285799000000001, 2024-09-14T09:07:22.001 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.001 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.001 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:22.001 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.001 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.001 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.001 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104502 2024-09-14T09:07:22.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.001 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681228+0000", 2024-09-14T09:07:22.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681228+0000", 2024-09-14T09:07:22.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681229+0000", 2024-09-14T09:07:22.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4399999999999999e-07 2024-09-14T09:07:22.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681232+0000", 2024-09-14T09:07:22.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8289999999999999e-06 2024-09-14T09:07:22.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681241+0000", 2024-09-14T09:07:22.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9239999999999996e-06 2024-09-14T09:07:22.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681248+0000", 2024-09-14T09:07:22.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2389999999999996e-06 2024-09-14T09:07:22.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.793253+0000", 2024-09-14T09:07:22.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11200555700000001 2024-09-14T09:07:22.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:22.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.793385+0000", 2024-09-14T09:07:22.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000131929 2024-09-14T09:07:22.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:22.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.794086+0000", 2024-09-14T09:07:22.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070076800000000005 2024-09-14T09:07:22.005 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.005 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.005 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.005 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104504 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 523264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:22.005 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681258+0000", 2024-09-14T09:07:22.005 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 1.2272554, 2024-09-14T09:07:22.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.113704203, 2024-09-14T09:07:22.005 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.005 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.005 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:22.005 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.005 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.006 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.006 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104504 2024-09-14T09:07:22.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.006 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681258+0000", 2024-09-14T09:07:22.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681258+0000", 2024-09-14T09:07:22.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681259+0000", 2024-09-14T09:07:22.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9999999999999997e-07 2024-09-14T09:07:22.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681271+0000", 2024-09-14T09:07:22.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1674e-05 2024-09-14T09:07:22.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681281+0000", 2024-09-14T09:07:22.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7100000000000002e-06 2024-09-14T09:07:22.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681288+0000", 2024-09-14T09:07:22.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2660000000000004e-06 2024-09-14T09:07:22.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.794140+0000", 2024-09-14T09:07:22.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.112852483 2024-09-14T09:07:22.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:22.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.794270+0000", 2024-09-14T09:07:22.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000129846 2024-09-14T09:07:22.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:22.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.794963+0000", 2024-09-14T09:07:22.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069252400000000003 2024-09-14T09:07:22.010 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.010 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.010 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.010 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.010 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.010 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:22.011 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:07:22.012 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:22.011+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:22.113 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:22.115+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:22.214 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:22.214 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:07:22.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.214 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105296 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.214 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841712+0000", 2024-09-14T09:07:22.214 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34811639500000002, 2024-09-14T09:07:22.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.348145437, 2024-09-14T09:07:22.215 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.215 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.215 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.215 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.215 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.215 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.215 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105296 2024-09-14T09:07:22.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.215 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841712+0000", 2024-09-14T09:07:22.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841712+0000", 2024-09-14T09:07:22.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841713+0000", 2024-09-14T09:07:22.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.540000000000001e-07 2024-09-14T09:07:22.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841716+0000", 2024-09-14T09:07:22.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.898e-06 2024-09-14T09:07:22.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841726+0000", 2024-09-14T09:07:22.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0210000000000001e-05 2024-09-14T09:07:22.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841735+0000", 2024-09-14T09:07:22.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5380000000000002e-06 2024-09-14T09:07:22.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842141+0000", 2024-09-14T09:07:22.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000406547 2024-09-14T09:07:22.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842147+0000", 2024-09-14T09:07:22.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3870000000000003e-06 2024-09-14T09:07:22.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857939+0000", 2024-09-14T09:07:22.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015792414000000001 2024-09-14T09:07:22.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857946+0000", 2024-09-14T09:07:22.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8040000000000004e-06 2024-09-14T09:07:22.219 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.219 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.219 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.219 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105361 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.219 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.845646+0000", 2024-09-14T09:07:22.219 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34418289200000002, 2024-09-14T09:07:22.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34429970300000001, 2024-09-14T09:07:22.219 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.220 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.220 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.220 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.220 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.220 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.220 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105361 2024-09-14T09:07:22.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.220 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845646+0000", 2024-09-14T09:07:22.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845646+0000", 2024-09-14T09:07:22.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845646+0000", 2024-09-14T09:07:22.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4399999999999999e-07 2024-09-14T09:07:22.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845650+0000", 2024-09-14T09:07:22.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1499999999999999e-06 2024-09-14T09:07:22.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845660+0000", 2024-09-14T09:07:22.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0849999999999999e-05 2024-09-14T09:07:22.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845669+0000", 2024-09-14T09:07:22.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8019999999999998e-06 2024-09-14T09:07:22.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845725+0000", 2024-09-14T09:07:22.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5658000000000001e-05 2024-09-14T09:07:22.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845730+0000", 2024-09-14T09:07:22.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9099999999999996e-06 2024-09-14T09:07:22.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860602+0000", 2024-09-14T09:07:22.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014871679 2024-09-14T09:07:22.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.224 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.224 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860607+0000", 2024-09-14T09:07:22.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.665e-06 2024-09-14T09:07:22.224 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.224 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.224 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.224 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105425 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.224 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.847934+0000", 2024-09-14T09:07:22.224 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34189483199999998, 2024-09-14T09:07:22.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.342087056, 2024-09-14T09:07:22.224 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.225 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.225 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.225 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.225 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.225 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.225 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105425 2024-09-14T09:07:22.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.225 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847934+0000", 2024-09-14T09:07:22.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847934+0000", 2024-09-14T09:07:22.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847935+0000", 2024-09-14T09:07:22.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9400000000000004e-07 2024-09-14T09:07:22.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847938+0000", 2024-09-14T09:07:22.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4659999999999999e-06 2024-09-14T09:07:22.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847949+0000", 2024-09-14T09:07:22.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0890000000000001e-05 2024-09-14T09:07:22.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847957+0000", 2024-09-14T09:07:22.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6039999999999993e-06 2024-09-14T09:07:22.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.848013+0000", 2024-09-14T09:07:22.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5173000000000002e-05 2024-09-14T09:07:22.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.848020+0000", 2024-09-14T09:07:22.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8719999999999996e-06 2024-09-14T09:07:22.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861799+0000", 2024-09-14T09:07:22.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013779392999999999 2024-09-14T09:07:22.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861808+0000", 2024-09-14T09:07:22.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.85e-06 2024-09-14T09:07:22.228 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.228 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.228 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.229 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105490 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.229 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857896+0000", 2024-09-14T09:07:22.229 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.331932897, 2024-09-14T09:07:22.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.332199093, 2024-09-14T09:07:22.229 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.229 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.229 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.230 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.230 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.230 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.230 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105490 2024-09-14T09:07:22.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.230 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857896+0000", 2024-09-14T09:07:22.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857896+0000", 2024-09-14T09:07:22.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857896+0000", 2024-09-14T09:07:22.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8599999999999998e-07 2024-09-14T09:07:22.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857899+0000", 2024-09-14T09:07:22.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9689999999999999e-06 2024-09-14T09:07:22.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857908+0000", 2024-09-14T09:07:22.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0669999999999998e-06 2024-09-14T09:07:22.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857916+0000", 2024-09-14T09:07:22.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4070000000000004e-06 2024-09-14T09:07:22.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863127+0000", 2024-09-14T09:07:22.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052110530000000002 2024-09-14T09:07:22.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863136+0000", 2024-09-14T09:07:22.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9910000000000008e-06 2024-09-14T09:07:22.236 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.236 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.236 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.237 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105553 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.237 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860716+0000", 2024-09-14T09:07:22.237 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32911243899999998, 2024-09-14T09:07:22.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32944050899999999, 2024-09-14T09:07:22.237 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.237 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.237 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.238 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.238 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.238 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.238 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105553 2024-09-14T09:07:22.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.238 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860716+0000", 2024-09-14T09:07:22.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860716+0000", 2024-09-14T09:07:22.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860717+0000", 2024-09-14T09:07:22.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9500000000000003e-07 2024-09-14T09:07:22.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860719+0000", 2024-09-14T09:07:22.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2819999999999999e-06 2024-09-14T09:07:22.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860726+0000", 2024-09-14T09:07:22.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.781e-06 2024-09-14T09:07:22.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860730+0000", 2024-09-14T09:07:22.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8019999999999999e-06 2024-09-14T09:07:22.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865013+0000", 2024-09-14T09:07:22.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042821580000000003 2024-09-14T09:07:22.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865019+0000", 2024-09-14T09:07:22.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3969999999999999e-06 2024-09-14T09:07:22.241 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.241 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.241 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.241 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105617 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.241 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866087+0000", 2024-09-14T09:07:22.241 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32374154999999999, 2024-09-14T09:07:22.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.324131377, 2024-09-14T09:07:22.241 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.241 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.241 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.241 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.242 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.242 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.242 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105617 2024-09-14T09:07:22.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.242 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866087+0000", 2024-09-14T09:07:22.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866087+0000", 2024-09-14T09:07:22.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866088+0000", 2024-09-14T09:07:22.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7999999999999993e-07 2024-09-14T09:07:22.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866092+0000", 2024-09-14T09:07:22.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4750000000000004e-06 2024-09-14T09:07:22.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866102+0000", 2024-09-14T09:07:22.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1619999999999997e-06 2024-09-14T09:07:22.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866109+0000", 2024-09-14T09:07:22.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1380000000000001e-06 2024-09-14T09:07:22.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866222+0000", 2024-09-14T09:07:22.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000113082 2024-09-14T09:07:22.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866237+0000", 2024-09-14T09:07:22.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5268999999999999e-05 2024-09-14T09:07:22.245 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.245 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.245 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.245 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105681 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.245 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868151+0000", 2024-09-14T09:07:22.245 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32167758400000002, 2024-09-14T09:07:22.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32212808999999998, 2024-09-14T09:07:22.245 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.245 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.245 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.245 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.246 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.246 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.246 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105681 2024-09-14T09:07:22.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.246 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868151+0000", 2024-09-14T09:07:22.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868151+0000", 2024-09-14T09:07:22.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868152+0000", 2024-09-14T09:07:22.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4470000000000001e-06 2024-09-14T09:07:22.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868165+0000", 2024-09-14T09:07:22.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2322e-05 2024-09-14T09:07:22.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868180+0000", 2024-09-14T09:07:22.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4779e-05 2024-09-14T09:07:22.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868193+0000", 2024-09-14T09:07:22.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3952e-05 2024-09-14T09:07:22.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868292+0000", 2024-09-14T09:07:22.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8925999999999994e-05 2024-09-14T09:07:22.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868301+0000", 2024-09-14T09:07:22.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2479999999999996e-06 2024-09-14T09:07:22.249 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.249 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.249 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.249 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105745 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.249 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.869531+0000", 2024-09-14T09:07:22.249 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32029719200000001, 2024-09-14T09:07:22.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32081153299999998, 2024-09-14T09:07:22.249 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.249 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.249 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.249 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.249 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.250 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.250 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105745 2024-09-14T09:07:22.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.250 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869531+0000", 2024-09-14T09:07:22.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869531+0000", 2024-09-14T09:07:22.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869532+0000", 2024-09-14T09:07:22.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7599999999999996e-07 2024-09-14T09:07:22.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869535+0000", 2024-09-14T09:07:22.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3459999999999998e-06 2024-09-14T09:07:22.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869546+0000", 2024-09-14T09:07:22.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0798000000000001e-05 2024-09-14T09:07:22.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869557+0000", 2024-09-14T09:07:22.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0426e-05 2024-09-14T09:07:22.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869830+0000", 2024-09-14T09:07:22.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00027363799999999998 2024-09-14T09:07:22.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869837+0000", 2024-09-14T09:07:22.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4509999999999997e-06 2024-09-14T09:07:22.252 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.252 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.253 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.253 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105300 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.253 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841762+0000", 2024-09-14T09:07:22.253 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.348066969, 2024-09-14T09:07:22.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34864263400000001, 2024-09-14T09:07:22.253 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.253 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.253 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.253 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.253 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.254 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.254 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105300 2024-09-14T09:07:22.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.254 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841762+0000", 2024-09-14T09:07:22.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841762+0000", 2024-09-14T09:07:22.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841762+0000", 2024-09-14T09:07:22.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5199999999999996e-07 2024-09-14T09:07:22.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841766+0000", 2024-09-14T09:07:22.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5540000000000002e-06 2024-09-14T09:07:22.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841776+0000", 2024-09-14T09:07:22.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0494e-05 2024-09-14T09:07:22.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841781+0000", 2024-09-14T09:07:22.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8579999999999999e-06 2024-09-14T09:07:22.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842205+0000", 2024-09-14T09:07:22.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00042328800000000002 2024-09-14T09:07:22.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842210+0000", 2024-09-14T09:07:22.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8470000000000001e-06 2024-09-14T09:07:22.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857984+0000", 2024-09-14T09:07:22.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015773249 2024-09-14T09:07:22.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857991+0000", 2024-09-14T09:07:22.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5789999999999998e-06 2024-09-14T09:07:22.257 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.257 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.257 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.258 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105363 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.258 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.845699+0000", 2024-09-14T09:07:22.258 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34412994200000002, 2024-09-14T09:07:22.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34478875799999997, 2024-09-14T09:07:22.258 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.258 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.258 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.258 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.258 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.258 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.258 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105363 2024-09-14T09:07:22.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.258 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845699+0000", 2024-09-14T09:07:22.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845699+0000", 2024-09-14T09:07:22.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845699+0000", 2024-09-14T09:07:22.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2600000000000001e-07 2024-09-14T09:07:22.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845703+0000", 2024-09-14T09:07:22.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.76e-06 2024-09-14T09:07:22.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845715+0000", 2024-09-14T09:07:22.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1822e-05 2024-09-14T09:07:22.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845724+0000", 2024-09-14T09:07:22.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.884e-06 2024-09-14T09:07:22.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845785+0000", 2024-09-14T09:07:22.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1484999999999998e-05 2024-09-14T09:07:22.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845790+0000", 2024-09-14T09:07:22.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9300000000000002e-06 2024-09-14T09:07:22.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860627+0000", 2024-09-14T09:07:22.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014836593 2024-09-14T09:07:22.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860631+0000", 2024-09-14T09:07:22.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2409999999999997e-06 2024-09-14T09:07:22.262 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.262 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.262 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.262 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105428 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.262 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.847968+0000", 2024-09-14T09:07:22.262 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34186037800000002, 2024-09-14T09:07:22.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34259215500000001, 2024-09-14T09:07:22.262 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.263 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.263 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.263 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.263 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.263 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.263 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105428 2024-09-14T09:07:22.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.263 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847968+0000", 2024-09-14T09:07:22.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847968+0000", 2024-09-14T09:07:22.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847969+0000", 2024-09-14T09:07:22.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4399999999999999e-07 2024-09-14T09:07:22.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847972+0000", 2024-09-14T09:07:22.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9900000000000002e-06 2024-09-14T09:07:22.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847982+0000", 2024-09-14T09:07:22.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0264e-05 2024-09-14T09:07:22.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847993+0000", 2024-09-14T09:07:22.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0893999999999999e-05 2024-09-14T09:07:22.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.848051+0000", 2024-09-14T09:07:22.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8322000000000002e-05 2024-09-14T09:07:22.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.848057+0000", 2024-09-14T09:07:22.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7219999999999996e-06 2024-09-14T09:07:22.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861848+0000", 2024-09-14T09:07:22.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013791358 2024-09-14T09:07:22.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861857+0000", 2024-09-14T09:07:22.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8449999999999992e-06 2024-09-14T09:07:22.267 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.267 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.267 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.267 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105491 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.267 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857939+0000", 2024-09-14T09:07:22.268 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33188927899999998, 2024-09-14T09:07:22.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33269299499999999, 2024-09-14T09:07:22.268 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.268 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.268 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.268 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.268 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.268 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.268 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105491 2024-09-14T09:07:22.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.268 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857939+0000", 2024-09-14T09:07:22.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857939+0000", 2024-09-14T09:07:22.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857940+0000", 2024-09-14T09:07:22.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.99e-07 2024-09-14T09:07:22.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857949+0000", 2024-09-14T09:07:22.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1309999999999995e-06 2024-09-14T09:07:22.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857959+0000", 2024-09-14T09:07:22.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.024e-05 2024-09-14T09:07:22.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857968+0000", 2024-09-14T09:07:22.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.303e-06 2024-09-14T09:07:22.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863173+0000", 2024-09-14T09:07:22.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.005205253 2024-09-14T09:07:22.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863182+0000", 2024-09-14T09:07:22.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.986e-06 2024-09-14T09:07:22.271 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.271 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.271 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.271 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105556 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.271 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860738+0000", 2024-09-14T09:07:22.272 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32909100600000002, 2024-09-14T09:07:22.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32995644400000002, 2024-09-14T09:07:22.272 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.272 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.272 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.272 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.272 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.272 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.272 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105556 2024-09-14T09:07:22.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.272 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860738+0000", 2024-09-14T09:07:22.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860738+0000", 2024-09-14T09:07:22.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860738+0000", 2024-09-14T09:07:22.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3000000000000001e-07 2024-09-14T09:07:22.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860740+0000", 2024-09-14T09:07:22.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2689999999999998e-06 2024-09-14T09:07:22.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860747+0000", 2024-09-14T09:07:22.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6919999999999996e-06 2024-09-14T09:07:22.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860752+0000", 2024-09-14T09:07:22.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4650000000000002e-06 2024-09-14T09:07:22.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865045+0000", 2024-09-14T09:07:22.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042923950000000001 2024-09-14T09:07:22.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865051+0000", 2024-09-14T09:07:22.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.516e-06 2024-09-14T09:07:22.275 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.275 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.275 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.276 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105618 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.276 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866139+0000", 2024-09-14T09:07:22.276 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32368935300000001, 2024-09-14T09:07:22.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32462035, 2024-09-14T09:07:22.276 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.276 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.276 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.276 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.276 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.276 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.276 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105618 2024-09-14T09:07:22.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.277 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866139+0000", 2024-09-14T09:07:22.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866139+0000", 2024-09-14T09:07:22.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866141+0000", 2024-09-14T09:07:22.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3829999999999999e-06 2024-09-14T09:07:22.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866154+0000", 2024-09-14T09:07:22.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3522e-05 2024-09-14T09:07:22.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866176+0000", 2024-09-14T09:07:22.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2098000000000001e-05 2024-09-14T09:07:22.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866213+0000", 2024-09-14T09:07:22.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6943000000000002e-05 2024-09-14T09:07:22.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866285+0000", 2024-09-14T09:07:22.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-09-14T09:07:22.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866296+0000", 2024-09-14T09:07:22.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0937999999999999e-05 2024-09-14T09:07:22.279 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.280 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.280 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.280 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105683 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.280 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868239+0000", 2024-09-14T09:07:22.280 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32158953499999998, 2024-09-14T09:07:22.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.322581216, 2024-09-14T09:07:22.280 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.280 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.280 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.280 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.280 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.281 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.281 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105683 2024-09-14T09:07:22.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.281 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868239+0000", 2024-09-14T09:07:22.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868239+0000", 2024-09-14T09:07:22.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868240+0000", 2024-09-14T09:07:22.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7899999999999997e-07 2024-09-14T09:07:22.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868243+0000", 2024-09-14T09:07:22.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0419999999999998e-06 2024-09-14T09:07:22.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868252+0000", 2024-09-14T09:07:22.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7080000000000003e-06 2024-09-14T09:07:22.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868258+0000", 2024-09-14T09:07:22.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4230000000000002e-06 2024-09-14T09:07:22.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868359+0000", 2024-09-14T09:07:22.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101433 2024-09-14T09:07:22.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868374+0000", 2024-09-14T09:07:22.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.428e-05 2024-09-14T09:07:22.284 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.284 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.284 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.285 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105747 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.285 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.869570+0000", 2024-09-14T09:07:22.285 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32025888000000002, 2024-09-14T09:07:22.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32131111899999998, 2024-09-14T09:07:22.285 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.285 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.285 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.285 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.285 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.285 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.285 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105747 2024-09-14T09:07:22.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.285 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869570+0000", 2024-09-14T09:07:22.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869570+0000", 2024-09-14T09:07:22.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869571+0000", 2024-09-14T09:07:22.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0899999999999999e-06 2024-09-14T09:07:22.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869575+0000", 2024-09-14T09:07:22.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7479999999999999e-06 2024-09-14T09:07:22.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869586+0000", 2024-09-14T09:07:22.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1168e-05 2024-09-14T09:07:22.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869594+0000", 2024-09-14T09:07:22.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5320000000000006e-06 2024-09-14T09:07:22.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869861+0000", 2024-09-14T09:07:22.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026729800000000001 2024-09-14T09:07:22.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869867+0000", 2024-09-14T09:07:22.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9309999999999996e-06 2024-09-14T09:07:22.288 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.288 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.288 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.289 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105301 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.289 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841801+0000", 2024-09-14T09:07:22.289 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34802760100000002, 2024-09-14T09:07:22.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34914010000000001, 2024-09-14T09:07:22.289 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.289 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.289 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.289 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.289 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.289 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.289 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105301 2024-09-14T09:07:22.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.290 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841801+0000", 2024-09-14T09:07:22.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841801+0000", 2024-09-14T09:07:22.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841802+0000", 2024-09-14T09:07:22.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0200000000000001e-07 2024-09-14T09:07:22.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841804+0000", 2024-09-14T09:07:22.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3719999999999999e-06 2024-09-14T09:07:22.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841811+0000", 2024-09-14T09:07:22.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0940000000000002e-06 2024-09-14T09:07:22.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841816+0000", 2024-09-14T09:07:22.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9760000000000003e-06 2024-09-14T09:07:22.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842275+0000", 2024-09-14T09:07:22.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00045885700000000001 2024-09-14T09:07:22.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842282+0000", 2024-09-14T09:07:22.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1740000000000003e-06 2024-09-14T09:07:22.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858033+0000", 2024-09-14T09:07:22.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015751061 2024-09-14T09:07:22.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858039+0000", 2024-09-14T09:07:22.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6999999999999996e-06 2024-09-14T09:07:22.293 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.293 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.294 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.294 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105365 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.294 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.845759+0000", 2024-09-14T09:07:22.294 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34406928199999998, 2024-09-14T09:07:22.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34525285999999999, 2024-09-14T09:07:22.294 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.294 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.294 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.294 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.294 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.294 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.294 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105365 2024-09-14T09:07:22.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.295 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845759+0000", 2024-09-14T09:07:22.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845759+0000", 2024-09-14T09:07:22.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845760+0000", 2024-09-14T09:07:22.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6799999999999999e-07 2024-09-14T09:07:22.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845764+0000", 2024-09-14T09:07:22.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6399999999999999e-06 2024-09-14T09:07:22.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845776+0000", 2024-09-14T09:07:22.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2249000000000001e-05 2024-09-14T09:07:22.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845785+0000", 2024-09-14T09:07:22.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7760000000000003e-06 2024-09-14T09:07:22.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845837+0000", 2024-09-14T09:07:22.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2077999999999998e-05 2024-09-14T09:07:22.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845854+0000", 2024-09-14T09:07:22.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.698e-05 2024-09-14T09:07:22.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860649+0000", 2024-09-14T09:07:22.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014794834 2024-09-14T09:07:22.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860653+0000", 2024-09-14T09:07:22.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1030000000000004e-06 2024-09-14T09:07:22.298 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.298 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.298 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.298 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105429 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.298 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.848004+0000", 2024-09-14T09:07:22.298 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34182437399999999, 2024-09-14T09:07:22.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34307966899999998, 2024-09-14T09:07:22.299 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.299 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.299 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.299 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.299 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.299 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.299 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105429 2024-09-14T09:07:22.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.299 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.848004+0000", 2024-09-14T09:07:22.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.848004+0000", 2024-09-14T09:07:22.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.848005+0000", 2024-09-14T09:07:22.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7599999999999996e-07 2024-09-14T09:07:22.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.848008+0000", 2024-09-14T09:07:22.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2140000000000001e-06 2024-09-14T09:07:22.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.848019+0000", 2024-09-14T09:07:22.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0705e-05 2024-09-14T09:07:22.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855070+0000", 2024-09-14T09:07:22.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0070514530000000001 2024-09-14T09:07:22.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861895+0000", 2024-09-14T09:07:22.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068250430000000003 2024-09-14T09:07:22.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861904+0000", 2024-09-14T09:07:22.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7579999999999998e-06 2024-09-14T09:07:22.302 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.302 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.302 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.303 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105492 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.303 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857980+0000", 2024-09-14T09:07:22.303 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.331848582, 2024-09-14T09:07:22.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33316538800000001, 2024-09-14T09:07:22.303 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.303 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.303 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.303 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.303 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.304 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.304 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105492 2024-09-14T09:07:22.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.304 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857980+0000", 2024-09-14T09:07:22.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857980+0000", 2024-09-14T09:07:22.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857981+0000", 2024-09-14T09:07:22.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3e-07 2024-09-14T09:07:22.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857984+0000", 2024-09-14T09:07:22.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0450000000000001e-06 2024-09-14T09:07:22.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857993+0000", 2024-09-14T09:07:22.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3349999999999996e-06 2024-09-14T09:07:22.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858001+0000", 2024-09-14T09:07:22.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4230000000000005e-06 2024-09-14T09:07:22.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863217+0000", 2024-09-14T09:07:22.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052151489999999997 2024-09-14T09:07:22.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863225+0000", 2024-09-14T09:07:22.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7479999999999999e-06 2024-09-14T09:07:22.307 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.307 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.307 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.307 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105557 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.307 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860779+0000", 2024-09-14T09:07:22.307 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.329049641, 2024-09-14T09:07:22.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33042799899999997, 2024-09-14T09:07:22.308 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.308 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.308 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.308 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.308 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.308 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.308 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105557 2024-09-14T09:07:22.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.308 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860779+0000", 2024-09-14T09:07:22.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860779+0000", 2024-09-14T09:07:22.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860779+0000", 2024-09-14T09:07:22.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.75e-07 2024-09-14T09:07:22.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860782+0000", 2024-09-14T09:07:22.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4849999999999999e-06 2024-09-14T09:07:22.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861058+0000", 2024-09-14T09:07:22.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00027628 2024-09-14T09:07:22.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861066+0000", 2024-09-14T09:07:22.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6520000000000006e-06 2024-09-14T09:07:22.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865077+0000", 2024-09-14T09:07:22.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040110270000000003 2024-09-14T09:07:22.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865083+0000", 2024-09-14T09:07:22.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3069999999999999e-06 2024-09-14T09:07:22.311 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.311 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.311 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.312 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105621 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.312 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866309+0000", 2024-09-14T09:07:22.312 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32351912700000002, 2024-09-14T09:07:22.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32495817399999999, 2024-09-14T09:07:22.312 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.312 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.312 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.312 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.312 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.312 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.312 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105621 2024-09-14T09:07:22.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.313 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866309+0000", 2024-09-14T09:07:22.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866309+0000", 2024-09-14T09:07:22.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866310+0000", 2024-09-14T09:07:22.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4e-07 2024-09-14T09:07:22.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866313+0000", 2024-09-14T09:07:22.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2040000000000002e-06 2024-09-14T09:07:22.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866322+0000", 2024-09-14T09:07:22.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2090000000000005e-06 2024-09-14T09:07:22.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866328+0000", 2024-09-14T09:07:22.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0379999999999996e-06 2024-09-14T09:07:22.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866407+0000", 2024-09-14T09:07:22.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9370999999999996e-05 2024-09-14T09:07:22.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866421+0000", 2024-09-14T09:07:22.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.385e-05 2024-09-14T09:07:22.316 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.316 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.316 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.316 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105685 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.316 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868272+0000", 2024-09-14T09:07:22.316 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32155689599999998, 2024-09-14T09:07:22.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32305691399999997, 2024-09-14T09:07:22.317 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.317 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.317 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.317 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.317 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.317 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.317 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105685 2024-09-14T09:07:22.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.317 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868272+0000", 2024-09-14T09:07:22.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868272+0000", 2024-09-14T09:07:22.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868273+0000", 2024-09-14T09:07:22.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9199999999999999e-07 2024-09-14T09:07:22.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868276+0000", 2024-09-14T09:07:22.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8500000000000004e-06 2024-09-14T09:07:22.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868289+0000", 2024-09-14T09:07:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2498e-05 2024-09-14T09:07:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868298+0000", 2024-09-14T09:07:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2240000000000001e-06 2024-09-14T09:07:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868428+0000", 2024-09-14T09:07:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012983299999999999 2024-09-14T09:07:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868435+0000", 2024-09-14T09:07:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5780000000000004e-06 2024-09-14T09:07:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.321 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105750 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.321 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.869606+0000", 2024-09-14T09:07:22.321 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32022282400000002, 2024-09-14T09:07:22.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.321787665, 2024-09-14T09:07:22.321 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.321 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.321 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.321 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.321 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105750 2024-09-14T09:07:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869606+0000", 2024-09-14T09:07:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869606+0000", 2024-09-14T09:07:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869606+0000", 2024-09-14T09:07:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5799999999999998e-07 2024-09-14T09:07:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869610+0000", 2024-09-14T09:07:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2200000000000001e-06 2024-09-14T09:07:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869621+0000", 2024-09-14T09:07:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1143e-05 2024-09-14T09:07:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869629+0000", 2024-09-14T09:07:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0479999999999999e-06 2024-09-14T09:07:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869893+0000", 2024-09-14T09:07:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026426800000000001 2024-09-14T09:07:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869899+0000", 2024-09-14T09:07:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0390000000000001e-06 2024-09-14T09:07:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105303 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841832+0000", 2024-09-14T09:07:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34799629999999998, 2024-09-14T09:07:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34962227899999998, 2024-09-14T09:07:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105303 2024-09-14T09:07:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841832+0000", 2024-09-14T09:07:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841832+0000", 2024-09-14T09:07:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841833+0000", 2024-09-14T09:07:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4000000000000002e-07 2024-09-14T09:07:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841835+0000", 2024-09-14T09:07:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.52e-06 2024-09-14T09:07:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841841+0000", 2024-09-14T09:07:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0480000000000004e-06 2024-09-14T09:07:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841846+0000", 2024-09-14T09:07:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.634e-06 2024-09-14T09:07:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842321+0000", 2024-09-14T09:07:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00047481599999999998 2024-09-14T09:07:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842326+0000", 2024-09-14T09:07:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9359999999999998e-06 2024-09-14T09:07:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858077+0000", 2024-09-14T09:07:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015751166 2024-09-14T09:07:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858085+0000", 2024-09-14T09:07:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3969999999999994e-06 2024-09-14T09:07:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105367 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.845811+0000", 2024-09-14T09:07:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34401747500000002, 2024-09-14T09:07:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34572119099999998, 2024-09-14T09:07:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105367 2024-09-14T09:07:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845811+0000", 2024-09-14T09:07:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845811+0000", 2024-09-14T09:07:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845812+0000", 2024-09-14T09:07:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7800000000000001e-07 2024-09-14T09:07:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845815+0000", 2024-09-14T09:07:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5240000000000001e-06 2024-09-14T09:07:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845826+0000", 2024-09-14T09:07:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1154e-05 2024-09-14T09:07:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845835+0000", 2024-09-14T09:07:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5059999999999995e-06 2024-09-14T09:07:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845879+0000", 2024-09-14T09:07:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3702999999999998e-05 2024-09-14T09:07:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845883+0000", 2024-09-14T09:07:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7690000000000004e-06 2024-09-14T09:07:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860670+0000", 2024-09-14T09:07:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014786987999999999 2024-09-14T09:07:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860675+0000", 2024-09-14T09:07:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3490000000000002e-06 2024-09-14T09:07:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105431 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.855094+0000", 2024-09-14T09:07:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33473502799999999, 2024-09-14T09:07:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33656592600000002, 2024-09-14T09:07:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105431 2024-09-14T09:07:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855094+0000", 2024-09-14T09:07:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855094+0000", 2024-09-14T09:07:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855095+0000", 2024-09-14T09:07:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2669999999999999e-06 2024-09-14T09:07:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855100+0000", 2024-09-14T09:07:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2889999999999997e-06 2024-09-14T09:07:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855111+0000", 2024-09-14T09:07:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1357e-05 2024-09-14T09:07:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855120+0000", 2024-09-14T09:07:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9679999999999995e-06 2024-09-14T09:07:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861945+0000", 2024-09-14T09:07:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068244029999999997 2024-09-14T09:07:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861954+0000", 2024-09-14T09:07:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9500000000000007e-06 2024-09-14T09:07:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105496 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.858014+0000", 2024-09-14T09:07:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33181502299999999, 2024-09-14T09:07:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33370752999999997, 2024-09-14T09:07:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105496 2024-09-14T09:07:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858014+0000", 2024-09-14T09:07:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858014+0000", 2024-09-14T09:07:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858014+0000", 2024-09-14T09:07:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-09-14T09:07:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858017+0000", 2024-09-14T09:07:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7930000000000002e-06 2024-09-14T09:07:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858025+0000", 2024-09-14T09:07:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5399999999999996e-06 2024-09-14T09:07:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858032+0000", 2024-09-14T09:07:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1e-06 2024-09-14T09:07:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863262+0000", 2024-09-14T09:07:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052301029999999998 2024-09-14T09:07:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863270+0000", 2024-09-14T09:07:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6570000000000003e-06 2024-09-14T09:07:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.344 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105559 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.344 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861079+0000", 2024-09-14T09:07:22.344 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.328749293, 2024-09-14T09:07:22.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33070638000000002, 2024-09-14T09:07:22.345 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.345 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.345 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.345 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.345 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.345 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.345 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105559 2024-09-14T09:07:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861079+0000", 2024-09-14T09:07:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861079+0000", 2024-09-14T09:07:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861080+0000", 2024-09-14T09:07:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9699999999999995e-07 2024-09-14T09:07:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861084+0000", 2024-09-14T09:07:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5930000000000002e-06 2024-09-14T09:07:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861092+0000", 2024-09-14T09:07:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3319999999999992e-06 2024-09-14T09:07:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861098+0000", 2024-09-14T09:07:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8629999999999996e-06 2024-09-14T09:07:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865109+0000", 2024-09-14T09:07:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040113429999999997 2024-09-14T09:07:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865116+0000", 2024-09-14T09:07:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6100000000000002e-06 2024-09-14T09:07:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105623 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866353+0000", 2024-09-14T09:07:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32347571200000003, 2024-09-14T09:07:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32549425799999998, 2024-09-14T09:07:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105623 2024-09-14T09:07:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866353+0000", 2024-09-14T09:07:22.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866353+0000", 2024-09-14T09:07:22.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866354+0000", 2024-09-14T09:07:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0359999999999999e-06 2024-09-14T09:07:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866358+0000", 2024-09-14T09:07:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4089999999999996e-06 2024-09-14T09:07:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866372+0000", 2024-09-14T09:07:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4195e-05 2024-09-14T09:07:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866383+0000", 2024-09-14T09:07:22.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0796e-05 2024-09-14T09:07:22.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866472+0000", 2024-09-14T09:07:22.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8245999999999995e-05 2024-09-14T09:07:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866485+0000", 2024-09-14T09:07:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3188e-05 2024-09-14T09:07:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105687 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868310+0000", 2024-09-14T09:07:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32151878699999997, 2024-09-14T09:07:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32360219099999998, 2024-09-14T09:07:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105687 2024-09-14T09:07:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868310+0000", 2024-09-14T09:07:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868310+0000", 2024-09-14T09:07:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868311+0000", 2024-09-14T09:07:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1710000000000001e-06 2024-09-14T09:07:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868315+0000", 2024-09-14T09:07:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6260000000000001e-06 2024-09-14T09:07:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868328+0000", 2024-09-14T09:07:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3543000000000001e-05 2024-09-14T09:07:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868338+0000", 2024-09-14T09:07:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7350000000000008e-06 2024-09-14T09:07:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868489+0000", 2024-09-14T09:07:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000151531 2024-09-14T09:07:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868502+0000", 2024-09-14T09:07:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2709e-05 2024-09-14T09:07:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105751 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.869640+0000", 2024-09-14T09:07:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32018865499999999, 2024-09-14T09:07:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32233614100000002, 2024-09-14T09:07:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105751 2024-09-14T09:07:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869640+0000", 2024-09-14T09:07:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869640+0000", 2024-09-14T09:07:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869641+0000", 2024-09-14T09:07:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0399999999999995e-07 2024-09-14T09:07:22.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869644+0000", 2024-09-14T09:07:22.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2200000000000001e-06 2024-09-14T09:07:22.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869654+0000", 2024-09-14T09:07:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0668e-05 2024-09-14T09:07:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869665+0000", 2024-09-14T09:07:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0159999999999999e-05 2024-09-14T09:07:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869926+0000", 2024-09-14T09:07:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026121299999999999 2024-09-14T09:07:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869933+0000", 2024-09-14T09:07:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8299999999999998e-06 2024-09-14T09:07:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105305 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841853+0000", 2024-09-14T09:07:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34797515200000001, 2024-09-14T09:07:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.350184357, 2024-09-14T09:07:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105305 2024-09-14T09:07:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841853+0000", 2024-09-14T09:07:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841853+0000", 2024-09-14T09:07:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841854+0000", 2024-09-14T09:07:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7199999999999998e-07 2024-09-14T09:07:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841858+0000", 2024-09-14T09:07:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.518e-06 2024-09-14T09:07:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841866+0000", 2024-09-14T09:07:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6619999999999994e-06 2024-09-14T09:07:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841871+0000", 2024-09-14T09:07:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4739999999999999e-06 2024-09-14T09:07:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842354+0000", 2024-09-14T09:07:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048341699999999998 2024-09-14T09:07:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842359+0000", 2024-09-14T09:07:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.673e-06 2024-09-14T09:07:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858120+0000", 2024-09-14T09:07:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015760957999999999 2024-09-14T09:07:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858126+0000", 2024-09-14T09:07:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8379999999999998e-06 2024-09-14T09:07:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105369 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.845847+0000", 2024-09-14T09:07:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.343981231, 2024-09-14T09:07:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34628608500000002, 2024-09-14T09:07:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105369 2024-09-14T09:07:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845847+0000", 2024-09-14T09:07:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845847+0000", 2024-09-14T09:07:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845848+0000", 2024-09-14T09:07:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5600000000000005e-07 2024-09-14T09:07:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845851+0000", 2024-09-14T09:07:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1580000000000001e-06 2024-09-14T09:07:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845862+0000", 2024-09-14T09:07:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0766e-05 2024-09-14T09:07:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845871+0000", 2024-09-14T09:07:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5399999999999996e-06 2024-09-14T09:07:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845907+0000", 2024-09-14T09:07:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6123e-05 2024-09-14T09:07:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845911+0000", 2024-09-14T09:07:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3470000000000001e-06 2024-09-14T09:07:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860693+0000", 2024-09-14T09:07:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014781765000000001 2024-09-14T09:07:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860697+0000", 2024-09-14T09:07:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1819999999999999e-06 2024-09-14T09:07:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105433 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.855131+0000", 2024-09-14T09:07:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33469737799999999, 2024-09-14T09:07:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.337080088, 2024-09-14T09:07:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105433 2024-09-14T09:07:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855131+0000", 2024-09-14T09:07:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855131+0000", 2024-09-14T09:07:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855132+0000", 2024-09-14T09:07:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9299999999999998e-07 2024-09-14T09:07:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855135+0000", 2024-09-14T09:07:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9979999999999999e-06 2024-09-14T09:07:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855143+0000", 2024-09-14T09:07:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1389999999999995e-06 2024-09-14T09:07:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855149+0000", 2024-09-14T09:07:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2530000000000001e-06 2024-09-14T09:07:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861999+0000", 2024-09-14T09:07:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068496520000000003 2024-09-14T09:07:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862008+0000", 2024-09-14T09:07:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.285e-06 2024-09-14T09:07:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105497 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.858041+0000", 2024-09-14T09:07:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33178795900000002, 2024-09-14T09:07:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33423186199999999, 2024-09-14T09:07:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105497 2024-09-14T09:07:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858041+0000", 2024-09-14T09:07:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858041+0000", 2024-09-14T09:07:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858041+0000", 2024-09-14T09:07:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.44e-07 2024-09-14T09:07:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858044+0000", 2024-09-14T09:07:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0510000000000001e-06 2024-09-14T09:07:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858053+0000", 2024-09-14T09:07:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9919999999999996e-06 2024-09-14T09:07:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858060+0000", 2024-09-14T09:07:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8900000000000001e-06 2024-09-14T09:07:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863306+0000", 2024-09-14T09:07:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052459519999999999 2024-09-14T09:07:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863315+0000", 2024-09-14T09:07:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2660000000000007e-06 2024-09-14T09:07:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105561 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861127+0000", 2024-09-14T09:07:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32870125500000003, 2024-09-14T09:07:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.331209278, 2024-09-14T09:07:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105561 2024-09-14T09:07:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861127+0000", 2024-09-14T09:07:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861127+0000", 2024-09-14T09:07:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861128+0000", 2024-09-14T09:07:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0809999999999999e-06 2024-09-14T09:07:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861134+0000", 2024-09-14T09:07:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5199999999999997e-06 2024-09-14T09:07:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861147+0000", 2024-09-14T09:07:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2693e-05 2024-09-14T09:07:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861155+0000", 2024-09-14T09:07:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9160000000000007e-06 2024-09-14T09:07:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865141+0000", 2024-09-14T09:07:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039858269999999999 2024-09-14T09:07:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865148+0000", 2024-09-14T09:07:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7599999999999997e-06 2024-09-14T09:07:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105624 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866398+0000", 2024-09-14T09:07:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32343075900000001, 2024-09-14T09:07:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32600036399999999, 2024-09-14T09:07:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105624 2024-09-14T09:07:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866398+0000", 2024-09-14T09:07:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866398+0000", 2024-09-14T09:07:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866399+0000", 2024-09-14T09:07:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.02e-06 2024-09-14T09:07:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866403+0000", 2024-09-14T09:07:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7419999999999999e-06 2024-09-14T09:07:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866415+0000", 2024-09-14T09:07:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2492e-05 2024-09-14T09:07:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866425+0000", 2024-09-14T09:07:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6330000000000008e-06 2024-09-14T09:07:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866532+0000", 2024-09-14T09:07:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010737400000000001 2024-09-14T09:07:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866545+0000", 2024-09-14T09:07:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3067e-05 2024-09-14T09:07:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105690 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868351+0000", 2024-09-14T09:07:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32147726999999998, 2024-09-14T09:07:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32411168400000001, 2024-09-14T09:07:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105690 2024-09-14T09:07:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868351+0000", 2024-09-14T09:07:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868351+0000", 2024-09-14T09:07:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868352+0000", 2024-09-14T09:07:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5000000000000001e-07 2024-09-14T09:07:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868356+0000", 2024-09-14T09:07:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6519999999999999e-06 2024-09-14T09:07:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868368+0000", 2024-09-14T09:07:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1838000000000001e-05 2024-09-14T09:07:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868377+0000", 2024-09-14T09:07:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3309999999999993e-06 2024-09-14T09:07:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868547+0000", 2024-09-14T09:07:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000169653 2024-09-14T09:07:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868554+0000", 2024-09-14T09:07:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9990000000000002e-06 2024-09-14T09:07:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105752 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.869676+0000", 2024-09-14T09:07:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32015228899999998, 2024-09-14T09:07:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32284731700000002, 2024-09-14T09:07:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105752 2024-09-14T09:07:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869676+0000", 2024-09-14T09:07:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869676+0000", 2024-09-14T09:07:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869677+0000", 2024-09-14T09:07:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8400000000000003e-07 2024-09-14T09:07:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869687+0000", 2024-09-14T09:07:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5119999999999997e-06 2024-09-14T09:07:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869699+0000", 2024-09-14T09:07:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2282e-05 2024-09-14T09:07:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869707+0000", 2024-09-14T09:07:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6319999999999997e-06 2024-09-14T09:07:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870323+0000", 2024-09-14T09:07:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00061505000000000002 2024-09-14T09:07:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870331+0000", 2024-09-14T09:07:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1710000000000002e-06 2024-09-14T09:07:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105307 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841877+0000", 2024-09-14T09:07:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34795120600000001, 2024-09-14T09:07:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35070989899999999, 2024-09-14T09:07:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105307 2024-09-14T09:07:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841877+0000", 2024-09-14T09:07:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841877+0000", 2024-09-14T09:07:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841878+0000", 2024-09-14T09:07:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.06e-07 2024-09-14T09:07:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841880+0000", 2024-09-14T09:07:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.934e-06 2024-09-14T09:07:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841885+0000", 2024-09-14T09:07:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3480000000000003e-06 2024-09-14T09:07:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841890+0000", 2024-09-14T09:07:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5120000000000002e-06 2024-09-14T09:07:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842386+0000", 2024-09-14T09:07:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00049617699999999999 2024-09-14T09:07:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842391+0000", 2024-09-14T09:07:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7489999999999998e-06 2024-09-14T09:07:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859506+0000", 2024-09-14T09:07:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017115004999999999 2024-09-14T09:07:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859518+0000", 2024-09-14T09:07:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2368000000000001e-05 2024-09-14T09:07:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105371 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.845882+0000", 2024-09-14T09:07:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34394637099999997, 2024-09-14T09:07:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34677997599999999, 2024-09-14T09:07:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105371 2024-09-14T09:07:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845882+0000", 2024-09-14T09:07:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845882+0000", 2024-09-14T09:07:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845883+0000", 2024-09-14T09:07:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7599999999999996e-07 2024-09-14T09:07:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845904+0000", 2024-09-14T09:07:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0818999999999998e-05 2024-09-14T09:07:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845917+0000", 2024-09-14T09:07:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3010000000000001e-05 2024-09-14T09:07:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845926+0000", 2024-09-14T09:07:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0259999999999996e-06 2024-09-14T09:07:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846003+0000", 2024-09-14T09:07:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7695999999999996e-05 2024-09-14T09:07:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846013+0000", 2024-09-14T09:07:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4639999999999995e-06 2024-09-14T09:07:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860715+0000", 2024-09-14T09:07:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014701727 2024-09-14T09:07:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860719+0000", 2024-09-14T09:07:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2300000000000002e-06 2024-09-14T09:07:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105436 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.855160+0000", 2024-09-14T09:07:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33466861399999998, 2024-09-14T09:07:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.337573976, 2024-09-14T09:07:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105436 2024-09-14T09:07:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855160+0000", 2024-09-14T09:07:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855160+0000", 2024-09-14T09:07:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855160+0000", 2024-09-14T09:07:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9999999999999998e-07 2024-09-14T09:07:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855163+0000", 2024-09-14T09:07:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9569999999999998e-06 2024-09-14T09:07:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855171+0000", 2024-09-14T09:07:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2450000000000001e-06 2024-09-14T09:07:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855176+0000", 2024-09-14T09:07:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5639999999999996e-06 2024-09-14T09:07:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862045+0000", 2024-09-14T09:07:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068692060000000001 2024-09-14T09:07:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862055+0000", 2024-09-14T09:07:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6339999999999996e-06 2024-09-14T09:07:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105498 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.858070+0000", 2024-09-14T09:07:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33175866999999998, 2024-09-14T09:07:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.334728256, 2024-09-14T09:07:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105498 2024-09-14T09:07:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858070+0000", 2024-09-14T09:07:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858070+0000", 2024-09-14T09:07:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858070+0000", 2024-09-14T09:07:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6199999999999998e-07 2024-09-14T09:07:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858073+0000", 2024-09-14T09:07:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5770000000000001e-06 2024-09-14T09:07:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858082+0000", 2024-09-14T09:07:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5790000000000004e-06 2024-09-14T09:07:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858089+0000", 2024-09-14T09:07:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-09-14T09:07:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863350+0000", 2024-09-14T09:07:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052611630000000001 2024-09-14T09:07:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863358+0000", 2024-09-14T09:07:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4130000000000006e-06 2024-09-14T09:07:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105563 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861179+0000", 2024-09-14T09:07:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32864975800000001, 2024-09-14T09:07:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33168328200000002, 2024-09-14T09:07:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105563 2024-09-14T09:07:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861179+0000", 2024-09-14T09:07:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861179+0000", 2024-09-14T09:07:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861180+0000", 2024-09-14T09:07:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9500000000000001e-07 2024-09-14T09:07:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861183+0000", 2024-09-14T09:07:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3979999999999999e-06 2024-09-14T09:07:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861192+0000", 2024-09-14T09:07:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2820000000000002e-06 2024-09-14T09:07:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861199+0000", 2024-09-14T09:07:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1849999999999998e-06 2024-09-14T09:07:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865174+0000", 2024-09-14T09:07:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039745420000000002 2024-09-14T09:07:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865180+0000", 2024-09-14T09:07:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3899999999999998e-06 2024-09-14T09:07:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105626 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866462+0000", 2024-09-14T09:07:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32336608900000002, 2024-09-14T09:07:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32646075200000002, 2024-09-14T09:07:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105626 2024-09-14T09:07:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866462+0000", 2024-09-14T09:07:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866462+0000", 2024-09-14T09:07:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866463+0000", 2024-09-14T09:07:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7399999999999991e-07 2024-09-14T09:07:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866467+0000", 2024-09-14T09:07:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8020000000000002e-06 2024-09-14T09:07:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866480+0000", 2024-09-14T09:07:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2724999999999999e-05 2024-09-14T09:07:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866490+0000", 2024-09-14T09:07:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0037999999999999e-05 2024-09-14T09:07:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866591+0000", 2024-09-14T09:07:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101043 2024-09-14T09:07:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866604+0000", 2024-09-14T09:07:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.312e-05 2024-09-14T09:07:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105691 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868427+0000", 2024-09-14T09:07:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32140120900000002, 2024-09-14T09:07:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32455941399999999, 2024-09-14T09:07:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105691 2024-09-14T09:07:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868427+0000", 2024-09-14T09:07:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868427+0000", 2024-09-14T09:07:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868428+0000", 2024-09-14T09:07:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.124e-06 2024-09-14T09:07:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868432+0000", 2024-09-14T09:07:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7340000000000002e-06 2024-09-14T09:07:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868446+0000", 2024-09-14T09:07:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3328e-05 2024-09-14T09:07:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868455+0000", 2024-09-14T09:07:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1560000000000001e-06 2024-09-14T09:07:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868605+0000", 2024-09-14T09:07:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015073500000000001 2024-09-14T09:07:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868618+0000", 2024-09-14T09:07:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2621e-05 2024-09-14T09:07:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105755 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.869732+0000", 2024-09-14T09:07:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32009629899999997, 2024-09-14T09:07:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32331545699999997, 2024-09-14T09:07:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105755 2024-09-14T09:07:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869732+0000", 2024-09-14T09:07:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869732+0000", 2024-09-14T09:07:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869733+0000", 2024-09-14T09:07:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1800000000000004e-07 2024-09-14T09:07:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869736+0000", 2024-09-14T09:07:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2880000000000001e-06 2024-09-14T09:07:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869748+0000", 2024-09-14T09:07:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1633e-05 2024-09-14T09:07:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869757+0000", 2024-09-14T09:07:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5979999999999997e-06 2024-09-14T09:07:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870388+0000", 2024-09-14T09:07:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063130900000000004 2024-09-14T09:07:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870403+0000", 2024-09-14T09:07:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4932e-05 2024-09-14T09:07:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105245 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.840052+0000", 2024-09-14T09:07:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34977645499999999, 2024-09-14T09:07:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35306204699999999, 2024-09-14T09:07:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105245 2024-09-14T09:07:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840052+0000", 2024-09-14T09:07:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840052+0000", 2024-09-14T09:07:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840054+0000", 2024-09-14T09:07:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2359999999999999e-06 2024-09-14T09:07:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840067+0000", 2024-09-14T09:07:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2867999999999999e-05 2024-09-14T09:07:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840093+0000", 2024-09-14T09:07:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.618e-05 2024-09-14T09:07:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840117+0000", 2024-09-14T09:07:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.349e-05 2024-09-14T09:07:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840289+0000", 2024-09-14T09:07:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017175799999999999 2024-09-14T09:07:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op must wait for map", 2024-09-14T09:07:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840303+0000", 2024-09-14T09:07:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.485e-05 2024-09-14T09:07:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855963+0000", 2024-09-14T09:07:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015659197999999999 2024-09-14T09:07:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855976+0000", 2024-09-14T09:07:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3465000000000001e-05 2024-09-14T09:07:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105308 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841895+0000", 2024-09-14T09:07:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34793335399999997, 2024-09-14T09:07:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35129829400000001, 2024-09-14T09:07:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105308 2024-09-14T09:07:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841895+0000", 2024-09-14T09:07:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841895+0000", 2024-09-14T09:07:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841896+0000", 2024-09-14T09:07:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7399999999999998e-07 2024-09-14T09:07:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841898+0000", 2024-09-14T09:07:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1679999999999998e-06 2024-09-14T09:07:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841903+0000", 2024-09-14T09:07:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2939999999999996e-06 2024-09-14T09:07:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841909+0000", 2024-09-14T09:07:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9599999999999997e-06 2024-09-14T09:07:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842418+0000", 2024-09-14T09:07:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050856000000000002 2024-09-14T09:07:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842422+0000", 2024-09-14T09:07:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7879999999999997e-06 2024-09-14T09:07:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859565+0000", 2024-09-14T09:07:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017142816000000002 2024-09-14T09:07:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859575+0000", 2024-09-14T09:07:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8779999999999993e-06 2024-09-14T09:07:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105373 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.846028+0000", 2024-09-14T09:07:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34380093099999998, 2024-09-14T09:07:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34723744200000001, 2024-09-14T09:07:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105373 2024-09-14T09:07:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.449 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846028+0000", 2024-09-14T09:07:22.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846028+0000", 2024-09-14T09:07:22.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846029+0000", 2024-09-14T09:07:22.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9399999999999993e-07 2024-09-14T09:07:22.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846033+0000", 2024-09-14T09:07:22.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1740000000000002e-06 2024-09-14T09:07:22.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846045+0000", 2024-09-14T09:07:22.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1739999999999999e-05 2024-09-14T09:07:22.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846054+0000", 2024-09-14T09:07:22.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5279999999999992e-06 2024-09-14T09:07:22.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846119+0000", 2024-09-14T09:07:22.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4797999999999999e-05 2024-09-14T09:07:22.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846127+0000", 2024-09-14T09:07:22.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6000000000000007e-06 2024-09-14T09:07:22.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860736+0000", 2024-09-14T09:07:22.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014608614000000001 2024-09-14T09:07:22.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860740+0000", 2024-09-14T09:07:22.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1400000000000002e-06 2024-09-14T09:07:22.453 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.454 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.454 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.454 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105437 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.454 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.855187+0000", 2024-09-14T09:07:22.454 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33464198499999998, 2024-09-14T09:07:22.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.338153437, 2024-09-14T09:07:22.454 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.455 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.455 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.455 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.455 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.455 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.455 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105437 2024-09-14T09:07:22.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.456 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.456 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.456 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855187+0000", 2024-09-14T09:07:22.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.456 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.456 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855187+0000", 2024-09-14T09:07:22.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855187+0000", 2024-09-14T09:07:22.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6499999999999999e-07 2024-09-14T09:07:22.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855190+0000", 2024-09-14T09:07:22.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7530000000000002e-06 2024-09-14T09:07:22.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855199+0000", 2024-09-14T09:07:22.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9630000000000004e-06 2024-09-14T09:07:22.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855206+0000", 2024-09-14T09:07:22.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0500000000000003e-06 2024-09-14T09:07:22.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862107+0000", 2024-09-14T09:07:22.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069014530000000001 2024-09-14T09:07:22.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862117+0000", 2024-09-14T09:07:22.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6250000000000002e-06 2024-09-14T09:07:22.460 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.460 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.460 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.461 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105502 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.461 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.858122+0000", 2024-09-14T09:07:22.461 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33170648600000002, 2024-09-14T09:07:22.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33537670400000003, 2024-09-14T09:07:22.461 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.462 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.462 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.462 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.462 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.462 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.462 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105502 2024-09-14T09:07:22.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.462 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858122+0000", 2024-09-14T09:07:22.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858122+0000", 2024-09-14T09:07:22.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858123+0000", 2024-09-14T09:07:22.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0100000000000001e-06 2024-09-14T09:07:22.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858128+0000", 2024-09-14T09:07:22.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6369999999999998e-06 2024-09-14T09:07:22.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858137+0000", 2024-09-14T09:07:22.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7499999999999998e-06 2024-09-14T09:07:22.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.858145+0000", 2024-09-14T09:07:22.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5009999999999998e-06 2024-09-14T09:07:22.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863407+0000", 2024-09-14T09:07:22.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052624300000000002 2024-09-14T09:07:22.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863420+0000", 2024-09-14T09:07:22.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2829e-05 2024-09-14T09:07:22.467 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.467 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.467 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.468 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105566 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.468 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861222+0000", 2024-09-14T09:07:22.468 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32860666900000002, 2024-09-14T09:07:22.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33233974500000002, 2024-09-14T09:07:22.468 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.468 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.468 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.468 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.468 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.469 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.469 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105566 2024-09-14T09:07:22.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.469 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861222+0000", 2024-09-14T09:07:22.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861222+0000", 2024-09-14T09:07:22.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861223+0000", 2024-09-14T09:07:22.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9999999999999997e-07 2024-09-14T09:07:22.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861226+0000", 2024-09-14T09:07:22.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1190000000000001e-06 2024-09-14T09:07:22.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861235+0000", 2024-09-14T09:07:22.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1300000000000007e-06 2024-09-14T09:07:22.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861242+0000", 2024-09-14T09:07:22.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9809999999999997e-06 2024-09-14T09:07:22.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865205+0000", 2024-09-14T09:07:22.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039636630000000001 2024-09-14T09:07:22.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865212+0000", 2024-09-14T09:07:22.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2500000000000003e-06 2024-09-14T09:07:22.472 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.472 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.472 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.472 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105629 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.472 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:07:22.472 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32330747999999998, 2024-09-14T09:07:22.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32710879300000001, 2024-09-14T09:07:22.472 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.472 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.472 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.472 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.473 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.473 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.473 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105629 2024-09-14T09:07:22.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.473 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:07:22.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:07:22.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866522+0000", 2024-09-14T09:07:22.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6899999999999996e-07 2024-09-14T09:07:22.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866526+0000", 2024-09-14T09:07:22.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0110000000000002e-06 2024-09-14T09:07:22.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866539+0000", 2024-09-14T09:07:22.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3013000000000001e-05 2024-09-14T09:07:22.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866549+0000", 2024-09-14T09:07:22.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8490000000000001e-06 2024-09-14T09:07:22.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866650+0000", 2024-09-14T09:07:22.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101054 2024-09-14T09:07:22.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866664+0000", 2024-09-14T09:07:22.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3805000000000001e-05 2024-09-14T09:07:22.476 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.476 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.476 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.477 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105694 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.477 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868482+0000", 2024-09-14T09:07:22.477 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32134676699999998, 2024-09-14T09:07:22.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32520923499999999, 2024-09-14T09:07:22.477 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.477 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.477 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.477 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.477 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.477 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.477 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105694 2024-09-14T09:07:22.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.477 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868482+0000", 2024-09-14T09:07:22.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868482+0000", 2024-09-14T09:07:22.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868483+0000", 2024-09-14T09:07:22.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7599999999999996e-07 2024-09-14T09:07:22.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868486+0000", 2024-09-14T09:07:22.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6279999999999998e-06 2024-09-14T09:07:22.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868498+0000", 2024-09-14T09:07:22.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1499e-05 2024-09-14T09:07:22.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868507+0000", 2024-09-14T09:07:22.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8419999999999994e-06 2024-09-14T09:07:22.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868680+0000", 2024-09-14T09:07:22.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017324800000000001 2024-09-14T09:07:22.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868689+0000", 2024-09-14T09:07:22.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.687e-06 2024-09-14T09:07:22.480 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.481 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.481 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.481 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105247 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.481 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.840148+0000", 2024-09-14T09:07:22.481 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.349680253, 2024-09-14T09:07:22.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.353607909, 2024-09-14T09:07:22.481 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.481 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.481 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.481 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.481 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.481 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.482 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105247 2024-09-14T09:07:22.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.482 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840148+0000", 2024-09-14T09:07:22.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840148+0000", 2024-09-14T09:07:22.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840149+0000", 2024-09-14T09:07:22.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.85e-07 2024-09-14T09:07:22.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840153+0000", 2024-09-14T09:07:22.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9709999999999998e-06 2024-09-14T09:07:22.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840160+0000", 2024-09-14T09:07:22.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.401e-06 2024-09-14T09:07:22.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840165+0000", 2024-09-14T09:07:22.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5430000000000004e-06 2024-09-14T09:07:22.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840439+0000", 2024-09-14T09:07:22.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00027423500000000001 2024-09-14T09:07:22.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840446+0000", 2024-09-14T09:07:22.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.221e-06 2024-09-14T09:07:22.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856672+0000", 2024-09-14T09:07:22.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016225197 2024-09-14T09:07:22.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856683+0000", 2024-09-14T09:07:22.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1693e-05 2024-09-14T09:07:22.486 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.486 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.486 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.486 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105312 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.486 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841934+0000", 2024-09-14T09:07:22.486 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34789463599999998, 2024-09-14T09:07:22.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.351967682, 2024-09-14T09:07:22.486 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.486 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.486 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.487 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.487 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.487 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.487 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105312 2024-09-14T09:07:22.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.487 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841934+0000", 2024-09-14T09:07:22.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841934+0000", 2024-09-14T09:07:22.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841935+0000", 2024-09-14T09:07:22.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.06e-06 2024-09-14T09:07:22.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841938+0000", 2024-09-14T09:07:22.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.292e-06 2024-09-14T09:07:22.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841950+0000", 2024-09-14T09:07:22.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1922e-05 2024-09-14T09:07:22.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841959+0000", 2024-09-14T09:07:22.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7809999999999994e-06 2024-09-14T09:07:22.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842454+0000", 2024-09-14T09:07:22.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00049498399999999996 2024-09-14T09:07:22.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842459+0000", 2024-09-14T09:07:22.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7110000000000003e-06 2024-09-14T09:07:22.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859608+0000", 2024-09-14T09:07:22.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017149640000000001 2024-09-14T09:07:22.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859613+0000", 2024-09-14T09:07:22.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9640000000000002e-06 2024-09-14T09:07:22.491 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.491 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.491 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.491 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105375 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.491 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.846141+0000", 2024-09-14T09:07:22.491 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34368779799999999, 2024-09-14T09:07:22.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34789921200000001, 2024-09-14T09:07:22.491 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.491 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.491 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.492 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.492 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.492 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.492 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105375 2024-09-14T09:07:22.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.492 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846141+0000", 2024-09-14T09:07:22.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846141+0000", 2024-09-14T09:07:22.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846142+0000", 2024-09-14T09:07:22.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.78e-07 2024-09-14T09:07:22.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846146+0000", 2024-09-14T09:07:22.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-06 2024-09-14T09:07:22.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846157+0000", 2024-09-14T09:07:22.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1258e-05 2024-09-14T09:07:22.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846166+0000", 2024-09-14T09:07:22.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.568e-06 2024-09-14T09:07:22.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846229+0000", 2024-09-14T09:07:22.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3742000000000001e-05 2024-09-14T09:07:22.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846238+0000", 2024-09-14T09:07:22.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5599999999999994e-06 2024-09-14T09:07:22.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860780+0000", 2024-09-14T09:07:22.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014542507 2024-09-14T09:07:22.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860789+0000", 2024-09-14T09:07:22.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1670000000000005e-06 2024-09-14T09:07:22.496 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.496 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.496 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.496 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105439 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.496 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.855219+0000", 2024-09-14T09:07:22.497 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.334609343, 2024-09-14T09:07:22.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.338942621, 2024-09-14T09:07:22.497 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.497 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.497 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.497 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.497 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.497 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.497 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105439 2024-09-14T09:07:22.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.497 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855219+0000", 2024-09-14T09:07:22.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855219+0000", 2024-09-14T09:07:22.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855220+0000", 2024-09-14T09:07:22.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4199999999999996e-07 2024-09-14T09:07:22.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855223+0000", 2024-09-14T09:07:22.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8720000000000002e-06 2024-09-14T09:07:22.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855230+0000", 2024-09-14T09:07:22.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5229999999999998e-06 2024-09-14T09:07:22.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.855235+0000", 2024-09-14T09:07:22.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3369999999999999e-06 2024-09-14T09:07:22.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862163+0000", 2024-09-14T09:07:22.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069271200000000002 2024-09-14T09:07:22.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862172+0000", 2024-09-14T09:07:22.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1470000000000007e-06 2024-09-14T09:07:22.500 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.500 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.500 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.500 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105503 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.500 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.859822+0000", 2024-09-14T09:07:22.501 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.330006103, 2024-09-14T09:07:22.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33440331299999998, 2024-09-14T09:07:22.501 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.501 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.501 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.501 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.501 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.501 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.501 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105503 2024-09-14T09:07:22.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.502 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859822+0000", 2024-09-14T09:07:22.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859822+0000", 2024-09-14T09:07:22.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859823+0000", 2024-09-14T09:07:22.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8100000000000001e-07 2024-09-14T09:07:22.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859828+0000", 2024-09-14T09:07:22.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3209999999999998e-06 2024-09-14T09:07:22.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859841+0000", 2024-09-14T09:07:22.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3739e-05 2024-09-14T09:07:22.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859850+0000", 2024-09-14T09:07:22.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4039999999999995e-06 2024-09-14T09:07:22.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863547+0000", 2024-09-14T09:07:22.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0036974149999999999 2024-09-14T09:07:22.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863571+0000", 2024-09-14T09:07:22.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3580000000000001e-05 2024-09-14T09:07:22.506 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.506 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.506 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.506 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105567 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.506 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861265+0000", 2024-09-14T09:07:22.506 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32856329000000001, 2024-09-14T09:07:22.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33302512899999998, 2024-09-14T09:07:22.507 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.507 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.507 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.507 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.507 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.507 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.507 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105567 2024-09-14T09:07:22.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.507 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861265+0000", 2024-09-14T09:07:22.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861265+0000", 2024-09-14T09:07:22.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861266+0000", 2024-09-14T09:07:22.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1099999999999995e-07 2024-09-14T09:07:22.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861269+0000", 2024-09-14T09:07:22.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.909e-06 2024-09-14T09:07:22.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861278+0000", 2024-09-14T09:07:22.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0070000000000003e-06 2024-09-14T09:07:22.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861285+0000", 2024-09-14T09:07:22.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0809999999999996e-06 2024-09-14T09:07:22.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865237+0000", 2024-09-14T09:07:22.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003952176 2024-09-14T09:07:22.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865243+0000", 2024-09-14T09:07:22.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2879999999999998e-06 2024-09-14T09:07:22.510 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.510 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.510 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.510 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105631 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.510 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:07:22.510 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32324826099999998, 2024-09-14T09:07:22.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32777435500000002, 2024-09-14T09:07:22.511 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.511 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.511 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.511 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.511 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.511 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.511 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105631 2024-09-14T09:07:22.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.511 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:07:22.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:07:22.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866581+0000", 2024-09-14T09:07:22.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.879999999999999e-07 2024-09-14T09:07:22.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866585+0000", 2024-09-14T09:07:22.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0239999999999999e-06 2024-09-14T09:07:22.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866598+0000", 2024-09-14T09:07:22.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2862e-05 2024-09-14T09:07:22.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866608+0000", 2024-09-14T09:07:22.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9890000000000004e-06 2024-09-14T09:07:22.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866708+0000", 2024-09-14T09:07:22.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9605999999999995e-05 2024-09-14T09:07:22.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866720+0000", 2024-09-14T09:07:22.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.261e-05 2024-09-14T09:07:22.514 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.514 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.515 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.515 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105695 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.515 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868542+0000", 2024-09-14T09:07:22.515 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32128653800000001, 2024-09-14T09:07:22.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32587398200000001, 2024-09-14T09:07:22.515 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.515 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.515 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.515 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.515 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.515 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.515 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105695 2024-09-14T09:07:22.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.515 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868542+0000", 2024-09-14T09:07:22.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868542+0000", 2024-09-14T09:07:22.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868543+0000", 2024-09-14T09:07:22.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:07:22.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868545+0000", 2024-09-14T09:07:22.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2019999999999998e-06 2024-09-14T09:07:22.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868552+0000", 2024-09-14T09:07:22.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9659999999999999e-06 2024-09-14T09:07:22.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868557+0000", 2024-09-14T09:07:22.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9799999999999998e-06 2024-09-14T09:07:22.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868726+0000", 2024-09-14T09:07:22.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016904800000000001 2024-09-14T09:07:22.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868733+0000", 2024-09-14T09:07:22.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7920000000000004e-06 2024-09-14T09:07:22.519 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.519 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.519 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.519 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105249 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.519 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.840235+0000", 2024-09-14T09:07:22.519 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34959389600000002, 2024-09-14T09:07:22.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35424513699999999, 2024-09-14T09:07:22.520 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.520 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.520 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.520 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.520 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.520 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.520 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105249 2024-09-14T09:07:22.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.520 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840235+0000", 2024-09-14T09:07:22.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840235+0000", 2024-09-14T09:07:22.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840235+0000", 2024-09-14T09:07:22.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-09-14T09:07:22.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840263+0000", 2024-09-14T09:07:22.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8096999999999999e-05 2024-09-14T09:07:22.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840273+0000", 2024-09-14T09:07:22.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8549999999999997e-06 2024-09-14T09:07:22.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840281+0000", 2024-09-14T09:07:22.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9079999999999995e-06 2024-09-14T09:07:22.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840492+0000", 2024-09-14T09:07:22.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021100599999999999 2024-09-14T09:07:22.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840498+0000", 2024-09-14T09:07:22.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5249999999999996e-06 2024-09-14T09:07:22.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856745+0000", 2024-09-14T09:07:22.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016247159000000001 2024-09-14T09:07:22.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856760+0000", 2024-09-14T09:07:22.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4876e-05 2024-09-14T09:07:22.524 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.525 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.525 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.525 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105313 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.525 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841969+0000", 2024-09-14T09:07:22.525 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34785987099999999, 2024-09-14T09:07:22.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35258236500000001, 2024-09-14T09:07:22.525 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.525 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.525 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.525 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.525 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.525 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.526 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105313 2024-09-14T09:07:22.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.526 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841969+0000", 2024-09-14T09:07:22.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841969+0000", 2024-09-14T09:07:22.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841969+0000", 2024-09-14T09:07:22.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1799999999999995e-07 2024-09-14T09:07:22.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841972+0000", 2024-09-14T09:07:22.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8119999999999999e-06 2024-09-14T09:07:22.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841978+0000", 2024-09-14T09:07:22.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6319999999999996e-06 2024-09-14T09:07:22.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841984+0000", 2024-09-14T09:07:22.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4919999999999998e-06 2024-09-14T09:07:22.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842488+0000", 2024-09-14T09:07:22.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050374000000000003 2024-09-14T09:07:22.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842492+0000", 2024-09-14T09:07:22.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6290000000000001e-06 2024-09-14T09:07:22.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859632+0000", 2024-09-14T09:07:22.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017139152000000001 2024-09-14T09:07:22.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859636+0000", 2024-09-14T09:07:22.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-09-14T09:07:22.529 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.529 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.529 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.530 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105376 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.530 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.846250+0000", 2024-09-14T09:07:22.530 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34357809, 2024-09-14T09:07:22.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34837680700000001, 2024-09-14T09:07:22.530 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.530 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.530 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.530 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.530 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.530 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.530 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105376 2024-09-14T09:07:22.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.530 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846250+0000", 2024-09-14T09:07:22.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846250+0000", 2024-09-14T09:07:22.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846251+0000", 2024-09-14T09:07:22.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6800000000000009e-07 2024-09-14T09:07:22.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846256+0000", 2024-09-14T09:07:22.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6020000000000002e-06 2024-09-14T09:07:22.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846268+0000", 2024-09-14T09:07:22.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1493e-05 2024-09-14T09:07:22.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846276+0000", 2024-09-14T09:07:22.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6319999999999997e-06 2024-09-14T09:07:22.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846339+0000", 2024-09-14T09:07:22.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3244000000000006e-05 2024-09-14T09:07:22.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846348+0000", 2024-09-14T09:07:22.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5420000000000005e-06 2024-09-14T09:07:22.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860824+0000", 2024-09-14T09:07:22.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014475949 2024-09-14T09:07:22.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860833+0000", 2024-09-14T09:07:22.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7020000000000007e-06 2024-09-14T09:07:22.534 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.534 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.534 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.534 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105441 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.534 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.856734+0000", 2024-09-14T09:07:22.534 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.333094159, 2024-09-14T09:07:22.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33796713900000003, 2024-09-14T09:07:22.535 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.535 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.535 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.535 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.535 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.535 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.535 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105441 2024-09-14T09:07:22.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.535 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856734+0000", 2024-09-14T09:07:22.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856734+0000", 2024-09-14T09:07:22.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856735+0000", 2024-09-14T09:07:22.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0499999999999999e-06 2024-09-14T09:07:22.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856742+0000", 2024-09-14T09:07:22.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4699999999999999e-06 2024-09-14T09:07:22.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856754+0000", 2024-09-14T09:07:22.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2140999999999999e-05 2024-09-14T09:07:22.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856767+0000", 2024-09-14T09:07:22.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2497000000000001e-05 2024-09-14T09:07:22.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862208+0000", 2024-09-14T09:07:22.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.005440973 2024-09-14T09:07:22.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862217+0000", 2024-09-14T09:07:22.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0739999999999999e-06 2024-09-14T09:07:22.538 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.538 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.538 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.538 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105504 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.538 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.859893+0000", 2024-09-14T09:07:22.538 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32993507300000002, 2024-09-14T09:07:22.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33486888799999998, 2024-09-14T09:07:22.539 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.539 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.539 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.539 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.539 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.539 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.539 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105504 2024-09-14T09:07:22.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.539 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859893+0000", 2024-09-14T09:07:22.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859893+0000", 2024-09-14T09:07:22.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859894+0000", 2024-09-14T09:07:22.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0200000000000001e-07 2024-09-14T09:07:22.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859899+0000", 2024-09-14T09:07:22.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.578e-06 2024-09-14T09:07:22.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859908+0000", 2024-09-14T09:07:22.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5070000000000006e-06 2024-09-14T09:07:22.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859915+0000", 2024-09-14T09:07:22.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.776e-06 2024-09-14T09:07:22.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863666+0000", 2024-09-14T09:07:22.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003751084 2024-09-14T09:07:22.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863681+0000", 2024-09-14T09:07:22.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4642e-05 2024-09-14T09:07:22.542 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.542 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.542 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.542 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105568 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.542 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861296+0000", 2024-09-14T09:07:22.542 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32853257499999999, 2024-09-14T09:07:22.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33353102000000001, 2024-09-14T09:07:22.543 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.543 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.543 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.543 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.543 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.543 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.543 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105568 2024-09-14T09:07:22.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.543 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861296+0000", 2024-09-14T09:07:22.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861296+0000", 2024-09-14T09:07:22.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861297+0000", 2024-09-14T09:07:22.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6599999999999996e-07 2024-09-14T09:07:22.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861300+0000", 2024-09-14T09:07:22.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4539999999999999e-06 2024-09-14T09:07:22.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861309+0000", 2024-09-14T09:07:22.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5290000000000008e-06 2024-09-14T09:07:22.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861315+0000", 2024-09-14T09:07:22.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4099999999999996e-06 2024-09-14T09:07:22.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865268+0000", 2024-09-14T09:07:22.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003953397 2024-09-14T09:07:22.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865275+0000", 2024-09-14T09:07:22.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2729999999999999e-06 2024-09-14T09:07:22.546 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.546 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.546 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.547 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105633 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.547 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:07:22.547 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.323189214, 2024-09-14T09:07:22.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32825189399999999, 2024-09-14T09:07:22.547 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.547 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.547 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.547 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.547 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.547 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.547 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105633 2024-09-14T09:07:22.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.547 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:07:22.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:07:22.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866640+0000", 2024-09-14T09:07:22.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9300000000000006e-07 2024-09-14T09:07:22.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866656+0000", 2024-09-14T09:07:22.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5291e-05 2024-09-14T09:07:22.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866669+0000", 2024-09-14T09:07:22.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2918000000000001e-05 2024-09-14T09:07:22.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866678+0000", 2024-09-14T09:07:22.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8369999999999992e-06 2024-09-14T09:07:22.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866771+0000", 2024-09-14T09:07:22.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2250999999999995e-05 2024-09-14T09:07:22.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866784+0000", 2024-09-14T09:07:22.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2991e-05 2024-09-14T09:07:22.551 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.551 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.551 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.551 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105697 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.551 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868569+0000", 2024-09-14T09:07:22.551 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32125991599999998, 2024-09-14T09:07:22.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32638397200000002, 2024-09-14T09:07:22.551 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.551 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.551 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.551 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.552 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.552 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.552 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105697 2024-09-14T09:07:22.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.552 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868569+0000", 2024-09-14T09:07:22.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868569+0000", 2024-09-14T09:07:22.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868570+0000", 2024-09-14T09:07:22.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.048e-06 2024-09-14T09:07:22.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868574+0000", 2024-09-14T09:07:22.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8580000000000002e-06 2024-09-14T09:07:22.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868585+0000", 2024-09-14T09:07:22.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1012000000000001e-05 2024-09-14T09:07:22.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868594+0000", 2024-09-14T09:07:22.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3959999999999995e-06 2024-09-14T09:07:22.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868763+0000", 2024-09-14T09:07:22.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000168858 2024-09-14T09:07:22.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868770+0000", 2024-09-14T09:07:22.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7240000000000003e-06 2024-09-14T09:07:22.555 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.555 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.555 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.556 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105251 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.556 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.840292+0000", 2024-09-14T09:07:22.556 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34953611699999998, 2024-09-14T09:07:22.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35472400100000001, 2024-09-14T09:07:22.556 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.556 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.556 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.556 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.556 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.556 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.556 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105251 2024-09-14T09:07:22.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.556 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840292+0000", 2024-09-14T09:07:22.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840292+0000", 2024-09-14T09:07:22.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840293+0000", 2024-09-14T09:07:22.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2500000000000005e-07 2024-09-14T09:07:22.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840297+0000", 2024-09-14T09:07:22.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3569999999999998e-06 2024-09-14T09:07:22.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840305+0000", 2024-09-14T09:07:22.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9449999999999999e-06 2024-09-14T09:07:22.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840313+0000", 2024-09-14T09:07:22.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6799999999999993e-06 2024-09-14T09:07:22.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840530+0000", 2024-09-14T09:07:22.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021637499999999999 2024-09-14T09:07:22.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840534+0000", 2024-09-14T09:07:22.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8720000000000001e-06 2024-09-14T09:07:22.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856851+0000", 2024-09-14T09:07:22.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.01631693 2024-09-14T09:07:22.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856861+0000", 2024-09-14T09:07:22.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0001e-05 2024-09-14T09:07:22.560 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.560 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.560 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.560 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105315 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.560 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.842018+0000", 2024-09-14T09:07:22.560 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.347810603, 2024-09-14T09:07:22.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35307000900000002, 2024-09-14T09:07:22.561 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.561 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.561 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.561 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.561 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.561 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.561 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105315 2024-09-14T09:07:22.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.561 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842018+0000", 2024-09-14T09:07:22.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842018+0000", 2024-09-14T09:07:22.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842018+0000", 2024-09-14T09:07:22.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.06e-07 2024-09-14T09:07:22.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842021+0000", 2024-09-14T09:07:22.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1280000000000002e-06 2024-09-14T09:07:22.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842027+0000", 2024-09-14T09:07:22.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1659999999999999e-06 2024-09-14T09:07:22.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842032+0000", 2024-09-14T09:07:22.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.882e-06 2024-09-14T09:07:22.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842519+0000", 2024-09-14T09:07:22.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048737599999999998 2024-09-14T09:07:22.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842524+0000", 2024-09-14T09:07:22.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6770000000000003e-06 2024-09-14T09:07:22.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859664+0000", 2024-09-14T09:07:22.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017140193000000001 2024-09-14T09:07:22.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859673+0000", 2024-09-14T09:07:22.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4460000000000007e-06 2024-09-14T09:07:22.565 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.565 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.565 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.565 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105379 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.565 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.846361+0000", 2024-09-14T09:07:22.565 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34346716100000002, 2024-09-14T09:07:22.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34880153800000002, 2024-09-14T09:07:22.566 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.566 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.566 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.566 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.566 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.566 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.566 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105379 2024-09-14T09:07:22.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.566 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846361+0000", 2024-09-14T09:07:22.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846361+0000", 2024-09-14T09:07:22.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846362+0000", 2024-09-14T09:07:22.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.006e-06 2024-09-14T09:07:22.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846366+0000", 2024-09-14T09:07:22.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9480000000000001e-06 2024-09-14T09:07:22.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846378+0000", 2024-09-14T09:07:22.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1506e-05 2024-09-14T09:07:22.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846387+0000", 2024-09-14T09:07:22.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7280000000000001e-06 2024-09-14T09:07:22.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846470+0000", 2024-09-14T09:07:22.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3319000000000006e-05 2024-09-14T09:07:22.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846478+0000", 2024-09-14T09:07:22.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4020000000000002e-06 2024-09-14T09:07:22.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860866+0000", 2024-09-14T09:07:22.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014387972000000001 2024-09-14T09:07:22.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860875+0000", 2024-09-14T09:07:22.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1169999999999993e-06 2024-09-14T09:07:22.570 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.570 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.570 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.571 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105443 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.571 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.856792+0000", 2024-09-14T09:07:22.571 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33303704099999998, 2024-09-14T09:07:22.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33844601000000002, 2024-09-14T09:07:22.571 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.571 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.571 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.571 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.571 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.571 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.571 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105443 2024-09-14T09:07:22.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.571 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856792+0000", 2024-09-14T09:07:22.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856792+0000", 2024-09-14T09:07:22.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856792+0000", 2024-09-14T09:07:22.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3300000000000001e-07 2024-09-14T09:07:22.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856807+0000", 2024-09-14T09:07:22.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4644000000000001e-05 2024-09-14T09:07:22.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856818+0000", 2024-09-14T09:07:22.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1017e-05 2024-09-14T09:07:22.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856826+0000", 2024-09-14T09:07:22.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5639999999999997e-06 2024-09-14T09:07:22.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862252+0000", 2024-09-14T09:07:22.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054259920000000001 2024-09-14T09:07:22.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862274+0000", 2024-09-14T09:07:22.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1393e-05 2024-09-14T09:07:22.575 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.575 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.575 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.575 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105507 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.575 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.859926+0000", 2024-09-14T09:07:22.575 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32990244099999999, 2024-09-14T09:07:22.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33537256999999998, 2024-09-14T09:07:22.575 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.575 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.575 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.575 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.576 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.576 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.576 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105507 2024-09-14T09:07:22.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.576 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859926+0000", 2024-09-14T09:07:22.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859926+0000", 2024-09-14T09:07:22.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859927+0000", 2024-09-14T09:07:22.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0800000000000005e-07 2024-09-14T09:07:22.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859929+0000", 2024-09-14T09:07:22.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7199999999999998e-06 2024-09-14T09:07:22.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859936+0000", 2024-09-14T09:07:22.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6959999999999999e-06 2024-09-14T09:07:22.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859942+0000", 2024-09-14T09:07:22.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2570000000000004e-06 2024-09-14T09:07:22.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863783+0000", 2024-09-14T09:07:22.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038406899999999999 2024-09-14T09:07:22.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863793+0000", 2024-09-14T09:07:22.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.594e-06 2024-09-14T09:07:22.579 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.579 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.579 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.579 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105572 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.579 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861324+0000", 2024-09-14T09:07:22.579 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32850469700000001, 2024-09-14T09:07:22.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.334038803, 2024-09-14T09:07:22.579 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.579 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.580 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.580 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.580 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.580 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.580 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105572 2024-09-14T09:07:22.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.580 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861324+0000", 2024-09-14T09:07:22.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861324+0000", 2024-09-14T09:07:22.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861324+0000", 2024-09-14T09:07:22.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7199999999999999e-07 2024-09-14T09:07:22.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861327+0000", 2024-09-14T09:07:22.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.446e-06 2024-09-14T09:07:22.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861335+0000", 2024-09-14T09:07:22.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3950000000000001e-06 2024-09-14T09:07:22.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861342+0000", 2024-09-14T09:07:22.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.832e-06 2024-09-14T09:07:22.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865300+0000", 2024-09-14T09:07:22.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003957889 2024-09-14T09:07:22.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865306+0000", 2024-09-14T09:07:22.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3389999999999998e-06 2024-09-14T09:07:22.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.583 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.583 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105635 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.583 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:07:22.584 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.323135387, 2024-09-14T09:07:22.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.328733479, 2024-09-14T09:07:22.584 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.584 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.584 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.584 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105635 2024-09-14T09:07:22.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.584 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:07:22.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:07:22.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866694+0000", 2024-09-14T09:07:22.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0210000000000001e-06 2024-09-14T09:07:22.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866698+0000", 2024-09-14T09:07:22.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.765e-06 2024-09-14T09:07:22.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866710+0000", 2024-09-14T09:07:22.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2320000000000001e-05 2024-09-14T09:07:22.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866722+0000", 2024-09-14T09:07:22.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2133000000000001e-05 2024-09-14T09:07:22.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866830+0000", 2024-09-14T09:07:22.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000107668 2024-09-14T09:07:22.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866843+0000", 2024-09-14T09:07:22.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2991e-05 2024-09-14T09:07:22.588 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.588 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.588 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.588 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105700 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.588 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868606+0000", 2024-09-14T09:07:22.588 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.321222956, 2024-09-14T09:07:22.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.326882389, 2024-09-14T09:07:22.588 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.588 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.588 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.588 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.589 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.589 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.589 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105700 2024-09-14T09:07:22.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.589 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868606+0000", 2024-09-14T09:07:22.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868606+0000", 2024-09-14T09:07:22.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868606+0000", 2024-09-14T09:07:22.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9999999999999997e-07 2024-09-14T09:07:22.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868610+0000", 2024-09-14T09:07:22.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2559999999999998e-06 2024-09-14T09:07:22.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868620+0000", 2024-09-14T09:07:22.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0719e-05 2024-09-14T09:07:22.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868629+0000", 2024-09-14T09:07:22.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4659999999999999e-06 2024-09-14T09:07:22.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868797+0000", 2024-09-14T09:07:22.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016843899999999999 2024-09-14T09:07:22.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868804+0000", 2024-09-14T09:07:22.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8260000000000004e-06 2024-09-14T09:07:22.591 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.592 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.592 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.592 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105254 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.592 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.840320+0000", 2024-09-14T09:07:22.592 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34950846299999999, 2024-09-14T09:07:22.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.355233032, 2024-09-14T09:07:22.592 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.592 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.592 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.592 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.592 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.593 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.593 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105254 2024-09-14T09:07:22.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.593 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840320+0000", 2024-09-14T09:07:22.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840320+0000", 2024-09-14T09:07:22.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840321+0000", 2024-09-14T09:07:22.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2500000000000001e-07 2024-09-14T09:07:22.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840338+0000", 2024-09-14T09:07:22.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7314999999999999e-05 2024-09-14T09:07:22.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840345+0000", 2024-09-14T09:07:22.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.204e-06 2024-09-14T09:07:22.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840349+0000", 2024-09-14T09:07:22.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1400000000000002e-06 2024-09-14T09:07:22.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840625+0000", 2024-09-14T09:07:22.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00027571399999999998 2024-09-14T09:07:22.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840640+0000", 2024-09-14T09:07:22.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5400000000000002e-05 2024-09-14T09:07:22.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856900+0000", 2024-09-14T09:07:22.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016259462999999998 2024-09-14T09:07:22.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856907+0000", 2024-09-14T09:07:22.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.878e-06 2024-09-14T09:07:22.596 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.596 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.596 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.597 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105316 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.597 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.842050+0000", 2024-09-14T09:07:22.597 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34777878099999998, 2024-09-14T09:07:22.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35357506, 2024-09-14T09:07:22.597 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.597 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.597 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.597 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.597 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.598 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.598 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105316 2024-09-14T09:07:22.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.598 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842050+0000", 2024-09-14T09:07:22.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842050+0000", 2024-09-14T09:07:22.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842050+0000", 2024-09-14T09:07:22.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6400000000000003e-07 2024-09-14T09:07:22.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842052+0000", 2024-09-14T09:07:22.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0660000000000002e-06 2024-09-14T09:07:22.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842058+0000", 2024-09-14T09:07:22.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1279999999999996e-06 2024-09-14T09:07:22.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842067+0000", 2024-09-14T09:07:22.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0359999999999995e-06 2024-09-14T09:07:22.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842551+0000", 2024-09-14T09:07:22.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048314199999999999 2024-09-14T09:07:22.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842555+0000", 2024-09-14T09:07:22.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.566e-06 2024-09-14T09:07:22.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859699+0000", 2024-09-14T09:07:22.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017144047999999999 2024-09-14T09:07:22.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859704+0000", 2024-09-14T09:07:22.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4939999999999997e-06 2024-09-14T09:07:22.602 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.602 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.602 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.602 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105381 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.603 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.846419+0000", 2024-09-14T09:07:22.603 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.343409148, 2024-09-14T09:07:22.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34927971800000002, 2024-09-14T09:07:22.603 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.603 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.603 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.603 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.603 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.603 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.603 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105381 2024-09-14T09:07:22.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.603 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846419+0000", 2024-09-14T09:07:22.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846419+0000", 2024-09-14T09:07:22.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846420+0000", 2024-09-14T09:07:22.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2800000000000005e-07 2024-09-14T09:07:22.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846424+0000", 2024-09-14T09:07:22.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.658e-06 2024-09-14T09:07:22.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846436+0000", 2024-09-14T09:07:22.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1642e-05 2024-09-14T09:07:22.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846445+0000", 2024-09-14T09:07:22.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9420000000000001e-06 2024-09-14T09:07:22.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846510+0000", 2024-09-14T09:07:22.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5890999999999996e-05 2024-09-14T09:07:22.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846530+0000", 2024-09-14T09:07:22.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9358e-05 2024-09-14T09:07:22.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860896+0000", 2024-09-14T09:07:22.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014366016000000001 2024-09-14T09:07:22.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860900+0000", 2024-09-14T09:07:22.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3710000000000002e-06 2024-09-14T09:07:22.607 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.607 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.607 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.607 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105445 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.607 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.856838+0000", 2024-09-14T09:07:22.607 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33299074400000001, 2024-09-14T09:07:22.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.338936391, 2024-09-14T09:07:22.608 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.608 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.608 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.608 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.608 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.608 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.608 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105445 2024-09-14T09:07:22.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.608 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856838+0000", 2024-09-14T09:07:22.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856838+0000", 2024-09-14T09:07:22.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856838+0000", 2024-09-14T09:07:22.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8100000000000002e-07 2024-09-14T09:07:22.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856842+0000", 2024-09-14T09:07:22.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1820000000000002e-06 2024-09-14T09:07:22.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856851+0000", 2024-09-14T09:07:22.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0110000000000006e-06 2024-09-14T09:07:22.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856857+0000", 2024-09-14T09:07:22.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6460000000000004e-06 2024-09-14T09:07:22.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862311+0000", 2024-09-14T09:07:22.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054539799999999998 2024-09-14T09:07:22.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862321+0000", 2024-09-14T09:07:22.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5799999999999998e-06 2024-09-14T09:07:22.614 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.614 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.615 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.615 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105509 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.615 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.859965+0000", 2024-09-14T09:07:22.615 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32986345299999997, 2024-09-14T09:07:22.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33587025300000001, 2024-09-14T09:07:22.616 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.616 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.616 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.616 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.616 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.617 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.617 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105509 2024-09-14T09:07:22.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.617 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859965+0000", 2024-09-14T09:07:22.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859965+0000", 2024-09-14T09:07:22.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859966+0000", 2024-09-14T09:07:22.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8699999999999995e-07 2024-09-14T09:07:22.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859969+0000", 2024-09-14T09:07:22.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.858e-06 2024-09-14T09:07:22.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859976+0000", 2024-09-14T09:07:22.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.79e-06 2024-09-14T09:07:22.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859983+0000", 2024-09-14T09:07:22.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3570000000000003e-06 2024-09-14T09:07:22.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863860+0000", 2024-09-14T09:07:22.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038775569999999998 2024-09-14T09:07:22.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863870+0000", 2024-09-14T09:07:22.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2939999999999994e-06 2024-09-14T09:07:22.624 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.624 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.624 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.625 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105573 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.625 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861368+0000", 2024-09-14T09:07:22.625 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.328460423, 2024-09-14T09:07:22.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33453122699999999, 2024-09-14T09:07:22.626 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.626 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.626 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.626 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.626 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.626 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.626 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105573 2024-09-14T09:07:22.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.627 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861368+0000", 2024-09-14T09:07:22.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861368+0000", 2024-09-14T09:07:22.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861369+0000", 2024-09-14T09:07:22.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3600000000000003e-07 2024-09-14T09:07:22.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861372+0000", 2024-09-14T09:07:22.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.083e-06 2024-09-14T09:07:22.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861381+0000", 2024-09-14T09:07:22.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8130000000000002e-06 2024-09-14T09:07:22.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861388+0000", 2024-09-14T09:07:22.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:07:22.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865332+0000", 2024-09-14T09:07:22.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039442269999999998 2024-09-14T09:07:22.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865338+0000", 2024-09-14T09:07:22.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3899999999999998e-06 2024-09-14T09:07:22.630 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.630 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.630 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.631 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105636 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.631 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:07:22.631 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32306749099999998, 2024-09-14T09:07:22.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32919948799999998, 2024-09-14T09:07:22.631 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.631 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.631 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.631 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.631 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.631 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.631 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105636 2024-09-14T09:07:22.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.632 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:07:22.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:07:22.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866762+0000", 2024-09-14T09:07:22.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4600000000000003e-07 2024-09-14T09:07:22.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866766+0000", 2024-09-14T09:07:22.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:07:22.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866779+0000", 2024-09-14T09:07:22.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3101999999999999e-05 2024-09-14T09:07:22.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866789+0000", 2024-09-14T09:07:22.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8039999999999996e-06 2024-09-14T09:07:22.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866889+0000", 2024-09-14T09:07:22.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100147 2024-09-14T09:07:22.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866902+0000", 2024-09-14T09:07:22.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3212e-05 2024-09-14T09:07:22.635 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.635 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.635 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.635 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105701 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.635 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868640+0000", 2024-09-14T09:07:22.635 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32118859900000002, 2024-09-14T09:07:22.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32738916200000001, 2024-09-14T09:07:22.635 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.635 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.635 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.635 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.635 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.636 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.636 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105701 2024-09-14T09:07:22.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.636 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868640+0000", 2024-09-14T09:07:22.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868640+0000", 2024-09-14T09:07:22.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868641+0000", 2024-09-14T09:07:22.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3200000000000004e-07 2024-09-14T09:07:22.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868644+0000", 2024-09-14T09:07:22.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4599999999999999e-06 2024-09-14T09:07:22.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868653+0000", 2024-09-14T09:07:22.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7819999999999999e-06 2024-09-14T09:07:22.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868658+0000", 2024-09-14T09:07:22.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.758e-06 2024-09-14T09:07:22.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868832+0000", 2024-09-14T09:07:22.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000173914 2024-09-14T09:07:22.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868839+0000", 2024-09-14T09:07:22.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9399999999999996e-06 2024-09-14T09:07:22.638 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.639 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.639 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.639 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105255 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.639 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.840356+0000", 2024-09-14T09:07:22.639 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34947238400000002, 2024-09-14T09:07:22.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35573704499999997, 2024-09-14T09:07:22.639 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.639 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.639 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.639 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.639 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.639 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.639 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105255 2024-09-14T09:07:22.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.640 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840356+0000", 2024-09-14T09:07:22.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840356+0000", 2024-09-14T09:07:22.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840357+0000", 2024-09-14T09:07:22.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4200000000000001e-07 2024-09-14T09:07:22.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840358+0000", 2024-09-14T09:07:22.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.623e-06 2024-09-14T09:07:22.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840364+0000", 2024-09-14T09:07:22.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1419999999999998e-06 2024-09-14T09:07:22.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840372+0000", 2024-09-14T09:07:22.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1579999999999999e-06 2024-09-14T09:07:22.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840736+0000", 2024-09-14T09:07:22.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00036479600000000002 2024-09-14T09:07:22.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840749+0000", 2024-09-14T09:07:22.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2626000000000001e-05 2024-09-14T09:07:22.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856954+0000", 2024-09-14T09:07:22.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016205161999999999 2024-09-14T09:07:22.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856961+0000", 2024-09-14T09:07:22.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0099999999999998e-06 2024-09-14T09:07:22.643 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.643 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.643 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.643 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105319 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.644 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.842077+0000", 2024-09-14T09:07:22.644 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34775157099999998, 2024-09-14T09:07:22.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35411157700000001, 2024-09-14T09:07:22.644 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.644 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.644 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.644 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.644 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.644 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.644 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105319 2024-09-14T09:07:22.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.644 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842077+0000", 2024-09-14T09:07:22.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842077+0000", 2024-09-14T09:07:22.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842077+0000", 2024-09-14T09:07:22.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1200000000000003e-07 2024-09-14T09:07:22.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842080+0000", 2024-09-14T09:07:22.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0420000000000001e-06 2024-09-14T09:07:22.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842085+0000", 2024-09-14T09:07:22.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6529999999999999e-06 2024-09-14T09:07:22.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842090+0000", 2024-09-14T09:07:22.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6879999999999998e-06 2024-09-14T09:07:22.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842581+0000", 2024-09-14T09:07:22.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00049160000000000002 2024-09-14T09:07:22.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842586+0000", 2024-09-14T09:07:22.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7929999999999997e-06 2024-09-14T09:07:22.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859721+0000", 2024-09-14T09:07:22.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017135147999999999 2024-09-14T09:07:22.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859726+0000", 2024-09-14T09:07:22.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3080000000000001e-06 2024-09-14T09:07:22.651 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.651 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.651 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.651 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105383 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.651 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.846457+0000", 2024-09-14T09:07:22.651 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34337156800000002, 2024-09-14T09:07:22.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34981007800000002, 2024-09-14T09:07:22.651 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.651 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.651 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.651 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.652 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.652 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.652 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105383 2024-09-14T09:07:22.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.652 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846457+0000", 2024-09-14T09:07:22.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846457+0000", 2024-09-14T09:07:22.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846458+0000", 2024-09-14T09:07:22.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4799999999999997e-07 2024-09-14T09:07:22.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846461+0000", 2024-09-14T09:07:22.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2499999999999998e-06 2024-09-14T09:07:22.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846472+0000", 2024-09-14T09:07:22.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.075e-05 2024-09-14T09:07:22.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846480+0000", 2024-09-14T09:07:22.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3799999999999994e-06 2024-09-14T09:07:22.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846564+0000", 2024-09-14T09:07:22.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3378e-05 2024-09-14T09:07:22.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846569+0000", 2024-09-14T09:07:22.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0710000000000002e-06 2024-09-14T09:07:22.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860918+0000", 2024-09-14T09:07:22.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014349041999999999 2024-09-14T09:07:22.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860922+0000", 2024-09-14T09:07:22.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.109e-06 2024-09-14T09:07:22.656 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.656 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.656 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.656 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105447 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.656 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.856867+0000", 2024-09-14T09:07:22.656 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33296118899999999, 2024-09-14T09:07:22.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33947504299999998, 2024-09-14T09:07:22.657 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.657 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.657 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.657 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.657 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.657 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.657 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105447 2024-09-14T09:07:22.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.657 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856867+0000", 2024-09-14T09:07:22.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856867+0000", 2024-09-14T09:07:22.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856868+0000", 2024-09-14T09:07:22.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8299999999999996e-07 2024-09-14T09:07:22.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856871+0000", 2024-09-14T09:07:22.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7190000000000001e-06 2024-09-14T09:07:22.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856880+0000", 2024-09-14T09:07:22.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2739999999999996e-06 2024-09-14T09:07:22.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856887+0000", 2024-09-14T09:07:22.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.764e-06 2024-09-14T09:07:22.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862366+0000", 2024-09-14T09:07:22.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054793999999999997 2024-09-14T09:07:22.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862376+0000", 2024-09-14T09:07:22.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7850000000000003e-06 2024-09-14T09:07:22.660 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.660 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.661 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.661 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105512 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.661 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860010+0000", 2024-09-14T09:07:22.661 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32981891099999999, 2024-09-14T09:07:22.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33639371800000001, 2024-09-14T09:07:22.661 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.661 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.661 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.661 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.661 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.662 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.662 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105512 2024-09-14T09:07:22.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.662 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860010+0000", 2024-09-14T09:07:22.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860010+0000", 2024-09-14T09:07:22.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860010+0000", 2024-09-14T09:07:22.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9200000000000001e-07 2024-09-14T09:07:22.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860013+0000", 2024-09-14T09:07:22.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9469999999999999e-06 2024-09-14T09:07:22.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860021+0000", 2024-09-14T09:07:22.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3409999999999997e-06 2024-09-14T09:07:22.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860028+0000", 2024-09-14T09:07:22.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1369999999999996e-06 2024-09-14T09:07:22.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863973+0000", 2024-09-14T09:07:22.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039457479999999998 2024-09-14T09:07:22.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863990+0000", 2024-09-14T09:07:22.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6415000000000001e-05 2024-09-14T09:07:22.665 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.665 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.665 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.666 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105575 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.666 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861415+0000", 2024-09-14T09:07:22.666 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32841312700000003, 2024-09-14T09:07:22.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.335052189, 2024-09-14T09:07:22.666 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.666 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.666 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.666 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.666 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.666 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.666 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105575 2024-09-14T09:07:22.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.666 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861415+0000", 2024-09-14T09:07:22.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861415+0000", 2024-09-14T09:07:22.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861416+0000", 2024-09-14T09:07:22.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.68e-07 2024-09-14T09:07:22.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861419+0000", 2024-09-14T09:07:22.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.999e-06 2024-09-14T09:07:22.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861428+0000", 2024-09-14T09:07:22.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.721e-06 2024-09-14T09:07:22.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861435+0000", 2024-09-14T09:07:22.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9999999999999999e-06 2024-09-14T09:07:22.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865364+0000", 2024-09-14T09:07:22.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003929147 2024-09-14T09:07:22.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865370+0000", 2024-09-14T09:07:22.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4180000000000002e-06 2024-09-14T09:07:22.670 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.670 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.670 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.670 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105639 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.670 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:07:22.670 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32300861600000003, 2024-09-14T09:07:22.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32970914200000001, 2024-09-14T09:07:22.670 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.671 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.671 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.671 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.671 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.671 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.671 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105639 2024-09-14T09:07:22.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.671 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:07:22.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:07:22.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866821+0000", 2024-09-14T09:07:22.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9800000000000002e-07 2024-09-14T09:07:22.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866825+0000", 2024-09-14T09:07:22.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9269999999999998e-06 2024-09-14T09:07:22.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866838+0000", 2024-09-14T09:07:22.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3040000000000001e-05 2024-09-14T09:07:22.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866848+0000", 2024-09-14T09:07:22.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7929999999999992e-06 2024-09-14T09:07:22.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866948+0000", 2024-09-14T09:07:22.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100337 2024-09-14T09:07:22.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866961+0000", 2024-09-14T09:07:22.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3106e-05 2024-09-14T09:07:22.674 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.674 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.674 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.674 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105703 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.674 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868673+0000", 2024-09-14T09:07:22.674 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.321155409, 2024-09-14T09:07:22.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32791994099999999, 2024-09-14T09:07:22.675 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.675 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.675 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.675 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.675 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.675 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.675 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105703 2024-09-14T09:07:22.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.675 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868673+0000", 2024-09-14T09:07:22.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868673+0000", 2024-09-14T09:07:22.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868674+0000", 2024-09-14T09:07:22.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1200000000000003e-07 2024-09-14T09:07:22.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868677+0000", 2024-09-14T09:07:22.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8559999999999998e-06 2024-09-14T09:07:22.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868688+0000", 2024-09-14T09:07:22.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1209999999999999e-05 2024-09-14T09:07:22.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868696+0000", 2024-09-14T09:07:22.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3999999999999992e-06 2024-09-14T09:07:22.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868868+0000", 2024-09-14T09:07:22.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017146800000000001 2024-09-14T09:07:22.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868874+0000", 2024-09-14T09:07:22.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.618e-06 2024-09-14T09:07:22.678 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.678 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.678 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.679 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105257 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.679 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.840400+0000", 2024-09-14T09:07:22.679 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34942852499999999, 2024-09-14T09:07:22.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35625688999999999, 2024-09-14T09:07:22.679 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.679 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.679 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.679 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.679 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.679 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.679 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105257 2024-09-14T09:07:22.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.679 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840400+0000", 2024-09-14T09:07:22.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840400+0000", 2024-09-14T09:07:22.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840401+0000", 2024-09-14T09:07:22.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5300000000000004e-07 2024-09-14T09:07:22.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840404+0000", 2024-09-14T09:07:22.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.18e-06 2024-09-14T09:07:22.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840413+0000", 2024-09-14T09:07:22.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4360000000000008e-06 2024-09-14T09:07:22.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840418+0000", 2024-09-14T09:07:22.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.865e-06 2024-09-14T09:07:22.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840825+0000", 2024-09-14T09:07:22.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040704199999999998 2024-09-14T09:07:22.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840837+0000", 2024-09-14T09:07:22.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2091e-05 2024-09-14T09:07:22.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857011+0000", 2024-09-14T09:07:22.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016174010999999999 2024-09-14T09:07:22.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857022+0000", 2024-09-14T09:07:22.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0576e-05 2024-09-14T09:07:22.683 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.683 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.683 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.684 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105321 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.684 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.842095+0000", 2024-09-14T09:07:22.684 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34773319000000003, 2024-09-14T09:07:22.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35463339199999999, 2024-09-14T09:07:22.684 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.684 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.684 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.684 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.684 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.684 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.684 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105321 2024-09-14T09:07:22.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.684 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842095+0000", 2024-09-14T09:07:22.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842095+0000", 2024-09-14T09:07:22.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842096+0000", 2024-09-14T09:07:22.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4999999999999998e-07 2024-09-14T09:07:22.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842098+0000", 2024-09-14T09:07:22.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7940000000000001e-06 2024-09-14T09:07:22.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842103+0000", 2024-09-14T09:07:22.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2279999999999998e-06 2024-09-14T09:07:22.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842107+0000", 2024-09-14T09:07:22.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5020000000000003e-06 2024-09-14T09:07:22.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842619+0000", 2024-09-14T09:07:22.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00051123499999999997 2024-09-14T09:07:22.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842623+0000", 2024-09-14T09:07:22.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6070000000000001e-06 2024-09-14T09:07:22.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859755+0000", 2024-09-14T09:07:22.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017131691000000001 2024-09-14T09:07:22.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859764+0000", 2024-09-14T09:07:22.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6670000000000002e-06 2024-09-14T09:07:22.688 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.688 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.688 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.689 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105385 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.689 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.846519+0000", 2024-09-14T09:07:22.689 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34330918300000002, 2024-09-14T09:07:22.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.350283389, 2024-09-14T09:07:22.689 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.689 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.689 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.689 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.689 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.689 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.689 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105385 2024-09-14T09:07:22.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.689 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846519+0000", 2024-09-14T09:07:22.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846519+0000", 2024-09-14T09:07:22.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846521+0000", 2024-09-14T09:07:22.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.254e-06 2024-09-14T09:07:22.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846525+0000", 2024-09-14T09:07:22.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.33e-06 2024-09-14T09:07:22.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846537+0000", 2024-09-14T09:07:22.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1926000000000001e-05 2024-09-14T09:07:22.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846546+0000", 2024-09-14T09:07:22.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3419999999999997e-06 2024-09-14T09:07:22.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846602+0000", 2024-09-14T09:07:22.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6258999999999998e-05 2024-09-14T09:07:22.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846607+0000", 2024-09-14T09:07:22.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4270000000000001e-06 2024-09-14T09:07:22.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860952+0000", 2024-09-14T09:07:22.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014344623000000001 2024-09-14T09:07:22.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860956+0000", 2024-09-14T09:07:22.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.826e-06 2024-09-14T09:07:22.693 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.693 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.693 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.694 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105449 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.694 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.856897+0000", 2024-09-14T09:07:22.694 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33293188299999998, 2024-09-14T09:07:22.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33998055700000002, 2024-09-14T09:07:22.694 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.694 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.694 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.694 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.694 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.694 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.694 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105449 2024-09-14T09:07:22.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.695 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856897+0000", 2024-09-14T09:07:22.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856897+0000", 2024-09-14T09:07:22.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856897+0000", 2024-09-14T09:07:22.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:07:22.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856900+0000", 2024-09-14T09:07:22.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.791e-06 2024-09-14T09:07:22.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856908+0000", 2024-09-14T09:07:22.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1759999999999993e-06 2024-09-14T09:07:22.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856915+0000", 2024-09-14T09:07:22.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1609999999999997e-06 2024-09-14T09:07:22.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862413+0000", 2024-09-14T09:07:22.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054979110000000003 2024-09-14T09:07:22.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862422+0000", 2024-09-14T09:07:22.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8559999999999996e-06 2024-09-14T09:07:22.698 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.698 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.698 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.698 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105513 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.698 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860056+0000", 2024-09-14T09:07:22.698 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32977222899999997, 2024-09-14T09:07:22.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33690209300000001, 2024-09-14T09:07:22.698 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.698 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.698 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.698 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.699 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.699 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.699 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105513 2024-09-14T09:07:22.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.699 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860056+0000", 2024-09-14T09:07:22.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860056+0000", 2024-09-14T09:07:22.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860057+0000", 2024-09-14T09:07:22.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.51e-07 2024-09-14T09:07:22.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860059+0000", 2024-09-14T09:07:22.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6010000000000002e-06 2024-09-14T09:07:22.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860067+0000", 2024-09-14T09:07:22.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1940000000000001e-06 2024-09-14T09:07:22.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860074+0000", 2024-09-14T09:07:22.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8480000000000003e-06 2024-09-14T09:07:22.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864067+0000", 2024-09-14T09:07:22.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039938109999999999 2024-09-14T09:07:22.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864082+0000", 2024-09-14T09:07:22.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4932e-05 2024-09-14T09:07:22.702 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.702 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.702 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.702 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105577 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.702 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861462+0000", 2024-09-14T09:07:22.703 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32836651900000002, 2024-09-14T09:07:22.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33556285699999999, 2024-09-14T09:07:22.703 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.703 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.703 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.703 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.703 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.703 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.703 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105577 2024-09-14T09:07:22.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.703 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861462+0000", 2024-09-14T09:07:22.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861462+0000", 2024-09-14T09:07:22.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861463+0000", 2024-09-14T09:07:22.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6700000000000003e-07 2024-09-14T09:07:22.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861466+0000", 2024-09-14T09:07:22.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-09-14T09:07:22.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861474+0000", 2024-09-14T09:07:22.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7099999999999996e-06 2024-09-14T09:07:22.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861481+0000", 2024-09-14T09:07:22.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.849e-06 2024-09-14T09:07:22.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865396+0000", 2024-09-14T09:07:22.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039149240000000002 2024-09-14T09:07:22.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865403+0000", 2024-09-14T09:07:22.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.455e-06 2024-09-14T09:07:22.706 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.706 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.706 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.707 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105641 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.707 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:07:22.707 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32294975100000001, 2024-09-14T09:07:22.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33020733200000002, 2024-09-14T09:07:22.707 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.707 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.707 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.707 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.707 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.707 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.707 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105641 2024-09-14T09:07:22.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.708 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:07:22.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:07:22.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866880+0000", 2024-09-14T09:07:22.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7699999999999992e-07 2024-09-14T09:07:22.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866884+0000", 2024-09-14T09:07:22.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9480000000000001e-06 2024-09-14T09:07:22.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866897+0000", 2024-09-14T09:07:22.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2832e-05 2024-09-14T09:07:22.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866906+0000", 2024-09-14T09:07:22.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7869999999999996e-06 2024-09-14T09:07:22.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867007+0000", 2024-09-14T09:07:22.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010103900000000001 2024-09-14T09:07:22.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867020+0000", 2024-09-14T09:07:22.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2896e-05 2024-09-14T09:07:22.711 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.711 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.711 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.711 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105704 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.711 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868723+0000", 2024-09-14T09:07:22.711 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32110513499999999, 2024-09-14T09:07:22.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32842664700000002, 2024-09-14T09:07:22.711 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.711 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.711 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.711 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.712 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.712 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.712 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105704 2024-09-14T09:07:22.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.712 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868723+0000", 2024-09-14T09:07:22.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868723+0000", 2024-09-14T09:07:22.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868724+0000", 2024-09-14T09:07:22.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7800000000000001e-07 2024-09-14T09:07:22.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868735+0000", 2024-09-14T09:07:22.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0356e-05 2024-09-14T09:07:22.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868748+0000", 2024-09-14T09:07:22.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3865e-05 2024-09-14T09:07:22.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868758+0000", 2024-09-14T09:07:22.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2699999999999993e-06 2024-09-14T09:07:22.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868904+0000", 2024-09-14T09:07:22.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014626999999999999 2024-09-14T09:07:22.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868911+0000", 2024-09-14T09:07:22.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9280000000000004e-06 2024-09-14T09:07:22.715 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.715 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.715 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.715 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105259 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.716 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.840425+0000", 2024-09-14T09:07:22.716 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34940337199999999, 2024-09-14T09:07:22.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.356980193, 2024-09-14T09:07:22.716 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.716 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.716 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.716 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.716 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.716 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.716 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105259 2024-09-14T09:07:22.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.716 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840425+0000", 2024-09-14T09:07:22.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840425+0000", 2024-09-14T09:07:22.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840426+0000", 2024-09-14T09:07:22.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.96e-07 2024-09-14T09:07:22.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840427+0000", 2024-09-14T09:07:22.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8139999999999999e-06 2024-09-14T09:07:22.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840432+0000", 2024-09-14T09:07:22.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6859999999999997e-06 2024-09-14T09:07:22.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840438+0000", 2024-09-14T09:07:22.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2779999999999999e-06 2024-09-14T09:07:22.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840918+0000", 2024-09-14T09:07:22.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000480106 2024-09-14T09:07:22.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840926+0000", 2024-09-14T09:07:22.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4059999999999999e-06 2024-09-14T09:07:22.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857080+0000", 2024-09-14T09:07:22.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016154354999999999 2024-09-14T09:07:22.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857089+0000", 2024-09-14T09:07:22.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1339999999999993e-06 2024-09-14T09:07:22.720 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.720 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.720 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.720 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105323 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.720 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.842137+0000", 2024-09-14T09:07:22.720 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34769108399999998, 2024-09-14T09:07:22.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35534040900000002, 2024-09-14T09:07:22.720 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.720 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.720 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.721 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.721 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.721 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.721 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105323 2024-09-14T09:07:22.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.721 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842137+0000", 2024-09-14T09:07:22.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842137+0000", 2024-09-14T09:07:22.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842138+0000", 2024-09-14T09:07:22.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6599999999999994e-07 2024-09-14T09:07:22.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842158+0000", 2024-09-14T09:07:22.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.91e-05 2024-09-14T09:07:22.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842169+0000", 2024-09-14T09:07:22.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1076e-05 2024-09-14T09:07:22.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842178+0000", 2024-09-14T09:07:22.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4339999999999998e-06 2024-09-14T09:07:22.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842649+0000", 2024-09-14T09:07:22.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00047121400000000002 2024-09-14T09:07:22.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842654+0000", 2024-09-14T09:07:22.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-09-14T09:07:22.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859795+0000", 2024-09-14T09:07:22.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017141126999999999 2024-09-14T09:07:22.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859804+0000", 2024-09-14T09:07:22.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1770000000000004e-06 2024-09-14T09:07:22.724 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.724 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.724 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.725 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105387 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.725 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.846590+0000", 2024-09-14T09:07:22.725 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34323820100000002, 2024-09-14T09:07:22.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35096351399999998, 2024-09-14T09:07:22.725 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.725 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.725 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.725 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.725 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.725 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.726 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105387 2024-09-14T09:07:22.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.726 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846590+0000", 2024-09-14T09:07:22.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846590+0000", 2024-09-14T09:07:22.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846591+0000", 2024-09-14T09:07:22.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4799999999999997e-07 2024-09-14T09:07:22.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846607+0000", 2024-09-14T09:07:22.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5794000000000002e-05 2024-09-14T09:07:22.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846620+0000", 2024-09-14T09:07:22.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2636000000000001e-05 2024-09-14T09:07:22.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846629+0000", 2024-09-14T09:07:22.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4860000000000003e-06 2024-09-14T09:07:22.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846715+0000", 2024-09-14T09:07:22.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6117000000000002e-05 2024-09-14T09:07:22.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.729 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.729 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846724+0000", 2024-09-14T09:07:22.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9460000000000004e-06 2024-09-14T09:07:22.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.729 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.729 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860974+0000", 2024-09-14T09:07:22.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014249585 2024-09-14T09:07:22.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860978+0000", 2024-09-14T09:07:22.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2479999999999998e-06 2024-09-14T09:07:22.730 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.730 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.730 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.730 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105450 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.730 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.856928+0000", 2024-09-14T09:07:22.730 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.332900686, 2024-09-14T09:07:22.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34070325899999998, 2024-09-14T09:07:22.731 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.731 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.731 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.731 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.731 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.731 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.731 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105450 2024-09-14T09:07:22.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.731 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856928+0000", 2024-09-14T09:07:22.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856928+0000", 2024-09-14T09:07:22.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856928+0000", 2024-09-14T09:07:22.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9800000000000003e-07 2024-09-14T09:07:22.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856932+0000", 2024-09-14T09:07:22.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1640000000000001e-06 2024-09-14T09:07:22.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856941+0000", 2024-09-14T09:07:22.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:07:22.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856949+0000", 2024-09-14T09:07:22.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4259999999999997e-06 2024-09-14T09:07:22.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862460+0000", 2024-09-14T09:07:22.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055109720000000003 2024-09-14T09:07:22.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862469+0000", 2024-09-14T09:07:22.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0650000000000005e-06 2024-09-14T09:07:22.734 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.734 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.734 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.734 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105516 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.734 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860103+0000", 2024-09-14T09:07:22.734 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32972542399999999, 2024-09-14T09:07:22.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33758891400000002, 2024-09-14T09:07:22.735 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.735 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.735 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.735 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.735 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.735 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.735 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105516 2024-09-14T09:07:22.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.735 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860103+0000", 2024-09-14T09:07:22.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860103+0000", 2024-09-14T09:07:22.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860111+0000", 2024-09-14T09:07:22.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2260000000000005e-06 2024-09-14T09:07:22.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860122+0000", 2024-09-14T09:07:22.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0716999999999999e-05 2024-09-14T09:07:22.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860131+0000", 2024-09-14T09:07:22.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8959999999999992e-06 2024-09-14T09:07:22.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860137+0000", 2024-09-14T09:07:22.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0730000000000001e-06 2024-09-14T09:07:22.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864165+0000", 2024-09-14T09:07:22.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040280200000000002 2024-09-14T09:07:22.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864180+0000", 2024-09-14T09:07:22.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4674e-05 2024-09-14T09:07:22.738 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.738 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.738 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.738 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105579 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.738 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861509+0000", 2024-09-14T09:07:22.738 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32831950900000001, 2024-09-14T09:07:22.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33625170999999998, 2024-09-14T09:07:22.739 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.739 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.739 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.739 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.739 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.739 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.739 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105579 2024-09-14T09:07:22.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.739 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861509+0000", 2024-09-14T09:07:22.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861509+0000", 2024-09-14T09:07:22.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861510+0000", 2024-09-14T09:07:22.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4499999999999997e-07 2024-09-14T09:07:22.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861513+0000", 2024-09-14T09:07:22.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.056e-06 2024-09-14T09:07:22.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861522+0000", 2024-09-14T09:07:22.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1279999999999997e-06 2024-09-14T09:07:22.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861529+0000", 2024-09-14T09:07:22.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.759e-06 2024-09-14T09:07:22.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865428+0000", 2024-09-14T09:07:22.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038996569999999999 2024-09-14T09:07:22.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865435+0000", 2024-09-14T09:07:22.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.686e-06 2024-09-14T09:07:22.742 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.742 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.742 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.742 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105643 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.742 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:07:22.742 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32289119199999999, 2024-09-14T09:07:22.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33088453800000001, 2024-09-14T09:07:22.742 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.743 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.743 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.743 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.743 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.743 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.743 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105643 2024-09-14T09:07:22.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.743 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:07:22.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:07:22.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866938+0000", 2024-09-14T09:07:22.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5600000000000004e-07 2024-09-14T09:07:22.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866942+0000", 2024-09-14T09:07:22.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0350000000000003e-06 2024-09-14T09:07:22.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866955+0000", 2024-09-14T09:07:22.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3097e-05 2024-09-14T09:07:22.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866965+0000", 2024-09-14T09:07:22.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8179999999999999e-06 2024-09-14T09:07:22.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867067+0000", 2024-09-14T09:07:22.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010128699999999999 2024-09-14T09:07:22.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867080+0000", 2024-09-14T09:07:22.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3089e-05 2024-09-14T09:07:22.746 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.746 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.746 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.746 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105707 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.746 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868771+0000", 2024-09-14T09:07:22.746 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32105741999999998, 2024-09-14T09:07:22.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32911500399999999, 2024-09-14T09:07:22.747 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.747 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.747 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.747 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.747 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.747 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.747 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105707 2024-09-14T09:07:22.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.747 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868771+0000", 2024-09-14T09:07:22.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868771+0000", 2024-09-14T09:07:22.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868772+0000", 2024-09-14T09:07:22.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7000000000000003e-07 2024-09-14T09:07:22.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868775+0000", 2024-09-14T09:07:22.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4400000000000001e-06 2024-09-14T09:07:22.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868787+0000", 2024-09-14T09:07:22.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1525999999999999e-05 2024-09-14T09:07:22.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868794+0000", 2024-09-14T09:07:22.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4440000000000002e-06 2024-09-14T09:07:22.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868940+0000", 2024-09-14T09:07:22.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014580100000000001 2024-09-14T09:07:22.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868949+0000", 2024-09-14T09:07:22.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3610000000000001e-06 2024-09-14T09:07:22.750 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.750 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.750 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.750 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105261 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.751 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.840448+0000", 2024-09-14T09:07:22.751 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34938085099999999, 2024-09-14T09:07:22.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35750195899999998, 2024-09-14T09:07:22.751 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.751 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.751 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.751 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.751 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.751 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.751 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105261 2024-09-14T09:07:22.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.751 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840448+0000", 2024-09-14T09:07:22.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840448+0000", 2024-09-14T09:07:22.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840448+0000", 2024-09-14T09:07:22.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3200000000000005e-07 2024-09-14T09:07:22.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840452+0000", 2024-09-14T09:07:22.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3000000000000002e-06 2024-09-14T09:07:22.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840460+0000", 2024-09-14T09:07:22.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7409999999999998e-06 2024-09-14T09:07:22.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840469+0000", 2024-09-14T09:07:22.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8060000000000001e-06 2024-09-14T09:07:22.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841008+0000", 2024-09-14T09:07:22.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00053872100000000003 2024-09-14T09:07:22.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841021+0000", 2024-09-14T09:07:22.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2631e-05 2024-09-14T09:07:22.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857129+0000", 2024-09-14T09:07:22.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016108168999999999 2024-09-14T09:07:22.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857137+0000", 2024-09-14T09:07:22.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0190000000000006e-06 2024-09-14T09:07:22.756 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.756 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.756 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.756 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105325 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.756 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.842196+0000", 2024-09-14T09:07:22.756 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34763235599999998, 2024-09-14T09:07:22.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.355825206, 2024-09-14T09:07:22.756 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.757 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.757 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.757 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.757 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.757 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.757 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105325 2024-09-14T09:07:22.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.757 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842196+0000", 2024-09-14T09:07:22.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842196+0000", 2024-09-14T09:07:22.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842197+0000", 2024-09-14T09:07:22.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4600000000000005e-07 2024-09-14T09:07:22.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842199+0000", 2024-09-14T09:07:22.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1339999999999998e-06 2024-09-14T09:07:22.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842205+0000", 2024-09-14T09:07:22.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2060000000000004e-06 2024-09-14T09:07:22.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842210+0000", 2024-09-14T09:07:22.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8670000000000002e-06 2024-09-14T09:07:22.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842679+0000", 2024-09-14T09:07:22.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00046904200000000003 2024-09-14T09:07:22.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842683+0000", 2024-09-14T09:07:22.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4689999999999999e-06 2024-09-14T09:07:22.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859884+0000", 2024-09-14T09:07:22.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017200666999999999 2024-09-14T09:07:22.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859893+0000", 2024-09-14T09:07:22.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9129999999999992e-06 2024-09-14T09:07:22.761 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.761 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.761 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.761 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105389 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.761 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.846741+0000", 2024-09-14T09:07:22.761 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34308723800000002, 2024-09-14T09:07:22.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35135464700000002, 2024-09-14T09:07:22.761 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.762 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.762 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.762 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.762 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.762 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.762 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105389 2024-09-14T09:07:22.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.762 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846741+0000", 2024-09-14T09:07:22.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846741+0000", 2024-09-14T09:07:22.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846742+0000", 2024-09-14T09:07:22.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1340000000000001e-06 2024-09-14T09:07:22.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846748+0000", 2024-09-14T09:07:22.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4160000000000003e-06 2024-09-14T09:07:22.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846760+0000", 2024-09-14T09:07:22.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.216e-05 2024-09-14T09:07:22.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846770+0000", 2024-09-14T09:07:22.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9629999999999993e-06 2024-09-14T09:07:22.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846850+0000", 2024-09-14T09:07:22.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0290000000000005e-05 2024-09-14T09:07:22.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846859+0000", 2024-09-14T09:07:22.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1020000000000003e-06 2024-09-14T09:07:22.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860995+0000", 2024-09-14T09:07:22.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014135586 2024-09-14T09:07:22.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860999+0000", 2024-09-14T09:07:22.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-09-14T09:07:22.766 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.766 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.766 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.766 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105454 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.766 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.856963+0000", 2024-09-14T09:07:22.766 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33286586800000001, 2024-09-14T09:07:22.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34120768899999998, 2024-09-14T09:07:22.766 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.766 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.766 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.767 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.767 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.767 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.767 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105454 2024-09-14T09:07:22.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.767 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856963+0000", 2024-09-14T09:07:22.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856963+0000", 2024-09-14T09:07:22.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856963+0000", 2024-09-14T09:07:22.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6599999999999996e-07 2024-09-14T09:07:22.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856966+0000", 2024-09-14T09:07:22.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0740000000000001e-06 2024-09-14T09:07:22.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856975+0000", 2024-09-14T09:07:22.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4980000000000006e-06 2024-09-14T09:07:22.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856982+0000", 2024-09-14T09:07:22.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6649999999999997e-06 2024-09-14T09:07:22.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862515+0000", 2024-09-14T09:07:22.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055338330000000002 2024-09-14T09:07:22.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862524+0000", 2024-09-14T09:07:22.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0420000000000008e-06 2024-09-14T09:07:22.770 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.770 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.770 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.770 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105517 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.770 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860158+0000", 2024-09-14T09:07:22.770 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.329670664, 2024-09-14T09:07:22.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33807300299999998, 2024-09-14T09:07:22.770 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.770 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.770 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.771 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.771 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.771 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.771 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105517 2024-09-14T09:07:22.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.771 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860158+0000", 2024-09-14T09:07:22.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860158+0000", 2024-09-14T09:07:22.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860158+0000", 2024-09-14T09:07:22.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5000000000000003e-07 2024-09-14T09:07:22.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860161+0000", 2024-09-14T09:07:22.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3309999999999998e-06 2024-09-14T09:07:22.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860168+0000", 2024-09-14T09:07:22.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1380000000000001e-06 2024-09-14T09:07:22.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860174+0000", 2024-09-14T09:07:22.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6779999999999997e-06 2024-09-14T09:07:22.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864238+0000", 2024-09-14T09:07:22.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004063952 2024-09-14T09:07:22.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864252+0000", 2024-09-14T09:07:22.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4052e-05 2024-09-14T09:07:22.774 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.774 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.774 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.774 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105581 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.774 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861554+0000", 2024-09-14T09:07:22.774 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32827420800000001, 2024-09-14T09:07:22.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33674024400000002, 2024-09-14T09:07:22.775 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.775 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.775 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.775 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.775 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.775 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.775 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105581 2024-09-14T09:07:22.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.775 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861554+0000", 2024-09-14T09:07:22.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861554+0000", 2024-09-14T09:07:22.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861555+0000", 2024-09-14T09:07:22.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7700000000000004e-07 2024-09-14T09:07:22.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861558+0000", 2024-09-14T09:07:22.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9229999999999998e-06 2024-09-14T09:07:22.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861567+0000", 2024-09-14T09:07:22.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7199999999999995e-06 2024-09-14T09:07:22.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861574+0000", 2024-09-14T09:07:22.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0040000000000002e-06 2024-09-14T09:07:22.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865461+0000", 2024-09-14T09:07:22.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038872490000000002 2024-09-14T09:07:22.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865467+0000", 2024-09-14T09:07:22.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-09-14T09:07:22.778 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.778 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.778 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.778 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105645 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.778 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:07:22.778 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.322830797, 2024-09-14T09:07:22.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33135728199999998, 2024-09-14T09:07:22.779 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.779 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.779 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.779 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.779 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.779 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.779 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105645 2024-09-14T09:07:22.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.779 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:07:22.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:07:22.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866999+0000", 2024-09-14T09:07:22.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7999999999999993e-07 2024-09-14T09:07:22.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867003+0000", 2024-09-14T09:07:22.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0090000000000001e-06 2024-09-14T09:07:22.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867016+0000", 2024-09-14T09:07:22.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3304e-05 2024-09-14T09:07:22.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867026+0000", 2024-09-14T09:07:22.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5889999999999992e-06 2024-09-14T09:07:22.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867128+0000", 2024-09-14T09:07:22.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000102101 2024-09-14T09:07:22.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867141+0000", 2024-09-14T09:07:22.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2903e-05 2024-09-14T09:07:22.782 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.782 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.782 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.782 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105709 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.782 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868808+0000", 2024-09-14T09:07:22.782 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32102043000000002, 2024-09-14T09:07:22.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.32961052899999999, 2024-09-14T09:07:22.782 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.782 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.783 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.783 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.783 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.783 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.783 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105709 2024-09-14T09:07:22.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.783 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868808+0000", 2024-09-14T09:07:22.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868808+0000", 2024-09-14T09:07:22.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868809+0000", 2024-09-14T09:07:22.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6000000000000003e-07 2024-09-14T09:07:22.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868812+0000", 2024-09-14T09:07:22.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9440000000000001e-06 2024-09-14T09:07:22.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868822+0000", 2024-09-14T09:07:22.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0451999999999999e-05 2024-09-14T09:07:22.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868831+0000", 2024-09-14T09:07:22.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6619999999999994e-06 2024-09-14T09:07:22.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869008+0000", 2024-09-14T09:07:22.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000176806 2024-09-14T09:07:22.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.786 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.786 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869022+0000", 2024-09-14T09:07:22.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4086e-05 2024-09-14T09:07:22.786 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.786 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.786 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.786 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105263 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.786 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.840491+0000", 2024-09-14T09:07:22.786 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34933726399999998, 2024-09-14T09:07:22.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35798843000000002, 2024-09-14T09:07:22.787 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.787 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.787 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.787 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.787 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.787 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.787 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105263 2024-09-14T09:07:22.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.787 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840491+0000", 2024-09-14T09:07:22.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840491+0000", 2024-09-14T09:07:22.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840492+0000", 2024-09-14T09:07:22.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0299999999999998e-07 2024-09-14T09:07:22.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840495+0000", 2024-09-14T09:07:22.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3859999999999999e-06 2024-09-14T09:07:22.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840505+0000", 2024-09-14T09:07:22.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0047000000000001e-05 2024-09-14T09:07:22.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840513+0000", 2024-09-14T09:07:22.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6869999999999994e-06 2024-09-14T09:07:22.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841086+0000", 2024-09-14T09:07:22.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00057279 2024-09-14T09:07:22.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841093+0000", 2024-09-14T09:07:22.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7089999999999996e-06 2024-09-14T09:07:22.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857165+0000", 2024-09-14T09:07:22.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016072347000000001 2024-09-14T09:07:22.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857172+0000", 2024-09-14T09:07:22.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0690000000000004e-06 2024-09-14T09:07:22.791 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.791 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.791 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.791 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105327 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.791 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.842216+0000", 2024-09-14T09:07:22.792 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34761250100000002, 2024-09-14T09:07:22.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.356337968, 2024-09-14T09:07:22.792 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.792 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.792 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.792 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.792 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.792 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.792 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105327 2024-09-14T09:07:22.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.792 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842216+0000", 2024-09-14T09:07:22.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842216+0000", 2024-09-14T09:07:22.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842217+0000", 2024-09-14T09:07:22.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-09-14T09:07:22.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842219+0000", 2024-09-14T09:07:22.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.79e-06 2024-09-14T09:07:22.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842230+0000", 2024-09-14T09:07:22.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0608000000000001e-05 2024-09-14T09:07:22.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842236+0000", 2024-09-14T09:07:22.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.592e-06 2024-09-14T09:07:22.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842709+0000", 2024-09-14T09:07:22.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00047352499999999998 2024-09-14T09:07:22.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842714+0000", 2024-09-14T09:07:22.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5839999999999996e-06 2024-09-14T09:07:22.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859926+0000", 2024-09-14T09:07:22.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017212852000000001 2024-09-14T09:07:22.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859935+0000", 2024-09-14T09:07:22.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8149999999999995e-06 2024-09-14T09:07:22.796 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.796 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.796 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.796 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105391 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.796 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.846880+0000", 2024-09-14T09:07:22.796 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34294829100000002, 2024-09-14T09:07:22.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.351748118, 2024-09-14T09:07:22.796 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.797 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.797 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.797 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.797 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.797 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.797 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105391 2024-09-14T09:07:22.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.797 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846880+0000", 2024-09-14T09:07:22.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846880+0000", 2024-09-14T09:07:22.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846881+0000", 2024-09-14T09:07:22.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0720000000000001e-06 2024-09-14T09:07:22.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846886+0000", 2024-09-14T09:07:22.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9339999999999997e-06 2024-09-14T09:07:22.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846937+0000", 2024-09-14T09:07:22.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0275000000000002e-05 2024-09-14T09:07:22.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846948+0000", 2024-09-14T09:07:22.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1136e-05 2024-09-14T09:07:22.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847039+0000", 2024-09-14T09:07:22.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1514999999999994e-05 2024-09-14T09:07:22.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847049+0000", 2024-09-14T09:07:22.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5759999999999995e-06 2024-09-14T09:07:22.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861016+0000", 2024-09-14T09:07:22.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013967297 2024-09-14T09:07:22.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861020+0000", 2024-09-14T09:07:22.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2080000000000002e-06 2024-09-14T09:07:22.801 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.801 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.801 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.801 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105455 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.801 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857004+0000", 2024-09-14T09:07:22.801 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33282433099999997, 2024-09-14T09:07:22.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34170436399999998, 2024-09-14T09:07:22.801 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.801 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.801 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.801 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.801 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.802 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.802 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105455 2024-09-14T09:07:22.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.802 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857004+0000", 2024-09-14T09:07:22.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857004+0000", 2024-09-14T09:07:22.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857005+0000", 2024-09-14T09:07:22.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9999999999999997e-07 2024-09-14T09:07:22.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857008+0000", 2024-09-14T09:07:22.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2279999999999998e-06 2024-09-14T09:07:22.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857018+0000", 2024-09-14T09:07:22.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4490000000000005e-06 2024-09-14T09:07:22.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857025+0000", 2024-09-14T09:07:22.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0870000000000001e-06 2024-09-14T09:07:22.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862588+0000", 2024-09-14T09:07:22.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055633790000000002 2024-09-14T09:07:22.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862593+0000", 2024-09-14T09:07:22.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7999999999999998e-06 2024-09-14T09:07:22.805 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.805 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.805 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.805 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105519 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.805 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860202+0000", 2024-09-14T09:07:22.806 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32962684799999997, 2024-09-14T09:07:22.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33856787999999999, 2024-09-14T09:07:22.806 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.806 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.806 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.806 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.806 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.806 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.806 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105519 2024-09-14T09:07:22.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.807 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860202+0000", 2024-09-14T09:07:22.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860202+0000", 2024-09-14T09:07:22.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860202+0000", 2024-09-14T09:07:22.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3599999999999999e-07 2024-09-14T09:07:22.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860205+0000", 2024-09-14T09:07:22.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.638e-06 2024-09-14T09:07:22.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860212+0000", 2024-09-14T09:07:22.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0430000000000002e-06 2024-09-14T09:07:22.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860217+0000", 2024-09-14T09:07:22.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2390000000000002e-06 2024-09-14T09:07:22.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864307+0000", 2024-09-14T09:07:22.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040896789999999997 2024-09-14T09:07:22.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864320+0000", 2024-09-14T09:07:22.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3662000000000001e-05 2024-09-14T09:07:22.811 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.811 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.811 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.811 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105584 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.811 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861602+0000", 2024-09-14T09:07:22.811 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32822677300000003, 2024-09-14T09:07:22.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.337231791, 2024-09-14T09:07:22.812 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.812 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.812 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.812 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.812 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.812 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.812 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105584 2024-09-14T09:07:22.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.813 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861602+0000", 2024-09-14T09:07:22.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861602+0000", 2024-09-14T09:07:22.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861602+0000", 2024-09-14T09:07:22.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9299999999999997e-07 2024-09-14T09:07:22.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861605+0000", 2024-09-14T09:07:22.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9510000000000002e-06 2024-09-14T09:07:22.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861614+0000", 2024-09-14T09:07:22.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9199999999999993e-06 2024-09-14T09:07:22.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861621+0000", 2024-09-14T09:07:22.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8460000000000002e-06 2024-09-14T09:07:22.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865493+0000", 2024-09-14T09:07:22.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003872183 2024-09-14T09:07:22.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865500+0000", 2024-09-14T09:07:22.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.545e-06 2024-09-14T09:07:22.817 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.817 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.817 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.817 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105647 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.817 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:07:22.817 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.322772279, 2024-09-14T09:07:22.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33183811699999999, 2024-09-14T09:07:22.817 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.817 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.817 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.817 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.818 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.818 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.818 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105647 2024-09-14T09:07:22.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.818 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:07:22.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:07:22.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867057+0000", 2024-09-14T09:07:22.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.95e-07 2024-09-14T09:07:22.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867061+0000", 2024-09-14T09:07:22.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9530000000000001e-06 2024-09-14T09:07:22.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867074+0000", 2024-09-14T09:07:22.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3061000000000001e-05 2024-09-14T09:07:22.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867086+0000", 2024-09-14T09:07:22.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1460999999999999e-05 2024-09-14T09:07:22.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867186+0000", 2024-09-14T09:07:22.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100181 2024-09-14T09:07:22.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867199+0000", 2024-09-14T09:07:22.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-05 2024-09-14T09:07:22.821 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.821 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.821 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.822 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105711 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.822 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868845+0000", 2024-09-14T09:07:22.822 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32098359999999998, 2024-09-14T09:07:22.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33011317000000001, 2024-09-14T09:07:22.822 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.822 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.822 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.822 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.822 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.822 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.822 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105711 2024-09-14T09:07:22.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.823 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868845+0000", 2024-09-14T09:07:22.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868845+0000", 2024-09-14T09:07:22.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868846+0000", 2024-09-14T09:07:22.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0400000000000005e-07 2024-09-14T09:07:22.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868849+0000", 2024-09-14T09:07:22.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4680000000000001e-06 2024-09-14T09:07:22.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868861+0000", 2024-09-14T09:07:22.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1534e-05 2024-09-14T09:07:22.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868869+0000", 2024-09-14T09:07:22.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4409999999999993e-06 2024-09-14T09:07:22.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869077+0000", 2024-09-14T09:07:22.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000207977 2024-09-14T09:07:22.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869084+0000", 2024-09-14T09:07:22.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0929999999999997e-06 2024-09-14T09:07:22.826 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.827 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.827 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.827 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105265 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.827 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.840524+0000", 2024-09-14T09:07:22.827 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34930492899999999, 2024-09-14T09:07:22.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35849534799999999, 2024-09-14T09:07:22.828 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.828 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.828 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.828 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.828 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.828 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.828 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105265 2024-09-14T09:07:22.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.829 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840524+0000", 2024-09-14T09:07:22.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840524+0000", 2024-09-14T09:07:22.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840524+0000", 2024-09-14T09:07:22.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7999999999999995e-07 2024-09-14T09:07:22.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840527+0000", 2024-09-14T09:07:22.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7779999999999999e-06 2024-09-14T09:07:22.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840535+0000", 2024-09-14T09:07:22.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0870000000000007e-06 2024-09-14T09:07:22.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840542+0000", 2024-09-14T09:07:22.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1620000000000002e-06 2024-09-14T09:07:22.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841152+0000", 2024-09-14T09:07:22.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060929699999999996 2024-09-14T09:07:22.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841158+0000", 2024-09-14T09:07:22.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7320000000000001e-06 2024-09-14T09:07:22.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857199+0000", 2024-09-14T09:07:22.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.016040769999999999 2024-09-14T09:07:22.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857207+0000", 2024-09-14T09:07:22.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6199999999999999e-06 2024-09-14T09:07:22.833 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.833 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.834 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.834 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105330 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.834 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.842266+0000", 2024-09-14T09:07:22.834 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34756281500000002, 2024-09-14T09:07:22.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35682931800000001, 2024-09-14T09:07:22.834 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.834 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.834 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.835 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.835 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.835 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.835 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105330 2024-09-14T09:07:22.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.835 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842266+0000", 2024-09-14T09:07:22.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842266+0000", 2024-09-14T09:07:22.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842266+0000", 2024-09-14T09:07:22.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1999999999999999e-07 2024-09-14T09:07:22.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842270+0000", 2024-09-14T09:07:22.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4340000000000001e-06 2024-09-14T09:07:22.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842284+0000", 2024-09-14T09:07:22.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4106e-05 2024-09-14T09:07:22.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842292+0000", 2024-09-14T09:07:22.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4479999999999994e-06 2024-09-14T09:07:22.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842739+0000", 2024-09-14T09:07:22.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00044664599999999998 2024-09-14T09:07:22.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842743+0000", 2024-09-14T09:07:22.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5170000000000001e-06 2024-09-14T09:07:22.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859971+0000", 2024-09-14T09:07:22.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017227095000000001 2024-09-14T09:07:22.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.859979+0000", 2024-09-14T09:07:22.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3019999999999995e-06 2024-09-14T09:07:22.839 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.839 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.839 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.839 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105394 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.839 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.846987+0000", 2024-09-14T09:07:22.839 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34284185, 2024-09-14T09:07:22.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35218238600000001, 2024-09-14T09:07:22.839 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.840 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.840 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.840 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.840 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.840 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.840 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105394 2024-09-14T09:07:22.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.840 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846987+0000", 2024-09-14T09:07:22.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846987+0000", 2024-09-14T09:07:22.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846988+0000", 2024-09-14T09:07:22.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5000000000000001e-07 2024-09-14T09:07:22.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.846992+0000", 2024-09-14T09:07:22.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6840000000000004e-06 2024-09-14T09:07:22.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847012+0000", 2024-09-14T09:07:22.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9672000000000001e-05 2024-09-14T09:07:22.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847032+0000", 2024-09-14T09:07:22.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9507000000000001e-05 2024-09-14T09:07:22.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847103+0000", 2024-09-14T09:07:22.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1273999999999996e-05 2024-09-14T09:07:22.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847111+0000", 2024-09-14T09:07:22.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2619999999999998e-06 2024-09-14T09:07:22.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861037+0000", 2024-09-14T09:07:22.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013925521999999999 2024-09-14T09:07:22.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861044+0000", 2024-09-14T09:07:22.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4229999999999999e-06 2024-09-14T09:07:22.844 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.844 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.844 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.844 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105457 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.844 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857051+0000", 2024-09-14T09:07:22.845 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33277729299999997, 2024-09-14T09:07:22.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34219214999999997, 2024-09-14T09:07:22.845 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.845 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.845 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.845 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.845 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.845 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.845 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105457 2024-09-14T09:07:22.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.845 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857051+0000", 2024-09-14T09:07:22.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857051+0000", 2024-09-14T09:07:22.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857052+0000", 2024-09-14T09:07:22.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.37e-07 2024-09-14T09:07:22.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857055+0000", 2024-09-14T09:07:22.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2940000000000001e-06 2024-09-14T09:07:22.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857065+0000", 2024-09-14T09:07:22.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0083e-05 2024-09-14T09:07:22.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857072+0000", 2024-09-14T09:07:22.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.109e-06 2024-09-14T09:07:22.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862612+0000", 2024-09-14T09:07:22.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055395100000000001 2024-09-14T09:07:22.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862616+0000", 2024-09-14T09:07:22.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3819999999999997e-06 2024-09-14T09:07:22.848 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.848 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.849 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.849 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105521 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.849 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860247+0000", 2024-09-14T09:07:22.849 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32958148799999998, 2024-09-14T09:07:22.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33905795599999999, 2024-09-14T09:07:22.849 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.849 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.849 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.849 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.850 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.850 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.850 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105521 2024-09-14T09:07:22.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.850 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860247+0000", 2024-09-14T09:07:22.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860247+0000", 2024-09-14T09:07:22.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860247+0000", 2024-09-14T09:07:22.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2500000000000001e-07 2024-09-14T09:07:22.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860250+0000", 2024-09-14T09:07:22.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.897e-06 2024-09-14T09:07:22.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860258+0000", 2024-09-14T09:07:22.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3810000000000001e-06 2024-09-14T09:07:22.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860263+0000", 2024-09-14T09:07:22.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.254e-06 2024-09-14T09:07:22.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864379+0000", 2024-09-14T09:07:22.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041164649999999997 2024-09-14T09:07:22.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864394+0000", 2024-09-14T09:07:22.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4062e-05 2024-09-14T09:07:22.856 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.856 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.856 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.856 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105585 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.856 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861649+0000", 2024-09-14T09:07:22.857 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32817990899999999, 2024-09-14T09:07:22.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33772064099999999, 2024-09-14T09:07:22.857 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.857 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.857 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.857 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.857 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.857 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.857 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105585 2024-09-14T09:07:22.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.857 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861649+0000", 2024-09-14T09:07:22.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861649+0000", 2024-09-14T09:07:22.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861649+0000", 2024-09-14T09:07:22.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6300000000000005e-07 2024-09-14T09:07:22.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861658+0000", 2024-09-14T09:07:22.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3979999999999999e-06 2024-09-14T09:07:22.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861667+0000", 2024-09-14T09:07:22.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2660000000000007e-06 2024-09-14T09:07:22.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861674+0000", 2024-09-14T09:07:22.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8009999999999998e-06 2024-09-14T09:07:22.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865525+0000", 2024-09-14T09:07:22.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038517059999999999 2024-09-14T09:07:22.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865532+0000", 2024-09-14T09:07:22.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4819999999999999e-06 2024-09-14T09:07:22.860 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.860 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.861 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.861 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105648 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.861 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:07:22.861 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.322710793, 2024-09-14T09:07:22.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.332312206, 2024-09-14T09:07:22.861 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.861 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.861 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.861 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.861 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.861 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.862 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105648 2024-09-14T09:07:22.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.862 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:07:22.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:07:22.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867119+0000", 2024-09-14T09:07:22.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8400000000000002e-07 2024-09-14T09:07:22.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867123+0000", 2024-09-14T09:07:22.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9639999999999997e-06 2024-09-14T09:07:22.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867136+0000", 2024-09-14T09:07:22.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.289e-05 2024-09-14T09:07:22.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867145+0000", 2024-09-14T09:07:22.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8919999999999995e-06 2024-09-14T09:07:22.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867245+0000", 2024-09-14T09:07:22.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9282000000000003e-05 2024-09-14T09:07:22.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867258+0000", 2024-09-14T09:07:22.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2935000000000001e-05 2024-09-14T09:07:22.865 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.865 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.865 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.866 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105713 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.866 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868899+0000", 2024-09-14T09:07:22.866 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32092989500000002, 2024-09-14T09:07:22.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33059540599999998, 2024-09-14T09:07:22.866 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.866 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.866 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.866 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.866 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.866 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.866 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105713 2024-09-14T09:07:22.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.867 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868899+0000", 2024-09-14T09:07:22.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868899+0000", 2024-09-14T09:07:22.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868900+0000", 2024-09-14T09:07:22.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8400000000000003e-07 2024-09-14T09:07:22.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868904+0000", 2024-09-14T09:07:22.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9779999999999999e-06 2024-09-14T09:07:22.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868917+0000", 2024-09-14T09:07:22.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3162e-05 2024-09-14T09:07:22.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868925+0000", 2024-09-14T09:07:22.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7120000000000006e-06 2024-09-14T09:07:22.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869136+0000", 2024-09-14T09:07:22.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021026199999999999 2024-09-14T09:07:22.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869148+0000", 2024-09-14T09:07:22.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2444e-05 2024-09-14T09:07:22.870 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.870 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.870 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.870 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105267 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.870 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.840642+0000", 2024-09-14T09:07:22.870 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34918693200000001, 2024-09-14T09:07:22.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35891321999999998, 2024-09-14T09:07:22.870 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.870 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.870 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.870 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.871 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.871 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.871 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105267 2024-09-14T09:07:22.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.871 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840642+0000", 2024-09-14T09:07:22.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840642+0000", 2024-09-14T09:07:22.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840644+0000", 2024-09-14T09:07:22.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.007e-06 2024-09-14T09:07:22.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840655+0000", 2024-09-14T09:07:22.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1039000000000001e-05 2024-09-14T09:07:22.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840690+0000", 2024-09-14T09:07:22.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5009000000000003e-05 2024-09-14T09:07:22.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840707+0000", 2024-09-14T09:07:22.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7603000000000001e-05 2024-09-14T09:07:22.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841233+0000", 2024-09-14T09:07:22.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00052552700000000003 2024-09-14T09:07:22.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841241+0000", 2024-09-14T09:07:22.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9030000000000004e-06 2024-09-14T09:07:22.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857234+0000", 2024-09-14T09:07:22.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015993212999999999 2024-09-14T09:07:22.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857241+0000", 2024-09-14T09:07:22.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7920000000000004e-06 2024-09-14T09:07:22.875 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.875 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.875 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.876 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105331 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.876 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.842319+0000", 2024-09-14T09:07:22.876 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34750952899999998, 2024-09-14T09:07:22.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35730988400000002, 2024-09-14T09:07:22.876 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.876 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.876 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.876 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.877 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.877 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.877 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105331 2024-09-14T09:07:22.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.877 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842319+0000", 2024-09-14T09:07:22.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842319+0000", 2024-09-14T09:07:22.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842320+0000", 2024-09-14T09:07:22.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.54e-07 2024-09-14T09:07:22.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842323+0000", 2024-09-14T09:07:22.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.546e-06 2024-09-14T09:07:22.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842334+0000", 2024-09-14T09:07:22.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0472000000000001e-05 2024-09-14T09:07:22.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842344+0000", 2024-09-14T09:07:22.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0145e-05 2024-09-14T09:07:22.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842769+0000", 2024-09-14T09:07:22.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00042476900000000003 2024-09-14T09:07:22.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842773+0000", 2024-09-14T09:07:22.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5000000000000001e-06 2024-09-14T09:07:22.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860017+0000", 2024-09-14T09:07:22.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017243743999999998 2024-09-14T09:07:22.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860027+0000", 2024-09-14T09:07:22.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8770000000000005e-06 2024-09-14T09:07:22.880 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.881 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.881 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.881 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105395 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.881 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.847064+0000", 2024-09-14T09:07:22.881 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34276463099999999, 2024-09-14T09:07:22.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.352639543, 2024-09-14T09:07:22.881 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.881 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.881 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.881 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.881 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.882 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.882 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105395 2024-09-14T09:07:22.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.882 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847064+0000", 2024-09-14T09:07:22.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847064+0000", 2024-09-14T09:07:22.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847065+0000", 2024-09-14T09:07:22.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4e-07 2024-09-14T09:07:22.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847082+0000", 2024-09-14T09:07:22.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6772000000000001e-05 2024-09-14T09:07:22.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847103+0000", 2024-09-14T09:07:22.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1766000000000001e-05 2024-09-14T09:07:22.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847112+0000", 2024-09-14T09:07:22.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8079999999999994e-06 2024-09-14T09:07:22.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847186+0000", 2024-09-14T09:07:22.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3338999999999997e-05 2024-09-14T09:07:22.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847194+0000", 2024-09-14T09:07:22.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8219999999999996e-06 2024-09-14T09:07:22.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861077+0000", 2024-09-14T09:07:22.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013882960999999999 2024-09-14T09:07:22.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861086+0000", 2024-09-14T09:07:22.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9379999999999998e-06 2024-09-14T09:07:22.886 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.886 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.886 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.887 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105459 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.887 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857083+0000", 2024-09-14T09:07:22.887 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33274522000000001, 2024-09-14T09:07:22.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.342709443, 2024-09-14T09:07:22.887 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.887 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.887 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.887 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.887 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.888 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.888 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105459 2024-09-14T09:07:22.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.888 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857083+0000", 2024-09-14T09:07:22.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857083+0000", 2024-09-14T09:07:22.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857084+0000", 2024-09-14T09:07:22.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-09-14T09:07:22.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857087+0000", 2024-09-14T09:07:22.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9069999999999999e-06 2024-09-14T09:07:22.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857315+0000", 2024-09-14T09:07:22.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000228178 2024-09-14T09:07:22.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857327+0000", 2024-09-14T09:07:22.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2478e-05 2024-09-14T09:07:22.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862634+0000", 2024-09-14T09:07:22.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053062480000000004 2024-09-14T09:07:22.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862638+0000", 2024-09-14T09:07:22.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1810000000000003e-06 2024-09-14T09:07:22.891 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.891 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.891 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.891 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105524 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.891 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860293+0000", 2024-09-14T09:07:22.891 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32953549900000001, 2024-09-14T09:07:22.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33956472799999998, 2024-09-14T09:07:22.892 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.892 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.892 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.892 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.892 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.892 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.892 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105524 2024-09-14T09:07:22.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.892 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860293+0000", 2024-09-14T09:07:22.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860293+0000", 2024-09-14T09:07:22.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860294+0000", 2024-09-14T09:07:22.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2e-07 2024-09-14T09:07:22.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860296+0000", 2024-09-14T09:07:22.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1280000000000002e-06 2024-09-14T09:07:22.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860303+0000", 2024-09-14T09:07:22.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2840000000000001e-06 2024-09-14T09:07:22.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860308+0000", 2024-09-14T09:07:22.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1030000000000001e-06 2024-09-14T09:07:22.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864456+0000", 2024-09-14T09:07:22.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041479589999999997 2024-09-14T09:07:22.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864470+0000", 2024-09-14T09:07:22.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3869999999999999e-05 2024-09-14T09:07:22.895 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.896 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.896 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.896 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105587 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.896 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861696+0000", 2024-09-14T09:07:22.896 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32813286200000003, 2024-09-14T09:07:22.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33822623800000001, 2024-09-14T09:07:22.896 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.896 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.896 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.896 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.896 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.896 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.897 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105587 2024-09-14T09:07:22.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.897 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861696+0000", 2024-09-14T09:07:22.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861696+0000", 2024-09-14T09:07:22.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861696+0000", 2024-09-14T09:07:22.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-07 2024-09-14T09:07:22.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861699+0000", 2024-09-14T09:07:22.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9450000000000002e-06 2024-09-14T09:07:22.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861708+0000", 2024-09-14T09:07:22.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0759999999999992e-06 2024-09-14T09:07:22.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861717+0000", 2024-09-14T09:07:22.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8470000000000002e-06 2024-09-14T09:07:22.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865558+0000", 2024-09-14T09:07:22.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038407049999999998 2024-09-14T09:07:22.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865564+0000", 2024-09-14T09:07:22.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4400000000000002e-06 2024-09-14T09:07:22.899 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.899 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.899 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.900 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105652 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.900 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:07:22.900 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32265277399999998, 2024-09-14T09:07:22.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.332807511, 2024-09-14T09:07:22.900 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.900 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.900 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.900 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.900 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.900 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.900 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105652 2024-09-14T09:07:22.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.901 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:07:22.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:07:22.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867177+0000", 2024-09-14T09:07:22.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9100000000000002e-07 2024-09-14T09:07:22.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867181+0000", 2024-09-14T09:07:22.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0029999999999996e-06 2024-09-14T09:07:22.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867193+0000", 2024-09-14T09:07:22.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2675e-05 2024-09-14T09:07:22.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867203+0000", 2024-09-14T09:07:22.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5599999999999999e-06 2024-09-14T09:07:22.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867306+0000", 2024-09-14T09:07:22.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010330100000000001 2024-09-14T09:07:22.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867319+0000", 2024-09-14T09:07:22.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3129000000000001e-05 2024-09-14T09:07:22.904 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.904 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.904 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.904 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105715 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.904 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868938+0000", 2024-09-14T09:07:22.905 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32089087300000002, 2024-09-14T09:07:22.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33110983199999999, 2024-09-14T09:07:22.905 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.905 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.905 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.905 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.906 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.906 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.906 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105715 2024-09-14T09:07:22.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.906 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868938+0000", 2024-09-14T09:07:22.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868938+0000", 2024-09-14T09:07:22.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868938+0000", 2024-09-14T09:07:22.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:07:22.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868941+0000", 2024-09-14T09:07:22.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.492e-06 2024-09-14T09:07:22.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868950+0000", 2024-09-14T09:07:22.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8219999999999996e-06 2024-09-14T09:07:22.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868956+0000", 2024-09-14T09:07:22.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3779999999999998e-06 2024-09-14T09:07:22.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869201+0000", 2024-09-14T09:07:22.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024469800000000001 2024-09-14T09:07:22.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869208+0000", 2024-09-14T09:07:22.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7109999999999995e-06 2024-09-14T09:07:22.909 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.909 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.909 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.909 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105270 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.909 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.840737+0000", 2024-09-14T09:07:22.910 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.349091386, 2024-09-14T09:07:22.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35938429900000002, 2024-09-14T09:07:22.910 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.910 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.910 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.910 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.910 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.910 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.910 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105270 2024-09-14T09:07:22.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.910 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840737+0000", 2024-09-14T09:07:22.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840737+0000", 2024-09-14T09:07:22.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840738+0000", 2024-09-14T09:07:22.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3289999999999999e-06 2024-09-14T09:07:22.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840744+0000", 2024-09-14T09:07:22.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7620000000000001e-06 2024-09-14T09:07:22.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840761+0000", 2024-09-14T09:07:22.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6277e-05 2024-09-14T09:07:22.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840774+0000", 2024-09-14T09:07:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3175e-05 2024-09-14T09:07:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841332+0000", 2024-09-14T09:07:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055835399999999995 2024-09-14T09:07:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841340+0000", 2024-09-14T09:07:22.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7209999999999994e-06 2024-09-14T09:07:22.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857266+0000", 2024-09-14T09:07:22.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015926624 2024-09-14T09:07:22.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857273+0000", 2024-09-14T09:07:22.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7360000000000004e-06 2024-09-14T09:07:22.914 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.915 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.915 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.915 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105332 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.915 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.842356+0000", 2024-09-14T09:07:22.915 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34747254999999999, 2024-09-14T09:07:22.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35784187899999997, 2024-09-14T09:07:22.915 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.915 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.915 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.916 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.916 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.916 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.916 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105332 2024-09-14T09:07:22.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.916 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842356+0000", 2024-09-14T09:07:22.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842356+0000", 2024-09-14T09:07:22.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842357+0000", 2024-09-14T09:07:22.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7000000000000004e-07 2024-09-14T09:07:22.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842360+0000", 2024-09-14T09:07:22.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1240000000000001e-06 2024-09-14T09:07:22.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842371+0000", 2024-09-14T09:07:22.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.098e-05 2024-09-14T09:07:22.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842379+0000", 2024-09-14T09:07:22.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0639999999999994e-06 2024-09-14T09:07:22.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842798+0000", 2024-09-14T09:07:22.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00041956600000000001 2024-09-14T09:07:22.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842803+0000", 2024-09-14T09:07:22.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5519999999999998e-06 2024-09-14T09:07:22.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860064+0000", 2024-09-14T09:07:22.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.01726072 2024-09-14T09:07:22.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860073+0000", 2024-09-14T09:07:22.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1020000000000003e-06 2024-09-14T09:07:22.920 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.920 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.920 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.921 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105396 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.921 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.847139+0000", 2024-09-14T09:07:22.921 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34268936500000002, 2024-09-14T09:07:22.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35313361100000001, 2024-09-14T09:07:22.921 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.921 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.921 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.921 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.921 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.921 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.921 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105396 2024-09-14T09:07:22.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.921 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847139+0000", 2024-09-14T09:07:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847139+0000", 2024-09-14T09:07:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847140+0000", 2024-09-14T09:07:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1379999999999999e-06 2024-09-14T09:07:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847144+0000", 2024-09-14T09:07:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.121e-06 2024-09-14T09:07:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847159+0000", 2024-09-14T09:07:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4090000000000001e-05 2024-09-14T09:07:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847167+0000", 2024-09-14T09:07:22.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9660000000000002e-06 2024-09-14T09:07:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847241+0000", 2024-09-14T09:07:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3794000000000005e-05 2024-09-14T09:07:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847250+0000", 2024-09-14T09:07:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3299999999999999e-06 2024-09-14T09:07:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861128+0000", 2024-09-14T09:07:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013878002 2024-09-14T09:07:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861136+0000", 2024-09-14T09:07:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5979999999999997e-06 2024-09-14T09:07:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.925 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105461 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857356+0000", 2024-09-14T09:07:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33247262799999999, 2024-09-14T09:07:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34299411800000001, 2024-09-14T09:07:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105461 2024-09-14T09:07:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857356+0000", 2024-09-14T09:07:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857356+0000", 2024-09-14T09:07:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857357+0000", 2024-09-14T09:07:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9400000000000004e-07 2024-09-14T09:07:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857362+0000", 2024-09-14T09:07:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.339e-06 2024-09-14T09:07:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857374+0000", 2024-09-14T09:07:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1783999999999999e-05 2024-09-14T09:07:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857382+0000", 2024-09-14T09:07:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1140000000000006e-06 2024-09-14T09:07:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862699+0000", 2024-09-14T09:07:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053168210000000002 2024-09-14T09:07:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862704+0000", 2024-09-14T09:07:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2870000000000004e-06 2024-09-14T09:07:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105525 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860338+0000", 2024-09-14T09:07:22.929 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32949102000000002, 2024-09-14T09:07:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34007672900000002, 2024-09-14T09:07:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105525 2024-09-14T09:07:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860338+0000", 2024-09-14T09:07:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860338+0000", 2024-09-14T09:07:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860338+0000", 2024-09-14T09:07:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6199999999999998e-07 2024-09-14T09:07:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860340+0000", 2024-09-14T09:07:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2199999999999999e-06 2024-09-14T09:07:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860347+0000", 2024-09-14T09:07:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9190000000000002e-06 2024-09-14T09:07:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860353+0000", 2024-09-14T09:07:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5310000000000001e-06 2024-09-14T09:07:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864527+0000", 2024-09-14T09:07:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041740010000000001 2024-09-14T09:07:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864541+0000", 2024-09-14T09:07:22.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3932e-05 2024-09-14T09:07:22.933 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.933 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.933 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.933 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105590 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.933 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861744+0000", 2024-09-14T09:07:22.933 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32808465399999998, 2024-09-14T09:07:22.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33873436000000001, 2024-09-14T09:07:22.934 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.934 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.934 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.934 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.934 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.934 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.935 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105590 2024-09-14T09:07:22.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.935 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861744+0000", 2024-09-14T09:07:22.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861744+0000", 2024-09-14T09:07:22.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861744+0000", 2024-09-14T09:07:22.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0299999999999999e-07 2024-09-14T09:07:22.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861747+0000", 2024-09-14T09:07:22.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.948e-06 2024-09-14T09:07:22.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861756+0000", 2024-09-14T09:07:22.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7760000000000003e-06 2024-09-14T09:07:22.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861763+0000", 2024-09-14T09:07:22.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6540000000000002e-06 2024-09-14T09:07:22.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865590+0000", 2024-09-14T09:07:22.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003827404 2024-09-14T09:07:22.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865597+0000", 2024-09-14T09:07:22.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2999999999999998e-06 2024-09-14T09:07:22.938 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.938 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.938 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.938 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105653 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.938 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:07:22.939 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32259410399999999, 2024-09-14T09:07:22.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33330440700000002, 2024-09-14T09:07:22.939 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.939 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.939 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.939 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.939 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.939 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.939 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105653 2024-09-14T09:07:22.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.939 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:07:22.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:07:22.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867235+0000", 2024-09-14T09:07:22.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9400000000000004e-07 2024-09-14T09:07:22.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867239+0000", 2024-09-14T09:07:22.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9380000000000002e-06 2024-09-14T09:07:22.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867252+0000", 2024-09-14T09:07:22.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.31e-05 2024-09-14T09:07:22.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867264+0000", 2024-09-14T09:07:22.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1275e-05 2024-09-14T09:07:22.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867367+0000", 2024-09-14T09:07:22.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103755 2024-09-14T09:07:22.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867382+0000", 2024-09-14T09:07:22.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4724e-05 2024-09-14T09:07:22.943 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.943 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.943 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.943 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105717 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.943 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868968+0000", 2024-09-14T09:07:22.943 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.320860847, 2024-09-14T09:07:22.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33163459699999998, 2024-09-14T09:07:22.944 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.944 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.944 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.944 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.944 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.944 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.944 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105717 2024-09-14T09:07:22.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.944 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868968+0000", 2024-09-14T09:07:22.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868968+0000", 2024-09-14T09:07:22.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868968+0000", 2024-09-14T09:07:22.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5000000000000002e-07 2024-09-14T09:07:22.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868972+0000", 2024-09-14T09:07:22.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1820000000000002e-06 2024-09-14T09:07:22.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868982+0000", 2024-09-14T09:07:22.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0776e-05 2024-09-14T09:07:22.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868991+0000", 2024-09-14T09:07:22.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8410000000000006e-06 2024-09-14T09:07:22.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869261+0000", 2024-09-14T09:07:22.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026958400000000001 2024-09-14T09:07:22.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869273+0000", 2024-09-14T09:07:22.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2164000000000001e-05 2024-09-14T09:07:22.947 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.947 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.947 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.948 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105271 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.948 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.840914+0000", 2024-09-14T09:07:22.948 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34891502400000002, 2024-09-14T09:07:22.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35975013099999997, 2024-09-14T09:07:22.948 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.948 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.948 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.948 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.948 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.948 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.948 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105271 2024-09-14T09:07:22.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.948 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840914+0000", 2024-09-14T09:07:22.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840914+0000", 2024-09-14T09:07:22.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840915+0000", 2024-09-14T09:07:22.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.077e-06 2024-09-14T09:07:22.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840920+0000", 2024-09-14T09:07:22.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0250000000000002e-06 2024-09-14T09:07:22.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840933+0000", 2024-09-14T09:07:22.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3395e-05 2024-09-14T09:07:22.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840948+0000", 2024-09-14T09:07:22.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5221999999999999e-05 2024-09-14T09:07:22.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841403+0000", 2024-09-14T09:07:22.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00045517000000000002 2024-09-14T09:07:22.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841410+0000", 2024-09-14T09:07:22.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3849999999999999e-06 2024-09-14T09:07:22.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857299+0000", 2024-09-14T09:07:22.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015889255000000001 2024-09-14T09:07:22.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857305+0000", 2024-09-14T09:07:22.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2009999999999996e-06 2024-09-14T09:07:22.953 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.953 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.953 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.953 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105335 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.953 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.842390+0000", 2024-09-14T09:07:22.953 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34743821800000002, 2024-09-14T09:07:22.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35835060400000002, 2024-09-14T09:07:22.953 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.953 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.953 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.954 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.954 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.954 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.954 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105335 2024-09-14T09:07:22.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.954 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842390+0000", 2024-09-14T09:07:22.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842390+0000", 2024-09-14T09:07:22.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842391+0000", 2024-09-14T09:07:22.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1799999999999995e-07 2024-09-14T09:07:22.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842394+0000", 2024-09-14T09:07:22.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.066e-06 2024-09-14T09:07:22.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842405+0000", 2024-09-14T09:07:22.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0648e-05 2024-09-14T09:07:22.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842412+0000", 2024-09-14T09:07:22.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6539999999999999e-06 2024-09-14T09:07:22.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842828+0000", 2024-09-14T09:07:22.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00041615800000000002 2024-09-14T09:07:22.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842833+0000", 2024-09-14T09:07:22.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5059999999999997e-06 2024-09-14T09:07:22.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860121+0000", 2024-09-14T09:07:22.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017288515000000001 2024-09-14T09:07:22.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860131+0000", 2024-09-14T09:07:22.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.037e-06 2024-09-14T09:07:22.959 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.959 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.959 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.959 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105399 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.959 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.847181+0000", 2024-09-14T09:07:22.959 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34264799400000001, 2024-09-14T09:07:22.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35363474900000003, 2024-09-14T09:07:22.959 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.960 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.960 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.960 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.960 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.960 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.960 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105399 2024-09-14T09:07:22.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.960 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847181+0000", 2024-09-14T09:07:22.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847181+0000", 2024-09-14T09:07:22.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847181+0000", 2024-09-14T09:07:22.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7800000000000001e-07 2024-09-14T09:07:22.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847185+0000", 2024-09-14T09:07:22.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4460000000000001e-06 2024-09-14T09:07:22.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847196+0000", 2024-09-14T09:07:22.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0849999999999999e-05 2024-09-14T09:07:22.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847205+0000", 2024-09-14T09:07:22.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7019999999999996e-06 2024-09-14T09:07:22.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847293+0000", 2024-09-14T09:07:22.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7985000000000005e-05 2024-09-14T09:07:22.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847302+0000", 2024-09-14T09:07:22.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7280000000000001e-06 2024-09-14T09:07:22.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861181+0000", 2024-09-14T09:07:22.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013879417 2024-09-14T09:07:22.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861191+0000", 2024-09-14T09:07:22.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3049999999999998e-06 2024-09-14T09:07:22.963 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.963 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.963 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.964 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105462 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.964 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857508+0000", 2024-09-14T09:07:22.964 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33232029200000002, 2024-09-14T09:07:22.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34337889500000002, 2024-09-14T09:07:22.964 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.964 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.964 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.964 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.964 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.964 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.964 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105462 2024-09-14T09:07:22.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.965 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857508+0000", 2024-09-14T09:07:22.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857508+0000", 2024-09-14T09:07:22.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857509+0000", 2024-09-14T09:07:22.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2099999999999996e-07 2024-09-14T09:07:22.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857512+0000", 2024-09-14T09:07:22.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0960000000000001e-06 2024-09-14T09:07:22.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857520+0000", 2024-09-14T09:07:22.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6369999999999999e-06 2024-09-14T09:07:22.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857524+0000", 2024-09-14T09:07:22.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8199999999999996e-06 2024-09-14T09:07:22.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862730+0000", 2024-09-14T09:07:22.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052060190000000001 2024-09-14T09:07:22.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.968 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862735+0000", 2024-09-14T09:07:22.968 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3699999999999997e-06 2024-09-14T09:07:22.968 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.968 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.968 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.968 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105527 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.968 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860360+0000", 2024-09-14T09:07:22.968 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32946851700000002, 2024-09-14T09:07:22.968 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34059335499999999, 2024-09-14T09:07:22.969 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.969 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.969 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.969 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.969 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.969 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.969 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105527 2024-09-14T09:07:22.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.969 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.969 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.969 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860360+0000", 2024-09-14T09:07:22.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860360+0000", 2024-09-14T09:07:22.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860361+0000", 2024-09-14T09:07:22.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9800000000000004e-07 2024-09-14T09:07:22.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860363+0000", 2024-09-14T09:07:22.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1629999999999999e-06 2024-09-14T09:07:22.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860369+0000", 2024-09-14T09:07:22.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4289999999999998e-06 2024-09-14T09:07:22.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860374+0000", 2024-09-14T09:07:22.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1000000000000003e-06 2024-09-14T09:07:22.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864586+0000", 2024-09-14T09:07:22.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042115590000000001 2024-09-14T09:07:22.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864593+0000", 2024-09-14T09:07:22.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5519999999999999e-06 2024-09-14T09:07:22.973 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.973 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.973 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.973 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105591 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.973 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861792+0000", 2024-09-14T09:07:22.973 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32803690000000002, 2024-09-14T09:07:22.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.339225322, 2024-09-14T09:07:22.973 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.973 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.973 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.974 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.974 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.974 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.974 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105591 2024-09-14T09:07:22.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.974 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.974 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861792+0000", 2024-09-14T09:07:22.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.974 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861792+0000", 2024-09-14T09:07:22.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861792+0000", 2024-09-14T09:07:22.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1399999999999996e-07 2024-09-14T09:07:22.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861795+0000", 2024-09-14T09:07:22.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9670000000000001e-06 2024-09-14T09:07:22.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861804+0000", 2024-09-14T09:07:22.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8070000000000006e-06 2024-09-14T09:07:22.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861813+0000", 2024-09-14T09:07:22.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6070000000000008e-06 2024-09-14T09:07:22.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865623+0000", 2024-09-14T09:07:22.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038098060000000002 2024-09-14T09:07:22.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865629+0000", 2024-09-14T09:07:22.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3130000000000004e-06 2024-09-14T09:07:22.977 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.977 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.977 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.977 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105654 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.977 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:07:22.977 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32253304799999999, 2024-09-14T09:07:22.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33378217900000001, 2024-09-14T09:07:22.977 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.977 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.978 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.978 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.978 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.978 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.978 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105654 2024-09-14T09:07:22.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.978 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:07:22.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:07:22.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867296+0000", 2024-09-14T09:07:22.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2399999999999996e-07 2024-09-14T09:07:22.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867300+0000", 2024-09-14T09:07:22.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8689999999999997e-06 2024-09-14T09:07:22.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867313+0000", 2024-09-14T09:07:22.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2660000000000001e-05 2024-09-14T09:07:22.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.980 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.980 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867322+0000", 2024-09-14T09:07:22.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.533e-06 2024-09-14T09:07:22.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.980 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.980 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867429+0000", 2024-09-14T09:07:22.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106566 2024-09-14T09:07:22.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.980 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.981 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867442+0000", 2024-09-14T09:07:22.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3244000000000001e-05 2024-09-14T09:07:22.981 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.981 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.981 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.981 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105719 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.981 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.869002+0000", 2024-09-14T09:07:22.981 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32082659600000002, 2024-09-14T09:07:22.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33213926799999999, 2024-09-14T09:07:22.981 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.981 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.981 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.982 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.982 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.982 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.982 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105719 2024-09-14T09:07:22.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.982 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869002+0000", 2024-09-14T09:07:22.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869002+0000", 2024-09-14T09:07:22.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869003+0000", 2024-09-14T09:07:22.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1399999999999996e-07 2024-09-14T09:07:22.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869006+0000", 2024-09-14T09:07:22.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8959999999999999e-06 2024-09-14T09:07:22.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869016+0000", 2024-09-14T09:07:22.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0713999999999999e-05 2024-09-14T09:07:22.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869025+0000", 2024-09-14T09:07:22.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3699999999999995e-06 2024-09-14T09:07:22.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869321+0000", 2024-09-14T09:07:22.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000296717 2024-09-14T09:07:22.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869331+0000", 2024-09-14T09:07:22.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5540000000000003e-06 2024-09-14T09:07:22.985 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.985 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.985 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.985 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105273 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.985 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.840994+0000", 2024-09-14T09:07:22.985 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34883499099999998, 2024-09-14T09:07:22.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.36020855699999998, 2024-09-14T09:07:22.985 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.985 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.985 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.985 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.985 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.986 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.986 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105273 2024-09-14T09:07:22.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.986 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.986 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840994+0000", 2024-09-14T09:07:22.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840994+0000", 2024-09-14T09:07:22.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.840995+0000", 2024-09-14T09:07:22.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2750000000000001e-06 2024-09-14T09:07:22.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841000+0000", 2024-09-14T09:07:22.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5310000000000001e-06 2024-09-14T09:07:22.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841018+0000", 2024-09-14T09:07:22.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7200000000000001e-05 2024-09-14T09:07:22.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841031+0000", 2024-09-14T09:07:22.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3251000000000001e-05 2024-09-14T09:07:22.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841459+0000", 2024-09-14T09:07:22.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00042804799999999999 2024-09-14T09:07:22.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841465+0000", 2024-09-14T09:07:22.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4400000000000002e-06 2024-09-14T09:07:22.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857331+0000", 2024-09-14T09:07:22.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015865442 2024-09-14T09:07:22.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857337+0000", 2024-09-14T09:07:22.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7240000000000003e-06 2024-09-14T09:07:22.991 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.991 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.991 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.991 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105337 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.991 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.842424+0000", 2024-09-14T09:07:22.991 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.347404502, 2024-09-14T09:07:22.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35885170700000002, 2024-09-14T09:07:22.991 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.991 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.991 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.991 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.991 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.992 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.992 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105337 2024-09-14T09:07:22.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.992 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842424+0000", 2024-09-14T09:07:22.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842424+0000", 2024-09-14T09:07:22.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842425+0000", 2024-09-14T09:07:22.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0899999999999999e-06 2024-09-14T09:07:22.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842428+0000", 2024-09-14T09:07:22.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.224e-06 2024-09-14T09:07:22.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842439+0000", 2024-09-14T09:07:22.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0698e-05 2024-09-14T09:07:22.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842447+0000", 2024-09-14T09:07:22.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9179999999999994e-06 2024-09-14T09:07:22.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842858+0000", 2024-09-14T09:07:22.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00041146800000000001 2024-09-14T09:07:22.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842863+0000", 2024-09-14T09:07:22.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4870000000000004e-06 2024-09-14T09:07:22.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860164+0000", 2024-09-14T09:07:22.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017301016999999998 2024-09-14T09:07:22.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:22.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860173+0000", 2024-09-14T09:07:22.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8340000000000005e-06 2024-09-14T09:07:22.996 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.996 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:22.996 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:22.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.996 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105401 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:22.996 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.847235+0000", 2024-09-14T09:07:22.996 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34259393799999999, 2024-09-14T09:07:22.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.354116027, 2024-09-14T09:07:22.996 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:22.996 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:22.996 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:22.996 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:22.997 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:22.997 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:22.997 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105401 2024-09-14T09:07:22.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.997 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:22.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:22.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847235+0000", 2024-09-14T09:07:22.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:22.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847235+0000", 2024-09-14T09:07:22.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:22.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:22.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847235+0000", 2024-09-14T09:07:22.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1999999999999998e-07 2024-09-14T09:07:22.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:22.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847239+0000", 2024-09-14T09:07:22.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5659999999999998e-06 2024-09-14T09:07:22.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:22.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847250+0000", 2024-09-14T09:07:22.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0902e-05 2024-09-14T09:07:22.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:22.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847258+0000", 2024-09-14T09:07:22.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2179999999999999e-06 2024-09-14T09:07:22.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:22.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847352+0000", 2024-09-14T09:07:22.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3795000000000006e-05 2024-09-14T09:07:22.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:22.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:22.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:22.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847360+0000", 2024-09-14T09:07:22.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2020000000000004e-06 2024-09-14T09:07:23.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861227+0000", 2024-09-14T09:07:23.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013866709 2024-09-14T09:07:23.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861236+0000", 2024-09-14T09:07:23.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8820000000000007e-06 2024-09-14T09:07:23.000 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.000 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.000 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.001 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105465 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.001 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857544+0000", 2024-09-14T09:07:23.001 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33228477299999998, 2024-09-14T09:07:23.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34387845500000003, 2024-09-14T09:07:23.001 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.001 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.001 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.001 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.001 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.001 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.001 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105465 2024-09-14T09:07:23.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.001 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857544+0000", 2024-09-14T09:07:23.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857544+0000", 2024-09-14T09:07:23.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857544+0000", 2024-09-14T09:07:23.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5299999999999999e-07 2024-09-14T09:07:23.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857546+0000", 2024-09-14T09:07:23.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.88e-06 2024-09-14T09:07:23.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857551+0000", 2024-09-14T09:07:23.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2270000000000001e-06 2024-09-14T09:07:23.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857555+0000", 2024-09-14T09:07:23.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8589999999999998e-06 2024-09-14T09:07:23.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862753+0000", 2024-09-14T09:07:23.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051975420000000003 2024-09-14T09:07:23.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862757+0000", 2024-09-14T09:07:23.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2400000000000001e-06 2024-09-14T09:07:23.004 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.004 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.005 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.005 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105529 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.005 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860381+0000", 2024-09-14T09:07:23.005 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32944798199999997, 2024-09-14T09:07:23.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34110530100000003, 2024-09-14T09:07:23.005 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.005 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.005 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.006 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.006 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.006 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.006 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105529 2024-09-14T09:07:23.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.006 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860381+0000", 2024-09-14T09:07:23.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860381+0000", 2024-09-14T09:07:23.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860381+0000", 2024-09-14T09:07:23.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.39e-07 2024-09-14T09:07:23.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860383+0000", 2024-09-14T09:07:23.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0700000000000001e-06 2024-09-14T09:07:23.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860390+0000", 2024-09-14T09:07:23.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4880000000000004e-06 2024-09-14T09:07:23.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860395+0000", 2024-09-14T09:07:23.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1259999999999997e-06 2024-09-14T09:07:23.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864620+0000", 2024-09-14T09:07:23.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042257579999999996 2024-09-14T09:07:23.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864627+0000", 2024-09-14T09:07:23.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.725e-06 2024-09-14T09:07:23.009 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.009 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.010 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.010 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105593 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.010 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861840+0000", 2024-09-14T09:07:23.010 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.327988112, 2024-09-14T09:07:23.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33970890799999998, 2024-09-14T09:07:23.011 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.011 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.011 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.011 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.011 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.011 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.012 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105593 2024-09-14T09:07:23.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.012 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861840+0000", 2024-09-14T09:07:23.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861840+0000", 2024-09-14T09:07:23.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861841+0000", 2024-09-14T09:07:23.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.13e-07 2024-09-14T09:07:23.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861844+0000", 2024-09-14T09:07:23.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9950000000000001e-06 2024-09-14T09:07:23.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861853+0000", 2024-09-14T09:07:23.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8550000000000008e-06 2024-09-14T09:07:23.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861860+0000", 2024-09-14T09:07:23.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9079999999999998e-06 2024-09-14T09:07:23.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865655+0000", 2024-09-14T09:07:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037947969999999999 2024-09-14T09:07:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865661+0000", 2024-09-14T09:07:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.302e-06 2024-09-14T09:07:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105657 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:07:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32247187500000002, 2024-09-14T09:07:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33425380199999999, 2024-09-14T09:07:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105657 2024-09-14T09:07:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:07:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:07:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867358+0000", 2024-09-14T09:07:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9700000000000005e-07 2024-09-14T09:07:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867367+0000", 2024-09-14T09:07:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8919999999999995e-06 2024-09-14T09:07:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867383+0000", 2024-09-14T09:07:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.579e-05 2024-09-14T09:07:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867393+0000", 2024-09-14T09:07:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3070000000000008e-06 2024-09-14T09:07:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867490+0000", 2024-09-14T09:07:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7703999999999998e-05 2024-09-14T09:07:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867503+0000", 2024-09-14T09:07:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2849e-05 2024-09-14T09:07:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105721 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.869054+0000", 2024-09-14T09:07:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.320774948, 2024-09-14T09:07:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33262055400000001, 2024-09-14T09:07:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105721 2024-09-14T09:07:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869054+0000", 2024-09-14T09:07:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869054+0000", 2024-09-14T09:07:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869054+0000", 2024-09-14T09:07:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9200000000000001e-07 2024-09-14T09:07:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869057+0000", 2024-09-14T09:07:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.79e-06 2024-09-14T09:07:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869066+0000", 2024-09-14T09:07:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9740000000000008e-06 2024-09-14T09:07:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869071+0000", 2024-09-14T09:07:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0200000000000002e-06 2024-09-14T09:07:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869388+0000", 2024-09-14T09:07:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031747299999999997 2024-09-14T09:07:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869401+0000", 2024-09-14T09:07:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2765999999999999e-05 2024-09-14T09:07:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105274 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841087+0000", 2024-09-14T09:07:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34874126900000002, 2024-09-14T09:07:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.36064758099999999, 2024-09-14T09:07:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105274 2024-09-14T09:07:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841087+0000", 2024-09-14T09:07:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841087+0000", 2024-09-14T09:07:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841089+0000", 2024-09-14T09:07:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.455e-06 2024-09-14T09:07:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841097+0000", 2024-09-14T09:07:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9449999999999993e-06 2024-09-14T09:07:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841115+0000", 2024-09-14T09:07:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8680000000000001e-05 2024-09-14T09:07:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841135+0000", 2024-09-14T09:07:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0049000000000001e-05 2024-09-14T09:07:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841520+0000", 2024-09-14T09:07:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038457200000000002 2024-09-14T09:07:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841525+0000", 2024-09-14T09:07:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2270000000000001e-06 2024-09-14T09:07:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857364+0000", 2024-09-14T09:07:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015838402000000001 2024-09-14T09:07:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857370+0000", 2024-09-14T09:07:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8229999999999997e-06 2024-09-14T09:07:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105339 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.842459+0000", 2024-09-14T09:07:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34736952999999998, 2024-09-14T09:07:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35934987499999999, 2024-09-14T09:07:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105339 2024-09-14T09:07:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842459+0000", 2024-09-14T09:07:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842459+0000", 2024-09-14T09:07:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842460+0000", 2024-09-14T09:07:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:07:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842463+0000", 2024-09-14T09:07:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5300000000000001e-06 2024-09-14T09:07:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842476+0000", 2024-09-14T09:07:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2387e-05 2024-09-14T09:07:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842483+0000", 2024-09-14T09:07:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3780000000000003e-06 2024-09-14T09:07:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842888+0000", 2024-09-14T09:07:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000405139 2024-09-14T09:07:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842893+0000", 2024-09-14T09:07:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5859999999999998e-06 2024-09-14T09:07:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860210+0000", 2024-09-14T09:07:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017316776999999998 2024-09-14T09:07:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860219+0000", 2024-09-14T09:07:23.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1400000000000006e-06 2024-09-14T09:07:23.035 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.035 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.035 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105403 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.847285+0000", 2024-09-14T09:07:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.342543761, 2024-09-14T09:07:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.354598564, 2024-09-14T09:07:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105403 2024-09-14T09:07:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847285+0000", 2024-09-14T09:07:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847285+0000", 2024-09-14T09:07:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847286+0000", 2024-09-14T09:07:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6400000000000001e-07 2024-09-14T09:07:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847290+0000", 2024-09-14T09:07:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8759999999999998e-06 2024-09-14T09:07:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847301+0000", 2024-09-14T09:07:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1161999999999999e-05 2024-09-14T09:07:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847309+0000", 2024-09-14T09:07:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1540000000000002e-06 2024-09-14T09:07:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847413+0000", 2024-09-14T09:07:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000104431 2024-09-14T09:07:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847424+0000", 2024-09-14T09:07:23.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0474e-05 2024-09-14T09:07:23.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861283+0000", 2024-09-14T09:07:23.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013858838 2024-09-14T09:07:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861292+0000", 2024-09-14T09:07:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1789999999999997e-06 2024-09-14T09:07:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105467 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857561+0000", 2024-09-14T09:07:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33226783100000001, 2024-09-14T09:07:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.344394382, 2024-09-14T09:07:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105467 2024-09-14T09:07:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857561+0000", 2024-09-14T09:07:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857561+0000", 2024-09-14T09:07:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857561+0000", 2024-09-14T09:07:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9499999999999998e-07 2024-09-14T09:07:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857569+0000", 2024-09-14T09:07:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0290000000000005e-06 2024-09-14T09:07:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857578+0000", 2024-09-14T09:07:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6570000000000003e-06 2024-09-14T09:07:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857585+0000", 2024-09-14T09:07:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.81e-06 2024-09-14T09:07:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862775+0000", 2024-09-14T09:07:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051903629999999999 2024-09-14T09:07:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862779+0000", 2024-09-14T09:07:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3869999999999997e-06 2024-09-14T09:07:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105532 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860410+0000", 2024-09-14T09:07:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32941893799999999, 2024-09-14T09:07:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34160892799999998, 2024-09-14T09:07:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105532 2024-09-14T09:07:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860410+0000", 2024-09-14T09:07:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860410+0000", 2024-09-14T09:07:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860410+0000", 2024-09-14T09:07:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4099999999999999e-07 2024-09-14T09:07:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860412+0000", 2024-09-14T09:07:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2570000000000002e-06 2024-09-14T09:07:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860419+0000", 2024-09-14T09:07:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8730000000000001e-06 2024-09-14T09:07:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860424+0000", 2024-09-14T09:07:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2490000000000001e-06 2024-09-14T09:07:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864654+0000", 2024-09-14T09:07:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004229366 2024-09-14T09:07:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864660+0000", 2024-09-14T09:07:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.64e-06 2024-09-14T09:07:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105595 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861887+0000", 2024-09-14T09:07:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32794112800000003, 2024-09-14T09:07:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.340191882, 2024-09-14T09:07:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105595 2024-09-14T09:07:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861887+0000", 2024-09-14T09:07:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861887+0000", 2024-09-14T09:07:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861888+0000", 2024-09-14T09:07:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5600000000000005e-07 2024-09-14T09:07:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861891+0000", 2024-09-14T09:07:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8779999999999998e-06 2024-09-14T09:07:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861900+0000", 2024-09-14T09:07:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8799999999999997e-06 2024-09-14T09:07:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861907+0000", 2024-09-14T09:07:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8000000000000001e-06 2024-09-14T09:07:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865687+0000", 2024-09-14T09:07:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003780308 2024-09-14T09:07:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865693+0000", 2024-09-14T09:07:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3559999999999998e-06 2024-09-14T09:07:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105658 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:07:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32240949099999999, 2024-09-14T09:07:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33472404900000002, 2024-09-14T09:07:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105658 2024-09-14T09:07:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:07:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:07:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867420+0000", 2024-09-14T09:07:23.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5000000000000001e-07 2024-09-14T09:07:23.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867424+0000", 2024-09-14T09:07:23.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8469999999999998e-06 2024-09-14T09:07:23.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867437+0000", 2024-09-14T09:07:23.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3156000000000001e-05 2024-09-14T09:07:23.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867448+0000", 2024-09-14T09:07:23.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1185e-05 2024-09-14T09:07:23.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867552+0000", 2024-09-14T09:07:23.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103362 2024-09-14T09:07:23.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867565+0000", 2024-09-14T09:07:23.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3018e-05 2024-09-14T09:07:23.063 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.063 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.063 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.063 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105723 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.063 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.869079+0000", 2024-09-14T09:07:23.063 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32074992400000002, 2024-09-14T09:07:23.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33312805099999998, 2024-09-14T09:07:23.063 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.063 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.063 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.063 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.064 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.064 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.064 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105723 2024-09-14T09:07:23.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.064 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869079+0000", 2024-09-14T09:07:23.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869079+0000", 2024-09-14T09:07:23.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869079+0000", 2024-09-14T09:07:23.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1200000000000003e-07 2024-09-14T09:07:23.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869081+0000", 2024-09-14T09:07:23.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9240000000000001e-06 2024-09-14T09:07:23.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869087+0000", 2024-09-14T09:07:23.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8799999999999996e-06 2024-09-14T09:07:23.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869095+0000", 2024-09-14T09:07:23.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2900000000000002e-06 2024-09-14T09:07:23.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869449+0000", 2024-09-14T09:07:23.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035382399999999998 2024-09-14T09:07:23.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869457+0000", 2024-09-14T09:07:23.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7449999999999995e-06 2024-09-14T09:07:23.067 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.067 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.067 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.067 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105278 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.067 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841161+0000", 2024-09-14T09:07:23.067 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34866757199999998, 2024-09-14T09:07:23.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.36110636899999998, 2024-09-14T09:07:23.068 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.068 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.068 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.068 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.068 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.068 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.068 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105278 2024-09-14T09:07:23.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.068 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841161+0000", 2024-09-14T09:07:23.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841161+0000", 2024-09-14T09:07:23.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841162+0000", 2024-09-14T09:07:23.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.277e-06 2024-09-14T09:07:23.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841206+0000", 2024-09-14T09:07:23.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3927999999999997e-05 2024-09-14T09:07:23.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841224+0000", 2024-09-14T09:07:23.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7464000000000001e-05 2024-09-14T09:07:23.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841234+0000", 2024-09-14T09:07:23.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0071999999999999e-05 2024-09-14T09:07:23.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841580+0000", 2024-09-14T09:07:23.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034642899999999999 2024-09-14T09:07:23.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841586+0000", 2024-09-14T09:07:23.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.032e-06 2024-09-14T09:07:23.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857393+0000", 2024-09-14T09:07:23.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015806899999999999 2024-09-14T09:07:23.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857399+0000", 2024-09-14T09:07:23.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2979999999999997e-06 2024-09-14T09:07:23.073 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.073 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.073 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.073 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105341 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.073 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.842494+0000", 2024-09-14T09:07:23.073 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34733472700000001, 2024-09-14T09:07:23.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.359849686, 2024-09-14T09:07:23.073 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.074 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.074 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.074 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.074 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.074 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.074 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105341 2024-09-14T09:07:23.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.074 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842494+0000", 2024-09-14T09:07:23.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842494+0000", 2024-09-14T09:07:23.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842495+0000", 2024-09-14T09:07:23.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1800000000000005e-07 2024-09-14T09:07:23.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842498+0000", 2024-09-14T09:07:23.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6660000000000001e-06 2024-09-14T09:07:23.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842508+0000", 2024-09-14T09:07:23.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0186e-05 2024-09-14T09:07:23.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842516+0000", 2024-09-14T09:07:23.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5979999999999999e-06 2024-09-14T09:07:23.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842917+0000", 2024-09-14T09:07:23.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040151299999999999 2024-09-14T09:07:23.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842922+0000", 2024-09-14T09:07:23.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4649999999999996e-06 2024-09-14T09:07:23.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860255+0000", 2024-09-14T09:07:23.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017333129999999999 2024-09-14T09:07:23.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860264+0000", 2024-09-14T09:07:23.079 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8019999999999998e-06 2024-09-14T09:07:23.079 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.079 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.079 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.079 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.079 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.079 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105405 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.079 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.847341+0000", 2024-09-14T09:07:23.079 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.342487127, 2024-09-14T09:07:23.079 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.355077274, 2024-09-14T09:07:23.079 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.079 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.079 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.080 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.080 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.080 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.080 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105405 2024-09-14T09:07:23.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.080 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.080 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.080 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.080 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847341+0000", 2024-09-14T09:07:23.080 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.081 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.081 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.081 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847341+0000", 2024-09-14T09:07:23.081 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.081 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.081 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.081 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.081 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847342+0000", 2024-09-14T09:07:23.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9400000000000004e-07 2024-09-14T09:07:23.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.082 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.082 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.082 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847346+0000", 2024-09-14T09:07:23.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6540000000000001e-06 2024-09-14T09:07:23.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.082 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.082 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.082 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847357+0000", 2024-09-14T09:07:23.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1345999999999999e-05 2024-09-14T09:07:23.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847365+0000", 2024-09-14T09:07:23.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0260000000000007e-06 2024-09-14T09:07:23.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847473+0000", 2024-09-14T09:07:23.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000107877 2024-09-14T09:07:23.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847483+0000", 2024-09-14T09:07:23.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5780000000000005e-06 2024-09-14T09:07:23.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.084 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861328+0000", 2024-09-14T09:07:23.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013845537 2024-09-14T09:07:23.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.084 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861337+0000", 2024-09-14T09:07:23.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7660000000000004e-06 2024-09-14T09:07:23.085 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.085 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.085 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.085 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105469 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.085 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857602+0000", 2024-09-14T09:07:23.085 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33222621899999999, 2024-09-14T09:07:23.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.344887469, 2024-09-14T09:07:23.085 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.085 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.085 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.085 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.085 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.086 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.086 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105469 2024-09-14T09:07:23.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.086 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.086 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.086 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857602+0000", 2024-09-14T09:07:23.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.086 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.086 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857602+0000", 2024-09-14T09:07:23.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.087 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.087 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857603+0000", 2024-09-14T09:07:23.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-09-14T09:07:23.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.087 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.087 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857606+0000", 2024-09-14T09:07:23.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0639999999999998e-06 2024-09-14T09:07:23.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857615+0000", 2024-09-14T09:07:23.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6670000000000002e-06 2024-09-14T09:07:23.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857622+0000", 2024-09-14T09:07:23.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8299999999999998e-06 2024-09-14T09:07:23.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862797+0000", 2024-09-14T09:07:23.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051756440000000001 2024-09-14T09:07:23.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862801+0000", 2024-09-14T09:07:23.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1779999999999996e-06 2024-09-14T09:07:23.089 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.089 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.089 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.089 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105533 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.089 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860457+0000", 2024-09-14T09:07:23.089 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32937165099999999, 2024-09-14T09:07:23.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.342096648, 2024-09-14T09:07:23.089 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.090 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.090 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.090 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.090 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.090 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.090 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105533 2024-09-14T09:07:23.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.090 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860457+0000", 2024-09-14T09:07:23.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860457+0000", 2024-09-14T09:07:23.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860457+0000", 2024-09-14T09:07:23.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8599999999999998e-07 2024-09-14T09:07:23.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860460+0000", 2024-09-14T09:07:23.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3659999999999999e-06 2024-09-14T09:07:23.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860468+0000", 2024-09-14T09:07:23.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8939999999999993e-06 2024-09-14T09:07:23.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860473+0000", 2024-09-14T09:07:23.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5280000000000003e-06 2024-09-14T09:07:23.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864687+0000", 2024-09-14T09:07:23.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042136780000000002 2024-09-14T09:07:23.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864693+0000", 2024-09-14T09:07:23.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4869999999999999e-06 2024-09-14T09:07:23.093 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.093 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.093 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.093 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105598 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.093 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.861937+0000", 2024-09-14T09:07:23.093 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32789193700000002, 2024-09-14T09:07:23.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34067736700000001, 2024-09-14T09:07:23.094 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.094 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.094 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.094 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.094 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.094 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.094 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105598 2024-09-14T09:07:23.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.094 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861937+0000", 2024-09-14T09:07:23.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861937+0000", 2024-09-14T09:07:23.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861937+0000", 2024-09-14T09:07:23.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4000000000000001e-07 2024-09-14T09:07:23.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861940+0000", 2024-09-14T09:07:23.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9629999999999998e-06 2024-09-14T09:07:23.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861949+0000", 2024-09-14T09:07:23.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6209999999999993e-06 2024-09-14T09:07:23.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861956+0000", 2024-09-14T09:07:23.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8839999999999997e-06 2024-09-14T09:07:23.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865719+0000", 2024-09-14T09:07:23.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037630889999999998 2024-09-14T09:07:23.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865725+0000", 2024-09-14T09:07:23.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3430000000000001e-06 2024-09-14T09:07:23.097 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.097 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.097 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.098 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105662 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.098 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:07:23.098 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32234842600000002, 2024-09-14T09:07:23.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33519758500000002, 2024-09-14T09:07:23.098 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.098 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.098 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.098 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.098 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.098 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.098 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105662 2024-09-14T09:07:23.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.099 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:07:23.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:07:23.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867481+0000", 2024-09-14T09:07:23.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1800000000000004e-07 2024-09-14T09:07:23.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867485+0000", 2024-09-14T09:07:23.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9199999999999997e-06 2024-09-14T09:07:23.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867498+0000", 2024-09-14T09:07:23.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2896e-05 2024-09-14T09:07:23.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867509+0000", 2024-09-14T09:07:23.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1202e-05 2024-09-14T09:07:23.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867611+0000", 2024-09-14T09:07:23.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010166999999999999 2024-09-14T09:07:23.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867624+0000", 2024-09-14T09:07:23.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2996000000000001e-05 2024-09-14T09:07:23.102 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.102 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.102 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.102 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105724 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.102 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.869124+0000", 2024-09-14T09:07:23.102 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32070485500000001, 2024-09-14T09:07:23.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33361805500000002, 2024-09-14T09:07:23.102 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.102 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.102 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.102 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.102 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.103 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.103 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105724 2024-09-14T09:07:23.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.103 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869124+0000", 2024-09-14T09:07:23.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869124+0000", 2024-09-14T09:07:23.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869124+0000", 2024-09-14T09:07:23.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8599999999999997e-07 2024-09-14T09:07:23.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869128+0000", 2024-09-14T09:07:23.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4620000000000001e-06 2024-09-14T09:07:23.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869139+0000", 2024-09-14T09:07:23.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0849999999999999e-05 2024-09-14T09:07:23.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869148+0000", 2024-09-14T09:07:23.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7779999999999996e-06 2024-09-14T09:07:23.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869494+0000", 2024-09-14T09:07:23.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034688399999999999 2024-09-14T09:07:23.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869502+0000", 2024-09-14T09:07:23.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9500000000000001e-06 2024-09-14T09:07:23.106 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.106 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.106 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.106 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105279 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.106 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841250+0000", 2024-09-14T09:07:23.106 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34857886300000002, 2024-09-14T09:07:23.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.36155253399999998, 2024-09-14T09:07:23.106 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.106 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.106 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.106 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.106 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.106 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.107 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105279 2024-09-14T09:07:23.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.107 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841250+0000", 2024-09-14T09:07:23.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841250+0000", 2024-09-14T09:07:23.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841250+0000", 2024-09-14T09:07:23.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8000000000000005e-07 2024-09-14T09:07:23.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841255+0000", 2024-09-14T09:07:23.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.262e-06 2024-09-14T09:07:23.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841266+0000", 2024-09-14T09:07:23.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1238e-05 2024-09-14T09:07:23.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841275+0000", 2024-09-14T09:07:23.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8400000000000001e-06 2024-09-14T09:07:23.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841629+0000", 2024-09-14T09:07:23.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035400299999999998 2024-09-14T09:07:23.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841634+0000", 2024-09-14T09:07:23.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5670000000000002e-06 2024-09-14T09:07:23.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857445+0000", 2024-09-14T09:07:23.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015810993999999998 2024-09-14T09:07:23.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857460+0000", 2024-09-14T09:07:23.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4354e-05 2024-09-14T09:07:23.110 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.110 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.110 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.111 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105343 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.111 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.842527+0000", 2024-09-14T09:07:23.111 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34730127500000002, 2024-09-14T09:07:23.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.36034891699999999, 2024-09-14T09:07:23.111 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.111 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.111 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.111 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.111 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.111 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.111 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105343 2024-09-14T09:07:23.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.112 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842527+0000", 2024-09-14T09:07:23.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842527+0000", 2024-09-14T09:07:23.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842528+0000", 2024-09-14T09:07:23.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4799999999999997e-07 2024-09-14T09:07:23.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842531+0000", 2024-09-14T09:07:23.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1499999999999999e-06 2024-09-14T09:07:23.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842541+0000", 2024-09-14T09:07:23.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0168e-05 2024-09-14T09:07:23.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842549+0000", 2024-09-14T09:07:23.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8800000000000008e-06 2024-09-14T09:07:23.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842948+0000", 2024-09-14T09:07:23.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00039881299999999998 2024-09-14T09:07:23.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842953+0000", 2024-09-14T09:07:23.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5809999999999998e-06 2024-09-14T09:07:23.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860302+0000", 2024-09-14T09:07:23.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.017349065 2024-09-14T09:07:23.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860310+0000", 2024-09-14T09:07:23.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3690000000000007e-06 2024-09-14T09:07:23.115 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.115 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.115 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.116 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105408 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.116 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.847402+0000", 2024-09-14T09:07:23.116 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34242653200000001, 2024-09-14T09:07:23.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35554886600000002, 2024-09-14T09:07:23.116 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.116 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.116 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.116 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.116 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.116 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.116 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105408 2024-09-14T09:07:23.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.116 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847402+0000", 2024-09-14T09:07:23.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847402+0000", 2024-09-14T09:07:23.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847403+0000", 2024-09-14T09:07:23.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-07 2024-09-14T09:07:23.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847407+0000", 2024-09-14T09:07:23.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-09-14T09:07:23.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847418+0000", 2024-09-14T09:07:23.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1616e-05 2024-09-14T09:07:23.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847427+0000", 2024-09-14T09:07:23.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7800000000000006e-06 2024-09-14T09:07:23.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847527+0000", 2024-09-14T09:07:23.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9916999999999999e-05 2024-09-14T09:07:23.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847535+0000", 2024-09-14T09:07:23.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0819999999999999e-06 2024-09-14T09:07:23.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861375+0000", 2024-09-14T09:07:23.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013839947999999999 2024-09-14T09:07:23.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861384+0000", 2024-09-14T09:07:23.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0440000000000001e-06 2024-09-14T09:07:23.120 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.120 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.120 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.120 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105470 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.120 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857631+0000", 2024-09-14T09:07:23.120 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33219734000000001, 2024-09-14T09:07:23.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34539747999999998, 2024-09-14T09:07:23.121 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.121 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.121 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.121 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.121 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.121 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.121 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105470 2024-09-14T09:07:23.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.121 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857631+0000", 2024-09-14T09:07:23.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857631+0000", 2024-09-14T09:07:23.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857632+0000", 2024-09-14T09:07:23.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9800000000000003e-07 2024-09-14T09:07:23.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857635+0000", 2024-09-14T09:07:23.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5189999999999999e-06 2024-09-14T09:07:23.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857645+0000", 2024-09-14T09:07:23.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0644e-05 2024-09-14T09:07:23.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857652+0000", 2024-09-14T09:07:23.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1940000000000001e-06 2024-09-14T09:07:23.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862819+0000", 2024-09-14T09:07:23.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051662469999999997 2024-09-14T09:07:23.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862823+0000", 2024-09-14T09:07:23.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1520000000000002e-06 2024-09-14T09:07:23.124 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.124 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.124 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.125 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105534 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.125 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860481+0000", 2024-09-14T09:07:23.125 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32934772200000001, 2024-09-14T09:07:23.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34261193899999998, 2024-09-14T09:07:23.125 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.125 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.125 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.125 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.125 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.125 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.125 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105534 2024-09-14T09:07:23.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.125 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860481+0000", 2024-09-14T09:07:23.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860481+0000", 2024-09-14T09:07:23.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860481+0000", 2024-09-14T09:07:23.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4799999999999999e-07 2024-09-14T09:07:23.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860484+0000", 2024-09-14T09:07:23.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.26e-06 2024-09-14T09:07:23.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860490+0000", 2024-09-14T09:07:23.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4459999999999998e-06 2024-09-14T09:07:23.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860495+0000", 2024-09-14T09:07:23.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4519999999999996e-06 2024-09-14T09:07:23.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864720+0000", 2024-09-14T09:07:23.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004224284 2024-09-14T09:07:23.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864726+0000", 2024-09-14T09:07:23.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5300000000000002e-06 2024-09-14T09:07:23.128 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.128 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.128 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.129 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105599 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.129 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.863523+0000", 2024-09-14T09:07:23.129 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32630551000000002, 2024-09-14T09:07:23.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33963037400000001, 2024-09-14T09:07:23.129 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.129 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.129 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.129 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.129 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.129 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.129 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105599 2024-09-14T09:07:23.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.130 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863523+0000", 2024-09-14T09:07:23.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863523+0000", 2024-09-14T09:07:23.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863524+0000", 2024-09-14T09:07:23.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4580000000000001e-06 2024-09-14T09:07:23.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863532+0000", 2024-09-14T09:07:23.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3599999999999998e-06 2024-09-14T09:07:23.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863548+0000", 2024-09-14T09:07:23.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6645000000000001e-05 2024-09-14T09:07:23.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863561+0000", 2024-09-14T09:07:23.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2969999999999999e-05 2024-09-14T09:07:23.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865751+0000", 2024-09-14T09:07:23.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021893619999999998 2024-09-14T09:07:23.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865757+0000", 2024-09-14T09:07:23.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3749999999999999e-06 2024-09-14T09:07:23.132 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.132 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.133 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.133 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105663 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.133 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:07:23.133 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32228737499999999, 2024-09-14T09:07:23.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33567592899999998, 2024-09-14T09:07:23.133 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.133 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.133 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.133 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.133 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.133 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.133 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105663 2024-09-14T09:07:23.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.134 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:07:23.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:07:23.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867542+0000", 2024-09-14T09:07:23.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1399999999999995e-07 2024-09-14T09:07:23.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867546+0000", 2024-09-14T09:07:23.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7900000000000001e-06 2024-09-14T09:07:23.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867559+0000", 2024-09-14T09:07:23.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2805e-05 2024-09-14T09:07:23.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867568+0000", 2024-09-14T09:07:23.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-09-14T09:07:23.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867669+0000", 2024-09-14T09:07:23.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100655 2024-09-14T09:07:23.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867682+0000", 2024-09-14T09:07:23.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3013000000000001e-05 2024-09-14T09:07:23.136 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.137 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.137 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.137 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105727 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.137 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.869176+0000", 2024-09-14T09:07:23.137 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32065229099999998, 2024-09-14T09:07:23.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33410419400000002, 2024-09-14T09:07:23.137 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.137 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.137 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.137 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.137 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.137 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.138 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105727 2024-09-14T09:07:23.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.138 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869176+0000", 2024-09-14T09:07:23.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869176+0000", 2024-09-14T09:07:23.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869177+0000", 2024-09-14T09:07:23.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8400000000000004e-07 2024-09-14T09:07:23.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869179+0000", 2024-09-14T09:07:23.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2840000000000001e-06 2024-09-14T09:07:23.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869186+0000", 2024-09-14T09:07:23.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7839999999999998e-06 2024-09-14T09:07:23.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869191+0000", 2024-09-14T09:07:23.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8239999999999999e-06 2024-09-14T09:07:23.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869531+0000", 2024-09-14T09:07:23.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034070500000000001 2024-09-14T09:07:23.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869538+0000", 2024-09-14T09:07:23.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.861e-06 2024-09-14T09:07:23.141 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.141 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.141 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.141 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105280 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.141 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841316+0000", 2024-09-14T09:07:23.141 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34851256600000002, 2024-09-14T09:07:23.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.36202501399999998, 2024-09-14T09:07:23.141 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.141 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.141 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.141 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.141 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.142 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.142 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105280 2024-09-14T09:07:23.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.142 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841316+0000", 2024-09-14T09:07:23.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841316+0000", 2024-09-14T09:07:23.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841317+0000", 2024-09-14T09:07:23.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0860000000000001e-06 2024-09-14T09:07:23.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841321+0000", 2024-09-14T09:07:23.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.658e-06 2024-09-14T09:07:23.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841334+0000", 2024-09-14T09:07:23.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3294e-05 2024-09-14T09:07:23.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841345+0000", 2024-09-14T09:07:23.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1056000000000001e-05 2024-09-14T09:07:23.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841668+0000", 2024-09-14T09:07:23.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032335699999999997 2024-09-14T09:07:23.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841674+0000", 2024-09-14T09:07:23.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1050000000000003e-06 2024-09-14T09:07:23.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857505+0000", 2024-09-14T09:07:23.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015831062999999999 2024-09-14T09:07:23.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857512+0000", 2024-09-14T09:07:23.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2579999999999998e-06 2024-09-14T09:07:23.145 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.146 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.146 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.146 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105345 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.146 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.842559+0000", 2024-09-14T09:07:23.146 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34726905899999999, 2024-09-14T09:07:23.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.36085594399999998, 2024-09-14T09:07:23.146 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.146 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.146 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.146 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.146 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.147 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.147 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105345 2024-09-14T09:07:23.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.147 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842559+0000", 2024-09-14T09:07:23.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842559+0000", 2024-09-14T09:07:23.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842560+0000", 2024-09-14T09:07:23.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6400000000000002e-07 2024-09-14T09:07:23.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842563+0000", 2024-09-14T09:07:23.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2559999999999998e-06 2024-09-14T09:07:23.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842574+0000", 2024-09-14T09:07:23.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0688e-05 2024-09-14T09:07:23.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842583+0000", 2024-09-14T09:07:23.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4360000000000002e-06 2024-09-14T09:07:23.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844684+0000", 2024-09-14T09:07:23.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021016609999999999 2024-09-14T09:07:23.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844694+0000", 2024-09-14T09:07:23.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7620000000000007e-06 2024-09-14T09:07:23.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860346+0000", 2024-09-14T09:07:23.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015652536000000002 2024-09-14T09:07:23.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860356+0000", 2024-09-14T09:07:23.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6069999999999997e-06 2024-09-14T09:07:23.151 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.151 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.151 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.151 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105409 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.151 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.847477+0000", 2024-09-14T09:07:23.151 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34235122400000001, 2024-09-14T09:07:23.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.356012791, 2024-09-14T09:07:23.151 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.151 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.151 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.151 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.152 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.152 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.152 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105409 2024-09-14T09:07:23.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.152 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847477+0000", 2024-09-14T09:07:23.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847477+0000", 2024-09-14T09:07:23.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847478+0000", 2024-09-14T09:07:23.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.034e-06 2024-09-14T09:07:23.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847483+0000", 2024-09-14T09:07:23.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5639999999999998e-06 2024-09-14T09:07:23.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847495+0000", 2024-09-14T09:07:23.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2187999999999999e-05 2024-09-14T09:07:23.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847516+0000", 2024-09-14T09:07:23.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0477999999999999e-05 2024-09-14T09:07:23.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847578+0000", 2024-09-14T09:07:23.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2809000000000001e-05 2024-09-14T09:07:23.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847586+0000", 2024-09-14T09:07:23.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7279999999999995e-06 2024-09-14T09:07:23.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861423+0000", 2024-09-14T09:07:23.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.01383737 2024-09-14T09:07:23.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861433+0000", 2024-09-14T09:07:23.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0299999999999999e-06 2024-09-14T09:07:23.156 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.156 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.156 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.156 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105473 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.156 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857666+0000", 2024-09-14T09:07:23.156 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33216237399999998, 2024-09-14T09:07:23.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34589518499999999, 2024-09-14T09:07:23.156 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.156 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.156 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.157 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.157 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.157 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.157 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105473 2024-09-14T09:07:23.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.157 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857666+0000", 2024-09-14T09:07:23.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857666+0000", 2024-09-14T09:07:23.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857667+0000", 2024-09-14T09:07:23.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1899999999999998e-07 2024-09-14T09:07:23.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857669+0000", 2024-09-14T09:07:23.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9439999999999999e-06 2024-09-14T09:07:23.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857674+0000", 2024-09-14T09:07:23.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1399999999999999e-06 2024-09-14T09:07:23.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857677+0000", 2024-09-14T09:07:23.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8020000000000002e-06 2024-09-14T09:07:23.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862840+0000", 2024-09-14T09:07:23.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051628400000000001 2024-09-14T09:07:23.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862844+0000", 2024-09-14T09:07:23.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1389999999999997e-06 2024-09-14T09:07:23.160 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.160 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.160 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.160 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105537 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.160 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860503+0000", 2024-09-14T09:07:23.160 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32932551399999999, 2024-09-14T09:07:23.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.343121868, 2024-09-14T09:07:23.161 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.161 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.161 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.161 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.161 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.161 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.161 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105537 2024-09-14T09:07:23.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.161 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860503+0000", 2024-09-14T09:07:23.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860503+0000", 2024-09-14T09:07:23.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860504+0000", 2024-09-14T09:07:23.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9299999999999998e-07 2024-09-14T09:07:23.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860506+0000", 2024-09-14T09:07:23.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0669999999999999e-06 2024-09-14T09:07:23.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860512+0000", 2024-09-14T09:07:23.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3489999999999997e-06 2024-09-14T09:07:23.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860517+0000", 2024-09-14T09:07:23.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2070000000000003e-06 2024-09-14T09:07:23.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864753+0000", 2024-09-14T09:07:23.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042355459999999998 2024-09-14T09:07:23.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864759+0000", 2024-09-14T09:07:23.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.443e-06 2024-09-14T09:07:23.164 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.164 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.164 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.165 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105600 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.165 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.863584+0000", 2024-09-14T09:07:23.165 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32624466099999999, 2024-09-14T09:07:23.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34010196300000001, 2024-09-14T09:07:23.165 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.165 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.165 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.165 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.165 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.165 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.165 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105600 2024-09-14T09:07:23.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.166 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863584+0000", 2024-09-14T09:07:23.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863584+0000", 2024-09-14T09:07:23.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863585+0000", 2024-09-14T09:07:23.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.077e-06 2024-09-14T09:07:23.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863602+0000", 2024-09-14T09:07:23.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7469000000000002e-05 2024-09-14T09:07:23.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863617+0000", 2024-09-14T09:07:23.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4717e-05 2024-09-14T09:07:23.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863761+0000", 2024-09-14T09:07:23.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000144091 2024-09-14T09:07:23.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865782+0000", 2024-09-14T09:07:23.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020212369999999999 2024-09-14T09:07:23.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865789+0000", 2024-09-14T09:07:23.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3099999999999997e-06 2024-09-14T09:07:23.169 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.169 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.169 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.169 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105665 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.169 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:07:23.169 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32222874400000001, 2024-09-14T09:07:23.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33615254500000002, 2024-09-14T09:07:23.169 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.169 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.169 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.170 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.170 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.170 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.170 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105665 2024-09-14T09:07:23.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.170 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:07:23.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:07:23.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867601+0000", 2024-09-14T09:07:23.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.569999999999999e-07 2024-09-14T09:07:23.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867605+0000", 2024-09-14T09:07:23.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8589999999999998e-06 2024-09-14T09:07:23.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867617+0000", 2024-09-14T09:07:23.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2833000000000001e-05 2024-09-14T09:07:23.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867627+0000", 2024-09-14T09:07:23.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5219999999999996e-06 2024-09-14T09:07:23.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867728+0000", 2024-09-14T09:07:23.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010057500000000001 2024-09-14T09:07:23.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867740+0000", 2024-09-14T09:07:23.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2744e-05 2024-09-14T09:07:23.173 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.173 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.173 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.173 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105729 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.173 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.869198+0000", 2024-09-14T09:07:23.173 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32063060300000001, 2024-09-14T09:07:23.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33461774100000002, 2024-09-14T09:07:23.174 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.174 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.174 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.174 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105729 2024-09-14T09:07:23.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.174 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869198+0000", 2024-09-14T09:07:23.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869198+0000", 2024-09-14T09:07:23.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869198+0000", 2024-09-14T09:07:23.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0399999999999996e-07 2024-09-14T09:07:23.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869206+0000", 2024-09-14T09:07:23.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7239999999999992e-06 2024-09-14T09:07:23.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869212+0000", 2024-09-14T09:07:23.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2779999999999999e-06 2024-09-14T09:07:23.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869221+0000", 2024-09-14T09:07:23.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4819999999999994e-06 2024-09-14T09:07:23.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869568+0000", 2024-09-14T09:07:23.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034684400000000002 2024-09-14T09:07:23.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869575+0000", 2024-09-14T09:07:23.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7789999999999998e-06 2024-09-14T09:07:23.177 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.177 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.177 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.178 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105283 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.178 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841396+0000", 2024-09-14T09:07:23.178 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34843232400000002, 2024-09-14T09:07:23.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.36248016599999999, 2024-09-14T09:07:23.178 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.178 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.178 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.178 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.178 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.178 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.178 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105283 2024-09-14T09:07:23.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.178 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841396+0000", 2024-09-14T09:07:23.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841396+0000", 2024-09-14T09:07:23.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841397+0000", 2024-09-14T09:07:23.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0399999999999995e-07 2024-09-14T09:07:23.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841400+0000", 2024-09-14T09:07:23.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3179999999999998e-06 2024-09-14T09:07:23.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841412+0000", 2024-09-14T09:07:23.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1780000000000001e-05 2024-09-14T09:07:23.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841422+0000", 2024-09-14T09:07:23.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0186e-05 2024-09-14T09:07:23.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841721+0000", 2024-09-14T09:07:23.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029863999999999997 2024-09-14T09:07:23.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841736+0000", 2024-09-14T09:07:23.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4729000000000001e-05 2024-09-14T09:07:23.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857553+0000", 2024-09-14T09:07:23.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015817656999999999 2024-09-14T09:07:23.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857562+0000", 2024-09-14T09:07:23.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9299999999999992e-06 2024-09-14T09:07:23.182 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.182 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.182 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.182 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105347 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.182 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.844613+0000", 2024-09-14T09:07:23.182 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34521542300000002, 2024-09-14T09:07:23.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35933748900000001, 2024-09-14T09:07:23.182 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.182 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.183 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.183 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.183 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.183 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.183 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105347 2024-09-14T09:07:23.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.183 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844613+0000", 2024-09-14T09:07:23.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844613+0000", 2024-09-14T09:07:23.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844614+0000", 2024-09-14T09:07:23.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3379999999999999e-06 2024-09-14T09:07:23.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844656+0000", 2024-09-14T09:07:23.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1600999999999997e-05 2024-09-14T09:07:23.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844685+0000", 2024-09-14T09:07:23.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8935999999999999e-05 2024-09-14T09:07:23.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844698+0000", 2024-09-14T09:07:23.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2628e-05 2024-09-14T09:07:23.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844782+0000", 2024-09-14T09:07:23.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4606999999999998e-05 2024-09-14T09:07:23.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844791+0000", 2024-09-14T09:07:23.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8519999999999993e-06 2024-09-14T09:07:23.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860386+0000", 2024-09-14T09:07:23.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015594812 2024-09-14T09:07:23.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860391+0000", 2024-09-14T09:07:23.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9259999999999999e-06 2024-09-14T09:07:23.186 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.187 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.187 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.187 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105411 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.187 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.847532+0000", 2024-09-14T09:07:23.187 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34229696100000001, 2024-09-14T09:07:23.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35649293300000001, 2024-09-14T09:07:23.187 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.187 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.187 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.187 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.187 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.187 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.187 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105411 2024-09-14T09:07:23.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.188 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847532+0000", 2024-09-14T09:07:23.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847532+0000", 2024-09-14T09:07:23.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847532+0000", 2024-09-14T09:07:23.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9199999999999999e-07 2024-09-14T09:07:23.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847536+0000", 2024-09-14T09:07:23.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4740000000000001e-06 2024-09-14T09:07:23.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847547+0000", 2024-09-14T09:07:23.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.111e-05 2024-09-14T09:07:23.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847556+0000", 2024-09-14T09:07:23.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9479999999999997e-06 2024-09-14T09:07:23.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847632+0000", 2024-09-14T09:07:23.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6304000000000007e-05 2024-09-14T09:07:23.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847640+0000", 2024-09-14T09:07:23.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4919999999999996e-06 2024-09-14T09:07:23.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861470+0000", 2024-09-14T09:07:23.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013830176 2024-09-14T09:07:23.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861479+0000", 2024-09-14T09:07:23.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9889999999999998e-06 2024-09-14T09:07:23.191 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.191 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.191 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.191 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105475 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.191 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857683+0000", 2024-09-14T09:07:23.192 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33214583800000003, 2024-09-14T09:07:23.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.346413939, 2024-09-14T09:07:23.192 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.192 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.192 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.192 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.192 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.192 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.192 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105475 2024-09-14T09:07:23.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.192 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857683+0000", 2024-09-14T09:07:23.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857683+0000", 2024-09-14T09:07:23.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857683+0000", 2024-09-14T09:07:23.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7899999999999999e-07 2024-09-14T09:07:23.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857685+0000", 2024-09-14T09:07:23.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.564e-06 2024-09-14T09:07:23.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857693+0000", 2024-09-14T09:07:23.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5499999999999995e-06 2024-09-14T09:07:23.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857701+0000", 2024-09-14T09:07:23.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7209999999999994e-06 2024-09-14T09:07:23.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862862+0000", 2024-09-14T09:07:23.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051615480000000002 2024-09-14T09:07:23.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862867+0000", 2024-09-14T09:07:23.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2139999999999998e-06 2024-09-14T09:07:23.195 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.195 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.195 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.195 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105539 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.196 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860530+0000", 2024-09-14T09:07:23.196 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.329298327, 2024-09-14T09:07:23.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34364832899999997, 2024-09-14T09:07:23.196 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.196 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.196 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.196 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.196 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.196 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.196 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105539 2024-09-14T09:07:23.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.196 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860530+0000", 2024-09-14T09:07:23.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860530+0000", 2024-09-14T09:07:23.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860531+0000", 2024-09-14T09:07:23.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.58e-07 2024-09-14T09:07:23.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860540+0000", 2024-09-14T09:07:23.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5389999999999996e-06 2024-09-14T09:07:23.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860548+0000", 2024-09-14T09:07:23.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7079999999999997e-06 2024-09-14T09:07:23.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860554+0000", 2024-09-14T09:07:23.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.161e-06 2024-09-14T09:07:23.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864785+0000", 2024-09-14T09:07:23.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004231042 2024-09-14T09:07:23.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864791+0000", 2024-09-14T09:07:23.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.409e-06 2024-09-14T09:07:23.199 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.199 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.199 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.200 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105603 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.200 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.863784+0000", 2024-09-14T09:07:23.200 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32604503200000001, 2024-09-14T09:07:23.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34046547599999999, 2024-09-14T09:07:23.200 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.200 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.200 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.200 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.200 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.200 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.200 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105603 2024-09-14T09:07:23.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.200 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863784+0000", 2024-09-14T09:07:23.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863784+0000", 2024-09-14T09:07:23.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863785+0000", 2024-09-14T09:07:23.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.322e-06 2024-09-14T09:07:23.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863791+0000", 2024-09-14T09:07:23.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0650000000000004e-06 2024-09-14T09:07:23.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863916+0000", 2024-09-14T09:07:23.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012504899999999999 2024-09-14T09:07:23.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863928+0000", 2024-09-14T09:07:23.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2e-05 2024-09-14T09:07:23.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865814+0000", 2024-09-14T09:07:23.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001886425 2024-09-14T09:07:23.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865821+0000", 2024-09-14T09:07:23.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2430000000000002e-06 2024-09-14T09:07:23.203 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.203 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.204 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.204 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105666 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.204 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867659+0000", 2024-09-14T09:07:23.204 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32216977800000002, 2024-09-14T09:07:23.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33665586800000002, 2024-09-14T09:07:23.204 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.204 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.204 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.204 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.204 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.204 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.205 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105666 2024-09-14T09:07:23.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.205 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867659+0000", 2024-09-14T09:07:23.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867659+0000", 2024-09-14T09:07:23.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867660+0000", 2024-09-14T09:07:23.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4600000000000003e-07 2024-09-14T09:07:23.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867664+0000", 2024-09-14T09:07:23.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9319999999999998e-06 2024-09-14T09:07:23.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867676+0000", 2024-09-14T09:07:23.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2656e-05 2024-09-14T09:07:23.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867686+0000", 2024-09-14T09:07:23.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8640000000000008e-06 2024-09-14T09:07:23.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867789+0000", 2024-09-14T09:07:23.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000102617 2024-09-14T09:07:23.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867802+0000", 2024-09-14T09:07:23.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3219e-05 2024-09-14T09:07:23.208 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.208 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.208 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.208 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105731 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.208 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.869250+0000", 2024-09-14T09:07:23.208 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32057896800000002, 2024-09-14T09:07:23.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33512611599999997, 2024-09-14T09:07:23.208 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.208 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.208 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.208 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.208 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.209 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.209 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105731 2024-09-14T09:07:23.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.209 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869250+0000", 2024-09-14T09:07:23.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869250+0000", 2024-09-14T09:07:23.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869250+0000", 2024-09-14T09:07:23.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4600000000000004e-07 2024-09-14T09:07:23.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869254+0000", 2024-09-14T09:07:23.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5540000000000002e-06 2024-09-14T09:07:23.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869265+0000", 2024-09-14T09:07:23.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1248e-05 2024-09-14T09:07:23.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869275+0000", 2024-09-14T09:07:23.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6379999999999999e-06 2024-09-14T09:07:23.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869601+0000", 2024-09-14T09:07:23.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032609000000000002 2024-09-14T09:07:23.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869607+0000", 2024-09-14T09:07:23.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4640000000000003e-06 2024-09-14T09:07:23.212 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.212 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.212 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.212 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105285 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.212 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841452+0000", 2024-09-14T09:07:23.212 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34837692300000001, 2024-09-14T09:07:23.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.36336796700000001, 2024-09-14T09:07:23.212 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.212 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.212 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.212 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.213 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.213 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.213 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105285 2024-09-14T09:07:23.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.213 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841452+0000", 2024-09-14T09:07:23.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841452+0000", 2024-09-14T09:07:23.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841452+0000", 2024-09-14T09:07:23.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5799999999999998e-07 2024-09-14T09:07:23.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841456+0000", 2024-09-14T09:07:23.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2440000000000002e-06 2024-09-14T09:07:23.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841467+0000", 2024-09-14T09:07:23.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1856e-05 2024-09-14T09:07:23.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841476+0000", 2024-09-14T09:07:23.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7960000000000001e-06 2024-09-14T09:07:23.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841789+0000", 2024-09-14T09:07:23.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031244200000000001 2024-09-14T09:07:23.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841794+0000", 2024-09-14T09:07:23.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5829999999999997e-06 2024-09-14T09:07:23.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857615+0000", 2024-09-14T09:07:23.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015821061000000001 2024-09-14T09:07:23.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857627+0000", 2024-09-14T09:07:23.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1514e-05 2024-09-14T09:07:23.217 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.217 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.217 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.217 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105349 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.217 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.844812+0000", 2024-09-14T09:07:23.217 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34501626699999999, 2024-09-14T09:07:23.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.36008529099999997, 2024-09-14T09:07:23.217 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.217 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.217 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.217 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.217 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105349 2024-09-14T09:07:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844812+0000", 2024-09-14T09:07:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844812+0000", 2024-09-14T09:07:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844813+0000", 2024-09-14T09:07:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.102e-06 2024-09-14T09:07:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844818+0000", 2024-09-14T09:07:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8160000000000001e-06 2024-09-14T09:07:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844831+0000", 2024-09-14T09:07:23.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2320000000000001e-05 2024-09-14T09:07:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844840+0000", 2024-09-14T09:07:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6479999999999998e-06 2024-09-14T09:07:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844915+0000", 2024-09-14T09:07:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5283000000000004e-05 2024-09-14T09:07:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844924+0000", 2024-09-14T09:07:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8040000000000007e-06 2024-09-14T09:07:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860411+0000", 2024-09-14T09:07:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015486272000000001 2024-09-14T09:07:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860415+0000", 2024-09-14T09:07:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.34e-06 2024-09-14T09:07:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.221 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105413 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.847568+0000", 2024-09-14T09:07:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34226079500000001, 2024-09-14T09:07:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35740234599999998, 2024-09-14T09:07:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105413 2024-09-14T09:07:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.222 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847568+0000", 2024-09-14T09:07:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847568+0000", 2024-09-14T09:07:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847569+0000", 2024-09-14T09:07:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1999999999999998e-07 2024-09-14T09:07:23.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847572+0000", 2024-09-14T09:07:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3079999999999999e-06 2024-09-14T09:07:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847583+0000", 2024-09-14T09:07:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0674e-05 2024-09-14T09:07:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.224 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847689+0000", 2024-09-14T09:07:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106419 2024-09-14T09:07:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847772+0000", 2024-09-14T09:07:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3133000000000005e-05 2024-09-14T09:07:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847780+0000", 2024-09-14T09:07:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5900000000000002e-06 2024-09-14T09:07:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861516+0000", 2024-09-14T09:07:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013736613 2024-09-14T09:07:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861525+0000", 2024-09-14T09:07:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8100000000000004e-06 2024-09-14T09:07:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105477 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857718+0000", 2024-09-14T09:07:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33211011299999998, 2024-09-14T09:07:23.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.347326093, 2024-09-14T09:07:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105477 2024-09-14T09:07:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857718+0000", 2024-09-14T09:07:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857718+0000", 2024-09-14T09:07:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857719+0000", 2024-09-14T09:07:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7000000000000005e-07 2024-09-14T09:07:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857722+0000", 2024-09-14T09:07:23.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9299999999999999e-06 2024-09-14T09:07:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857731+0000", 2024-09-14T09:07:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5930000000000006e-06 2024-09-14T09:07:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857739+0000", 2024-09-14T09:07:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8400000000000001e-06 2024-09-14T09:07:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.230 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862884+0000", 2024-09-14T09:07:23.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051445800000000002 2024-09-14T09:07:23.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.230 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.230 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862888+0000", 2024-09-14T09:07:23.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2010000000000001e-06 2024-09-14T09:07:23.230 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.230 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.230 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.230 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105541 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.230 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860562+0000", 2024-09-14T09:07:23.231 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32926699799999998, 2024-09-14T09:07:23.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34454648599999999, 2024-09-14T09:07:23.231 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.231 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.231 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.231 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.231 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.231 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.231 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105541 2024-09-14T09:07:23.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.231 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860562+0000", 2024-09-14T09:07:23.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860562+0000", 2024-09-14T09:07:23.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860562+0000", 2024-09-14T09:07:23.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5199999999999997e-07 2024-09-14T09:07:23.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860564+0000", 2024-09-14T09:07:23.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0839999999999999e-06 2024-09-14T09:07:23.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860571+0000", 2024-09-14T09:07:23.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4620000000000001e-06 2024-09-14T09:07:23.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860576+0000", 2024-09-14T09:07:23.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0119999999999996e-06 2024-09-14T09:07:23.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864818+0000", 2024-09-14T09:07:23.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042419809999999997 2024-09-14T09:07:23.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864824+0000", 2024-09-14T09:07:23.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4239999999999998e-06 2024-09-14T09:07:23.234 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.235 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.235 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.235 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105605 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.235 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.863949+0000", 2024-09-14T09:07:23.235 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.325879796, 2024-09-14T09:07:23.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.341220001, 2024-09-14T09:07:23.235 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.235 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.235 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.235 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.235 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.236 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.236 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105605 2024-09-14T09:07:23.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.236 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863949+0000", 2024-09-14T09:07:23.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863949+0000", 2024-09-14T09:07:23.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863950+0000", 2024-09-14T09:07:23.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2160000000000001e-06 2024-09-14T09:07:23.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863970+0000", 2024-09-14T09:07:23.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9649000000000001e-05 2024-09-14T09:07:23.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863984+0000", 2024-09-14T09:07:23.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4450000000000001e-05 2024-09-14T09:07:23.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863999+0000", 2024-09-14T09:07:23.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4795e-05 2024-09-14T09:07:23.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865847+0000", 2024-09-14T09:07:23.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018476530000000001 2024-09-14T09:07:23.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865853+0000", 2024-09-14T09:07:23.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2779999999999999e-06 2024-09-14T09:07:23.239 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.239 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.239 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.239 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105670 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.239 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867717+0000", 2024-09-14T09:07:23.239 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32211140199999999, 2024-09-14T09:07:23.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33751581800000002, 2024-09-14T09:07:23.239 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.239 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.240 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.240 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.240 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.240 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.240 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105670 2024-09-14T09:07:23.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.240 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867717+0000", 2024-09-14T09:07:23.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867717+0000", 2024-09-14T09:07:23.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867718+0000", 2024-09-14T09:07:23.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4600000000000003e-07 2024-09-14T09:07:23.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867722+0000", 2024-09-14T09:07:23.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0659999999999997e-06 2024-09-14T09:07:23.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867735+0000", 2024-09-14T09:07:23.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2877999999999999e-05 2024-09-14T09:07:23.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867746+0000", 2024-09-14T09:07:23.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1178e-05 2024-09-14T09:07:23.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867847+0000", 2024-09-14T09:07:23.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101091 2024-09-14T09:07:23.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867860+0000", 2024-09-14T09:07:23.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2906e-05 2024-09-14T09:07:23.243 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.243 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.243 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.243 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105733 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.243 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.869315+0000", 2024-09-14T09:07:23.243 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32051377800000003, 2024-09-14T09:07:23.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33597928900000001, 2024-09-14T09:07:23.244 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.244 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.244 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.244 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.244 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.244 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.244 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105733 2024-09-14T09:07:23.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.244 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869315+0000", 2024-09-14T09:07:23.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869315+0000", 2024-09-14T09:07:23.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869315+0000", 2024-09-14T09:07:23.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4799999999999998e-07 2024-09-14T09:07:23.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869318+0000", 2024-09-14T09:07:23.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2919999999999998e-06 2024-09-14T09:07:23.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869325+0000", 2024-09-14T09:07:23.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9360000000000002e-06 2024-09-14T09:07:23.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869330+0000", 2024-09-14T09:07:23.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.972e-06 2024-09-14T09:07:23.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869633+0000", 2024-09-14T09:07:23.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00030392599999999999 2024-09-14T09:07:23.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869639+0000", 2024-09-14T09:07:23.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0449999999999997e-06 2024-09-14T09:07:23.247 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.247 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.247 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.247 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105287 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.248 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841516+0000", 2024-09-14T09:07:23.248 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34831266500000002, 2024-09-14T09:07:23.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.36384237699999999, 2024-09-14T09:07:23.248 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.248 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.248 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.248 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.248 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.248 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.248 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105287 2024-09-14T09:07:23.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.248 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841516+0000", 2024-09-14T09:07:23.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841516+0000", 2024-09-14T09:07:23.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841517+0000", 2024-09-14T09:07:23.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4000000000000001e-07 2024-09-14T09:07:23.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841520+0000", 2024-09-14T09:07:23.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4860000000000002e-06 2024-09-14T09:07:23.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841531+0000", 2024-09-14T09:07:23.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1253999999999999e-05 2024-09-14T09:07:23.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841540+0000", 2024-09-14T09:07:23.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6759999999999996e-06 2024-09-14T09:07:23.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841853+0000", 2024-09-14T09:07:23.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031293299999999999 2024-09-14T09:07:23.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841859+0000", 2024-09-14T09:07:23.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.46e-06 2024-09-14T09:07:23.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857674+0000", 2024-09-14T09:07:23.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015814910000000001 2024-09-14T09:07:23.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857683+0000", 2024-09-14T09:07:23.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2400000000000007e-06 2024-09-14T09:07:23.252 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.252 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.252 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.252 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105351 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.252 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.844943+0000", 2024-09-14T09:07:23.253 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34488513199999998, 2024-09-14T09:07:23.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.36048929499999999, 2024-09-14T09:07:23.253 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.253 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.253 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.253 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.253 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.253 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.253 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105351 2024-09-14T09:07:23.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.253 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844943+0000", 2024-09-14T09:07:23.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844943+0000", 2024-09-14T09:07:23.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844944+0000", 2024-09-14T09:07:23.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.02e-06 2024-09-14T09:07:23.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844948+0000", 2024-09-14T09:07:23.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0439999999999997e-06 2024-09-14T09:07:23.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844960+0000", 2024-09-14T09:07:23.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1174e-05 2024-09-14T09:07:23.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.844969+0000", 2024-09-14T09:07:23.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1460000000000002e-06 2024-09-14T09:07:23.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845040+0000", 2024-09-14T09:07:23.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1674999999999994e-05 2024-09-14T09:07:23.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845049+0000", 2024-09-14T09:07:23.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6079999999999996e-06 2024-09-14T09:07:23.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860460+0000", 2024-09-14T09:07:23.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015411223999999999 2024-09-14T09:07:23.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860465+0000", 2024-09-14T09:07:23.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8779999999999997e-06 2024-09-14T09:07:23.257 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.257 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.257 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.257 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105415 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.258 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.847722+0000", 2024-09-14T09:07:23.258 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34210653800000002, 2024-09-14T09:07:23.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35778232500000001, 2024-09-14T09:07:23.260 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.260 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.260 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.260 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.260 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.260 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.260 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105415 2024-09-14T09:07:23.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.260 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847722+0000", 2024-09-14T09:07:23.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847722+0000", 2024-09-14T09:07:23.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847723+0000", 2024-09-14T09:07:23.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2600000000000001e-07 2024-09-14T09:07:23.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847728+0000", 2024-09-14T09:07:23.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0100000000000003e-06 2024-09-14T09:07:23.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847740+0000", 2024-09-14T09:07:23.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2442e-05 2024-09-14T09:07:23.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847750+0000", 2024-09-14T09:07:23.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.73e-06 2024-09-14T09:07:23.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847816+0000", 2024-09-14T09:07:23.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6098000000000003e-05 2024-09-14T09:07:23.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847823+0000", 2024-09-14T09:07:23.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.63e-06 2024-09-14T09:07:23.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861561+0000", 2024-09-14T09:07:23.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013738174000000001 2024-09-14T09:07:23.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861570+0000", 2024-09-14T09:07:23.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9600000000000006e-06 2024-09-14T09:07:23.264 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.264 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.264 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.265 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105479 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.265 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857760+0000", 2024-09-14T09:07:23.265 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33206833899999999, 2024-09-14T09:07:23.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34781838799999998, 2024-09-14T09:07:23.265 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.265 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.265 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.265 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.265 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.265 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.265 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105479 2024-09-14T09:07:23.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.265 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857760+0000", 2024-09-14T09:07:23.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857760+0000", 2024-09-14T09:07:23.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857761+0000", 2024-09-14T09:07:23.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0900000000000002e-07 2024-09-14T09:07:23.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857763+0000", 2024-09-14T09:07:23.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.906e-06 2024-09-14T09:07:23.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857768+0000", 2024-09-14T09:07:23.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4070000000000001e-06 2024-09-14T09:07:23.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857772+0000", 2024-09-14T09:07:23.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1450000000000001e-06 2024-09-14T09:07:23.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862922+0000", 2024-09-14T09:07:23.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051497909999999999 2024-09-14T09:07:23.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862930+0000", 2024-09-14T09:07:23.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3760000000000008e-06 2024-09-14T09:07:23.268 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.268 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.268 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.269 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105542 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.269 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860596+0000", 2024-09-14T09:07:23.269 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.329232419, 2024-09-14T09:07:23.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34504633800000001, 2024-09-14T09:07:23.269 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.269 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.269 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.269 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.269 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.269 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.269 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105542 2024-09-14T09:07:23.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.270 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860596+0000", 2024-09-14T09:07:23.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860596+0000", 2024-09-14T09:07:23.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860597+0000", 2024-09-14T09:07:23.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7100000000000002e-07 2024-09-14T09:07:23.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860599+0000", 2024-09-14T09:07:23.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4490000000000002e-06 2024-09-14T09:07:23.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860608+0000", 2024-09-14T09:07:23.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7010000000000002e-06 2024-09-14T09:07:23.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860614+0000", 2024-09-14T09:07:23.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8370000000000002e-06 2024-09-14T09:07:23.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864850+0000", 2024-09-14T09:07:23.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042360710000000001 2024-09-14T09:07:23.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864856+0000", 2024-09-14T09:07:23.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.297e-06 2024-09-14T09:07:23.273 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.273 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.273 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.273 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105607 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.273 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.864058+0000", 2024-09-14T09:07:23.273 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32577041200000001, 2024-09-14T09:07:23.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34164510199999998, 2024-09-14T09:07:23.273 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.273 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.273 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.274 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.274 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.274 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.274 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105607 2024-09-14T09:07:23.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.274 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864058+0000", 2024-09-14T09:07:23.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864058+0000", 2024-09-14T09:07:23.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864059+0000", 2024-09-14T09:07:23.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2300000000000001e-06 2024-09-14T09:07:23.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864066+0000", 2024-09-14T09:07:23.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9099999999999999e-06 2024-09-14T09:07:23.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864081+0000", 2024-09-14T09:07:23.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5128e-05 2024-09-14T09:07:23.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864093+0000", 2024-09-14T09:07:23.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1644000000000001e-05 2024-09-14T09:07:23.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865878+0000", 2024-09-14T09:07:23.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001785114 2024-09-14T09:07:23.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865884+0000", 2024-09-14T09:07:23.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2500000000000003e-06 2024-09-14T09:07:23.277 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.277 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.277 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.277 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105671 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.277 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867779+0000", 2024-09-14T09:07:23.278 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32204981599999999, 2024-09-14T09:07:23.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.337988012, 2024-09-14T09:07:23.278 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.278 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.278 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.278 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.278 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.278 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.278 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105671 2024-09-14T09:07:23.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.278 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867779+0000", 2024-09-14T09:07:23.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867779+0000", 2024-09-14T09:07:23.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867780+0000", 2024-09-14T09:07:23.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8599999999999996e-07 2024-09-14T09:07:23.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867784+0000", 2024-09-14T09:07:23.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.907e-06 2024-09-14T09:07:23.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867797+0000", 2024-09-14T09:07:23.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.294e-05 2024-09-14T09:07:23.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867806+0000", 2024-09-14T09:07:23.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5449999999999992e-06 2024-09-14T09:07:23.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867908+0000", 2024-09-14T09:07:23.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101927 2024-09-14T09:07:23.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867921+0000", 2024-09-14T09:07:23.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2989e-05 2024-09-14T09:07:23.281 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.282 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.282 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.282 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105734 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.282 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.869352+0000", 2024-09-14T09:07:23.282 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32047604800000001, 2024-09-14T09:07:23.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33647508799999998, 2024-09-14T09:07:23.282 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.282 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.282 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.282 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.282 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.282 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.283 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105734 2024-09-14T09:07:23.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.283 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869352+0000", 2024-09-14T09:07:23.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869352+0000", 2024-09-14T09:07:23.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869353+0000", 2024-09-14T09:07:23.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7400000000000002e-07 2024-09-14T09:07:23.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869357+0000", 2024-09-14T09:07:23.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5580000000000001e-06 2024-09-14T09:07:23.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869368+0000", 2024-09-14T09:07:23.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0947e-05 2024-09-14T09:07:23.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869377+0000", 2024-09-14T09:07:23.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-09-14T09:07:23.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869666+0000", 2024-09-14T09:07:23.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028890099999999999 2024-09-14T09:07:23.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869672+0000", 2024-09-14T09:07:23.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3450000000000002e-06 2024-09-14T09:07:23.286 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.286 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.286 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.286 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105289 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.286 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841570+0000", 2024-09-14T09:07:23.286 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34825860600000003, 2024-09-14T09:07:23.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.36432136599999998, 2024-09-14T09:07:23.287 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.287 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.287 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.287 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.287 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.287 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.287 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105289 2024-09-14T09:07:23.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.287 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841570+0000", 2024-09-14T09:07:23.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841570+0000", 2024-09-14T09:07:23.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841570+0000", 2024-09-14T09:07:23.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9599999999999999e-07 2024-09-14T09:07:23.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841573+0000", 2024-09-14T09:07:23.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.17e-06 2024-09-14T09:07:23.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841579+0000", 2024-09-14T09:07:23.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0340000000000002e-06 2024-09-14T09:07:23.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841583+0000", 2024-09-14T09:07:23.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6840000000000004e-06 2024-09-14T09:07:23.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841905+0000", 2024-09-14T09:07:23.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032152999999999999 2024-09-14T09:07:23.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841910+0000", 2024-09-14T09:07:23.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5720000000000002e-06 2024-09-14T09:07:23.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857727+0000", 2024-09-14T09:07:23.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015817055999999999 2024-09-14T09:07:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857738+0000", 2024-09-14T09:07:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0977e-05 2024-09-14T09:07:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105353 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.845068+0000", 2024-09-14T09:07:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34476058500000001, 2024-09-14T09:07:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.36089787499999998, 2024-09-14T09:07:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105353 2024-09-14T09:07:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845068+0000", 2024-09-14T09:07:23.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845068+0000", 2024-09-14T09:07:23.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845069+0000", 2024-09-14T09:07:23.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0139999999999999e-06 2024-09-14T09:07:23.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845073+0000", 2024-09-14T09:07:23.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.794e-06 2024-09-14T09:07:23.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845085+0000", 2024-09-14T09:07:23.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1878e-05 2024-09-14T09:07:23.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845093+0000", 2024-09-14T09:07:23.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.85e-06 2024-09-14T09:07:23.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845172+0000", 2024-09-14T09:07:23.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8045000000000002e-05 2024-09-14T09:07:23.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845180+0000", 2024-09-14T09:07:23.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4700000000000002e-06 2024-09-14T09:07:23.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860485+0000", 2024-09-14T09:07:23.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015305098 2024-09-14T09:07:23.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860489+0000", 2024-09-14T09:07:23.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-09-14T09:07:23.296 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.296 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.296 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.297 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105416 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.297 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.847765+0000", 2024-09-14T09:07:23.297 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34206317800000002, 2024-09-14T09:07:23.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35827228700000002, 2024-09-14T09:07:23.297 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.297 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.297 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.297 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.297 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.297 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.297 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105416 2024-09-14T09:07:23.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.297 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847765+0000", 2024-09-14T09:07:23.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847765+0000", 2024-09-14T09:07:23.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847766+0000", 2024-09-14T09:07:23.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.78e-07 2024-09-14T09:07:23.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:07:23.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4869999999999998e-06 2024-09-14T09:07:23.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847781+0000", 2024-09-14T09:07:23.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1626e-05 2024-09-14T09:07:23.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847793+0000", 2024-09-14T09:07:23.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1464000000000001e-05 2024-09-14T09:07:23.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847855+0000", 2024-09-14T09:07:23.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2347999999999993e-05 2024-09-14T09:07:23.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847862+0000", 2024-09-14T09:07:23.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6270000000000002e-06 2024-09-14T09:07:23.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861610+0000", 2024-09-14T09:07:23.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013747943 2024-09-14T09:07:23.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861619+0000", 2024-09-14T09:07:23.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9849999999999995e-06 2024-09-14T09:07:23.301 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.301 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.301 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.301 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105481 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.301 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857778+0000", 2024-09-14T09:07:23.301 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33205078199999999, 2024-09-14T09:07:23.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.348334598, 2024-09-14T09:07:23.301 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.302 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.302 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.302 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.302 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.302 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.302 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105481 2024-09-14T09:07:23.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.302 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857778+0000", 2024-09-14T09:07:23.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857778+0000", 2024-09-14T09:07:23.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857778+0000", 2024-09-14T09:07:23.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8500000000000002e-07 2024-09-14T09:07:23.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857780+0000", 2024-09-14T09:07:23.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5179999999999999e-06 2024-09-14T09:07:23.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857784+0000", 2024-09-14T09:07:23.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4970000000000003e-06 2024-09-14T09:07:23.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857788+0000", 2024-09-14T09:07:23.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0380000000000001e-06 2024-09-14T09:07:23.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862967+0000", 2024-09-14T09:07:23.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051784530000000004 2024-09-14T09:07:23.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.862976+0000", 2024-09-14T09:07:23.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0739999999999999e-06 2024-09-14T09:07:23.305 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.305 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.305 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.305 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105546 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.305 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860623+0000", 2024-09-14T09:07:23.305 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32920601599999999, 2024-09-14T09:07:23.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34555434499999998, 2024-09-14T09:07:23.305 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.305 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.305 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.305 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.306 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.306 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.306 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105546 2024-09-14T09:07:23.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.306 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860623+0000", 2024-09-14T09:07:23.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860623+0000", 2024-09-14T09:07:23.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860623+0000", 2024-09-14T09:07:23.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2599999999999998e-07 2024-09-14T09:07:23.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860625+0000", 2024-09-14T09:07:23.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2680000000000001e-06 2024-09-14T09:07:23.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860632+0000", 2024-09-14T09:07:23.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2450000000000001e-06 2024-09-14T09:07:23.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860638+0000", 2024-09-14T09:07:23.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8019999999999997e-06 2024-09-14T09:07:23.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864882+0000", 2024-09-14T09:07:23.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004244027 2024-09-14T09:07:23.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864889+0000", 2024-09-14T09:07:23.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6579999999999996e-06 2024-09-14T09:07:23.308 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.309 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.309 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.309 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105608 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.309 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.864153+0000", 2024-09-14T09:07:23.309 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32567538200000001, 2024-09-14T09:07:23.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34208451000000001, 2024-09-14T09:07:23.309 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.309 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.309 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.309 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.309 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.310 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.310 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105608 2024-09-14T09:07:23.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.310 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864153+0000", 2024-09-14T09:07:23.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864153+0000", 2024-09-14T09:07:23.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864154+0000", 2024-09-14T09:07:23.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1400000000000001e-06 2024-09-14T09:07:23.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864169+0000", 2024-09-14T09:07:23.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4443999999999999e-05 2024-09-14T09:07:23.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864183+0000", 2024-09-14T09:07:23.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4548e-05 2024-09-14T09:07:23.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864194+0000", 2024-09-14T09:07:23.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1114999999999999e-05 2024-09-14T09:07:23.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865910+0000", 2024-09-14T09:07:23.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017155040000000001 2024-09-14T09:07:23.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865916+0000", 2024-09-14T09:07:23.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.28e-06 2024-09-14T09:07:23.313 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.313 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.313 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.313 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105673 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.313 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867837+0000", 2024-09-14T09:07:23.313 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.321991735, 2024-09-14T09:07:23.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33846482300000003, 2024-09-14T09:07:23.313 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.313 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.313 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.314 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.314 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.314 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.314 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105673 2024-09-14T09:07:23.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.314 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867837+0000", 2024-09-14T09:07:23.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867837+0000", 2024-09-14T09:07:23.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867838+0000", 2024-09-14T09:07:23.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5600000000000004e-07 2024-09-14T09:07:23.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867842+0000", 2024-09-14T09:07:23.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0110000000000002e-06 2024-09-14T09:07:23.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867855+0000", 2024-09-14T09:07:23.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2972e-05 2024-09-14T09:07:23.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867866+0000", 2024-09-14T09:07:23.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1257000000000001e-05 2024-09-14T09:07:23.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867970+0000", 2024-09-14T09:07:23.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103726 2024-09-14T09:07:23.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867983+0000", 2024-09-14T09:07:23.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3030000000000001e-05 2024-09-14T09:07:23.317 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.317 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.317 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.317 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105738 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.317 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.869388+0000", 2024-09-14T09:07:23.317 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32044021700000003, 2024-09-14T09:07:23.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.336974151, 2024-09-14T09:07:23.317 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.318 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.318 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.318 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.318 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.318 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.318 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105738 2024-09-14T09:07:23.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.318 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869388+0000", 2024-09-14T09:07:23.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869388+0000", 2024-09-14T09:07:23.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869389+0000", 2024-09-14T09:07:23.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2799999999999995e-07 2024-09-14T09:07:23.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869392+0000", 2024-09-14T09:07:23.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2780000000000002e-06 2024-09-14T09:07:23.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869403+0000", 2024-09-14T09:07:23.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0882e-05 2024-09-14T09:07:23.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869412+0000", 2024-09-14T09:07:23.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4999999999999999e-06 2024-09-14T09:07:23.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869698+0000", 2024-09-14T09:07:23.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028610400000000002 2024-09-14T09:07:23.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869704+0000", 2024-09-14T09:07:23.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6549999999999998e-06 2024-09-14T09:07:23.321 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.321 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.321 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.321 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105292 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.321 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841590+0000", 2024-09-14T09:07:23.321 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34823855999999997, 2024-09-14T09:07:23.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.36483623799999998, 2024-09-14T09:07:23.322 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.322 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.322 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.322 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.322 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.322 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.322 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105292 2024-09-14T09:07:23.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.322 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841590+0000", 2024-09-14T09:07:23.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841590+0000", 2024-09-14T09:07:23.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841590+0000", 2024-09-14T09:07:23.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8800000000000003e-07 2024-09-14T09:07:23.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841592+0000", 2024-09-14T09:07:23.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8899999999999999e-06 2024-09-14T09:07:23.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841598+0000", 2024-09-14T09:07:23.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2859999999999999e-06 2024-09-14T09:07:23.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841602+0000", 2024-09-14T09:07:23.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.42e-06 2024-09-14T09:07:23.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841974+0000", 2024-09-14T09:07:23.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037208599999999999 2024-09-14T09:07:23.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841981+0000", 2024-09-14T09:07:23.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1559999999999998e-06 2024-09-14T09:07:23.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857785+0000", 2024-09-14T09:07:23.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015803615999999999 2024-09-14T09:07:23.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857793+0000", 2024-09-14T09:07:23.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1699999999999997e-06 2024-09-14T09:07:23.326 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.326 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.326 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.327 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105355 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.327 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.845199+0000", 2024-09-14T09:07:23.327 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34462967999999999, 2024-09-14T09:07:23.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.361303971, 2024-09-14T09:07:23.327 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.327 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.327 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.327 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.327 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.327 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.327 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105355 2024-09-14T09:07:23.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.327 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845199+0000", 2024-09-14T09:07:23.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845199+0000", 2024-09-14T09:07:23.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845200+0000", 2024-09-14T09:07:23.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8599999999999996e-07 2024-09-14T09:07:23.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845204+0000", 2024-09-14T09:07:23.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9979999999999997e-06 2024-09-14T09:07:23.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845480+0000", 2024-09-14T09:07:23.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00027655600000000001 2024-09-14T09:07:23.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845493+0000", 2024-09-14T09:07:23.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2296000000000001e-05 2024-09-14T09:07:23.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845582+0000", 2024-09-14T09:07:23.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8882999999999996e-05 2024-09-14T09:07:23.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845588+0000", 2024-09-14T09:07:23.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4509999999999997e-06 2024-09-14T09:07:23.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860507+0000", 2024-09-14T09:07:23.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014919059 2024-09-14T09:07:23.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860511+0000", 2024-09-14T09:07:23.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2549999999999999e-06 2024-09-14T09:07:23.331 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.331 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.331 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.331 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105420 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.331 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.847807+0000", 2024-09-14T09:07:23.331 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34202169300000002, 2024-09-14T09:07:23.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35876801200000003, 2024-09-14T09:07:23.332 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.332 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.332 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.332 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.332 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.332 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.332 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105420 2024-09-14T09:07:23.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.332 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847807+0000", 2024-09-14T09:07:23.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847807+0000", 2024-09-14T09:07:23.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847808+0000", 2024-09-14T09:07:23.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8999999999999995e-07 2024-09-14T09:07:23.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847820+0000", 2024-09-14T09:07:23.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1884e-05 2024-09-14T09:07:23.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847832+0000", 2024-09-14T09:07:23.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2082000000000001e-05 2024-09-14T09:07:23.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847841+0000", 2024-09-14T09:07:23.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6719999999999999e-06 2024-09-14T09:07:23.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847894+0000", 2024-09-14T09:07:23.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2287000000000003e-05 2024-09-14T09:07:23.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847900+0000", 2024-09-14T09:07:23.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3409999999999999e-06 2024-09-14T09:07:23.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861657+0000", 2024-09-14T09:07:23.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013757238999999999 2024-09-14T09:07:23.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861667+0000", 2024-09-14T09:07:23.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3929999999999997e-06 2024-09-14T09:07:23.336 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.336 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.336 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.337 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105483 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.337 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857793+0000", 2024-09-14T09:07:23.337 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33203568700000002, 2024-09-14T09:07:23.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34885796299999999, 2024-09-14T09:07:23.337 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.337 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.337 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.337 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.337 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.337 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.337 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105483 2024-09-14T09:07:23.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.337 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857793+0000", 2024-09-14T09:07:23.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857793+0000", 2024-09-14T09:07:23.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857793+0000", 2024-09-14T09:07:23.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.77e-07 2024-09-14T09:07:23.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857795+0000", 2024-09-14T09:07:23.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5409999999999999e-06 2024-09-14T09:07:23.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857801+0000", 2024-09-14T09:07:23.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9359999999999996e-06 2024-09-14T09:07:23.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857808+0000", 2024-09-14T09:07:23.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9310000000000002e-06 2024-09-14T09:07:23.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863010+0000", 2024-09-14T09:07:23.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052025129999999998 2024-09-14T09:07:23.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863016+0000", 2024-09-14T09:07:23.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2600000000000002e-06 2024-09-14T09:07:23.340 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.340 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.340 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.341 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105547 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.341 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860646+0000", 2024-09-14T09:07:23.341 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32918220300000001, 2024-09-14T09:07:23.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34606869200000001, 2024-09-14T09:07:23.341 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.341 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.341 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.341 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.341 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.341 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.341 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105547 2024-09-14T09:07:23.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.341 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860646+0000", 2024-09-14T09:07:23.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860646+0000", 2024-09-14T09:07:23.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860647+0000", 2024-09-14T09:07:23.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8999999999999996e-07 2024-09-14T09:07:23.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860649+0000", 2024-09-14T09:07:23.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2139999999999999e-06 2024-09-14T09:07:23.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860656+0000", 2024-09-14T09:07:23.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8580000000000002e-06 2024-09-14T09:07:23.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860662+0000", 2024-09-14T09:07:23.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6130000000000003e-06 2024-09-14T09:07:23.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864915+0000", 2024-09-14T09:07:23.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042530900000000002 2024-09-14T09:07:23.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864921+0000", 2024-09-14T09:07:23.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2269999999999998e-06 2024-09-14T09:07:23.345 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.345 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.345 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.345 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105612 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.345 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.864227+0000", 2024-09-14T09:07:23.345 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32560126499999997, 2024-09-14T09:07:23.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34254827599999998, 2024-09-14T09:07:23.345 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.345 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.345 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.345 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.345 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.346 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.346 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105612 2024-09-14T09:07:23.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.346 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864227+0000", 2024-09-14T09:07:23.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864227+0000", 2024-09-14T09:07:23.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864228+0000", 2024-09-14T09:07:23.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.114e-06 2024-09-14T09:07:23.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864233+0000", 2024-09-14T09:07:23.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5299999999999998e-06 2024-09-14T09:07:23.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864247+0000", 2024-09-14T09:07:23.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4090000000000001e-05 2024-09-14T09:07:23.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864257+0000", 2024-09-14T09:07:23.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.043e-05 2024-09-14T09:07:23.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865942+0000", 2024-09-14T09:07:23.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001684589 2024-09-14T09:07:23.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865949+0000", 2024-09-14T09:07:23.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4999999999999996e-06 2024-09-14T09:07:23.349 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.349 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.349 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.349 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105674 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.349 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867898+0000", 2024-09-14T09:07:23.349 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.321930679, 2024-09-14T09:07:23.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33894184700000002, 2024-09-14T09:07:23.350 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.350 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.350 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.350 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.350 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.350 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.350 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105674 2024-09-14T09:07:23.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.350 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867898+0000", 2024-09-14T09:07:23.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867898+0000", 2024-09-14T09:07:23.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867899+0000", 2024-09-14T09:07:23.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.015e-06 2024-09-14T09:07:23.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867903+0000", 2024-09-14T09:07:23.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9219999999999999e-06 2024-09-14T09:07:23.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867916+0000", 2024-09-14T09:07:23.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2858e-05 2024-09-14T09:07:23.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867927+0000", 2024-09-14T09:07:23.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1231e-05 2024-09-14T09:07:23.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868028+0000", 2024-09-14T09:07:23.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101084 2024-09-14T09:07:23.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868041+0000", 2024-09-14T09:07:23.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3179e-05 2024-09-14T09:07:23.353 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.353 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.353 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.354 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105739 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.354 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.869423+0000", 2024-09-14T09:07:23.354 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32040596100000002, 2024-09-14T09:07:23.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33747871299999999, 2024-09-14T09:07:23.354 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.354 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.354 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.354 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.354 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.354 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.354 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105739 2024-09-14T09:07:23.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.354 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869423+0000", 2024-09-14T09:07:23.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869423+0000", 2024-09-14T09:07:23.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869423+0000", 2024-09-14T09:07:23.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3e-07 2024-09-14T09:07:23.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869427+0000", 2024-09-14T09:07:23.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4580000000000002e-06 2024-09-14T09:07:23.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869434+0000", 2024-09-14T09:07:23.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8240000000000002e-06 2024-09-14T09:07:23.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869438+0000", 2024-09-14T09:07:23.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7679999999999999e-06 2024-09-14T09:07:23.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869732+0000", 2024-09-14T09:07:23.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000293491 2024-09-14T09:07:23.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869738+0000", 2024-09-14T09:07:23.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5989999999999998e-06 2024-09-14T09:07:23.357 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.357 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.358 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.358 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105293 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.358 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841622+0000", 2024-09-14T09:07:23.358 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34820654400000001, 2024-09-14T09:07:23.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.36534319700000001, 2024-09-14T09:07:23.358 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.358 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.358 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.358 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.358 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.358 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.359 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105293 2024-09-14T09:07:23.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.359 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841622+0000", 2024-09-14T09:07:23.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841622+0000", 2024-09-14T09:07:23.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841623+0000", 2024-09-14T09:07:23.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0800000000000005e-07 2024-09-14T09:07:23.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841625+0000", 2024-09-14T09:07:23.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0020000000000001e-06 2024-09-14T09:07:23.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841631+0000", 2024-09-14T09:07:23.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9939999999999997e-06 2024-09-14T09:07:23.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841635+0000", 2024-09-14T09:07:23.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7899999999999999e-06 2024-09-14T09:07:23.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842024+0000", 2024-09-14T09:07:23.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038907200000000002 2024-09-14T09:07:23.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842030+0000", 2024-09-14T09:07:23.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4709999999999998e-06 2024-09-14T09:07:23.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857837+0000", 2024-09-14T09:07:23.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015807244000000002 2024-09-14T09:07:23.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857848+0000", 2024-09-14T09:07:23.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0900999999999999e-05 2024-09-14T09:07:23.362 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.362 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.363 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.363 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105356 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.363 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.845539+0000", 2024-09-14T09:07:23.363 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34428940099999999, 2024-09-14T09:07:23.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.36149977799999999, 2024-09-14T09:07:23.363 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.363 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.363 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.363 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.363 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.363 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.363 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105356 2024-09-14T09:07:23.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.364 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845539+0000", 2024-09-14T09:07:23.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845539+0000", 2024-09-14T09:07:23.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845540+0000", 2024-09-14T09:07:23.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1659999999999999e-06 2024-09-14T09:07:23.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845555+0000", 2024-09-14T09:07:23.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4552000000000001e-05 2024-09-14T09:07:23.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845576+0000", 2024-09-14T09:07:23.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1556000000000002e-05 2024-09-14T09:07:23.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845587+0000", 2024-09-14T09:07:23.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0178e-05 2024-09-14T09:07:23.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845635+0000", 2024-09-14T09:07:23.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8231e-05 2024-09-14T09:07:23.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845640+0000", 2024-09-14T09:07:23.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1549999999999998e-06 2024-09-14T09:07:23.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860531+0000", 2024-09-14T09:07:23.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014891066 2024-09-14T09:07:23.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860535+0000", 2024-09-14T09:07:23.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1980000000000003e-06 2024-09-14T09:07:23.367 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.367 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.367 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.368 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105421 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.368 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.847860+0000", 2024-09-14T09:07:23.368 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34196841900000002, 2024-09-14T09:07:23.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35925075299999998, 2024-09-14T09:07:23.368 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.368 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.368 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.368 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.368 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.368 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.368 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105421 2024-09-14T09:07:23.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.368 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847860+0000", 2024-09-14T09:07:23.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847860+0000", 2024-09-14T09:07:23.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847861+0000", 2024-09-14T09:07:23.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1399999999999995e-07 2024-09-14T09:07:23.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847865+0000", 2024-09-14T09:07:23.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8039999999999999e-06 2024-09-14T09:07:23.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847877+0000", 2024-09-14T09:07:23.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2082000000000001e-05 2024-09-14T09:07:23.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847885+0000", 2024-09-14T09:07:23.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9619999999999993e-06 2024-09-14T09:07:23.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847934+0000", 2024-09-14T09:07:23.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9427e-05 2024-09-14T09:07:23.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847941+0000", 2024-09-14T09:07:23.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2140000000000001e-06 2024-09-14T09:07:23.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861703+0000", 2024-09-14T09:07:23.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013762196000000001 2024-09-14T09:07:23.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861712+0000", 2024-09-14T09:07:23.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0529999999999996e-06 2024-09-14T09:07:23.372 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.372 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.372 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.372 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105485 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.372 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857828+0000", 2024-09-14T09:07:23.373 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33200031800000002, 2024-09-14T09:07:23.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34935679200000003, 2024-09-14T09:07:23.373 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.373 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.373 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.373 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.373 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.373 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.373 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105485 2024-09-14T09:07:23.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.373 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857828+0000", 2024-09-14T09:07:23.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857828+0000", 2024-09-14T09:07:23.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857829+0000", 2024-09-14T09:07:23.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8699999999999995e-07 2024-09-14T09:07:23.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857832+0000", 2024-09-14T09:07:23.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-06 2024-09-14T09:07:23.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857840+0000", 2024-09-14T09:07:23.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6489999999999997e-06 2024-09-14T09:07:23.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857849+0000", 2024-09-14T09:07:23.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6510000000000007e-06 2024-09-14T09:07:23.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863041+0000", 2024-09-14T09:07:23.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051925959999999998 2024-09-14T09:07:23.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863046+0000", 2024-09-14T09:07:23.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5580000000000002e-06 2024-09-14T09:07:23.376 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.376 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.376 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.376 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105550 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.377 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860673+0000", 2024-09-14T09:07:23.377 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32915603300000001, 2024-09-14T09:07:23.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34657644799999998, 2024-09-14T09:07:23.377 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.377 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.377 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.377 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.377 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.377 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.377 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105550 2024-09-14T09:07:23.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.377 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860673+0000", 2024-09-14T09:07:23.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860673+0000", 2024-09-14T09:07:23.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860673+0000", 2024-09-14T09:07:23.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8699999999999995e-07 2024-09-14T09:07:23.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860675+0000", 2024-09-14T09:07:23.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.041e-06 2024-09-14T09:07:23.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860682+0000", 2024-09-14T09:07:23.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9070000000000001e-06 2024-09-14T09:07:23.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860687+0000", 2024-09-14T09:07:23.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2750000000000003e-06 2024-09-14T09:07:23.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864948+0000", 2024-09-14T09:07:23.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004260532 2024-09-14T09:07:23.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864954+0000", 2024-09-14T09:07:23.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3629999999999999e-06 2024-09-14T09:07:23.380 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.380 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.380 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.380 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105613 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.380 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.864295+0000", 2024-09-14T09:07:23.380 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32553305300000002, 2024-09-14T09:07:23.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34301470299999998, 2024-09-14T09:07:23.381 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.381 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.381 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.381 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.381 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.381 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.381 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105613 2024-09-14T09:07:23.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.381 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864295+0000", 2024-09-14T09:07:23.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864295+0000", 2024-09-14T09:07:23.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864297+0000", 2024-09-14T09:07:23.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.052e-06 2024-09-14T09:07:23.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864301+0000", 2024-09-14T09:07:23.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.34e-06 2024-09-14T09:07:23.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864315+0000", 2024-09-14T09:07:23.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3867e-05 2024-09-14T09:07:23.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864325+0000", 2024-09-14T09:07:23.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0715e-05 2024-09-14T09:07:23.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865980+0000", 2024-09-14T09:07:23.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016547249999999999 2024-09-14T09:07:23.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.865987+0000", 2024-09-14T09:07:23.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5270000000000004e-06 2024-09-14T09:07:23.384 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.384 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.384 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.384 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105677 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.384 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867959+0000", 2024-09-14T09:07:23.384 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32186925999999999, 2024-09-14T09:07:23.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33941563600000002, 2024-09-14T09:07:23.385 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.385 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.385 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.385 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.385 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.385 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.385 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105677 2024-09-14T09:07:23.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.385 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867959+0000", 2024-09-14T09:07:23.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867959+0000", 2024-09-14T09:07:23.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867960+0000", 2024-09-14T09:07:23.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.78e-07 2024-09-14T09:07:23.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867964+0000", 2024-09-14T09:07:23.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999999e-06 2024-09-14T09:07:23.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867977+0000", 2024-09-14T09:07:23.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2863e-05 2024-09-14T09:07:23.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867987+0000", 2024-09-14T09:07:23.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.577e-06 2024-09-14T09:07:23.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868086+0000", 2024-09-14T09:07:23.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8838999999999993e-05 2024-09-14T09:07:23.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868099+0000", 2024-09-14T09:07:23.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3138e-05 2024-09-14T09:07:23.388 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.388 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.388 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.389 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105741 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.389 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.869447+0000", 2024-09-14T09:07:23.389 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32038155099999999, 2024-09-14T09:07:23.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33798895000000001, 2024-09-14T09:07:23.389 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.389 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.389 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.389 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.389 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.389 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.389 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105741 2024-09-14T09:07:23.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.390 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869447+0000", 2024-09-14T09:07:23.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869447+0000", 2024-09-14T09:07:23.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869448+0000", 2024-09-14T09:07:23.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0200000000000001e-07 2024-09-14T09:07:23.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869451+0000", 2024-09-14T09:07:23.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1360000000000001e-06 2024-09-14T09:07:23.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869461+0000", 2024-09-14T09:07:23.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9960000000000005e-06 2024-09-14T09:07:23.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869469+0000", 2024-09-14T09:07:23.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4200000000000007e-06 2024-09-14T09:07:23.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869765+0000", 2024-09-14T09:07:23.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029527900000000002 2024-09-14T09:07:23.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869771+0000", 2024-09-14T09:07:23.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5629999999999997e-06 2024-09-14T09:07:23.393 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.393 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.393 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.393 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105295 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.393 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.841663+0000", 2024-09-14T09:07:23.393 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34816524799999998, 2024-09-14T09:07:23.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.36583586299999998, 2024-09-14T09:07:23.393 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.394 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.394 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.394 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.394 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.394 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.394 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105295 2024-09-14T09:07:23.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.394 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841663+0000", 2024-09-14T09:07:23.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841663+0000", 2024-09-14T09:07:23.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841664+0000", 2024-09-14T09:07:23.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8800000000000003e-07 2024-09-14T09:07:23.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841666+0000", 2024-09-14T09:07:23.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2579999999999998e-06 2024-09-14T09:07:23.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841672+0000", 2024-09-14T09:07:23.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8479999999999997e-06 2024-09-14T09:07:23.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.841676+0000", 2024-09-14T09:07:23.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4259999999999996e-06 2024-09-14T09:07:23.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842092+0000", 2024-09-14T09:07:23.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00041565200000000002 2024-09-14T09:07:23.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.842099+0000", 2024-09-14T09:07:23.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0079999999999996e-06 2024-09-14T09:07:23.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857897+0000", 2024-09-14T09:07:23.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015798293000000001 2024-09-14T09:07:23.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857904+0000", 2024-09-14T09:07:23.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1559999999999998e-06 2024-09-14T09:07:23.398 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.398 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.398 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.398 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105359 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.398 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.845608+0000", 2024-09-14T09:07:23.398 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34422092900000001, 2024-09-14T09:07:23.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.36196557699999998, 2024-09-14T09:07:23.398 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.399 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.399 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.399 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.399 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.399 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.399 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105359 2024-09-14T09:07:23.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.399 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845608+0000", 2024-09-14T09:07:23.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.400 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.400 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845608+0000", 2024-09-14T09:07:23.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.400 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.400 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845609+0000", 2024-09-14T09:07:23.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7399999999999991e-07 2024-09-14T09:07:23.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.400 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.400 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845613+0000", 2024-09-14T09:07:23.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9319999999999998e-06 2024-09-14T09:07:23.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845624+0000", 2024-09-14T09:07:23.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1562e-05 2024-09-14T09:07:23.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845633+0000", 2024-09-14T09:07:23.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2939999999999994e-06 2024-09-14T09:07:23.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845667+0000", 2024-09-14T09:07:23.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3448999999999998e-05 2024-09-14T09:07:23.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.845672+0000", 2024-09-14T09:07:23.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9439999999999996e-06 2024-09-14T09:07:23.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860553+0000", 2024-09-14T09:07:23.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.014880763999999999 2024-09-14T09:07:23.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860568+0000", 2024-09-14T09:07:23.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5820000000000001e-05 2024-09-14T09:07:23.403 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.403 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.403 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.403 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105423 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.403 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.847898+0000", 2024-09-14T09:07:23.403 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.34193082400000002, 2024-09-14T09:07:23.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.35974757000000002, 2024-09-14T09:07:23.403 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.403 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.404 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.404 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.404 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.404 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.404 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105423 2024-09-14T09:07:23.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.404 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847898+0000", 2024-09-14T09:07:23.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847898+0000", 2024-09-14T09:07:23.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847899+0000", 2024-09-14T09:07:23.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2200000000000003e-07 2024-09-14T09:07:23.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847902+0000", 2024-09-14T09:07:23.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4240000000000002e-06 2024-09-14T09:07:23.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847913+0000", 2024-09-14T09:07:23.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0994e-05 2024-09-14T09:07:23.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847921+0000", 2024-09-14T09:07:23.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8639999999999996e-06 2024-09-14T09:07:23.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847973+0000", 2024-09-14T09:07:23.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1975999999999998e-05 2024-09-14T09:07:23.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting_for_map not empty", 2024-09-14T09:07:23.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.847979+0000", 2024-09-14T09:07:23.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9920000000000004e-06 2024-09-14T09:07:23.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861752+0000", 2024-09-14T09:07:23.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013772888 2024-09-14T09:07:23.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.861761+0000", 2024-09-14T09:07:23.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.003e-06 2024-09-14T09:07:23.408 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.408 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.408 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.408 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105487 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.408 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.857870+0000", 2024-09-14T09:07:23.408 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.33195836899999998, 2024-09-14T09:07:23.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34985894000000001, 2024-09-14T09:07:23.408 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.408 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.408 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.408 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.408 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.409 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.409 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105487 2024-09-14T09:07:23.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.409 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857870+0000", 2024-09-14T09:07:23.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857870+0000", 2024-09-14T09:07:23.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857871+0000", 2024-09-14T09:07:23.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1399999999999997e-07 2024-09-14T09:07:23.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857873+0000", 2024-09-14T09:07:23.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0269999999999998e-06 2024-09-14T09:07:23.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857879+0000", 2024-09-14T09:07:23.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-09-14T09:07:23.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.857886+0000", 2024-09-14T09:07:23.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.657e-06 2024-09-14T09:07:23.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863081+0000", 2024-09-14T09:07:23.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051949789999999997 2024-09-14T09:07:23.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.863090+0000", 2024-09-14T09:07:23.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0780000000000002e-06 2024-09-14T09:07:23.412 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.412 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.412 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.412 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105551 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.412 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.860694+0000", 2024-09-14T09:07:23.412 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32913429799999999, 2024-09-14T09:07:23.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34709875600000001, 2024-09-14T09:07:23.412 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.412 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.412 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.412 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.413 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.413 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.413 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105551 2024-09-14T09:07:23.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.413 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860694+0000", 2024-09-14T09:07:23.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860694+0000", 2024-09-14T09:07:23.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860695+0000", 2024-09-14T09:07:23.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4999999999999998e-07 2024-09-14T09:07:23.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860697+0000", 2024-09-14T09:07:23.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2029999999999999e-06 2024-09-14T09:07:23.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860704+0000", 2024-09-14T09:07:23.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.7900000000000002e-06 2024-09-14T09:07:23.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.860709+0000", 2024-09-14T09:07:23.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4779999999999999e-06 2024-09-14T09:07:23.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864980+0000", 2024-09-14T09:07:23.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042711010000000002 2024-09-14T09:07:23.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864987+0000", 2024-09-14T09:07:23.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6200000000000001e-06 2024-09-14T09:07:23.416 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.416 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.416 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.416 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105615 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.416 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:07:23.416 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.325461202, 2024-09-14T09:07:23.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.34348697700000003, 2024-09-14T09:07:23.416 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.416 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.417 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.417 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.417 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.417 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.417 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105615 2024-09-14T09:07:23.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.417 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:07:23.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:07:23.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864368+0000", 2024-09-14T09:07:23.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.043e-06 2024-09-14T09:07:23.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864373+0000", 2024-09-14T09:07:23.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4669999999999998e-06 2024-09-14T09:07:23.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864387+0000", 2024-09-14T09:07:23.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3876000000000001e-05 2024-09-14T09:07:23.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864397+0000", 2024-09-14T09:07:23.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0592e-05 2024-09-14T09:07:23.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866069+0000", 2024-09-14T09:07:23.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016714410000000001 2024-09-14T09:07:23.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866076+0000", 2024-09-14T09:07:23.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:07:23.420 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.420 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.420 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.420 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105679 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.420 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.868018+0000", 2024-09-14T09:07:23.420 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32181101600000001, 2024-09-14T09:07:23.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33990300200000001, 2024-09-14T09:07:23.421 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.421 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.421 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.421 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.421 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.421 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.421 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105679 2024-09-14T09:07:23.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.421 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868018+0000", 2024-09-14T09:07:23.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868018+0000", 2024-09-14T09:07:23.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868019+0000", 2024-09-14T09:07:23.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.029e-06 2024-09-14T09:07:23.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868022+0000", 2024-09-14T09:07:23.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9049999999999999e-06 2024-09-14T09:07:23.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868035+0000", 2024-09-14T09:07:23.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2917e-05 2024-09-14T09:07:23.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868045+0000", 2024-09-14T09:07:23.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7329999999999998e-06 2024-09-14T09:07:23.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868172+0000", 2024-09-14T09:07:23.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000127083 2024-09-14T09:07:23.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.868186+0000", 2024-09-14T09:07:23.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3534e-05 2024-09-14T09:07:23.424 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.424 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.424 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.425 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105742 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:23.425 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.869493+0000", 2024-09-14T09:07:23.425 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.32033546400000001, 2024-09-14T09:07:23.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.33848805799999998, 2024-09-14T09:07:23.425 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:23.425 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:23.425 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:23.425 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:23.425 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:23.425 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:23.425 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105742 2024-09-14T09:07:23.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.425 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:23.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:23.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869493+0000", 2024-09-14T09:07:23.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:23.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869493+0000", 2024-09-14T09:07:23.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:23.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:23.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869494+0000", 2024-09-14T09:07:23.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:07:23.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:23.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869498+0000", 2024-09-14T09:07:23.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0300000000000004e-06 2024-09-14T09:07:23.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:23.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869510+0000", 2024-09-14T09:07:23.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2186e-05 2024-09-14T09:07:23.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:23.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869519+0000", 2024-09-14T09:07:23.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8419999999999994e-06 2024-09-14T09:07:23.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:23.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869798+0000", 2024-09-14T09:07:23.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00027890899999999998 2024-09-14T09:07:23.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:23.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:23.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:23.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.869804+0000", 2024-09-14T09:07:23.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.652e-06 2024-09-14T09:07:23.428 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.428 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:23.428 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.428 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:23.429 INFO:teuthology.orchestra.run.smithi183.stdout: ], 2024-09-14T09:07:23.429 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 256 2024-09-14T09:07:23.429 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:23.429 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:07:23.431 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:07:23.432 INFO:teuthology.orchestra.run.smithi176.stdout:5404 writesame 0xe566c19 thru 0xe57554e data_size 0x1(0xe936 bytes) 2024-09-14T09:07:23.432 INFO:teuthology.orchestra.run.smithi176.stdout:5405 read 0x3180a47 thru 0x318579f (0x4d59 bytes) 2024-09-14T09:07:23.433 INFO:teuthology.orchestra.run.smithi176.stdout:5406 punch from 0x1b474a2 to 0x1b53fcf, (0xcb2d bytes) 2024-09-14T09:07:23.433 INFO:teuthology.orchestra.run.smithi176.stdout:5407 write 0x14bd37a thru 0x14c737c (0xa003 bytes) 2024-09-14T09:07:23.433 INFO:teuthology.orchestra.run.smithi176.stdout:5408 read 0xb1b8f37 thru 0xb1b9484 (0x54e bytes) 2024-09-14T09:07:23.433 INFO:teuthology.orchestra.run.smithi176.stdout:5409 write 0x24c5415 thru 0x24d1f6d (0xcb59 bytes) 2024-09-14T09:07:23.433 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:23.433 INFO:teuthology.orchestra.run.smithi176.stdout:5413 punch from 0x82bc27e to 0x82bef2f, (0x2cb1 bytes) 2024-09-14T09:07:23.433 INFO:teuthology.orchestra.run.smithi176.stdout:5414 punch from 0xa3ee479 to 0xa3f88c1, (0xa448 bytes) 2024-09-14T09:07:23.433 INFO:teuthology.orchestra.run.smithi176.stdout:5415 writesame 0x9f8fb75 thru 0x9f930d1 data_size 0x1(0x355d bytes) 2024-09-14T09:07:23.433 INFO:teuthology.orchestra.run.smithi176.stdout:5416 trunc from 0xe57554f to 0x8b9dc0f 2024-09-14T09:07:23.433 INFO:teuthology.orchestra.run.smithi176.stdout:5417 write 0x2db817d thru 0x2db883f (0x6c3 bytes) 2024-09-14T09:07:23.433 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:23.433 INFO:teuthology.orchestra.run.smithi176.stdout:5420 trunc from 0x8b9dc0f to 0x6967384 2024-09-14T09:07:23.433 INFO:teuthology.orchestra.run.smithi176.stdout:5421 punch from 0xdce9f7 to 0xdd687a, (0x7e83 bytes) 2024-09-14T09:07:23.434 INFO:teuthology.orchestra.run.smithi176.stdout:5422 write 0x5f82c8 thru 0x60351c (0xb255 bytes) 2024-09-14T09:07:23.434 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:23.431+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:23.531 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:23.535+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:23.605 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:23.605 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:23.605 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:07:23.605 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:07:23.605 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:23.626 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:07:23.631 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:23.631 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:23.656 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:23.656 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:23.670 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:23.675+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:23.680 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:23.680 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:23.704 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:23.704 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:23.727 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:07:23.727 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:07:23.733 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:23.735+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:23.750 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:23.750 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:23.756 INFO:teuthology.orchestra.run.smithi176.stdout:5423 trunc from 0x6967384 to 0xee46396 2024-09-14T09:07:23.763 INFO:teuthology.orchestra.run.smithi176.stdout:5426 writesame 0x608818a thru 0x608cbfd data_size 0xa(0x4a74 bytes) 2024-09-14T09:07:23.769 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:23.769 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:23.778 INFO:teuthology.orchestra.run.smithi176.stdout:5427 writesame 0x439f639 thru 0x43adcf1 data_size 0x1(0xe6b9 bytes) 2024-09-14T09:07:23.796 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:23.797 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:23.797 INFO:teuthology.orchestra.run.smithi176.stdout:5431 write 0x7c45acd thru 0x7c50a48 (0xaf7c bytes) 2024-09-14T09:07:23.811 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:23.811 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:23.811 INFO:teuthology.orchestra.run.smithi176.stdout:5432 write 0xe63d23d thru 0xe64793c (0xa700 bytes) 2024-09-14T09:07:23.824 INFO:teuthology.orchestra.run.smithi176.stdout:5433 writesame 0xb0db245 thru 0xb0e5786 data_size 0x21(0xa542 bytes) 2024-09-14T09:07:23.829 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:07:23.834 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:23.835+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:23.836 INFO:teuthology.orchestra.run.smithi176.stdout:5435 read 0x1d918d1 thru 0x1da0c64 (0xf394 bytes) 2024-09-14T09:07:23.837 INFO:teuthology.orchestra.run.smithi176.stdout:5436 read 0xb117772 thru 0xb11b794 (0x4023 bytes) 2024-09-14T09:07:23.837 INFO:teuthology.orchestra.run.smithi176.stdout:5437 write 0x192235d thru 0x192e968 (0xc60c bytes) 2024-09-14T09:07:23.857 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:23.857 INFO:teuthology.orchestra.run.smithi176.stdout:5439 writesame 0x5a89b45 thru 0x5a954b2 data_size 0x65(0xb96e bytes) 2024-09-14T09:07:23.865 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:23.865 INFO:teuthology.orchestra.run.smithi176.stdout:5441 writesame 0x29b48d7 thru 0x29b83eb data_size 0xb(0x3b15 bytes) 2024-09-14T09:07:23.870 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T09:07:23.887 INFO:teuthology.orchestra.run.smithi176.stdout:5442 read 0x293a053 thru 0x293ff41 (0x5eef bytes) 2024-09-14T09:07:23.888 INFO:teuthology.orchestra.run.smithi176.stdout:5443 write 0x6147e7a thru 0x614f707 (0x788e bytes) 2024-09-14T09:07:23.891 INFO:teuthology.orchestra.run.smithi176.stdout:5444 punch from 0x288673c to 0x288827e, (0x1b42 bytes) 2024-09-14T09:07:23.892 INFO:teuthology.orchestra.run.smithi176.stdout:5445 punch from 0xd25c8bf to 0xd26a24c, (0xd98d bytes) 2024-09-14T09:07:23.892 INFO:teuthology.orchestra.run.smithi176.stdout:5446 write 0xd570598 thru 0xd57d264 (0xcccd bytes) 2024-09-14T09:07:23.906 INFO:teuthology.orchestra.run.smithi176.stdout:5447 punch from 0x4c1a566 to 0x4c24e07, (0xa8a1 bytes) 2024-09-14T09:07:23.906 INFO:teuthology.orchestra.run.smithi176.stdout:5448 writesame 0x8025ec5 thru 0x802a8e3 data_size 0x4b(0x4a1f bytes) 2024-09-14T09:07:23.925 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:23.925 INFO:teuthology.orchestra.run.smithi176.stdout:5450 write 0x99643e5 thru 0x99670e5 (0x2d01 bytes) 2024-09-14T09:07:23.935 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:23.939+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:23.943 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:23.943 INFO:teuthology.orchestra.run.smithi176.stdout:5453 read 0x3fe9536 thru 0x3ff23ab (0x8e76 bytes) 2024-09-14T09:07:23.943 INFO:teuthology.orchestra.run.smithi176.stdout:5455 write 0x54a7808 thru 0x54afb38 (0x8331 bytes) 2024-09-14T09:07:23.961 INFO:teuthology.orchestra.run.smithi176.stdout:5456 punch from 0x277df28 to 0x277f32b, (0x1403 bytes) 2024-09-14T09:07:23.961 INFO:teuthology.orchestra.run.smithi176.stdout:5457 read 0xb08d2ee thru 0xb0990bf (0xbdd2 bytes) 2024-09-14T09:07:23.962 INFO:teuthology.orchestra.run.smithi176.stdout:5458 trunc from 0xee46396 to 0xe923724 2024-09-14T09:07:23.968 INFO:teuthology.orchestra.run.smithi176.stdout:5460 write 0xdf8e1d thru 0xdfa408 (0x15ec bytes) 2024-09-14T09:07:24.005 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:24.006 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:24.020 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:07:24.063 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:24.067+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:24.137 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:24.135+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:24.200 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:24.200 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:24.216 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:07:24.266 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:24.271+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:24.339 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:24.339+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:24.343 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:24.343 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:24.359 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:07:24.454 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:24.459+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:24.528 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:24.528 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:24.541 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:24.539+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:24.543 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:07:24.641 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:24.639+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:24.713 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:24.713 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:24.727 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:07:24.775 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:24.779+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:24.842 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:24.843+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:24.857 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:24.858 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:24.872 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:07:24.944 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:24.947+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:25.007 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:25.007 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:25.021 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:07:25.044 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:25.047+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:25.087 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is set 2024-09-14T09:07:25.146 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:25.147+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:25.157 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:25.157 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:25.171 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:07:25.246 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:25.251+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:25.333 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:25.333 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:25.347 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:07:25.347 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:25.347+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:25.448 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:25.447+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:25.512 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:25.513 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:25.526 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:07:25.549 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:25.551+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:25.650 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:25.651+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:25.691 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:25.691 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:25.705 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:07:25.754 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:25.755+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:25.821 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:07:25.827 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:25.827 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:25.842 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:07:25.852 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:25.855+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:25.954 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:25.955+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:25.979 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:25.979 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:25.997 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:25.997 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:26.015 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:26.015 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:26.015 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:26.016 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:26.016 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:26.016 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:26.030 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:07:26.033 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:26.033 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:26.054 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:26.055+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:26.055 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:07:26.055 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:07:26.076 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:26.076 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:26.103 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:07:26.104 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:07:26.104 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T09:07:26.104 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T09:07:26.107 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:26.107 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:26.116 INFO:teuthology.orchestra.run.smithi176.stdout:5461 writesame 0x8bbdab9 thru 0x8bc8368 data_size 0x8(0xa8b0 bytes) 2024-09-14T09:07:26.136 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:26.136 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:26.154 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:26.155+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:26.162 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:26.162 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:26.162 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:26.162 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:26.163 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:26.177 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:07:26.266 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:26.267+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:26.345 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:26.345 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:26.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:26.345 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:26.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.345 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106705 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 518144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.345 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:07:26.345 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.496703009, 2024-09-14T09:07:26.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3001326799999999, 2024-09-14T09:07:26.346 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.346 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.346 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.346 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.346 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.346 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.346 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106705 2024-09-14T09:07:26.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.346 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:07:26.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:07:26.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:07:26.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4699999999999998e-07 2024-09-14T09:07:26.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847500+0000", 2024-09-14T09:07:26.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0860000000000001e-06 2024-09-14T09:07:26.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847503+0000", 2024-09-14T09:07:26.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1269999999999999e-06 2024-09-14T09:07:26.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847506+0000", 2024-09-14T09:07:26.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5780000000000001e-06 2024-09-14T09:07:26.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858911+0000", 2024-09-14T09:07:26.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011404925 2024-09-14T09:07:26.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858919+0000", 2024-09-14T09:07:26.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8620000000000003e-06 2024-09-14T09:07:26.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147254+0000", 2024-09-14T09:07:26.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2883353719999999 2024-09-14T09:07:26.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147408+0000", 2024-09-14T09:07:26.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015389200000000001 2024-09-14T09:07:26.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147631+0000", 2024-09-14T09:07:26.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022359100000000001 2024-09-14T09:07:26.350 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.350 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.350 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.351 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106707 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 522240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.351 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:07:26.351 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4966925389999997, 2024-09-14T09:07:26.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.300471801, 2024-09-14T09:07:26.351 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.351 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.351 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.351 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.351 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.351 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.351 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106707 2024-09-14T09:07:26.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.352 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:07:26.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:07:26.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:07:26.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.23e-07 2024-09-14T09:07:26.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847514+0000", 2024-09-14T09:07:26.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9150000000000004e-06 2024-09-14T09:07:26.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847520+0000", 2024-09-14T09:07:26.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.203e-06 2024-09-14T09:07:26.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847532+0000", 2024-09-14T09:07:26.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2441e-05 2024-09-14T09:07:26.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858953+0000", 2024-09-14T09:07:26.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01142108 2024-09-14T09:07:26.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858957+0000", 2024-09-14T09:07:26.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2830000000000003e-06 2024-09-14T09:07:26.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147667+0000", 2024-09-14T09:07:26.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2887094619999999 2024-09-14T09:07:26.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147784+0000", 2024-09-14T09:07:26.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117465 2024-09-14T09:07:26.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147981+0000", 2024-09-14T09:07:26.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019672900000000001 2024-09-14T09:07:26.356 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.356 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.356 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.356 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106709 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 3072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.356 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:07:26.356 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4966642480000001, 2024-09-14T09:07:26.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.301335192, 2024-09-14T09:07:26.356 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.356 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.357 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.357 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106709 2024-09-14T09:07:26.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.357 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:07:26.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:07:26.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847538+0000", 2024-09-14T09:07:26.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2800000000000003e-07 2024-09-14T09:07:26.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847539+0000", 2024-09-14T09:07:26.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2720000000000001e-06 2024-09-14T09:07:26.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847544+0000", 2024-09-14T09:07:26.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7219999999999999e-06 2024-09-14T09:07:26.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847547+0000", 2024-09-14T09:07:26.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9960000000000002e-06 2024-09-14T09:07:26.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858991+0000", 2024-09-14T09:07:26.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011444218000000001 2024-09-14T09:07:26.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858999+0000", 2024-09-14T09:07:26.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9419999999999995e-06 2024-09-14T09:07:26.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148053+0000", 2024-09-14T09:07:26.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2890544099999999 2024-09-14T09:07:26.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148203+0000", 2024-09-14T09:07:26.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149419 2024-09-14T09:07:26.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148873+0000", 2024-09-14T09:07:26.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00066988499999999995 2024-09-14T09:07:26.361 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.361 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.361 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.361 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106711 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 7168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.361 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:07:26.361 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.49665099, 2024-09-14T09:07:26.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3022241219999999, 2024-09-14T09:07:26.362 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.362 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.362 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.362 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.362 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.362 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.362 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106711 2024-09-14T09:07:26.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.362 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:07:26.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:07:26.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:07:26.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8000000000000002e-07 2024-09-14T09:07:26.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847552+0000", 2024-09-14T09:07:26.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1340000000000001e-06 2024-09-14T09:07:26.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847557+0000", 2024-09-14T09:07:26.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6179999999999997e-06 2024-09-14T09:07:26.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847560+0000", 2024-09-14T09:07:26.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5899999999999999e-06 2024-09-14T09:07:26.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859050+0000", 2024-09-14T09:07:26.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01148914 2024-09-14T09:07:26.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859056+0000", 2024-09-14T09:07:26.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6630000000000004e-06 2024-09-14T09:07:26.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148948+0000", 2024-09-14T09:07:26.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2898914370000001 2024-09-14T09:07:26.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149079+0000", 2024-09-14T09:07:26.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013145400000000001 2024-09-14T09:07:26.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149775+0000", 2024-09-14T09:07:26.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00069580599999999999 2024-09-14T09:07:26.366 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.366 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.366 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.366 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106713 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 11264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.367 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:07:26.367 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4966375249999997, 2024-09-14T09:07:26.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3035188959999999, 2024-09-14T09:07:26.367 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.367 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.367 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.367 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.367 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.367 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.367 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106713 2024-09-14T09:07:26.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.367 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:07:26.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:07:26.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:07:26.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6e-07 2024-09-14T09:07:26.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847566+0000", 2024-09-14T09:07:26.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1039999999999999e-06 2024-09-14T09:07:26.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847569+0000", 2024-09-14T09:07:26.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.191e-06 2024-09-14T09:07:26.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847571+0000", 2024-09-14T09:07:26.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6390000000000001e-06 2024-09-14T09:07:26.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859099+0000", 2024-09-14T09:07:26.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011527379000000001 2024-09-14T09:07:26.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859106+0000", 2024-09-14T09:07:26.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3679999999999996e-06 2024-09-14T09:07:26.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149878+0000", 2024-09-14T09:07:26.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.29077208 2024-09-14T09:07:26.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.150175+0000", 2024-09-14T09:07:26.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029685699999999998 2024-09-14T09:07:26.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151083+0000", 2024-09-14T09:07:26.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00090801800000000002 2024-09-14T09:07:26.371 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.371 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.371 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.372 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106715 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 15360~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.372 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:07:26.372 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4966265160000001, 2024-09-14T09:07:26.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3047478800000001, 2024-09-14T09:07:26.372 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.372 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.372 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.372 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.372 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.372 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.372 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106715 2024-09-14T09:07:26.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.373 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:07:26.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:07:26.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:07:26.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6e-07 2024-09-14T09:07:26.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847580+0000", 2024-09-14T09:07:26.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9250000000000003e-06 2024-09-14T09:07:26.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847585+0000", 2024-09-14T09:07:26.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8409999999999999e-06 2024-09-14T09:07:26.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847588+0000", 2024-09-14T09:07:26.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6989999999999999e-06 2024-09-14T09:07:26.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859151+0000", 2024-09-14T09:07:26.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011562604000000001 2024-09-14T09:07:26.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859159+0000", 2024-09-14T09:07:26.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4179999999999997e-06 2024-09-14T09:07:26.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151155+0000", 2024-09-14T09:07:26.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.291996068 2024-09-14T09:07:26.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151383+0000", 2024-09-14T09:07:26.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022746100000000001 2024-09-14T09:07:26.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152323+0000", 2024-09-14T09:07:26.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00094060400000000005 2024-09-14T09:07:26.377 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.377 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.377 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.377 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106717 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 19456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.377 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:07:26.377 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4966098280000004, 2024-09-14T09:07:26.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.305574579, 2024-09-14T09:07:26.377 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.377 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.377 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.377 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.378 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.378 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.378 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106717 2024-09-14T09:07:26.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.378 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:07:26.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:07:26.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:07:26.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1199999999999999e-07 2024-09-14T09:07:26.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847593+0000", 2024-09-14T09:07:26.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.257e-06 2024-09-14T09:07:26.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847598+0000", 2024-09-14T09:07:26.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4139999999999996e-06 2024-09-14T09:07:26.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847602+0000", 2024-09-14T09:07:26.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3440000000000003e-06 2024-09-14T09:07:26.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859203+0000", 2024-09-14T09:07:26.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011600572999999999 2024-09-14T09:07:26.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859211+0000", 2024-09-14T09:07:26.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-06 2024-09-14T09:07:26.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152398+0000", 2024-09-14T09:07:26.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.293187812 2024-09-14T09:07:26.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152543+0000", 2024-09-14T09:07:26.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014478000000000001 2024-09-14T09:07:26.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153166+0000", 2024-09-14T09:07:26.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000623257 2024-09-14T09:07:26.382 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.382 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.382 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.382 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106719 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 23552~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.382 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:07:26.382 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4965955119999999, 2024-09-14T09:07:26.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.306394982, 2024-09-14T09:07:26.383 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.383 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.383 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.383 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.383 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.383 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.383 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106719 2024-09-14T09:07:26.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.383 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:07:26.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:07:26.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:07:26.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5100000000000001e-07 2024-09-14T09:07:26.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847608+0000", 2024-09-14T09:07:26.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.189e-06 2024-09-14T09:07:26.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847612+0000", 2024-09-14T09:07:26.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9519999999999996e-06 2024-09-14T09:07:26.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847614+0000", 2024-09-14T09:07:26.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6280000000000001e-06 2024-09-14T09:07:26.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859254+0000", 2024-09-14T09:07:26.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011640157 2024-09-14T09:07:26.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859261+0000", 2024-09-14T09:07:26.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0570000000000004e-06 2024-09-14T09:07:26.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153236+0000", 2024-09-14T09:07:26.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2939744040000001 2024-09-14T09:07:26.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153380+0000", 2024-09-14T09:07:26.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014410399999999999 2024-09-14T09:07:26.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154001+0000", 2024-09-14T09:07:26.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062124000000000001 2024-09-14T09:07:26.387 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.387 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.387 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.388 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106721 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 27648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.388 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:07:26.388 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4965838619999996, 2024-09-14T09:07:26.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3074446659999999, 2024-09-14T09:07:26.388 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.388 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.388 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.388 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.388 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.388 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.388 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106721 2024-09-14T09:07:26.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.389 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:07:26.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:07:26.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:07:26.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4299999999999999e-07 2024-09-14T09:07:26.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847619+0000", 2024-09-14T09:07:26.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T09:07:26.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847624+0000", 2024-09-14T09:07:26.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4229999999999998e-06 2024-09-14T09:07:26.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847627+0000", 2024-09-14T09:07:26.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0120000000000001e-06 2024-09-14T09:07:26.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859305+0000", 2024-09-14T09:07:26.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01167812 2024-09-14T09:07:26.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859313+0000", 2024-09-14T09:07:26.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2409999999999995e-06 2024-09-14T09:07:26.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154081+0000", 2024-09-14T09:07:26.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.294768296 2024-09-14T09:07:26.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154280+0000", 2024-09-14T09:07:26.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019824 2024-09-14T09:07:26.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155063+0000", 2024-09-14T09:07:26.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00078303400000000005 2024-09-14T09:07:26.393 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.393 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.393 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.393 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106723 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 31744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.393 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:07:26.393 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4965714319999996, 2024-09-14T09:07:26.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308293648, 2024-09-14T09:07:26.393 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.393 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.393 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.393 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.394 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106723 2024-09-14T09:07:26.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.394 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:07:26.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:07:26.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847631+0000", 2024-09-14T09:07:26.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6100000000000002e-07 2024-09-14T09:07:26.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847632+0000", 2024-09-14T09:07:26.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0589999999999999e-06 2024-09-14T09:07:26.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847635+0000", 2024-09-14T09:07:26.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.551e-06 2024-09-14T09:07:26.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847638+0000", 2024-09-14T09:07:26.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.644e-06 2024-09-14T09:07:26.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859344+0000", 2024-09-14T09:07:26.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011706397 2024-09-14T09:07:26.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859348+0000", 2024-09-14T09:07:26.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5779999999999999e-06 2024-09-14T09:07:26.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155135+0000", 2024-09-14T09:07:26.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.295787448 2024-09-14T09:07:26.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155279+0000", 2024-09-14T09:07:26.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014400700000000001 2024-09-14T09:07:26.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155924+0000", 2024-09-14T09:07:26.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00064470300000000005 2024-09-14T09:07:26.398 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.398 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.398 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.398 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106725 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 35840~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.398 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:07:26.398 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4965603210000005, 2024-09-14T09:07:26.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.309103218, 2024-09-14T09:07:26.398 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.398 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.398 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.398 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.398 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.399 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.399 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106725 2024-09-14T09:07:26.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.399 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:07:26.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:07:26.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847642+0000", 2024-09-14T09:07:26.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4499999999999998e-07 2024-09-14T09:07:26.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847643+0000", 2024-09-14T09:07:26.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4e-07 2024-09-14T09:07:26.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847647+0000", 2024-09-14T09:07:26.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6190000000000002e-06 2024-09-14T09:07:26.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847655+0000", 2024-09-14T09:07:26.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:07:26.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859381+0000", 2024-09-14T09:07:26.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011725793 2024-09-14T09:07:26.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859391+0000", 2024-09-14T09:07:26.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4390000000000006e-06 2024-09-14T09:07:26.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155997+0000", 2024-09-14T09:07:26.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2966067830000001 2024-09-14T09:07:26.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156133+0000", 2024-09-14T09:07:26.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000135609 2024-09-14T09:07:26.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156745+0000", 2024-09-14T09:07:26.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000611592 2024-09-14T09:07:26.403 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.403 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.403 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.403 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106727 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 39936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.403 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:07:26.403 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4965420209999998, 2024-09-14T09:07:26.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3099046560000001, 2024-09-14T09:07:26.403 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.404 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.404 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.404 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.404 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.404 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.404 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106727 2024-09-14T09:07:26.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.404 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:07:26.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:07:26.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:07:26.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1300000000000001e-07 2024-09-14T09:07:26.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847661+0000", 2024-09-14T09:07:26.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.192e-06 2024-09-14T09:07:26.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847665+0000", 2024-09-14T09:07:26.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0999999999999997e-06 2024-09-14T09:07:26.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847668+0000", 2024-09-14T09:07:26.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.644e-06 2024-09-14T09:07:26.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859422+0000", 2024-09-14T09:07:26.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011753559 2024-09-14T09:07:26.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859425+0000", 2024-09-14T09:07:26.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.343e-06 2024-09-14T09:07:26.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156817+0000", 2024-09-14T09:07:26.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2973924800000001 2024-09-14T09:07:26.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156956+0000", 2024-09-14T09:07:26.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013835 2024-09-14T09:07:26.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157564+0000", 2024-09-14T09:07:26.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060867500000000004 2024-09-14T09:07:26.408 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.408 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.408 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.408 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106729 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 44032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.408 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:07:26.408 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4965300480000003, 2024-09-14T09:07:26.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3107195970000001, 2024-09-14T09:07:26.408 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.408 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.409 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.409 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.409 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.409 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.409 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106729 2024-09-14T09:07:26.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.409 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:07:26.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:07:26.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:07:26.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.29e-07 2024-09-14T09:07:26.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847673+0000", 2024-09-14T09:07:26.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.096e-06 2024-09-14T09:07:26.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847678+0000", 2024-09-14T09:07:26.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.527e-06 2024-09-14T09:07:26.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847681+0000", 2024-09-14T09:07:26.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.788e-06 2024-09-14T09:07:26.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859462+0000", 2024-09-14T09:07:26.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01178032 2024-09-14T09:07:26.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859471+0000", 2024-09-14T09:07:26.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3430000000000002e-06 2024-09-14T09:07:26.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157635+0000", 2024-09-14T09:07:26.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2981640320000001 2024-09-14T09:07:26.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157773+0000", 2024-09-14T09:07:26.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763 2024-09-14T09:07:26.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158391+0000", 2024-09-14T09:07:26.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061863200000000004 2024-09-14T09:07:26.413 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.413 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.413 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.413 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106731 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 48128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.413 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:07:26.413 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4965144029999999, 2024-09-14T09:07:26.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3115142020000001, 2024-09-14T09:07:26.413 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.414 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.414 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.414 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.414 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.414 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.414 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106731 2024-09-14T09:07:26.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.414 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:07:26.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:07:26.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847688+0000", 2024-09-14T09:07:26.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0199999999999998e-07 2024-09-14T09:07:26.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847689+0000", 2024-09-14T09:07:26.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4529999999999999e-06 2024-09-14T09:07:26.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847694+0000", 2024-09-14T09:07:26.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9080000000000003e-06 2024-09-14T09:07:26.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847697+0000", 2024-09-14T09:07:26.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4920000000000002e-06 2024-09-14T09:07:26.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859499+0000", 2024-09-14T09:07:26.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011801676000000001 2024-09-14T09:07:26.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859502+0000", 2024-09-14T09:07:26.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2899999999999998e-06 2024-09-14T09:07:26.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158461+0000", 2024-09-14T09:07:26.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.298958796 2024-09-14T09:07:26.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158594+0000", 2024-09-14T09:07:26.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013321599999999999 2024-09-14T09:07:26.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159202+0000", 2024-09-14T09:07:26.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060706900000000001 2024-09-14T09:07:26.418 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.418 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.418 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.419 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106733 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 52224~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.419 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:07:26.419 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4965001469999999, 2024-09-14T09:07:26.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3123343809999999, 2024-09-14T09:07:26.419 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.419 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.419 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.419 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.419 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.419 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.419 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106733 2024-09-14T09:07:26.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.419 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:07:26.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:07:26.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:07:26.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5800000000000001e-07 2024-09-14T09:07:26.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847703+0000", 2024-09-14T09:07:26.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.127e-06 2024-09-14T09:07:26.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847706+0000", 2024-09-14T09:07:26.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3210000000000001e-06 2024-09-14T09:07:26.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847709+0000", 2024-09-14T09:07:26.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7539999999999998e-06 2024-09-14T09:07:26.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859538+0000", 2024-09-14T09:07:26.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01182942 2024-09-14T09:07:26.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859547+0000", 2024-09-14T09:07:26.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T09:07:26.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159273+0000", 2024-09-14T09:07:26.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2997263700000001 2024-09-14T09:07:26.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159415+0000", 2024-09-14T09:07:26.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000141765 2024-09-14T09:07:26.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160036+0000", 2024-09-14T09:07:26.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062122100000000003 2024-09-14T09:07:26.424 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.424 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.424 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.424 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106735 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 56320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.424 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:07:26.424 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4964887029999998, 2024-09-14T09:07:26.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3131339309999999, 2024-09-14T09:07:26.424 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.424 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.424 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.424 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.425 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.425 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.425 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106735 2024-09-14T09:07:26.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.425 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:07:26.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:07:26.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:07:26.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2100000000000001e-07 2024-09-14T09:07:26.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847714+0000", 2024-09-14T09:07:26.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.051e-06 2024-09-14T09:07:26.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847719+0000", 2024-09-14T09:07:26.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3640000000000001e-06 2024-09-14T09:07:26.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847723+0000", 2024-09-14T09:07:26.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8789999999999996e-06 2024-09-14T09:07:26.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859577+0000", 2024-09-14T09:07:26.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011854416 2024-09-14T09:07:26.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859580+0000", 2024-09-14T09:07:26.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.292e-06 2024-09-14T09:07:26.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160108+0000", 2024-09-14T09:07:26.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.30052758 2024-09-14T09:07:26.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160238+0000", 2024-09-14T09:07:26.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013043500000000001 2024-09-14T09:07:26.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160847+0000", 2024-09-14T09:07:26.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060869299999999999 2024-09-14T09:07:26.429 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.429 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.429 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.429 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106738 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 60416~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.429 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:07:26.430 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4964754989999998, 2024-09-14T09:07:26.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3139482179999999, 2024-09-14T09:07:26.430 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.430 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.430 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.430 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.430 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.430 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.430 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106738 2024-09-14T09:07:26.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.430 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:07:26.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:07:26.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:07:26.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3999999999999998e-07 2024-09-14T09:07:26.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847733+0000", 2024-09-14T09:07:26.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8070000000000002e-06 2024-09-14T09:07:26.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847738+0000", 2024-09-14T09:07:26.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9740000000000001e-06 2024-09-14T09:07:26.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847742+0000", 2024-09-14T09:07:26.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6009999999999999e-06 2024-09-14T09:07:26.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859612+0000", 2024-09-14T09:07:26.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011870537 2024-09-14T09:07:26.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859621+0000", 2024-09-14T09:07:26.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3019999999999995e-06 2024-09-14T09:07:26.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160917+0000", 2024-09-14T09:07:26.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3012961080000001 2024-09-14T09:07:26.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161054+0000", 2024-09-14T09:07:26.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000137422 2024-09-14T09:07:26.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161674+0000", 2024-09-14T09:07:26.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000620227 2024-09-14T09:07:26.434 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.434 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.434 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.435 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106739 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 64512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.435 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:07:26.435 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4964556590000004, 2024-09-14T09:07:26.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.314753777, 2024-09-14T09:07:26.435 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.435 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.435 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.435 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.435 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.435 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.435 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106739 2024-09-14T09:07:26.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.436 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:07:26.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:07:26.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:07:26.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9700000000000003e-07 2024-09-14T09:07:26.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847748+0000", 2024-09-14T09:07:26.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2589999999999999e-06 2024-09-14T09:07:26.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847752+0000", 2024-09-14T09:07:26.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6099999999999999e-06 2024-09-14T09:07:26.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847755+0000", 2024-09-14T09:07:26.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0520000000000002e-06 2024-09-14T09:07:26.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859653+0000", 2024-09-14T09:07:26.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011897575000000001 2024-09-14T09:07:26.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859658+0000", 2024-09-14T09:07:26.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8810000000000004e-06 2024-09-14T09:07:26.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161745+0000", 2024-09-14T09:07:26.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3020870819999999 2024-09-14T09:07:26.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161877+0000", 2024-09-14T09:07:26.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000132657 2024-09-14T09:07:26.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162500+0000", 2024-09-14T09:07:26.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062236400000000003 2024-09-14T09:07:26.440 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.440 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.440 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.440 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106741 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 68608~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.440 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:07:26.440 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4964428669999998, 2024-09-14T09:07:26.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3155568449999999, 2024-09-14T09:07:26.440 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.440 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.440 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.440 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.441 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.441 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.441 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106741 2024-09-14T09:07:26.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.441 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:07:26.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:07:26.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:07:26.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5800000000000001e-07 2024-09-14T09:07:26.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847760+0000", 2024-09-14T09:07:26.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:07:26.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847764+0000", 2024-09-14T09:07:26.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3189999999999999e-06 2024-09-14T09:07:26.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847766+0000", 2024-09-14T09:07:26.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6429999999999999e-06 2024-09-14T09:07:26.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859693+0000", 2024-09-14T09:07:26.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011926724 2024-09-14T09:07:26.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859703+0000", 2024-09-14T09:07:26.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6160000000000008e-06 2024-09-14T09:07:26.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162575+0000", 2024-09-14T09:07:26.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.302872128 2024-09-14T09:07:26.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162709+0000", 2024-09-14T09:07:26.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013446000000000001 2024-09-14T09:07:26.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163316+0000", 2024-09-14T09:07:26.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060650700000000005 2024-09-14T09:07:26.445 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.445 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.445 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.445 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106744 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 76800~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:26.445 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:07:26.445 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 4.4964317559999998, 2024-09-14T09:07:26.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3163901789999999, 2024-09-14T09:07:26.445 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.445 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.445 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.445 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.445 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.445 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.446 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106744 2024-09-14T09:07:26.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.446 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:07:26.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:07:26.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996947+0000", 2024-09-14T09:07:26.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.149177154 2024-09-14T09:07:26.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996964+0000", 2024-09-14T09:07:26.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.679e-05 2024-09-14T09:07:26.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996999+0000", 2024-09-14T09:07:26.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5463000000000002e-05 2024-09-14T09:07:26.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.997029+0000", 2024-09-14T09:07:26.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9567e-05 2024-09-14T09:07:26.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163390+0000", 2024-09-14T09:07:26.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.16636056799999999 2024-09-14T09:07:26.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163526+0000", 2024-09-14T09:07:26.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013655899999999999 2024-09-14T09:07:26.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.164160+0000", 2024-09-14T09:07:26.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00063407800000000005 2024-09-14T09:07:26.449 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.449 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.449 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.449 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.449 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.449 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:26.451 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:07:26.451 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:07:26.452 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:26.452 INFO:teuthology.orchestra.run.smithi176.stdout: "epoch": 117, 2024-09-14T09:07:26.452 INFO:teuthology.orchestra.run.smithi176.stdout: "fsid": "c635667d-994e-454f-bcea-51b45637b2a2", 2024-09-14T09:07:26.452 INFO:teuthology.orchestra.run.smithi176.stdout: "created": "2024-09-14T09:04:57.077293+0000", 2024-09-14T09:07:26.452 INFO:teuthology.orchestra.run.smithi176.stdout: "modified": "2024-09-14T09:07:26.085280+0000", 2024-09-14T09:07:26.452 INFO:teuthology.orchestra.run.smithi176.stdout: "last_up_change": "2024-09-14T09:05:11.210835+0000", 2024-09-14T09:07:26.452 INFO:teuthology.orchestra.run.smithi176.stdout: "last_in_change": "2024-09-14T09:07:21.060249+0000", 2024-09-14T09:07:26.452 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T09:07:26.452 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_num": 5801984, 2024-09-14T09:07:26.452 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_set": [ 2024-09-14T09:07:26.452 INFO:teuthology.orchestra.run.smithi176.stdout: "noscrub", 2024-09-14T09:07:26.452 INFO:teuthology.orchestra.run.smithi176.stdout: "pglog_hardlimit", 2024-09-14T09:07:26.452 INFO:teuthology.orchestra.run.smithi176.stdout: "purged_snapdirs", 2024-09-14T09:07:26.453 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_deletes", 2024-09-14T09:07:26.453 INFO:teuthology.orchestra.run.smithi176.stdout: "sortbitwise" 2024-09-14T09:07:26.453 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:26.453 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_version": 5, 2024-09-14T09:07:26.453 INFO:teuthology.orchestra.run.smithi176.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T09:07:26.453 INFO:teuthology.orchestra.run.smithi176.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T09:07:26.453 INFO:teuthology.orchestra.run.smithi176.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T09:07:26.453 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_snapshot": "", 2024-09-14T09:07:26.453 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_max": 3, 2024-09-14T09:07:26.453 INFO:teuthology.orchestra.run.smithi176.stdout: "max_osd": 8, 2024-09-14T09:07:26.453 INFO:teuthology.orchestra.run.smithi176.stdout: "require_min_compat_client": "luminous", 2024-09-14T09:07:26.453 INFO:teuthology.orchestra.run.smithi176.stdout: "min_compat_client": "luminous", 2024-09-14T09:07:26.453 INFO:teuthology.orchestra.run.smithi176.stdout: "require_osd_release": "squid", 2024-09-14T09:07:26.454 INFO:teuthology.orchestra.run.smithi176.stdout: "allow_crimson": false, 2024-09-14T09:07:26.454 INFO:teuthology.orchestra.run.smithi176.stdout: "pools": [ 2024-09-14T09:07:26.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.454 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 1, 2024-09-14T09:07:26.454 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_name": ".mgr", 2024-09-14T09:07:26.454 INFO:teuthology.orchestra.run.smithi176.stdout: "create_time": "2024-09-14T09:05:11.997673+0000", 2024-09-14T09:07:26.454 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": 1, 2024-09-14T09:07:26.454 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_names": "hashpspool", 2024-09-14T09:07:26.454 INFO:teuthology.orchestra.run.smithi176.stdout: "type": 1, 2024-09-14T09:07:26.454 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 2, 2024-09-14T09:07:26.454 INFO:teuthology.orchestra.run.smithi176.stdout: "min_size": 1, 2024-09-14T09:07:26.454 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_rule": 0, 2024-09-14T09:07:26.454 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_count": 0, 2024-09-14T09:07:26.455 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_target": 0, 2024-09-14T09:07:26.455 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T09:07:26.455 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T09:07:26.455 INFO:teuthology.orchestra.run.smithi176.stdout: "is_stretch_pool": false, 2024-09-14T09:07:26.455 INFO:teuthology.orchestra.run.smithi176.stdout: "object_hash": 2, 2024-09-14T09:07:26.455 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_autoscale_mode": "off", 2024-09-14T09:07:26.455 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num": 1, 2024-09-14T09:07:26.455 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num": 1, 2024-09-14T09:07:26.455 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num_target": 1, 2024-09-14T09:07:26.455 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_target": 1, 2024-09-14T09:07:26.455 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_pending": 1, 2024-09-14T09:07:26.455 INFO:teuthology.orchestra.run.smithi176.stdout: "last_pg_merge_meta": { 2024-09-14T09:07:26.455 INFO:teuthology.orchestra.run.smithi176.stdout: "source_pgid": "0.0", 2024-09-14T09:07:26.456 INFO:teuthology.orchestra.run.smithi176.stdout: "ready_epoch": 0, 2024-09-14T09:07:26.456 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_started": 0, 2024-09-14T09:07:26.456 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_clean": 0, 2024-09-14T09:07:26.456 INFO:teuthology.orchestra.run.smithi176.stdout: "source_version": "0'0", 2024-09-14T09:07:26.456 INFO:teuthology.orchestra.run.smithi176.stdout: "target_version": "0'0" 2024-09-14T09:07:26.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.456 INFO:teuthology.orchestra.run.smithi176.stdout: "last_change": "18", 2024-09-14T09:07:26.456 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend": "0", 2024-09-14T09:07:26.456 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T09:07:26.456 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T09:07:26.456 INFO:teuthology.orchestra.run.smithi176.stdout: "auid": 0, 2024-09-14T09:07:26.456 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_mode": "selfmanaged", 2024-09-14T09:07:26.456 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_seq": 0, 2024-09-14T09:07:26.457 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_epoch": 0, 2024-09-14T09:07:26.457 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_snaps": [], 2024-09-14T09:07:26.457 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps": "[]", 2024-09-14T09:07:26.457 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_bytes": 0, 2024-09-14T09:07:26.457 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_objects": 0, 2024-09-14T09:07:26.457 INFO:teuthology.orchestra.run.smithi176.stdout: "tiers": [], 2024-09-14T09:07:26.457 INFO:teuthology.orchestra.run.smithi176.stdout: "tier_of": -1, 2024-09-14T09:07:26.457 INFO:teuthology.orchestra.run.smithi176.stdout: "read_tier": -1, 2024-09-14T09:07:26.457 INFO:teuthology.orchestra.run.smithi176.stdout: "write_tier": -1, 2024-09-14T09:07:26.457 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_mode": "none", 2024-09-14T09:07:26.457 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_bytes": 0, 2024-09-14T09:07:26.457 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_objects": 0, 2024-09-14T09:07:26.457 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T09:07:26.457 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T09:07:26.458 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T09:07:26.458 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_flush_age": 0, 2024-09-14T09:07:26.458 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_evict_age": 0, 2024-09-14T09:07:26.458 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profile": "", 2024-09-14T09:07:26.458 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_params": { 2024-09-14T09:07:26.458 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "none" 2024-09-14T09:07:26.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.458 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_period": 0, 2024-09-14T09:07:26.458 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_count": 0, 2024-09-14T09:07:26.458 INFO:teuthology.orchestra.run.smithi176.stdout: "use_gmt_hitset": true, 2024-09-14T09:07:26.458 INFO:teuthology.orchestra.run.smithi176.stdout: "min_read_recency_for_promote": 0, 2024-09-14T09:07:26.458 INFO:teuthology.orchestra.run.smithi176.stdout: "min_write_recency_for_promote": 0, 2024-09-14T09:07:26.458 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T09:07:26.458 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_search_last_n": 0, 2024-09-14T09:07:26.459 INFO:teuthology.orchestra.run.smithi176.stdout: "grade_table": [], 2024-09-14T09:07:26.459 INFO:teuthology.orchestra.run.smithi176.stdout: "stripe_width": 0, 2024-09-14T09:07:26.459 INFO:teuthology.orchestra.run.smithi176.stdout: "expected_num_objects": 0, 2024-09-14T09:07:26.459 INFO:teuthology.orchestra.run.smithi176.stdout: "fast_read": false, 2024-09-14T09:07:26.459 INFO:teuthology.orchestra.run.smithi176.stdout: "options": { 2024-09-14T09:07:26.459 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_max": 32, 2024-09-14T09:07:26.459 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_min": 1 2024-09-14T09:07:26.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.459 INFO:teuthology.orchestra.run.smithi176.stdout: "application_metadata": { 2024-09-14T09:07:26.459 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": {} 2024-09-14T09:07:26.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.459 INFO:teuthology.orchestra.run.smithi176.stdout: "read_balance": { 2024-09-14T09:07:26.459 INFO:teuthology.orchestra.run.smithi176.stdout: "score_type": "Fair distribution", 2024-09-14T09:07:26.459 INFO:teuthology.orchestra.run.smithi176.stdout: "score_acting": 8, 2024-09-14T09:07:26.459 INFO:teuthology.orchestra.run.smithi176.stdout: "score_stable": 8, 2024-09-14T09:07:26.460 INFO:teuthology.orchestra.run.smithi176.stdout: "optimal_score": 0.25, 2024-09-14T09:07:26.460 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_acting": 2, 2024-09-14T09:07:26.460 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_stable": 2, 2024-09-14T09:07:26.460 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity_weighted": 1, 2024-09-14T09:07:26.460 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity": 1, 2024-09-14T09:07:26.460 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity_weighted": 1 2024-09-14T09:07:26.460 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.460 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 2, 2024-09-14T09:07:26.460 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_name": "rbd", 2024-09-14T09:07:26.460 INFO:teuthology.orchestra.run.smithi176.stdout: "create_time": "2024-09-14T09:05:15.352551+0000", 2024-09-14T09:07:26.460 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": 8193, 2024-09-14T09:07:26.461 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T09:07:26.461 INFO:teuthology.orchestra.run.smithi176.stdout: "type": 1, 2024-09-14T09:07:26.461 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 2, 2024-09-14T09:07:26.461 INFO:teuthology.orchestra.run.smithi176.stdout: "min_size": 1, 2024-09-14T09:07:26.461 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_rule": 0, 2024-09-14T09:07:26.461 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_count": 0, 2024-09-14T09:07:26.461 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_target": 0, 2024-09-14T09:07:26.461 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T09:07:26.461 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T09:07:26.461 INFO:teuthology.orchestra.run.smithi176.stdout: "is_stretch_pool": false, 2024-09-14T09:07:26.461 INFO:teuthology.orchestra.run.smithi176.stdout: "object_hash": 2, 2024-09-14T09:07:26.461 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_autoscale_mode": "off", 2024-09-14T09:07:26.461 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num": 8, 2024-09-14T09:07:26.461 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num": 8, 2024-09-14T09:07:26.462 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num_target": 8, 2024-09-14T09:07:26.462 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_target": 8, 2024-09-14T09:07:26.462 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_pending": 8, 2024-09-14T09:07:26.462 INFO:teuthology.orchestra.run.smithi176.stdout: "last_pg_merge_meta": { 2024-09-14T09:07:26.462 INFO:teuthology.orchestra.run.smithi176.stdout: "source_pgid": "0.0", 2024-09-14T09:07:26.462 INFO:teuthology.orchestra.run.smithi176.stdout: "ready_epoch": 0, 2024-09-14T09:07:26.462 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_started": 0, 2024-09-14T09:07:26.462 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_clean": 0, 2024-09-14T09:07:26.462 INFO:teuthology.orchestra.run.smithi176.stdout: "source_version": "0'0", 2024-09-14T09:07:26.462 INFO:teuthology.orchestra.run.smithi176.stdout: "target_version": "0'0" 2024-09-14T09:07:26.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.462 INFO:teuthology.orchestra.run.smithi176.stdout: "last_change": "22", 2024-09-14T09:07:26.462 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend": "0", 2024-09-14T09:07:26.462 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T09:07:26.463 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T09:07:26.463 INFO:teuthology.orchestra.run.smithi176.stdout: "auid": 0, 2024-09-14T09:07:26.463 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_mode": "selfmanaged", 2024-09-14T09:07:26.463 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_seq": 2, 2024-09-14T09:07:26.463 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_epoch": 22, 2024-09-14T09:07:26.463 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_snaps": [], 2024-09-14T09:07:26.463 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps": "[]", 2024-09-14T09:07:26.463 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_bytes": 0, 2024-09-14T09:07:26.463 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_objects": 0, 2024-09-14T09:07:26.463 INFO:teuthology.orchestra.run.smithi176.stdout: "tiers": [], 2024-09-14T09:07:26.463 INFO:teuthology.orchestra.run.smithi176.stdout: "tier_of": -1, 2024-09-14T09:07:26.463 INFO:teuthology.orchestra.run.smithi176.stdout: "read_tier": -1, 2024-09-14T09:07:26.463 INFO:teuthology.orchestra.run.smithi176.stdout: "write_tier": -1, 2024-09-14T09:07:26.463 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_mode": "none", 2024-09-14T09:07:26.464 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_bytes": 0, 2024-09-14T09:07:26.464 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_objects": 0, 2024-09-14T09:07:26.464 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T09:07:26.464 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T09:07:26.464 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T09:07:26.464 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_flush_age": 0, 2024-09-14T09:07:26.464 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_evict_age": 0, 2024-09-14T09:07:26.464 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profile": "", 2024-09-14T09:07:26.464 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_params": { 2024-09-14T09:07:26.464 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "none" 2024-09-14T09:07:26.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.464 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_period": 0, 2024-09-14T09:07:26.464 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_count": 0, 2024-09-14T09:07:26.464 INFO:teuthology.orchestra.run.smithi176.stdout: "use_gmt_hitset": true, 2024-09-14T09:07:26.465 INFO:teuthology.orchestra.run.smithi176.stdout: "min_read_recency_for_promote": 0, 2024-09-14T09:07:26.465 INFO:teuthology.orchestra.run.smithi176.stdout: "min_write_recency_for_promote": 0, 2024-09-14T09:07:26.465 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T09:07:26.465 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_search_last_n": 0, 2024-09-14T09:07:26.465 INFO:teuthology.orchestra.run.smithi176.stdout: "grade_table": [], 2024-09-14T09:07:26.465 INFO:teuthology.orchestra.run.smithi176.stdout: "stripe_width": 0, 2024-09-14T09:07:26.465 INFO:teuthology.orchestra.run.smithi176.stdout: "expected_num_objects": 0, 2024-09-14T09:07:26.465 INFO:teuthology.orchestra.run.smithi176.stdout: "fast_read": false, 2024-09-14T09:07:26.465 INFO:teuthology.orchestra.run.smithi176.stdout: "options": {}, 2024-09-14T09:07:26.465 INFO:teuthology.orchestra.run.smithi176.stdout: "application_metadata": { 2024-09-14T09:07:26.465 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd": {} 2024-09-14T09:07:26.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.465 INFO:teuthology.orchestra.run.smithi176.stdout: "read_balance": { 2024-09-14T09:07:26.465 INFO:teuthology.orchestra.run.smithi176.stdout: "score_type": "Fair distribution", 2024-09-14T09:07:26.466 INFO:teuthology.orchestra.run.smithi176.stdout: "score_acting": 1.9900000095367432, 2024-09-14T09:07:26.466 INFO:teuthology.orchestra.run.smithi176.stdout: "score_stable": 1.9900000095367432, 2024-09-14T09:07:26.466 INFO:teuthology.orchestra.run.smithi176.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T09:07:26.466 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_acting": 1.75, 2024-09-14T09:07:26.466 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_stable": 1.75, 2024-09-14T09:07:26.466 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity_weighted": 1, 2024-09-14T09:07:26.466 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity": 1, 2024-09-14T09:07:26.466 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity_weighted": 1 2024-09-14T09:07:26.466 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.466 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 3, 2024-09-14T09:07:26.466 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_name": "pool_client.0", 2024-09-14T09:07:26.467 INFO:teuthology.orchestra.run.smithi176.stdout: "create_time": "2024-09-14T09:05:25.840963+0000", 2024-09-14T09:07:26.467 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": 8193, 2024-09-14T09:07:26.467 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T09:07:26.467 INFO:teuthology.orchestra.run.smithi176.stdout: "type": 1, 2024-09-14T09:07:26.467 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 2, 2024-09-14T09:07:26.467 INFO:teuthology.orchestra.run.smithi176.stdout: "min_size": 1, 2024-09-14T09:07:26.467 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_rule": 0, 2024-09-14T09:07:26.467 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_count": 0, 2024-09-14T09:07:26.467 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_target": 0, 2024-09-14T09:07:26.467 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T09:07:26.467 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T09:07:26.467 INFO:teuthology.orchestra.run.smithi176.stdout: "is_stretch_pool": false, 2024-09-14T09:07:26.467 INFO:teuthology.orchestra.run.smithi176.stdout: "object_hash": 2, 2024-09-14T09:07:26.467 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_autoscale_mode": "off", 2024-09-14T09:07:26.468 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num": 32, 2024-09-14T09:07:26.468 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num": 32, 2024-09-14T09:07:26.468 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num_target": 32, 2024-09-14T09:07:26.468 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_target": 32, 2024-09-14T09:07:26.468 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_pending": 32, 2024-09-14T09:07:26.468 INFO:teuthology.orchestra.run.smithi176.stdout: "last_pg_merge_meta": { 2024-09-14T09:07:26.468 INFO:teuthology.orchestra.run.smithi176.stdout: "source_pgid": "0.0", 2024-09-14T09:07:26.468 INFO:teuthology.orchestra.run.smithi176.stdout: "ready_epoch": 0, 2024-09-14T09:07:26.468 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_started": 0, 2024-09-14T09:07:26.468 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_clean": 0, 2024-09-14T09:07:26.468 INFO:teuthology.orchestra.run.smithi176.stdout: "source_version": "0'0", 2024-09-14T09:07:26.468 INFO:teuthology.orchestra.run.smithi176.stdout: "target_version": "0'0" 2024-09-14T09:07:26.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.469 INFO:teuthology.orchestra.run.smithi176.stdout: "last_change": "111", 2024-09-14T09:07:26.469 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend": "0", 2024-09-14T09:07:26.469 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T09:07:26.469 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T09:07:26.469 INFO:teuthology.orchestra.run.smithi176.stdout: "auid": 0, 2024-09-14T09:07:26.469 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_mode": "selfmanaged", 2024-09-14T09:07:26.469 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_seq": 40, 2024-09-14T09:07:26.469 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_epoch": 111, 2024-09-14T09:07:26.469 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_snaps": [], 2024-09-14T09:07:26.469 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps": "[]", 2024-09-14T09:07:26.469 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_bytes": 0, 2024-09-14T09:07:26.469 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_objects": 0, 2024-09-14T09:07:26.469 INFO:teuthology.orchestra.run.smithi176.stdout: "tiers": [], 2024-09-14T09:07:26.470 INFO:teuthology.orchestra.run.smithi176.stdout: "tier_of": -1, 2024-09-14T09:07:26.470 INFO:teuthology.orchestra.run.smithi176.stdout: "read_tier": -1, 2024-09-14T09:07:26.470 INFO:teuthology.orchestra.run.smithi176.stdout: "write_tier": -1, 2024-09-14T09:07:26.470 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_mode": "none", 2024-09-14T09:07:26.470 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_bytes": 0, 2024-09-14T09:07:26.470 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_objects": 0, 2024-09-14T09:07:26.470 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T09:07:26.470 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T09:07:26.470 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T09:07:26.470 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_flush_age": 0, 2024-09-14T09:07:26.470 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_evict_age": 0, 2024-09-14T09:07:26.470 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profile": "", 2024-09-14T09:07:26.470 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_params": { 2024-09-14T09:07:26.470 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "none" 2024-09-14T09:07:26.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.471 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_period": 0, 2024-09-14T09:07:26.471 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_count": 0, 2024-09-14T09:07:26.471 INFO:teuthology.orchestra.run.smithi176.stdout: "use_gmt_hitset": true, 2024-09-14T09:07:26.471 INFO:teuthology.orchestra.run.smithi176.stdout: "min_read_recency_for_promote": 0, 2024-09-14T09:07:26.471 INFO:teuthology.orchestra.run.smithi176.stdout: "min_write_recency_for_promote": 0, 2024-09-14T09:07:26.471 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T09:07:26.471 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_search_last_n": 0, 2024-09-14T09:07:26.471 INFO:teuthology.orchestra.run.smithi176.stdout: "grade_table": [], 2024-09-14T09:07:26.471 INFO:teuthology.orchestra.run.smithi176.stdout: "stripe_width": 0, 2024-09-14T09:07:26.471 INFO:teuthology.orchestra.run.smithi176.stdout: "expected_num_objects": 0, 2024-09-14T09:07:26.471 INFO:teuthology.orchestra.run.smithi176.stdout: "fast_read": false, 2024-09-14T09:07:26.471 INFO:teuthology.orchestra.run.smithi176.stdout: "options": {}, 2024-09-14T09:07:26.471 INFO:teuthology.orchestra.run.smithi176.stdout: "application_metadata": { 2024-09-14T09:07:26.471 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd": {} 2024-09-14T09:07:26.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.472 INFO:teuthology.orchestra.run.smithi176.stdout: "read_balance": { 2024-09-14T09:07:26.472 INFO:teuthology.orchestra.run.smithi176.stdout: "score_type": "Fair distribution", 2024-09-14T09:07:26.472 INFO:teuthology.orchestra.run.smithi176.stdout: "score_acting": 1.75, 2024-09-14T09:07:26.472 INFO:teuthology.orchestra.run.smithi176.stdout: "score_stable": 1.5, 2024-09-14T09:07:26.472 INFO:teuthology.orchestra.run.smithi176.stdout: "optimal_score": 1, 2024-09-14T09:07:26.472 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_acting": 1.75, 2024-09-14T09:07:26.472 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_stable": 1.5, 2024-09-14T09:07:26.472 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity_weighted": 1, 2024-09-14T09:07:26.472 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity": 1, 2024-09-14T09:07:26.472 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity_weighted": 1 2024-09-14T09:07:26.472 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.472 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.473 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:26.473 INFO:teuthology.orchestra.run.smithi176.stdout: "osds": [ 2024-09-14T09:07:26.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.473 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 0, 2024-09-14T09:07:26.473 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "4ceedb4f-6398-467f-9562-ccf27200860b", 2024-09-14T09:07:26.473 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:07:26.473 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:07:26.473 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:07:26.473 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:07:26.473 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:07:26.473 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:07:26.473 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:07:26.473 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 116, 2024-09-14T09:07:26.473 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:07:26.474 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:07:26.474 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:07:26.474 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.474 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.474 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6824", 2024-09-14T09:07:26.474 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:07:26.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.474 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.474 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6825", 2024-09-14T09:07:26.474 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:07:26.474 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.474 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.475 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:07:26.475 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.475 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.475 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6826", 2024-09-14T09:07:26.475 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:07:26.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.475 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.475 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6827", 2024-09-14T09:07:26.475 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:07:26.475 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.475 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.476 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:07:26.476 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.476 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.476 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6830", 2024-09-14T09:07:26.476 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:07:26.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.476 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.476 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6831", 2024-09-14T09:07:26.476 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:07:26.476 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.476 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.477 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:07:26.477 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.477 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.477 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6828", 2024-09-14T09:07:26.477 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:07:26.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.477 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.477 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6829", 2024-09-14T09:07:26.477 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:07:26.477 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.477 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.478 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6825/3384710819", 2024-09-14T09:07:26.478 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6827/3384710819", 2024-09-14T09:07:26.478 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6831/3384710819", 2024-09-14T09:07:26.478 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6829/3384710819", 2024-09-14T09:07:26.478 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:07:26.478 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:07:26.478 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:07:26.478 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.478 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 1, 2024-09-14T09:07:26.478 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "604e2f62-7c13-46be-a97f-029b28c44a89", 2024-09-14T09:07:26.478 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:07:26.479 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:07:26.479 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:07:26.479 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:07:26.479 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:07:26.479 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:07:26.479 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:07:26.479 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 116, 2024-09-14T09:07:26.479 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:07:26.479 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:07:26.479 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:07:26.479 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.479 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.479 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6800", 2024-09-14T09:07:26.480 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:07:26.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.480 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.480 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6801", 2024-09-14T09:07:26.480 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:07:26.480 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.480 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.480 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:07:26.480 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.480 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.480 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6802", 2024-09-14T09:07:26.480 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:07:26.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.481 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.481 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6803", 2024-09-14T09:07:26.481 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:07:26.481 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.481 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.481 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:07:26.481 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.481 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.481 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6806", 2024-09-14T09:07:26.481 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:07:26.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.482 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.482 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6807", 2024-09-14T09:07:26.482 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:07:26.482 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.482 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.482 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:07:26.482 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.482 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.482 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6804", 2024-09-14T09:07:26.482 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:07:26.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.483 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.483 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6805", 2024-09-14T09:07:26.483 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:07:26.483 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.483 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.483 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6801/612012679", 2024-09-14T09:07:26.483 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6803/612012679", 2024-09-14T09:07:26.483 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6807/612012679", 2024-09-14T09:07:26.483 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6805/612012679", 2024-09-14T09:07:26.483 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:07:26.483 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:07:26.483 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:07:26.483 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.484 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 2, 2024-09-14T09:07:26.484 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "1f65589e-8b0a-4033-94c0-4ab66531db2b", 2024-09-14T09:07:26.484 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:07:26.484 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:07:26.484 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:07:26.484 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:07:26.484 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:07:26.484 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:07:26.484 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:07:26.484 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 113, 2024-09-14T09:07:26.484 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:07:26.484 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:07:26.484 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:07:26.485 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.485 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.485 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6808", 2024-09-14T09:07:26.485 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:07:26.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.485 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.485 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6809", 2024-09-14T09:07:26.485 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:07:26.485 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.485 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.485 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:07:26.486 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.486 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.486 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6810", 2024-09-14T09:07:26.486 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:07:26.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.486 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.486 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6811", 2024-09-14T09:07:26.486 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:07:26.486 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.486 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.486 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:07:26.487 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.487 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.487 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6814", 2024-09-14T09:07:26.487 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:07:26.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.487 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.487 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6815", 2024-09-14T09:07:26.487 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:07:26.487 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.487 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.488 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:07:26.488 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.488 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.488 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6812", 2024-09-14T09:07:26.488 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:07:26.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.488 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.488 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6813", 2024-09-14T09:07:26.488 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:07:26.488 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.488 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.489 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6809/4167379279", 2024-09-14T09:07:26.489 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6811/4167379279", 2024-09-14T09:07:26.489 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6815/4167379279", 2024-09-14T09:07:26.489 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6813/4167379279", 2024-09-14T09:07:26.489 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:07:26.489 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:07:26.489 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:07:26.489 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.489 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 3, 2024-09-14T09:07:26.489 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "209d4f65-79f9-4a7d-9d23-832a4209507f", 2024-09-14T09:07:26.489 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:07:26.489 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:07:26.489 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:07:26.490 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:07:26.490 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:07:26.490 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:07:26.490 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:07:26.490 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 116, 2024-09-14T09:07:26.490 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:07:26.490 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:07:26.490 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:07:26.490 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.490 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.490 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6816", 2024-09-14T09:07:26.490 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:07:26.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.491 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.491 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6817", 2024-09-14T09:07:26.491 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:07:26.491 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.491 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.491 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:07:26.491 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.491 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.491 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6818", 2024-09-14T09:07:26.491 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:07:26.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.492 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.492 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6819", 2024-09-14T09:07:26.492 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:07:26.492 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.492 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.492 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:07:26.492 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.492 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.492 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6822", 2024-09-14T09:07:26.492 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:07:26.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.492 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.493 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6823", 2024-09-14T09:07:26.493 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:07:26.493 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.493 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.493 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:07:26.493 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.493 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.493 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6820", 2024-09-14T09:07:26.493 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:07:26.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.493 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.494 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6821", 2024-09-14T09:07:26.494 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:07:26.494 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.494 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.494 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6817/1658209015", 2024-09-14T09:07:26.494 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6819/1658209015", 2024-09-14T09:07:26.494 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6823/1658209015", 2024-09-14T09:07:26.494 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6821/1658209015", 2024-09-14T09:07:26.494 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:07:26.494 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:07:26.494 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:07:26.494 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.495 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 4, 2024-09-14T09:07:26.495 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "8f9a8c9b-282e-4523-9e62-5824f4aba401", 2024-09-14T09:07:26.495 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:07:26.495 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:07:26.495 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:07:26.495 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:07:26.495 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:07:26.495 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:07:26.495 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:07:26.495 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 116, 2024-09-14T09:07:26.495 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:07:26.495 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:07:26.495 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:07:26.495 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.496 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.496 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6808", 2024-09-14T09:07:26.496 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:07:26.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.496 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.496 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6809", 2024-09-14T09:07:26.496 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:07:26.496 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.496 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.496 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:07:26.496 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.497 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.497 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6810", 2024-09-14T09:07:26.497 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:07:26.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.497 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.497 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6811", 2024-09-14T09:07:26.497 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:07:26.497 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.497 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.497 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:07:26.497 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.498 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.498 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6814", 2024-09-14T09:07:26.498 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:07:26.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.498 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.498 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6815", 2024-09-14T09:07:26.498 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:07:26.498 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.498 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.498 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:07:26.498 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.499 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.499 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6812", 2024-09-14T09:07:26.499 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:07:26.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.499 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.499 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6813", 2024-09-14T09:07:26.499 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:07:26.499 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.499 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.499 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6809/1576054389", 2024-09-14T09:07:26.499 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6811/1576054389", 2024-09-14T09:07:26.499 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6815/1576054389", 2024-09-14T09:07:26.499 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6813/1576054389", 2024-09-14T09:07:26.500 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:07:26.500 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:07:26.500 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:07:26.500 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.500 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 5, 2024-09-14T09:07:26.500 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80", 2024-09-14T09:07:26.500 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:07:26.500 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:07:26.500 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:07:26.500 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:07:26.500 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:07:26.501 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:07:26.501 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:07:26.501 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 116, 2024-09-14T09:07:26.501 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:07:26.501 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:07:26.501 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:07:26.501 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.501 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.501 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6824", 2024-09-14T09:07:26.501 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:07:26.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.502 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.502 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6825", 2024-09-14T09:07:26.502 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:07:26.502 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.502 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.502 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:07:26.502 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.502 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.502 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6826", 2024-09-14T09:07:26.502 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:07:26.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.502 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.503 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6827", 2024-09-14T09:07:26.503 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:07:26.503 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.503 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.503 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:07:26.503 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.503 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.503 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6830", 2024-09-14T09:07:26.503 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:07:26.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.503 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.504 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6831", 2024-09-14T09:07:26.504 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:07:26.504 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.504 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.504 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:07:26.504 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.504 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.504 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6828", 2024-09-14T09:07:26.504 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:07:26.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.505 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.505 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6829", 2024-09-14T09:07:26.505 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:07:26.505 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.505 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.505 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6825/1711292386", 2024-09-14T09:07:26.505 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6827/1711292386", 2024-09-14T09:07:26.505 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6831/1711292386", 2024-09-14T09:07:26.505 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6829/1711292386", 2024-09-14T09:07:26.505 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:07:26.505 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:07:26.505 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:07:26.506 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.506 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 6, 2024-09-14T09:07:26.506 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "59d9259d-b9dd-4326-8ab9-9d599a2bf57b", 2024-09-14T09:07:26.506 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:07:26.506 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:07:26.506 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:07:26.506 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:07:26.506 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:07:26.506 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:07:26.506 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:07:26.506 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 112, 2024-09-14T09:07:26.506 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:07:26.507 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:07:26.507 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:07:26.507 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.507 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.507 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6816", 2024-09-14T09:07:26.507 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:07:26.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.507 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.507 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6817", 2024-09-14T09:07:26.507 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:07:26.507 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.507 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.508 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:07:26.508 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.508 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.508 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6818", 2024-09-14T09:07:26.508 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:07:26.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.508 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.508 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6819", 2024-09-14T09:07:26.508 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:07:26.508 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.509 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.509 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:07:26.509 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.509 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.509 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6822", 2024-09-14T09:07:26.509 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:07:26.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.509 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.509 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6823", 2024-09-14T09:07:26.509 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:07:26.509 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.510 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.510 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:07:26.510 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.510 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.510 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6820", 2024-09-14T09:07:26.510 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:07:26.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.510 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.510 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6821", 2024-09-14T09:07:26.510 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:07:26.510 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.511 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.511 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6817/2721245607", 2024-09-14T09:07:26.511 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6819/2721245607", 2024-09-14T09:07:26.511 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6823/2721245607", 2024-09-14T09:07:26.511 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6821/2721245607", 2024-09-14T09:07:26.511 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:07:26.511 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:07:26.511 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:07:26.511 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.511 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 7, 2024-09-14T09:07:26.512 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "e2f27d42-786b-449b-a2fb-9dd28f4e9788", 2024-09-14T09:07:26.512 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:07:26.512 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:07:26.512 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:07:26.512 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:07:26.512 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:07:26.512 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:07:26.512 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:07:26.512 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 115, 2024-09-14T09:07:26.512 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:07:26.512 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:07:26.512 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:07:26.512 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.513 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.513 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6800", 2024-09-14T09:07:26.513 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:07:26.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.513 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.513 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6801", 2024-09-14T09:07:26.513 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:07:26.513 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.513 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.513 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:07:26.513 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.514 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.514 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6802", 2024-09-14T09:07:26.514 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:07:26.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.514 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.514 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6803", 2024-09-14T09:07:26.514 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:07:26.514 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.514 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.514 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:07:26.514 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.515 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.515 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6806", 2024-09-14T09:07:26.515 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:07:26.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.515 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.515 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6807", 2024-09-14T09:07:26.515 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:07:26.515 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.515 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.515 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:07:26.515 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:26.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.516 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:26.516 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6804", 2024-09-14T09:07:26.516 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:07:26.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.516 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:26.516 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6805", 2024-09-14T09:07:26.516 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:07:26.516 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.516 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.516 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6801/2122823465", 2024-09-14T09:07:26.516 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6803/2122823465", 2024-09-14T09:07:26.516 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6807/2122823465", 2024-09-14T09:07:26.517 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6805/2122823465", 2024-09-14T09:07:26.517 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:07:26.517 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:07:26.517 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:07:26.517 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.517 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.517 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:26.517 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_xinfo": [ 2024-09-14T09:07:26.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.517 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 0, 2024-09-14T09:07:26.517 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:07:26.517 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:07:26.517 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:07:26.517 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:07:26.518 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:07:26.518 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.411789+0000", 2024-09-14T09:07:26.518 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:07:26.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.518 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 1, 2024-09-14T09:07:26.518 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:07:26.518 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:07:26.518 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:07:26.518 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:07:26.518 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:07:26.518 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:05.801873+0000", 2024-09-14T09:07:26.518 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:07:26.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.519 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 2, 2024-09-14T09:07:26.519 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:07:26.519 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:07:26.519 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:07:26.519 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:07:26.519 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:07:26.519 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.206725+0000", 2024-09-14T09:07:26.519 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:07:26.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.519 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 3, 2024-09-14T09:07:26.519 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:07:26.520 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:07:26.520 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:07:26.520 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:07:26.520 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:07:26.520 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:05.882180+0000", 2024-09-14T09:07:26.520 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:07:26.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.520 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 4, 2024-09-14T09:07:26.520 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:07:26.520 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:07:26.520 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:07:26.520 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:07:26.520 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:07:26.520 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.013864+0000", 2024-09-14T09:07:26.521 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:07:26.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.521 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 5, 2024-09-14T09:07:26.521 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:07:26.521 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:07:26.521 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:07:26.521 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:07:26.521 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:07:26.521 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.652975+0000", 2024-09-14T09:07:26.521 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:07:26.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.521 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 6, 2024-09-14T09:07:26.522 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:07:26.522 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:07:26.522 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:07:26.522 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:07:26.522 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:07:26.522 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.741359+0000", 2024-09-14T09:07:26.522 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:07:26.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.522 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 7, 2024-09-14T09:07:26.522 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:07:26.522 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:07:26.522 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:07:26.523 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:07:26.523 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:07:26.523 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:05.873675+0000", 2024-09-14T09:07:26.523 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:07:26.523 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.523 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:26.523 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_upmap": [ 2024-09-14T09:07:26.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.523 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "2.3", 2024-09-14T09:07:26.523 INFO:teuthology.orchestra.run.smithi176.stdout: "osds": [ 2024-09-14T09:07:26.523 INFO:teuthology.orchestra.run.smithi176.stdout: 3, 2024-09-14T09:07:26.523 INFO:teuthology.orchestra.run.smithi176.stdout: 2 2024-09-14T09:07:26.523 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.524 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "3.4", 2024-09-14T09:07:26.524 INFO:teuthology.orchestra.run.smithi176.stdout: "osds": [ 2024-09-14T09:07:26.524 INFO:teuthology.orchestra.run.smithi176.stdout: 1, 2024-09-14T09:07:26.524 INFO:teuthology.orchestra.run.smithi176.stdout: 7 2024-09-14T09:07:26.524 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.524 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.524 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:26.524 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_upmap_items": [ 2024-09-14T09:07:26.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.524 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "3.9", 2024-09-14T09:07:26.524 INFO:teuthology.orchestra.run.smithi176.stdout: "mappings": [ 2024-09-14T09:07:26.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.525 INFO:teuthology.orchestra.run.smithi176.stdout: "from": 4, 2024-09-14T09:07:26.525 INFO:teuthology.orchestra.run.smithi176.stdout: "to": 6 2024-09-14T09:07:26.525 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.525 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.525 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "3.1b", 2024-09-14T09:07:26.525 INFO:teuthology.orchestra.run.smithi176.stdout: "mappings": [ 2024-09-14T09:07:26.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.525 INFO:teuthology.orchestra.run.smithi176.stdout: "from": 4, 2024-09-14T09:07:26.525 INFO:teuthology.orchestra.run.smithi176.stdout: "to": 6 2024-09-14T09:07:26.525 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.525 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.525 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.526 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:26.526 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_upmap_primaries": [], 2024-09-14T09:07:26.526 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_temp": [ 2024-09-14T09:07:26.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.526 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "3.4", 2024-09-14T09:07:26.526 INFO:teuthology.orchestra.run.smithi176.stdout: "osds": [ 2024-09-14T09:07:26.526 INFO:teuthology.orchestra.run.smithi176.stdout: 1 2024-09-14T09:07:26.526 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.526 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "3.13", 2024-09-14T09:07:26.526 INFO:teuthology.orchestra.run.smithi176.stdout: "osds": [ 2024-09-14T09:07:26.526 INFO:teuthology.orchestra.run.smithi176.stdout: 2, 2024-09-14T09:07:26.526 INFO:teuthology.orchestra.run.smithi176.stdout: 4 2024-09-14T09:07:26.527 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.527 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "3.15", 2024-09-14T09:07:26.527 INFO:teuthology.orchestra.run.smithi176.stdout: "osds": [ 2024-09-14T09:07:26.527 INFO:teuthology.orchestra.run.smithi176.stdout: 3 2024-09-14T09:07:26.527 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.527 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.527 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:26.527 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_temp": [], 2024-09-14T09:07:26.527 INFO:teuthology.orchestra.run.smithi176.stdout: "blocklist": {}, 2024-09-14T09:07:26.527 INFO:teuthology.orchestra.run.smithi176.stdout: "range_blocklist": {}, 2024-09-14T09:07:26.527 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profiles": { 2024-09-14T09:07:26.527 INFO:teuthology.orchestra.run.smithi176.stdout: "default": { 2024-09-14T09:07:26.527 INFO:teuthology.orchestra.run.smithi176.stdout: "crush-failure-domain": "osd", 2024-09-14T09:07:26.528 INFO:teuthology.orchestra.run.smithi176.stdout: "k": "2", 2024-09-14T09:07:26.528 INFO:teuthology.orchestra.run.smithi176.stdout: "m": "1", 2024-09-14T09:07:26.528 INFO:teuthology.orchestra.run.smithi176.stdout: "plugin": "jerasure", 2024-09-14T09:07:26.528 INFO:teuthology.orchestra.run.smithi176.stdout: "technique": "reed_sol_van" 2024-09-14T09:07:26.528 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.528 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps_queue": [], 2024-09-14T09:07:26.528 INFO:teuthology.orchestra.run.smithi176.stdout: "new_removed_snaps": [], 2024-09-14T09:07:26.528 INFO:teuthology.orchestra.run.smithi176.stdout: "new_purged_snaps": [], 2024-09-14T09:07:26.528 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_node_flags": {}, 2024-09-14T09:07:26.528 INFO:teuthology.orchestra.run.smithi176.stdout: "device_class_flags": {}, 2024-09-14T09:07:26.528 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_mode": { 2024-09-14T09:07:26.528 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_mode_enabled": false, 2024-09-14T09:07:26.529 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_bucket_count": 0, 2024-09-14T09:07:26.529 INFO:teuthology.orchestra.run.smithi176.stdout: "degraded_stretch_mode": 0, 2024-09-14T09:07:26.529 INFO:teuthology.orchestra.run.smithi176.stdout: "recovering_stretch_mode": 0, 2024-09-14T09:07:26.529 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_mode_bucket": 0 2024-09-14T09:07:26.529 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.529 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:26.530 INFO:tasks.thrashosds.thrasher:j is {'epoch': 117, 'fsid': 'c635667d-994e-454f-bcea-51b45637b2a2', 'created': '2024-09-14T09:04:57.077293+0000', 'modified': '2024-09-14T09:07:26.085280+0000', 'last_up_change': '2024-09-14T09:05:11.210835+0000', 'last_in_change': '2024-09-14T09:07:21.060249+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T09:05:11.997673+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T09:05:15.352551+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T09:05:25.840963+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 40, 'snap_epoch': 111, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '4ceedb4f-6398-467f-9562-ccf27200860b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6824', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6825', 'nonce': 3384710819}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6826', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6827', 'nonce': 3384710819}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6830', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6831', 'nonce': 3384710819}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6828', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6829', 'nonce': 3384710819}]}, 'public_addr': '172.21.15.176:6825/3384710819', 'cluster_addr': '172.21.15.176:6827/3384710819', 'heartbeat_back_addr': '172.21.15.176:6831/3384710819', 'heartbeat_front_addr': '172.21.15.176:6829/3384710819', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '604e2f62-7c13-46be-a97f-029b28c44a89', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6800', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6801', 'nonce': 612012679}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6802', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6803', 'nonce': 612012679}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6806', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6807', 'nonce': 612012679}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6804', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6805', 'nonce': 612012679}]}, 'public_addr': '172.21.15.176:6801/612012679', 'cluster_addr': '172.21.15.176:6803/612012679', 'heartbeat_back_addr': '172.21.15.176:6807/612012679', 'heartbeat_front_addr': '172.21.15.176:6805/612012679', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1f65589e-8b0a-4033-94c0-4ab66531db2b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6808', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6809', 'nonce': 4167379279}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6810', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6811', 'nonce': 4167379279}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6814', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6815', 'nonce': 4167379279}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6812', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6813', 'nonce': 4167379279}]}, 'public_addr': '172.21.15.176:6809/4167379279', 'cluster_addr': '172.21.15.176:6811/4167379279', 'heartbeat_back_addr': '172.21.15.176:6815/4167379279', 'heartbeat_front_addr': '172.21.15.176:6813/4167379279', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '209d4f65-79f9-4a7d-9d23-832a4209507f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6816', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6817', 'nonce': 1658209015}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6818', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6819', 'nonce': 1658209015}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6822', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6823', 'nonce': 1658209015}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6820', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6821', 'nonce': 1658209015}]}, 'public_addr': '172.21.15.176:6817/1658209015', 'cluster_addr': '172.21.15.176:6819/1658209015', 'heartbeat_back_addr': '172.21.15.176:6823/1658209015', 'heartbeat_front_addr': '172.21.15.176:6821/1658209015', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8f9a8c9b-282e-4523-9e62-5824f4aba401', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6808', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6809', 'nonce': 1576054389}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6810', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6811', 'nonce': 1576054389}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6814', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6815', 'nonce': 1576054389}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6812', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6813', 'nonce': 1576054389}]}, 'public_addr': '172.21.15.183:6809/1576054389', 'cluster_addr': '172.21.15.183:6811/1576054389', 'heartbeat_back_addr': '172.21.15.183:6815/1576054389', 'heartbeat_front_addr': '172.21.15.183:6813/1576054389', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6824', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6825', 'nonce': 1711292386}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6826', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6827', 'nonce': 1711292386}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6830', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6831', 'nonce': 1711292386}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6828', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6829', 'nonce': 1711292386}]}, 'public_addr': '172.21.15.183:6825/1711292386', 'cluster_addr': '172.21.15.183:6827/1711292386', 'heartbeat_back_addr': '172.21.15.183:6831/1711292386', 'heartbeat_front_addr': '172.21.15.183:6829/1711292386', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '59d9259d-b9dd-4326-8ab9-9d599a2bf57b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6816', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6817', 'nonce': 2721245607}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6818', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6819', 'nonce': 2721245607}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6822', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6823', 'nonce': 2721245607}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6820', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6821', 'nonce': 2721245607}]}, 'public_addr': '172.21.15.183:6817/2721245607', 'cluster_addr': '172.21.15.183:6819/2721245607', 'heartbeat_back_addr': '172.21.15.183:6823/2721245607', 'heartbeat_front_addr': '172.21.15.183:6821/2721245607', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'e2f27d42-786b-449b-a2fb-9dd28f4e9788', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 115, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6800', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6801', 'nonce': 2122823465}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6802', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6803', 'nonce': 2122823465}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6806', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6807', 'nonce': 2122823465}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6804', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6805', 'nonce': 2122823465}]}, 'public_addr': '172.21.15.183:6801/2122823465', 'cluster_addr': '172.21.15.183:6803/2122823465', 'heartbeat_back_addr': '172.21.15.183:6807/2122823465', 'heartbeat_front_addr': '172.21.15.183:6805/2122823465', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.411789+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:05.801873+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.206725+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:05.882180+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.013864+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.652975+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.741359+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:05.873675+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.3', 'osds': [3, 2]}, {'pgid': '3.4', 'osds': [1, 7]}], 'pg_upmap_items': [{'pgid': '3.9', 'mappings': [{'from': 4, 'to': 6}]}, {'pgid': '3.1b', 'mappings': [{'from': 4, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.4', 'osds': [1]}, {'pgid': '3.13', 'osds': [2, 4]}, {'pgid': '3.15', 'osds': [3]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T09:07:26.530 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:07:26.531 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:26.451+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:26.574 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:26.575+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:26.653 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:26.655+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:26.664 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:26.664 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:26.664 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:26.664 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:26.680 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:07:26.754 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:26.759+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:26.838 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:26.838 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:26.838 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:26.839 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:26.839 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:26.854 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:07:26.855 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:26.855+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:26.858 INFO:teuthology.orchestra.run.smithi176.stdout:5462 write 0x495d55f thru 0x496ba0f (0xe4b1 bytes) 2024-09-14T09:07:26.863 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:07:26.864 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:07:26.882 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-14T09:07:26.840865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1288569639,"num_objects":1994,"num_object_clones":605,"num_object_copies":3988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":76,"num_objects_unfound":0,"num_objects_dirty":1994,"num_whiteouts":710,"num_read":148134,"num_read_kb":736587,"num_write":19816,"num_write_kb":323881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3036,"num_bytes_recovered":157326827,"num_keys_recovered":535,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15652,"ondisk_log_size":15652,"up":82,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":442604,"kb_used_data":68136,"kb_used_omap":12,"kb_used_meta":374259,"kb_avail":754532116,"statfs":{"total":773094113280,"available":772640886784,"internally_reserved":0,"allocated":69771264,"data_stored":489817756,"data_compressed":16085537,"data_compressed_allocated":42635264,"data_compressed_original":474396394,"omap_allocated":12751,"internal_metadata":383241777},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,256,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":116929938,"num_objects":281,"num_object_clones":67,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":-2,"num_objects_degraded":2,"num_objects_misplaced":76,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":84,"num_read":20580,"num_read_kb":67078,"num_write":3028,"num_write_kb":37075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1057,"num_bytes_recovered":42506013,"num_keys_recovered":167,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.993700"},"pg_stats":[{"pgid":"3.1f","version":"111'1526","reported_seq":5980,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T09:07:21.841250+0000","last_change":"2024-09-14T09:06:41.348384+0000","last_active":"2024-09-14T09:07:21.841250+0000","last_peered":"2024-09-14T09:07:21.841250+0000","last_clean":"2024-09-14T09:07:21.841250+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:07:21.841250+0000","last_undegraded":"2024-09-14T09:07:21.841250+0000","last_fullsized":"2024-09-14T09:07:21.841250+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1526,"log_dups_size":0,"ondisk_log_size":1526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:09.268982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001387512,"stat_sum":{"num_bytes":19547248,"num_objects":58,"num_object_clones":18,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":21,"num_read":11903,"num_read_kb":21263,"num_write":2504,"num_write_kb":7452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"117'324","reported_seq":3747,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T09:07:26.092261+0000","last_change":"2024-09-14T09:05:29.039503+0000","last_active":"2024-09-14T09:07:26.092261+0000","last_peered":"2024-09-14T09:07:26.092261+0000","last_clean":"2024-09-14T09:07:26.092261+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:07:26.092261+0000","last_undegraded":"2024-09-14T09:07:26.092261+0000","last_fullsized":"2024-09-14T09:07:26.092261+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:47:29.581979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001657176,"stat_sum":{"num_bytes":45813783,"num_objects":86,"num_object_clones":28,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":30,"num_read":3312,"num_read_kb":29433,"num_write":353,"num_write_kb":12955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"111'478","reported_seq":4572,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T09:07:21.856496+0000","last_change":"2024-09-14T09:05:28.835298+0000","last_active":"2024-09-14T09:07:21.856496+0000","last_peered":"2024-09-14T09:07:21.856496+0000","last_clean":"2024-09-14T09:07:21.856496+0000","last_became_active":"2024-09-14T09:05:26.835372+0000","last_became_peered":"2024-09-14T09:05:26.835372+0000","last_unstale":"2024-09-14T09:07:21.856496+0000","last_undegraded":"2024-09-14T09:07:21.856496+0000","last_fullsized":"2024-09-14T09:07:21.856496+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:39:55.686148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014149329999999999,"stat_sum":{"num_bytes":64178109,"num_objects":70,"num_object_clones":22,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":23,"num_read":4764,"num_read_kb":55952,"num_write":607,"num_write_kb":11304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"111'628","reported_seq":4252,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T09:07:21.850137+0000","last_change":"2024-09-14T09:06:44.867658+0000","last_active":"2024-09-14T09:07:21.850137+0000","last_peered":"2024-09-14T09:07:21.850137+0000","last_clean":"2024-09-14T09:07:21.850137+0000","last_became_active":"2024-09-14T09:06:44.862945+0000","last_became_peered":"2024-09-14T09:06:44.862945+0000","last_unstale":"2024-09-14T09:07:21.850137+0000","last_undegraded":"2024-09-14T09:07:21.850137+0000","last_fullsized":"2024-09-14T09:07:21.850137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:29:11.089635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014661419999999999,"stat_sum":{"num_bytes":44032933,"num_objects":62,"num_object_clones":21,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":23,"num_read":6362,"num_read_kb":25684,"num_write":840,"num_write_kb":9043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"111'582","reported_seq":4749,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T09:07:21.839028+0000","last_change":"2024-09-14T09:07:06.843637+0000","last_active":"2024-09-14T09:07:21.839028+0000","last_peered":"2024-09-14T09:07:21.839028+0000","last_clean":"2024-09-14T09:07:21.839028+0000","last_became_active":"2024-09-14T09:07:06.842337+0000","last_became_peered":"2024-09-14T09:07:06.842337+0000","last_unstale":"2024-09-14T09:07:21.839028+0000","last_undegraded":"2024-09-14T09:07:21.839028+0000","last_fullsized":"2024-09-14T09:07:21.839028+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:58:53.730540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060373399999999998,"stat_sum":{"num_bytes":35482818,"num_objects":67,"num_object_clones":25,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":24,"num_read":5765,"num_read_kb":22784,"num_write":779,"num_write_kb":15455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":14145319,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"114'579","reported_seq":4668,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T09:07:26.095867+0000","last_change":"2024-09-14T09:06:44.872988+0000","last_active":"2024-09-14T09:07:26.095867+0000","last_peered":"2024-09-14T09:07:26.095867+0000","last_clean":"2024-09-14T09:07:26.095867+0000","last_became_active":"2024-09-14T09:06:44.869103+0000","last_became_peered":"2024-09-14T09:06:44.869103+0000","last_unstale":"2024-09-14T09:07:26.095867+0000","last_undegraded":"2024-09-14T09:07:26.095867+0000","last_fullsized":"2024-09-14T09:07:26.095867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:30:11.721163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020845550000000001,"stat_sum":{"num_bytes":46021401,"num_objects":60,"num_object_clones":16,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":18,"num_read":5538,"num_read_kb":27393,"num_write":873,"num_write_kb":12964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"114'263","reported_seq":2396,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T09:07:26.094333+0000","last_change":"2024-09-14T09:05:28.822565+0000","last_active":"2024-09-14T09:07:26.094333+0000","last_peered":"2024-09-14T09:07:26.094333+0000","last_clean":"2024-09-14T09:07:26.094333+0000","last_became_active":"2024-09-14T09:05:26.816132+0000","last_became_peered":"2024-09-14T09:05:26.816132+0000","last_unstale":"2024-09-14T09:07:26.094333+0000","last_undegraded":"2024-09-14T09:07:26.094333+0000","last_fullsized":"2024-09-14T09:07:26.094333+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:34:47.276260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092420299999999996,"stat_sum":{"num_bytes":24281410,"num_objects":48,"num_object_clones":15,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":17,"num_read":1998,"num_read_kb":7781,"num_write":298,"num_write_kb":5487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"114'409","reported_seq":4293,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T09:07:26.092078+0000","last_change":"2024-09-14T09:06:41.342724+0000","last_active":"2024-09-14T09:07:26.092078+0000","last_peered":"2024-09-14T09:07:26.092078+0000","last_clean":"2024-09-14T09:07:26.092078+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:07:26.092078+0000","last_undegraded":"2024-09-14T09:07:26.092078+0000","last_fullsized":"2024-09-14T09:07:26.092078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:45:32.033554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081579899999999997,"stat_sum":{"num_bytes":61228569,"num_objects":72,"num_object_clones":23,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":22,"num_read":5004,"num_read_kb":13560,"num_write":395,"num_write_kb":6846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"111'742","reported_seq":5605,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T09:07:21.841382+0000","last_change":"2024-09-14T09:05:28.839928+0000","last_active":"2024-09-14T09:07:21.841382+0000","last_peered":"2024-09-14T09:07:21.841382+0000","last_clean":"2024-09-14T09:07:21.841382+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:07:21.841382+0000","last_undegraded":"2024-09-14T09:07:21.841382+0000","last_fullsized":"2024-09-14T09:07:21.841382+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:17:53.856515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014343089999999999,"stat_sum":{"num_bytes":40693925,"num_objects":49,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":16,"num_read":7788,"num_read_kb":27533,"num_write":1018,"num_write_kb":8055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"111'450","reported_seq":2756,"reported_epoch":112,"state":"remapped+peering","last_fresh":"2024-09-14T09:07:21.079645+0000","last_change":"2024-09-14T09:07:21.855055+0000","last_active":"2024-09-14T09:07:21.079645+0000","last_peered":"2024-09-14T09:07:20.887287+0000","last_clean":"2024-09-14T09:07:20.887287+0000","last_became_active":"2024-09-14T09:07:08.846820+0000","last_became_peered":"2024-09-14T09:07:08.846820+0000","last_unstale":"2024-09-14T09:07:21.079645+0000","last_undegraded":"2024-09-14T09:07:21.079645+0000","last_fullsized":"2024-09-14T09:07:21.079645+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015277190000000001,"stat_sum":{"num_bytes":20124530,"num_objects":53,"num_object_clones":12,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":15,"num_read":4003,"num_read_kb":14080,"num_write":577,"num_write_kb":6008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":4094848,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"114'938","reported_seq":2942,"reported_epoch":117,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-14T09:07:26.146792+0000","last_change":"2024-09-14T09:07:25.815827+0000","last_active":"2024-09-14T09:07:26.146792+0000","last_peered":"2024-09-14T09:07:26.146792+0000","last_clean":"2024-09-14T09:07:20.854707+0000","last_became_active":"2024-09-14T09:07:22.991330+0000","last_became_peered":"2024-09-14T09:07:22.991330+0000","last_unstale":"2024-09-14T09:07:26.146792+0000","last_undegraded":"2024-09-14T09:07:26.146792+0000","last_fullsized":"2024-09-14T09:07:22.857715+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":938,"log_dups_size":0,"ondisk_log_size":938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098594000000000004,"stat_sum":{"num_bytes":19546320,"num_objects":55,"num_object_clones":16,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":21,"num_read":6101,"num_read_kb":11186,"num_write":1448,"num_write_kb":6691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":2790638,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[3],"avail_no_missing":["3","4"],"object_location_counts":[{"shards":"3","objects":41},{"shards":"3,4","objects":14}],"blocked_by":[],"up_primary":7,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"117'494","reported_seq":2758,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T09:07:26.113954+0000","last_change":"2024-09-14T09:07:26.110470+0000","last_active":"2024-09-14T09:07:26.113954+0000","last_peered":"2024-09-14T09:07:26.113954+0000","last_clean":"2024-09-14T09:07:26.113954+0000","last_became_active":"2024-09-14T09:07:26.106734+0000","last_became_peered":"2024-09-14T09:07:26.106734+0000","last_unstale":"2024-09-14T09:07:26.113954+0000","last_undegraded":"2024-09-14T09:07:26.113954+0000","last_fullsized":"2024-09-14T09:07:26.113954+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:51:58.592492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014079679999999999,"stat_sum":{"num_bytes":25034255,"num_objects":65,"num_object_clones":20,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":25,"num_read":2973,"num_read_kb":7239,"num_write":575,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":2686146,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":214,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T09:07:21.857666+0000","last_change":"2024-09-14T09:07:21.857426+0000","last_active":"2024-09-14T09:07:21.857666+0000","last_peered":"2024-09-14T09:07:21.857666+0000","last_clean":"2024-09-14T09:07:21.857666+0000","last_became_active":"2024-09-14T09:07:21.856579+0000","last_became_peered":"2024-09-14T09:07:21.856579+0000","last_unstale":"2024-09-14T09:07:21.857666+0000","last_undegraded":"2024-09-14T09:07:21.857666+0000","last_fullsized":"2024-09-14T09:07:21.857666+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:56.694930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"111'470","reported_seq":4721,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T09:07:21.839495+0000","last_change":"2024-09-14T09:05:28.839614+0000","last_active":"2024-09-14T09:07:21.839495+0000","last_peered":"2024-09-14T09:07:21.839495+0000","last_clean":"2024-09-14T09:07:21.839495+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:07:21.839495+0000","last_undegraded":"2024-09-14T09:07:21.839495+0000","last_fullsized":"2024-09-14T09:07:21.839495+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:25:50.513152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014245869999999999,"stat_sum":{"num_bytes":43479799,"num_objects":60,"num_object_clones":17,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":23,"num_read":6461,"num_read_kb":42137,"num_write":524,"num_write_kb":7305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":209,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T09:07:26.094745+0000","last_change":"2024-09-14T09:05:19.314029+0000","last_active":"2024-09-14T09:07:26.094745+0000","last_peered":"2024-09-14T09:07:26.094745+0000","last_clean":"2024-09-14T09:07:26.094745+0000","last_became_active":"2024-09-14T09:05:17.282890+0000","last_became_peered":"2024-09-14T09:05:17.282890+0000","last_unstale":"2024-09-14T09:07:26.094745+0000","last_undegraded":"2024-09-14T09:07:26.094745+0000","last_fullsized":"2024-09-14T09:07:26.094745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:25.304105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"114'470","reported_seq":3801,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T09:07:26.103842+0000","last_change":"2024-09-14T09:07:26.103749+0000","last_active":"2024-09-14T09:07:26.103842+0000","last_peered":"2024-09-14T09:07:26.103842+0000","last_clean":"2024-09-14T09:07:26.103842+0000","last_became_active":"2024-09-14T09:07:26.102666+0000","last_became_peered":"2024-09-14T09:07:26.102666+0000","last_unstale":"2024-09-14T09:07:26.103842+0000","last_undegraded":"2024-09-14T09:07:26.103842+0000","last_fullsized":"2024-09-14T09:07:26.103842+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:38.198182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019213749999999999,"stat_sum":{"num_bytes":41643891,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":14,"num_read":4171,"num_read_kb":14543,"num_write":589,"num_write_kb":6764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":8334712,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T09:07:26.103137+0000","last_change":"2024-09-14T09:07:21.863290+0000","last_active":"2024-09-14T09:07:26.103137+0000","last_peered":"2024-09-14T09:07:26.103137+0000","last_clean":"2024-09-14T09:07:26.103137+0000","last_became_active":"2024-09-14T09:07:21.862352+0000","last_became_peered":"2024-09-14T09:07:21.862352+0000","last_unstale":"2024-09-14T09:07:26.103137+0000","last_undegraded":"2024-09-14T09:07:26.103137+0000","last_fullsized":"2024-09-14T09:07:26.103137+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:48:30.332891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"117'641","reported_seq":4446,"reported_epoch":117,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-14T09:07:26.287840+0000","last_change":"2024-09-14T09:07:26.213626+0000","last_active":"2024-09-14T09:07:26.287840+0000","last_peered":"2024-09-14T09:07:26.287840+0000","last_clean":"2024-09-14T09:07:20.960794+0000","last_became_active":"2024-09-14T09:07:22.985120+0000","last_became_peered":"2024-09-14T09:07:22.985120+0000","last_unstale":"2024-09-14T09:07:26.287840+0000","last_undegraded":"2024-09-14T09:07:22.852603+0000","last_fullsized":"2024-09-14T09:07:22.848817+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014087959999999999,"stat_sum":{"num_bytes":24639386,"num_objects":73,"num_object_clones":25,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":62,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":32,"num_read":6347,"num_read_kb":19793,"num_write":865,"num_write_kb":8471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":4490741,"num_keys_recovered":57,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1","objects":9},{"shards":"1,2","objects":62}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":210,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T09:07:26.094575+0000","last_change":"2024-09-14T09:05:19.314129+0000","last_active":"2024-09-14T09:07:26.094575+0000","last_peered":"2024-09-14T09:07:26.094575+0000","last_clean":"2024-09-14T09:07:26.094575+0000","last_became_active":"2024-09-14T09:05:17.295196+0000","last_became_peered":"2024-09-14T09:05:17.295196+0000","last_unstale":"2024-09-14T09:07:26.094575+0000","last_undegraded":"2024-09-14T09:07:26.094575+0000","last_fullsized":"2024-09-14T09:07:26.094575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:35.269115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011613800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"111'522","reported_seq":2017,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T09:07:21.856548+0000","last_change":"2024-09-14T09:05:28.835976+0000","last_active":"2024-09-14T09:07:21.856548+0000","last_peered":"2024-09-14T09:07:21.856548+0000","last_clean":"2024-09-14T09:07:21.856548+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:07:21.856548+0000","last_undegraded":"2024-09-14T09:07:21.856548+0000","last_fullsized":"2024-09-14T09:07:21.856548+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:37:08.558933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017480779999999999,"stat_sum":{"num_bytes":58240649,"num_objects":74,"num_object_clones":25,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":32,"num_read":2803,"num_read_kb":15431,"num_write":698,"num_write_kb":12813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"117'505","reported_seq":3299,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T09:07:26.096204+0000","last_change":"2024-09-14T09:06:46.882194+0000","last_active":"2024-09-14T09:07:26.096204+0000","last_peered":"2024-09-14T09:07:26.096204+0000","last_clean":"2024-09-14T09:07:26.096204+0000","last_became_active":"2024-09-14T09:06:46.880786+0000","last_became_peered":"2024-09-14T09:06:46.880786+0000","last_unstale":"2024-09-14T09:07:26.096204+0000","last_undegraded":"2024-09-14T09:07:26.096204+0000","last_fullsized":"2024-09-14T09:07:26.096204+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:48:27.425272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014128,"stat_sum":{"num_bytes":28932831,"num_objects":62,"num_object_clones":20,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":21,"num_read":4823,"num_read_kb":19425,"num_write":578,"num_write_kb":11737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":205,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T09:07:21.850062+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:07:21.850062+0000","last_peered":"2024-09-14T09:07:21.850062+0000","last_clean":"2024-09-14T09:07:21.850062+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:07:21.850062+0000","last_undegraded":"2024-09-14T09:07:21.850062+0000","last_fullsized":"2024-09-14T09:07:21.850062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"117'271","reported_seq":1835,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T09:07:26.094418+0000","last_change":"2024-09-14T09:05:28.822791+0000","last_active":"2024-09-14T09:07:26.094418+0000","last_peered":"2024-09-14T09:07:26.094418+0000","last_clean":"2024-09-14T09:07:26.094418+0000","last_became_active":"2024-09-14T09:05:26.817109+0000","last_became_peered":"2024-09-14T09:05:26.817109+0000","last_unstale":"2024-09-14T09:07:26.094418+0000","last_undegraded":"2024-09-14T09:07:26.094418+0000","last_fullsized":"2024-09-14T09:07:26.094418+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:11:25.957291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011728439999999999,"stat_sum":{"num_bytes":25392903,"num_objects":66,"num_object_clones":20,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":25,"num_read":1625,"num_read_kb":10253,"num_write":283,"num_write_kb":4459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":203,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T09:07:21.841136+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:07:21.841136+0000","last_peered":"2024-09-14T09:07:21.841136+0000","last_clean":"2024-09-14T09:07:21.841136+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:07:21.841136+0000","last_undegraded":"2024-09-14T09:07:21.841136+0000","last_fullsized":"2024-09-14T09:07:21.841136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"111'324","reported_seq":3272,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T09:07:21.838784+0000","last_change":"2024-09-14T09:06:45.868939+0000","last_active":"2024-09-14T09:07:21.838784+0000","last_peered":"2024-09-14T09:07:21.838784+0000","last_clean":"2024-09-14T09:07:21.838784+0000","last_became_active":"2024-09-14T09:06:45.868194+0000","last_became_peered":"2024-09-14T09:06:45.868194+0000","last_unstale":"2024-09-14T09:07:21.838784+0000","last_undegraded":"2024-09-14T09:07:21.838784+0000","last_fullsized":"2024-09-14T09:07:21.838784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:05:11.216524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074815999999999999,"stat_sum":{"num_bytes":48330152,"num_objects":60,"num_object_clones":21,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":27,"num_read":2650,"num_read_kb":22719,"num_write":315,"num_write_kb":13762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T09:07:26.102487+0000","last_change":"2024-09-14T09:07:21.856721+0000","last_active":"2024-09-14T09:07:26.102487+0000","last_peered":"2024-09-14T09:07:26.102487+0000","last_clean":"2024-09-14T09:07:26.102487+0000","last_became_active":"2024-09-14T09:07:21.856091+0000","last_became_peered":"2024-09-14T09:07:21.856091+0000","last_unstale":"2024-09-14T09:07:26.102487+0000","last_undegraded":"2024-09-14T09:07:26.102487+0000","last_fullsized":"2024-09-14T09:07:26.102487+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:16:17.683341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"114'305","reported_seq":2214,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T09:07:26.092553+0000","last_change":"2024-09-14T09:05:29.038389+0000","last_active":"2024-09-14T09:07:26.092553+0000","last_peered":"2024-09-14T09:07:26.092553+0000","last_clean":"2024-09-14T09:07:26.092553+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:07:26.092553+0000","last_undegraded":"2024-09-14T09:07:26.092553+0000","last_fullsized":"2024-09-14T09:07:26.092553+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:11:10.248212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001479076,"stat_sum":{"num_bytes":33120438,"num_objects":60,"num_object_clones":18,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":22,"num_read":2014,"num_read_kb":10421,"num_write":351,"num_write_kb":6652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":112,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T09:07:26.092673+0000","last_change":"2024-09-14T09:06:25.227644+0000","last_active":"2024-09-14T09:07:26.092673+0000","last_peered":"2024-09-14T09:07:26.092673+0000","last_clean":"2024-09-14T09:07:26.092673+0000","last_became_active":"2024-09-14T09:06:25.222344+0000","last_became_peered":"2024-09-14T09:06:25.222344+0000","last_unstale":"2024-09-14T09:07:26.092673+0000","last_undegraded":"2024-09-14T09:07:26.092673+0000","last_fullsized":"2024-09-14T09:07:26.092673+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:23:33.414859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T09:07:26.102163+0000","last_change":"2024-09-14T09:07:21.876180+0000","last_active":"2024-09-14T09:07:26.102163+0000","last_peered":"2024-09-14T09:07:26.102163+0000","last_clean":"2024-09-14T09:07:26.102163+0000","last_became_active":"2024-09-14T09:07:21.860117+0000","last_became_peered":"2024-09-14T09:07:21.860117+0000","last_unstale":"2024-09-14T09:07:26.102163+0000","last_undegraded":"2024-09-14T09:07:26.102163+0000","last_fullsized":"2024-09-14T09:07:26.102163+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:58:36.142971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"114'364","reported_seq":4125,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T09:07:26.092155+0000","last_change":"2024-09-14T09:05:29.037923+0000","last_active":"2024-09-14T09:07:26.092155+0000","last_peered":"2024-09-14T09:07:26.092155+0000","last_clean":"2024-09-14T09:07:26.092155+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:07:26.092155+0000","last_undegraded":"2024-09-14T09:07:26.092155+0000","last_fullsized":"2024-09-14T09:07:26.092155+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:33:35.172879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099199800000000001,"stat_sum":{"num_bytes":53735258,"num_objects":61,"num_object_clones":16,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":21,"num_read":4430,"num_read_kb":34768,"num_write":376,"num_write_kb":12584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"111'313","reported_seq":2276,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T09:07:21.849786+0000","last_change":"2024-09-14T09:07:06.854914+0000","last_active":"2024-09-14T09:07:21.849786+0000","last_peered":"2024-09-14T09:07:21.849786+0000","last_clean":"2024-09-14T09:07:21.849786+0000","last_became_active":"2024-09-14T09:07:06.851020+0000","last_became_peered":"2024-09-14T09:07:06.851020+0000","last_unstale":"2024-09-14T09:07:21.849786+0000","last_undegraded":"2024-09-14T09:07:21.849786+0000","last_fullsized":"2024-09-14T09:07:21.849786+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:00:49.568888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001266028,"stat_sum":{"num_bytes":39336494,"num_objects":63,"num_object_clones":18,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":22,"num_read":1674,"num_read_kb":13916,"num_write":292,"num_write_kb":18611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":12315549,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"111'536","reported_seq":4259,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T09:07:21.848295+0000","last_change":"2024-09-14T09:06:45.870752+0000","last_active":"2024-09-14T09:07:21.848295+0000","last_peered":"2024-09-14T09:07:21.848295+0000","last_clean":"2024-09-14T09:07:21.848295+0000","last_became_active":"2024-09-14T09:06:45.869266+0000","last_became_peered":"2024-09-14T09:06:45.869266+0000","last_unstale":"2024-09-14T09:07:21.848295+0000","last_undegraded":"2024-09-14T09:07:21.848295+0000","last_fullsized":"2024-09-14T09:07:21.848295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:07:04.580426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185731,"stat_sum":{"num_bytes":54116513,"num_objects":60,"num_object_clones":19,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":24,"num_read":5060,"num_read_kb":37686,"num_write":725,"num_write_kb":18778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"115'395","reported_seq":8605,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T09:07:26.092763+0000","last_change":"2024-09-14T09:06:02.837388+0000","last_active":"2024-09-14T09:07:26.092763+0000","last_peered":"2024-09-14T09:07:26.092763+0000","last_clean":"2024-09-14T09:07:26.092763+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:07:26.092763+0000","last_undegraded":"2024-09-14T09:07:26.092763+0000","last_fullsized":"2024-09-14T09:07:26.092763+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:21:00.113612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001585295,"stat_sum":{"num_bytes":48749754,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":18,"num_read":9337,"num_read_kb":58223,"num_write":395,"num_write_kb":10388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"111'302","reported_seq":2132,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T09:07:21.854674+0000","last_change":"2024-09-14T09:05:28.834647+0000","last_active":"2024-09-14T09:07:21.854674+0000","last_peered":"2024-09-14T09:07:21.854674+0000","last_clean":"2024-09-14T09:07:21.854674+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:07:21.854674+0000","last_undegraded":"2024-09-14T09:07:21.854674+0000","last_fullsized":"2024-09-14T09:07:21.854674+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:34:03.733334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244290000000001,"stat_sum":{"num_bytes":30577727,"num_objects":63,"num_object_clones":19,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":20,"num_read":2175,"num_read_kb":10173,"num_write":314,"num_write_kb":9087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"114'340","reported_seq":4574,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-14T09:07:26.102339+0000","last_change":"2024-09-14T09:07:26.102023+0000","last_active":"2024-09-14T09:07:26.102023+0000","last_peered":"2024-09-14T09:05:26.325409+0000","last_clean":"2024-09-14T09:05:26.325409+0000","last_became_active":"2024-09-14T09:07:22.991938+0000","last_became_peered":"2024-09-14T09:07:22.991938+0000","last_unstale":"2024-09-14T09:07:26.102339+0000","last_undegraded":"2024-09-14T09:07:26.102339+0000","last_fullsized":"2024-09-14T09:07:26.102339+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072987200000000001,"stat_sum":{"num_bytes":32393738,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":4556,"num_read_kb":28552,"num_write":392,"num_write_kb":16162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":12503355,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"114'294","reported_seq":2674,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-09-14T09:07:26.101965+0000","last_change":"2024-09-14T09:07:26.101650+0000","last_active":"2024-09-14T09:07:26.101650+0000","last_peered":"2024-09-14T09:05:26.325409+0000","last_clean":"2024-09-14T09:05:26.325409+0000","last_became_active":"2024-09-14T09:07:22.976753+0000","last_became_peered":"2024-09-14T09:07:22.976753+0000","last_unstale":"2024-09-14T09:07:26.101965+0000","last_undegraded":"2024-09-14T09:07:26.101965+0000","last_fullsized":"2024-09-14T09:07:26.101965+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057596100000000005,"stat_sum":{"num_bytes":38930044,"num_objects":53,"num_object_clones":13,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":17,"num_read":2564,"num_read_kb":15494,"num_write":280,"num_write_kb":6516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":8595878,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"114'355","reported_seq":2925,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T09:07:26.102741+0000","last_change":"2024-09-14T09:07:25.117894+0000","last_active":"2024-09-14T09:07:26.102741+0000","last_peered":"2024-09-14T09:07:26.102741+0000","last_clean":"2024-09-14T09:07:26.102741+0000","last_became_active":"2024-09-14T09:07:25.104940+0000","last_became_peered":"2024-09-14T09:07:25.104940+0000","last_unstale":"2024-09-14T09:07:26.102741+0000","last_undegraded":"2024-09-14T09:07:26.102741+0000","last_fullsized":"2024-09-14T09:07:26.102741+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:45.041381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000909463,"stat_sum":{"num_bytes":33685643,"num_objects":68,"num_object_clones":18,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":22,"num_read":2295,"num_read_kb":9557,"num_write":365,"num_write_kb":9574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":333,"num_bytes_recovered":11864981,"num_keys_recovered":45,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"111'372","reported_seq":5048,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-14T09:07:21.852382+0000","last_change":"2024-09-14T09:06:00.166357+0000","last_active":"2024-09-14T09:07:21.852382+0000","last_peered":"2024-09-14T09:07:21.852382+0000","last_clean":"2024-09-14T09:07:21.852382+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:07:21.852382+0000","last_undegraded":"2024-09-14T09:07:21.852382+0000","last_fullsized":"2024-09-14T09:07:21.852382+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:20:23.086057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011381609999999999,"stat_sum":{"num_bytes":96588690,"num_objects":69,"num_object_clones":22,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":26,"num_read":4713,"num_read_kb":57585,"num_write":454,"num_write_kb":19634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"114'439","reported_seq":3749,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T09:07:26.102371+0000","last_change":"2024-09-14T09:07:25.107931+0000","last_active":"2024-09-14T09:07:26.102371+0000","last_peered":"2024-09-14T09:07:26.102371+0000","last_clean":"2024-09-14T09:07:26.102371+0000","last_became_active":"2024-09-14T09:07:25.105011+0000","last_became_peered":"2024-09-14T09:07:25.105011+0000","last_unstale":"2024-09-14T09:07:26.102371+0000","last_undegraded":"2024-09-14T09:07:26.102371+0000","last_fullsized":"2024-09-14T09:07:26.102371+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:23:44.713009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092949299999999999,"stat_sum":{"num_bytes":18845984,"num_objects":74,"num_object_clones":26,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":30,"num_read":4031,"num_read_kb":9129,"num_write":532,"num_write_kb":6121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":7405746,"num_keys_recovered":72,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"110'582","reported_seq":3458,"reported_epoch":112,"state":"remapped+peering","last_fresh":"2024-09-14T09:07:21.075105+0000","last_change":"2024-09-14T09:07:21.841685+0000","last_active":"2024-09-14T09:07:21.075105+0000","last_peered":"2024-09-14T09:07:20.077709+0000","last_clean":"2024-09-14T09:07:20.077709+0000","last_became_active":"2024-09-14T09:07:08.840121+0000","last_became_peered":"2024-09-14T09:07:08.840121+0000","last_unstale":"2024-09-14T09:07:21.075105+0000","last_undegraded":"2024-09-14T09:07:21.075105+0000","last_fullsized":"2024-09-14T09:07:21.075105+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018357099999999999,"stat_sum":{"num_bytes":63490824,"num_objects":60,"num_object_clones":20,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":24,"num_read":4762,"num_read_kb":14072,"num_write":775,"num_write_kb":8515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":4867868,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"111'364","reported_seq":5404,"reported_epoch":113,"state":"remapped+peering","last_fresh":"2024-09-14T09:07:21.840828+0000","last_change":"2024-09-14T09:07:21.840828+0000","last_active":"2024-09-14T09:07:21.840828+0000","last_peered":"2024-09-14T09:07:20.866500+0000","last_clean":"2024-09-14T09:07:20.866500+0000","last_became_active":"2024-09-14T09:07:09.867934+0000","last_became_peered":"2024-09-14T09:07:09.867934+0000","last_unstale":"2024-09-14T09:07:21.840828+0000","last_undegraded":"2024-09-14T09:07:21.840828+0000","last_fullsized":"2024-09-14T09:07:21.840828+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061978900000000004,"stat_sum":{"num_bytes":27763233,"num_objects":56,"num_object_clones":18,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":19,"num_read":6066,"num_read_kb":28767,"num_write":343,"num_write_kb":9114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":15353859,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1287979252,"num_objects":1990,"num_object_clones":605,"num_object_copies":3980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":76,"num_objects_unfound":0,"num_objects_dirty":1990,"num_whiteouts":710,"num_read":148068,"num_read_kb":736532,"num_write":19713,"num_write_kb":322251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3031,"num_bytes_recovered":156146091,"num_keys_recovered":535,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":59240448,"data_stored":485397878,"data_compressed":16209804,"data_compressed_allocated":42799104,"data_compressed_original":474953450,"omap_allocated":0,"internal_metadata":0},"log_size":15577,"ondisk_log_size":15577,"up":64,"acting":60,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20292,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48868,"kb_used_data":9356,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322972,"statfs":{"total":96636764160,"available":96586723328,"internally_reserved":0,"allocated":9580544,"data_stored":74972804,"data_compressed":2241442,"data_compressed_allocated":6529024,"data_compressed_original":73317931,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75656,"kb_used_data":8372,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94296184,"statfs":{"total":96636764160,"available":96559292416,"internally_reserved":0,"allocated":8572928,"data_stored":62179142,"data_compressed":1866596,"data_compressed_allocated":5296128,"data_compressed_original":60373479,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,256],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61664,"kb_used_data":4108,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310176,"statfs":{"total":96636764160,"available":96573620224,"internally_reserved":0,"allocated":4206592,"data_stored":24286158,"data_compressed":708769,"data_compressed_allocated":2134016,"data_compressed_original":23198971,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509473,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59940,"kb_used_data":10696,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94311900,"statfs":{"total":96636764160,"available":96575385600,"internally_reserved":0,"allocated":10952704,"data_stored":83086114,"data_compressed":2576101,"data_compressed_allocated":6905856,"data_compressed_original":80703204,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542174,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60512,"kb_used_data":7112,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311328,"statfs":{"total":96636764160,"available":96574799872,"internally_reserved":0,"allocated":7282688,"data_stored":46480700,"data_compressed":1563530,"data_compressed_allocated":4116480,"data_compressed_original":44611976,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542175,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42452,"kb_used_data":11256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329388,"statfs":{"total":96636764160,"available":96593293312,"internally_reserved":0,"allocated":11526144,"data_stored":82807818,"data_compressed":2755177,"data_compressed_allocated":7241728,"data_compressed_original":80323661,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542175,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59080,"kb_used_data":9836,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94312760,"statfs":{"total":96636764160,"available":96576266240,"internally_reserved":0,"allocated":10072064,"data_stored":70206550,"data_compressed":2571577,"data_compressed_allocated":6131712,"data_compressed_original":67943545,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34432,"kb_used_data":7400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337408,"statfs":{"total":96636764160,"available":96601505792,"internally_reserved":0,"allocated":7577600,"data_stored":45798470,"data_compressed":1802345,"data_compressed_allocated":4280320,"data_compressed_original":43923627,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9568256,"data_stored":81876592,"data_compressed":2569889,"data_compressed_allocated":6868992,"data_compressed_original":80129764,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6213632,"data_stored":45164812,"data_compressed":1802345,"data_compressed_allocated":4280320,"data_compressed_original":43923627,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2867200,"data_stored":23660485,"data_compressed":708769,"data_compressed_allocated":2134016,"data_compressed_original":23198971,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8704000,"data_stored":69568488,"data_compressed":2571577,"data_compressed_allocated":6131712,"data_compressed_original":67943545,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10178560,"data_stored":82179845,"data_compressed":2755177,"data_compressed_allocated":7241728,"data_compressed_original":80323661,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7229440,"data_stored":61553450,"data_compressed":1866596,"data_compressed_allocated":5296128,"data_compressed_original":60373479,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8212480,"data_stored":73763282,"data_compressed":2234402,"data_compressed_allocated":6492160,"data_compressed_original":72728107,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6266880,"data_stored":47630924,"data_compressed":1701049,"data_compressed_allocated":4354048,"data_compressed_original":46332296,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:07:26.883 INFO:tasks.thrashosds.thrasher:Setting 3.19 to [0, 2, 5, 1] 2024-09-14T09:07:26.884 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.19', '0', '2', '5', '1'] 2024-09-14T09:07:26.884 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.19 0 2 5 1 2024-09-14T09:07:26.957 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:26.959+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:26.972 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:26.973 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:26.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:26.973 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:26.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.973 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:26.973 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:26.973 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.888991843, 2024-09-14T09:07:26.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:07:26.973 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.973 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.973 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.973 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.973 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.974 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.974 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:07:26.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.974 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:26.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:26.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:07:26.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:07:26.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:07:26.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:07:26.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:07:26.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:07:26.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:07:26.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:07:26.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:07:26.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:07:26.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:07:26.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:07:26.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:07:26.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:07:26.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:07:26.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:07:26.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:07:26.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:07:26.978 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.978 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.978 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.978 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:26.978 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:26.978 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.888937185, 2024-09-14T09:07:26.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:07:26.978 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.978 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.978 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.978 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.978 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.979 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.979 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:07:26.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.979 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:26.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:26.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:26.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:07:26.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:07:26.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:07:26.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:07:26.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:07:26.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:07:26.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:07:26.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:07:26.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:07:26.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:07:26.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:07:26.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:07:26.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:07:26.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:07:26.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:07:26.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:07:26.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:07:26.983 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.983 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.983 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.983 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:26.983 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:26.983 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.888831811, 2024-09-14T09:07:26.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:07:26.983 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.983 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.983 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.983 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.983 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.983 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.984 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:07:26.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.984 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:26.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:26.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:07:26.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:07:26.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:07:26.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:07:26.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:07:26.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:07:26.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:07:26.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:07:26.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:07:26.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:07:26.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:07:26.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:07:26.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:07:26.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:07:26.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:07:26.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:07:26.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:07:26.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:07:26.988 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.988 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.988 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.988 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:26.988 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:26.988 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.888475047, 2024-09-14T09:07:26.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:07:26.988 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.988 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.988 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.989 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.989 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.989 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.989 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:07:26.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.989 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:26.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:26.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:07:26.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:07:26.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:07:26.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:07:26.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:07:26.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:07:26.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:07:26.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:07:26.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:07:26.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:07:26.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:07:26.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:07:26.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:07:26.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:07:26.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:07:26.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:07:26.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:07:26.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:07:26.993 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.993 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.993 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.993 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:26.993 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:26.993 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.888390309, 2024-09-14T09:07:26.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:07:26.994 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.994 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.994 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.994 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.994 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.994 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.994 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:07:26.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.994 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:26.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:26.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:26.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:26.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:26.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:26.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:26.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:07:26.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:07:26.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:26.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:07:26.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:07:26.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:26.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:07:26.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:07:26.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:26.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:07:26.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:07:26.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:07:26.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:07:26.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:26.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:07:26.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:07:26.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:26.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:07:26.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:07:26.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:26.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:07:26.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:07:26.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:26.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:07:26.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:07:26.998 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.998 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:26.998 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:26.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:26.999 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:26.999 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:26.999 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 113.888302623, 2024-09-14T09:07:26.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:07:26.999 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:26.999 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:26.999 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:26.999 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:26.999 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:26.999 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:26.999 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:07:26.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:26.999 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:27.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:27.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:27.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:27.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:27.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:27.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:07:27.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:07:27.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:27.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:07:27.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:07:27.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:27.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:07:27.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:07:27.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:07:27.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:07:27.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:27.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:07:27.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:07:27.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:27.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:07:27.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:07:27.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:27.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:07:27.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:07:27.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:27.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:07:27.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:07:27.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:27.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:07:27.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:07:27.003 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.003 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:27.004 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.004 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:07:27.004 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:27.004 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.812010635999997, 2024-09-14T09:07:27.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:07:27.004 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:27.004 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:27.004 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:27.004 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:27.004 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:27.004 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.005 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:07:27.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.005 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:27.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:27.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:27.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:27.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:27.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:27.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:07:27.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:07:27.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:27.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:07:27.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:07:27.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:27.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:07:27.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:07:27.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:07:27.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:07:27.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:27.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:07:27.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:07:27.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:27.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:07:27.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:07:27.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:27.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:07:27.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:07:27.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:27.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:07:27.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:07:27.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:27.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:07:27.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:07:27.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:27.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:07:27.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:07:27.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:27.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:07:27.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:07:27.009 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.009 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:27.009 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.010 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:84137 3.7 3:e15b49d9:::rbd_data.115e77b799c3.00000000000000e0:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:07:27.010 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:27.010 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.649263238000003, 2024-09-14T09:07:27.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11205707500000001, 2024-09-14T09:07:27.010 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:27.010 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:27.010 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:27.010 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:27.010 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:27.010 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.010 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 84137 2024-09-14T09:07:27.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.010 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:27.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:27.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:27.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:27.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:27.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:27.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323362+0000", 2024-09-14T09:07:27.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0839999999999999e-06 2024-09-14T09:07:27.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:27.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323370+0000", 2024-09-14T09:07:27.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0180000000000001e-06 2024-09-14T09:07:27.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:27.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323388+0000", 2024-09-14T09:07:27.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8326999999999999e-05 2024-09-14T09:07:27.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323397+0000", 2024-09-14T09:07:27.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1940000000000004e-06 2024-09-14T09:07:27.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:27.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323452+0000", 2024-09-14T09:07:27.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5043000000000001e-05 2024-09-14T09:07:27.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:27.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323550+0000", 2024-09-14T09:07:27.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8201000000000004e-05 2024-09-14T09:07:27.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:27.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.328273+0000", 2024-09-14T09:07:27.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0047227079999999999 2024-09-14T09:07:27.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:27.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435232+0000", 2024-09-14T09:07:27.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.106959339 2024-09-14T09:07:27.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:27.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435256+0000", 2024-09-14T09:07:27.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3774e-05 2024-09-14T09:07:27.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:27.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435314+0000", 2024-09-14T09:07:27.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7636999999999997e-05 2024-09-14T09:07:27.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:27.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435417+0000", 2024-09-14T09:07:27.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010275 2024-09-14T09:07:27.015 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.015 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:27.015 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.015 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:97796 3.15 3:aaaa5f7f:::rbd_data.115e113dc8ca.0000000000000029:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:07:27.016 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:27.016 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.878141766000001, 2024-09-14T09:07:27.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14537731300000001, 2024-09-14T09:07:27.016 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:27.016 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:27.016 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:27.016 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:27.016 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:27.016 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.016 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97796 2024-09-14T09:07:27.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.016 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:27.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:27.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:27.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:27.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:27.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:27.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094482+0000", 2024-09-14T09:07:27.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6400000000000001e-07 2024-09-14T09:07:27.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:27.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094485+0000", 2024-09-14T09:07:27.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.693e-06 2024-09-14T09:07:27.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:27.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094494+0000", 2024-09-14T09:07:27.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9590000000000001e-06 2024-09-14T09:07:27.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094502+0000", 2024-09-14T09:07:27.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3180000000000001e-06 2024-09-14T09:07:27.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:27.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094567+0000", 2024-09-14T09:07:27.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5526999999999998e-05 2024-09-14T09:07:27.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:27.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094726+0000", 2024-09-14T09:07:27.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015842499999999999 2024-09-14T09:07:27.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:07:27.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.101940+0000", 2024-09-14T09:07:27.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0072146559999999998 2024-09-14T09:07:27.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:27.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239665+0000", 2024-09-14T09:07:27.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.137724501 2024-09-14T09:07:27.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:27.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239726+0000", 2024-09-14T09:07:27.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1550999999999994e-05 2024-09-14T09:07:27.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.021 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:27.021 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239800+0000", 2024-09-14T09:07:27.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3318000000000005e-05 2024-09-14T09:07:27.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.021 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:27.021 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239858+0000", 2024-09-14T09:07:27.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8600999999999997e-05 2024-09-14T09:07:27.021 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.021 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:27.021 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.021 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:97796 3.15 e106/103 ondisk, result = 0)", 2024-09-14T09:07:27.022 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:27.022 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.855795897, 2024-09-14T09:07:27.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123043465, 2024-09-14T09:07:27.022 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:27.022 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:27.022 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:27.022 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:27.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:27.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:27.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:27.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:27.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:27.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116828+0000", 2024-09-14T09:07:27.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8100000000000002e-07 2024-09-14T09:07:27.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:27.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116831+0000", 2024-09-14T09:07:27.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3009999999999998e-06 2024-09-14T09:07:27.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:27.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116841+0000", 2024-09-14T09:07:27.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7829999999999993e-06 2024-09-14T09:07:27.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116851+0000", 2024-09-14T09:07:27.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0135999999999999e-05 2024-09-14T09:07:27.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:27.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239707+0000", 2024-09-14T09:07:27.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12285581499999999 2024-09-14T09:07:27.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:27.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239718+0000", 2024-09-14T09:07:27.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1804000000000001e-05 2024-09-14T09:07:27.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:27.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239870+0000", 2024-09-14T09:07:27.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015194500000000001 2024-09-14T09:07:27.025 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.025 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:27.025 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.026 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107601 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:27.026 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403134+0000", 2024-09-14T09:07:27.026 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.5694891609999999, 2024-09-14T09:07:27.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111355695, 2024-09-14T09:07:27.026 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:27.026 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:27.026 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:27.026 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:27.026 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:27.026 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.026 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107601 2024-09-14T09:07:27.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.026 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:27.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:27.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403134+0000", 2024-09-14T09:07:27.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:27.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403134+0000", 2024-09-14T09:07:27.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:27.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403134+0000", 2024-09-14T09:07:27.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2000000000000001e-07 2024-09-14T09:07:27.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:27.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403136+0000", 2024-09-14T09:07:27.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7430000000000001e-06 2024-09-14T09:07:27.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:27.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403140+0000", 2024-09-14T09:07:27.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:07:27.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403144+0000", 2024-09-14T09:07:27.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.766e-06 2024-09-14T09:07:27.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:27.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514478+0000", 2024-09-14T09:07:27.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111334212 2024-09-14T09:07:27.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:27.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514489+0000", 2024-09-14T09:07:27.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1236999999999999e-05 2024-09-14T09:07:27.029 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.029 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:27.030 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.030 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107603 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:27.030 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403149+0000", 2024-09-14T09:07:27.030 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.5694738930000001, 2024-09-14T09:07:27.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111385997, 2024-09-14T09:07:27.030 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:27.030 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:27.030 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:27.030 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:27.030 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:27.030 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.030 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107603 2024-09-14T09:07:27.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.031 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:27.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:27.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403149+0000", 2024-09-14T09:07:27.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:27.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403149+0000", 2024-09-14T09:07:27.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.031 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:27.031 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403149+0000", 2024-09-14T09:07:27.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8299999999999998e-07 2024-09-14T09:07:27.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:27.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403152+0000", 2024-09-14T09:07:27.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T09:07:27.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:27.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403158+0000", 2024-09-14T09:07:27.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2940000000000002e-06 2024-09-14T09:07:27.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403165+0000", 2024-09-14T09:07:27.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.849e-06 2024-09-14T09:07:27.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:27.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514524+0000", 2024-09-14T09:07:27.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111358923 2024-09-14T09:07:27.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:27.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514535+0000", 2024-09-14T09:07:27.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0913999999999999e-05 2024-09-14T09:07:27.033 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.034 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:27.034 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.034 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107605 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:27.034 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403207+0000", 2024-09-14T09:07:27.034 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.5694161279999999, 2024-09-14T09:07:27.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11137927, 2024-09-14T09:07:27.034 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:27.034 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:27.034 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:27.034 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:27.034 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:27.034 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.035 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107605 2024-09-14T09:07:27.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.035 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:27.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:27.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403207+0000", 2024-09-14T09:07:27.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:27.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403207+0000", 2024-09-14T09:07:27.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:27.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403207+0000", 2024-09-14T09:07:27.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1800000000000005e-07 2024-09-14T09:07:27.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:27.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403211+0000", 2024-09-14T09:07:27.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3629999999999998e-06 2024-09-14T09:07:27.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:27.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403222+0000", 2024-09-14T09:07:27.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1606e-05 2024-09-14T09:07:27.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403230+0000", 2024-09-14T09:07:27.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6760000000000007e-06 2024-09-14T09:07:27.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:27.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514572+0000", 2024-09-14T09:07:27.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111341991 2024-09-14T09:07:27.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:27.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514586+0000", 2024-09-14T09:07:27.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3916e-05 2024-09-14T09:07:27.038 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.038 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:27.038 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.038 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107607 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:27.038 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403242+0000", 2024-09-14T09:07:27.038 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.569381146, 2024-09-14T09:07:27.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111392375, 2024-09-14T09:07:27.038 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:27.038 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:27.038 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:27.038 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:27.038 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:27.039 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.039 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107607 2024-09-14T09:07:27.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.039 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:27.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:27.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403242+0000", 2024-09-14T09:07:27.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:27.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403242+0000", 2024-09-14T09:07:27.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:27.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403242+0000", 2024-09-14T09:07:27.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2600000000000002e-07 2024-09-14T09:07:27.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:27.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403245+0000", 2024-09-14T09:07:27.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.128e-06 2024-09-14T09:07:27.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:27.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403254+0000", 2024-09-14T09:07:27.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6219999999999998e-06 2024-09-14T09:07:27.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.041 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.041 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403261+0000", 2024-09-14T09:07:27.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1999999999999997e-06 2024-09-14T09:07:27.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.041 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:27.041 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514621+0000", 2024-09-14T09:07:27.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111359977 2024-09-14T09:07:27.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.041 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:27.041 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514634+0000", 2024-09-14T09:07:27.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2822e-05 2024-09-14T09:07:27.042 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.042 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:27.042 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.042 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107609 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:27.042 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403271+0000", 2024-09-14T09:07:27.042 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.5693518850000001, 2024-09-14T09:07:27.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111392167, 2024-09-14T09:07:27.042 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:27.042 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:27.042 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:27.042 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:27.042 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:27.043 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.043 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107609 2024-09-14T09:07:27.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.043 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:27.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:27.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403271+0000", 2024-09-14T09:07:27.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:27.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403271+0000", 2024-09-14T09:07:27.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:27.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403271+0000", 2024-09-14T09:07:27.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3499999999999996e-07 2024-09-14T09:07:27.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:27.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403319+0000", 2024-09-14T09:07:27.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7450000000000001e-05 2024-09-14T09:07:27.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:27.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403332+0000", 2024-09-14T09:07:27.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3066e-05 2024-09-14T09:07:27.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403341+0000", 2024-09-14T09:07:27.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5739999999999996e-06 2024-09-14T09:07:27.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:27.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514657+0000", 2024-09-14T09:07:27.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11131680100000001 2024-09-14T09:07:27.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:27.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514663+0000", 2024-09-14T09:07:27.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6409999999999999e-06 2024-09-14T09:07:27.046 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.046 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:27.046 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.046 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107613 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:27.046 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403384+0000", 2024-09-14T09:07:27.046 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.5692387239999999, 2024-09-14T09:07:27.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11137016399999999, 2024-09-14T09:07:27.046 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:27.046 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:27.047 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:27.047 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:27.047 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:27.047 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.047 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107613 2024-09-14T09:07:27.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.047 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:27.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:27.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403384+0000", 2024-09-14T09:07:27.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:27.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403384+0000", 2024-09-14T09:07:27.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:27.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403385+0000", 2024-09-14T09:07:27.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5600000000000005e-07 2024-09-14T09:07:27.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:27.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403388+0000", 2024-09-14T09:07:27.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0139999999999999e-06 2024-09-14T09:07:27.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:27.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403397+0000", 2024-09-14T09:07:27.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1980000000000007e-06 2024-09-14T09:07:27.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403404+0000", 2024-09-14T09:07:27.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3669999999999999e-06 2024-09-14T09:07:27.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:27.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514742+0000", 2024-09-14T09:07:27.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11133736299999999 2024-09-14T09:07:27.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:27.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514754+0000", 2024-09-14T09:07:27.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2566e-05 2024-09-14T09:07:27.050 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.050 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:27.050 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.050 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107615 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:27.050 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403415+0000", 2024-09-14T09:07:27.051 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.5692081390000001, 2024-09-14T09:07:27.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11137188200000001, 2024-09-14T09:07:27.051 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:27.051 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:27.051 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:27.051 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:27.051 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:27.051 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.051 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107615 2024-09-14T09:07:27.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.051 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:27.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:27.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403415+0000", 2024-09-14T09:07:27.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:27.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403415+0000", 2024-09-14T09:07:27.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:27.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403415+0000", 2024-09-14T09:07:27.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1500000000000004e-07 2024-09-14T09:07:27.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:27.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403418+0000", 2024-09-14T09:07:27.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9170000000000002e-06 2024-09-14T09:07:27.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:27.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403427+0000", 2024-09-14T09:07:27.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1740000000000006e-06 2024-09-14T09:07:27.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403435+0000", 2024-09-14T09:07:27.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2849999999999997e-06 2024-09-14T09:07:27.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:27.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514776+0000", 2024-09-14T09:07:27.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11134168799999999 2024-09-14T09:07:27.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:27.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514786+0000", 2024-09-14T09:07:27.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0203e-05 2024-09-14T09:07:27.054 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.054 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:27.054 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.054 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107617 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:27.055 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403444+0000", 2024-09-14T09:07:27.055 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.569178602, 2024-09-14T09:07:27.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111359085, 2024-09-14T09:07:27.055 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:27.055 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:27.055 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:27.055 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:27.055 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:27.055 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.055 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107617 2024-09-14T09:07:27.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.055 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:27.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:27.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403444+0000", 2024-09-14T09:07:27.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:27.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403444+0000", 2024-09-14T09:07:27.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:27.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403445+0000", 2024-09-14T09:07:27.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0399999999999996e-07 2024-09-14T09:07:27.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:27.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403447+0000", 2024-09-14T09:07:27.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7989999999999998e-06 2024-09-14T09:07:27.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:27.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403459+0000", 2024-09-14T09:07:27.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1086e-05 2024-09-14T09:07:27.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403465+0000", 2024-09-14T09:07:27.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9709999999999998e-06 2024-09-14T09:07:27.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:27.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514798+0000", 2024-09-14T09:07:27.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111332495 2024-09-14T09:07:27.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:27.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514803+0000", 2024-09-14T09:07:27.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2299999999999999e-06 2024-09-14T09:07:27.058 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.058 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:27.058 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.058 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107619 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:27.059 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403476+0000", 2024-09-14T09:07:27.059 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.569146688, 2024-09-14T09:07:27.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111354988, 2024-09-14T09:07:27.059 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:27.059 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:27.059 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:27.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:27.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:27.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.059 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107619 2024-09-14T09:07:27.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.059 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:27.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:27.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403476+0000", 2024-09-14T09:07:27.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:27.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403476+0000", 2024-09-14T09:07:27.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:27.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403477+0000", 2024-09-14T09:07:27.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8199999999999999e-07 2024-09-14T09:07:27.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:27.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403480+0000", 2024-09-14T09:07:27.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9979999999999999e-06 2024-09-14T09:07:27.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:27.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403493+0000", 2024-09-14T09:07:27.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3478999999999999e-05 2024-09-14T09:07:27.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403508+0000", 2024-09-14T09:07:27.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4419e-05 2024-09-14T09:07:27.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:27.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514821+0000", 2024-09-14T09:07:27.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111313379 2024-09-14T09:07:27.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:27.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514831+0000", 2024-09-14T09:07:27.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0030999999999999e-05 2024-09-14T09:07:27.062 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.062 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:27.062 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.062 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107625 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:27.063 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403586+0000", 2024-09-14T09:07:27.063 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 3.5690367150000002, 2024-09-14T09:07:27.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111365832, 2024-09-14T09:07:27.063 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:27.063 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:27.063 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:27.063 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:27.063 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:27.063 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.063 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107625 2024-09-14T09:07:27.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.063 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:27.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:27.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403586+0000", 2024-09-14T09:07:27.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:27.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403586+0000", 2024-09-14T09:07:27.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:27.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:27.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.404090+0000", 2024-09-14T09:07:27.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050411300000000004 2024-09-14T09:07:27.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:27.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.404096+0000", 2024-09-14T09:07:27.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2790000000000004e-06 2024-09-14T09:07:27.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:27.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.404110+0000", 2024-09-14T09:07:27.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3784000000000001e-05 2024-09-14T09:07:27.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.404120+0000", 2024-09-14T09:07:27.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8039999999999996e-06 2024-09-14T09:07:27.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:27.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514944+0000", 2024-09-14T09:07:27.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.110824099 2024-09-14T09:07:27.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:27.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:27.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:27.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514952+0000", 2024-09-14T09:07:27.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7530000000000001e-06 2024-09-14T09:07:27.066 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.066 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:27.066 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.066 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:27.066 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:27.066 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:27.068 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:07:27.068 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:27.067+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:27.118 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T09:07:27.169 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:27.167+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:27.212 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:27.212 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:27.213 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:07:27.213 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:27.218 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:07:27.270 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:27.271+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:27.335 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:27.335 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:27.335 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:07:27.335 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:07:27.335 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:27.349 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:07:27.394 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:27.399+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:27.472 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:27.475+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:27.474 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:07:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:07:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:07:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:76803 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-14T09:07:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:43.858877+0000", 2024-09-14T09:07:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 43.618453230999997, 2024-09-14T09:07:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0137577900000001, 2024-09-14T09:07:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.476 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 76803 2024-09-14T09:07:27.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.476 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858877+0000", 2024-09-14T09:07:27.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858877+0000", 2024-09-14T09:07:27.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858879+0000", 2024-09-14T09:07:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.097e-06 2024-09-14T09:07:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858887+0000", 2024-09-14T09:07:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6510000000000001e-06 2024-09-14T09:07:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858903+0000", 2024-09-14T09:07:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6560999999999999e-05 2024-09-14T09:07:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858912+0000", 2024-09-14T09:07:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2019999999999993e-06 2024-09-14T09:07:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858964+0000", 2024-09-14T09:07:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2046999999999998e-05 2024-09-14T09:07:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858972+0000", 2024-09-14T09:07:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6029999999999999e-06 2024-09-14T09:07:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:44.871832+0000", 2024-09-14T09:07:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.012860109 2024-09-14T09:07:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:44.872248+0000", 2024-09-14T09:07:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00041549300000000001 2024-09-14T09:07:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:44.872635+0000", 2024-09-14T09:07:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038702700000000002 2024-09-14T09:07:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:84403 3.14 3:2fc85a9a:::rbd_data.115e77b799c3.0000000000000018:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:07:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:53.878069+0000", 2024-09-14T09:07:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.599261362, 2024-09-14T09:07:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.128852405, 2024-09-14T09:07:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 84403 2024-09-14T09:07:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878069+0000", 2024-09-14T09:07:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878069+0000", 2024-09-14T09:07:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878071+0000", 2024-09-14T09:07:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0899999999999999e-06 2024-09-14T09:07:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878082+0000", 2024-09-14T09:07:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1226999999999999e-05 2024-09-14T09:07:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878108+0000", 2024-09-14T09:07:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5568999999999999e-05 2024-09-14T09:07:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878122+0000", 2024-09-14T09:07:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.488e-05 2024-09-14T09:07:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878247+0000", 2024-09-14T09:07:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000124039 2024-09-14T09:07:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878387+0000", 2024-09-14T09:07:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014041600000000001 2024-09-14T09:07:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.886261+0000", 2024-09-14T09:07:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0078738499999999999 2024-09-14T09:07:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006598+0000", 2024-09-14T09:07:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.120337316 2024-09-14T09:07:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006724+0000", 2024-09-14T09:07:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012601600000000001 2024-09-14T09:07:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006827+0000", 2024-09-14T09:07:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103299 2024-09-14T09:07:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006921+0000", 2024-09-14T09:07:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3702999999999994e-05 2024-09-14T09:07:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:93272 3.3 3:c49b4fbc:::rbd_data.115e7770ddc5.000000000000001a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:07:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.971357+0000", 2024-09-14T09:07:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.505973303000001, 2024-09-14T09:07:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.112268252, 2024-09-14T09:07:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 93272 2024-09-14T09:07:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971357+0000", 2024-09-14T09:07:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971357+0000", 2024-09-14T09:07:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971360+0000", 2024-09-14T09:07:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8250000000000001e-06 2024-09-14T09:07:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971371+0000", 2024-09-14T09:07:27.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1537e-05 2024-09-14T09:07:27.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971399+0000", 2024-09-14T09:07:27.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8274999999999999e-05 2024-09-14T09:07:27.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971417+0000", 2024-09-14T09:07:27.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7241999999999999e-05 2024-09-14T09:07:27.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971537+0000", 2024-09-14T09:07:27.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012010600000000001 2024-09-14T09:07:27.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971685+0000", 2024-09-14T09:07:27.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014777599999999999 2024-09-14T09:07:27.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:27.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.979172+0000", 2024-09-14T09:07:27.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0074876810000000004 2024-09-14T09:07:27.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:27.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083438+0000", 2024-09-14T09:07:27.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.104265969 2024-09-14T09:07:27.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:27.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083466+0000", 2024-09-14T09:07:27.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7665999999999999e-05 2024-09-14T09:07:27.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:27.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083553+0000", 2024-09-14T09:07:27.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6809000000000002e-05 2024-09-14T09:07:27.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083625+0000", 2024-09-14T09:07:27.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2366000000000005e-05 2024-09-14T09:07:27.494 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.494 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.494 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.494 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:96694 3.11 3:8a7e3c58:::rbd_data.115e63832e35.000000000000001c:head [stat,truncate 131072] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-14T09:07:27.494 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:07.004845+0000", 2024-09-14T09:07:27.494 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.472485287000001, 2024-09-14T09:07:27.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.85224771499999996, 2024-09-14T09:07:27.494 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.494 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.495 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:27.495 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:27.495 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:27.495 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.495 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 96694 2024-09-14T09:07:27.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.495 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004845+0000", 2024-09-14T09:07:27.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004845+0000", 2024-09-14T09:07:27.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004848+0000", 2024-09-14T09:07:27.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:07:27.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004862+0000", 2024-09-14T09:07:27.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4426e-05 2024-09-14T09:07:27.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004903+0000", 2024-09-14T09:07:27.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0447000000000001e-05 2024-09-14T09:07:27.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004926+0000", 2024-09-14T09:07:27.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3260000000000001e-05 2024-09-14T09:07:27.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.005104+0000", 2024-09-14T09:07:27.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000178018 2024-09-14T09:07:27.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:27.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.005120+0000", 2024-09-14T09:07:27.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6422999999999998e-05 2024-09-14T09:07:27.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.851775+0000", 2024-09-14T09:07:27.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.84665435499999997 2024-09-14T09:07:27.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.852176+0000", 2024-09-14T09:07:27.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000400787 2024-09-14T09:07:27.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-09-14T09:07:27.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.852662+0000", 2024-09-14T09:07:27.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048621899999999997 2024-09-14T09:07:27.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:27.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.855261+0000", 2024-09-14T09:07:27.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025991479999999999 2024-09-14T09:07:27.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:27.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.856839+0000", 2024-09-14T09:07:27.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015779819999999999 2024-09-14T09:07:27.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:27.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.856993+0000", 2024-09-14T09:07:27.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015431900000000001 2024-09-14T09:07:27.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.857093+0000", 2024-09-14T09:07:27.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9314000000000005e-05 2024-09-14T09:07:27.501 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.501 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.501 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.502 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:97612 3.e 3:7298f82d:::rbd_data.115e113dc8ca.00000000000000e8:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:07:27.502 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:11.399529+0000", 2024-09-14T09:07:27.502 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.077801233999999, 2024-09-14T09:07:27.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.18555535000000001, 2024-09-14T09:07:27.502 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.502 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.503 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:27.503 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:27.503 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:27.503 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.503 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 97612 2024-09-14T09:07:27.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.503 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399529+0000", 2024-09-14T09:07:27.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399529+0000", 2024-09-14T09:07:27.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399532+0000", 2024-09-14T09:07:27.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6599999999999999e-06 2024-09-14T09:07:27.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399544+0000", 2024-09-14T09:07:27.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2527000000000001e-05 2024-09-14T09:07:27.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399574+0000", 2024-09-14T09:07:27.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.97e-05 2024-09-14T09:07:27.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399588+0000", 2024-09-14T09:07:27.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4653e-05 2024-09-14T09:07:27.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399740+0000", 2024-09-14T09:07:27.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015180900000000001 2024-09-14T09:07:27.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399952+0000", 2024-09-14T09:07:27.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000211435 2024-09-14T09:07:27.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:27.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.411686+0000", 2024-09-14T09:07:27.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011733883000000001 2024-09-14T09:07:27.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:27.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584791+0000", 2024-09-14T09:07:27.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.17310596 2024-09-14T09:07:27.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:27.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584915+0000", 2024-09-14T09:07:27.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012330800000000001 2024-09-14T09:07:27.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:27.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.585010+0000", 2024-09-14T09:07:27.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5216000000000005e-05 2024-09-14T09:07:27.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.585084+0000", 2024-09-14T09:07:27.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4198999999999999e-05 2024-09-14T09:07:27.508 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.508 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.508 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.508 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:97612 3.e e106/103 ondisk, result = 0)", 2024-09-14T09:07:27.508 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:11.451700+0000", 2024-09-14T09:07:27.508 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.025629940000002, 2024-09-14T09:07:27.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13339934000000001, 2024-09-14T09:07:27.509 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.509 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.509 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:27.509 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451700+0000", 2024-09-14T09:07:27.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451700+0000", 2024-09-14T09:07:27.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451702+0000", 2024-09-14T09:07:27.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.34e-06 2024-09-14T09:07:27.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451713+0000", 2024-09-14T09:07:27.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1e-05 2024-09-14T09:07:27.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451751+0000", 2024-09-14T09:07:27.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7762999999999997e-05 2024-09-14T09:07:27.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451770+0000", 2024-09-14T09:07:27.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8626000000000001e-05 2024-09-14T09:07:27.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584888+0000", 2024-09-14T09:07:27.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13311764500000001 2024-09-14T09:07:27.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584903+0000", 2024-09-14T09:07:27.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.499e-05 2024-09-14T09:07:27.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.585099+0000", 2024-09-14T09:07:27.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019697599999999999 2024-09-14T09:07:27.513 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.513 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.513 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.513 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104478 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 470016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:27.513 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680713+0000", 2024-09-14T09:07:27.513 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.7966169330000001, 2024-09-14T09:07:27.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10242654800000001, 2024-09-14T09:07:27.514 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.514 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.514 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:27.514 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:27.514 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:27.514 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.514 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104478 2024-09-14T09:07:27.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.514 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680713+0000", 2024-09-14T09:07:27.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680713+0000", 2024-09-14T09:07:27.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680714+0000", 2024-09-14T09:07:27.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3300000000000001e-07 2024-09-14T09:07:27.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680717+0000", 2024-09-14T09:07:27.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:07:27.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680729+0000", 2024-09-14T09:07:27.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1643e-05 2024-09-14T09:07:27.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680736+0000", 2024-09-14T09:07:27.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8730000000000007e-06 2024-09-14T09:07:27.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.782300+0000", 2024-09-14T09:07:27.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10156356499999999 2024-09-14T09:07:27.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.782436+0000", 2024-09-14T09:07:27.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013629899999999999 2024-09-14T09:07:27.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.783140+0000", 2024-09-14T09:07:27.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070341800000000003 2024-09-14T09:07:27.518 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.518 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.518 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.518 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104480 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 474112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:27.518 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680747+0000", 2024-09-14T09:07:27.518 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.7965834599999999, 2024-09-14T09:07:27.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.103294211, 2024-09-14T09:07:27.519 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.519 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.519 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:27.519 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:27.519 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:27.519 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.519 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104480 2024-09-14T09:07:27.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.519 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680747+0000", 2024-09-14T09:07:27.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680747+0000", 2024-09-14T09:07:27.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680747+0000", 2024-09-14T09:07:27.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3300000000000001e-07 2024-09-14T09:07:27.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680750+0000", 2024-09-14T09:07:27.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9160000000000001e-06 2024-09-14T09:07:27.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680759+0000", 2024-09-14T09:07:27.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8699999999999998e-06 2024-09-14T09:07:27.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680766+0000", 2024-09-14T09:07:27.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1339999999999998e-06 2024-09-14T09:07:27.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.783218+0000", 2024-09-14T09:07:27.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.102451925 2024-09-14T09:07:27.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.783350+0000", 2024-09-14T09:07:27.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013207200000000001 2024-09-14T09:07:27.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.784041+0000", 2024-09-14T09:07:27.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069056100000000002 2024-09-14T09:07:27.522 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.522 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.523 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.523 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104482 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 478208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:27.523 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680777+0000", 2024-09-14T09:07:27.523 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.7965535710000005, 2024-09-14T09:07:27.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.104207155, 2024-09-14T09:07:27.523 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.523 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.524 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:27.524 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:27.524 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:27.524 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.524 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104482 2024-09-14T09:07:27.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.524 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680777+0000", 2024-09-14T09:07:27.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680777+0000", 2024-09-14T09:07:27.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680777+0000", 2024-09-14T09:07:27.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8700000000000005e-07 2024-09-14T09:07:27.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680801+0000", 2024-09-14T09:07:27.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3963000000000001e-05 2024-09-14T09:07:27.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680832+0000", 2024-09-14T09:07:27.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1029999999999999e-05 2024-09-14T09:07:27.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680841+0000", 2024-09-14T09:07:27.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3030000000000005e-06 2024-09-14T09:07:27.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.784136+0000", 2024-09-14T09:07:27.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.103294752 2024-09-14T09:07:27.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.784275+0000", 2024-09-14T09:07:27.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013825900000000001 2024-09-14T09:07:27.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.784984+0000", 2024-09-14T09:07:27.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070916100000000004 2024-09-14T09:07:27.527 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.527 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.527 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.528 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104484 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 482304~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:27.528 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680855+0000", 2024-09-14T09:07:27.528 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.796475085, 2024-09-14T09:07:27.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.105039178, 2024-09-14T09:07:27.528 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.528 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.528 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:27.528 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:27.528 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:27.528 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.528 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104484 2024-09-14T09:07:27.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.529 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680855+0000", 2024-09-14T09:07:27.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680855+0000", 2024-09-14T09:07:27.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680856+0000", 2024-09-14T09:07:27.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3399999999999997e-07 2024-09-14T09:07:27.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680859+0000", 2024-09-14T09:07:27.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4800000000000001e-06 2024-09-14T09:07:27.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680872+0000", 2024-09-14T09:07:27.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2918999999999999e-05 2024-09-14T09:07:27.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680880+0000", 2024-09-14T09:07:27.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9370000000000004e-06 2024-09-14T09:07:27.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.785063+0000", 2024-09-14T09:07:27.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.104182498 2024-09-14T09:07:27.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.785202+0000", 2024-09-14T09:07:27.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000139362 2024-09-14T09:07:27.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.785894+0000", 2024-09-14T09:07:27.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069204800000000001 2024-09-14T09:07:27.533 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.533 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.533 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.533 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104486 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 486400~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:27.534 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680891+0000", 2024-09-14T09:07:27.534 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.7964394449999999, 2024-09-14T09:07:27.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.105913091, 2024-09-14T09:07:27.534 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.534 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.534 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:27.534 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:27.534 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:27.534 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.534 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104486 2024-09-14T09:07:27.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.534 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680891+0000", 2024-09-14T09:07:27.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680891+0000", 2024-09-14T09:07:27.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680891+0000", 2024-09-14T09:07:27.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1299999999999999e-07 2024-09-14T09:07:27.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680894+0000", 2024-09-14T09:07:27.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0199999999999999e-06 2024-09-14T09:07:27.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680904+0000", 2024-09-14T09:07:27.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2140000000000002e-06 2024-09-14T09:07:27.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680911+0000", 2024-09-14T09:07:27.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1130000000000003e-06 2024-09-14T09:07:27.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.785969+0000", 2024-09-14T09:07:27.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.105058734 2024-09-14T09:07:27.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.786104+0000", 2024-09-14T09:07:27.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013463900000000001 2024-09-14T09:07:27.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.786804+0000", 2024-09-14T09:07:27.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069965800000000003 2024-09-14T09:07:27.538 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.538 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.538 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.539 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104488 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 490496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:27.539 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680925+0000", 2024-09-14T09:07:27.539 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.7964052419999996, 2024-09-14T09:07:27.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10678225700000001, 2024-09-14T09:07:27.539 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.539 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.539 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:27.539 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:27.539 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:27.539 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.539 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104488 2024-09-14T09:07:27.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.540 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680925+0000", 2024-09-14T09:07:27.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680925+0000", 2024-09-14T09:07:27.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680926+0000", 2024-09-14T09:07:27.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-07 2024-09-14T09:07:27.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680928+0000", 2024-09-14T09:07:27.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8870000000000001e-06 2024-09-14T09:07:27.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680940+0000", 2024-09-14T09:07:27.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1717e-05 2024-09-14T09:07:27.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680948+0000", 2024-09-14T09:07:27.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8429999999999993e-06 2024-09-14T09:07:27.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.786883+0000", 2024-09-14T09:07:27.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10593530399999999 2024-09-14T09:07:27.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.787015+0000", 2024-09-14T09:07:27.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013217200000000001 2024-09-14T09:07:27.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.787707+0000", 2024-09-14T09:07:27.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069162399999999995 2024-09-14T09:07:27.543 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.544 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.544 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.544 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104490 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 494592~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:27.544 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680960+0000", 2024-09-14T09:07:27.544 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.7963699049999997, 2024-09-14T09:07:27.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.107672767, 2024-09-14T09:07:27.544 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.545 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.545 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:27.545 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:27.545 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:27.545 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.545 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104490 2024-09-14T09:07:27.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.545 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680960+0000", 2024-09-14T09:07:27.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680960+0000", 2024-09-14T09:07:27.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680961+0000", 2024-09-14T09:07:27.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-07 2024-09-14T09:07:27.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680964+0000", 2024-09-14T09:07:27.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3859999999999999e-06 2024-09-14T09:07:27.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680974+0000", 2024-09-14T09:07:27.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3700000000000001e-06 2024-09-14T09:07:27.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680981+0000", 2024-09-14T09:07:27.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1230000000000002e-06 2024-09-14T09:07:27.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.787780+0000", 2024-09-14T09:07:27.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.106799291 2024-09-14T09:07:27.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.787914+0000", 2024-09-14T09:07:27.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013379200000000001 2024-09-14T09:07:27.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.788633+0000", 2024-09-14T09:07:27.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000719095 2024-09-14T09:07:27.548 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.549 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.549 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.549 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104493 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 498688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:27.549 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681058+0000", 2024-09-14T09:07:27.549 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.7962723860000001, 2024-09-14T09:07:27.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.108480707, 2024-09-14T09:07:27.549 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.549 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.549 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:27.549 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:27.549 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:27.549 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.550 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104493 2024-09-14T09:07:27.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.550 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681058+0000", 2024-09-14T09:07:27.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681058+0000", 2024-09-14T09:07:27.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681059+0000", 2024-09-14T09:07:27.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1200000000000001e-06 2024-09-14T09:07:27.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681062+0000", 2024-09-14T09:07:27.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3100000000000001e-06 2024-09-14T09:07:27.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681078+0000", 2024-09-14T09:07:27.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5630000000000001e-05 2024-09-14T09:07:27.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681096+0000", 2024-09-14T09:07:27.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8036000000000001e-05 2024-09-14T09:07:27.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.788719+0000", 2024-09-14T09:07:27.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.107623651 2024-09-14T09:07:27.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.788851+0000", 2024-09-14T09:07:27.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000131402 2024-09-14T09:07:27.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.789538+0000", 2024-09-14T09:07:27.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068755800000000001 2024-09-14T09:07:27.553 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.554 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.554 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.554 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104494 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 502784~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:27.554 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681108+0000", 2024-09-14T09:07:27.554 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.7962216230000001, 2024-09-14T09:07:27.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10933915700000001, 2024-09-14T09:07:27.554 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.554 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.554 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:27.555 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:27.555 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:27.555 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.555 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104494 2024-09-14T09:07:27.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.555 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681108+0000", 2024-09-14T09:07:27.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681108+0000", 2024-09-14T09:07:27.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681109+0000", 2024-09-14T09:07:27.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0999999999999997e-07 2024-09-14T09:07:27.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681113+0000", 2024-09-14T09:07:27.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4659999999999999e-06 2024-09-14T09:07:27.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681122+0000", 2024-09-14T09:07:27.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5140000000000007e-06 2024-09-14T09:07:27.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681129+0000", 2024-09-14T09:07:27.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1130000000000003e-06 2024-09-14T09:07:27.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.789610+0000", 2024-09-14T09:07:27.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10848097700000001 2024-09-14T09:07:27.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.789748+0000", 2024-09-14T09:07:27.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013734899999999999 2024-09-14T09:07:27.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.790448+0000", 2024-09-14T09:07:27.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069992800000000005 2024-09-14T09:07:27.558 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.558 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.559 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.559 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104496 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 506880~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:27.559 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681140+0000", 2024-09-14T09:07:27.559 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.7961904769999997, 2024-09-14T09:07:27.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.110205621, 2024-09-14T09:07:27.559 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.559 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.559 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:27.559 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:27.559 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:27.559 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.560 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104496 2024-09-14T09:07:27.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.560 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681140+0000", 2024-09-14T09:07:27.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681140+0000", 2024-09-14T09:07:27.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681141+0000", 2024-09-14T09:07:27.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8700000000000004e-07 2024-09-14T09:07:27.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681144+0000", 2024-09-14T09:07:27.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0259999999999999e-06 2024-09-14T09:07:27.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681153+0000", 2024-09-14T09:07:27.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9770000000000006e-06 2024-09-14T09:07:27.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681159+0000", 2024-09-14T09:07:27.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.72e-06 2024-09-14T09:07:27.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.790527+0000", 2024-09-14T09:07:27.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.109368144 2024-09-14T09:07:27.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.790660+0000", 2024-09-14T09:07:27.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013242500000000001 2024-09-14T09:07:27.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.791345+0000", 2024-09-14T09:07:27.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068534200000000003 2024-09-14T09:07:27.563 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.563 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.563 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.564 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104498 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 510976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:27.564 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681169+0000", 2024-09-14T09:07:27.564 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.7961607099999997, 2024-09-14T09:07:27.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.111099946, 2024-09-14T09:07:27.564 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.564 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.564 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:27.564 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:27.564 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:27.564 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.564 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104498 2024-09-14T09:07:27.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.565 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681169+0000", 2024-09-14T09:07:27.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681169+0000", 2024-09-14T09:07:27.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681170+0000", 2024-09-14T09:07:27.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9599999999999999e-07 2024-09-14T09:07:27.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681173+0000", 2024-09-14T09:07:27.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:07:27.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681182+0000", 2024-09-14T09:07:27.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8100000000000004e-06 2024-09-14T09:07:27.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681189+0000", 2024-09-14T09:07:27.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-06 2024-09-14T09:07:27.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.791421+0000", 2024-09-14T09:07:27.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11023223 2024-09-14T09:07:27.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.791553+0000", 2024-09-14T09:07:27.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013137899999999999 2024-09-14T09:07:27.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.792269+0000", 2024-09-14T09:07:27.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071678099999999999 2024-09-14T09:07:27.568 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.568 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.568 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.568 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104500 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 515072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:27.568 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681199+0000", 2024-09-14T09:07:27.569 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.7961311699999998, 2024-09-14T09:07:27.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.111976886, 2024-09-14T09:07:27.569 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.569 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.569 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:27.569 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:27.569 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:27.569 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.569 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104500 2024-09-14T09:07:27.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.569 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681199+0000", 2024-09-14T09:07:27.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681199+0000", 2024-09-14T09:07:27.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681200+0000", 2024-09-14T09:07:27.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3600000000000003e-07 2024-09-14T09:07:27.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681202+0000", 2024-09-14T09:07:27.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7870000000000002e-06 2024-09-14T09:07:27.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681211+0000", 2024-09-14T09:07:27.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7730000000000005e-06 2024-09-14T09:07:27.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681218+0000", 2024-09-14T09:07:27.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9670000000000004e-06 2024-09-14T09:07:27.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.792355+0000", 2024-09-14T09:07:27.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11113640599999999 2024-09-14T09:07:27.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.792486+0000", 2024-09-14T09:07:27.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013117900000000001 2024-09-14T09:07:27.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.793176+0000", 2024-09-14T09:07:27.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069003800000000002 2024-09-14T09:07:27.573 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.573 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.573 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.573 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104502 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 519168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:27.573 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681228+0000", 2024-09-14T09:07:27.574 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.7961019509999998, 2024-09-14T09:07:27.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11285799000000001, 2024-09-14T09:07:27.574 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.574 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.574 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:27.574 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:27.574 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:27.574 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.574 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104502 2024-09-14T09:07:27.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.574 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681228+0000", 2024-09-14T09:07:27.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681228+0000", 2024-09-14T09:07:27.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681229+0000", 2024-09-14T09:07:27.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4399999999999999e-07 2024-09-14T09:07:27.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681232+0000", 2024-09-14T09:07:27.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8289999999999999e-06 2024-09-14T09:07:27.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681241+0000", 2024-09-14T09:07:27.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9239999999999996e-06 2024-09-14T09:07:27.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681248+0000", 2024-09-14T09:07:27.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2389999999999996e-06 2024-09-14T09:07:27.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.793253+0000", 2024-09-14T09:07:27.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11200555700000001 2024-09-14T09:07:27.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.793385+0000", 2024-09-14T09:07:27.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000131929 2024-09-14T09:07:27.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.794086+0000", 2024-09-14T09:07:27.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070076800000000005 2024-09-14T09:07:27.578 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.579 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.579 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.579 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104504 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 523264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:27.579 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681258+0000", 2024-09-14T09:07:27.579 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.7960716449999996, 2024-09-14T09:07:27.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.113704203, 2024-09-14T09:07:27.579 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:27.579 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:27.579 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:27.580 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:27.580 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:27.580 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:27.580 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104504 2024-09-14T09:07:27.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.580 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:27.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:27.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681258+0000", 2024-09-14T09:07:27.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:27.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681258+0000", 2024-09-14T09:07:27.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:27.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:27.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681259+0000", 2024-09-14T09:07:27.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9999999999999997e-07 2024-09-14T09:07:27.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:27.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681271+0000", 2024-09-14T09:07:27.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1674e-05 2024-09-14T09:07:27.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:27.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681281+0000", 2024-09-14T09:07:27.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7100000000000002e-06 2024-09-14T09:07:27.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:27.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681288+0000", 2024-09-14T09:07:27.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2660000000000004e-06 2024-09-14T09:07:27.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:27.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.794140+0000", 2024-09-14T09:07:27.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.112852483 2024-09-14T09:07:27.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:27.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.794270+0000", 2024-09-14T09:07:27.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000129846 2024-09-14T09:07:27.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:27.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:27.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:27.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.794963+0000", 2024-09-14T09:07:27.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069252400000000003 2024-09-14T09:07:27.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.583 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:27.584 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:27.584 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:27.585 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:07:27.586 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:27.583+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:27.686 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:27.687+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:27.753 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:27.753 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:27.753 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:07:27.753 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:27.769 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:07:27.814 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:27.819+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:27.844 INFO:teuthology.orchestra.run.smithi176.stderr:set 3.19 pg_upmap_items mapping to [0->2,5->1] 2024-09-14T09:07:27.844 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is set 2024-09-14T09:07:27.901 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:27.901 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:27.901 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:07:27.901 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:07:27.901 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:27.915 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:07:27.918 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:27.919+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:27.989 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:27.987+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:28.085 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:28.086 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:07:28.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:07:28.086 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:07:28.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.086 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105615 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 382976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.086 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:07:28.086 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.2235108730000004, 2024-09-14T09:07:28.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117270900000001, 2024-09-14T09:07:28.086 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.086 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.086 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:28.086 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.086 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.087 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.087 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105615 2024-09-14T09:07:28.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.087 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.087 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.087 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:07:28.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.087 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.087 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:07:28.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864368+0000", 2024-09-14T09:07:28.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.043e-06 2024-09-14T09:07:28.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864373+0000", 2024-09-14T09:07:28.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4669999999999998e-06 2024-09-14T09:07:28.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864387+0000", 2024-09-14T09:07:28.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3876000000000001e-05 2024-09-14T09:07:28.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864397+0000", 2024-09-14T09:07:28.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0592e-05 2024-09-14T09:07:28.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866069+0000", 2024-09-14T09:07:28.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016714410000000001 2024-09-14T09:07:28.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866076+0000", 2024-09-14T09:07:28.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:07:28.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.075854+0000", 2024-09-14T09:07:28.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.209778501 2024-09-14T09:07:28.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:28.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.075940+0000", 2024-09-14T09:07:28.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5357000000000003e-05 2024-09-14T09:07:28.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.076094+0000", 2024-09-14T09:07:28.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000154745 2024-09-14T09:07:28.091 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.091 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.092 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.092 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105629 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 411648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.092 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:07:28.092 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.2213571510000003, 2024-09-14T09:07:28.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2115591910000001, 2024-09-14T09:07:28.092 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.092 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.092 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:28.092 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.092 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.092 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.092 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105629 2024-09-14T09:07:28.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.093 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:07:28.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:07:28.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866522+0000", 2024-09-14T09:07:28.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6899999999999996e-07 2024-09-14T09:07:28.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866526+0000", 2024-09-14T09:07:28.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0110000000000002e-06 2024-09-14T09:07:28.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866539+0000", 2024-09-14T09:07:28.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3013000000000001e-05 2024-09-14T09:07:28.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866549+0000", 2024-09-14T09:07:28.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8490000000000001e-06 2024-09-14T09:07:28.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866650+0000", 2024-09-14T09:07:28.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101054 2024-09-14T09:07:28.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866664+0000", 2024-09-14T09:07:28.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3805000000000001e-05 2024-09-14T09:07:28.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.077849+0000", 2024-09-14T09:07:28.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2111855419999999 2024-09-14T09:07:28.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:28.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.077945+0000", 2024-09-14T09:07:28.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6086000000000001e-05 2024-09-14T09:07:28.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078080+0000", 2024-09-14T09:07:28.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000134862 2024-09-14T09:07:28.097 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.097 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.097 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.097 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105631 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 415744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.097 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:07:28.097 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.2212979319999997, 2024-09-14T09:07:28.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119463509999999, 2024-09-14T09:07:28.097 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.097 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.097 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:28.097 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.097 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.098 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.098 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105631 2024-09-14T09:07:28.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.098 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:07:28.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:07:28.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866581+0000", 2024-09-14T09:07:28.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.879999999999999e-07 2024-09-14T09:07:28.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866585+0000", 2024-09-14T09:07:28.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0239999999999999e-06 2024-09-14T09:07:28.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866598+0000", 2024-09-14T09:07:28.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2862e-05 2024-09-14T09:07:28.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866608+0000", 2024-09-14T09:07:28.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9890000000000004e-06 2024-09-14T09:07:28.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866708+0000", 2024-09-14T09:07:28.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9605999999999995e-05 2024-09-14T09:07:28.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866720+0000", 2024-09-14T09:07:28.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.261e-05 2024-09-14T09:07:28.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078137+0000", 2024-09-14T09:07:28.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211416571 2024-09-14T09:07:28.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:28.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078293+0000", 2024-09-14T09:07:28.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000156 2024-09-14T09:07:28.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078527+0000", 2024-09-14T09:07:28.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000233701 2024-09-14T09:07:28.102 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.102 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.102 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.102 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105633 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.102 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:07:28.102 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.221238885, 2024-09-14T09:07:28.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211941734, 2024-09-14T09:07:28.103 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.103 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.103 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:28.103 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.103 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.103 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.103 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105633 2024-09-14T09:07:28.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.103 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:07:28.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:07:28.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866640+0000", 2024-09-14T09:07:28.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9300000000000006e-07 2024-09-14T09:07:28.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866656+0000", 2024-09-14T09:07:28.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5291e-05 2024-09-14T09:07:28.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866669+0000", 2024-09-14T09:07:28.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2918000000000001e-05 2024-09-14T09:07:28.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866678+0000", 2024-09-14T09:07:28.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8369999999999992e-06 2024-09-14T09:07:28.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866771+0000", 2024-09-14T09:07:28.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2250999999999995e-05 2024-09-14T09:07:28.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866784+0000", 2024-09-14T09:07:28.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2991e-05 2024-09-14T09:07:28.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078566+0000", 2024-09-14T09:07:28.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117822570000001 2024-09-14T09:07:28.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078581+0000", 2024-09-14T09:07:28.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5196e-05 2024-09-14T09:07:28.107 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.107 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.107 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.107 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105635 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.107 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:07:28.107 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.2211850579999997, 2024-09-14T09:07:28.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119313250000001, 2024-09-14T09:07:28.108 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.108 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.108 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:28.108 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.108 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.108 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.108 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105635 2024-09-14T09:07:28.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.108 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:07:28.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:07:28.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866694+0000", 2024-09-14T09:07:28.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0210000000000001e-06 2024-09-14T09:07:28.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866698+0000", 2024-09-14T09:07:28.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.765e-06 2024-09-14T09:07:28.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866710+0000", 2024-09-14T09:07:28.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2320000000000001e-05 2024-09-14T09:07:28.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866722+0000", 2024-09-14T09:07:28.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2133000000000001e-05 2024-09-14T09:07:28.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866830+0000", 2024-09-14T09:07:28.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000107668 2024-09-14T09:07:28.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866843+0000", 2024-09-14T09:07:28.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2991e-05 2024-09-14T09:07:28.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078616+0000", 2024-09-14T09:07:28.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117732459999999 2024-09-14T09:07:28.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078624+0000", 2024-09-14T09:07:28.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1810000000000001e-06 2024-09-14T09:07:28.112 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.112 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.112 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.112 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105636 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.112 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:07:28.112 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.2211171619999996, 2024-09-14T09:07:28.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119085119999999, 2024-09-14T09:07:28.113 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.113 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.113 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:28.113 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.113 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.113 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.113 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105636 2024-09-14T09:07:28.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.113 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:07:28.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:07:28.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866762+0000", 2024-09-14T09:07:28.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4600000000000003e-07 2024-09-14T09:07:28.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866766+0000", 2024-09-14T09:07:28.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:07:28.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866779+0000", 2024-09-14T09:07:28.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3101999999999999e-05 2024-09-14T09:07:28.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866789+0000", 2024-09-14T09:07:28.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8039999999999996e-06 2024-09-14T09:07:28.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866889+0000", 2024-09-14T09:07:28.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100147 2024-09-14T09:07:28.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866902+0000", 2024-09-14T09:07:28.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3212e-05 2024-09-14T09:07:28.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078659+0000", 2024-09-14T09:07:28.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211756362 2024-09-14T09:07:28.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078670+0000", 2024-09-14T09:07:28.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0956e-05 2024-09-14T09:07:28.118 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.118 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.118 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.119 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105639 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.119 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:07:28.119 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.221058287, 2024-09-14T09:07:28.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118829259999999, 2024-09-14T09:07:28.119 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.119 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.119 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:28.119 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.119 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.119 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.119 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105639 2024-09-14T09:07:28.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.120 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:07:28.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:07:28.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866821+0000", 2024-09-14T09:07:28.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9800000000000002e-07 2024-09-14T09:07:28.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866825+0000", 2024-09-14T09:07:28.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9269999999999998e-06 2024-09-14T09:07:28.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866838+0000", 2024-09-14T09:07:28.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3040000000000001e-05 2024-09-14T09:07:28.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866848+0000", 2024-09-14T09:07:28.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7929999999999992e-06 2024-09-14T09:07:28.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866948+0000", 2024-09-14T09:07:28.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100337 2024-09-14T09:07:28.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866961+0000", 2024-09-14T09:07:28.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3106e-05 2024-09-14T09:07:28.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078693+0000", 2024-09-14T09:07:28.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117320380000001 2024-09-14T09:07:28.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078703+0000", 2024-09-14T09:07:28.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6870000000000006e-06 2024-09-14T09:07:28.123 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.123 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.123 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.124 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105641 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.124 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:07:28.124 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.2209994220000002, 2024-09-14T09:07:28.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118727169999999, 2024-09-14T09:07:28.124 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.124 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.124 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:28.124 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.124 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.124 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.124 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105641 2024-09-14T09:07:28.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.124 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:07:28.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:07:28.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866880+0000", 2024-09-14T09:07:28.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7699999999999992e-07 2024-09-14T09:07:28.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866884+0000", 2024-09-14T09:07:28.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9480000000000001e-06 2024-09-14T09:07:28.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866897+0000", 2024-09-14T09:07:28.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2832e-05 2024-09-14T09:07:28.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866906+0000", 2024-09-14T09:07:28.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7869999999999996e-06 2024-09-14T09:07:28.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867007+0000", 2024-09-14T09:07:28.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010103900000000001 2024-09-14T09:07:28.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867020+0000", 2024-09-14T09:07:28.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2896e-05 2024-09-14T09:07:28.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078739+0000", 2024-09-14T09:07:28.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211718305 2024-09-14T09:07:28.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078752+0000", 2024-09-14T09:07:28.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2933e-05 2024-09-14T09:07:28.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.129 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.129 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105643 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.129 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:07:28.129 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.220940863, 2024-09-14T09:07:28.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211858353, 2024-09-14T09:07:28.129 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.129 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.129 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:28.129 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.129 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.130 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.130 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105643 2024-09-14T09:07:28.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.130 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:07:28.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:07:28.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866938+0000", 2024-09-14T09:07:28.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5600000000000004e-07 2024-09-14T09:07:28.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866942+0000", 2024-09-14T09:07:28.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0350000000000003e-06 2024-09-14T09:07:28.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866955+0000", 2024-09-14T09:07:28.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3097e-05 2024-09-14T09:07:28.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866965+0000", 2024-09-14T09:07:28.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8179999999999999e-06 2024-09-14T09:07:28.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867067+0000", 2024-09-14T09:07:28.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010128699999999999 2024-09-14T09:07:28.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867080+0000", 2024-09-14T09:07:28.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3089e-05 2024-09-14T09:07:28.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078786+0000", 2024-09-14T09:07:28.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117065250000001 2024-09-14T09:07:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078796+0000", 2024-09-14T09:07:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5459999999999997e-06 2024-09-14T09:07:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105645 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:07:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.2208804679999998, 2024-09-14T09:07:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211848641, 2024-09-14T09:07:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105645 2024-09-14T09:07:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:07:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:07:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866999+0000", 2024-09-14T09:07:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7999999999999993e-07 2024-09-14T09:07:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867003+0000", 2024-09-14T09:07:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0090000000000001e-06 2024-09-14T09:07:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867016+0000", 2024-09-14T09:07:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3304e-05 2024-09-14T09:07:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867026+0000", 2024-09-14T09:07:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5889999999999992e-06 2024-09-14T09:07:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867128+0000", 2024-09-14T09:07:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000102101 2024-09-14T09:07:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867141+0000", 2024-09-14T09:07:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2903e-05 2024-09-14T09:07:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078833+0000", 2024-09-14T09:07:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116920959999999 2024-09-14T09:07:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078846+0000", 2024-09-14T09:07:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3658999999999999e-05 2024-09-14T09:07:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105647 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:07:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.2208219500000004, 2024-09-14T09:07:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211830097, 2024-09-14T09:07:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105647 2024-09-14T09:07:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:07:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:07:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867057+0000", 2024-09-14T09:07:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.95e-07 2024-09-14T09:07:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867061+0000", 2024-09-14T09:07:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9530000000000001e-06 2024-09-14T09:07:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867074+0000", 2024-09-14T09:07:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3061000000000001e-05 2024-09-14T09:07:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867086+0000", 2024-09-14T09:07:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1460999999999999e-05 2024-09-14T09:07:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867186+0000", 2024-09-14T09:07:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100181 2024-09-14T09:07:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867199+0000", 2024-09-14T09:07:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-05 2024-09-14T09:07:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078874+0000", 2024-09-14T09:07:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211674683 2024-09-14T09:07:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078886+0000", 2024-09-14T09:07:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2853e-05 2024-09-14T09:07:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105648 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:07:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.2207604639999996, 2024-09-14T09:07:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118073570000001, 2024-09-14T09:07:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105648 2024-09-14T09:07:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:07:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:07:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867119+0000", 2024-09-14T09:07:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8400000000000002e-07 2024-09-14T09:07:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867123+0000", 2024-09-14T09:07:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9639999999999997e-06 2024-09-14T09:07:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867136+0000", 2024-09-14T09:07:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.289e-05 2024-09-14T09:07:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867145+0000", 2024-09-14T09:07:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8919999999999995e-06 2024-09-14T09:07:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867245+0000", 2024-09-14T09:07:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9282000000000003e-05 2024-09-14T09:07:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867258+0000", 2024-09-14T09:07:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2935000000000001e-05 2024-09-14T09:07:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078910+0000", 2024-09-14T09:07:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211652352 2024-09-14T09:07:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078925+0000", 2024-09-14T09:07:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5058000000000001e-05 2024-09-14T09:07:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105652 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:07:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.2207024449999997, 2024-09-14T09:07:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211775737, 2024-09-14T09:07:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105652 2024-09-14T09:07:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:07:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:07:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867177+0000", 2024-09-14T09:07:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9100000000000002e-07 2024-09-14T09:07:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867181+0000", 2024-09-14T09:07:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0029999999999996e-06 2024-09-14T09:07:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867193+0000", 2024-09-14T09:07:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2675e-05 2024-09-14T09:07:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867203+0000", 2024-09-14T09:07:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5599999999999999e-06 2024-09-14T09:07:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867306+0000", 2024-09-14T09:07:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010330100000000001 2024-09-14T09:07:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867319+0000", 2024-09-14T09:07:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3129000000000001e-05 2024-09-14T09:07:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078944+0000", 2024-09-14T09:07:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116247520000001 2024-09-14T09:07:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078952+0000", 2024-09-14T09:07:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4259999999999997e-06 2024-09-14T09:07:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105653 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:07:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.2206437750000001, 2024-09-14T09:07:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211774895, 2024-09-14T09:07:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105653 2024-09-14T09:07:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:07:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:07:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867235+0000", 2024-09-14T09:07:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9400000000000004e-07 2024-09-14T09:07:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867239+0000", 2024-09-14T09:07:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9380000000000002e-06 2024-09-14T09:07:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867252+0000", 2024-09-14T09:07:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.31e-05 2024-09-14T09:07:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867264+0000", 2024-09-14T09:07:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1275e-05 2024-09-14T09:07:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867367+0000", 2024-09-14T09:07:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103755 2024-09-14T09:07:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867382+0000", 2024-09-14T09:07:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4724e-05 2024-09-14T09:07:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078995+0000", 2024-09-14T09:07:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116132669999999 2024-09-14T09:07:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079009+0000", 2024-09-14T09:07:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3942e-05 2024-09-14T09:07:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105654 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:07:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.2205827190000003, 2024-09-14T09:07:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117364070000001, 2024-09-14T09:07:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105654 2024-09-14T09:07:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:07:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:07:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867296+0000", 2024-09-14T09:07:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2399999999999996e-07 2024-09-14T09:07:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867300+0000", 2024-09-14T09:07:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8689999999999997e-06 2024-09-14T09:07:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867313+0000", 2024-09-14T09:07:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2660000000000001e-05 2024-09-14T09:07:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867322+0000", 2024-09-14T09:07:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.533e-06 2024-09-14T09:07:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867429+0000", 2024-09-14T09:07:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106566 2024-09-14T09:07:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867442+0000", 2024-09-14T09:07:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3244000000000001e-05 2024-09-14T09:07:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079024+0000", 2024-09-14T09:07:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211581813 2024-09-14T09:07:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079032+0000", 2024-09-14T09:07:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7980000000000006e-06 2024-09-14T09:07:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105657 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:07:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.2205215459999996, 2024-09-14T09:07:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211700657, 2024-09-14T09:07:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105657 2024-09-14T09:07:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:07:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:07:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867358+0000", 2024-09-14T09:07:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9700000000000005e-07 2024-09-14T09:07:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867367+0000", 2024-09-14T09:07:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8919999999999995e-06 2024-09-14T09:07:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867383+0000", 2024-09-14T09:07:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.579e-05 2024-09-14T09:07:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867393+0000", 2024-09-14T09:07:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3070000000000008e-06 2024-09-14T09:07:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867490+0000", 2024-09-14T09:07:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7703999999999998e-05 2024-09-14T09:07:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867503+0000", 2024-09-14T09:07:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2849e-05 2024-09-14T09:07:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079050+0000", 2024-09-14T09:07:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211546899 2024-09-14T09:07:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079057+0000", 2024-09-14T09:07:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3189999999999997e-06 2024-09-14T09:07:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105658 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:07:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.220459162, 2024-09-14T09:07:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211691217, 2024-09-14T09:07:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105658 2024-09-14T09:07:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:07:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:07:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867420+0000", 2024-09-14T09:07:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5000000000000001e-07 2024-09-14T09:07:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867424+0000", 2024-09-14T09:07:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8469999999999998e-06 2024-09-14T09:07:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867437+0000", 2024-09-14T09:07:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3156000000000001e-05 2024-09-14T09:07:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867448+0000", 2024-09-14T09:07:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1185e-05 2024-09-14T09:07:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867552+0000", 2024-09-14T09:07:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103362 2024-09-14T09:07:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867565+0000", 2024-09-14T09:07:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3018e-05 2024-09-14T09:07:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079092+0000", 2024-09-14T09:07:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211527013 2024-09-14T09:07:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079110+0000", 2024-09-14T09:07:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8686e-05 2024-09-14T09:07:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105662 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:07:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.2203980970000003, 2024-09-14T09:07:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211652226, 2024-09-14T09:07:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105662 2024-09-14T09:07:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:07:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:07:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867481+0000", 2024-09-14T09:07:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1800000000000004e-07 2024-09-14T09:07:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867485+0000", 2024-09-14T09:07:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9199999999999997e-06 2024-09-14T09:07:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867498+0000", 2024-09-14T09:07:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2896e-05 2024-09-14T09:07:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867509+0000", 2024-09-14T09:07:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1202e-05 2024-09-14T09:07:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867611+0000", 2024-09-14T09:07:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010166999999999999 2024-09-14T09:07:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867624+0000", 2024-09-14T09:07:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2996000000000001e-05 2024-09-14T09:07:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079125+0000", 2024-09-14T09:07:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2115009299999999 2024-09-14T09:07:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079132+0000", 2024-09-14T09:07:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6939999999999995e-06 2024-09-14T09:07:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105663 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:07:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.220337046, 2024-09-14T09:07:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116218169999999, 2024-09-14T09:07:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105663 2024-09-14T09:07:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:07:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:07:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867542+0000", 2024-09-14T09:07:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1399999999999995e-07 2024-09-14T09:07:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867546+0000", 2024-09-14T09:07:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7900000000000001e-06 2024-09-14T09:07:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867559+0000", 2024-09-14T09:07:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2805e-05 2024-09-14T09:07:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867568+0000", 2024-09-14T09:07:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-09-14T09:07:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867669+0000", 2024-09-14T09:07:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100655 2024-09-14T09:07:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867682+0000", 2024-09-14T09:07:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3013000000000001e-05 2024-09-14T09:07:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079151+0000", 2024-09-14T09:07:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2114689089999999 2024-09-14T09:07:28.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079163+0000", 2024-09-14T09:07:28.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1962e-05 2024-09-14T09:07:28.187 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.187 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.187 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105665 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:07:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 6.2202784150000001, 2024-09-14T09:07:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211584848, 2024-09-14T09:07:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105665 2024-09-14T09:07:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:07:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:07:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867601+0000", 2024-09-14T09:07:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.569999999999999e-07 2024-09-14T09:07:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867605+0000", 2024-09-14T09:07:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8589999999999998e-06 2024-09-14T09:07:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867617+0000", 2024-09-14T09:07:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2833000000000001e-05 2024-09-14T09:07:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867627+0000", 2024-09-14T09:07:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5219999999999996e-06 2024-09-14T09:07:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867728+0000", 2024-09-14T09:07:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010057500000000001 2024-09-14T09:07:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867740+0000", 2024-09-14T09:07:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2744e-05 2024-09-14T09:07:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079178+0000", 2024-09-14T09:07:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211437552 2024-09-14T09:07:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079185+0000", 2024-09-14T09:07:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8059999999999997e-06 2024-09-14T09:07:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:28.193 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:28.193 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:28.193 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:07:28.194 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:28.195+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:28.195 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:07:28.295 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:28.295+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:28.325 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:28.325 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:28.326 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:07:28.326 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:28.339 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:07:28.345 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:28.345 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:28.361 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:28.362 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:28.380 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:28.380 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:28.395 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:28.395 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:28.396 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:28.399+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:28.413 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:07:28.413 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:07:28.431 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:28.431 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:28.451 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:28.451 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:28.457 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:28.457 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:28.471 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:07:28.472 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:28.472 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:28.534 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:28.535+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:28.597 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:28.595+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:28.638 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:28.638 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:28.653 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:07:28.698 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:28.699+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:28.775 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:28.776 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:28.791 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:07:28.838 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:28.843+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:28.851 INFO:teuthology.orchestra.run.smithi176.stdout:5463 clone 39 order 25 su 4194304 sc 7 2024-09-14T09:07:28.900 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:28.899+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:28.917 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:28.917 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:28.931 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:07:29.019 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:29.023+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:29.102 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:29.103+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:29.102 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:29.102 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:29.115 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:07:29.202 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:29.203+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:29.248 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:29.248 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:29.262 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:07:29.302 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:29.307+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:29.380 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:29.380 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:29.394 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:07:29.442 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:29.443+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:29.504 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:29.503+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:29.513 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:29.513 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:29.527 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:07:29.606 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:29.607+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:29.695 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:29.695 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:29.707 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:29.707+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:29.709 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:07:29.807 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:29.807+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:29.877 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T09:07:29.882 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:29.882 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:29.898 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:07:29.909 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:29.911+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:30.009 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:30.011+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:30.042 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone38 from 0xe923724 (overlap 0x39cb2d4) to 0x5c7848 2024-09-14T09:07:30.077 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:30.078 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:30.098 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:07:30.110 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:30.115+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:30.211 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:30.211+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:30.271 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:30.271 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:30.294 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:07:30.312 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:30.315+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:30.413 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:30.415+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:30.482 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:30.482 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:30.490 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:07:30.498 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:07:30.513 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:30.515+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:30.614 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:30.619+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:30.635 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:30.635 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:30.651 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:30.651 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:30.660 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:30.660 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:30.669 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:30.669 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:30.674 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:07:30.681 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-14T09:07:30.689 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:30.689 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:30.715 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:30.719+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:30.722 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:07:30.723 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:07:30.743 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:30.743 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:30.763 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:30.764 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:30.785 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:30.785 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:30.790 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:30.790 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:30.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.790 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:114405 3.1e 3.97f681e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e121)", 2024-09-14T09:07:30.790 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:30.790488+0000", 2024-09-14T09:07:30.790 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.00077721799999999998, 2024-09-14T09:07:30.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00080329700000000002, 2024-09-14T09:07:30.790 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:30.790 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:30.790 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:07:30.790 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:30.790 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:30.791 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:30.791 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 114405 2024-09-14T09:07:30.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.791 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:30.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:30.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790488+0000", 2024-09-14T09:07:30.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:30.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:30.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790488+0000", 2024-09-14T09:07:30.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:30.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:30.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790488+0000", 2024-09-14T09:07:30.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0100000000000005e-07 2024-09-14T09:07:30.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:30.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790491+0000", 2024-09-14T09:07:30.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.824e-06 2024-09-14T09:07:30.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:30.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790498+0000", 2024-09-14T09:07:30.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.951e-06 2024-09-14T09:07:30.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:30.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790504+0000", 2024-09-14T09:07:30.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4099999999999999e-06 2024-09-14T09:07:30.793 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:30.793 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:30.793 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:30.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.793 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:114409 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121)", 2024-09-14T09:07:30.793 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:30.790517+0000", 2024-09-14T09:07:30.793 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.000747961, 2024-09-14T09:07:30.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00083705100000000001, 2024-09-14T09:07:30.793 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:30.794 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:30.794 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:30.794 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:30.794 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:30.794 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:30.794 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 114409 2024-09-14T09:07:30.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.794 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:30.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:30.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790517+0000", 2024-09-14T09:07:30.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:30.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:30.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790517+0000", 2024-09-14T09:07:30.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:30.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:30.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790517+0000", 2024-09-14T09:07:30.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2e-07 2024-09-14T09:07:30.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:30.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790520+0000", 2024-09-14T09:07:30.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.103e-06 2024-09-14T09:07:30.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:30.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790526+0000", 2024-09-14T09:07:30.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2720000000000003e-06 2024-09-14T09:07:30.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:30.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790530+0000", 2024-09-14T09:07:30.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4970000000000003e-06 2024-09-14T09:07:30.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:30.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790712+0000", 2024-09-14T09:07:30.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000181509 2024-09-14T09:07:30.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:30.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790778+0000", 2024-09-14T09:07:30.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6443000000000001e-05 2024-09-14T09:07:30.797 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:30.797 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:30.797 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:30.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.797 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:114410 3.18 3:18779e98:::rbd_data.115e500fbcd9.0000000000000007:27 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121)", 2024-09-14T09:07:30.797 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:30.790545+0000", 2024-09-14T09:07:30.797 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.00071964200000000005, 2024-09-14T09:07:30.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00087187799999999998, 2024-09-14T09:07:30.797 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:30.797 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:30.798 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:30.798 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:30.798 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:30.798 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:30.798 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 114410 2024-09-14T09:07:30.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.798 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:30.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:30.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790545+0000", 2024-09-14T09:07:30.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:30.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:30.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790545+0000", 2024-09-14T09:07:30.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:30.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:30.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790546+0000", 2024-09-14T09:07:30.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.63e-07 2024-09-14T09:07:30.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:30.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790548+0000", 2024-09-14T09:07:30.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3319999999999999e-06 2024-09-14T09:07:30.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:30.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790555+0000", 2024-09-14T09:07:30.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8560000000000001e-06 2024-09-14T09:07:30.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:30.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790560+0000", 2024-09-14T09:07:30.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9509999999999997e-06 2024-09-14T09:07:30.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:30.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790840+0000", 2024-09-14T09:07:30.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028036199999999998 2024-09-14T09:07:30.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:30.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790926+0000", 2024-09-14T09:07:30.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5966999999999999e-05 2024-09-14T09:07:30.801 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:30.801 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:30.801 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:30.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.801 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:114413 3.7 3:e29f10f6:::rbd_data.115e500fbcd9.000000000000000a:27 [sparse-read 262144~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121)", 2024-09-14T09:07:30.801 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:30.790567+0000", 2024-09-14T09:07:30.801 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.00069790800000000001, 2024-09-14T09:07:30.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000910496, 2024-09-14T09:07:30.801 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:30.801 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:30.801 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:30.801 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:30.802 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:30.802 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:30.802 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 114413 2024-09-14T09:07:30.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.802 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:30.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:30.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790567+0000", 2024-09-14T09:07:30.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:30.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:30.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790567+0000", 2024-09-14T09:07:30.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:30.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:30.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790568+0000", 2024-09-14T09:07:30.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4000000000000002e-07 2024-09-14T09:07:30.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:30.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790570+0000", 2024-09-14T09:07:30.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.091e-06 2024-09-14T09:07:30.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:30.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790576+0000", 2024-09-14T09:07:30.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0499999999999997e-06 2024-09-14T09:07:30.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:30.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790581+0000", 2024-09-14T09:07:30.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9880000000000004e-06 2024-09-14T09:07:30.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:30.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.791166+0000", 2024-09-14T09:07:30.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00058577599999999998 2024-09-14T09:07:30.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:30.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.791306+0000", 2024-09-14T09:07:30.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013987099999999999 2024-09-14T09:07:30.805 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:30.805 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:30.805 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:30.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.805 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:114422 3.1e 3.96f6653e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e121)", 2024-09-14T09:07:30.805 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:30.790614+0000", 2024-09-14T09:07:30.805 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.00065149299999999995, 2024-09-14T09:07:30.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00092222799999999998, 2024-09-14T09:07:30.805 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:30.805 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:30.805 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:07:30.805 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:30.805 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:30.805 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:30.806 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 114422 2024-09-14T09:07:30.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.806 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:30.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:30.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790614+0000", 2024-09-14T09:07:30.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:30.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:30.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790614+0000", 2024-09-14T09:07:30.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:30.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:30.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790614+0000", 2024-09-14T09:07:30.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2500000000000001e-07 2024-09-14T09:07:30.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:30.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790616+0000", 2024-09-14T09:07:30.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1339999999999998e-06 2024-09-14T09:07:30.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:30.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790623+0000", 2024-09-14T09:07:30.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4119999999999998e-06 2024-09-14T09:07:30.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:30.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:30.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:30.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:30.790627+0000", 2024-09-14T09:07:30.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7450000000000003e-06 2024-09-14T09:07:30.808 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:30.808 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:30.808 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:30.808 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:30.808 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:30.808 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 5 2024-09-14T09:07:30.808 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:30.815 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:30.815+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:30.823 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:07:30.862 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is unset 2024-09-14T09:07:30.926 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:30.927+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:30.972 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:30.973 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:30.973 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:30.973 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:30.973 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:30.987 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:07:31.017 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:31.019+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:31.118 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:31.119+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:31.149 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:31.149 INFO:teuthology.orchestra.run.smithi176.stdout:5467 write 0xe7696b1 thru 0xe769f86 (0x8d6 bytes) 2024-09-14T09:07:31.151 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.066758416, 2024-09-14T09:07:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:07:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:31.152 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:07:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:07:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:07:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:07:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:07:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:07:31.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:07:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:07:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:07:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:07:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:07:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:31.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:07:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:07:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:07:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:07:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:07:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:07:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:07:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:07:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.066703758, 2024-09-14T09:07:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:07:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.157 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:07:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:07:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:07:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:07:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:07:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:07:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:07:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:07:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:07:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:07:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:07:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:07:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:07:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:07:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:07:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:07:31.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:07:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:07:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.066598384, 2024-09-14T09:07:31.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:07:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:07:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:07:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:07:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:07:31.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:07:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:07:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:07:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:07:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:07:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:07:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:07:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:07:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:07:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:07:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:07:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:07:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:07:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:07:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:07:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.06624162, 2024-09-14T09:07:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:07:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:07:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.168 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:07:31.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:07:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:07:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:07:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:07:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:07:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:07:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:07:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:07:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:07:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:07:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:07:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:07:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:07:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:07:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:07:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:07:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:07:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.066156882, 2024-09-14T09:07:31.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:07:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:07:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:07:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:07:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:07:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:07:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:07:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:07:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:07:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:07:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:07:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:07:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:07:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:07:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:07:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:07:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:07:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:07:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:07:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:07:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.066069196, 2024-09-14T09:07:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:07:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.177 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:07:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:31.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:07:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:07:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:07:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:07:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:07:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:07:31.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:07:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:07:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:07:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:07:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:07:31.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:07:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:07:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:07:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:07:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:07:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:07:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:07:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:07:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 84.989777208999996, 2024-09-14T09:07:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:07:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:31.182 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:07:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:07:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:07:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:07:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:07:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:07:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:07:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:07:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:07:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:07:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:07:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:07:31.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:07:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:07:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:07:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:07:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:07:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:07:31.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:07:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:07:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:07:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:07:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:07:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:84137 3.7 3:e15b49d9:::rbd_data.115e77b799c3.00000000000000e0:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:07:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.827029811000003, 2024-09-14T09:07:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11205707500000001, 2024-09-14T09:07:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 84137 2024-09-14T09:07:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.188 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323359+0000", 2024-09-14T09:07:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323362+0000", 2024-09-14T09:07:31.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0839999999999999e-06 2024-09-14T09:07:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323370+0000", 2024-09-14T09:07:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0180000000000001e-06 2024-09-14T09:07:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323388+0000", 2024-09-14T09:07:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8326999999999999e-05 2024-09-14T09:07:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323397+0000", 2024-09-14T09:07:31.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1940000000000004e-06 2024-09-14T09:07:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323452+0000", 2024-09-14T09:07:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5043000000000001e-05 2024-09-14T09:07:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.323550+0000", 2024-09-14T09:07:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8201000000000004e-05 2024-09-14T09:07:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.328273+0000", 2024-09-14T09:07:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0047227079999999999 2024-09-14T09:07:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435232+0000", 2024-09-14T09:07:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.106959339 2024-09-14T09:07:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435256+0000", 2024-09-14T09:07:31.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3774e-05 2024-09-14T09:07:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435314+0000", 2024-09-14T09:07:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7636999999999997e-05 2024-09-14T09:07:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:53.435417+0000", 2024-09-14T09:07:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010275 2024-09-14T09:07:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:97796 3.15 3:aaaa5f7f:::rbd_data.115e113dc8ca.0000000000000029:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:07:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.055908338999998, 2024-09-14T09:07:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14537731300000001, 2024-09-14T09:07:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97796 2024-09-14T09:07:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.194 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094482+0000", 2024-09-14T09:07:31.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6400000000000001e-07 2024-09-14T09:07:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094485+0000", 2024-09-14T09:07:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.693e-06 2024-09-14T09:07:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094494+0000", 2024-09-14T09:07:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9590000000000001e-06 2024-09-14T09:07:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094502+0000", 2024-09-14T09:07:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3180000000000001e-06 2024-09-14T09:07:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094567+0000", 2024-09-14T09:07:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5526999999999998e-05 2024-09-14T09:07:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094726+0000", 2024-09-14T09:07:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015842499999999999 2024-09-14T09:07:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:07:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.101940+0000", 2024-09-14T09:07:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0072146559999999998 2024-09-14T09:07:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239665+0000", 2024-09-14T09:07:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.137724501 2024-09-14T09:07:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239726+0000", 2024-09-14T09:07:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1550999999999994e-05 2024-09-14T09:07:31.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239800+0000", 2024-09-14T09:07:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3318000000000005e-05 2024-09-14T09:07:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239858+0000", 2024-09-14T09:07:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8600999999999997e-05 2024-09-14T09:07:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.199 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:97796 3.15 e106/103 ondisk, result = 0)", 2024-09-14T09:07:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 19.03356247, 2024-09-14T09:07:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123043465, 2024-09-14T09:07:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116828+0000", 2024-09-14T09:07:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8100000000000002e-07 2024-09-14T09:07:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116831+0000", 2024-09-14T09:07:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3009999999999998e-06 2024-09-14T09:07:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116841+0000", 2024-09-14T09:07:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7829999999999993e-06 2024-09-14T09:07:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116851+0000", 2024-09-14T09:07:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0135999999999999e-05 2024-09-14T09:07:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239707+0000", 2024-09-14T09:07:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12285581499999999 2024-09-14T09:07:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239718+0000", 2024-09-14T09:07:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1804000000000001e-05 2024-09-14T09:07:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239870+0000", 2024-09-14T09:07:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015194500000000001 2024-09-14T09:07:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107601 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403134+0000", 2024-09-14T09:07:31.203 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.7472557340000003, 2024-09-14T09:07:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111355695, 2024-09-14T09:07:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107601 2024-09-14T09:07:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403134+0000", 2024-09-14T09:07:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403134+0000", 2024-09-14T09:07:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403134+0000", 2024-09-14T09:07:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2000000000000001e-07 2024-09-14T09:07:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403136+0000", 2024-09-14T09:07:31.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7430000000000001e-06 2024-09-14T09:07:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403140+0000", 2024-09-14T09:07:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:07:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403144+0000", 2024-09-14T09:07:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.766e-06 2024-09-14T09:07:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514478+0000", 2024-09-14T09:07:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111334212 2024-09-14T09:07:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514489+0000", 2024-09-14T09:07:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1236999999999999e-05 2024-09-14T09:07:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107603 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403149+0000", 2024-09-14T09:07:31.207 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.747240466, 2024-09-14T09:07:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111385997, 2024-09-14T09:07:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107603 2024-09-14T09:07:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403149+0000", 2024-09-14T09:07:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403149+0000", 2024-09-14T09:07:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403149+0000", 2024-09-14T09:07:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8299999999999998e-07 2024-09-14T09:07:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403152+0000", 2024-09-14T09:07:31.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T09:07:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403158+0000", 2024-09-14T09:07:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2940000000000002e-06 2024-09-14T09:07:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403165+0000", 2024-09-14T09:07:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.849e-06 2024-09-14T09:07:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514524+0000", 2024-09-14T09:07:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111358923 2024-09-14T09:07:31.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514535+0000", 2024-09-14T09:07:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0913999999999999e-05 2024-09-14T09:07:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107605 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403207+0000", 2024-09-14T09:07:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.7471827009999998, 2024-09-14T09:07:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11137927, 2024-09-14T09:07:31.211 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107605 2024-09-14T09:07:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403207+0000", 2024-09-14T09:07:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403207+0000", 2024-09-14T09:07:31.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403207+0000", 2024-09-14T09:07:31.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1800000000000005e-07 2024-09-14T09:07:31.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403211+0000", 2024-09-14T09:07:31.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3629999999999998e-06 2024-09-14T09:07:31.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403222+0000", 2024-09-14T09:07:31.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1606e-05 2024-09-14T09:07:31.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403230+0000", 2024-09-14T09:07:31.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6760000000000007e-06 2024-09-14T09:07:31.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514572+0000", 2024-09-14T09:07:31.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111341991 2024-09-14T09:07:31.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514586+0000", 2024-09-14T09:07:31.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3916e-05 2024-09-14T09:07:31.215 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.215 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.215 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.215 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107607 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:31.215 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403242+0000", 2024-09-14T09:07:31.215 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.747147719, 2024-09-14T09:07:31.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111392375, 2024-09-14T09:07:31.215 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.215 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.215 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:31.215 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:31.215 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:31.216 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.216 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107607 2024-09-14T09:07:31.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.216 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403242+0000", 2024-09-14T09:07:31.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403242+0000", 2024-09-14T09:07:31.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403242+0000", 2024-09-14T09:07:31.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2600000000000002e-07 2024-09-14T09:07:31.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403245+0000", 2024-09-14T09:07:31.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.128e-06 2024-09-14T09:07:31.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403254+0000", 2024-09-14T09:07:31.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6219999999999998e-06 2024-09-14T09:07:31.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403261+0000", 2024-09-14T09:07:31.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1999999999999997e-06 2024-09-14T09:07:31.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514621+0000", 2024-09-14T09:07:31.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111359977 2024-09-14T09:07:31.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514634+0000", 2024-09-14T09:07:31.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2822e-05 2024-09-14T09:07:31.218 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.218 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.219 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.219 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107609 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:31.219 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403271+0000", 2024-09-14T09:07:31.219 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.7471184580000001, 2024-09-14T09:07:31.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111392167, 2024-09-14T09:07:31.219 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.219 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.219 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:31.219 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:31.219 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:31.219 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.219 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107609 2024-09-14T09:07:31.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.219 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403271+0000", 2024-09-14T09:07:31.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403271+0000", 2024-09-14T09:07:31.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403271+0000", 2024-09-14T09:07:31.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3499999999999996e-07 2024-09-14T09:07:31.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403319+0000", 2024-09-14T09:07:31.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7450000000000001e-05 2024-09-14T09:07:31.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403332+0000", 2024-09-14T09:07:31.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3066e-05 2024-09-14T09:07:31.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403341+0000", 2024-09-14T09:07:31.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5739999999999996e-06 2024-09-14T09:07:31.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514657+0000", 2024-09-14T09:07:31.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11131680100000001 2024-09-14T09:07:31.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514663+0000", 2024-09-14T09:07:31.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6409999999999999e-06 2024-09-14T09:07:31.222 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.222 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.222 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.222 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107613 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:31.222 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403384+0000", 2024-09-14T09:07:31.223 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.7470052970000003, 2024-09-14T09:07:31.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11137016399999999, 2024-09-14T09:07:31.223 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.223 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.223 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:31.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:31.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:31.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.223 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107613 2024-09-14T09:07:31.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.223 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403384+0000", 2024-09-14T09:07:31.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403384+0000", 2024-09-14T09:07:31.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403385+0000", 2024-09-14T09:07:31.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5600000000000005e-07 2024-09-14T09:07:31.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403388+0000", 2024-09-14T09:07:31.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0139999999999999e-06 2024-09-14T09:07:31.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403397+0000", 2024-09-14T09:07:31.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1980000000000007e-06 2024-09-14T09:07:31.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403404+0000", 2024-09-14T09:07:31.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3669999999999999e-06 2024-09-14T09:07:31.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514742+0000", 2024-09-14T09:07:31.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11133736299999999 2024-09-14T09:07:31.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514754+0000", 2024-09-14T09:07:31.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2566e-05 2024-09-14T09:07:31.226 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.226 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.226 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.226 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107615 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:31.226 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403415+0000", 2024-09-14T09:07:31.226 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.7469747120000001, 2024-09-14T09:07:31.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11137188200000001, 2024-09-14T09:07:31.226 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.226 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.226 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:31.226 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:31.227 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:31.227 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.227 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107615 2024-09-14T09:07:31.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.227 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403415+0000", 2024-09-14T09:07:31.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403415+0000", 2024-09-14T09:07:31.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403415+0000", 2024-09-14T09:07:31.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1500000000000004e-07 2024-09-14T09:07:31.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403418+0000", 2024-09-14T09:07:31.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9170000000000002e-06 2024-09-14T09:07:31.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403427+0000", 2024-09-14T09:07:31.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1740000000000006e-06 2024-09-14T09:07:31.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403435+0000", 2024-09-14T09:07:31.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2849999999999997e-06 2024-09-14T09:07:31.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514776+0000", 2024-09-14T09:07:31.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11134168799999999 2024-09-14T09:07:31.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514786+0000", 2024-09-14T09:07:31.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0203e-05 2024-09-14T09:07:31.229 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.229 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.230 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.230 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107617 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:31.230 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403444+0000", 2024-09-14T09:07:31.230 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.7469451750000005, 2024-09-14T09:07:31.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111359085, 2024-09-14T09:07:31.230 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.230 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.230 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:31.230 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:31.230 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:31.230 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.230 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107617 2024-09-14T09:07:31.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.230 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403444+0000", 2024-09-14T09:07:31.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403444+0000", 2024-09-14T09:07:31.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403445+0000", 2024-09-14T09:07:31.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0399999999999996e-07 2024-09-14T09:07:31.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403447+0000", 2024-09-14T09:07:31.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7989999999999998e-06 2024-09-14T09:07:31.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403459+0000", 2024-09-14T09:07:31.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1086e-05 2024-09-14T09:07:31.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403465+0000", 2024-09-14T09:07:31.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9709999999999998e-06 2024-09-14T09:07:31.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514798+0000", 2024-09-14T09:07:31.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111332495 2024-09-14T09:07:31.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514803+0000", 2024-09-14T09:07:31.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2299999999999999e-06 2024-09-14T09:07:31.233 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.233 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.233 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.233 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107619 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:31.234 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403476+0000", 2024-09-14T09:07:31.236 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.7469132609999996, 2024-09-14T09:07:31.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111354988, 2024-09-14T09:07:31.236 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.236 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.236 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:31.236 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:31.236 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:31.236 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.236 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107619 2024-09-14T09:07:31.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.236 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403476+0000", 2024-09-14T09:07:31.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403476+0000", 2024-09-14T09:07:31.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403477+0000", 2024-09-14T09:07:31.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8199999999999999e-07 2024-09-14T09:07:31.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403480+0000", 2024-09-14T09:07:31.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9979999999999999e-06 2024-09-14T09:07:31.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403493+0000", 2024-09-14T09:07:31.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3478999999999999e-05 2024-09-14T09:07:31.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403508+0000", 2024-09-14T09:07:31.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4419e-05 2024-09-14T09:07:31.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514821+0000", 2024-09-14T09:07:31.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111313379 2024-09-14T09:07:31.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514831+0000", 2024-09-14T09:07:31.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0030999999999999e-05 2024-09-14T09:07:31.239 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.239 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.239 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.240 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:107625 3.7 3.6f08f947 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:31.240 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:23.403586+0000", 2024-09-14T09:07:31.240 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 7.7468032879999997, 2024-09-14T09:07:31.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111365832, 2024-09-14T09:07:31.240 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:31.240 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:31.240 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "reached pg", 2024-09-14T09:07:31.240 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:31.240 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:31.240 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.240 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 107625 2024-09-14T09:07:31.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.241 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:31.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:31.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403586+0000", 2024-09-14T09:07:31.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:31.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.403586+0000", 2024-09-14T09:07:31.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:31.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:31.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.404090+0000", 2024-09-14T09:07:31.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050411300000000004 2024-09-14T09:07:31.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:31.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.404096+0000", 2024-09-14T09:07:31.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2790000000000004e-06 2024-09-14T09:07:31.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:31.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.404110+0000", 2024-09-14T09:07:31.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3784000000000001e-05 2024-09-14T09:07:31.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.404120+0000", 2024-09-14T09:07:31.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.8039999999999996e-06 2024-09-14T09:07:31.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:31.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514944+0000", 2024-09-14T09:07:31.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.110824099 2024-09-14T09:07:31.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:31.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:31.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:31.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.514952+0000", 2024-09-14T09:07:31.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.7530000000000001e-06 2024-09-14T09:07:31.244 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.244 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.244 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.244 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:31.244 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:31.244 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:31.246 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:07:31.246 INFO:teuthology.orchestra.run.smithi176.stdout:5468 writesame 0xb53c431 thru 0xb541c74 data_size 0x6(0x5844 bytes) 2024-09-14T09:07:31.246 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:31.243+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:31.347 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:31.347+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:31.379 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:31.379 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:31.379 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:07:31.379 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:31.393 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:07:31.448 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:31.451+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:31.512 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:31.512 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:31.512 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:07:31.512 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:07:31.512 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:31.516 INFO:teuthology.orchestra.run.smithi176.stdout:5469 read 0xcbad855 thru 0xcbb6e00 (0x95ac bytes) 2024-09-14T09:07:31.526 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:07:31.535 INFO:teuthology.orchestra.run.smithi176.stdout:5471 trunc from 0xe923724 to 0x78bc4d 2024-09-14T09:07:31.570 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:31.575+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:31.650 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:31.651+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:31.723 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:31.724 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:07:31.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:07:31.724 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:07:31.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.724 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:76803 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-14T09:07:31.724 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:43.858877+0000", 2024-09-14T09:07:31.724 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.868794190000003, 2024-09-14T09:07:31.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0137577900000001, 2024-09-14T09:07:31.724 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.724 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.724 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:31.724 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:31.724 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:31.725 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.725 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 76803 2024-09-14T09:07:31.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.725 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858877+0000", 2024-09-14T09:07:31.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858877+0000", 2024-09-14T09:07:31.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858879+0000", 2024-09-14T09:07:31.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.097e-06 2024-09-14T09:07:31.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858887+0000", 2024-09-14T09:07:31.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6510000000000001e-06 2024-09-14T09:07:31.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858903+0000", 2024-09-14T09:07:31.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6560999999999999e-05 2024-09-14T09:07:31.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858912+0000", 2024-09-14T09:07:31.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2019999999999993e-06 2024-09-14T09:07:31.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858964+0000", 2024-09-14T09:07:31.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2046999999999998e-05 2024-09-14T09:07:31.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:31.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:43.858972+0000", 2024-09-14T09:07:31.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6029999999999999e-06 2024-09-14T09:07:31.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:44.871832+0000", 2024-09-14T09:07:31.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.012860109 2024-09-14T09:07:31.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:44.872248+0000", 2024-09-14T09:07:31.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00041549300000000001 2024-09-14T09:07:31.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.729 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:44.872635+0000", 2024-09-14T09:07:31.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038702700000000002 2024-09-14T09:07:31.729 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.729 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.729 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.729 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:84403 3.14 3:2fc85a9a:::rbd_data.115e77b799c3.0000000000000018:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-14T09:07:31.729 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:53.878069+0000", 2024-09-14T09:07:31.729 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.849602320999999, 2024-09-14T09:07:31.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.128852405, 2024-09-14T09:07:31.729 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.729 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.730 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:31.730 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:31.730 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:31.730 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.730 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 84403 2024-09-14T09:07:31.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.730 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878069+0000", 2024-09-14T09:07:31.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878069+0000", 2024-09-14T09:07:31.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878071+0000", 2024-09-14T09:07:31.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0899999999999999e-06 2024-09-14T09:07:31.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878082+0000", 2024-09-14T09:07:31.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1226999999999999e-05 2024-09-14T09:07:31.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878108+0000", 2024-09-14T09:07:31.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5568999999999999e-05 2024-09-14T09:07:31.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878122+0000", 2024-09-14T09:07:31.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.488e-05 2024-09-14T09:07:31.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878247+0000", 2024-09-14T09:07:31.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000124039 2024-09-14T09:07:31.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.878387+0000", 2024-09-14T09:07:31.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014041600000000001 2024-09-14T09:07:31.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:31.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:53.886261+0000", 2024-09-14T09:07:31.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0078738499999999999 2024-09-14T09:07:31.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:31.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006598+0000", 2024-09-14T09:07:31.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.120337316 2024-09-14T09:07:31.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:31.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006724+0000", 2024-09-14T09:07:31.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012601600000000001 2024-09-14T09:07:31.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:31.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006827+0000", 2024-09-14T09:07:31.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103299 2024-09-14T09:07:31.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:54.006921+0000", 2024-09-14T09:07:31.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3702999999999994e-05 2024-09-14T09:07:31.735 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.735 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.735 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.735 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:93272 3.3 3:c49b4fbc:::rbd_data.115e7770ddc5.000000000000001a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T09:07:31.735 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:57.971357+0000", 2024-09-14T09:07:31.735 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.756314261999997, 2024-09-14T09:07:31.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.112268252, 2024-09-14T09:07:31.735 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.735 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.735 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:31.736 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:31.736 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:31.736 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.736 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 93272 2024-09-14T09:07:31.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.736 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971357+0000", 2024-09-14T09:07:31.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971357+0000", 2024-09-14T09:07:31.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971360+0000", 2024-09-14T09:07:31.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8250000000000001e-06 2024-09-14T09:07:31.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971371+0000", 2024-09-14T09:07:31.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1537e-05 2024-09-14T09:07:31.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971399+0000", 2024-09-14T09:07:31.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8274999999999999e-05 2024-09-14T09:07:31.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971417+0000", 2024-09-14T09:07:31.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7241999999999999e-05 2024-09-14T09:07:31.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971537+0000", 2024-09-14T09:07:31.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012010600000000001 2024-09-14T09:07:31.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.971685+0000", 2024-09-14T09:07:31.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014777599999999999 2024-09-14T09:07:31.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:31.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:57.979172+0000", 2024-09-14T09:07:31.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0074876810000000004 2024-09-14T09:07:31.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:31.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083438+0000", 2024-09-14T09:07:31.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.104265969 2024-09-14T09:07:31.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:31.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083466+0000", 2024-09-14T09:07:31.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7665999999999999e-05 2024-09-14T09:07:31.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:31.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083553+0000", 2024-09-14T09:07:31.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6809000000000002e-05 2024-09-14T09:07:31.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:58.083625+0000", 2024-09-14T09:07:31.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2366000000000005e-05 2024-09-14T09:07:31.741 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.741 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.741 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.741 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:96694 3.11 3:8a7e3c58:::rbd_data.115e63832e35.000000000000001c:head [stat,truncate 131072] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-14T09:07:31.741 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:07.004845+0000", 2024-09-14T09:07:31.741 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 24.722826246, 2024-09-14T09:07:31.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.85224771499999996, 2024-09-14T09:07:31.741 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.741 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.741 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:31.741 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:31.741 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:31.742 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.742 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 96694 2024-09-14T09:07:31.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.742 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004845+0000", 2024-09-14T09:07:31.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004845+0000", 2024-09-14T09:07:31.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004848+0000", 2024-09-14T09:07:31.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:07:31.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004862+0000", 2024-09-14T09:07:31.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4426e-05 2024-09-14T09:07:31.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004903+0000", 2024-09-14T09:07:31.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0447000000000001e-05 2024-09-14T09:07:31.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.004926+0000", 2024-09-14T09:07:31.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3260000000000001e-05 2024-09-14T09:07:31.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.005104+0000", 2024-09-14T09:07:31.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000178018 2024-09-14T09:07:31.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:31.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.005120+0000", 2024-09-14T09:07:31.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6422999999999998e-05 2024-09-14T09:07:31.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.851775+0000", 2024-09-14T09:07:31.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.84665435499999997 2024-09-14T09:07:31.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.852176+0000", 2024-09-14T09:07:31.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000400787 2024-09-14T09:07:31.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-09-14T09:07:31.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.852662+0000", 2024-09-14T09:07:31.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048621899999999997 2024-09-14T09:07:31.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:31.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.855261+0000", 2024-09-14T09:07:31.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025991479999999999 2024-09-14T09:07:31.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:31.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.856839+0000", 2024-09-14T09:07:31.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015779819999999999 2024-09-14T09:07:31.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:31.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.856993+0000", 2024-09-14T09:07:31.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015431900000000001 2024-09-14T09:07:31.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:07.857093+0000", 2024-09-14T09:07:31.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9314000000000005e-05 2024-09-14T09:07:31.748 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.748 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.748 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.748 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:97612 3.e 3:7298f82d:::rbd_data.115e113dc8ca.00000000000000e8:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:07:31.748 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:11.399529+0000", 2024-09-14T09:07:31.748 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.328142193000001, 2024-09-14T09:07:31.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.18555535000000001, 2024-09-14T09:07:31.748 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.748 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.748 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:31.748 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:31.749 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:31.749 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.749 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 97612 2024-09-14T09:07:31.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.749 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399529+0000", 2024-09-14T09:07:31.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399529+0000", 2024-09-14T09:07:31.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399532+0000", 2024-09-14T09:07:31.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6599999999999999e-06 2024-09-14T09:07:31.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399544+0000", 2024-09-14T09:07:31.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2527000000000001e-05 2024-09-14T09:07:31.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399574+0000", 2024-09-14T09:07:31.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.97e-05 2024-09-14T09:07:31.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399588+0000", 2024-09-14T09:07:31.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4653e-05 2024-09-14T09:07:31.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399740+0000", 2024-09-14T09:07:31.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015180900000000001 2024-09-14T09:07:31.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.399952+0000", 2024-09-14T09:07:31.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000211435 2024-09-14T09:07:31.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:31.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.411686+0000", 2024-09-14T09:07:31.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011733883000000001 2024-09-14T09:07:31.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:31.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584791+0000", 2024-09-14T09:07:31.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.17310596 2024-09-14T09:07:31.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:31.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584915+0000", 2024-09-14T09:07:31.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012330800000000001 2024-09-14T09:07:31.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:31.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.585010+0000", 2024-09-14T09:07:31.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5216000000000005e-05 2024-09-14T09:07:31.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.585084+0000", 2024-09-14T09:07:31.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4198999999999999e-05 2024-09-14T09:07:31.754 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.754 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.754 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.754 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop_reply(client.4446.0:97612 3.e e106/103 ondisk, result = 0)", 2024-09-14T09:07:31.754 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:11.451700+0000", 2024-09-14T09:07:31.754 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.275970899000001, 2024-09-14T09:07:31.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13339934000000001, 2024-09-14T09:07:31.754 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.754 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.754 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:31.755 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451700+0000", 2024-09-14T09:07:31.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451700+0000", 2024-09-14T09:07:31.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451702+0000", 2024-09-14T09:07:31.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.34e-06 2024-09-14T09:07:31.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451713+0000", 2024-09-14T09:07:31.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1e-05 2024-09-14T09:07:31.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451751+0000", 2024-09-14T09:07:31.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7762999999999997e-05 2024-09-14T09:07:31.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.451770+0000", 2024-09-14T09:07:31.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8626000000000001e-05 2024-09-14T09:07:31.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584888+0000", 2024-09-14T09:07:31.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13311764500000001 2024-09-14T09:07:31.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.584903+0000", 2024-09-14T09:07:31.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.499e-05 2024-09-14T09:07:31.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:11.585099+0000", 2024-09-14T09:07:31.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019697599999999999 2024-09-14T09:07:31.758 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.758 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.758 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.758 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104478 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 470016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:31.758 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680713+0000", 2024-09-14T09:07:31.758 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 11.046957892, 2024-09-14T09:07:31.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10242654800000001, 2024-09-14T09:07:31.758 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.758 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.759 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:31.759 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:31.759 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:31.759 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.759 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104478 2024-09-14T09:07:31.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.759 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680713+0000", 2024-09-14T09:07:31.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680713+0000", 2024-09-14T09:07:31.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680714+0000", 2024-09-14T09:07:31.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3300000000000001e-07 2024-09-14T09:07:31.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680717+0000", 2024-09-14T09:07:31.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0170000000000001e-06 2024-09-14T09:07:31.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680729+0000", 2024-09-14T09:07:31.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1643e-05 2024-09-14T09:07:31.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680736+0000", 2024-09-14T09:07:31.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8730000000000007e-06 2024-09-14T09:07:31.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.782300+0000", 2024-09-14T09:07:31.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10156356499999999 2024-09-14T09:07:31.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.782436+0000", 2024-09-14T09:07:31.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013629899999999999 2024-09-14T09:07:31.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.783140+0000", 2024-09-14T09:07:31.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070341800000000003 2024-09-14T09:07:31.762 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.762 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.762 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.762 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104480 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 474112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:31.762 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680747+0000", 2024-09-14T09:07:31.762 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 11.046924419, 2024-09-14T09:07:31.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.103294211, 2024-09-14T09:07:31.763 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.763 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.763 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:31.763 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:31.763 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:31.763 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.763 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104480 2024-09-14T09:07:31.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.763 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680747+0000", 2024-09-14T09:07:31.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680747+0000", 2024-09-14T09:07:31.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680747+0000", 2024-09-14T09:07:31.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3300000000000001e-07 2024-09-14T09:07:31.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680750+0000", 2024-09-14T09:07:31.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9160000000000001e-06 2024-09-14T09:07:31.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680759+0000", 2024-09-14T09:07:31.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8699999999999998e-06 2024-09-14T09:07:31.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680766+0000", 2024-09-14T09:07:31.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1339999999999998e-06 2024-09-14T09:07:31.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.783218+0000", 2024-09-14T09:07:31.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.102451925 2024-09-14T09:07:31.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.783350+0000", 2024-09-14T09:07:31.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013207200000000001 2024-09-14T09:07:31.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.784041+0000", 2024-09-14T09:07:31.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069056100000000002 2024-09-14T09:07:31.766 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.766 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.766 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.767 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104482 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 478208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:31.767 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680777+0000", 2024-09-14T09:07:31.767 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 11.046894529999999, 2024-09-14T09:07:31.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.104207155, 2024-09-14T09:07:31.767 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.767 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.767 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:31.767 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:31.767 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:31.767 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.767 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104482 2024-09-14T09:07:31.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.768 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680777+0000", 2024-09-14T09:07:31.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680777+0000", 2024-09-14T09:07:31.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680777+0000", 2024-09-14T09:07:31.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8700000000000005e-07 2024-09-14T09:07:31.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680801+0000", 2024-09-14T09:07:31.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3963000000000001e-05 2024-09-14T09:07:31.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680832+0000", 2024-09-14T09:07:31.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1029999999999999e-05 2024-09-14T09:07:31.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680841+0000", 2024-09-14T09:07:31.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3030000000000005e-06 2024-09-14T09:07:31.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.784136+0000", 2024-09-14T09:07:31.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.103294752 2024-09-14T09:07:31.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.784275+0000", 2024-09-14T09:07:31.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013825900000000001 2024-09-14T09:07:31.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.784984+0000", 2024-09-14T09:07:31.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070916100000000004 2024-09-14T09:07:31.771 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.771 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.771 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.771 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104484 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 482304~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:31.771 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680855+0000", 2024-09-14T09:07:31.772 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 11.046816044, 2024-09-14T09:07:31.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.105039178, 2024-09-14T09:07:31.772 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.772 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.772 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:31.772 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:31.772 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:31.772 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.772 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104484 2024-09-14T09:07:31.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.772 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680855+0000", 2024-09-14T09:07:31.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680855+0000", 2024-09-14T09:07:31.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680856+0000", 2024-09-14T09:07:31.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3399999999999997e-07 2024-09-14T09:07:31.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680859+0000", 2024-09-14T09:07:31.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4800000000000001e-06 2024-09-14T09:07:31.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680872+0000", 2024-09-14T09:07:31.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2918999999999999e-05 2024-09-14T09:07:31.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680880+0000", 2024-09-14T09:07:31.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9370000000000004e-06 2024-09-14T09:07:31.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.785063+0000", 2024-09-14T09:07:31.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.104182498 2024-09-14T09:07:31.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.785202+0000", 2024-09-14T09:07:31.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000139362 2024-09-14T09:07:31.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.785894+0000", 2024-09-14T09:07:31.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069204800000000001 2024-09-14T09:07:31.776 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.776 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.776 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.776 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104486 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 486400~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:31.776 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680891+0000", 2024-09-14T09:07:31.776 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 11.046780404, 2024-09-14T09:07:31.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.105913091, 2024-09-14T09:07:31.776 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.776 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.777 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:31.777 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:31.777 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:31.777 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.777 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104486 2024-09-14T09:07:31.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.777 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680891+0000", 2024-09-14T09:07:31.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680891+0000", 2024-09-14T09:07:31.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680891+0000", 2024-09-14T09:07:31.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1299999999999999e-07 2024-09-14T09:07:31.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680894+0000", 2024-09-14T09:07:31.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0199999999999999e-06 2024-09-14T09:07:31.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680904+0000", 2024-09-14T09:07:31.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2140000000000002e-06 2024-09-14T09:07:31.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680911+0000", 2024-09-14T09:07:31.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1130000000000003e-06 2024-09-14T09:07:31.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.785969+0000", 2024-09-14T09:07:31.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.105058734 2024-09-14T09:07:31.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.786104+0000", 2024-09-14T09:07:31.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013463900000000001 2024-09-14T09:07:31.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.786804+0000", 2024-09-14T09:07:31.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069965800000000003 2024-09-14T09:07:31.781 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.781 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.781 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.781 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104488 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 490496~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:31.781 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680925+0000", 2024-09-14T09:07:31.781 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 11.046746200999999, 2024-09-14T09:07:31.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10678225700000001, 2024-09-14T09:07:31.781 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.781 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.781 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:31.781 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:31.782 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:31.782 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.782 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104488 2024-09-14T09:07:31.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.782 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680925+0000", 2024-09-14T09:07:31.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680925+0000", 2024-09-14T09:07:31.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680926+0000", 2024-09-14T09:07:31.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-07 2024-09-14T09:07:31.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680928+0000", 2024-09-14T09:07:31.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8870000000000001e-06 2024-09-14T09:07:31.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680940+0000", 2024-09-14T09:07:31.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1717e-05 2024-09-14T09:07:31.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680948+0000", 2024-09-14T09:07:31.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8429999999999993e-06 2024-09-14T09:07:31.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.786883+0000", 2024-09-14T09:07:31.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10593530399999999 2024-09-14T09:07:31.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.787015+0000", 2024-09-14T09:07:31.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013217200000000001 2024-09-14T09:07:31.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.787707+0000", 2024-09-14T09:07:31.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069162399999999995 2024-09-14T09:07:31.785 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.785 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.785 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.785 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104490 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 494592~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:31.786 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.680960+0000", 2024-09-14T09:07:31.786 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 11.046710864, 2024-09-14T09:07:31.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.107672767, 2024-09-14T09:07:31.786 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.786 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.786 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:31.786 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:31.786 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:31.786 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.786 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104490 2024-09-14T09:07:31.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.786 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.786 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680960+0000", 2024-09-14T09:07:31.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680960+0000", 2024-09-14T09:07:31.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680961+0000", 2024-09-14T09:07:31.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-07 2024-09-14T09:07:31.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680964+0000", 2024-09-14T09:07:31.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3859999999999999e-06 2024-09-14T09:07:31.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680974+0000", 2024-09-14T09:07:31.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3700000000000001e-06 2024-09-14T09:07:31.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.680981+0000", 2024-09-14T09:07:31.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1230000000000002e-06 2024-09-14T09:07:31.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.787780+0000", 2024-09-14T09:07:31.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.106799291 2024-09-14T09:07:31.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.787914+0000", 2024-09-14T09:07:31.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013379200000000001 2024-09-14T09:07:31.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.788633+0000", 2024-09-14T09:07:31.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000719095 2024-09-14T09:07:31.790 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.790 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.790 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.790 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104493 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 498688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:31.790 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681058+0000", 2024-09-14T09:07:31.790 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 11.046613345000001, 2024-09-14T09:07:31.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.108480707, 2024-09-14T09:07:31.791 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.791 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.791 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:31.791 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:31.791 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:31.791 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.791 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104493 2024-09-14T09:07:31.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.791 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681058+0000", 2024-09-14T09:07:31.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681058+0000", 2024-09-14T09:07:31.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681059+0000", 2024-09-14T09:07:31.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1200000000000001e-06 2024-09-14T09:07:31.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681062+0000", 2024-09-14T09:07:31.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3100000000000001e-06 2024-09-14T09:07:31.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681078+0000", 2024-09-14T09:07:31.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5630000000000001e-05 2024-09-14T09:07:31.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681096+0000", 2024-09-14T09:07:31.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8036000000000001e-05 2024-09-14T09:07:31.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.788719+0000", 2024-09-14T09:07:31.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.107623651 2024-09-14T09:07:31.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.788851+0000", 2024-09-14T09:07:31.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000131402 2024-09-14T09:07:31.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.789538+0000", 2024-09-14T09:07:31.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068755800000000001 2024-09-14T09:07:31.794 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.794 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.794 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.795 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104494 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 502784~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:31.795 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681108+0000", 2024-09-14T09:07:31.795 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 11.046562582, 2024-09-14T09:07:31.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10933915700000001, 2024-09-14T09:07:31.795 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.795 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.795 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:31.795 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:31.795 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:31.795 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.795 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104494 2024-09-14T09:07:31.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.796 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681108+0000", 2024-09-14T09:07:31.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681108+0000", 2024-09-14T09:07:31.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681109+0000", 2024-09-14T09:07:31.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0999999999999997e-07 2024-09-14T09:07:31.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681113+0000", 2024-09-14T09:07:31.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4659999999999999e-06 2024-09-14T09:07:31.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681122+0000", 2024-09-14T09:07:31.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5140000000000007e-06 2024-09-14T09:07:31.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681129+0000", 2024-09-14T09:07:31.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1130000000000003e-06 2024-09-14T09:07:31.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.789610+0000", 2024-09-14T09:07:31.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.10848097700000001 2024-09-14T09:07:31.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.789748+0000", 2024-09-14T09:07:31.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013734899999999999 2024-09-14T09:07:31.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.790448+0000", 2024-09-14T09:07:31.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069992800000000005 2024-09-14T09:07:31.799 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.799 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.799 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.799 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104496 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 506880~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:31.799 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681140+0000", 2024-09-14T09:07:31.799 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 11.046531436, 2024-09-14T09:07:31.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.110205621, 2024-09-14T09:07:31.799 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.799 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.799 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:31.799 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:31.800 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:31.800 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.800 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104496 2024-09-14T09:07:31.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.800 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681140+0000", 2024-09-14T09:07:31.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681140+0000", 2024-09-14T09:07:31.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681141+0000", 2024-09-14T09:07:31.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8700000000000004e-07 2024-09-14T09:07:31.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681144+0000", 2024-09-14T09:07:31.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0259999999999999e-06 2024-09-14T09:07:31.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681153+0000", 2024-09-14T09:07:31.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9770000000000006e-06 2024-09-14T09:07:31.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681159+0000", 2024-09-14T09:07:31.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.72e-06 2024-09-14T09:07:31.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.790527+0000", 2024-09-14T09:07:31.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.109368144 2024-09-14T09:07:31.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.790660+0000", 2024-09-14T09:07:31.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013242500000000001 2024-09-14T09:07:31.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.791345+0000", 2024-09-14T09:07:31.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068534200000000003 2024-09-14T09:07:31.803 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.803 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.803 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.803 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104498 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 510976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:31.803 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681169+0000", 2024-09-14T09:07:31.804 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 11.046501669, 2024-09-14T09:07:31.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.111099946, 2024-09-14T09:07:31.804 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.804 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.804 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:31.804 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:31.804 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:31.804 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.804 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104498 2024-09-14T09:07:31.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.804 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681169+0000", 2024-09-14T09:07:31.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681169+0000", 2024-09-14T09:07:31.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681170+0000", 2024-09-14T09:07:31.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9599999999999999e-07 2024-09-14T09:07:31.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681173+0000", 2024-09-14T09:07:31.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:07:31.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681182+0000", 2024-09-14T09:07:31.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8100000000000004e-06 2024-09-14T09:07:31.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681189+0000", 2024-09-14T09:07:31.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-06 2024-09-14T09:07:31.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.791421+0000", 2024-09-14T09:07:31.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11023223 2024-09-14T09:07:31.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.791553+0000", 2024-09-14T09:07:31.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013137899999999999 2024-09-14T09:07:31.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.792269+0000", 2024-09-14T09:07:31.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071678099999999999 2024-09-14T09:07:31.809 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.809 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.809 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.810 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104500 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 515072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:31.810 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681199+0000", 2024-09-14T09:07:31.810 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 11.046472129, 2024-09-14T09:07:31.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.111976886, 2024-09-14T09:07:31.810 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.810 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.810 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:31.810 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:31.810 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:31.810 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.810 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104500 2024-09-14T09:07:31.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.811 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681199+0000", 2024-09-14T09:07:31.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681199+0000", 2024-09-14T09:07:31.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681200+0000", 2024-09-14T09:07:31.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3600000000000003e-07 2024-09-14T09:07:31.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681202+0000", 2024-09-14T09:07:31.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7870000000000002e-06 2024-09-14T09:07:31.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681211+0000", 2024-09-14T09:07:31.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7730000000000005e-06 2024-09-14T09:07:31.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681218+0000", 2024-09-14T09:07:31.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9670000000000004e-06 2024-09-14T09:07:31.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.792355+0000", 2024-09-14T09:07:31.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11113640599999999 2024-09-14T09:07:31.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.792486+0000", 2024-09-14T09:07:31.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013117900000000001 2024-09-14T09:07:31.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.793176+0000", 2024-09-14T09:07:31.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069003800000000002 2024-09-14T09:07:31.814 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.814 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.814 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.814 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104502 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 519168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:31.814 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681228+0000", 2024-09-14T09:07:31.814 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 11.04644291, 2024-09-14T09:07:31.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11285799000000001, 2024-09-14T09:07:31.814 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.814 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.815 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:31.815 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:31.815 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:31.815 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.815 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104502 2024-09-14T09:07:31.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.815 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681228+0000", 2024-09-14T09:07:31.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681228+0000", 2024-09-14T09:07:31.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681229+0000", 2024-09-14T09:07:31.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4399999999999999e-07 2024-09-14T09:07:31.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681232+0000", 2024-09-14T09:07:31.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8289999999999999e-06 2024-09-14T09:07:31.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681241+0000", 2024-09-14T09:07:31.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9239999999999996e-06 2024-09-14T09:07:31.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681248+0000", 2024-09-14T09:07:31.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2389999999999996e-06 2024-09-14T09:07:31.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.793253+0000", 2024-09-14T09:07:31.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.11200555700000001 2024-09-14T09:07:31.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.793385+0000", 2024-09-14T09:07:31.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000131929 2024-09-14T09:07:31.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.794086+0000", 2024-09-14T09:07:31.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070076800000000005 2024-09-14T09:07:31.818 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.818 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.818 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.819 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:104504 3.3 3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27 [read 523264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-14T09:07:31.819 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:20.681258+0000", 2024-09-14T09:07:31.819 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 11.046412604, 2024-09-14T09:07:31.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.113704203, 2024-09-14T09:07:31.819 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:31.819 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:31.819 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:31.819 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:31.819 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:31.819 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:31.819 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 104504 2024-09-14T09:07:31.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.819 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:31.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:31.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681258+0000", 2024-09-14T09:07:31.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:31.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681258+0000", 2024-09-14T09:07:31.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:31.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:31.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681259+0000", 2024-09-14T09:07:31.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9999999999999997e-07 2024-09-14T09:07:31.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:31.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681271+0000", 2024-09-14T09:07:31.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1674e-05 2024-09-14T09:07:31.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:31.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681281+0000", 2024-09-14T09:07:31.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7100000000000002e-06 2024-09-14T09:07:31.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:31.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.681288+0000", 2024-09-14T09:07:31.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2660000000000004e-06 2024-09-14T09:07:31.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:31.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.794140+0000", 2024-09-14T09:07:31.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.112852483 2024-09-14T09:07:31.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:31.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.794270+0000", 2024-09-14T09:07:31.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000129846 2024-09-14T09:07:31.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:31.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:31.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:31.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:20.794963+0000", 2024-09-14T09:07:31.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00069252400000000003 2024-09-14T09:07:31.823 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.823 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.823 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.823 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:31.823 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:31.823 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:31.824 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:07:31.825 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:31.827+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:31.925 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:31.923+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:31.990 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:31.990 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:31.990 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:07:31.990 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:32.005 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:07:32.026 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:32.027+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:32.127 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:32.127+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:32.144 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:32.144 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:32.144 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:07:32.144 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:07:32.144 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:32.158 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:07:32.189 INFO:teuthology.orchestra.run.smithi176.stdout:5472 write 0x8775557 thru 0x8778e61 (0x390b bytes) 2024-09-14T09:07:32.195 INFO:teuthology.orchestra.run.smithi176.stdout:5473 read 0x455a1fa thru 0x455e916 (0x471d bytes) 2024-09-14T09:07:32.195 INFO:teuthology.orchestra.run.smithi176.stdout:5474 write 0x6383608 thru 0x6387756 (0x414f bytes) 2024-09-14T09:07:32.200 INFO:teuthology.orchestra.run.smithi176.stdout:5476 read 0x39b447b thru 0x39b74e9 (0x306f bytes) 2024-09-14T09:07:32.214 INFO:teuthology.orchestra.run.smithi176.stdout:5477 write 0xa2c4696 thru 0xa2c8244 (0x3baf bytes) 2024-09-14T09:07:32.218 INFO:teuthology.orchestra.run.smithi176.stdout:5478 read 0x5a24335 thru 0x5a2c377 (0x8043 bytes) 2024-09-14T09:07:32.218 INFO:teuthology.orchestra.run.smithi176.stdout:5480 write 0x49fabc0 thru 0x4a054d3 (0xa914 bytes) 2024-09-14T09:07:32.228 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:32.227+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:32.228 INFO:teuthology.orchestra.run.smithi176.stdout:5482 read 0x7b84043 thru 0x7b86c8c (0x2c4a bytes) 2024-09-14T09:07:32.252 INFO:teuthology.orchestra.run.smithi176.stdout:5483 trunc from 0xa2c8245 to 0x1622eda 2024-09-14T09:07:32.284 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:32.284 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:07:32.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:07:32.285 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:07:32.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.285 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105615 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 382976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.285 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:07:32.285 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.422358093, 2024-09-14T09:07:32.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117270900000001, 2024-09-14T09:07:32.285 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.285 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.285 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.285 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.285 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.285 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.285 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105615 2024-09-14T09:07:32.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.286 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:07:32.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:07:32.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864368+0000", 2024-09-14T09:07:32.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.043e-06 2024-09-14T09:07:32.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864373+0000", 2024-09-14T09:07:32.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4669999999999998e-06 2024-09-14T09:07:32.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864387+0000", 2024-09-14T09:07:32.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3876000000000001e-05 2024-09-14T09:07:32.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864397+0000", 2024-09-14T09:07:32.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0592e-05 2024-09-14T09:07:32.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866069+0000", 2024-09-14T09:07:32.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016714410000000001 2024-09-14T09:07:32.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866076+0000", 2024-09-14T09:07:32.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:07:32.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.075854+0000", 2024-09-14T09:07:32.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.209778501 2024-09-14T09:07:32.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.075940+0000", 2024-09-14T09:07:32.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5357000000000003e-05 2024-09-14T09:07:32.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.076094+0000", 2024-09-14T09:07:32.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000154745 2024-09-14T09:07:32.290 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.290 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.290 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.290 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105629 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 411648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.290 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:07:32.290 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.420204371000001, 2024-09-14T09:07:32.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2115591910000001, 2024-09-14T09:07:32.290 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.290 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.290 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.290 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.290 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.291 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.291 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105629 2024-09-14T09:07:32.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.291 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:07:32.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:07:32.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866522+0000", 2024-09-14T09:07:32.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6899999999999996e-07 2024-09-14T09:07:32.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866526+0000", 2024-09-14T09:07:32.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0110000000000002e-06 2024-09-14T09:07:32.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866539+0000", 2024-09-14T09:07:32.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3013000000000001e-05 2024-09-14T09:07:32.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866549+0000", 2024-09-14T09:07:32.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8490000000000001e-06 2024-09-14T09:07:32.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866650+0000", 2024-09-14T09:07:32.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101054 2024-09-14T09:07:32.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866664+0000", 2024-09-14T09:07:32.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3805000000000001e-05 2024-09-14T09:07:32.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.077849+0000", 2024-09-14T09:07:32.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2111855419999999 2024-09-14T09:07:32.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.077945+0000", 2024-09-14T09:07:32.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6086000000000001e-05 2024-09-14T09:07:32.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078080+0000", 2024-09-14T09:07:32.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000134862 2024-09-14T09:07:32.295 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.295 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.295 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.295 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105631 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 415744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.295 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:07:32.295 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.420145152, 2024-09-14T09:07:32.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119463509999999, 2024-09-14T09:07:32.295 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.295 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.295 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.295 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.295 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.296 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.296 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105631 2024-09-14T09:07:32.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.296 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:07:32.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:07:32.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866581+0000", 2024-09-14T09:07:32.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.879999999999999e-07 2024-09-14T09:07:32.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866585+0000", 2024-09-14T09:07:32.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0239999999999999e-06 2024-09-14T09:07:32.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866598+0000", 2024-09-14T09:07:32.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2862e-05 2024-09-14T09:07:32.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866608+0000", 2024-09-14T09:07:32.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9890000000000004e-06 2024-09-14T09:07:32.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866708+0000", 2024-09-14T09:07:32.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9605999999999995e-05 2024-09-14T09:07:32.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866720+0000", 2024-09-14T09:07:32.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.261e-05 2024-09-14T09:07:32.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078137+0000", 2024-09-14T09:07:32.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211416571 2024-09-14T09:07:32.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078293+0000", 2024-09-14T09:07:32.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000156 2024-09-14T09:07:32.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078527+0000", 2024-09-14T09:07:32.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000233701 2024-09-14T09:07:32.300 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.300 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.300 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.300 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105633 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.300 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:07:32.300 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.420086104999999, 2024-09-14T09:07:32.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211941734, 2024-09-14T09:07:32.300 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.301 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.301 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:32.301 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.301 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.301 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.301 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105633 2024-09-14T09:07:32.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.301 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:07:32.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:07:32.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866640+0000", 2024-09-14T09:07:32.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9300000000000006e-07 2024-09-14T09:07:32.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866656+0000", 2024-09-14T09:07:32.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5291e-05 2024-09-14T09:07:32.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866669+0000", 2024-09-14T09:07:32.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2918000000000001e-05 2024-09-14T09:07:32.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866678+0000", 2024-09-14T09:07:32.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8369999999999992e-06 2024-09-14T09:07:32.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866771+0000", 2024-09-14T09:07:32.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2250999999999995e-05 2024-09-14T09:07:32.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866784+0000", 2024-09-14T09:07:32.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2991e-05 2024-09-14T09:07:32.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078566+0000", 2024-09-14T09:07:32.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117822570000001 2024-09-14T09:07:32.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078581+0000", 2024-09-14T09:07:32.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5196e-05 2024-09-14T09:07:32.305 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.305 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.305 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.305 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105635 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.305 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:07:32.305 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.420032278000001, 2024-09-14T09:07:32.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119313250000001, 2024-09-14T09:07:32.305 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.305 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.305 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:32.305 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.306 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.306 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.306 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105635 2024-09-14T09:07:32.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.306 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:07:32.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:07:32.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866694+0000", 2024-09-14T09:07:32.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0210000000000001e-06 2024-09-14T09:07:32.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866698+0000", 2024-09-14T09:07:32.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.765e-06 2024-09-14T09:07:32.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866710+0000", 2024-09-14T09:07:32.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2320000000000001e-05 2024-09-14T09:07:32.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866722+0000", 2024-09-14T09:07:32.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2133000000000001e-05 2024-09-14T09:07:32.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866830+0000", 2024-09-14T09:07:32.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000107668 2024-09-14T09:07:32.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866843+0000", 2024-09-14T09:07:32.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2991e-05 2024-09-14T09:07:32.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078616+0000", 2024-09-14T09:07:32.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117732459999999 2024-09-14T09:07:32.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078624+0000", 2024-09-14T09:07:32.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1810000000000001e-06 2024-09-14T09:07:32.309 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.309 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.310 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.310 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105636 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.310 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:07:32.310 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.419964382, 2024-09-14T09:07:32.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119085119999999, 2024-09-14T09:07:32.310 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.310 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.310 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:32.310 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.310 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.310 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.310 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105636 2024-09-14T09:07:32.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.311 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:07:32.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:07:32.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866762+0000", 2024-09-14T09:07:32.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4600000000000003e-07 2024-09-14T09:07:32.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866766+0000", 2024-09-14T09:07:32.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:07:32.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866779+0000", 2024-09-14T09:07:32.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3101999999999999e-05 2024-09-14T09:07:32.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866789+0000", 2024-09-14T09:07:32.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8039999999999996e-06 2024-09-14T09:07:32.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866889+0000", 2024-09-14T09:07:32.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100147 2024-09-14T09:07:32.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866902+0000", 2024-09-14T09:07:32.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3212e-05 2024-09-14T09:07:32.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078659+0000", 2024-09-14T09:07:32.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211756362 2024-09-14T09:07:32.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078670+0000", 2024-09-14T09:07:32.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0956e-05 2024-09-14T09:07:32.314 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.314 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.314 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.315 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105639 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.315 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:07:32.315 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.419905506999999, 2024-09-14T09:07:32.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118829259999999, 2024-09-14T09:07:32.315 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.315 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.315 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:32.315 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.315 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.315 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.315 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105639 2024-09-14T09:07:32.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.315 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:07:32.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:07:32.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866821+0000", 2024-09-14T09:07:32.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9800000000000002e-07 2024-09-14T09:07:32.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866825+0000", 2024-09-14T09:07:32.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9269999999999998e-06 2024-09-14T09:07:32.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866838+0000", 2024-09-14T09:07:32.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3040000000000001e-05 2024-09-14T09:07:32.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866848+0000", 2024-09-14T09:07:32.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7929999999999992e-06 2024-09-14T09:07:32.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866948+0000", 2024-09-14T09:07:32.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100337 2024-09-14T09:07:32.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866961+0000", 2024-09-14T09:07:32.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3106e-05 2024-09-14T09:07:32.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078693+0000", 2024-09-14T09:07:32.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117320380000001 2024-09-14T09:07:32.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078703+0000", 2024-09-14T09:07:32.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6870000000000006e-06 2024-09-14T09:07:32.319 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.319 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.319 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.319 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105641 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.319 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:07:32.319 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.419846642, 2024-09-14T09:07:32.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118727169999999, 2024-09-14T09:07:32.320 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.320 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.320 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:32.320 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.320 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.320 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.320 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105641 2024-09-14T09:07:32.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.320 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:07:32.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:07:32.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866880+0000", 2024-09-14T09:07:32.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7699999999999992e-07 2024-09-14T09:07:32.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866884+0000", 2024-09-14T09:07:32.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9480000000000001e-06 2024-09-14T09:07:32.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866897+0000", 2024-09-14T09:07:32.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2832e-05 2024-09-14T09:07:32.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866906+0000", 2024-09-14T09:07:32.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7869999999999996e-06 2024-09-14T09:07:32.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867007+0000", 2024-09-14T09:07:32.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010103900000000001 2024-09-14T09:07:32.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867020+0000", 2024-09-14T09:07:32.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2896e-05 2024-09-14T09:07:32.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078739+0000", 2024-09-14T09:07:32.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211718305 2024-09-14T09:07:32.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078752+0000", 2024-09-14T09:07:32.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2933e-05 2024-09-14T09:07:32.323 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.323 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.324 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.324 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105643 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.324 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:07:32.324 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.419788083, 2024-09-14T09:07:32.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211858353, 2024-09-14T09:07:32.324 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.324 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.324 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:32.324 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.324 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.324 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.324 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105643 2024-09-14T09:07:32.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.325 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:07:32.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:07:32.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866938+0000", 2024-09-14T09:07:32.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5600000000000004e-07 2024-09-14T09:07:32.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866942+0000", 2024-09-14T09:07:32.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0350000000000003e-06 2024-09-14T09:07:32.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866955+0000", 2024-09-14T09:07:32.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3097e-05 2024-09-14T09:07:32.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866965+0000", 2024-09-14T09:07:32.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8179999999999999e-06 2024-09-14T09:07:32.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867067+0000", 2024-09-14T09:07:32.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010128699999999999 2024-09-14T09:07:32.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867080+0000", 2024-09-14T09:07:32.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3089e-05 2024-09-14T09:07:32.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078786+0000", 2024-09-14T09:07:32.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117065250000001 2024-09-14T09:07:32.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078796+0000", 2024-09-14T09:07:32.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5459999999999997e-06 2024-09-14T09:07:32.328 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.328 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.328 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.328 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105645 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.329 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:07:32.329 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.419727688, 2024-09-14T09:07:32.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211848641, 2024-09-14T09:07:32.329 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.329 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.329 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:32.329 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.329 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.329 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.329 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105645 2024-09-14T09:07:32.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.329 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:07:32.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:07:32.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866999+0000", 2024-09-14T09:07:32.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7999999999999993e-07 2024-09-14T09:07:32.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867003+0000", 2024-09-14T09:07:32.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0090000000000001e-06 2024-09-14T09:07:32.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867016+0000", 2024-09-14T09:07:32.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3304e-05 2024-09-14T09:07:32.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867026+0000", 2024-09-14T09:07:32.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5889999999999992e-06 2024-09-14T09:07:32.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867128+0000", 2024-09-14T09:07:32.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000102101 2024-09-14T09:07:32.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867141+0000", 2024-09-14T09:07:32.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2903e-05 2024-09-14T09:07:32.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078833+0000", 2024-09-14T09:07:32.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116920959999999 2024-09-14T09:07:32.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078846+0000", 2024-09-14T09:07:32.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3658999999999999e-05 2024-09-14T09:07:32.333 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.333 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.333 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.333 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105647 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.333 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:07:32.333 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.419669170000001, 2024-09-14T09:07:32.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211830097, 2024-09-14T09:07:32.333 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.333 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.333 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:32.333 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.334 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.334 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.334 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105647 2024-09-14T09:07:32.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.334 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:07:32.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:07:32.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867057+0000", 2024-09-14T09:07:32.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.95e-07 2024-09-14T09:07:32.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867061+0000", 2024-09-14T09:07:32.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9530000000000001e-06 2024-09-14T09:07:32.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867074+0000", 2024-09-14T09:07:32.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3061000000000001e-05 2024-09-14T09:07:32.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867086+0000", 2024-09-14T09:07:32.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1460999999999999e-05 2024-09-14T09:07:32.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867186+0000", 2024-09-14T09:07:32.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100181 2024-09-14T09:07:32.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867199+0000", 2024-09-14T09:07:32.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-05 2024-09-14T09:07:32.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078874+0000", 2024-09-14T09:07:32.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211674683 2024-09-14T09:07:32.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078886+0000", 2024-09-14T09:07:32.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2853e-05 2024-09-14T09:07:32.337 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.337 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.337 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.338 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105648 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.338 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:07:32.338 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.419607684000001, 2024-09-14T09:07:32.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118073570000001, 2024-09-14T09:07:32.338 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.338 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.338 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:32.338 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.338 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.338 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.338 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105648 2024-09-14T09:07:32.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.338 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:07:32.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:07:32.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867119+0000", 2024-09-14T09:07:32.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8400000000000002e-07 2024-09-14T09:07:32.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867123+0000", 2024-09-14T09:07:32.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9639999999999997e-06 2024-09-14T09:07:32.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867136+0000", 2024-09-14T09:07:32.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.289e-05 2024-09-14T09:07:32.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867145+0000", 2024-09-14T09:07:32.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8919999999999995e-06 2024-09-14T09:07:32.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867245+0000", 2024-09-14T09:07:32.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9282000000000003e-05 2024-09-14T09:07:32.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867258+0000", 2024-09-14T09:07:32.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2935000000000001e-05 2024-09-14T09:07:32.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078910+0000", 2024-09-14T09:07:32.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211652352 2024-09-14T09:07:32.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078925+0000", 2024-09-14T09:07:32.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5058000000000001e-05 2024-09-14T09:07:32.342 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.342 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.342 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.343 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105652 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.343 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:07:32.343 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.419549665, 2024-09-14T09:07:32.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211775737, 2024-09-14T09:07:32.343 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.343 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.343 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:32.343 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.344 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.344 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.344 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105652 2024-09-14T09:07:32.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.344 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:07:32.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:07:32.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867177+0000", 2024-09-14T09:07:32.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9100000000000002e-07 2024-09-14T09:07:32.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867181+0000", 2024-09-14T09:07:32.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0029999999999996e-06 2024-09-14T09:07:32.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867193+0000", 2024-09-14T09:07:32.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2675e-05 2024-09-14T09:07:32.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867203+0000", 2024-09-14T09:07:32.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5599999999999999e-06 2024-09-14T09:07:32.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867306+0000", 2024-09-14T09:07:32.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010330100000000001 2024-09-14T09:07:32.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867319+0000", 2024-09-14T09:07:32.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3129000000000001e-05 2024-09-14T09:07:32.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078944+0000", 2024-09-14T09:07:32.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116247520000001 2024-09-14T09:07:32.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078952+0000", 2024-09-14T09:07:32.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4259999999999997e-06 2024-09-14T09:07:32.348 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.348 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.348 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.348 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105653 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.348 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:07:32.348 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.419490995, 2024-09-14T09:07:32.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211774895, 2024-09-14T09:07:32.349 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.349 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.349 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:32.349 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.349 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.349 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.349 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105653 2024-09-14T09:07:32.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.349 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:07:32.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:07:32.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867235+0000", 2024-09-14T09:07:32.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9400000000000004e-07 2024-09-14T09:07:32.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867239+0000", 2024-09-14T09:07:32.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9380000000000002e-06 2024-09-14T09:07:32.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867252+0000", 2024-09-14T09:07:32.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.31e-05 2024-09-14T09:07:32.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867264+0000", 2024-09-14T09:07:32.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1275e-05 2024-09-14T09:07:32.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867367+0000", 2024-09-14T09:07:32.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103755 2024-09-14T09:07:32.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867382+0000", 2024-09-14T09:07:32.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4724e-05 2024-09-14T09:07:32.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078995+0000", 2024-09-14T09:07:32.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116132669999999 2024-09-14T09:07:32.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079009+0000", 2024-09-14T09:07:32.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3942e-05 2024-09-14T09:07:32.353 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.353 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.353 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.353 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105654 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.353 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:07:32.353 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.419429939, 2024-09-14T09:07:32.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117364070000001, 2024-09-14T09:07:32.354 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.354 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.354 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:32.354 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.354 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.354 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.354 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105654 2024-09-14T09:07:32.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.354 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:07:32.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:07:32.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867296+0000", 2024-09-14T09:07:32.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2399999999999996e-07 2024-09-14T09:07:32.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867300+0000", 2024-09-14T09:07:32.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8689999999999997e-06 2024-09-14T09:07:32.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867313+0000", 2024-09-14T09:07:32.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2660000000000001e-05 2024-09-14T09:07:32.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867322+0000", 2024-09-14T09:07:32.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.533e-06 2024-09-14T09:07:32.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867429+0000", 2024-09-14T09:07:32.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106566 2024-09-14T09:07:32.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867442+0000", 2024-09-14T09:07:32.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3244000000000001e-05 2024-09-14T09:07:32.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079024+0000", 2024-09-14T09:07:32.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211581813 2024-09-14T09:07:32.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079032+0000", 2024-09-14T09:07:32.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7980000000000006e-06 2024-09-14T09:07:32.358 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.358 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.358 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.358 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105657 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.358 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:07:32.359 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.419368766, 2024-09-14T09:07:32.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211700657, 2024-09-14T09:07:32.359 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.359 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.359 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:32.359 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.359 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.359 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.359 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105657 2024-09-14T09:07:32.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.359 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:07:32.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:07:32.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867358+0000", 2024-09-14T09:07:32.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9700000000000005e-07 2024-09-14T09:07:32.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867367+0000", 2024-09-14T09:07:32.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8919999999999995e-06 2024-09-14T09:07:32.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867383+0000", 2024-09-14T09:07:32.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.579e-05 2024-09-14T09:07:32.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867393+0000", 2024-09-14T09:07:32.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3070000000000008e-06 2024-09-14T09:07:32.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867490+0000", 2024-09-14T09:07:32.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7703999999999998e-05 2024-09-14T09:07:32.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867503+0000", 2024-09-14T09:07:32.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2849e-05 2024-09-14T09:07:32.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079050+0000", 2024-09-14T09:07:32.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211546899 2024-09-14T09:07:32.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079057+0000", 2024-09-14T09:07:32.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3189999999999997e-06 2024-09-14T09:07:32.363 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.363 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.363 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.363 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105658 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.363 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:07:32.363 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.419306382, 2024-09-14T09:07:32.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211691217, 2024-09-14T09:07:32.364 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.364 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.364 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:32.364 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.364 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.364 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.364 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105658 2024-09-14T09:07:32.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.364 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:07:32.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:07:32.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867420+0000", 2024-09-14T09:07:32.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5000000000000001e-07 2024-09-14T09:07:32.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867424+0000", 2024-09-14T09:07:32.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8469999999999998e-06 2024-09-14T09:07:32.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867437+0000", 2024-09-14T09:07:32.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3156000000000001e-05 2024-09-14T09:07:32.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867448+0000", 2024-09-14T09:07:32.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1185e-05 2024-09-14T09:07:32.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867552+0000", 2024-09-14T09:07:32.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103362 2024-09-14T09:07:32.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867565+0000", 2024-09-14T09:07:32.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3018e-05 2024-09-14T09:07:32.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079092+0000", 2024-09-14T09:07:32.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211527013 2024-09-14T09:07:32.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079110+0000", 2024-09-14T09:07:32.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8686e-05 2024-09-14T09:07:32.368 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.368 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.368 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.368 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105662 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.368 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:07:32.368 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.419245317, 2024-09-14T09:07:32.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211652226, 2024-09-14T09:07:32.368 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.368 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.368 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:32.368 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.368 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.369 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.369 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105662 2024-09-14T09:07:32.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.369 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:07:32.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:07:32.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867481+0000", 2024-09-14T09:07:32.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1800000000000004e-07 2024-09-14T09:07:32.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867485+0000", 2024-09-14T09:07:32.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9199999999999997e-06 2024-09-14T09:07:32.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867498+0000", 2024-09-14T09:07:32.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2896e-05 2024-09-14T09:07:32.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867509+0000", 2024-09-14T09:07:32.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1202e-05 2024-09-14T09:07:32.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867611+0000", 2024-09-14T09:07:32.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010166999999999999 2024-09-14T09:07:32.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867624+0000", 2024-09-14T09:07:32.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2996000000000001e-05 2024-09-14T09:07:32.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079125+0000", 2024-09-14T09:07:32.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2115009299999999 2024-09-14T09:07:32.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079132+0000", 2024-09-14T09:07:32.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6939999999999995e-06 2024-09-14T09:07:32.372 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.372 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.372 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.373 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105663 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.373 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:07:32.373 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.419184266, 2024-09-14T09:07:32.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116218169999999, 2024-09-14T09:07:32.373 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.373 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.373 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:32.373 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.373 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.373 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.373 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105663 2024-09-14T09:07:32.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.373 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:07:32.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:07:32.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867542+0000", 2024-09-14T09:07:32.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1399999999999995e-07 2024-09-14T09:07:32.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867546+0000", 2024-09-14T09:07:32.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7900000000000001e-06 2024-09-14T09:07:32.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867559+0000", 2024-09-14T09:07:32.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2805e-05 2024-09-14T09:07:32.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867568+0000", 2024-09-14T09:07:32.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-09-14T09:07:32.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867669+0000", 2024-09-14T09:07:32.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100655 2024-09-14T09:07:32.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867682+0000", 2024-09-14T09:07:32.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3013000000000001e-05 2024-09-14T09:07:32.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079151+0000", 2024-09-14T09:07:32.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2114689089999999 2024-09-14T09:07:32.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079163+0000", 2024-09-14T09:07:32.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1962e-05 2024-09-14T09:07:32.377 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.377 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.377 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.377 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105665 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:32.377 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:07:32.377 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.419125635, 2024-09-14T09:07:32.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211584848, 2024-09-14T09:07:32.378 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.378 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.378 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:32.378 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.378 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.378 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.378 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105665 2024-09-14T09:07:32.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.378 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:07:32.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:07:32.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867601+0000", 2024-09-14T09:07:32.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.569999999999999e-07 2024-09-14T09:07:32.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867605+0000", 2024-09-14T09:07:32.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8589999999999998e-06 2024-09-14T09:07:32.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867617+0000", 2024-09-14T09:07:32.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2833000000000001e-05 2024-09-14T09:07:32.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867627+0000", 2024-09-14T09:07:32.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5219999999999996e-06 2024-09-14T09:07:32.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867728+0000", 2024-09-14T09:07:32.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010057500000000001 2024-09-14T09:07:32.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:32.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867740+0000", 2024-09-14T09:07:32.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2744e-05 2024-09-14T09:07:32.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079178+0000", 2024-09-14T09:07:32.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211437552 2024-09-14T09:07:32.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079185+0000", 2024-09-14T09:07:32.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8059999999999997e-06 2024-09-14T09:07:32.382 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.382 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.382 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.382 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.382 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.382 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:32.384 INFO:teuthology.orchestra.run.smithi176.stdout:5484 trunc from 0x1622eda to 0x7c14620 2024-09-14T09:07:32.384 INFO:teuthology.orchestra.run.smithi176.stdout:5486 writesame 0xdab460f thru 0xdac2b81 data_size 0x1(0xe573 bytes) 2024-09-14T09:07:32.384 INFO:teuthology.orchestra.run.smithi176.stdout:5487 trunc from 0xdac2b82 to 0xe2c4a8f 2024-09-14T09:07:32.384 INFO:teuthology.orchestra.run.smithi176.stdout:5488 read 0x404931a thru 0x4056c94 (0xd97b bytes) 2024-09-14T09:07:32.384 INFO:teuthology.orchestra.run.smithi176.stdout:5489 read 0x7fdd404 thru 0x7fea3a3 (0xcfa0 bytes) 2024-09-14T09:07:32.384 INFO:teuthology.orchestra.run.smithi176.stdout:5490 clone 40 order 19 su 65536 sc 13 2024-09-14T09:07:32.384 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:07:32.385 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:32.386+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:32.485 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:32.483+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:32.555 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:32.555 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:32.555 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:07:32.555 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:32.569 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:07:32.585 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:32.583+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:32.686 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:32.686+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:32.686 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:32.686 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:32.687 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:07:32.687 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:07:32.687 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:32.700 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:07:32.787 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:32.790+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:32.815 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:07:32.864 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:32.864 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:07:32.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:07:32.865 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:07:32.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.865 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65139 3.a 3:54c90b7d:::rbd_data.115e21c31fcf.0000000000000008:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:07:32.865 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:07:32.865 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.487198707999994, 2024-09-14T09:07:32.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14429895700000001, 2024-09-14T09:07:32.865 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.865 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.865 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:32.865 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.865 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.865 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.865 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65139 2024-09-14T09:07:32.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.866 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:07:32.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:07:32.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379715+0000", 2024-09-14T09:07:32.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.576e-06 2024-09-14T09:07:32.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379724+0000", 2024-09-14T09:07:32.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1779999999999992e-06 2024-09-14T09:07:32.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379746+0000", 2024-09-14T09:07:32.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1661000000000001e-05 2024-09-14T09:07:32.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379759+0000", 2024-09-14T09:07:32.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.324e-05 2024-09-14T09:07:32.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379882+0000", 2024-09-14T09:07:32.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012239400000000001 2024-09-14T09:07:32.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.380023+0000", 2024-09-14T09:07:32.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000141402 2024-09-14T09:07:32.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.391553+0000", 2024-09-14T09:07:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011530197000000001 2024-09-14T09:07:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523927+0000", 2024-09-14T09:07:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.132373935 2024-09-14T09:07:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523941+0000", 2024-09-14T09:07:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3326000000000001e-05 2024-09-14T09:07:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523977+0000", 2024-09-14T09:07:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6185e-05 2024-09-14T09:07:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.524012+0000", 2024-09-14T09:07:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4863000000000001e-05 2024-09-14T09:07:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.870 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107606 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 14336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:32.871 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:07:32.871 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4607418069999998, 2024-09-14T09:07:32.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14293513799999999, 2024-09-14T09:07:32.871 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.871 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.871 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.871 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.871 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.871 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.871 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107606 2024-09-14T09:07:32.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.871 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:07:32.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:07:32.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:07:32.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1600000000000001e-07 2024-09-14T09:07:32.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406174+0000", 2024-09-14T09:07:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4199999999999999e-06 2024-09-14T09:07:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406188+0000", 2024-09-14T09:07:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4114000000000001e-05 2024-09-14T09:07:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406198+0000", 2024-09-14T09:07:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.041e-05 2024-09-14T09:07:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.548700+0000", 2024-09-14T09:07:32.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14250141299999999 2024-09-14T09:07:32.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.548783+0000", 2024-09-14T09:07:32.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3578999999999994e-05 2024-09-14T09:07:32.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549105+0000", 2024-09-14T09:07:32.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032158600000000001 2024-09-14T09:07:32.874 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.874 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.874 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107608 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:07:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.460675771, 2024-09-14T09:07:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14603042799999999, 2024-09-14T09:07:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107608 2024-09-14T09:07:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.875 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:07:32.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:07:32.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:07:32.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1200000000000002e-07 2024-09-14T09:07:32.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406241+0000", 2024-09-14T09:07:32.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7539999999999997e-06 2024-09-14T09:07:32.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406275+0000", 2024-09-14T09:07:32.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3512e-05 2024-09-14T09:07:32.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406322+0000", 2024-09-14T09:07:32.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6764999999999997e-05 2024-09-14T09:07:32.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549149+0000", 2024-09-14T09:07:32.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14282725499999999 2024-09-14T09:07:32.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549198+0000", 2024-09-14T09:07:32.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9599999999999999e-05 2024-09-14T09:07:32.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552266+0000", 2024-09-14T09:07:32.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030677299999999998 2024-09-14T09:07:32.882 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.882 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107610 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 22528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:07:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4605538960000004, 2024-09-14T09:07:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146437919, 2024-09-14T09:07:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107610 2024-09-14T09:07:32.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:07:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:07:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:07:32.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.02e-07 2024-09-14T09:07:32.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406363+0000", 2024-09-14T09:07:32.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8099999999999997e-06 2024-09-14T09:07:32.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406390+0000", 2024-09-14T09:07:32.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7004000000000001e-05 2024-09-14T09:07:32.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406404+0000", 2024-09-14T09:07:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3682e-05 2024-09-14T09:07:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552335+0000", 2024-09-14T09:07:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14593077800000001 2024-09-14T09:07:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552431+0000", 2024-09-14T09:07:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6086000000000001e-05 2024-09-14T09:07:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552795+0000", 2024-09-14T09:07:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000364657 2024-09-14T09:07:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107612 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 26624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:07:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4604886330000006, 2024-09-14T09:07:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146762061, 2024-09-14T09:07:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.887 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107612 2024-09-14T09:07:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:07:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:07:32.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406424+0000", 2024-09-14T09:07:32.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2600000000000002e-07 2024-09-14T09:07:32.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406451+0000", 2024-09-14T09:07:32.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7250000000000002e-05 2024-09-14T09:07:32.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406487+0000", 2024-09-14T09:07:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6086e-05 2024-09-14T09:07:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406498+0000", 2024-09-14T09:07:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0858e-05 2024-09-14T09:07:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552821+0000", 2024-09-14T09:07:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14632319099999999 2024-09-14T09:07:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552875+0000", 2024-09-14T09:07:32.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4262999999999999e-05 2024-09-14T09:07:32.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553185+0000", 2024-09-14T09:07:32.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000309687 2024-09-14T09:07:32.891 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.891 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.891 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.892 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107614 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:32.892 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:07:32.892 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4604010990000003, 2024-09-14T09:07:32.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14737121, 2024-09-14T09:07:32.892 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.892 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.892 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.892 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.892 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.892 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.892 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107614 2024-09-14T09:07:32.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.892 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:07:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:07:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406511+0000", 2024-09-14T09:07:32.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1799999999999995e-07 2024-09-14T09:07:32.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406514+0000", 2024-09-14T09:07:32.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8660000000000002e-06 2024-09-14T09:07:32.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406523+0000", 2024-09-14T09:07:32.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1940000000000004e-06 2024-09-14T09:07:32.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406531+0000", 2024-09-14T09:07:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3159999999999997e-06 2024-09-14T09:07:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553233+0000", 2024-09-14T09:07:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14670117899999999 2024-09-14T09:07:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553334+0000", 2024-09-14T09:07:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100982 2024-09-14T09:07:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553882+0000", 2024-09-14T09:07:32.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00054805499999999998 2024-09-14T09:07:32.896 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.896 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.896 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.896 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107616 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 34816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:32.896 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:07:32.896 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4603694639999993, 2024-09-14T09:07:32.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14780390500000001, 2024-09-14T09:07:32.896 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.896 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.897 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.897 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.897 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.897 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.897 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107616 2024-09-14T09:07:32.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.897 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:07:32.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:07:32.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406543+0000", 2024-09-14T09:07:32.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0200000000000001e-07 2024-09-14T09:07:32.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406545+0000", 2024-09-14T09:07:32.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3360000000000002e-06 2024-09-14T09:07:32.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406552+0000", 2024-09-14T09:07:32.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9779999999999999e-06 2024-09-14T09:07:32.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406559+0000", 2024-09-14T09:07:32.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6919999999999996e-06 2024-09-14T09:07:32.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553917+0000", 2024-09-14T09:07:32.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14735791600000001 2024-09-14T09:07:32.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554033+0000", 2024-09-14T09:07:32.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116047 2024-09-14T09:07:32.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554346+0000", 2024-09-14T09:07:32.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031323400000000002 2024-09-14T09:07:32.900 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.900 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.900 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.901 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107618 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 38912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:32.901 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:07:32.901 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4603406640000003, 2024-09-14T09:07:32.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14816009599999999, 2024-09-14T09:07:32.901 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.901 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.901 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.901 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.901 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.901 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.901 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107618 2024-09-14T09:07:32.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.901 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:07:32.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:07:32.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:07:32.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9999999999999998e-07 2024-09-14T09:07:32.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406574+0000", 2024-09-14T09:07:32.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2560000000000001e-06 2024-09-14T09:07:32.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406581+0000", 2024-09-14T09:07:32.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1539999999999996e-06 2024-09-14T09:07:32.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406586+0000", 2024-09-14T09:07:32.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.904e-06 2024-09-14T09:07:32.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554385+0000", 2024-09-14T09:07:32.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.147799443 2024-09-14T09:07:32.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554441+0000", 2024-09-14T09:07:32.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5921000000000002e-05 2024-09-14T09:07:32.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554731+0000", 2024-09-14T09:07:32.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028991800000000001 2024-09-14T09:07:32.905 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.905 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.905 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.905 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107620 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:32.905 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:07:32.905 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4603188819999993, 2024-09-14T09:07:32.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.148512643, 2024-09-14T09:07:32.905 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.905 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.906 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.906 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.906 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.906 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.906 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107620 2024-09-14T09:07:32.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.906 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:07:32.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:07:32.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:07:32.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9999999999999998e-07 2024-09-14T09:07:32.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406595+0000", 2024-09-14T09:07:32.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9360000000000002e-06 2024-09-14T09:07:32.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406605+0000", 2024-09-14T09:07:32.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0268000000000001e-05 2024-09-14T09:07:32.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406614+0000", 2024-09-14T09:07:32.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7180000000000002e-06 2024-09-14T09:07:32.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554750+0000", 2024-09-14T09:07:32.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14813599699999999 2024-09-14T09:07:32.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554794+0000", 2024-09-14T09:07:32.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.401e-05 2024-09-14T09:07:32.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555105+0000", 2024-09-14T09:07:32.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031131399999999998 2024-09-14T09:07:32.909 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.909 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.909 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.910 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107622 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:32.910 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:07:32.910 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4602876259999995, 2024-09-14T09:07:32.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14902820899999999, 2024-09-14T09:07:32.910 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.910 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.910 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.910 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.910 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.910 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.910 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107622 2024-09-14T09:07:32.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.910 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:07:32.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:07:32.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:07:32.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9200000000000001e-07 2024-09-14T09:07:32.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406636+0000", 2024-09-14T09:07:32.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.189e-05 2024-09-14T09:07:32.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406649+0000", 2024-09-14T09:07:32.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2442e-05 2024-09-14T09:07:32.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406658+0000", 2024-09-14T09:07:32.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3239999999999992e-06 2024-09-14T09:07:32.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555287+0000", 2024-09-14T09:07:32.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14862868400000001 2024-09-14T09:07:32.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555348+0000", 2024-09-14T09:07:32.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1686999999999994e-05 2024-09-14T09:07:32.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555652+0000", 2024-09-14T09:07:32.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00030369000000000001 2024-09-14T09:07:32.914 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.914 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.914 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.914 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107624 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:32.914 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:07:32.914 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4602439329999992, 2024-09-14T09:07:32.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14934019900000001, 2024-09-14T09:07:32.915 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.915 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.915 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.915 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.915 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.915 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.915 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107624 2024-09-14T09:07:32.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.915 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:07:32.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:07:32.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:07:32.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1399999999999997e-07 2024-09-14T09:07:32.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406670+0000", 2024-09-14T09:07:32.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4099999999999998e-06 2024-09-14T09:07:32.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406680+0000", 2024-09-14T09:07:32.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6460000000000005e-06 2024-09-14T09:07:32.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406688+0000", 2024-09-14T09:07:32.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5859999999999999e-06 2024-09-14T09:07:32.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555693+0000", 2024-09-14T09:07:32.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14900544499999999 2024-09-14T09:07:32.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555743+0000", 2024-09-14T09:07:32.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9966999999999998e-05 2024-09-14T09:07:32.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556008+0000", 2024-09-14T09:07:32.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026463099999999998 2024-09-14T09:07:32.919 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.919 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.919 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.919 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107626 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:32.919 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:07:32.919 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4602119590000004, 2024-09-14T09:07:32.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14983149500000001, 2024-09-14T09:07:32.919 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.919 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.919 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.919 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.919 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.919 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.920 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107626 2024-09-14T09:07:32.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.920 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:07:32.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:07:32.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408087+0000", 2024-09-14T09:07:32.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001387735 2024-09-14T09:07:32.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408094+0000", 2024-09-14T09:07:32.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9399999999999996e-06 2024-09-14T09:07:32.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408122+0000", 2024-09-14T09:07:32.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8076e-05 2024-09-14T09:07:32.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408147+0000", 2024-09-14T09:07:32.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4726e-05 2024-09-14T09:07:32.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556048+0000", 2024-09-14T09:07:32.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14790144299999999 2024-09-14T09:07:32.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556175+0000", 2024-09-14T09:07:32.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012663 2024-09-14T09:07:32.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556531+0000", 2024-09-14T09:07:32.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035594499999999998 2024-09-14T09:07:32.923 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.923 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.923 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.923 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107628 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:32.923 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:07:32.924 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4587423479999995, 2024-09-14T09:07:32.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14882648900000001, 2024-09-14T09:07:32.924 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.924 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.924 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.924 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.924 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.924 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.924 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107628 2024-09-14T09:07:32.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.924 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:07:32.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:07:32.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408878+0000", 2024-09-14T09:07:32.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070858800000000001 2024-09-14T09:07:32.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408885+0000", 2024-09-14T09:07:32.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4839999999999998e-06 2024-09-14T09:07:32.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408900+0000", 2024-09-14T09:07:32.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5174000000000001e-05 2024-09-14T09:07:32.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408918+0000", 2024-09-14T09:07:32.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.808e-05 2024-09-14T09:07:32.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556572+0000", 2024-09-14T09:07:32.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14765399500000001 2024-09-14T09:07:32.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556651+0000", 2024-09-14T09:07:32.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8919999999999997e-05 2024-09-14T09:07:32.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556996+0000", 2024-09-14T09:07:32.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034424800000000001 2024-09-14T09:07:32.928 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.928 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.928 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.928 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107630 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 63488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:32.928 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:07:32.928 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4579735140000007, 2024-09-14T09:07:32.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14855159200000001, 2024-09-14T09:07:32.928 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.928 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.928 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.929 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.929 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.929 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.929 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107630 2024-09-14T09:07:32.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.929 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:07:32.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:07:32.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409510+0000", 2024-09-14T09:07:32.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00057189700000000003 2024-09-14T09:07:32.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409518+0000", 2024-09-14T09:07:32.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7619999999999995e-06 2024-09-14T09:07:32.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409534+0000", 2024-09-14T09:07:32.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.594e-05 2024-09-14T09:07:32.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409546+0000", 2024-09-14T09:07:32.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2604e-05 2024-09-14T09:07:32.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557048+0000", 2024-09-14T09:07:32.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14750137899999999 2024-09-14T09:07:32.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557117+0000", 2024-09-14T09:07:32.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8998999999999994e-05 2024-09-14T09:07:32.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557490+0000", 2024-09-14T09:07:32.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037301100000000002 2024-09-14T09:07:32.932 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.932 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.932 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.933 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107632 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:32.933 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:07:32.933 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4573425689999997, 2024-09-14T09:07:32.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14833944199999999, 2024-09-14T09:07:32.933 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.933 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.933 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.933 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.933 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.933 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.933 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107632 2024-09-14T09:07:32.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.933 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:07:32.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:07:32.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412609+0000", 2024-09-14T09:07:32.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003040105 2024-09-14T09:07:32.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412618+0000", 2024-09-14T09:07:32.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2960000000000004e-06 2024-09-14T09:07:32.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412640+0000", 2024-09-14T09:07:32.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1773999999999999e-05 2024-09-14T09:07:32.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412655+0000", 2024-09-14T09:07:32.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4538e-05 2024-09-14T09:07:32.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557538+0000", 2024-09-14T09:07:32.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14488311800000001 2024-09-14T09:07:32.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557594+0000", 2024-09-14T09:07:32.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6122999999999998e-05 2024-09-14T09:07:32.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557908+0000", 2024-09-14T09:07:32.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031448799999999997 2024-09-14T09:07:32.937 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.937 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.937 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.937 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107634 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:32.937 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:07:32.937 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4542249219999999, 2024-09-14T09:07:32.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.145666186, 2024-09-14T09:07:32.937 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.938 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.938 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.938 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.938 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.938 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.938 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107634 2024-09-14T09:07:32.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.938 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:07:32.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:07:32.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412688+0000", 2024-09-14T09:07:32.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9000000000000005e-07 2024-09-14T09:07:32.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412707+0000", 2024-09-14T09:07:32.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9232000000000002e-05 2024-09-14T09:07:32.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412722+0000", 2024-09-14T09:07:32.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5722e-05 2024-09-14T09:07:32.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412733+0000", 2024-09-14T09:07:32.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0909e-05 2024-09-14T09:07:32.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557946+0000", 2024-09-14T09:07:32.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14521263100000001 2024-09-14T09:07:32.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558020+0000", 2024-09-14T09:07:32.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4294000000000004e-05 2024-09-14T09:07:32.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558353+0000", 2024-09-14T09:07:32.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033240800000000002 2024-09-14T09:07:32.941 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.942 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.942 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.942 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107636 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:32.942 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:07:32.942 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4541606189999996, 2024-09-14T09:07:32.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14599377399999999, 2024-09-14T09:07:32.942 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.942 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.942 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.942 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.942 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.942 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.943 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107636 2024-09-14T09:07:32.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.943 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:07:32.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:07:32.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412752+0000", 2024-09-14T09:07:32.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.156e-06 2024-09-14T09:07:32.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412770+0000", 2024-09-14T09:07:32.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7521999999999999e-05 2024-09-14T09:07:32.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412790+0000", 2024-09-14T09:07:32.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0942e-05 2024-09-14T09:07:32.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412809+0000", 2024-09-14T09:07:32.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8887999999999999e-05 2024-09-14T09:07:32.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558387+0000", 2024-09-14T09:07:32.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14557798799999999 2024-09-14T09:07:32.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558448+0000", 2024-09-14T09:07:32.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0862000000000001e-05 2024-09-14T09:07:32.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558745+0000", 2024-09-14T09:07:32.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029641599999999998 2024-09-14T09:07:32.946 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.946 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.946 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.947 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107638 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:32.947 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:07:32.947 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4540733629999991, 2024-09-14T09:07:32.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146344799, 2024-09-14T09:07:32.947 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.947 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.947 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.947 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.947 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.947 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.947 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107638 2024-09-14T09:07:32.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.947 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:07:32.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:07:32.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412839+0000", 2024-09-14T09:07:32.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0720000000000001e-06 2024-09-14T09:07:32.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412844+0000", 2024-09-14T09:07:32.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7759999999999997e-06 2024-09-14T09:07:32.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412857+0000", 2024-09-14T09:07:32.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3138e-05 2024-09-14T09:07:32.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412867+0000", 2024-09-14T09:07:32.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7229999999999999e-06 2024-09-14T09:07:32.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558785+0000", 2024-09-14T09:07:32.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14591828700000001 2024-09-14T09:07:32.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558833+0000", 2024-09-14T09:07:32.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7790000000000002e-05 2024-09-14T09:07:32.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559183+0000", 2024-09-14T09:07:32.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035001299999999998 2024-09-14T09:07:32.951 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.951 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.951 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.951 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107640 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:32.951 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:07:32.951 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4540282120000008, 2024-09-14T09:07:32.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146686804, 2024-09-14T09:07:32.952 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.952 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.952 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.952 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.952 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.952 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.952 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107640 2024-09-14T09:07:32.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.952 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:07:32.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:07:32.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420446+0000", 2024-09-14T09:07:32.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075632529999999998 2024-09-14T09:07:32.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420456+0000", 2024-09-14T09:07:32.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0850000000000003e-06 2024-09-14T09:07:32.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420475+0000", 2024-09-14T09:07:32.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9406e-05 2024-09-14T09:07:32.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420488+0000", 2024-09-14T09:07:32.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3278000000000001e-05 2024-09-14T09:07:32.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559205+0000", 2024-09-14T09:07:32.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13871644999999999 2024-09-14T09:07:32.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559252+0000", 2024-09-14T09:07:32.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7666000000000001e-05 2024-09-14T09:07:32.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559570+0000", 2024-09-14T09:07:32.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031766599999999998 2024-09-14T09:07:32.955 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.956 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.956 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.956 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107660 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 124928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:32.956 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:07:32.956 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4459579700000003, 2024-09-14T09:07:32.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.142954628, 2024-09-14T09:07:32.956 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:32.956 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:32.956 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:32.956 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:32.956 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:32.957 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:32.957 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107660 2024-09-14T09:07:32.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.957 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:32.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:32.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:07:32.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:32.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:07:32.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:32.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:32.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428190+0000", 2024-09-14T09:07:32.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072360339999999997 2024-09-14T09:07:32.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:32.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428398+0000", 2024-09-14T09:07:32.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020898499999999999 2024-09-14T09:07:32.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:32.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428458+0000", 2024-09-14T09:07:32.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9985999999999998e-05 2024-09-14T09:07:32.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:32.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428480+0000", 2024-09-14T09:07:32.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1284000000000002e-05 2024-09-14T09:07:32.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:32.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563583+0000", 2024-09-14T09:07:32.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13510361800000001 2024-09-14T09:07:32.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:32.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563644+0000", 2024-09-14T09:07:32.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1092000000000004e-05 2024-09-14T09:07:32.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:32.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:32.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:32.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563908+0000", 2024-09-14T09:07:32.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026362899999999998 2024-09-14T09:07:32.960 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.960 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.960 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.960 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:32.960 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:32.960 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:32.960 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:07:32.961 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:07:32.961 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T09:07:32.961 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T09:07:32.961 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T09:07:32.963 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:07:32.964 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:32.966+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:32.980 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:32.980 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:33.014 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:33.015 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:33.037 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:33.038 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:33.057 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:33.058 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:33.064 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone39 from 0xe2c4a8f (overlap 0x78bc4d) to 0x304b52 2024-09-14T09:07:33.064 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:33.066+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:33.080 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:07:33.080 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:07:33.106 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:33.106 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:33.129 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:33.129 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:33.150 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:33.150 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:33.152 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:33.152 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:33.164 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:33.166+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:33.166 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:07:33.265 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:33.266+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:33.285 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-14T09:07:33.329 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:07:33.330 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:33.330 INFO:teuthology.orchestra.run.smithi176.stdout: "epoch": 123, 2024-09-14T09:07:33.330 INFO:teuthology.orchestra.run.smithi176.stdout: "fsid": "c635667d-994e-454f-bcea-51b45637b2a2", 2024-09-14T09:07:33.330 INFO:teuthology.orchestra.run.smithi176.stdout: "created": "2024-09-14T09:04:57.077293+0000", 2024-09-14T09:07:33.330 INFO:teuthology.orchestra.run.smithi176.stdout: "modified": "2024-09-14T09:07:32.860265+0000", 2024-09-14T09:07:33.330 INFO:teuthology.orchestra.run.smithi176.stdout: "last_up_change": "2024-09-14T09:05:11.210835+0000", 2024-09-14T09:07:33.330 INFO:teuthology.orchestra.run.smithi176.stdout: "last_in_change": "2024-09-14T09:07:21.060249+0000", 2024-09-14T09:07:33.330 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T09:07:33.330 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_num": 5804032, 2024-09-14T09:07:33.330 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_set": [ 2024-09-14T09:07:33.331 INFO:teuthology.orchestra.run.smithi176.stdout: "nodeep-scrub", 2024-09-14T09:07:33.331 INFO:teuthology.orchestra.run.smithi176.stdout: "pglog_hardlimit", 2024-09-14T09:07:33.331 INFO:teuthology.orchestra.run.smithi176.stdout: "purged_snapdirs", 2024-09-14T09:07:33.331 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_deletes", 2024-09-14T09:07:33.331 INFO:teuthology.orchestra.run.smithi176.stdout: "sortbitwise" 2024-09-14T09:07:33.331 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:33.331 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_version": 5, 2024-09-14T09:07:33.331 INFO:teuthology.orchestra.run.smithi176.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T09:07:33.331 INFO:teuthology.orchestra.run.smithi176.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T09:07:33.331 INFO:teuthology.orchestra.run.smithi176.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T09:07:33.331 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_snapshot": "", 2024-09-14T09:07:33.332 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_max": 3, 2024-09-14T09:07:33.332 INFO:teuthology.orchestra.run.smithi176.stdout: "max_osd": 8, 2024-09-14T09:07:33.332 INFO:teuthology.orchestra.run.smithi176.stdout: "require_min_compat_client": "luminous", 2024-09-14T09:07:33.332 INFO:teuthology.orchestra.run.smithi176.stdout: "min_compat_client": "luminous", 2024-09-14T09:07:33.332 INFO:teuthology.orchestra.run.smithi176.stdout: "require_osd_release": "squid", 2024-09-14T09:07:33.332 INFO:teuthology.orchestra.run.smithi176.stdout: "allow_crimson": false, 2024-09-14T09:07:33.332 INFO:teuthology.orchestra.run.smithi176.stdout: "pools": [ 2024-09-14T09:07:33.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.332 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 1, 2024-09-14T09:07:33.332 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_name": ".mgr", 2024-09-14T09:07:33.332 INFO:teuthology.orchestra.run.smithi176.stdout: "create_time": "2024-09-14T09:05:11.997673+0000", 2024-09-14T09:07:33.332 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": 1, 2024-09-14T09:07:33.333 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_names": "hashpspool", 2024-09-14T09:07:33.333 INFO:teuthology.orchestra.run.smithi176.stdout: "type": 1, 2024-09-14T09:07:33.333 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 2, 2024-09-14T09:07:33.333 INFO:teuthology.orchestra.run.smithi176.stdout: "min_size": 1, 2024-09-14T09:07:33.333 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_rule": 0, 2024-09-14T09:07:33.333 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_count": 0, 2024-09-14T09:07:33.333 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_target": 0, 2024-09-14T09:07:33.333 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T09:07:33.333 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T09:07:33.333 INFO:teuthology.orchestra.run.smithi176.stdout: "is_stretch_pool": false, 2024-09-14T09:07:33.334 INFO:teuthology.orchestra.run.smithi176.stdout: "object_hash": 2, 2024-09-14T09:07:33.334 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_autoscale_mode": "off", 2024-09-14T09:07:33.334 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num": 1, 2024-09-14T09:07:33.334 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num": 1, 2024-09-14T09:07:33.334 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num_target": 1, 2024-09-14T09:07:33.334 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_target": 1, 2024-09-14T09:07:33.334 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_pending": 1, 2024-09-14T09:07:33.334 INFO:teuthology.orchestra.run.smithi176.stdout: "last_pg_merge_meta": { 2024-09-14T09:07:33.334 INFO:teuthology.orchestra.run.smithi176.stdout: "source_pgid": "0.0", 2024-09-14T09:07:33.334 INFO:teuthology.orchestra.run.smithi176.stdout: "ready_epoch": 0, 2024-09-14T09:07:33.334 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_started": 0, 2024-09-14T09:07:33.334 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_clean": 0, 2024-09-14T09:07:33.334 INFO:teuthology.orchestra.run.smithi176.stdout: "source_version": "0'0", 2024-09-14T09:07:33.335 INFO:teuthology.orchestra.run.smithi176.stdout: "target_version": "0'0" 2024-09-14T09:07:33.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.335 INFO:teuthology.orchestra.run.smithi176.stdout: "last_change": "18", 2024-09-14T09:07:33.335 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend": "0", 2024-09-14T09:07:33.335 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T09:07:33.335 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T09:07:33.335 INFO:teuthology.orchestra.run.smithi176.stdout: "auid": 0, 2024-09-14T09:07:33.335 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_mode": "selfmanaged", 2024-09-14T09:07:33.335 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_seq": 0, 2024-09-14T09:07:33.335 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_epoch": 0, 2024-09-14T09:07:33.335 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_snaps": [], 2024-09-14T09:07:33.335 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps": "[]", 2024-09-14T09:07:33.335 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_bytes": 0, 2024-09-14T09:07:33.336 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_objects": 0, 2024-09-14T09:07:33.336 INFO:teuthology.orchestra.run.smithi176.stdout: "tiers": [], 2024-09-14T09:07:33.336 INFO:teuthology.orchestra.run.smithi176.stdout: "tier_of": -1, 2024-09-14T09:07:33.336 INFO:teuthology.orchestra.run.smithi176.stdout: "read_tier": -1, 2024-09-14T09:07:33.336 INFO:teuthology.orchestra.run.smithi176.stdout: "write_tier": -1, 2024-09-14T09:07:33.336 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_mode": "none", 2024-09-14T09:07:33.336 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_bytes": 0, 2024-09-14T09:07:33.336 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_objects": 0, 2024-09-14T09:07:33.336 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T09:07:33.336 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T09:07:33.336 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T09:07:33.336 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_flush_age": 0, 2024-09-14T09:07:33.337 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_evict_age": 0, 2024-09-14T09:07:33.337 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profile": "", 2024-09-14T09:07:33.337 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_params": { 2024-09-14T09:07:33.337 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "none" 2024-09-14T09:07:33.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.337 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_period": 0, 2024-09-14T09:07:33.337 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_count": 0, 2024-09-14T09:07:33.337 INFO:teuthology.orchestra.run.smithi176.stdout: "use_gmt_hitset": true, 2024-09-14T09:07:33.337 INFO:teuthology.orchestra.run.smithi176.stdout: "min_read_recency_for_promote": 0, 2024-09-14T09:07:33.337 INFO:teuthology.orchestra.run.smithi176.stdout: "min_write_recency_for_promote": 0, 2024-09-14T09:07:33.337 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T09:07:33.337 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_search_last_n": 0, 2024-09-14T09:07:33.337 INFO:teuthology.orchestra.run.smithi176.stdout: "grade_table": [], 2024-09-14T09:07:33.337 INFO:teuthology.orchestra.run.smithi176.stdout: "stripe_width": 0, 2024-09-14T09:07:33.338 INFO:teuthology.orchestra.run.smithi176.stdout: "expected_num_objects": 0, 2024-09-14T09:07:33.338 INFO:teuthology.orchestra.run.smithi176.stdout: "fast_read": false, 2024-09-14T09:07:33.338 INFO:teuthology.orchestra.run.smithi176.stdout: "options": { 2024-09-14T09:07:33.338 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_max": 32, 2024-09-14T09:07:33.338 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_min": 1 2024-09-14T09:07:33.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.338 INFO:teuthology.orchestra.run.smithi176.stdout: "application_metadata": { 2024-09-14T09:07:33.338 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": {} 2024-09-14T09:07:33.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.338 INFO:teuthology.orchestra.run.smithi176.stdout: "read_balance": { 2024-09-14T09:07:33.339 INFO:teuthology.orchestra.run.smithi176.stdout: "score_type": "Fair distribution", 2024-09-14T09:07:33.339 INFO:teuthology.orchestra.run.smithi176.stdout: "score_acting": 8, 2024-09-14T09:07:33.339 INFO:teuthology.orchestra.run.smithi176.stdout: "score_stable": 8, 2024-09-14T09:07:33.339 INFO:teuthology.orchestra.run.smithi176.stdout: "optimal_score": 0.25, 2024-09-14T09:07:33.339 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_acting": 2, 2024-09-14T09:07:33.339 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_stable": 2, 2024-09-14T09:07:33.339 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity_weighted": 1, 2024-09-14T09:07:33.339 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity": 1, 2024-09-14T09:07:33.339 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity_weighted": 1 2024-09-14T09:07:33.339 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.340 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 2, 2024-09-14T09:07:33.340 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_name": "rbd", 2024-09-14T09:07:33.340 INFO:teuthology.orchestra.run.smithi176.stdout: "create_time": "2024-09-14T09:05:15.352551+0000", 2024-09-14T09:07:33.340 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": 8193, 2024-09-14T09:07:33.340 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T09:07:33.340 INFO:teuthology.orchestra.run.smithi176.stdout: "type": 1, 2024-09-14T09:07:33.340 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 2, 2024-09-14T09:07:33.340 INFO:teuthology.orchestra.run.smithi176.stdout: "min_size": 1, 2024-09-14T09:07:33.340 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_rule": 0, 2024-09-14T09:07:33.340 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_count": 0, 2024-09-14T09:07:33.341 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_target": 0, 2024-09-14T09:07:33.341 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T09:07:33.341 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T09:07:33.341 INFO:teuthology.orchestra.run.smithi176.stdout: "is_stretch_pool": false, 2024-09-14T09:07:33.341 INFO:teuthology.orchestra.run.smithi176.stdout: "object_hash": 2, 2024-09-14T09:07:33.341 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_autoscale_mode": "off", 2024-09-14T09:07:33.341 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num": 8, 2024-09-14T09:07:33.341 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num": 8, 2024-09-14T09:07:33.341 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num_target": 8, 2024-09-14T09:07:33.341 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_target": 8, 2024-09-14T09:07:33.341 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_pending": 8, 2024-09-14T09:07:33.341 INFO:teuthology.orchestra.run.smithi176.stdout: "last_pg_merge_meta": { 2024-09-14T09:07:33.342 INFO:teuthology.orchestra.run.smithi176.stdout: "source_pgid": "0.0", 2024-09-14T09:07:33.342 INFO:teuthology.orchestra.run.smithi176.stdout: "ready_epoch": 0, 2024-09-14T09:07:33.342 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_started": 0, 2024-09-14T09:07:33.342 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_clean": 0, 2024-09-14T09:07:33.342 INFO:teuthology.orchestra.run.smithi176.stdout: "source_version": "0'0", 2024-09-14T09:07:33.342 INFO:teuthology.orchestra.run.smithi176.stdout: "target_version": "0'0" 2024-09-14T09:07:33.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.342 INFO:teuthology.orchestra.run.smithi176.stdout: "last_change": "22", 2024-09-14T09:07:33.342 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend": "0", 2024-09-14T09:07:33.342 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T09:07:33.342 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T09:07:33.342 INFO:teuthology.orchestra.run.smithi176.stdout: "auid": 0, 2024-09-14T09:07:33.342 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_mode": "selfmanaged", 2024-09-14T09:07:33.343 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_seq": 2, 2024-09-14T09:07:33.343 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_epoch": 22, 2024-09-14T09:07:33.343 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_snaps": [], 2024-09-14T09:07:33.343 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps": "[]", 2024-09-14T09:07:33.343 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_bytes": 0, 2024-09-14T09:07:33.343 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_objects": 0, 2024-09-14T09:07:33.343 INFO:teuthology.orchestra.run.smithi176.stdout: "tiers": [], 2024-09-14T09:07:33.343 INFO:teuthology.orchestra.run.smithi176.stdout: "tier_of": -1, 2024-09-14T09:07:33.343 INFO:teuthology.orchestra.run.smithi176.stdout: "read_tier": -1, 2024-09-14T09:07:33.343 INFO:teuthology.orchestra.run.smithi176.stdout: "write_tier": -1, 2024-09-14T09:07:33.343 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_mode": "none", 2024-09-14T09:07:33.343 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_bytes": 0, 2024-09-14T09:07:33.343 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_objects": 0, 2024-09-14T09:07:33.343 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T09:07:33.344 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T09:07:33.344 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T09:07:33.344 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_flush_age": 0, 2024-09-14T09:07:33.344 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_evict_age": 0, 2024-09-14T09:07:33.344 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profile": "", 2024-09-14T09:07:33.344 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_params": { 2024-09-14T09:07:33.344 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "none" 2024-09-14T09:07:33.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.344 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_period": 0, 2024-09-14T09:07:33.344 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_count": 0, 2024-09-14T09:07:33.344 INFO:teuthology.orchestra.run.smithi176.stdout: "use_gmt_hitset": true, 2024-09-14T09:07:33.344 INFO:teuthology.orchestra.run.smithi176.stdout: "min_read_recency_for_promote": 0, 2024-09-14T09:07:33.344 INFO:teuthology.orchestra.run.smithi176.stdout: "min_write_recency_for_promote": 0, 2024-09-14T09:07:33.345 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T09:07:33.345 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_search_last_n": 0, 2024-09-14T09:07:33.345 INFO:teuthology.orchestra.run.smithi176.stdout: "grade_table": [], 2024-09-14T09:07:33.345 INFO:teuthology.orchestra.run.smithi176.stdout: "stripe_width": 0, 2024-09-14T09:07:33.345 INFO:teuthology.orchestra.run.smithi176.stdout: "expected_num_objects": 0, 2024-09-14T09:07:33.345 INFO:teuthology.orchestra.run.smithi176.stdout: "fast_read": false, 2024-09-14T09:07:33.345 INFO:teuthology.orchestra.run.smithi176.stdout: "options": {}, 2024-09-14T09:07:33.345 INFO:teuthology.orchestra.run.smithi176.stdout: "application_metadata": { 2024-09-14T09:07:33.345 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd": {} 2024-09-14T09:07:33.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.345 INFO:teuthology.orchestra.run.smithi176.stdout: "read_balance": { 2024-09-14T09:07:33.345 INFO:teuthology.orchestra.run.smithi176.stdout: "score_type": "Fair distribution", 2024-09-14T09:07:33.345 INFO:teuthology.orchestra.run.smithi176.stdout: "score_acting": 1.9900000095367432, 2024-09-14T09:07:33.346 INFO:teuthology.orchestra.run.smithi176.stdout: "score_stable": 1.9900000095367432, 2024-09-14T09:07:33.346 INFO:teuthology.orchestra.run.smithi176.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T09:07:33.346 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_acting": 1.75, 2024-09-14T09:07:33.346 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_stable": 1.75, 2024-09-14T09:07:33.346 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity_weighted": 1, 2024-09-14T09:07:33.346 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity": 1, 2024-09-14T09:07:33.346 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity_weighted": 1 2024-09-14T09:07:33.346 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.346 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 3, 2024-09-14T09:07:33.346 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_name": "pool_client.0", 2024-09-14T09:07:33.347 INFO:teuthology.orchestra.run.smithi176.stdout: "create_time": "2024-09-14T09:05:25.840963+0000", 2024-09-14T09:07:33.347 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": 8193, 2024-09-14T09:07:33.347 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T09:07:33.347 INFO:teuthology.orchestra.run.smithi176.stdout: "type": 1, 2024-09-14T09:07:33.347 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 2, 2024-09-14T09:07:33.347 INFO:teuthology.orchestra.run.smithi176.stdout: "min_size": 1, 2024-09-14T09:07:33.348 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_rule": 0, 2024-09-14T09:07:33.348 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_count": 0, 2024-09-14T09:07:33.348 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_target": 0, 2024-09-14T09:07:33.348 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T09:07:33.348 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T09:07:33.348 INFO:teuthology.orchestra.run.smithi176.stdout: "is_stretch_pool": false, 2024-09-14T09:07:33.348 INFO:teuthology.orchestra.run.smithi176.stdout: "object_hash": 2, 2024-09-14T09:07:33.348 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_autoscale_mode": "off", 2024-09-14T09:07:33.348 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num": 32, 2024-09-14T09:07:33.348 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num": 32, 2024-09-14T09:07:33.348 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num_target": 32, 2024-09-14T09:07:33.348 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_target": 32, 2024-09-14T09:07:33.349 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_pending": 32, 2024-09-14T09:07:33.349 INFO:teuthology.orchestra.run.smithi176.stdout: "last_pg_merge_meta": { 2024-09-14T09:07:33.349 INFO:teuthology.orchestra.run.smithi176.stdout: "source_pgid": "0.0", 2024-09-14T09:07:33.349 INFO:teuthology.orchestra.run.smithi176.stdout: "ready_epoch": 0, 2024-09-14T09:07:33.349 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_started": 0, 2024-09-14T09:07:33.349 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_clean": 0, 2024-09-14T09:07:33.349 INFO:teuthology.orchestra.run.smithi176.stdout: "source_version": "0'0", 2024-09-14T09:07:33.349 INFO:teuthology.orchestra.run.smithi176.stdout: "target_version": "0'0" 2024-09-14T09:07:33.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.349 INFO:teuthology.orchestra.run.smithi176.stdout: "last_change": "123", 2024-09-14T09:07:33.349 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend": "0", 2024-09-14T09:07:33.349 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T09:07:33.350 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T09:07:33.350 INFO:teuthology.orchestra.run.smithi176.stdout: "auid": 0, 2024-09-14T09:07:33.350 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_mode": "selfmanaged", 2024-09-14T09:07:33.350 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_seq": 42, 2024-09-14T09:07:33.350 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_epoch": 123, 2024-09-14T09:07:33.350 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_snaps": [], 2024-09-14T09:07:33.350 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps": "[]", 2024-09-14T09:07:33.350 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_bytes": 0, 2024-09-14T09:07:33.350 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_objects": 0, 2024-09-14T09:07:33.350 INFO:teuthology.orchestra.run.smithi176.stdout: "tiers": [], 2024-09-14T09:07:33.350 INFO:teuthology.orchestra.run.smithi176.stdout: "tier_of": -1, 2024-09-14T09:07:33.351 INFO:teuthology.orchestra.run.smithi176.stdout: "read_tier": -1, 2024-09-14T09:07:33.351 INFO:teuthology.orchestra.run.smithi176.stdout: "write_tier": -1, 2024-09-14T09:07:33.351 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_mode": "none", 2024-09-14T09:07:33.351 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_bytes": 0, 2024-09-14T09:07:33.351 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_objects": 0, 2024-09-14T09:07:33.351 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T09:07:33.351 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T09:07:33.351 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T09:07:33.351 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_flush_age": 0, 2024-09-14T09:07:33.351 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_evict_age": 0, 2024-09-14T09:07:33.351 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profile": "", 2024-09-14T09:07:33.351 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_params": { 2024-09-14T09:07:33.351 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "none" 2024-09-14T09:07:33.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.352 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_period": 0, 2024-09-14T09:07:33.352 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_count": 0, 2024-09-14T09:07:33.352 INFO:teuthology.orchestra.run.smithi176.stdout: "use_gmt_hitset": true, 2024-09-14T09:07:33.352 INFO:teuthology.orchestra.run.smithi176.stdout: "min_read_recency_for_promote": 0, 2024-09-14T09:07:33.352 INFO:teuthology.orchestra.run.smithi176.stdout: "min_write_recency_for_promote": 0, 2024-09-14T09:07:33.352 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T09:07:33.352 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_search_last_n": 0, 2024-09-14T09:07:33.352 INFO:teuthology.orchestra.run.smithi176.stdout: "grade_table": [], 2024-09-14T09:07:33.352 INFO:teuthology.orchestra.run.smithi176.stdout: "stripe_width": 0, 2024-09-14T09:07:33.352 INFO:teuthology.orchestra.run.smithi176.stdout: "expected_num_objects": 0, 2024-09-14T09:07:33.352 INFO:teuthology.orchestra.run.smithi176.stdout: "fast_read": false, 2024-09-14T09:07:33.352 INFO:teuthology.orchestra.run.smithi176.stdout: "options": {}, 2024-09-14T09:07:33.352 INFO:teuthology.orchestra.run.smithi176.stdout: "application_metadata": { 2024-09-14T09:07:33.353 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd": {} 2024-09-14T09:07:33.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.353 INFO:teuthology.orchestra.run.smithi176.stdout: "read_balance": { 2024-09-14T09:07:33.353 INFO:teuthology.orchestra.run.smithi176.stdout: "score_type": "Fair distribution", 2024-09-14T09:07:33.353 INFO:teuthology.orchestra.run.smithi176.stdout: "score_acting": 1.5, 2024-09-14T09:07:33.353 INFO:teuthology.orchestra.run.smithi176.stdout: "score_stable": 1.5, 2024-09-14T09:07:33.353 INFO:teuthology.orchestra.run.smithi176.stdout: "optimal_score": 1, 2024-09-14T09:07:33.353 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_acting": 1.5, 2024-09-14T09:07:33.353 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_stable": 1.5, 2024-09-14T09:07:33.353 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity_weighted": 1, 2024-09-14T09:07:33.353 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity": 1, 2024-09-14T09:07:33.353 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity_weighted": 1 2024-09-14T09:07:33.353 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.353 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.354 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:33.354 INFO:teuthology.orchestra.run.smithi176.stdout: "osds": [ 2024-09-14T09:07:33.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.354 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 0, 2024-09-14T09:07:33.354 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "4ceedb4f-6398-467f-9562-ccf27200860b", 2024-09-14T09:07:33.354 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:07:33.354 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:07:33.354 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:07:33.354 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:07:33.354 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:07:33.354 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:07:33.354 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:07:33.354 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 116, 2024-09-14T09:07:33.354 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:07:33.354 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:07:33.355 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:07:33.355 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.355 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.355 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6824", 2024-09-14T09:07:33.355 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:07:33.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.355 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.355 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6825", 2024-09-14T09:07:33.355 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:07:33.355 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.355 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.355 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:07:33.356 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.356 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.356 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6826", 2024-09-14T09:07:33.356 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:07:33.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.356 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.356 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6827", 2024-09-14T09:07:33.356 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:07:33.356 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.356 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.357 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:07:33.357 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.357 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.357 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6830", 2024-09-14T09:07:33.357 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:07:33.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.357 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.357 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6831", 2024-09-14T09:07:33.357 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:07:33.358 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.358 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.358 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:07:33.358 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.358 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.358 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6828", 2024-09-14T09:07:33.358 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:07:33.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.359 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.359 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6829", 2024-09-14T09:07:33.359 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:07:33.359 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.359 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.359 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6825/3384710819", 2024-09-14T09:07:33.359 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6827/3384710819", 2024-09-14T09:07:33.359 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6831/3384710819", 2024-09-14T09:07:33.359 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6829/3384710819", 2024-09-14T09:07:33.360 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:07:33.360 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:07:33.360 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:07:33.360 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.360 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 1, 2024-09-14T09:07:33.360 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "604e2f62-7c13-46be-a97f-029b28c44a89", 2024-09-14T09:07:33.360 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:07:33.360 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:07:33.360 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:07:33.360 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:07:33.361 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:07:33.361 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:07:33.361 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:07:33.361 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 118, 2024-09-14T09:07:33.361 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:07:33.361 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:07:33.361 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:07:33.361 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.361 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.361 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6800", 2024-09-14T09:07:33.361 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:07:33.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.362 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.362 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6801", 2024-09-14T09:07:33.362 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:07:33.362 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.362 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.362 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:07:33.362 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.363 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.363 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6802", 2024-09-14T09:07:33.363 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:07:33.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.363 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.363 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6803", 2024-09-14T09:07:33.363 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:07:33.363 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.363 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.363 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:07:33.363 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.364 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.364 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6806", 2024-09-14T09:07:33.364 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:07:33.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.364 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.364 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6807", 2024-09-14T09:07:33.364 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:07:33.364 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.364 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.364 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:07:33.364 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.365 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.365 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6804", 2024-09-14T09:07:33.365 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:07:33.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.365 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.365 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6805", 2024-09-14T09:07:33.365 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:07:33.365 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.365 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.365 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6801/612012679", 2024-09-14T09:07:33.366 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6803/612012679", 2024-09-14T09:07:33.366 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6807/612012679", 2024-09-14T09:07:33.366 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6805/612012679", 2024-09-14T09:07:33.366 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:07:33.366 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:07:33.366 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:07:33.366 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.366 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 2, 2024-09-14T09:07:33.366 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "1f65589e-8b0a-4033-94c0-4ab66531db2b", 2024-09-14T09:07:33.366 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:07:33.366 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:07:33.366 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:07:33.367 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:07:33.367 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:07:33.367 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:07:33.367 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:07:33.367 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 118, 2024-09-14T09:07:33.367 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:07:33.367 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:07:33.367 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:07:33.367 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.367 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.367 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6808", 2024-09-14T09:07:33.368 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:07:33.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.368 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.368 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6809", 2024-09-14T09:07:33.368 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:07:33.368 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.368 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.368 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:07:33.368 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.368 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.368 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6810", 2024-09-14T09:07:33.369 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:07:33.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.369 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.369 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6811", 2024-09-14T09:07:33.369 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:07:33.369 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.369 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.369 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:07:33.369 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.369 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.370 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6814", 2024-09-14T09:07:33.370 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:07:33.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.370 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.370 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6815", 2024-09-14T09:07:33.370 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:07:33.370 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.370 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.370 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:07:33.370 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.370 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.371 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6812", 2024-09-14T09:07:33.371 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:07:33.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.371 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.371 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6813", 2024-09-14T09:07:33.371 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:07:33.371 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.371 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.371 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6809/4167379279", 2024-09-14T09:07:33.371 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6811/4167379279", 2024-09-14T09:07:33.371 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6815/4167379279", 2024-09-14T09:07:33.372 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6813/4167379279", 2024-09-14T09:07:33.372 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:07:33.372 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:07:33.372 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:07:33.372 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.372 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 3, 2024-09-14T09:07:33.372 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "209d4f65-79f9-4a7d-9d23-832a4209507f", 2024-09-14T09:07:33.372 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:07:33.372 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:07:33.372 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:07:33.372 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:07:33.373 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:07:33.373 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:07:33.373 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:07:33.373 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 117, 2024-09-14T09:07:33.373 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:07:33.373 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:07:33.373 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:07:33.373 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.373 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.373 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6816", 2024-09-14T09:07:33.373 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:07:33.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.374 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.374 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6817", 2024-09-14T09:07:33.374 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:07:33.374 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.374 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.374 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:07:33.374 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.374 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.374 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6818", 2024-09-14T09:07:33.374 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:07:33.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.375 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.375 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6819", 2024-09-14T09:07:33.375 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:07:33.375 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.375 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.375 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:07:33.375 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.375 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.375 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6822", 2024-09-14T09:07:33.375 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:07:33.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.376 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.376 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6823", 2024-09-14T09:07:33.376 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:07:33.376 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.376 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.376 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:07:33.376 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.376 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.376 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6820", 2024-09-14T09:07:33.377 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:07:33.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.377 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.377 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6821", 2024-09-14T09:07:33.377 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:07:33.377 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.377 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.377 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6817/1658209015", 2024-09-14T09:07:33.377 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6819/1658209015", 2024-09-14T09:07:33.377 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6823/1658209015", 2024-09-14T09:07:33.378 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6821/1658209015", 2024-09-14T09:07:33.378 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:07:33.378 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:07:33.378 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:07:33.378 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.378 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 4, 2024-09-14T09:07:33.378 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "8f9a8c9b-282e-4523-9e62-5824f4aba401", 2024-09-14T09:07:33.378 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:07:33.379 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:07:33.379 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:07:33.379 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:07:33.379 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:07:33.379 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:07:33.379 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:07:33.379 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 116, 2024-09-14T09:07:33.379 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:07:33.379 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:07:33.380 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:07:33.380 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.380 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.380 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6808", 2024-09-14T09:07:33.380 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:07:33.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.382 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.382 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6809", 2024-09-14T09:07:33.382 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:07:33.382 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.382 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.382 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:07:33.382 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6810", 2024-09-14T09:07:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:07:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6811", 2024-09-14T09:07:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:07:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:07:33.383 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6814", 2024-09-14T09:07:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:07:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6815", 2024-09-14T09:07:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:07:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:07:33.384 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6812", 2024-09-14T09:07:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:07:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6813", 2024-09-14T09:07:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:07:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6809/1576054389", 2024-09-14T09:07:33.385 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6811/1576054389", 2024-09-14T09:07:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6815/1576054389", 2024-09-14T09:07:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6813/1576054389", 2024-09-14T09:07:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:07:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:07:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:07:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 5, 2024-09-14T09:07:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80", 2024-09-14T09:07:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:07:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:07:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:07:33.386 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:07:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:07:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:07:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:07:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 116, 2024-09-14T09:07:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:07:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:07:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:07:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6824", 2024-09-14T09:07:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:07:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6825", 2024-09-14T09:07:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:07:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:07:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.388 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6826", 2024-09-14T09:07:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:07:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6827", 2024-09-14T09:07:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:07:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:07:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6830", 2024-09-14T09:07:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:07:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6831", 2024-09-14T09:07:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:07:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:07:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6828", 2024-09-14T09:07:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:07:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6829", 2024-09-14T09:07:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:07:33.391 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6825/1711292386", 2024-09-14T09:07:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6827/1711292386", 2024-09-14T09:07:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6831/1711292386", 2024-09-14T09:07:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6829/1711292386", 2024-09-14T09:07:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:07:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:07:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:07:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 6, 2024-09-14T09:07:33.392 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "59d9259d-b9dd-4326-8ab9-9d599a2bf57b", 2024-09-14T09:07:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:07:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:07:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:07:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:07:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:07:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:07:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:07:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 112, 2024-09-14T09:07:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:07:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:07:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:07:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.393 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6816", 2024-09-14T09:07:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:07:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6817", 2024-09-14T09:07:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:07:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:07:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.394 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6818", 2024-09-14T09:07:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:07:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6819", 2024-09-14T09:07:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:07:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:07:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6822", 2024-09-14T09:07:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:07:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6823", 2024-09-14T09:07:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:07:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:07:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6820", 2024-09-14T09:07:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:07:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6821", 2024-09-14T09:07:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:07:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6817/2721245607", 2024-09-14T09:07:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6819/2721245607", 2024-09-14T09:07:33.397 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6823/2721245607", 2024-09-14T09:07:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6821/2721245607", 2024-09-14T09:07:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:07:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:07:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:07:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 7, 2024-09-14T09:07:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "e2f27d42-786b-449b-a2fb-9dd28f4e9788", 2024-09-14T09:07:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:07:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:07:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:07:33.398 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:07:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:07:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:07:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:07:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 119, 2024-09-14T09:07:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:07:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:07:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:07:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6800", 2024-09-14T09:07:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:07:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6801", 2024-09-14T09:07:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:07:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:07:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6802", 2024-09-14T09:07:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:07:33.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6803", 2024-09-14T09:07:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:07:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:07:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6806", 2024-09-14T09:07:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:07:33.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6807", 2024-09-14T09:07:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:07:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:07:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:07:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:07:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6804", 2024-09-14T09:07:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:07:33.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:07:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6805", 2024-09-14T09:07:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:07:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6801/2122823465", 2024-09-14T09:07:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6803/2122823465", 2024-09-14T09:07:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6807/2122823465", 2024-09-14T09:07:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6805/2122823465", 2024-09-14T09:07:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:07:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:07:33.403 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:07:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_xinfo": [ 2024-09-14T09:07:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 0, 2024-09-14T09:07:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:07:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:07:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:07:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:07:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:07:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.411789+0000", 2024-09-14T09:07:33.404 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:07:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 1, 2024-09-14T09:07:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:07:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:07:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:07:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:07:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:07:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:05.801873+0000", 2024-09-14T09:07:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:07:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 2, 2024-09-14T09:07:33.405 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:07:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:07:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:07:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:07:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:07:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.206725+0000", 2024-09-14T09:07:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:07:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 3, 2024-09-14T09:07:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:07:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:07:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:07:33.406 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:07:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:07:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:05.882180+0000", 2024-09-14T09:07:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:07:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 4, 2024-09-14T09:07:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:07:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:07:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:07:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:07:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:07:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.013864+0000", 2024-09-14T09:07:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:07:33.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 5, 2024-09-14T09:07:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:07:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:07:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:07:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:07:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:07:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.652975+0000", 2024-09-14T09:07:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:07:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 6, 2024-09-14T09:07:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:07:33.408 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:07:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:07:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:07:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:07:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.741359+0000", 2024-09-14T09:07:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:07:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 7, 2024-09-14T09:07:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:07:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:07:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:07:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:07:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:07:33.409 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:05.873675+0000", 2024-09-14T09:07:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:07:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_upmap": [ 2024-09-14T09:07:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "2.3", 2024-09-14T09:07:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: "osds": [ 2024-09-14T09:07:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: 3, 2024-09-14T09:07:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: 2 2024-09-14T09:07:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "3.4", 2024-09-14T09:07:33.410 INFO:teuthology.orchestra.run.smithi176.stdout: "osds": [ 2024-09-14T09:07:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: 1, 2024-09-14T09:07:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: 7 2024-09-14T09:07:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_upmap_items": [ 2024-09-14T09:07:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "3.9", 2024-09-14T09:07:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: "mappings": [ 2024-09-14T09:07:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: "from": 4, 2024-09-14T09:07:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: "to": 6 2024-09-14T09:07:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.411 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "3.1b", 2024-09-14T09:07:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "mappings": [ 2024-09-14T09:07:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "from": 4, 2024-09-14T09:07:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "to": 6 2024-09-14T09:07:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_upmap_primaries": [], 2024-09-14T09:07:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_temp": [], 2024-09-14T09:07:33.412 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_temp": [], 2024-09-14T09:07:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "blocklist": {}, 2024-09-14T09:07:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "range_blocklist": {}, 2024-09-14T09:07:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profiles": { 2024-09-14T09:07:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "default": { 2024-09-14T09:07:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "crush-failure-domain": "osd", 2024-09-14T09:07:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "k": "2", 2024-09-14T09:07:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "m": "1", 2024-09-14T09:07:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "plugin": "jerasure", 2024-09-14T09:07:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "technique": "reed_sol_van" 2024-09-14T09:07:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps_queue": [], 2024-09-14T09:07:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "new_removed_snaps": [], 2024-09-14T09:07:33.413 INFO:teuthology.orchestra.run.smithi176.stdout: "new_purged_snaps": [], 2024-09-14T09:07:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_node_flags": {}, 2024-09-14T09:07:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "device_class_flags": {}, 2024-09-14T09:07:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_mode": { 2024-09-14T09:07:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_mode_enabled": false, 2024-09-14T09:07:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_bucket_count": 0, 2024-09-14T09:07:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "degraded_stretch_mode": 0, 2024-09-14T09:07:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "recovering_stretch_mode": 0, 2024-09-14T09:07:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_mode_bucket": 0 2024-09-14T09:07:33.414 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:33.414 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:33.416 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:33.416 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:33.417 INFO:tasks.thrashosds.thrasher:j is {'epoch': 123, 'fsid': 'c635667d-994e-454f-bcea-51b45637b2a2', 'created': '2024-09-14T09:04:57.077293+0000', 'modified': '2024-09-14T09:07:32.860265+0000', 'last_up_change': '2024-09-14T09:05:11.210835+0000', 'last_in_change': '2024-09-14T09:07:21.060249+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T09:05:11.997673+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T09:05:15.352551+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T09:05:25.840963+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 42, 'snap_epoch': 123, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '4ceedb4f-6398-467f-9562-ccf27200860b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6824', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6825', 'nonce': 3384710819}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6826', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6827', 'nonce': 3384710819}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6830', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6831', 'nonce': 3384710819}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6828', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6829', 'nonce': 3384710819}]}, 'public_addr': '172.21.15.176:6825/3384710819', 'cluster_addr': '172.21.15.176:6827/3384710819', 'heartbeat_back_addr': '172.21.15.176:6831/3384710819', 'heartbeat_front_addr': '172.21.15.176:6829/3384710819', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '604e2f62-7c13-46be-a97f-029b28c44a89', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 118, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6800', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6801', 'nonce': 612012679}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6802', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6803', 'nonce': 612012679}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6806', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6807', 'nonce': 612012679}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6804', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6805', 'nonce': 612012679}]}, 'public_addr': '172.21.15.176:6801/612012679', 'cluster_addr': '172.21.15.176:6803/612012679', 'heartbeat_back_addr': '172.21.15.176:6807/612012679', 'heartbeat_front_addr': '172.21.15.176:6805/612012679', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1f65589e-8b0a-4033-94c0-4ab66531db2b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 118, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6808', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6809', 'nonce': 4167379279}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6810', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6811', 'nonce': 4167379279}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6814', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6815', 'nonce': 4167379279}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6812', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6813', 'nonce': 4167379279}]}, 'public_addr': '172.21.15.176:6809/4167379279', 'cluster_addr': '172.21.15.176:6811/4167379279', 'heartbeat_back_addr': '172.21.15.176:6815/4167379279', 'heartbeat_front_addr': '172.21.15.176:6813/4167379279', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '209d4f65-79f9-4a7d-9d23-832a4209507f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6816', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6817', 'nonce': 1658209015}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6818', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6819', 'nonce': 1658209015}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6822', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6823', 'nonce': 1658209015}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6820', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6821', 'nonce': 1658209015}]}, 'public_addr': '172.21.15.176:6817/1658209015', 'cluster_addr': '172.21.15.176:6819/1658209015', 'heartbeat_back_addr': '172.21.15.176:6823/1658209015', 'heartbeat_front_addr': '172.21.15.176:6821/1658209015', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8f9a8c9b-282e-4523-9e62-5824f4aba401', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6808', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6809', 'nonce': 1576054389}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6810', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6811', 'nonce': 1576054389}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6814', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6815', 'nonce': 1576054389}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6812', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6813', 'nonce': 1576054389}]}, 'public_addr': '172.21.15.183:6809/1576054389', 'cluster_addr': '172.21.15.183:6811/1576054389', 'heartbeat_back_addr': '172.21.15.183:6815/1576054389', 'heartbeat_front_addr': '172.21.15.183:6813/1576054389', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6824', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6825', 'nonce': 1711292386}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6826', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6827', 'nonce': 1711292386}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6830', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6831', 'nonce': 1711292386}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6828', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6829', 'nonce': 1711292386}]}, 'public_addr': '172.21.15.183:6825/1711292386', 'cluster_addr': '172.21.15.183:6827/1711292386', 'heartbeat_back_addr': '172.21.15.183:6831/1711292386', 'heartbeat_front_addr': '172.21.15.183:6829/1711292386', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '59d9259d-b9dd-4326-8ab9-9d599a2bf57b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6816', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6817', 'nonce': 2721245607}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6818', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6819', 'nonce': 2721245607}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6822', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6823', 'nonce': 2721245607}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6820', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6821', 'nonce': 2721245607}]}, 'public_addr': '172.21.15.183:6817/2721245607', 'cluster_addr': '172.21.15.183:6819/2721245607', 'heartbeat_back_addr': '172.21.15.183:6823/2721245607', 'heartbeat_front_addr': '172.21.15.183:6821/2721245607', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'e2f27d42-786b-449b-a2fb-9dd28f4e9788', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 119, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6800', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6801', 'nonce': 2122823465}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6802', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6803', 'nonce': 2122823465}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6806', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6807', 'nonce': 2122823465}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6804', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6805', 'nonce': 2122823465}]}, 'public_addr': '172.21.15.183:6801/2122823465', 'cluster_addr': '172.21.15.183:6803/2122823465', 'heartbeat_back_addr': '172.21.15.183:6807/2122823465', 'heartbeat_front_addr': '172.21.15.183:6805/2122823465', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.411789+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:05.801873+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.206725+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:05.882180+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.013864+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.652975+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.741359+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:05.873675+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.3', 'osds': [3, 2]}, {'pgid': '3.4', 'osds': [1, 7]}], 'pg_upmap_items': [{'pgid': '3.9', 'mappings': [{'from': 4, 'to': 6}]}, {'pgid': '3.1b', 'mappings': [{'from': 4, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T09:07:33.417 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:07:33.418 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:07:33.418 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:33.418+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:33.506 INFO:teuthology.orchestra.run.smithi176.stdout:5491 read 0xc16dd50 thru 0xc17668c (0x893d bytes) 2024-09-14T09:07:33.507 INFO:teuthology.orchestra.run.smithi176.stdout:5492 punch from 0xc8bd156 to 0xc8c8bf6, (0xbaa0 bytes) 2024-09-14T09:07:33.512 INFO:teuthology.orchestra.run.smithi176.stdout:5494 write 0x516779 thru 0x51c8a3 (0x612b bytes) 2024-09-14T09:07:33.515 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:33.515+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:33.524 INFO:teuthology.orchestra.run.smithi176.stdout:5497 writesame 0x5c267a3 thru 0x5c26fdd data_size 0x7(0x83b bytes) 2024-09-14T09:07:33.529 INFO:teuthology.orchestra.run.smithi176.stdout:5498 writesame 0xb399130 thru 0xb39b311 data_size 0x1(0x21e2 bytes) 2024-09-14T09:07:33.537 INFO:teuthology.orchestra.run.smithi176.stdout:5499 punch from 0x35c9b9a to 0x35cc383, (0x27e9 bytes) 2024-09-14T09:07:33.537 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:33.537 INFO:teuthology.orchestra.run.smithi176.stdout:5501 read 0x14ea638 thru 0x14f4645 (0xa00e bytes) 2024-09-14T09:07:33.537 INFO:teuthology.orchestra.run.smithi176.stdout:5502 writesame 0x3592c0c thru 0x359e89f data_size 0x95(0xbc94 bytes) 2024-09-14T09:07:33.545 INFO:teuthology.orchestra.run.smithi176.stdout:5503 punch from 0x5d78e97 to 0x5d8431d, (0xb486 bytes) 2024-09-14T09:07:33.545 INFO:teuthology.orchestra.run.smithi176.stdout:5504 write 0x4b949ec thru 0x4b9be08 (0x741d bytes) 2024-09-14T09:07:33.565 INFO:teuthology.orchestra.run.smithi176.stdout:5505 trunc from 0xe2c4a8f to 0x3d92d4d 2024-09-14T09:07:33.595 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:33.595 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:33.600 INFO:teuthology.orchestra.run.smithi176.stdout:5506 punch from 0x5afb97 to 0x5b8e68, (0x92d1 bytes) 2024-09-14T09:07:33.610 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:07:33.622 INFO:teuthology.orchestra.run.smithi176.stdout:5507 write 0xcf66e62 thru 0xcf6dd5d (0x6efc bytes) 2024-09-14T09:07:33.622 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:33.623+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:33.629 INFO:teuthology.orchestra.run.smithi176.stdout:5508 read 0xaf9ae20 thru 0xafa743f (0xc620 bytes) 2024-09-14T09:07:33.629 INFO:teuthology.orchestra.run.smithi176.stdout:5509 write 0x8165996 thru 0x81734f3 (0xdb5e bytes) 2024-09-14T09:07:33.638 INFO:teuthology.orchestra.run.smithi176.stdout:5510 clone 41 order 19 su 8192 sc 13 2024-09-14T09:07:33.718 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:33.719+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:33.725 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:33.726 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:33.739 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:07:33.758 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:07:33.759 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:07:33.775 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-14T09:07:32.872349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1307552874,"num_objects":2400,"num_object_clones":800,"num_object_copies":4800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2400,"num_whiteouts":1014,"num_read":155319,"num_read_kb":753578,"num_write":20374,"num_write_kb":331437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3687,"num_bytes_recovered":199279531,"num_keys_recovered":652,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16487,"ondisk_log_size":16487,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":452124,"kb_used_data":65200,"kb_used_omap":12,"kb_used_meta":386739,"kb_avail":754522596,"statfs":{"total":773094113280,"available":772631138304,"internally_reserved":0,"allocated":66764800,"data_stored":458016950,"data_compressed":14967055,"data_compressed_allocated":39256064,"data_compressed_original":442578895,"omap_allocated":12751,"internal_metadata":396021297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7800505,"num_objects":335,"num_object_clones":195,"num_object_copies":670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2,"num_objects_misplaced":-76,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":304,"num_read":5656,"num_read_kb":15789,"num_write":439,"num_write_kb":4632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":23460643,"num_keys_recovered":58,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.876848"},"pg_stats":[{"pgid":"3.1f","version":"121'1552","reported_seq":6458,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:31.940098+0000","last_change":"2024-09-14T09:06:41.348384+0000","last_active":"2024-09-14T09:07:31.940098+0000","last_peered":"2024-09-14T09:07:31.940098+0000","last_clean":"2024-09-14T09:07:31.940098+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:07:31.940098+0000","last_undegraded":"2024-09-14T09:07:31.940098+0000","last_fullsized":"2024-09-14T09:07:31.940098+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1552,"log_dups_size":0,"ondisk_log_size":1552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:02:12.219525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001387512,"stat_sum":{"num_bytes":21045360,"num_objects":73,"num_object_clones":25,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":29,"num_read":12424,"num_read_kb":22906,"num_write":2526,"num_write_kb":7988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"121'344","reported_seq":4292,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.925132+0000","last_change":"2024-09-14T09:05:29.039503+0000","last_active":"2024-09-14T09:07:30.925132+0000","last_peered":"2024-09-14T09:07:30.925132+0000","last_clean":"2024-09-14T09:07:30.925132+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:07:30.925132+0000","last_undegraded":"2024-09-14T09:07:30.925132+0000","last_fullsized":"2024-09-14T09:07:30.925132+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:37:58.141404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001657176,"stat_sum":{"num_bytes":45813783,"num_objects":96,"num_object_clones":35,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":40,"num_read":3840,"num_read_kb":30041,"num_write":363,"num_write_kb":12955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"121'520","reported_seq":5754,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:31.557759+0000","last_change":"2024-09-14T09:05:28.835298+0000","last_active":"2024-09-14T09:07:31.557759+0000","last_peered":"2024-09-14T09:07:31.557759+0000","last_clean":"2024-09-14T09:07:31.557759+0000","last_became_active":"2024-09-14T09:05:26.835372+0000","last_became_peered":"2024-09-14T09:05:26.835372+0000","last_unstale":"2024-09-14T09:07:31.557759+0000","last_undegraded":"2024-09-14T09:07:31.557759+0000","last_fullsized":"2024-09-14T09:07:31.557759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:43:33.096405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014149329999999999,"stat_sum":{"num_bytes":65075133,"num_objects":87,"num_object_clones":31,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":34,"num_read":6068,"num_read_kb":57296,"num_write":636,"num_write_kb":11341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"121'665","reported_seq":4327,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:31.556790+0000","last_change":"2024-09-14T09:06:44.867658+0000","last_active":"2024-09-14T09:07:31.556790+0000","last_peered":"2024-09-14T09:07:31.556790+0000","last_clean":"2024-09-14T09:07:31.556790+0000","last_became_active":"2024-09-14T09:06:44.862945+0000","last_became_peered":"2024-09-14T09:06:44.862945+0000","last_unstale":"2024-09-14T09:07:31.556790+0000","last_undegraded":"2024-09-14T09:07:31.556790+0000","last_fullsized":"2024-09-14T09:07:31.556790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:12.268283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014661419999999999,"stat_sum":{"num_bytes":45277982,"num_objects":82,"num_object_clones":30,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":35,"num_read":6401,"num_read_kb":25977,"num_write":865,"num_write_kb":9080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"122'599","reported_seq":4817,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:31.544036+0000","last_change":"2024-09-14T09:07:06.843637+0000","last_active":"2024-09-14T09:07:31.544036+0000","last_peered":"2024-09-14T09:07:31.544036+0000","last_clean":"2024-09-14T09:07:31.544036+0000","last_became_active":"2024-09-14T09:07:06.842337+0000","last_became_peered":"2024-09-14T09:07:06.842337+0000","last_unstale":"2024-09-14T09:07:31.544036+0000","last_undegraded":"2024-09-14T09:07:31.544036+0000","last_fullsized":"2024-09-14T09:07:31.544036+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:47:20.376370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060373399999999998,"stat_sum":{"num_bytes":35899947,"num_objects":75,"num_object_clones":27,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":30,"num_read":5882,"num_read_kb":23130,"num_write":790,"num_write_kb":15763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":14145319,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"121'601","reported_seq":4723,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.667837+0000","last_change":"2024-09-14T09:06:44.872988+0000","last_active":"2024-09-14T09:07:30.667837+0000","last_peered":"2024-09-14T09:07:30.667837+0000","last_clean":"2024-09-14T09:07:30.667837+0000","last_became_active":"2024-09-14T09:06:44.869103+0000","last_became_peered":"2024-09-14T09:06:44.869103+0000","last_unstale":"2024-09-14T09:07:30.667837+0000","last_undegraded":"2024-09-14T09:07:30.667837+0000","last_fullsized":"2024-09-14T09:07:30.667837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:19:15.083814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020845550000000001,"stat_sum":{"num_bytes":46021401,"num_objects":71,"num_object_clones":23,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":29,"num_read":5575,"num_read_kb":27424,"num_write":884,"num_write_kb":12964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"121'281","reported_seq":2422,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.863024+0000","last_change":"2024-09-14T09:05:28.822565+0000","last_active":"2024-09-14T09:07:30.863024+0000","last_peered":"2024-09-14T09:07:30.863024+0000","last_clean":"2024-09-14T09:07:30.863024+0000","last_became_active":"2024-09-14T09:05:26.816132+0000","last_became_peered":"2024-09-14T09:05:26.816132+0000","last_unstale":"2024-09-14T09:07:30.863024+0000","last_undegraded":"2024-09-14T09:07:30.863024+0000","last_fullsized":"2024-09-14T09:07:30.863024+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:32.134922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092420299999999996,"stat_sum":{"num_bytes":24543554,"num_objects":57,"num_object_clones":20,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":25,"num_read":2007,"num_read_kb":7789,"num_write":307,"num_write_kb":5743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"121'421","reported_seq":4345,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.924958+0000","last_change":"2024-09-14T09:06:41.342724+0000","last_active":"2024-09-14T09:07:30.924958+0000","last_peered":"2024-09-14T09:07:30.924958+0000","last_clean":"2024-09-14T09:07:30.924958+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:07:30.924958+0000","last_undegraded":"2024-09-14T09:07:30.924958+0000","last_fullsized":"2024-09-14T09:07:30.924958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:24:27.415478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081579899999999997,"stat_sum":{"num_bytes":61228569,"num_objects":78,"num_object_clones":28,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":28,"num_read":5113,"num_read_kb":14179,"num_write":401,"num_write_kb":6846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"122'758","reported_seq":5670,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:32.220551+0000","last_change":"2024-09-14T09:05:28.839928+0000","last_active":"2024-09-14T09:07:32.220551+0000","last_peered":"2024-09-14T09:07:32.220551+0000","last_clean":"2024-09-14T09:07:32.220551+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:07:32.220551+0000","last_undegraded":"2024-09-14T09:07:32.220551+0000","last_fullsized":"2024-09-14T09:07:32.220551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:55:10.432216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014343089999999999,"stat_sum":{"num_bytes":45156594,"num_objects":58,"num_object_clones":18,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":20,"num_read":7901,"num_read_kb":28388,"num_write":1030,"num_write_kb":10695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"121'473","reported_seq":3014,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:31.558590+0000","last_change":"2024-09-14T09:07:25.107391+0000","last_active":"2024-09-14T09:07:31.558590+0000","last_peered":"2024-09-14T09:07:31.558590+0000","last_clean":"2024-09-14T09:07:31.558590+0000","last_became_active":"2024-09-14T09:07:25.105674+0000","last_became_peered":"2024-09-14T09:07:25.105674+0000","last_unstale":"2024-09-14T09:07:31.558590+0000","last_undegraded":"2024-09-14T09:07:31.558590+0000","last_fullsized":"2024-09-14T09:07:31.558590+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:11:50.162523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015277190000000001,"stat_sum":{"num_bytes":21209970,"num_objects":66,"num_object_clones":19,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":23,"num_read":4013,"num_read_kb":14592,"num_write":590,"num_write_kb":6540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":8467448,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"121'953","reported_seq":2795,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.967361+0000","last_change":"2024-09-14T09:07:27.856516+0000","last_active":"2024-09-14T09:07:30.967361+0000","last_peered":"2024-09-14T09:07:30.967361+0000","last_clean":"2024-09-14T09:07:30.967361+0000","last_became_active":"2024-09-14T09:07:27.854998+0000","last_became_peered":"2024-09-14T09:07:27.854998+0000","last_unstale":"2024-09-14T09:07:30.967361+0000","last_undegraded":"2024-09-14T09:07:30.967361+0000","last_fullsized":"2024-09-14T09:07:30.967361+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:30:52.209816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098594000000000004,"stat_sum":{"num_bytes":19731664,"num_objects":63,"num_object_clones":18,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":28,"num_read":6113,"num_read_kb":11189,"num_write":1456,"num_write_kb":6698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":4170600,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"121'512","reported_seq":2781,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.461328+0000","last_change":"2024-09-14T09:07:26.110470+0000","last_active":"2024-09-14T09:07:30.461328+0000","last_peered":"2024-09-14T09:07:30.461328+0000","last_clean":"2024-09-14T09:07:30.461328+0000","last_became_active":"2024-09-14T09:07:26.106734+0000","last_became_peered":"2024-09-14T09:07:26.106734+0000","last_unstale":"2024-09-14T09:07:30.461328+0000","last_undegraded":"2024-09-14T09:07:30.461328+0000","last_fullsized":"2024-09-14T09:07:30.461328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:04:36.500342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014079679999999999,"stat_sum":{"num_bytes":25034255,"num_objects":74,"num_object_clones":24,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":34,"num_read":2979,"num_read_kb":7239,"num_write":584,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":2686146,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:31.166942+0000","last_change":"2024-09-14T09:07:21.857426+0000","last_active":"2024-09-14T09:07:31.166942+0000","last_peered":"2024-09-14T09:07:31.166942+0000","last_clean":"2024-09-14T09:07:31.166942+0000","last_became_active":"2024-09-14T09:07:21.856579+0000","last_became_peered":"2024-09-14T09:07:21.856579+0000","last_unstale":"2024-09-14T09:07:31.166942+0000","last_undegraded":"2024-09-14T09:07:31.166942+0000","last_fullsized":"2024-09-14T09:07:31.166942+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:56.694930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"122'526","reported_seq":5458,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:32.214445+0000","last_change":"2024-09-14T09:05:28.839614+0000","last_active":"2024-09-14T09:07:32.214445+0000","last_peered":"2024-09-14T09:07:32.214445+0000","last_clean":"2024-09-14T09:07:32.214445+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:07:32.214445+0000","last_undegraded":"2024-09-14T09:07:32.214445+0000","last_fullsized":"2024-09-14T09:07:32.214445+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:07:56.683627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014245869999999999,"stat_sum":{"num_bytes":44735223,"num_objects":85,"num_object_clones":26,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":38,"num_read":7220,"num_read_kb":43447,"num_write":560,"num_write_kb":7857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":220,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.863268+0000","last_change":"2024-09-14T09:05:19.314029+0000","last_active":"2024-09-14T09:07:30.863268+0000","last_peered":"2024-09-14T09:07:30.863268+0000","last_clean":"2024-09-14T09:07:30.863268+0000","last_became_active":"2024-09-14T09:05:17.282890+0000","last_became_peered":"2024-09-14T09:05:17.282890+0000","last_unstale":"2024-09-14T09:07:30.863268+0000","last_undegraded":"2024-09-14T09:07:30.863268+0000","last_fullsized":"2024-09-14T09:07:30.863268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:25.304105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001098244,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"121'511","reported_seq":4356,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.925341+0000","last_change":"2024-09-14T09:07:26.103749+0000","last_active":"2024-09-14T09:07:30.925341+0000","last_peered":"2024-09-14T09:07:30.925341+0000","last_clean":"2024-09-14T09:07:30.925341+0000","last_became_active":"2024-09-14T09:07:26.102666+0000","last_became_peered":"2024-09-14T09:07:26.102666+0000","last_unstale":"2024-09-14T09:07:30.925341+0000","last_undegraded":"2024-09-14T09:07:30.925341+0000","last_fullsized":"2024-09-14T09:07:30.925341+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:10:23.206109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019213749999999999,"stat_sum":{"num_bytes":41830259,"num_objects":76,"num_object_clones":29,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":33,"num_read":4703,"num_read_kb":15145,"num_write":610,"num_write_kb":6771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":8334712,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.967136+0000","last_change":"2024-09-14T09:07:21.863290+0000","last_active":"2024-09-14T09:07:30.967136+0000","last_peered":"2024-09-14T09:07:30.967136+0000","last_clean":"2024-09-14T09:07:30.967136+0000","last_became_active":"2024-09-14T09:07:21.862352+0000","last_became_peered":"2024-09-14T09:07:21.862352+0000","last_unstale":"2024-09-14T09:07:30.967136+0000","last_undegraded":"2024-09-14T09:07:30.967136+0000","last_fullsized":"2024-09-14T09:07:30.967136+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:48:30.332891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"121'673","reported_seq":4720,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.863176+0000","last_change":"2024-09-14T09:07:27.851340+0000","last_active":"2024-09-14T09:07:30.863176+0000","last_peered":"2024-09-14T09:07:30.863176+0000","last_clean":"2024-09-14T09:07:30.863176+0000","last_became_active":"2024-09-14T09:07:27.848139+0000","last_became_peered":"2024-09-14T09:07:27.848139+0000","last_unstale":"2024-09-14T09:07:30.863176+0000","last_undegraded":"2024-09-14T09:07:30.863176+0000","last_fullsized":"2024-09-14T09:07:30.863176+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:00:08.281626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014087959999999999,"stat_sum":{"num_bytes":24639174,"num_objects":79,"num_object_clones":27,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":37,"num_read":6477,"num_read_kb":20409,"num_write":910,"num_write_kb":8510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":8151041,"num_keys_recovered":91,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":220,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.863234+0000","last_change":"2024-09-14T09:05:19.314129+0000","last_active":"2024-09-14T09:07:30.863234+0000","last_peered":"2024-09-14T09:07:30.863234+0000","last_clean":"2024-09-14T09:07:30.863234+0000","last_became_active":"2024-09-14T09:05:17.295196+0000","last_became_peered":"2024-09-14T09:05:17.295196+0000","last_unstale":"2024-09-14T09:07:30.863234+0000","last_undegraded":"2024-09-14T09:07:30.863234+0000","last_fullsized":"2024-09-14T09:07:30.863234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:35.269115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011613800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"122'550","reported_seq":2097,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:31.564142+0000","last_change":"2024-09-14T09:05:28.835976+0000","last_active":"2024-09-14T09:07:31.564142+0000","last_peered":"2024-09-14T09:07:31.564142+0000","last_clean":"2024-09-14T09:07:31.564142+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:07:31.564142+0000","last_undegraded":"2024-09-14T09:07:31.564142+0000","last_fullsized":"2024-09-14T09:07:31.564142+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:58:11.080584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017480779999999999,"stat_sum":{"num_bytes":59249289,"num_objects":90,"num_object_clones":32,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":41,"num_read":2925,"num_read_kb":15797,"num_write":717,"num_write_kb":13102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"121'515","reported_seq":3352,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.796724+0000","last_change":"2024-09-14T09:06:46.882194+0000","last_active":"2024-09-14T09:07:30.796724+0000","last_peered":"2024-09-14T09:07:30.796724+0000","last_clean":"2024-09-14T09:07:30.796724+0000","last_became_active":"2024-09-14T09:06:46.880786+0000","last_became_peered":"2024-09-14T09:06:46.880786+0000","last_unstale":"2024-09-14T09:07:30.796724+0000","last_undegraded":"2024-09-14T09:07:30.796724+0000","last_fullsized":"2024-09-14T09:07:30.796724+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:57:12.240918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014128,"stat_sum":{"num_bytes":28932831,"num_objects":67,"num_object_clones":23,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":26,"num_read":4943,"num_read_kb":20031,"num_write":583,"num_write_kb":11737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":223,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.867245+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:07:30.867245+0000","last_peered":"2024-09-14T09:07:30.867245+0000","last_clean":"2024-09-14T09:07:30.867245+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:07:30.867245+0000","last_undegraded":"2024-09-14T09:07:30.867245+0000","last_fullsized":"2024-09-14T09:07:30.867245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"121'288","reported_seq":1859,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:31.160737+0000","last_change":"2024-09-14T09:05:28.822791+0000","last_active":"2024-09-14T09:07:31.160737+0000","last_peered":"2024-09-14T09:07:31.160737+0000","last_clean":"2024-09-14T09:07:31.160737+0000","last_became_active":"2024-09-14T09:05:26.817109+0000","last_became_peered":"2024-09-14T09:05:26.817109+0000","last_unstale":"2024-09-14T09:07:31.160737+0000","last_undegraded":"2024-09-14T09:07:31.160737+0000","last_fullsized":"2024-09-14T09:07:31.160737+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:34:21.190046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011728439999999999,"stat_sum":{"num_bytes":25578775,"num_objects":75,"num_object_clones":26,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":33,"num_read":1629,"num_read_kb":10259,"num_write":292,"num_write_kb":4467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":221,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.861686+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:07:30.861686+0000","last_peered":"2024-09-14T09:07:30.861686+0000","last_clean":"2024-09-14T09:07:30.861686+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:07:30.861686+0000","last_undegraded":"2024-09-14T09:07:30.861686+0000","last_fullsized":"2024-09-14T09:07:30.861686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"121'368","reported_seq":3908,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:31.743563+0000","last_change":"2024-09-14T09:06:45.868939+0000","last_active":"2024-09-14T09:07:31.743563+0000","last_peered":"2024-09-14T09:07:31.743563+0000","last_clean":"2024-09-14T09:07:31.743563+0000","last_became_active":"2024-09-14T09:06:45.868194+0000","last_became_peered":"2024-09-14T09:06:45.868194+0000","last_unstale":"2024-09-14T09:07:31.743563+0000","last_undegraded":"2024-09-14T09:07:31.743563+0000","last_fullsized":"2024-09-14T09:07:31.743563+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:45:42.424083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074815999999999999,"stat_sum":{"num_bytes":49455397,"num_objects":84,"num_object_clones":31,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":42,"num_read":3248,"num_read_kb":23974,"num_write":344,"num_write_kb":13814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.966722+0000","last_change":"2024-09-14T09:07:21.856721+0000","last_active":"2024-09-14T09:07:30.966722+0000","last_peered":"2024-09-14T09:07:30.966722+0000","last_clean":"2024-09-14T09:07:30.966722+0000","last_became_active":"2024-09-14T09:07:21.856091+0000","last_became_peered":"2024-09-14T09:07:21.856091+0000","last_unstale":"2024-09-14T09:07:30.966722+0000","last_undegraded":"2024-09-14T09:07:30.966722+0000","last_fullsized":"2024-09-14T09:07:30.966722+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:16:17.683341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"121'317","reported_seq":2365,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.924984+0000","last_change":"2024-09-14T09:05:29.038389+0000","last_active":"2024-09-14T09:07:30.924984+0000","last_peered":"2024-09-14T09:07:30.924984+0000","last_clean":"2024-09-14T09:07:30.924984+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:07:30.924984+0000","last_undegraded":"2024-09-14T09:07:30.924984+0000","last_fullsized":"2024-09-14T09:07:30.924984+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:23:13.684863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001479076,"stat_sum":{"num_bytes":33120438,"num_objects":66,"num_object_clones":20,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":28,"num_read":2149,"num_read_kb":10517,"num_write":357,"num_write_kb":6652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":122,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.925010+0000","last_change":"2024-09-14T09:06:25.227644+0000","last_active":"2024-09-14T09:07:30.925010+0000","last_peered":"2024-09-14T09:07:30.925010+0000","last_clean":"2024-09-14T09:07:30.925010+0000","last_became_active":"2024-09-14T09:06:25.222344+0000","last_became_peered":"2024-09-14T09:06:25.222344+0000","last_unstale":"2024-09-14T09:07:30.925010+0000","last_undegraded":"2024-09-14T09:07:30.925010+0000","last_fullsized":"2024-09-14T09:07:30.925010+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:23:33.414859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":35,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.966700+0000","last_change":"2024-09-14T09:07:21.876180+0000","last_active":"2024-09-14T09:07:30.966700+0000","last_peered":"2024-09-14T09:07:30.966700+0000","last_clean":"2024-09-14T09:07:30.966700+0000","last_became_active":"2024-09-14T09:07:21.860117+0000","last_became_peered":"2024-09-14T09:07:21.860117+0000","last_unstale":"2024-09-14T09:07:30.966700+0000","last_undegraded":"2024-09-14T09:07:30.966700+0000","last_fullsized":"2024-09-14T09:07:30.966700+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:58:36.142971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"121'380","reported_seq":4149,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.924962+0000","last_change":"2024-09-14T09:05:29.037923+0000","last_active":"2024-09-14T09:07:30.924962+0000","last_peered":"2024-09-14T09:07:30.924962+0000","last_clean":"2024-09-14T09:07:30.924962+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:07:30.924962+0000","last_undegraded":"2024-09-14T09:07:30.924962+0000","last_fullsized":"2024-09-14T09:07:30.924962+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:30:29.931183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099199800000000001,"stat_sum":{"num_bytes":53735258,"num_objects":69,"num_object_clones":23,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":29,"num_read":4438,"num_read_kb":35289,"num_write":384,"num_write_kb":12584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"121'346","reported_seq":2327,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:31.556943+0000","last_change":"2024-09-14T09:07:06.854914+0000","last_active":"2024-09-14T09:07:31.556943+0000","last_peered":"2024-09-14T09:07:31.556943+0000","last_clean":"2024-09-14T09:07:31.556943+0000","last_became_active":"2024-09-14T09:07:06.851020+0000","last_became_peered":"2024-09-14T09:07:06.851020+0000","last_unstale":"2024-09-14T09:07:31.556943+0000","last_undegraded":"2024-09-14T09:07:31.556943+0000","last_fullsized":"2024-09-14T09:07:31.556943+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:39:42.112091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001266028,"stat_sum":{"num_bytes":40275356,"num_objects":81,"num_object_clones":26,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":34,"num_read":1692,"num_read_kb":14190,"num_write":312,"num_write_kb":18897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":12315549,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"121'581","reported_seq":4325,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:31.565041+0000","last_change":"2024-09-14T09:06:45.870752+0000","last_active":"2024-09-14T09:07:31.565041+0000","last_peered":"2024-09-14T09:07:31.565041+0000","last_clean":"2024-09-14T09:07:31.565041+0000","last_became_active":"2024-09-14T09:06:45.869266+0000","last_became_peered":"2024-09-14T09:06:45.869266+0000","last_unstale":"2024-09-14T09:07:31.565041+0000","last_undegraded":"2024-09-14T09:07:31.565041+0000","last_fullsized":"2024-09-14T09:07:31.565041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:21:56.523332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185731,"stat_sum":{"num_bytes":55367632,"num_objects":79,"num_object_clones":26,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":34,"num_read":5131,"num_read_kb":38510,"num_write":767,"num_write_kb":19592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"121'411","reported_seq":8628,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.925154+0000","last_change":"2024-09-14T09:06:02.837388+0000","last_active":"2024-09-14T09:07:30.925154+0000","last_peered":"2024-09-14T09:07:30.925154+0000","last_clean":"2024-09-14T09:07:30.925154+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:07:30.925154+0000","last_undegraded":"2024-09-14T09:07:30.925154+0000","last_fullsized":"2024-09-14T09:07:30.925154+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:31:55.504126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001585295,"stat_sum":{"num_bytes":48749754,"num_objects":62,"num_object_clones":18,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":26,"num_read":9344,"num_read_kb":58223,"num_write":403,"num_write_kb":10388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"122'331","reported_seq":2180,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:31.566227+0000","last_change":"2024-09-14T09:05:28.834647+0000","last_active":"2024-09-14T09:07:31.566227+0000","last_peered":"2024-09-14T09:07:31.566227+0000","last_clean":"2024-09-14T09:07:31.566227+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:07:31.566227+0000","last_undegraded":"2024-09-14T09:07:31.566227+0000","last_fullsized":"2024-09-14T09:07:31.566227+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:40:34.022981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244290000000001,"stat_sum":{"num_bytes":31376447,"num_objects":77,"num_object_clones":25,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":29,"num_read":2190,"num_read_kb":10705,"num_write":332,"num_write_kb":9620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"121'362","reported_seq":4609,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.967125+0000","last_change":"2024-09-14T09:07:26.857275+0000","last_active":"2024-09-14T09:07:30.967125+0000","last_peered":"2024-09-14T09:07:30.967125+0000","last_clean":"2024-09-14T09:07:30.967125+0000","last_became_active":"2024-09-14T09:07:26.853961+0000","last_became_peered":"2024-09-14T09:07:26.853961+0000","last_unstale":"2024-09-14T09:07:30.967125+0000","last_undegraded":"2024-09-14T09:07:30.967125+0000","last_fullsized":"2024-09-14T09:07:30.967125+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:03:04.704761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072987200000000001,"stat_sum":{"num_bytes":32393738,"num_objects":65,"num_object_clones":19,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":26,"num_read":4564,"num_read_kb":28662,"num_write":403,"num_write_kb":16162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":25396220,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"121'319","reported_seq":2745,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.967221+0000","last_change":"2024-09-14T09:07:26.860392+0000","last_active":"2024-09-14T09:07:30.967221+0000","last_peered":"2024-09-14T09:07:30.967221+0000","last_clean":"2024-09-14T09:07:30.967221+0000","last_became_active":"2024-09-14T09:07:26.855371+0000","last_became_peered":"2024-09-14T09:07:26.855371+0000","last_unstale":"2024-09-14T09:07:30.967221+0000","last_undegraded":"2024-09-14T09:07:30.967221+0000","last_fullsized":"2024-09-14T09:07:30.967221+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:48:03.621693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057596100000000005,"stat_sum":{"num_bytes":39163516,"num_objects":66,"num_object_clones":18,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":29,"num_read":2683,"num_read_kb":15584,"num_write":293,"num_write_kb":6521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":14123394,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"121'373","reported_seq":3081,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.967425+0000","last_change":"2024-09-14T09:07:25.117894+0000","last_active":"2024-09-14T09:07:30.967425+0000","last_peered":"2024-09-14T09:07:30.967425+0000","last_clean":"2024-09-14T09:07:30.967425+0000","last_became_active":"2024-09-14T09:07:25.104940+0000","last_became_peered":"2024-09-14T09:07:25.104940+0000","last_unstale":"2024-09-14T09:07:30.967425+0000","last_undegraded":"2024-09-14T09:07:30.967425+0000","last_fullsized":"2024-09-14T09:07:30.967425+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:50:01.287155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000909463,"stat_sum":{"num_bytes":33685643,"num_objects":77,"num_object_clones":25,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":30,"num_read":2435,"num_read_kb":9681,"num_write":374,"num_write_kb":9574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":333,"num_bytes_recovered":11864981,"num_keys_recovered":45,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"122'401","reported_seq":5605,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:31.565680+0000","last_change":"2024-09-14T09:06:00.166357+0000","last_active":"2024-09-14T09:07:31.565680+0000","last_peered":"2024-09-14T09:07:31.565680+0000","last_clean":"2024-09-14T09:07:31.565680+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:07:31.565680+0000","last_undegraded":"2024-09-14T09:07:31.565680+0000","last_fullsized":"2024-09-14T09:07:31.565680+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:06:36.918610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011381609999999999,"stat_sum":{"num_bytes":97288082,"num_objects":85,"num_object_clones":30,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":37,"num_read":5241,"num_read_kb":58881,"num_write":473,"num_write_kb":19920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"121'473","reported_seq":3792,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.966755+0000","last_change":"2024-09-14T09:07:25.107931+0000","last_active":"2024-09-14T09:07:30.966755+0000","last_peered":"2024-09-14T09:07:30.966755+0000","last_clean":"2024-09-14T09:07:30.966755+0000","last_became_active":"2024-09-14T09:07:25.105011+0000","last_became_peered":"2024-09-14T09:07:25.105011+0000","last_unstale":"2024-09-14T09:07:30.966755+0000","last_undegraded":"2024-09-14T09:07:30.966755+0000","last_fullsized":"2024-09-14T09:07:30.966755+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:19:56.842722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092949299999999999,"stat_sum":{"num_bytes":18845984,"num_objects":91,"num_object_clones":30,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":47,"num_read":4048,"num_read_kb":9641,"num_write":549,"num_write_kb":6121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":7405746,"num_keys_recovered":72,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"122'611","reported_seq":4291,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:32.209225+0000","last_change":"2024-09-14T09:07:26.105039+0000","last_active":"2024-09-14T09:07:32.209225+0000","last_peered":"2024-09-14T09:07:32.209225+0000","last_clean":"2024-09-14T09:07:32.209225+0000","last_became_active":"2024-09-14T09:07:26.101586+0000","last_became_peered":"2024-09-14T09:07:26.101586+0000","last_unstale":"2024-09-14T09:07:32.209225+0000","last_undegraded":"2024-09-14T09:07:32.209225+0000","last_fullsized":"2024-09-14T09:07:32.209225+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:30:30.393589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018357099999999999,"stat_sum":{"num_bytes":64086808,"num_objects":69,"num_object_clones":23,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":29,"num_read":5420,"num_read_kb":15473,"num_write":812,"num_write_kb":8805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":10260744,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"121'397","reported_seq":5831,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-14T09:07:30.966816+0000","last_change":"2024-09-14T09:07:28.854184+0000","last_active":"2024-09-14T09:07:30.966816+0000","last_peered":"2024-09-14T09:07:30.966816+0000","last_clean":"2024-09-14T09:07:30.966816+0000","last_became_active":"2024-09-14T09:07:28.851840+0000","last_became_peered":"2024-09-14T09:07:28.851840+0000","last_unstale":"2024-09-14T09:07:30.966816+0000","last_undegraded":"2024-09-14T09:07:30.966816+0000","last_fullsized":"2024-09-14T09:07:30.966816+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:43:31.581954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061978900000000004,"stat_sum":{"num_bytes":28414671,"num_objects":73,"num_object_clones":25,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":31,"num_read":6457,"num_read_kb":28955,"num_write":364,"num_write_kb":9156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":24080444,"num_keys_recovered":69,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1306962487,"num_objects":2396,"num_object_clones":800,"num_object_copies":4792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2396,"num_whiteouts":1014,"num_read":155253,"num_read_kb":753523,"num_write":20271,"num_write_kb":329807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3682,"num_bytes_recovered":198098795,"num_keys_recovered":652,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":55676928,"data_stored":456909041,"data_compressed":15315181,"data_compressed_allocated":39739392,"data_compressed_original":446907415,"omap_allocated":0,"internal_metadata":0},"log_size":16412,"ondisk_log_size":16412,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20292,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47240,"kb_used_data":7724,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94324600,"statfs":{"total":96636764160,"available":96588390400,"internally_reserved":0,"allocated":7909376,"data_stored":57911221,"data_compressed":1780069,"data_compressed_allocated":4993024,"data_compressed_original":56340011,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80416,"kb_used_data":8972,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94291424,"statfs":{"total":96636764160,"available":96554418176,"internally_reserved":0,"allocated":9187328,"data_stored":65751110,"data_compressed":1981666,"data_compressed_allocated":5488640,"data_compressed_original":63694311,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509473,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60368,"kb_used_data":2812,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311472,"statfs":{"total":96636764160,"available":96574947328,"internally_reserved":0,"allocated":2879488,"data_stored":12028047,"data_compressed":407788,"data_compressed_allocated":1064960,"data_compressed_original":11141920,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509475,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59256,"kb_used_data":10024,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94312584,"statfs":{"total":96636764160,"available":96576086016,"internally_reserved":0,"allocated":10264576,"data_stored":76231873,"data_compressed":2397752,"data_compressed_allocated":6270976,"data_compressed_original":74001196,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542175,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66828,"kb_used_data":9276,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94305012,"statfs":{"total":96636764160,"available":96568332288,"internally_reserved":0,"allocated":9498624,"data_stored":63502163,"data_compressed":2094835,"data_compressed_allocated":5578752,"data_compressed_original":61171080,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542176,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42200,"kb_used_data":11008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329640,"statfs":{"total":96636764160,"available":96593551360,"internally_reserved":0,"allocated":11272192,"data_stored":81410442,"data_compressed":2636481,"data_compressed_allocated":6987776,"data_compressed_original":78926925,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542176,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58468,"kb_used_data":9228,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94313372,"statfs":{"total":96636764160,"available":96576892928,"internally_reserved":0,"allocated":9449472,"data_stored":65038360,"data_compressed":2314837,"data_compressed_allocated":5578752,"data_compressed_original":62827641,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37348,"kb_used_data":6156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334492,"statfs":{"total":96636764160,"available":96598519808,"internally_reserved":0,"allocated":6303744,"data_stored":36143734,"data_compressed":1353627,"data_compressed_allocated":3293184,"data_compressed_original":34475811,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9052160,"data_stored":77131852,"data_compressed":2552998,"data_compressed_allocated":6516736,"data_compressed_original":75594540,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4861952,"data_stored":35935164,"data_compressed":1384119,"data_compressed_allocated":3293184,"data_compressed_original":34941803,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1429504,"data_stored":11347843,"data_compressed":407788,"data_compressed_allocated":1064960,"data_compressed_original":11141920,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8028160,"data_stored":65093133,"data_compressed":2347305,"data_compressed_allocated":5595136,"data_compressed_original":63556729,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":9834496,"data_stored":80736127,"data_compressed":2636481,"data_compressed_allocated":6987776,"data_compressed_original":78926925,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8028160,"data_stored":67241439,"data_compressed":2143124,"data_compressed_allocated":5771264,"data_compressed_original":65861095,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6422528,"data_stored":57186003,"data_compressed":1755571,"data_compressed_allocated":4968448,"data_compressed_original":56303147,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8019968,"data_stored":62237480,"data_compressed":2087795,"data_compressed_allocated":5541888,"data_compressed_original":60581256,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:07:33.777 INFO:tasks.thrashosds.thrasher:Setting 3.19 to [3, 0, 2, 6] 2024-09-14T09:07:33.777 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.19', '3', '0', '2', '6'] 2024-09-14T09:07:33.778 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.19 3 0 2 6 2024-09-14T09:07:33.819 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:33.822+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:33.875 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is unset 2024-09-14T09:07:33.877 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:33.877 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:33.901 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:07:33.919 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:33.922+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:34.019 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:34.019+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:34.029 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:34.030 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:34.043 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:07:34.099 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone40 from 0xcf6dd5e (overlap 0x3d92d4d) to 0x376a8c5 2024-09-14T09:07:34.121 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:34.122+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:34.212 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:34.212 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:34.221 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:34.222+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:34.225 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:07:34.322 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:34.323+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:34.382 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:34.382 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:34.398 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:07:34.442 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:34.443+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:34.456 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-14T09:07:34.490 INFO:teuthology.orchestra.run.smithi176.stdout:5511 writesame 0xa7fa347 thru 0xa7fe97c data_size 0x2(0x4636 bytes) 2024-09-14T09:07:34.513 INFO:teuthology.orchestra.run.smithi176.stdout:5512 read 0x28865fb thru 0x2893098 (0xca9e bytes) 2024-09-14T09:07:34.516 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:34.516 INFO:teuthology.orchestra.run.smithi176.stdout:5514 write 0x8609979 thru 0x86178ad (0xdf35 bytes) 2024-09-14T09:07:34.524 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:34.523+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:34.542 INFO:teuthology.orchestra.run.smithi176.stdout:5515 punch from 0x88730ff to 0x887a53e, (0x743f bytes) 2024-09-14T09:07:34.542 INFO:teuthology.orchestra.run.smithi176.stdout:5516 writesame 0x3332f8f thru 0x3333ddc data_size 0x1(0xe4e bytes) 2024-09-14T09:07:34.560 INFO:teuthology.orchestra.run.smithi176.stdout:5518 read 0xb9abefe thru 0xb9b3947 (0x7a4a bytes) 2024-09-14T09:07:34.562 INFO:teuthology.orchestra.run.smithi176.stdout:5519 trunc from 0xcf6dd5e to 0xe3258cf 2024-09-14T09:07:34.571 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:34.571 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:34.589 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:07:34.593 INFO:teuthology.orchestra.run.smithi176.stdout:5521 trunc from 0xe3258cf to 0x4ef4454 2024-09-14T09:07:34.625 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:34.626+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:34.643 INFO:teuthology.orchestra.run.smithi176.stdout:5522 read 0x935c50 thru 0x93a9e3 (0x4d94 bytes) 2024-09-14T09:07:34.643 INFO:teuthology.orchestra.run.smithi176.stdout:5523 read 0x1348a93 thru 0x134f979 (0x6ee7 bytes) 2024-09-14T09:07:34.645 INFO:teuthology.orchestra.run.smithi176.stdout:5524 punch from 0x3cf74c0 to 0x3cfe767, (0x72a7 bytes) 2024-09-14T09:07:34.645 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:34.645 INFO:teuthology.orchestra.run.smithi176.stdout:5526 flatten 2024-09-14T09:07:34.726 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:34.727+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:34.734 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:34.734 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:34.750 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:07:34.828 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:34.830+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:34.888 INFO:teuthology.orchestra.run.smithi176.stderr:set 3.19 pg_upmap_items mapping to [3->0,2->6] 2024-09-14T09:07:34.929 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:34.927+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:34.953 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:34.953 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:34.973 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:07:35.062 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:35.063+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:35.131 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:35.131+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:35.134 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:35.134 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:35.148 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:07:35.175 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:07:35.234 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:35.235+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:35.313 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:35.314 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:35.329 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:07:35.333 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:35.334 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:35.336 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:35.331+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:35.433 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:35.431+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:35.448 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:35.449 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:35.463 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:07:35.558 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:35.559+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:35.633 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:35.633 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:35.635 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:35.635+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:35.647 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:07:35.736 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:35.735+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:35.778 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:35.778 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:35.791 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:07:35.838 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:35.842+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:35.906 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T09:07:35.915 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:35.915 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:35.929 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:07:35.970 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:35.974+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:36.038 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:36.039+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:36.095 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:36.095 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:36.109 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:07:36.139 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:36.139+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:36.233 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:36.233 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:36.240 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:36.242+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:36.248 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:07:36.341 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:36.339+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:36.413 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:36.413 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:36.427 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:07:36.442 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:36.443+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:36.543 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:36.546+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:36.548 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:36.548 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:36.562 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:07:36.654 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:36.658+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:36.732 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:36.733 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:36.744 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:36.746+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:36.746 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:07:36.845 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:36.846+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:36.910 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is set 2024-09-14T09:07:36.917 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:36.917 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:36.931 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:07:36.955 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:36.956 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:36.974 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:36.978+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:36.975 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:36.975 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:36.997 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:36.997 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:37.018 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:07:37.019 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:07:37.046 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:37.047+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:37.046 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:37.046 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:37.073 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:37.074 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:37.102 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:37.102 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:37.118 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:37.119 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:07:37.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:07:37.119 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:07:37.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.119 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65139 3.a 3:54c90b7d:::rbd_data.115e21c31fcf.0000000000000008:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:07:37.119 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:07:37.119 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 75.741444169000005, 2024-09-14T09:07:37.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14429895700000001, 2024-09-14T09:07:37.119 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.119 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.119 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:37.119 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.120 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.120 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.120 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65139 2024-09-14T09:07:37.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.120 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:07:37.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:07:37.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379715+0000", 2024-09-14T09:07:37.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.576e-06 2024-09-14T09:07:37.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379724+0000", 2024-09-14T09:07:37.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1779999999999992e-06 2024-09-14T09:07:37.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379746+0000", 2024-09-14T09:07:37.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1661000000000001e-05 2024-09-14T09:07:37.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379759+0000", 2024-09-14T09:07:37.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.324e-05 2024-09-14T09:07:37.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379882+0000", 2024-09-14T09:07:37.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012239400000000001 2024-09-14T09:07:37.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.380023+0000", 2024-09-14T09:07:37.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000141402 2024-09-14T09:07:37.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:37.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.391553+0000", 2024-09-14T09:07:37.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011530197000000001 2024-09-14T09:07:37.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:37.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523927+0000", 2024-09-14T09:07:37.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.132373935 2024-09-14T09:07:37.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:37.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523941+0000", 2024-09-14T09:07:37.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3326000000000001e-05 2024-09-14T09:07:37.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:37.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523977+0000", 2024-09-14T09:07:37.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6185e-05 2024-09-14T09:07:37.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.524012+0000", 2024-09-14T09:07:37.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4863000000000001e-05 2024-09-14T09:07:37.124 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.124 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.125 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.125 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107606 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 14336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:37.125 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:07:37.125 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.714987268, 2024-09-14T09:07:37.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14293513799999999, 2024-09-14T09:07:37.125 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.125 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.125 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.125 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.125 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.125 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.125 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107606 2024-09-14T09:07:37.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.126 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:07:37.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:07:37.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:07:37.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1600000000000001e-07 2024-09-14T09:07:37.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406174+0000", 2024-09-14T09:07:37.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4199999999999999e-06 2024-09-14T09:07:37.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406188+0000", 2024-09-14T09:07:37.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4114000000000001e-05 2024-09-14T09:07:37.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406198+0000", 2024-09-14T09:07:37.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.041e-05 2024-09-14T09:07:37.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.548700+0000", 2024-09-14T09:07:37.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14250141299999999 2024-09-14T09:07:37.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.548783+0000", 2024-09-14T09:07:37.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3578999999999994e-05 2024-09-14T09:07:37.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549105+0000", 2024-09-14T09:07:37.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032158600000000001 2024-09-14T09:07:37.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.129 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.129 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107608 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:37.129 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:07:37.129 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.714921232, 2024-09-14T09:07:37.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14603042799999999, 2024-09-14T09:07:37.129 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.129 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.129 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.130 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.130 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.130 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.130 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107608 2024-09-14T09:07:37.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.130 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:07:37.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:07:37.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:07:37.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1200000000000002e-07 2024-09-14T09:07:37.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406241+0000", 2024-09-14T09:07:37.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7539999999999997e-06 2024-09-14T09:07:37.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406275+0000", 2024-09-14T09:07:37.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3512e-05 2024-09-14T09:07:37.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406322+0000", 2024-09-14T09:07:37.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6764999999999997e-05 2024-09-14T09:07:37.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549149+0000", 2024-09-14T09:07:37.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14282725499999999 2024-09-14T09:07:37.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549198+0000", 2024-09-14T09:07:37.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9599999999999999e-05 2024-09-14T09:07:37.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552266+0000", 2024-09-14T09:07:37.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030677299999999998 2024-09-14T09:07:37.133 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.133 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.133 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.133 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107610 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 22528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:37.133 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:07:37.134 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.714799357, 2024-09-14T09:07:37.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146437919, 2024-09-14T09:07:37.134 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.134 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.134 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.134 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.134 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.134 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.134 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107610 2024-09-14T09:07:37.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.134 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:07:37.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:07:37.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:07:37.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.02e-07 2024-09-14T09:07:37.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406363+0000", 2024-09-14T09:07:37.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8099999999999997e-06 2024-09-14T09:07:37.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406390+0000", 2024-09-14T09:07:37.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7004000000000001e-05 2024-09-14T09:07:37.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406404+0000", 2024-09-14T09:07:37.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3682e-05 2024-09-14T09:07:37.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552335+0000", 2024-09-14T09:07:37.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14593077800000001 2024-09-14T09:07:37.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552431+0000", 2024-09-14T09:07:37.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6086000000000001e-05 2024-09-14T09:07:37.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552795+0000", 2024-09-14T09:07:37.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000364657 2024-09-14T09:07:37.137 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.137 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.138 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.138 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107612 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 26624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:37.138 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:07:37.138 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.714734094000001, 2024-09-14T09:07:37.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146762061, 2024-09-14T09:07:37.138 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.138 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.138 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.138 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.138 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.138 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.138 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107612 2024-09-14T09:07:37.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.139 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:07:37.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:07:37.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406424+0000", 2024-09-14T09:07:37.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2600000000000002e-07 2024-09-14T09:07:37.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406451+0000", 2024-09-14T09:07:37.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7250000000000002e-05 2024-09-14T09:07:37.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406487+0000", 2024-09-14T09:07:37.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6086e-05 2024-09-14T09:07:37.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406498+0000", 2024-09-14T09:07:37.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0858e-05 2024-09-14T09:07:37.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552821+0000", 2024-09-14T09:07:37.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14632319099999999 2024-09-14T09:07:37.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552875+0000", 2024-09-14T09:07:37.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4262999999999999e-05 2024-09-14T09:07:37.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553185+0000", 2024-09-14T09:07:37.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000309687 2024-09-14T09:07:37.142 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.142 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.142 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.143 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107614 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:37.143 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:07:37.143 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.71464656, 2024-09-14T09:07:37.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14737121, 2024-09-14T09:07:37.143 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.143 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.143 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.143 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.143 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.143 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.143 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107614 2024-09-14T09:07:37.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.143 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:07:37.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:07:37.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406511+0000", 2024-09-14T09:07:37.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1799999999999995e-07 2024-09-14T09:07:37.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406514+0000", 2024-09-14T09:07:37.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8660000000000002e-06 2024-09-14T09:07:37.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406523+0000", 2024-09-14T09:07:37.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1940000000000004e-06 2024-09-14T09:07:37.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406531+0000", 2024-09-14T09:07:37.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3159999999999997e-06 2024-09-14T09:07:37.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553233+0000", 2024-09-14T09:07:37.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14670117899999999 2024-09-14T09:07:37.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553334+0000", 2024-09-14T09:07:37.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100982 2024-09-14T09:07:37.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553882+0000", 2024-09-14T09:07:37.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00054805499999999998 2024-09-14T09:07:37.147 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.147 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.147 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.147 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107616 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 34816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:37.147 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:07:37.147 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.714614924999999, 2024-09-14T09:07:37.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14780390500000001, 2024-09-14T09:07:37.147 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.147 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.147 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.148 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.148 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.148 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.148 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107616 2024-09-14T09:07:37.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.148 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:07:37.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:07:37.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406543+0000", 2024-09-14T09:07:37.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0200000000000001e-07 2024-09-14T09:07:37.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406545+0000", 2024-09-14T09:07:37.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3360000000000002e-06 2024-09-14T09:07:37.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406552+0000", 2024-09-14T09:07:37.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9779999999999999e-06 2024-09-14T09:07:37.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406559+0000", 2024-09-14T09:07:37.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6919999999999996e-06 2024-09-14T09:07:37.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553917+0000", 2024-09-14T09:07:37.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14735791600000001 2024-09-14T09:07:37.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554033+0000", 2024-09-14T09:07:37.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116047 2024-09-14T09:07:37.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554346+0000", 2024-09-14T09:07:37.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031323400000000002 2024-09-14T09:07:37.151 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.152 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.152 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.152 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107618 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 38912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:37.152 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:07:37.152 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.714586125, 2024-09-14T09:07:37.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14816009599999999, 2024-09-14T09:07:37.152 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.152 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.152 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.152 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.152 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.152 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.153 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107618 2024-09-14T09:07:37.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.153 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:07:37.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:07:37.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:07:37.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9999999999999998e-07 2024-09-14T09:07:37.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406574+0000", 2024-09-14T09:07:37.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2560000000000001e-06 2024-09-14T09:07:37.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406581+0000", 2024-09-14T09:07:37.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1539999999999996e-06 2024-09-14T09:07:37.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406586+0000", 2024-09-14T09:07:37.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.904e-06 2024-09-14T09:07:37.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554385+0000", 2024-09-14T09:07:37.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.147799443 2024-09-14T09:07:37.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554441+0000", 2024-09-14T09:07:37.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5921000000000002e-05 2024-09-14T09:07:37.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554731+0000", 2024-09-14T09:07:37.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028991800000000001 2024-09-14T09:07:37.156 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.156 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.156 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.156 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107620 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:37.156 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:07:37.156 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.714564342999999, 2024-09-14T09:07:37.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.148512643, 2024-09-14T09:07:37.157 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.157 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.157 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.157 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.157 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.157 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.157 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107620 2024-09-14T09:07:37.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.157 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:07:37.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:07:37.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:07:37.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9999999999999998e-07 2024-09-14T09:07:37.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406595+0000", 2024-09-14T09:07:37.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9360000000000002e-06 2024-09-14T09:07:37.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406605+0000", 2024-09-14T09:07:37.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0268000000000001e-05 2024-09-14T09:07:37.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406614+0000", 2024-09-14T09:07:37.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7180000000000002e-06 2024-09-14T09:07:37.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554750+0000", 2024-09-14T09:07:37.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14813599699999999 2024-09-14T09:07:37.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554794+0000", 2024-09-14T09:07:37.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.401e-05 2024-09-14T09:07:37.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555105+0000", 2024-09-14T09:07:37.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031131399999999998 2024-09-14T09:07:37.160 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.161 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.161 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.161 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107622 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:37.161 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:07:37.161 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.714533087, 2024-09-14T09:07:37.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14902820899999999, 2024-09-14T09:07:37.161 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.161 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.161 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.161 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.161 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.161 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.162 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107622 2024-09-14T09:07:37.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.162 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:07:37.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:07:37.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:07:37.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9200000000000001e-07 2024-09-14T09:07:37.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406636+0000", 2024-09-14T09:07:37.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.189e-05 2024-09-14T09:07:37.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406649+0000", 2024-09-14T09:07:37.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2442e-05 2024-09-14T09:07:37.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406658+0000", 2024-09-14T09:07:37.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3239999999999992e-06 2024-09-14T09:07:37.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555287+0000", 2024-09-14T09:07:37.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14862868400000001 2024-09-14T09:07:37.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555348+0000", 2024-09-14T09:07:37.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1686999999999994e-05 2024-09-14T09:07:37.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555652+0000", 2024-09-14T09:07:37.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00030369000000000001 2024-09-14T09:07:37.165 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.165 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.165 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.165 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107624 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:37.165 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:07:37.166 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.714489393999999, 2024-09-14T09:07:37.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14934019900000001, 2024-09-14T09:07:37.166 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.166 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.166 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.166 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.166 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.166 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.166 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107624 2024-09-14T09:07:37.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.166 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:07:37.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:07:37.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:07:37.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1399999999999997e-07 2024-09-14T09:07:37.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406670+0000", 2024-09-14T09:07:37.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4099999999999998e-06 2024-09-14T09:07:37.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406680+0000", 2024-09-14T09:07:37.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6460000000000005e-06 2024-09-14T09:07:37.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406688+0000", 2024-09-14T09:07:37.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5859999999999999e-06 2024-09-14T09:07:37.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555693+0000", 2024-09-14T09:07:37.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14900544499999999 2024-09-14T09:07:37.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555743+0000", 2024-09-14T09:07:37.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9966999999999998e-05 2024-09-14T09:07:37.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556008+0000", 2024-09-14T09:07:37.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026463099999999998 2024-09-14T09:07:37.173 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.173 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.173 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.173 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107626 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:37.173 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:07:37.173 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.71445742, 2024-09-14T09:07:37.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14983149500000001, 2024-09-14T09:07:37.173 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.173 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.173 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.173 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.174 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107626 2024-09-14T09:07:37.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.174 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:07:37.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:07:37.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408087+0000", 2024-09-14T09:07:37.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001387735 2024-09-14T09:07:37.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408094+0000", 2024-09-14T09:07:37.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9399999999999996e-06 2024-09-14T09:07:37.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408122+0000", 2024-09-14T09:07:37.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8076e-05 2024-09-14T09:07:37.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408147+0000", 2024-09-14T09:07:37.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4726e-05 2024-09-14T09:07:37.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556048+0000", 2024-09-14T09:07:37.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14790144299999999 2024-09-14T09:07:37.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556175+0000", 2024-09-14T09:07:37.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012663 2024-09-14T09:07:37.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556531+0000", 2024-09-14T09:07:37.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035594499999999998 2024-09-14T09:07:37.177 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.177 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.177 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.178 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107628 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:37.178 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:07:37.178 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.712987808999999, 2024-09-14T09:07:37.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14882648900000001, 2024-09-14T09:07:37.178 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.178 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.178 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.178 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.178 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.178 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.178 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107628 2024-09-14T09:07:37.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.178 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:07:37.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:07:37.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408878+0000", 2024-09-14T09:07:37.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070858800000000001 2024-09-14T09:07:37.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408885+0000", 2024-09-14T09:07:37.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4839999999999998e-06 2024-09-14T09:07:37.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408900+0000", 2024-09-14T09:07:37.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5174000000000001e-05 2024-09-14T09:07:37.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408918+0000", 2024-09-14T09:07:37.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.808e-05 2024-09-14T09:07:37.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556572+0000", 2024-09-14T09:07:37.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14765399500000001 2024-09-14T09:07:37.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556651+0000", 2024-09-14T09:07:37.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8919999999999997e-05 2024-09-14T09:07:37.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556996+0000", 2024-09-14T09:07:37.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034424800000000001 2024-09-14T09:07:37.182 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.182 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.182 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.182 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107630 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 63488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:37.182 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:07:37.182 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.712218975000001, 2024-09-14T09:07:37.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14855159200000001, 2024-09-14T09:07:37.182 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.182 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.182 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.183 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.183 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.183 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.183 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107630 2024-09-14T09:07:37.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.183 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:07:37.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:07:37.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409510+0000", 2024-09-14T09:07:37.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00057189700000000003 2024-09-14T09:07:37.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409518+0000", 2024-09-14T09:07:37.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7619999999999995e-06 2024-09-14T09:07:37.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409534+0000", 2024-09-14T09:07:37.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.594e-05 2024-09-14T09:07:37.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409546+0000", 2024-09-14T09:07:37.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2604e-05 2024-09-14T09:07:37.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557048+0000", 2024-09-14T09:07:37.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14750137899999999 2024-09-14T09:07:37.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557117+0000", 2024-09-14T09:07:37.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8998999999999994e-05 2024-09-14T09:07:37.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557490+0000", 2024-09-14T09:07:37.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037301100000000002 2024-09-14T09:07:37.186 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.186 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.187 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.187 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107632 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:37.187 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:07:37.187 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.71158803, 2024-09-14T09:07:37.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14833944199999999, 2024-09-14T09:07:37.187 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.187 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.187 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.187 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.187 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.187 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.187 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107632 2024-09-14T09:07:37.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.188 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:07:37.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:07:37.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412609+0000", 2024-09-14T09:07:37.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003040105 2024-09-14T09:07:37.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412618+0000", 2024-09-14T09:07:37.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2960000000000004e-06 2024-09-14T09:07:37.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412640+0000", 2024-09-14T09:07:37.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1773999999999999e-05 2024-09-14T09:07:37.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412655+0000", 2024-09-14T09:07:37.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4538e-05 2024-09-14T09:07:37.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557538+0000", 2024-09-14T09:07:37.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14488311800000001 2024-09-14T09:07:37.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557594+0000", 2024-09-14T09:07:37.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6122999999999998e-05 2024-09-14T09:07:37.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557908+0000", 2024-09-14T09:07:37.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031448799999999997 2024-09-14T09:07:37.191 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.191 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.191 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.191 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107634 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:37.191 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:07:37.191 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.708470383, 2024-09-14T09:07:37.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.145666186, 2024-09-14T09:07:37.191 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.192 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.192 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.192 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.192 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.192 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.192 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107634 2024-09-14T09:07:37.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.192 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:07:37.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:07:37.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412688+0000", 2024-09-14T09:07:37.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9000000000000005e-07 2024-09-14T09:07:37.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412707+0000", 2024-09-14T09:07:37.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9232000000000002e-05 2024-09-14T09:07:37.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412722+0000", 2024-09-14T09:07:37.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5722e-05 2024-09-14T09:07:37.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412733+0000", 2024-09-14T09:07:37.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0909e-05 2024-09-14T09:07:37.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557946+0000", 2024-09-14T09:07:37.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14521263100000001 2024-09-14T09:07:37.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558020+0000", 2024-09-14T09:07:37.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4294000000000004e-05 2024-09-14T09:07:37.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558353+0000", 2024-09-14T09:07:37.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033240800000000002 2024-09-14T09:07:37.196 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.196 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.196 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.196 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107636 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:37.196 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:07:37.196 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.70840608, 2024-09-14T09:07:37.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14599377399999999, 2024-09-14T09:07:37.196 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.196 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.196 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.196 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.197 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.197 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.197 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107636 2024-09-14T09:07:37.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.197 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:07:37.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:07:37.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412752+0000", 2024-09-14T09:07:37.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.156e-06 2024-09-14T09:07:37.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412770+0000", 2024-09-14T09:07:37.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7521999999999999e-05 2024-09-14T09:07:37.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412790+0000", 2024-09-14T09:07:37.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0942e-05 2024-09-14T09:07:37.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412809+0000", 2024-09-14T09:07:37.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8887999999999999e-05 2024-09-14T09:07:37.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558387+0000", 2024-09-14T09:07:37.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14557798799999999 2024-09-14T09:07:37.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558448+0000", 2024-09-14T09:07:37.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0862000000000001e-05 2024-09-14T09:07:37.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558745+0000", 2024-09-14T09:07:37.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029641599999999998 2024-09-14T09:07:37.200 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.200 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.200 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.200 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107638 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:37.201 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:07:37.201 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.708318823999999, 2024-09-14T09:07:37.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146344799, 2024-09-14T09:07:37.201 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.201 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.201 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.201 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.201 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.201 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.201 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107638 2024-09-14T09:07:37.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.201 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:07:37.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:07:37.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412839+0000", 2024-09-14T09:07:37.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0720000000000001e-06 2024-09-14T09:07:37.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412844+0000", 2024-09-14T09:07:37.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7759999999999997e-06 2024-09-14T09:07:37.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412857+0000", 2024-09-14T09:07:37.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3138e-05 2024-09-14T09:07:37.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412867+0000", 2024-09-14T09:07:37.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7229999999999999e-06 2024-09-14T09:07:37.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558785+0000", 2024-09-14T09:07:37.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14591828700000001 2024-09-14T09:07:37.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558833+0000", 2024-09-14T09:07:37.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7790000000000002e-05 2024-09-14T09:07:37.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559183+0000", 2024-09-14T09:07:37.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035001299999999998 2024-09-14T09:07:37.205 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.205 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.205 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.205 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107640 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:37.205 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:07:37.205 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.708273673000001, 2024-09-14T09:07:37.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146686804, 2024-09-14T09:07:37.205 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.205 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.205 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.205 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.206 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.206 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.206 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107640 2024-09-14T09:07:37.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.206 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:07:37.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:07:37.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420446+0000", 2024-09-14T09:07:37.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075632529999999998 2024-09-14T09:07:37.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420456+0000", 2024-09-14T09:07:37.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0850000000000003e-06 2024-09-14T09:07:37.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420475+0000", 2024-09-14T09:07:37.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9406e-05 2024-09-14T09:07:37.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420488+0000", 2024-09-14T09:07:37.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3278000000000001e-05 2024-09-14T09:07:37.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559205+0000", 2024-09-14T09:07:37.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13871644999999999 2024-09-14T09:07:37.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559252+0000", 2024-09-14T09:07:37.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7666000000000001e-05 2024-09-14T09:07:37.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559570+0000", 2024-09-14T09:07:37.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031766599999999998 2024-09-14T09:07:37.209 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.209 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.209 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.209 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107660 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 124928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:37.209 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:07:37.209 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 13.700203431, 2024-09-14T09:07:37.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.142954628, 2024-09-14T09:07:37.209 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.210 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.210 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.210 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.210 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.210 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.210 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107660 2024-09-14T09:07:37.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.210 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:07:37.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:07:37.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428190+0000", 2024-09-14T09:07:37.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072360339999999997 2024-09-14T09:07:37.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428398+0000", 2024-09-14T09:07:37.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020898499999999999 2024-09-14T09:07:37.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428458+0000", 2024-09-14T09:07:37.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9985999999999998e-05 2024-09-14T09:07:37.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428480+0000", 2024-09-14T09:07:37.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1284000000000002e-05 2024-09-14T09:07:37.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563583+0000", 2024-09-14T09:07:37.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13510361800000001 2024-09-14T09:07:37.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563644+0000", 2024-09-14T09:07:37.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1092000000000004e-05 2024-09-14T09:07:37.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563908+0000", 2024-09-14T09:07:37.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026362899999999998 2024-09-14T09:07:37.213 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.213 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.213 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.214 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.214 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.214 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:37.216 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:07:37.216 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:37.218+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:37.318 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:37.319+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:37.377 INFO:teuthology.orchestra.run.smithi176.stdout:5527 writesame 0x3fea400 thru 0x3ff5d2a data_size 0x9(0xb92b bytes) 2024-09-14T09:07:37.387 INFO:teuthology.orchestra.run.smithi176.stdout:5529 write 0x63418c thru 0x634e06 (0xc7b bytes) 2024-09-14T09:07:37.395 INFO:teuthology.orchestra.run.smithi176.stdout:5530 trunc from 0x4ef4454 to 0xa579715 2024-09-14T09:07:37.402 INFO:teuthology.orchestra.run.smithi176.stdout:5531 read 0x12c56f3 thru 0x12d4686 (0xef94 bytes) 2024-09-14T09:07:37.406 INFO:teuthology.orchestra.run.smithi176.stdout:5533 trunc from 0xa579715 to 0xd308199 2024-09-14T09:07:37.411 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:37.411 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:37.411 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:07:37.411 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:37.412 INFO:teuthology.orchestra.run.smithi176.stdout:5535 writesame 0x184b35e thru 0x184de57 data_size 0x1(0x2afa bytes) 2024-09-14T09:07:37.417 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:37.418+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:37.421 INFO:teuthology.orchestra.run.smithi176.stdout:5536 trunc from 0xd308199 to 0xc31cb36 2024-09-14T09:07:37.426 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:07:37.427 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:37.427 INFO:teuthology.orchestra.run.smithi176.stdout:5538 read 0x2d8f803 thru 0x2d9dd89 (0xe587 bytes) 2024-09-14T09:07:37.428 INFO:teuthology.orchestra.run.smithi176.stdout:5541 write 0x83a823a thru 0x83b81ef (0xffb6 bytes) 2024-09-14T09:07:37.442 INFO:teuthology.orchestra.run.smithi176.stdout:5543 read 0x4c3521d thru 0x4c3c2e6 (0x70ca bytes) 2024-09-14T09:07:37.453 INFO:teuthology.orchestra.run.smithi176.stdout:5544 trunc from 0xc31cb36 to 0x32cbf78 2024-09-14T09:07:37.484 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:37.493 INFO:teuthology.orchestra.run.smithi176.stdout:5546 writesame 0x49b7f50 thru 0x49c1961 data_size 0x4a(0x9a12 bytes) 2024-09-14T09:07:37.493 INFO:teuthology.orchestra.run.smithi176.stdout:5547 punch from 0x436fc07 to 0x437dff8, (0xe3f1 bytes) 2024-09-14T09:07:37.506 INFO:teuthology.orchestra.run.smithi176.stdout:5548 write 0xa974a13 thru 0xa975aa0 (0x108e bytes) 2024-09-14T09:07:37.510 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:37.511 INFO:teuthology.orchestra.run.smithi176.stdout:5553 read 0x5e89434 thru 0x5e98314 (0xeee1 bytes) 2024-09-14T09:07:37.511 INFO:teuthology.orchestra.run.smithi176.stdout:5554 writesame 0x19402a1 thru 0x194b2bd data_size 0x47(0xb01d bytes) 2024-09-14T09:07:37.517 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:37.518+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:37.521 INFO:teuthology.orchestra.run.smithi176.stdout:5555 trunc from 0xa975aa1 to 0x8c70160 2024-09-14T09:07:37.534 INFO:teuthology.orchestra.run.smithi176.stdout:5556 punch from 0x7e85b17 to 0x7e93c81, (0xe16a bytes) 2024-09-14T09:07:37.534 INFO:teuthology.orchestra.run.smithi176.stdout:5557 punch from 0x5f23a4b to 0x5f2dcc5, (0xa27a bytes) 2024-09-14T09:07:37.534 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:37.534 INFO:teuthology.orchestra.run.smithi176.stdout:5560 punch from 0x64b2108 to 0x64bd5a0, (0xb498 bytes) 2024-09-14T09:07:37.534 INFO:teuthology.orchestra.run.smithi176.stdout:5561 read 0x4cb1c42 thru 0x4cbc2d7 (0xa696 bytes) 2024-09-14T09:07:37.534 INFO:teuthology.orchestra.run.smithi176.stdout:5562 read 0x7378f72 thru 0x737eac2 (0x5b51 bytes) 2024-09-14T09:07:37.543 INFO:teuthology.orchestra.run.smithi176.stdout:5563 trunc from 0x8c70160 to 0x8de076 2024-09-14T09:07:37.563 INFO:teuthology.orchestra.run.smithi176.stdout:5564 read 0x814918 thru 0x81ca61 (0x814a bytes) 2024-09-14T09:07:37.592 INFO:teuthology.orchestra.run.smithi176.stdout:5566 write 0xd1baad2 thru 0xd1c2bcc (0x80fb bytes) 2024-09-14T09:07:37.596 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:37.596 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:37.596 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:07:37.596 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:07:37.596 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:37.603 INFO:teuthology.orchestra.run.smithi176.stdout:5567 write 0x987ee70 thru 0x988229e (0x342f bytes) 2024-09-14T09:07:37.611 INFO:teuthology.orchestra.run.smithi176.stdout:5569 read 0x73e7bd4 thru 0x73f3b03 (0xbf30 bytes) 2024-09-14T09:07:37.611 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:37.611 INFO:teuthology.orchestra.run.smithi176.stdout:5571 read 0x1fc54d8 thru 0x1fcd4a4 (0x7fcd bytes) 2024-09-14T09:07:37.611 INFO:teuthology.orchestra.run.smithi176.stdout:5572 read 0x6ff9984 thru 0x7007107 (0xd784 bytes) 2024-09-14T09:07:37.611 INFO:teuthology.orchestra.run.smithi176.stdout:5573 write 0x73c1f85 thru 0x73ce01e (0xc09a bytes) 2024-09-14T09:07:37.612 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:07:37.632 INFO:teuthology.orchestra.run.smithi176.stdout:5574 writesame 0x2569da3 thru 0x257825a data_size 0x34(0xe4b8 bytes) 2024-09-14T09:07:37.653 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:37.653 INFO:teuthology.orchestra.run.smithi176.stdout:5577 read 0x652f3d6 thru 0x6530616 (0x1241 bytes) 2024-09-14T09:07:37.653 INFO:teuthology.orchestra.run.smithi176.stdout:5578 write 0x98b917a thru 0x98bd941 (0x47c8 bytes) 2024-09-14T09:07:37.654 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:37.658+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:37.662 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:37.663 INFO:teuthology.orchestra.run.smithi176.stdout:5581 punch from 0x1fac2b9 to 0x1fb0419, (0x4160 bytes) 2024-09-14T09:07:37.681 INFO:teuthology.orchestra.run.smithi176.stdout:5582 trunc from 0xd1c2bcd to 0x42298f6 2024-09-14T09:07:37.704 INFO:teuthology.orchestra.run.smithi176.stdout:5583 write 0x3117106 thru 0x3120626 (0x9521 bytes) 2024-09-14T09:07:37.718 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:37.722+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:37.743 INFO:teuthology.orchestra.run.smithi176.stdout:5584 writesame 0xaf1f4f8 thru 0xaf2d672 data_size 0x47(0xe17b bytes) 2024-09-14T09:07:37.743 INFO:teuthology.orchestra.run.smithi176.stdout:5586 writesame 0x6cf0a38 thru 0x6cfae91 data_size 0x6d(0xa45a bytes) 2024-09-14T09:07:37.758 INFO:teuthology.orchestra.run.smithi176.stdout:5590 punch from 0x1a4c245 to 0x1a4c379, (0x134 bytes) 2024-09-14T09:07:37.758 INFO:teuthology.orchestra.run.smithi176.stdout:5591 read 0x37a53c thru 0x381bec (0x76b1 bytes) 2024-09-14T09:07:37.760 INFO:teuthology.orchestra.run.smithi176.stdout:5592 write 0x1f67edd thru 0x1f68703 (0x827 bytes) 2024-09-14T09:07:37.773 INFO:teuthology.orchestra.run.smithi176.stdout:5594 write 0xaf2abe0 thru 0xaf32f9e (0x83bf bytes) 2024-09-14T09:07:37.787 INFO:teuthology.orchestra.run.smithi176.stdout:5595 punch from 0x2d26920 to 0x2d32030, (0xb710 bytes) 2024-09-14T09:07:37.788 INFO:teuthology.orchestra.run.smithi176.stdout:5596 write 0x633e405 thru 0x6348d48 (0xa944 bytes) 2024-09-14T09:07:37.790 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:37.790 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:07:37.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:07:37.790 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:07:37.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.790 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38593 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [watch reconnect cookie 139718844451680 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:07:37.790 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:07:37.790 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 95.973863301999998, 2024-09-14T09:07:37.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0218296630000001, 2024-09-14T09:07:37.791 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.791 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.791 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.791 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.791 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.791 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.791 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38593 2024-09-14T09:07:37.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.791 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:07:37.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:07:37.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818420+0000", 2024-09-14T09:07:37.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8399999999999999e-06 2024-09-14T09:07:37.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818431+0000", 2024-09-14T09:07:37.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0859999999999999e-05 2024-09-14T09:07:37.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818466+0000", 2024-09-14T09:07:37.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4486999999999997e-05 2024-09-14T09:07:37.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818481+0000", 2024-09-14T09:07:37.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5362999999999998e-05 2024-09-14T09:07:37.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818659+0000", 2024-09-14T09:07:37.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017807500000000001 2024-09-14T09:07:37.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op must wait for map", 2024-09-14T09:07:37.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818681+0000", 2024-09-14T09:07:37.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2096999999999999e-05 2024-09-14T09:07:37.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825472+0000", 2024-09-14T09:07:37.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067905170000000003 2024-09-14T09:07:37.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:37.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825491+0000", 2024-09-14T09:07:37.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9372999999999999e-05 2024-09-14T09:07:37.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835402+0000", 2024-09-14T09:07:37.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0099111620000001 2024-09-14T09:07:37.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835548+0000", 2024-09-14T09:07:37.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014607300000000001 2024-09-14T09:07:37.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.840247+0000", 2024-09-14T09:07:37.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046988159999999998 2024-09-14T09:07:37.796 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.796 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.796 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.796 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38615 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:07:37.796 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:07:37.796 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 95.925498094000005, 2024-09-14T09:07:37.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.97014128899999996, 2024-09-14T09:07:37.796 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.796 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.797 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:37.797 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.797 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.797 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.797 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38615 2024-09-14T09:07:37.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.797 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:07:37.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:07:37.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866787+0000", 2024-09-14T09:07:37.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.476e-06 2024-09-14T09:07:37.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866806+0000", 2024-09-14T09:07:37.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8606000000000001e-05 2024-09-14T09:07:37.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866839+0000", 2024-09-14T09:07:37.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3510000000000003e-05 2024-09-14T09:07:37.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866867+0000", 2024-09-14T09:07:37.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7610000000000002e-05 2024-09-14T09:07:37.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867031+0000", 2024-09-14T09:07:37.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016414199999999999 2024-09-14T09:07:37.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:37.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867054+0000", 2024-09-14T09:07:37.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2693000000000001e-05 2024-09-14T09:07:37.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836462+0000", 2024-09-14T09:07:37.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.96940797700000003 2024-09-14T09:07:37.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:37.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836547+0000", 2024-09-14T09:07:37.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5294e-05 2024-09-14T09:07:37.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836924+0000", 2024-09-14T09:07:37.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037698099999999997 2024-09-14T09:07:37.801 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.801 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.801 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.801 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105723 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:37.801 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:07:37.801 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.697064050000002, 2024-09-14T09:07:37.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77512799700000001, 2024-09-14T09:07:37.801 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.802 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.802 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:37.802 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.802 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.802 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.802 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105723 2024-09-14T09:07:37.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.802 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:07:37.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:07:37.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:07:37.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1399999999999997e-07 2024-09-14T09:07:37.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095220+0000", 2024-09-14T09:07:37.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0510000000000001e-06 2024-09-14T09:07:37.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095228+0000", 2024-09-14T09:07:37.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4159999999999998e-06 2024-09-14T09:07:37.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095234+0000", 2024-09-14T09:07:37.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3010000000000003e-06 2024-09-14T09:07:37.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103902+0000", 2024-09-14T09:07:37.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086678299999999996 2024-09-14T09:07:37.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:37.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103905+0000", 2024-09-14T09:07:37.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5020000000000001e-06 2024-09-14T09:07:37.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870332+0000", 2024-09-14T09:07:37.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76642700900000005 2024-09-14T09:07:37.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870345+0000", 2024-09-14T09:07:37.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2274e-05 2024-09-14T09:07:37.806 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.806 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.806 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.807 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105724 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:37.807 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:07:37.807 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.697039536999998, 2024-09-14T09:07:37.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77515698099999997, 2024-09-14T09:07:37.807 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.807 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.807 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:37.807 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.807 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.808 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.808 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105724 2024-09-14T09:07:37.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.808 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:07:37.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:07:37.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095242+0000", 2024-09-14T09:07:37.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9400000000000001e-07 2024-09-14T09:07:37.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095243+0000", 2024-09-14T09:07:37.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7540000000000001e-06 2024-09-14T09:07:37.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095248+0000", 2024-09-14T09:07:37.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1320000000000004e-06 2024-09-14T09:07:37.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095251+0000", 2024-09-14T09:07:37.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2770000000000001e-06 2024-09-14T09:07:37.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103947+0000", 2024-09-14T09:07:37.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086960089999999993 2024-09-14T09:07:37.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:37.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103955+0000", 2024-09-14T09:07:37.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9519999999999994e-06 2024-09-14T09:07:37.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870386+0000", 2024-09-14T09:07:37.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76643141100000001 2024-09-14T09:07:37.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870398+0000", 2024-09-14T09:07:37.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2052000000000001e-05 2024-09-14T09:07:37.812 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.812 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.812 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.812 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105727 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:37.812 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:07:37.812 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.696999830999999, 2024-09-14T09:07:37.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517292900000001, 2024-09-14T09:07:37.812 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.813 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.813 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:37.813 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.813 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.813 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.813 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105727 2024-09-14T09:07:37.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.813 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:07:37.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:07:37.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:07:37.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3300000000000002e-07 2024-09-14T09:07:37.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095284+0000", 2024-09-14T09:07:37.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9790000000000002e-06 2024-09-14T09:07:37.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095292+0000", 2024-09-14T09:07:37.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3810000000000001e-06 2024-09-14T09:07:37.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095298+0000", 2024-09-14T09:07:37.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9710000000000001e-06 2024-09-14T09:07:37.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103987+0000", 2024-09-14T09:07:37.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086893820000000007 2024-09-14T09:07:37.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:37.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103991+0000", 2024-09-14T09:07:37.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5480000000000002e-06 2024-09-14T09:07:37.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870442+0000", 2024-09-14T09:07:37.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.766451145 2024-09-14T09:07:37.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870454+0000", 2024-09-14T09:07:37.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.199e-05 2024-09-14T09:07:37.816 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.817 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.817 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.817 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105729 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:37.817 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:07:37.817 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.696951473999999, 2024-09-14T09:07:37.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517686500000005, 2024-09-14T09:07:37.817 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.817 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.817 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:37.817 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.818 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.818 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.818 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105729 2024-09-14T09:07:37.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.818 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:07:37.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:07:37.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095330+0000", 2024-09-14T09:07:37.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-09-14T09:07:37.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095332+0000", 2024-09-14T09:07:37.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1560000000000002e-06 2024-09-14T09:07:37.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095338+0000", 2024-09-14T09:07:37.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.592e-06 2024-09-14T09:07:37.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095341+0000", 2024-09-14T09:07:37.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8139999999999998e-06 2024-09-14T09:07:37.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104031+0000", 2024-09-14T09:07:37.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086896330000000004 2024-09-14T09:07:37.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:37.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104039+0000", 2024-09-14T09:07:37.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0900000000000005e-06 2024-09-14T09:07:37.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870495+0000", 2024-09-14T09:07:37.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76645611199999997 2024-09-14T09:07:37.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870506+0000", 2024-09-14T09:07:37.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0986e-05 2024-09-14T09:07:37.822 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.822 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.822 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.822 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105731 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:37.822 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:07:37.822 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.696933207000001, 2024-09-14T09:07:37.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77520598600000001, 2024-09-14T09:07:37.822 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.822 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.822 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:37.822 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.822 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.823 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.823 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105731 2024-09-14T09:07:37.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.823 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:07:37.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:07:37.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:07:37.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8599999999999999e-07 2024-09-14T09:07:37.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095350+0000", 2024-09-14T09:07:37.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1110000000000002e-06 2024-09-14T09:07:37.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095357+0000", 2024-09-14T09:07:37.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9020000000000002e-06 2024-09-14T09:07:37.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095360+0000", 2024-09-14T09:07:37.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1939999999999998e-06 2024-09-14T09:07:37.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104073+0000", 2024-09-14T09:07:37.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087131210000000008 2024-09-14T09:07:37.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:37.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104077+0000", 2024-09-14T09:07:37.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.501e-06 2024-09-14T09:07:37.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870541+0000", 2024-09-14T09:07:37.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76646461700000001 2024-09-14T09:07:37.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870554+0000", 2024-09-14T09:07:37.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2153999999999999e-05 2024-09-14T09:07:37.826 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.826 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.826 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.826 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105733 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:37.827 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:07:37.827 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.696913175999999, 2024-09-14T09:07:37.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.775237277, 2024-09-14T09:07:37.827 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.827 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.827 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:37.827 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.827 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.827 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.827 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105733 2024-09-14T09:07:37.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.827 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:07:37.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:07:37.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:07:37.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8699999999999995e-07 2024-09-14T09:07:37.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095371+0000", 2024-09-14T09:07:37.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5450000000000002e-06 2024-09-14T09:07:37.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095377+0000", 2024-09-14T09:07:37.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.776e-06 2024-09-14T09:07:37.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095383+0000", 2024-09-14T09:07:37.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3129999999999998e-06 2024-09-14T09:07:37.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104124+0000", 2024-09-14T09:07:37.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087409840000000003 2024-09-14T09:07:37.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:37.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104132+0000", 2024-09-14T09:07:37.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3159999999999997e-06 2024-09-14T09:07:37.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870593+0000", 2024-09-14T09:07:37.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76646064800000002 2024-09-14T09:07:37.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870605+0000", 2024-09-14T09:07:37.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2208000000000001e-05 2024-09-14T09:07:37.831 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.831 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.831 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.831 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105734 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:37.831 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:07:37.831 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.696890094, 2024-09-14T09:07:37.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77526569599999995, 2024-09-14T09:07:37.831 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.831 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.831 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:37.831 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.832 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.832 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.832 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105734 2024-09-14T09:07:37.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.832 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:07:37.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:07:37.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:07:37.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7399999999999998e-07 2024-09-14T09:07:37.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095394+0000", 2024-09-14T09:07:37.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8030000000000001e-06 2024-09-14T09:07:37.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095401+0000", 2024-09-14T09:07:37.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.046e-06 2024-09-14T09:07:37.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095407+0000", 2024-09-14T09:07:37.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6060000000000002e-06 2024-09-14T09:07:37.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104167+0000", 2024-09-14T09:07:37.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087602530000000008 2024-09-14T09:07:37.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:37.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104171+0000", 2024-09-14T09:07:37.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7620000000000001e-06 2024-09-14T09:07:37.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870645+0000", 2024-09-14T09:07:37.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76647400600000004 2024-09-14T09:07:37.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870656+0000", 2024-09-14T09:07:37.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1746000000000001e-05 2024-09-14T09:07:37.835 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.836 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.836 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.836 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105738 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:37.836 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:07:37.836 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.696850367, 2024-09-14T09:07:37.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77527530700000002, 2024-09-14T09:07:37.836 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.836 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.836 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:37.836 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.836 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.836 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.837 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105738 2024-09-14T09:07:37.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.837 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:07:37.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:07:37.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095431+0000", 2024-09-14T09:07:37.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0999999999999999e-07 2024-09-14T09:07:37.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095434+0000", 2024-09-14T09:07:37.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.757e-06 2024-09-14T09:07:37.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095442+0000", 2024-09-14T09:07:37.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9929999999999995e-06 2024-09-14T09:07:37.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095448+0000", 2024-09-14T09:07:37.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1340000000000001e-06 2024-09-14T09:07:37.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104213+0000", 2024-09-14T09:07:37.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087650000000000002 2024-09-14T09:07:37.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:37.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104221+0000", 2024-09-14T09:07:37.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4589999999999998e-06 2024-09-14T09:07:37.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870694+0000", 2024-09-14T09:07:37.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76647303200000005 2024-09-14T09:07:37.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870706+0000", 2024-09-14T09:07:37.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1522000000000001e-05 2024-09-14T09:07:37.840 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.840 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.840 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.840 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105739 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:37.841 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:07:37.841 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.696825837999999, 2024-09-14T09:07:37.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77530221700000002, 2024-09-14T09:07:37.841 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.841 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.841 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:37.841 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.841 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.841 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.841 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105739 2024-09-14T09:07:37.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.841 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:07:37.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:07:37.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:07:37.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.15e-07 2024-09-14T09:07:37.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095457+0000", 2024-09-14T09:07:37.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6470000000000001e-06 2024-09-14T09:07:37.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095461+0000", 2024-09-14T09:07:37.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0910000000000003e-06 2024-09-14T09:07:37.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095464+0000", 2024-09-14T09:07:37.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2059999999999999e-06 2024-09-14T09:07:37.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104253+0000", 2024-09-14T09:07:37.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087884120000000007 2024-09-14T09:07:37.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:37.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104256+0000", 2024-09-14T09:07:37.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5439999999999999e-06 2024-09-14T09:07:37.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870745+0000", 2024-09-14T09:07:37.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76648908000000004 2024-09-14T09:07:37.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870757+0000", 2024-09-14T09:07:37.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1822e-05 2024-09-14T09:07:37.845 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.845 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.845 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.845 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105741 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:37.845 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:07:37.845 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.696756711999999, 2024-09-14T09:07:37.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77528469899999997, 2024-09-14T09:07:37.846 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.846 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.846 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:37.846 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.846 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.846 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.846 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105741 2024-09-14T09:07:37.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.846 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:07:37.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:07:37.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095525+0000", 2024-09-14T09:07:37.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3740000000000001e-06 2024-09-14T09:07:37.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095533+0000", 2024-09-14T09:07:37.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6380000000000004e-06 2024-09-14T09:07:37.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095549+0000", 2024-09-14T09:07:37.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5554999999999999e-05 2024-09-14T09:07:37.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095566+0000", 2024-09-14T09:07:37.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7146000000000002e-05 2024-09-14T09:07:37.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104297+0000", 2024-09-14T09:07:37.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087315750000000001 2024-09-14T09:07:37.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:37.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104305+0000", 2024-09-14T09:07:37.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1389999999999995e-06 2024-09-14T09:07:37.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870797+0000", 2024-09-14T09:07:37.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76649129599999999 2024-09-14T09:07:37.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870809+0000", 2024-09-14T09:07:37.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1976e-05 2024-09-14T09:07:37.850 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.850 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.850 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.850 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105742 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:37.850 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:07:37.850 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.696691928, 2024-09-14T09:07:37.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.775271873, 2024-09-14T09:07:37.851 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.851 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.851 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:37.851 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.851 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.851 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.851 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105742 2024-09-14T09:07:37.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.851 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:07:37.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:07:37.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095590+0000", 2024-09-14T09:07:37.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0400000000000005e-07 2024-09-14T09:07:37.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095594+0000", 2024-09-14T09:07:37.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5639999999999998e-06 2024-09-14T09:07:37.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095620+0000", 2024-09-14T09:07:37.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5494999999999999e-05 2024-09-14T09:07:37.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095639+0000", 2024-09-14T09:07:37.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9639999999999998e-05 2024-09-14T09:07:37.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104340+0000", 2024-09-14T09:07:37.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087004950000000008 2024-09-14T09:07:37.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:37.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104343+0000", 2024-09-14T09:07:37.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4020000000000002e-06 2024-09-14T09:07:37.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870850+0000", 2024-09-14T09:07:37.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76650648499999996 2024-09-14T09:07:37.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870861+0000", 2024-09-14T09:07:37.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0888e-05 2024-09-14T09:07:37.854 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.855 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.855 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.855 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105745 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:37.855 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:07:37.855 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.696593313000001, 2024-09-14T09:07:37.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77521998700000005, 2024-09-14T09:07:37.855 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.855 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.855 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:37.855 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.855 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.855 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.855 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105745 2024-09-14T09:07:37.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.856 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:07:37.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:07:37.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095688+0000", 2024-09-14T09:07:37.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2699999999999998e-07 2024-09-14T09:07:37.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095693+0000", 2024-09-14T09:07:37.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0479999999999998e-06 2024-09-14T09:07:37.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095707+0000", 2024-09-14T09:07:37.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.342e-05 2024-09-14T09:07:37.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095716+0000", 2024-09-14T09:07:37.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9800000000000004e-06 2024-09-14T09:07:37.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104388+0000", 2024-09-14T09:07:37.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086717349999999999 2024-09-14T09:07:37.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:37.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104396+0000", 2024-09-14T09:07:37.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1769999999999998e-06 2024-09-14T09:07:37.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870897+0000", 2024-09-14T09:07:37.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76650102600000003 2024-09-14T09:07:37.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870907+0000", 2024-09-14T09:07:37.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0674e-05 2024-09-14T09:07:37.859 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.859 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.859 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.859 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105747 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:37.859 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:07:37.859 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.696547093, 2024-09-14T09:07:37.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77522190300000005, 2024-09-14T09:07:37.860 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.860 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.860 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:37.860 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.860 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.860 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.860 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105747 2024-09-14T09:07:37.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.860 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:07:37.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:07:37.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:07:37.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8999999999999995e-07 2024-09-14T09:07:37.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095738+0000", 2024-09-14T09:07:37.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8430000000000003e-06 2024-09-14T09:07:37.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095750+0000", 2024-09-14T09:07:37.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1229999999999999e-05 2024-09-14T09:07:37.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095758+0000", 2024-09-14T09:07:37.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.337e-06 2024-09-14T09:07:37.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104431+0000", 2024-09-14T09:07:37.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086728780000000002 2024-09-14T09:07:37.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:37.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104434+0000", 2024-09-14T09:07:37.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5049999999999999e-06 2024-09-14T09:07:37.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870945+0000", 2024-09-14T09:07:37.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651031800000002 2024-09-14T09:07:37.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870956+0000", 2024-09-14T09:07:37.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1002000000000001e-05 2024-09-14T09:07:37.864 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.864 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.864 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.864 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105750 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:37.864 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:07:37.864 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.696477147, 2024-09-14T09:07:37.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77519961800000003, 2024-09-14T09:07:37.864 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.864 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.864 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:37.864 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.864 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.865 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.865 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105750 2024-09-14T09:07:37.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.865 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:07:37.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:07:37.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:07:37.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3600000000000002e-07 2024-09-14T09:07:37.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095809+0000", 2024-09-14T09:07:37.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4000000000000002e-06 2024-09-14T09:07:37.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095822+0000", 2024-09-14T09:07:37.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2704000000000001e-05 2024-09-14T09:07:37.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095831+0000", 2024-09-14T09:07:37.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9530000000000005e-06 2024-09-14T09:07:37.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104477+0000", 2024-09-14T09:07:37.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086459989999999997 2024-09-14T09:07:37.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:37.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104488+0000", 2024-09-14T09:07:37.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:07:37.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870992+0000", 2024-09-14T09:07:37.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.766504097 2024-09-14T09:07:37.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871003+0000", 2024-09-14T09:07:37.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1358999999999999e-05 2024-09-14T09:07:37.868 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.868 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.869 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.869 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105751 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:37.869 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:07:37.869 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.696419261999999, 2024-09-14T09:07:37.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77518966700000003, 2024-09-14T09:07:37.869 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.869 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.869 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:37.869 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.869 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.869 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.870 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105751 2024-09-14T09:07:37.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.870 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:07:37.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:07:37.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:07:37.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7300000000000005e-07 2024-09-14T09:07:37.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095867+0000", 2024-09-14T09:07:37.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2729999999999996e-06 2024-09-14T09:07:37.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095879+0000", 2024-09-14T09:07:37.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2917e-05 2024-09-14T09:07:37.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095888+0000", 2024-09-14T09:07:37.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2609999999999993e-06 2024-09-14T09:07:37.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104520+0000", 2024-09-14T09:07:37.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086322970000000006 2024-09-14T09:07:37.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:37.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104524+0000", 2024-09-14T09:07:37.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.54e-06 2024-09-14T09:07:37.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871040+0000", 2024-09-14T09:07:37.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651615399999995 2024-09-14T09:07:37.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871051+0000", 2024-09-14T09:07:37.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1452e-05 2024-09-14T09:07:37.873 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.873 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.873 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.874 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105752 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:37.874 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:07:37.874 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.696358497999999, 2024-09-14T09:07:37.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517684899999995, 2024-09-14T09:07:37.874 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.874 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.874 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:37.874 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.874 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.874 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.874 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105752 2024-09-14T09:07:37.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.874 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:07:37.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:07:37.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095923+0000", 2024-09-14T09:07:37.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3300000000000001e-07 2024-09-14T09:07:37.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095927+0000", 2024-09-14T09:07:37.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:07:37.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095940+0000", 2024-09-14T09:07:37.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2309e-05 2024-09-14T09:07:37.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095948+0000", 2024-09-14T09:07:37.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9199999999999993e-06 2024-09-14T09:07:37.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104566+0000", 2024-09-14T09:07:37.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086171449999999997 2024-09-14T09:07:37.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:37.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104574+0000", 2024-09-14T09:07:37.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0690000000000002e-06 2024-09-14T09:07:37.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871087+0000", 2024-09-14T09:07:37.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651370699999999 2024-09-14T09:07:37.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871099+0000", 2024-09-14T09:07:37.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1718e-05 2024-09-14T09:07:37.878 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.878 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.878 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.879 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105755 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:37.879 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:07:37.879 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.696300555000001, 2024-09-14T09:07:37.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77516787899999995, 2024-09-14T09:07:37.879 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.879 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.879 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:37.879 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.879 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.879 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.879 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105755 2024-09-14T09:07:37.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.879 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:07:37.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:07:37.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095981+0000", 2024-09-14T09:07:37.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2699999999999998e-07 2024-09-14T09:07:37.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095985+0000", 2024-09-14T09:07:37.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4349999999999999e-06 2024-09-14T09:07:37.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095999+0000", 2024-09-14T09:07:37.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3139000000000001e-05 2024-09-14T09:07:37.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096007+0000", 2024-09-14T09:07:37.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6610000000000006e-06 2024-09-14T09:07:37.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104591+0000", 2024-09-14T09:07:37.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0085841709999999998 2024-09-14T09:07:37.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:37.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104595+0000", 2024-09-14T09:07:37.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5590000000000001e-06 2024-09-14T09:07:37.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871137+0000", 2024-09-14T09:07:37.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76654202100000002 2024-09-14T09:07:37.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871148+0000", 2024-09-14T09:07:37.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1066e-05 2024-09-14T09:07:37.883 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.883 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.883 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.883 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105757 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:37.883 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:07:37.883 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 16.696241975, 2024-09-14T09:07:37.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77516401300000004, 2024-09-14T09:07:37.883 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:37.883 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:37.883 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:37.883 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:37.884 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:37.884 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:37.884 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105757 2024-09-14T09:07:37.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.884 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:37.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:37.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:07:37.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:37.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:07:37.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:37.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:37.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856228+0000", 2024-09-14T09:07:37.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76018875100000005 2024-09-14T09:07:37.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:37.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856240+0000", 2024-09-14T09:07:37.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-05 2024-09-14T09:07:37.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:37.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856268+0000", 2024-09-14T09:07:37.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7628999999999999e-05 2024-09-14T09:07:37.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:37.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856307+0000", 2024-09-14T09:07:37.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8967000000000001e-05 2024-09-14T09:07:37.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:37.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871192+0000", 2024-09-14T09:07:37.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.01488449 2024-09-14T09:07:37.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:37.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:37.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:37.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871203+0000", 2024-09-14T09:07:37.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1266e-05 2024-09-14T09:07:37.887 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.887 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.887 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.887 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:37.887 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:37.887 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:37.889 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:07:37.891 INFO:teuthology.orchestra.run.smithi176.stdout:5597 write 0x2121efb thru 0x2129e69 (0x7f6f bytes) 2024-09-14T09:07:37.891 INFO:teuthology.orchestra.run.smithi176.stdout:5600 read 0x2c7b51f thru 0x2c82c26 (0x7708 bytes) 2024-09-14T09:07:37.891 INFO:teuthology.orchestra.run.smithi176.stdout:5602 read 0x5616545 thru 0x5618770 (0x222c bytes) 2024-09-14T09:07:37.891 INFO:teuthology.orchestra.run.smithi176.stdout:5603 write 0x27ef586 thru 0x27fe03e (0xeab9 bytes) 2024-09-14T09:07:37.891 INFO:teuthology.orchestra.run.smithi176.stdout:5604 write 0x5071aed thru 0x507cb51 (0xb065 bytes) 2024-09-14T09:07:37.891 INFO:teuthology.orchestra.run.smithi176.stdout:5605 punch from 0x1a23ccc to 0x1a2cf66, (0x929a bytes) 2024-09-14T09:07:37.891 INFO:teuthology.orchestra.run.smithi176.stdout:5606 punch from 0x397ee05 to 0x39820ba, (0x32b5 bytes) 2024-09-14T09:07:37.891 INFO:teuthology.orchestra.run.smithi176.stdout:5607 write 0x63a9de thru 0x644dc6 (0xa3e9 bytes) 2024-09-14T09:07:37.891 INFO:teuthology.orchestra.run.smithi176.stdout:5610 writesame 0xc2b1d6f thru 0xc2b5cd8 data_size 0x1(0x3f6a bytes) 2024-09-14T09:07:37.891 INFO:teuthology.orchestra.run.smithi176.stdout:5613 trunc from 0xc2b5cd9 to 0xc3384a7 2024-09-14T09:07:37.891 INFO:teuthology.orchestra.run.smithi176.stdout:5617 read 0xafcbf56 thru 0xafd3640 (0x76eb bytes) 2024-09-14T09:07:37.891 INFO:teuthology.orchestra.run.smithi176.stdout:5618 read 0x4161b13 thru 0x4167adf (0x5fcd bytes) 2024-09-14T09:07:37.891 INFO:teuthology.orchestra.run.smithi176.stdout:5621 read 0xa85dd96 thru 0xa85ffd2 (0x223d bytes) 2024-09-14T09:07:37.891 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:37.892 INFO:teuthology.orchestra.run.smithi176.stdout:5625 write 0x2c878c8 thru 0x2c87a56 (0x18f bytes) 2024-09-14T09:07:37.892 INFO:teuthology.orchestra.run.smithi176.stdout:5627 read 0xb1c4f76 thru 0xb1d1dba (0xce45 bytes) 2024-09-14T09:07:37.892 INFO:teuthology.orchestra.run.smithi176.stdout:5629 read 0x9bb3970 thru 0x9bb86c6 (0x4d57 bytes) 2024-09-14T09:07:37.892 INFO:teuthology.orchestra.run.smithi176.stdout:5630 trunc from 0xc3384a7 to 0x8666e2d 2024-09-14T09:07:37.892 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:37.887+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:37.929 INFO:teuthology.orchestra.run.smithi176.stdout:5631 write 0xcf39911 thru 0xcf450a6 (0xb796 bytes) 2024-09-14T09:07:37.949 INFO:teuthology.orchestra.run.smithi176.stdout:5633 read 0x1b25a58 thru 0x1b2d02f (0x75d8 bytes) 2024-09-14T09:07:37.950 INFO:teuthology.orchestra.run.smithi176.stdout:5634 write 0x979c450 thru 0x979e531 (0x20e2 bytes) 2024-09-14T09:07:37.960 INFO:teuthology.orchestra.run.smithi176.stdout:5635 trunc from 0xcf450a7 to 0xb1c5ebe 2024-09-14T09:07:37.979 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:37.979 INFO:teuthology.orchestra.run.smithi176.stdout:5637 read 0x83645b6 thru 0x83679ca (0x3415 bytes) 2024-09-14T09:07:37.979 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:37.979 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:37.984 INFO:teuthology.orchestra.run.smithi176.stdout:5640 trunc from 0xb1c5ebe to 0xe6eba4f 2024-09-14T09:07:37.989 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:37.987+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:37.992 INFO:teuthology.orchestra.run.smithi176.stdout:5641 write 0x8f3726c thru 0x8f3ff8c (0x8d21 bytes) 2024-09-14T09:07:38.005 INFO:teuthology.orchestra.run.smithi176.stdout:5643 write 0xa386775 thru 0xa388ddd (0x2669 bytes) 2024-09-14T09:07:38.013 INFO:teuthology.orchestra.run.smithi176.stdout:5644 read 0x15ba52a thru 0x15bd2ed (0x2dc4 bytes) 2024-09-14T09:07:38.019 INFO:teuthology.orchestra.run.smithi176.stdout:5645 trunc from 0xe6eba4f to 0x9ab983d 2024-09-14T09:07:38.033 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop(client.4446.0:130044 3.3 e128/86 3:c3353b4d:::rbd_data.115e2baf3d9a.0000000000000135:head v 128'538, mlcod=128'536)", 2024-09-14T09:07:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:38.034145+0000", 2024-09-14T09:07:38.033 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.00094171999999999995, 2024-09-14T09:07:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00097812500000000004, 2024-09-14T09:07:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:38.034145+0000", 2024-09-14T09:07:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:38.034145+0000", 2024-09-14T09:07:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:38.034146+0000", 2024-09-14T09:07:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3239999999999999e-06 2024-09-14T09:07:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:38.034154+0000", 2024-09-14T09:07:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8029999999999997e-06 2024-09-14T09:07:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:38.034170+0000", 2024-09-14T09:07:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6793999999999999e-05 2024-09-14T09:07:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:38.034178+0000", 2024-09-14T09:07:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0620000000000003e-06 2024-09-14T09:07:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:38.034261+0000", 2024-09-14T09:07:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3030999999999998e-05 2024-09-14T09:07:38.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:38.034300+0000", 2024-09-14T09:07:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9004999999999997e-05 2024-09-14T09:07:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:38.037 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 1 2024-09-14T09:07:38.037 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:38.050 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:07:38.054 INFO:teuthology.orchestra.run.smithi176.stdout:5646 trunc from 0x9ab983d to 0x2477aa2 2024-09-14T09:07:38.090 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:38.094+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:38.182 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:38.183 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:38.183 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:38.183 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:38.183 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:38.192 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:38.194+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:38.196 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:07:38.292 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:38.294+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:38.321 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:38.321 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:38.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:38.322 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:38.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.322 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:71052 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:07:38.322 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:07:38.322 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.582737034999994, 2024-09-14T09:07:38.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14854635599999999, 2024-09-14T09:07:38.322 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.322 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.322 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:38.322 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.322 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.322 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.322 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 71052 2024-09-14T09:07:38.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.323 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:07:38.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:07:38.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737570+0000", 2024-09-14T09:07:38.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4770000000000002e-06 2024-09-14T09:07:38.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.324 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.324 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737584+0000", 2024-09-14T09:07:38.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3457e-05 2024-09-14T09:07:38.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737608+0000", 2024-09-14T09:07:38.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3893999999999999e-05 2024-09-14T09:07:38.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737620+0000", 2024-09-14T09:07:38.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2157000000000001e-05 2024-09-14T09:07:38.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737738+0000", 2024-09-14T09:07:38.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118409 2024-09-14T09:07:38.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737892+0000", 2024-09-14T09:07:38.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154222 2024-09-14T09:07:38.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:07:38.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.749622+0000", 2024-09-14T09:07:38.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011729537 2024-09-14T09:07:38.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:38.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885934+0000", 2024-09-14T09:07:38.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13631222200000001 2024-09-14T09:07:38.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:38.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885996+0000", 2024-09-14T09:07:38.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1555000000000003e-05 2024-09-14T09:07:38.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:38.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886060+0000", 2024-09-14T09:07:38.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3980999999999996e-05 2024-09-14T09:07:38.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886114+0000", 2024-09-14T09:07:38.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4444999999999998e-05 2024-09-14T09:07:38.327 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.327 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.328 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.328 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:71052 3.1 e71/34 ondisk, result = 0)", 2024-09-14T09:07:38.328 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:07:38.328 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.539833657000003, 2024-09-14T09:07:38.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10565334799999999, 2024-09-14T09:07:38.328 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.328 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.328 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.328 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:07:38.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:07:38.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780473+0000", 2024-09-14T09:07:38.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8980000000000001e-06 2024-09-14T09:07:38.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780482+0000", 2024-09-14T09:07:38.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9080000000000001e-06 2024-09-14T09:07:38.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780511+0000", 2024-09-14T09:07:38.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9490999999999999e-05 2024-09-14T09:07:38.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780527+0000", 2024-09-14T09:07:38.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.508e-05 2024-09-14T09:07:38.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885978+0000", 2024-09-14T09:07:38.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.105451328 2024-09-14T09:07:38.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885989+0000", 2024-09-14T09:07:38.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0875e-05 2024-09-14T09:07:38.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886125+0000", 2024-09-14T09:07:38.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013576800000000001 2024-09-14T09:07:38.331 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.331 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.331 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.332 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:98007 3.12 3:4faf1f2a:::rbd_data.115e113dc8ca.000000000000004f:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:07:38.332 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:07:38.332 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 25.462861433, 2024-09-14T09:07:38.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.098233707000000003, 2024-09-14T09:07:38.332 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.332 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.332 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:38.332 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.332 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.332 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.332 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 98007 2024-09-14T09:07:38.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.332 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:07:38.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:07:38.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857446+0000", 2024-09-14T09:07:38.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7599999999999998e-06 2024-09-14T09:07:38.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857459+0000", 2024-09-14T09:07:38.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3123e-05 2024-09-14T09:07:38.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857491+0000", 2024-09-14T09:07:38.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1603000000000002e-05 2024-09-14T09:07:38.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857506+0000", 2024-09-14T09:07:38.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.537e-05 2024-09-14T09:07:38.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857694+0000", 2024-09-14T09:07:38.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018766599999999999 2024-09-14T09:07:38.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857955+0000", 2024-09-14T09:07:38.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000260979 2024-09-14T09:07:38.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:07:38.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.863125+0000", 2024-09-14T09:07:38.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.005170582 2024-09-14T09:07:38.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:38.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955268+0000", 2024-09-14T09:07:38.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092142659000000002 2024-09-14T09:07:38.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:38.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955385+0000", 2024-09-14T09:07:38.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116723 2024-09-14T09:07:38.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:38.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955553+0000", 2024-09-14T09:07:38.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016809299999999999 2024-09-14T09:07:38.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955677+0000", 2024-09-14T09:07:38.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012414899999999999 2024-09-14T09:07:38.338 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.338 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.338 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.338 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118153 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 445440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:38.338 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:07:38.338 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.4823040580000004, 2024-09-14T09:07:38.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095358268999999996, 2024-09-14T09:07:38.338 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.339 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.339 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.339 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.339 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.339 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.339 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118153 2024-09-14T09:07:38.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.339 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:07:38.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:07:38.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:07:38.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0999999999999999e-07 2024-09-14T09:07:38.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838003+0000", 2024-09-14T09:07:38.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2000000000000001e-06 2024-09-14T09:07:38.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838009+0000", 2024-09-14T09:07:38.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.105e-06 2024-09-14T09:07:38.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838015+0000", 2024-09-14T09:07:38.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9030000000000001e-06 2024-09-14T09:07:38.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933049+0000", 2024-09-14T09:07:38.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095033513999999999 2024-09-14T09:07:38.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933113+0000", 2024-09-14T09:07:38.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3616999999999998e-05 2024-09-14T09:07:38.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933359+0000", 2024-09-14T09:07:38.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024652000000000002 2024-09-14T09:07:38.342 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.342 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.342 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.343 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118155 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 449536~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:38.343 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:07:38.343 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.4822815330000001, 2024-09-14T09:07:38.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095668000000000003, 2024-09-14T09:07:38.343 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.343 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.343 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.343 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.343 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118155 2024-09-14T09:07:38.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.343 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:07:38.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:07:38.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838024+0000", 2024-09-14T09:07:38.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0400000000000002e-07 2024-09-14T09:07:38.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838026+0000", 2024-09-14T09:07:38.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.508e-06 2024-09-14T09:07:38.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838033+0000", 2024-09-14T09:07:38.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.392e-06 2024-09-14T09:07:38.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838038+0000", 2024-09-14T09:07:38.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1050000000000003e-06 2024-09-14T09:07:38.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933400+0000", 2024-09-14T09:07:38.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095362279999999994 2024-09-14T09:07:38.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933451+0000", 2024-09-14T09:07:38.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0927000000000002e-05 2024-09-14T09:07:38.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933691+0000", 2024-09-14T09:07:38.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024038400000000001 2024-09-14T09:07:38.347 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.347 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.347 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.347 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118158 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 453632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:38.347 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:07:38.348 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.4822605109999998, 2024-09-14T09:07:38.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096024231000000002, 2024-09-14T09:07:38.348 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.348 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.348 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.348 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.348 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.348 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.348 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118158 2024-09-14T09:07:38.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.348 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:07:38.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:07:38.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838045+0000", 2024-09-14T09:07:38.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2199999999999999e-07 2024-09-14T09:07:38.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838047+0000", 2024-09-14T09:07:38.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1059999999999998e-06 2024-09-14T09:07:38.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838054+0000", 2024-09-14T09:07:38.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4089999999999997e-06 2024-09-14T09:07:38.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838059+0000", 2024-09-14T09:07:38.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8590000000000004e-06 2024-09-14T09:07:38.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933729+0000", 2024-09-14T09:07:38.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095670274999999999 2024-09-14T09:07:38.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933794+0000", 2024-09-14T09:07:38.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4846999999999998e-05 2024-09-14T09:07:38.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934069+0000", 2024-09-14T09:07:38.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027431299999999999 2024-09-14T09:07:38.352 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.352 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.352 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.352 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118160 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 457728~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:38.352 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:07:38.353 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.4822383910000001, 2024-09-14T09:07:38.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096394812999999996, 2024-09-14T09:07:38.353 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.353 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.353 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.353 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.354 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.354 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.354 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118160 2024-09-14T09:07:38.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.354 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:07:38.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:07:38.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838067+0000", 2024-09-14T09:07:38.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9399999999999995e-07 2024-09-14T09:07:38.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838069+0000", 2024-09-14T09:07:38.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1579999999999999e-06 2024-09-14T09:07:38.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838077+0000", 2024-09-14T09:07:38.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6939999999999995e-06 2024-09-14T09:07:38.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838082+0000", 2024-09-14T09:07:38.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2290000000000002e-06 2024-09-14T09:07:38.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934106+0000", 2024-09-14T09:07:38.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096023874999999995 2024-09-14T09:07:38.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934158+0000", 2024-09-14T09:07:38.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2052000000000002e-05 2024-09-14T09:07:38.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934461+0000", 2024-09-14T09:07:38.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030331100000000001 2024-09-14T09:07:38.361 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.361 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.361 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.361 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118162 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 461824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:38.361 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:07:38.361 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.4822158109999997, 2024-09-14T09:07:38.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096615294000000004, 2024-09-14T09:07:38.361 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.361 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.361 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.361 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.362 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.362 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.362 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118162 2024-09-14T09:07:38.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.362 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:07:38.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:07:38.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838090+0000", 2024-09-14T09:07:38.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9699999999999996e-07 2024-09-14T09:07:38.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838092+0000", 2024-09-14T09:07:38.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1119999999999999e-06 2024-09-14T09:07:38.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838098+0000", 2024-09-14T09:07:38.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4359999999999999e-06 2024-09-14T09:07:38.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838103+0000", 2024-09-14T09:07:38.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1200000000000001e-06 2024-09-14T09:07:38.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934504+0000", 2024-09-14T09:07:38.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096400885000000006 2024-09-14T09:07:38.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934596+0000", 2024-09-14T09:07:38.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1775999999999997e-05 2024-09-14T09:07:38.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934704+0000", 2024-09-14T09:07:38.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010846800000000001 2024-09-14T09:07:38.365 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.365 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.365 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.365 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118164 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 465920~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:38.365 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:07:38.366 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.4821945510000001, 2024-09-14T09:07:38.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096951372999999993, 2024-09-14T09:07:38.366 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.366 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.366 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.366 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.366 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118164 2024-09-14T09:07:38.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.366 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:07:38.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:07:38.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838111+0000", 2024-09-14T09:07:38.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0100000000000005e-07 2024-09-14T09:07:38.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838113+0000", 2024-09-14T09:07:38.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0339999999999999e-06 2024-09-14T09:07:38.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838119+0000", 2024-09-14T09:07:38.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.409e-06 2024-09-14T09:07:38.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838124+0000", 2024-09-14T09:07:38.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9030000000000003e-06 2024-09-14T09:07:38.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934739+0000", 2024-09-14T09:07:38.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096615124999999996 2024-09-14T09:07:38.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934789+0000", 2024-09-14T09:07:38.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9554e-05 2024-09-14T09:07:38.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935062+0000", 2024-09-14T09:07:38.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000272847 2024-09-14T09:07:38.369 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.370 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.370 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.370 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118166 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 470016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:38.370 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:07:38.370 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.4821737969999997, 2024-09-14T09:07:38.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097306410999999995, 2024-09-14T09:07:38.370 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.370 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.370 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.370 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.371 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118166 2024-09-14T09:07:38.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.371 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:07:38.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:07:38.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:07:38.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.03e-07 2024-09-14T09:07:38.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838133+0000", 2024-09-14T09:07:38.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9479999999999998e-06 2024-09-14T09:07:38.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838141+0000", 2024-09-14T09:07:38.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5739999999999998e-06 2024-09-14T09:07:38.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838146+0000", 2024-09-14T09:07:38.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3750000000000002e-06 2024-09-14T09:07:38.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935095+0000", 2024-09-14T09:07:38.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096948435999999999 2024-09-14T09:07:38.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935152+0000", 2024-09-14T09:07:38.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.694e-05 2024-09-14T09:07:38.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935437+0000", 2024-09-14T09:07:38.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028573500000000001 2024-09-14T09:07:38.374 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.374 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.374 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.374 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118168 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 474112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:38.374 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:07:38.374 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.4821515759999997, 2024-09-14T09:07:38.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097658024999999996, 2024-09-14T09:07:38.374 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.375 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.375 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.375 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.375 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.375 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.375 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118168 2024-09-14T09:07:38.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.375 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:07:38.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:07:38.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838154+0000", 2024-09-14T09:07:38.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4200000000000001e-07 2024-09-14T09:07:38.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838160+0000", 2024-09-14T09:07:38.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8510000000000004e-06 2024-09-14T09:07:38.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838168+0000", 2024-09-14T09:07:38.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9899999999999997e-06 2024-09-14T09:07:38.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838172+0000", 2024-09-14T09:07:38.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.887e-06 2024-09-14T09:07:38.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935473+0000", 2024-09-14T09:07:38.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097300132999999997 2024-09-14T09:07:38.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935523+0000", 2024-09-14T09:07:38.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0862000000000002e-05 2024-09-14T09:07:38.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935811+0000", 2024-09-14T09:07:38.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028786000000000003 2024-09-14T09:07:38.379 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.379 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.379 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.379 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118170 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 478208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:38.379 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:07:38.379 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.482124765, 2024-09-14T09:07:38.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.098077529999999996, 2024-09-14T09:07:38.379 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.380 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.380 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.380 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118170 2024-09-14T09:07:38.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.380 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:07:38.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:07:38.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842901+0000", 2024-09-14T09:07:38.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004721416 2024-09-14T09:07:38.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842906+0000", 2024-09-14T09:07:38.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7199999999999997e-06 2024-09-14T09:07:38.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842917+0000", 2024-09-14T09:07:38.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0671e-05 2024-09-14T09:07:38.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842923+0000", 2024-09-14T09:07:38.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.088e-06 2024-09-14T09:07:38.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935857+0000", 2024-09-14T09:07:38.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092933624000000006 2024-09-14T09:07:38.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935958+0000", 2024-09-14T09:07:38.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100933 2024-09-14T09:07:38.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.936258+0000", 2024-09-14T09:07:38.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030007800000000001 2024-09-14T09:07:38.383 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.383 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.383 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.384 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118181 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 498688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:38.384 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:07:38.384 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.4773184959999996, 2024-09-14T09:07:38.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095212643999999999, 2024-09-14T09:07:38.384 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.384 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.384 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.384 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.384 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.384 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.384 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118181 2024-09-14T09:07:38.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.385 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:07:38.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:07:38.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842987+0000", 2024-09-14T09:07:38.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.29e-07 2024-09-14T09:07:38.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842988+0000", 2024-09-14T09:07:38.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.229e-06 2024-09-14T09:07:38.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842993+0000", 2024-09-14T09:07:38.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.792e-06 2024-09-14T09:07:38.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842996+0000", 2024-09-14T09:07:38.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3349999999999998e-06 2024-09-14T09:07:38.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.937807+0000", 2024-09-14T09:07:38.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.094811585000000004 2024-09-14T09:07:38.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.937899+0000", 2024-09-14T09:07:38.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1719999999999996e-05 2024-09-14T09:07:38.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938199+0000", 2024-09-14T09:07:38.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000299754 2024-09-14T09:07:38.388 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.388 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.389 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.389 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118182 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 502784~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:38.389 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:07:38.389 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.4773053190000001, 2024-09-14T09:07:38.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095557953000000001, 2024-09-14T09:07:38.389 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.389 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.389 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.389 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118182 2024-09-14T09:07:38.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.390 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:07:38.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:07:38.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:07:38.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2999999999999999e-07 2024-09-14T09:07:38.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843001+0000", 2024-09-14T09:07:38.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.652e-06 2024-09-14T09:07:38.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843005+0000", 2024-09-14T09:07:38.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2780000000000002e-06 2024-09-14T09:07:38.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843007+0000", 2024-09-14T09:07:38.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.553e-06 2024-09-14T09:07:38.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938240+0000", 2024-09-14T09:07:38.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095232864 2024-09-14T09:07:38.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938295+0000", 2024-09-14T09:07:38.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5095e-05 2024-09-14T09:07:38.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938557+0000", 2024-09-14T09:07:38.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026228100000000001 2024-09-14T09:07:38.393 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.393 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.393 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.393 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118184 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 506880~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:38.393 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:07:38.393 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.4772941189999997, 2024-09-14T09:07:38.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095950246000000003, 2024-09-14T09:07:38.394 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.394 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.394 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.394 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118184 2024-09-14T09:07:38.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.394 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:07:38.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:07:38.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:07:38.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2399999999999999e-07 2024-09-14T09:07:38.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843012+0000", 2024-09-14T09:07:38.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.269e-06 2024-09-14T09:07:38.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843015+0000", 2024-09-14T09:07:38.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0469999999999998e-06 2024-09-14T09:07:38.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843018+0000", 2024-09-14T09:07:38.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.044e-06 2024-09-14T09:07:38.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938597+0000", 2024-09-14T09:07:38.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095578688999999994 2024-09-14T09:07:38.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938687+0000", 2024-09-14T09:07:38.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0284000000000005e-05 2024-09-14T09:07:38.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938961+0000", 2024-09-14T09:07:38.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027368899999999997 2024-09-14T09:07:38.402 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.402 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.402 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.402 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118186 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 510976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:38.402 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:07:38.402 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.4772831589999997, 2024-09-14T09:07:38.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096286497999999998, 2024-09-14T09:07:38.403 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.403 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.403 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.403 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.403 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.403 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.403 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118186 2024-09-14T09:07:38.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.403 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:07:38.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:07:38.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:07:38.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.35e-07 2024-09-14T09:07:38.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843023+0000", 2024-09-14T09:07:38.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.195e-06 2024-09-14T09:07:38.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843026+0000", 2024-09-14T09:07:38.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.055e-06 2024-09-14T09:07:38.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843029+0000", 2024-09-14T09:07:38.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6539999999999999e-06 2024-09-14T09:07:38.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939000+0000", 2024-09-14T09:07:38.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095971476 2024-09-14T09:07:38.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939049+0000", 2024-09-14T09:07:38.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8986999999999997e-05 2024-09-14T09:07:38.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939308+0000", 2024-09-14T09:07:38.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000258896 2024-09-14T09:07:38.406 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.406 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.407 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.407 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118188 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 515072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:38.407 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:07:38.407 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.4772727699999999, 2024-09-14T09:07:38.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096722717999999999, 2024-09-14T09:07:38.407 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.407 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.407 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.407 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.407 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.407 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.407 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118188 2024-09-14T09:07:38.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.408 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:07:38.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:07:38.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:07:38.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2999999999999999e-07 2024-09-14T09:07:38.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843034+0000", 2024-09-14T09:07:38.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T09:07:38.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843037+0000", 2024-09-14T09:07:38.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0989999999999999e-06 2024-09-14T09:07:38.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843039+0000", 2024-09-14T09:07:38.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.666e-06 2024-09-14T09:07:38.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939351+0000", 2024-09-14T09:07:38.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096311789999999994 2024-09-14T09:07:38.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939440+0000", 2024-09-14T09:07:38.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8843000000000003e-05 2024-09-14T09:07:38.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939755+0000", 2024-09-14T09:07:38.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031486399999999999 2024-09-14T09:07:38.412 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.412 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.412 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.413 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118191 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 519168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:38.413 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:07:38.413 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.4772620769999998, 2024-09-14T09:07:38.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097080607999999999, 2024-09-14T09:07:38.413 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.413 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.413 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.413 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.413 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.413 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.413 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118191 2024-09-14T09:07:38.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.414 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:07:38.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:07:38.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:07:38.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2100000000000001e-07 2024-09-14T09:07:38.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843044+0000", 2024-09-14T09:07:38.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2249999999999999e-06 2024-09-14T09:07:38.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843047+0000", 2024-09-14T09:07:38.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0240000000000002e-06 2024-09-14T09:07:38.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843050+0000", 2024-09-14T09:07:38.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.508e-06 2024-09-14T09:07:38.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939791+0000", 2024-09-14T09:07:38.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096741399000000006 2024-09-14T09:07:38.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939849+0000", 2024-09-14T09:07:38.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7912999999999999e-05 2024-09-14T09:07:38.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940123+0000", 2024-09-14T09:07:38.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027431800000000001 2024-09-14T09:07:38.417 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.417 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.417 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.417 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118192 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 523264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:38.418 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:07:38.418 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.4772510170000004, 2024-09-14T09:07:38.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097587224, 2024-09-14T09:07:38.418 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.418 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.418 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.418 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.418 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.418 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.418 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118192 2024-09-14T09:07:38.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.418 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:07:38.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:07:38.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844132+0000", 2024-09-14T09:07:38.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010777810000000001 2024-09-14T09:07:38.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844140+0000", 2024-09-14T09:07:38.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4470000000000006e-06 2024-09-14T09:07:38.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844155+0000", 2024-09-14T09:07:38.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4924e-05 2024-09-14T09:07:38.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844163+0000", 2024-09-14T09:07:38.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9980000000000003e-06 2024-09-14T09:07:38.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940199+0000", 2024-09-14T09:07:38.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096036047999999999 2024-09-14T09:07:38.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940282+0000", 2024-09-14T09:07:38.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3073999999999998e-05 2024-09-14T09:07:38.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940641+0000", 2024-09-14T09:07:38.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000358952 2024-09-14T09:07:38.421 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.421 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.422 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.422 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118197 3.17 3:ec5ec63a:::rbd_data.115e96ceebb2.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 3718221] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:38.422 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:07:38.422 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 6.4724716019999997, 2024-09-14T09:07:38.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.141131069, 2024-09-14T09:07:38.422 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.422 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.422 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:38.422 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.422 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.422 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.423 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118197 2024-09-14T09:07:38.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.423 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:07:38.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:07:38.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847834+0000", 2024-09-14T09:07:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2500000000000004e-07 2024-09-14T09:07:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849227+0000", 2024-09-14T09:07:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0013932149999999999 2024-09-14T09:07:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849246+0000", 2024-09-14T09:07:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8958e-05 2024-09-14T09:07:38.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849254+0000", 2024-09-14T09:07:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2360000000000004e-06 2024-09-14T09:07:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940154+0000", 2024-09-14T09:07:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.090899468999999997 2024-09-14T09:07:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940312+0000", 2024-09-14T09:07:38.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000158326 2024-09-14T09:07:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:07:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940728+0000", 2024-09-14T09:07:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00041583100000000002 2024-09-14T09:07:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.962731+0000", 2024-09-14T09:07:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022002608999999999 2024-09-14T09:07:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988881+0000", 2024-09-14T09:07:38.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026150620999999999 2024-09-14T09:07:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988919+0000", 2024-09-14T09:07:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8108999999999997e-05 2024-09-14T09:07:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988964+0000", 2024-09-14T09:07:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4870000000000002e-05 2024-09-14T09:07:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.427 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.428 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:38.429 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:07:38.429 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:38.427+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:38.530 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:38.531+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:38.597 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:38.597 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:38.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.597 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:130080 3.19 3.e5048859 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-14T09:07:38.597 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:38.071000+0000", 2024-09-14T09:07:38.597 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.52753735999999996, 2024-09-14T09:07:38.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.52758247599999997, 2024-09-14T09:07:38.597 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 130080 2024-09-14T09:07:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:38.071000+0000", 2024-09-14T09:07:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:38.071000+0000", 2024-09-14T09:07:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:38.071002+0000", 2024-09-14T09:07:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7749999999999999e-06 2024-09-14T09:07:38.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:38.071008+0000", 2024-09-14T09:07:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.0900000000000001e-06 2024-09-14T09:07:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:38.071022+0000", 2024-09-14T09:07:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4245e-05 2024-09-14T09:07:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:38.071030+0000", 2024-09-14T09:07:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0360000000000006e-06 2024-09-14T09:07:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:38.071099+0000", 2024-09-14T09:07:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9058000000000002e-05 2024-09-14T09:07:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:38.071112+0000", 2024-09-14T09:07:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2357999999999999e-05 2024-09-14T09:07:38.601 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.602 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.602 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.602 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.602 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:38.602 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 1 2024-09-14T09:07:38.602 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:38.611 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:07:38.654 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:38.655+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:38.732 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:38.734+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:38.778 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:38.778 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:38.779 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:38.779 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:38.779 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:38.794 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:07:38.833 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:38.834+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:38.915 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:38.916 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:38.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:38.916 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:38.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.916 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73542 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15319040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:38.916 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:38.916 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.645026444999999, 2024-09-14T09:07:38.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0237091359999999, 2024-09-14T09:07:38.916 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.916 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.916 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.916 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.917 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.917 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.917 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73542 2024-09-14T09:07:38.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.917 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:38.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:38.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268762+0000", 2024-09-14T09:07:38.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:38.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268766+0000", 2024-09-14T09:07:38.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5880000000000002e-06 2024-09-14T09:07:38.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268776+0000", 2024-09-14T09:07:38.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0168e-05 2024-09-14T09:07:38.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268789+0000", 2024-09-14T09:07:38.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:07:38.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282053+0000", 2024-09-14T09:07:38.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013264496000000001 2024-09-14T09:07:38.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:38.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282057+0000", 2024-09-14T09:07:38.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6090000000000001e-06 2024-09-14T09:07:38.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292052+0000", 2024-09-14T09:07:38.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0099950070000001 2024-09-14T09:07:38.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292262+0000", 2024-09-14T09:07:38.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021009800000000001 2024-09-14T09:07:38.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292470+0000", 2024-09-14T09:07:38.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000208314 2024-09-14T09:07:38.921 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.921 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.921 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.922 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73543 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:38.922 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:38.922 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.644987654999994, 2024-09-14T09:07:38.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023985473, 2024-09-14T09:07:38.922 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.922 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.922 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.922 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.922 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73543 2024-09-14T09:07:38.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.922 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:38.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:38.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268801+0000", 2024-09-14T09:07:38.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:38.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268803+0000", 2024-09-14T09:07:38.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7439999999999999e-06 2024-09-14T09:07:38.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268812+0000", 2024-09-14T09:07:38.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7080000000000003e-06 2024-09-14T09:07:38.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268819+0000", 2024-09-14T09:07:38.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T09:07:38.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282068+0000", 2024-09-14T09:07:38.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013249540000000001 2024-09-14T09:07:38.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:38.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282072+0000", 2024-09-14T09:07:38.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0960000000000001e-06 2024-09-14T09:07:38.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292573+0000", 2024-09-14T09:07:38.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0105014560000001 2024-09-14T09:07:38.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292679+0000", 2024-09-14T09:07:38.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010545899999999999 2024-09-14T09:07:38.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292786+0000", 2024-09-14T09:07:38.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107052 2024-09-14T09:07:38.926 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.926 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.926 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.927 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73544 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:38.927 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:38.927 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.644959952999997, 2024-09-14T09:07:38.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0244830549999999, 2024-09-14T09:07:38.927 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.927 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.927 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.927 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.927 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.927 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.927 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73544 2024-09-14T09:07:38.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.928 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:38.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:38.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:38.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5599999999999995e-07 2024-09-14T09:07:38.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268831+0000", 2024-09-14T09:07:38.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.57e-06 2024-09-14T09:07:38.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.929 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268844+0000", 2024-09-14T09:07:38.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2811e-05 2024-09-14T09:07:38.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.929 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268851+0000", 2024-09-14T09:07:38.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9140000000000002e-06 2024-09-14T09:07:38.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282083+0000", 2024-09-14T09:07:38.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013232111 2024-09-14T09:07:38.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:38.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282086+0000", 2024-09-14T09:07:38.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2909999999999999e-06 2024-09-14T09:07:38.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292884+0000", 2024-09-14T09:07:38.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0107975069999999 2024-09-14T09:07:38.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293100+0000", 2024-09-14T09:07:38.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021639399999999999 2024-09-14T09:07:38.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293311+0000", 2024-09-14T09:07:38.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021090099999999999 2024-09-14T09:07:38.932 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.932 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.932 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.932 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73545 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15564800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:38.932 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:38.932 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.644908767999993, 2024-09-14T09:07:38.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0249746740000001, 2024-09-14T09:07:38.932 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.932 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.932 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.932 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.932 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.933 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.933 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73545 2024-09-14T09:07:38.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.933 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:38.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.933 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:38.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268880+0000", 2024-09-14T09:07:38.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5600000000000005e-07 2024-09-14T09:07:38.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268889+0000", 2024-09-14T09:07:38.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8559999999999996e-06 2024-09-14T09:07:38.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268913+0000", 2024-09-14T09:07:38.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3958e-05 2024-09-14T09:07:38.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268921+0000", 2024-09-14T09:07:38.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7479999999999999e-06 2024-09-14T09:07:38.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282098+0000", 2024-09-14T09:07:38.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013176297999999999 2024-09-14T09:07:38.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:38.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282101+0000", 2024-09-14T09:07:38.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2329999999999998e-06 2024-09-14T09:07:38.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293435+0000", 2024-09-14T09:07:38.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0113339459999999 2024-09-14T09:07:38.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293648+0000", 2024-09-14T09:07:38.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021307100000000001 2024-09-14T09:07:38.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293854+0000", 2024-09-14T09:07:38.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020580800000000001 2024-09-14T09:07:38.937 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.937 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.937 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.937 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73546 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15646720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:38.937 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:38.937 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.644854897000002, 2024-09-14T09:07:38.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0254846360000001, 2024-09-14T09:07:38.938 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.938 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.938 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.938 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.938 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.938 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.938 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73546 2024-09-14T09:07:38.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.938 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:38.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:38.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268934+0000", 2024-09-14T09:07:38.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7999999999999995e-07 2024-09-14T09:07:38.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268937+0000", 2024-09-14T09:07:38.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9840000000000001e-06 2024-09-14T09:07:38.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268947+0000", 2024-09-14T09:07:38.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0522e-05 2024-09-14T09:07:38.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268955+0000", 2024-09-14T09:07:38.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8059999999999995e-06 2024-09-14T09:07:38.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282112+0000", 2024-09-14T09:07:38.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013157189 2024-09-14T09:07:38.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:38.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282115+0000", 2024-09-14T09:07:38.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2949999999999998e-06 2024-09-14T09:07:38.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293968+0000", 2024-09-14T09:07:38.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0118530990000001 2024-09-14T09:07:38.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294197+0000", 2024-09-14T09:07:38.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228507 2024-09-14T09:07:38.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294417+0000", 2024-09-14T09:07:38.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000220554 2024-09-14T09:07:38.942 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.942 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.942 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.942 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73547 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:38.942 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:38.942 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.644822450999996, 2024-09-14T09:07:38.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.025970525, 2024-09-14T09:07:38.943 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.943 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.943 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.943 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.943 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.943 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.943 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73547 2024-09-14T09:07:38.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.943 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:38.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:38.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268966+0000", 2024-09-14T09:07:38.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3799999999999997e-07 2024-09-14T09:07:38.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268969+0000", 2024-09-14T09:07:38.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8739999999999999e-06 2024-09-14T09:07:38.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268979+0000", 2024-09-14T09:07:38.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9939999999999995e-06 2024-09-14T09:07:38.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268986+0000", 2024-09-14T09:07:38.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3459999999999996e-06 2024-09-14T09:07:38.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282127+0000", 2024-09-14T09:07:38.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01314039 2024-09-14T09:07:38.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:38.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282130+0000", 2024-09-14T09:07:38.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2219999999999998e-06 2024-09-14T09:07:38.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294540+0000", 2024-09-14T09:07:38.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124105880000001 2024-09-14T09:07:38.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294759+0000", 2024-09-14T09:07:38.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000218798 2024-09-14T09:07:38.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294936+0000", 2024-09-14T09:07:38.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000176675 2024-09-14T09:07:38.947 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.947 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.947 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.947 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73548 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15810560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:38.948 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:38.948 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.644791338999994, 2024-09-14T09:07:38.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0265045129999999, 2024-09-14T09:07:38.948 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.948 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.948 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.948 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.948 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.948 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.948 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73548 2024-09-14T09:07:38.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.948 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:38.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:38.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268997+0000", 2024-09-14T09:07:38.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7599999999999996e-07 2024-09-14T09:07:38.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269000+0000", 2024-09-14T09:07:38.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0580000000000002e-06 2024-09-14T09:07:38.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269010+0000", 2024-09-14T09:07:38.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124e-05 2024-09-14T09:07:38.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269018+0000", 2024-09-14T09:07:38.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5399999999999998e-06 2024-09-14T09:07:38.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282141+0000", 2024-09-14T09:07:38.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013123012 2024-09-14T09:07:38.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:38.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282144+0000", 2024-09-14T09:07:38.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.196e-06 2024-09-14T09:07:38.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295042+0000", 2024-09-14T09:07:38.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0128975689999999 2024-09-14T09:07:38.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295272+0000", 2024-09-14T09:07:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023048099999999999 2024-09-14T09:07:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295501+0000", 2024-09-14T09:07:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228657 2024-09-14T09:07:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.952 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73549 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15892480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.644745459999996, 2024-09-14T09:07:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0268354099999999, 2024-09-14T09:07:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73549 2024-09-14T09:07:38.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269043+0000", 2024-09-14T09:07:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6199999999999997e-07 2024-09-14T09:07:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269046+0000", 2024-09-14T09:07:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2480000000000001e-06 2024-09-14T09:07:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269057+0000", 2024-09-14T09:07:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0436e-05 2024-09-14T09:07:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269064+0000", 2024-09-14T09:07:38.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5179999999999998e-06 2024-09-14T09:07:38.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282155+0000", 2024-09-14T09:07:38.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013091190000000001 2024-09-14T09:07:38.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:38.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282159+0000", 2024-09-14T09:07:38.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.366e-06 2024-09-14T09:07:38.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295599+0000", 2024-09-14T09:07:38.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0134405559999999 2024-09-14T09:07:38.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295765+0000", 2024-09-14T09:07:38.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016575499999999999 2024-09-14T09:07:38.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295878+0000", 2024-09-14T09:07:38.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112679 2024-09-14T09:07:38.957 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.958 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.958 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.958 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73550 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15974400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:38.958 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:38.958 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.644712859999998, 2024-09-14T09:07:38.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027357657, 2024-09-14T09:07:38.958 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.958 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.958 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.958 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.958 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.958 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.959 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73550 2024-09-14T09:07:38.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.959 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:38.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:38.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269076+0000", 2024-09-14T09:07:38.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:07:38.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269079+0000", 2024-09-14T09:07:38.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:07:38.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269098+0000", 2024-09-14T09:07:38.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9698e-05 2024-09-14T09:07:38.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269106+0000", 2024-09-14T09:07:38.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6179999999999997e-06 2024-09-14T09:07:38.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282170+0000", 2024-09-14T09:07:38.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013063874 2024-09-14T09:07:38.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:38.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282173+0000", 2024-09-14T09:07:38.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1999999999999999e-06 2024-09-14T09:07:38.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295996+0000", 2024-09-14T09:07:38.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.013823151 2024-09-14T09:07:38.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296214+0000", 2024-09-14T09:07:38.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021817700000000001 2024-09-14T09:07:38.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296433+0000", 2024-09-14T09:07:38.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021829099999999999 2024-09-14T09:07:38.963 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.963 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.963 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.963 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73551 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:38.963 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:38.963 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.644671442999993, 2024-09-14T09:07:38.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027730276, 2024-09-14T09:07:38.963 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.963 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.963 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.964 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73551 2024-09-14T09:07:38.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.964 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:38.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:38.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269117+0000", 2024-09-14T09:07:38.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:07:38.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269120+0000", 2024-09-14T09:07:38.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.056e-06 2024-09-14T09:07:38.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269130+0000", 2024-09-14T09:07:38.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0084e-05 2024-09-14T09:07:38.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269137+0000", 2024-09-14T09:07:38.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1359999999999999e-06 2024-09-14T09:07:38.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282184+0000", 2024-09-14T09:07:38.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013046537 2024-09-14T09:07:38.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:38.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282188+0000", 2024-09-14T09:07:38.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3220000000000003e-06 2024-09-14T09:07:38.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296529+0000", 2024-09-14T09:07:38.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0143406109999999 2024-09-14T09:07:38.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296695+0000", 2024-09-14T09:07:38.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016604799999999999 2024-09-14T09:07:38.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296847+0000", 2024-09-14T09:07:38.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000151822 2024-09-14T09:07:38.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.968 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.968 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.968 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73552 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:38.968 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:38.968 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.644640863000006, 2024-09-14T09:07:38.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0281767500000001, 2024-09-14T09:07:38.968 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.968 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.968 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.968 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.969 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73552 2024-09-14T09:07:38.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.969 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:38.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:38.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:38.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:07:38.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269150+0000", 2024-09-14T09:07:38.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9119999999999998e-06 2024-09-14T09:07:38.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269160+0000", 2024-09-14T09:07:38.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6819999999999998e-06 2024-09-14T09:07:38.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269168+0000", 2024-09-14T09:07:38.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6079999999999998e-06 2024-09-14T09:07:38.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.971 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282205+0000", 2024-09-14T09:07:38.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013036982000000001 2024-09-14T09:07:38.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:38.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282208+0000", 2024-09-14T09:07:38.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4620000000000001e-06 2024-09-14T09:07:38.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296968+0000", 2024-09-14T09:07:38.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0147601209999999 2024-09-14T09:07:38.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297115+0000", 2024-09-14T09:07:38.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014673900000000001 2024-09-14T09:07:38.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297324+0000", 2024-09-14T09:07:38.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020866400000000001 2024-09-14T09:07:38.973 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.973 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.973 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.974 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73553 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:38.974 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:38.974 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.644610799000006, 2024-09-14T09:07:38.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0286556739999999, 2024-09-14T09:07:38.974 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.974 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.974 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.974 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.974 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.974 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.974 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73553 2024-09-14T09:07:38.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.974 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:38.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:38.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269178+0000", 2024-09-14T09:07:38.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:07:38.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269180+0000", 2024-09-14T09:07:38.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T09:07:38.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.976 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.976 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269189+0000", 2024-09-14T09:07:38.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2e-06 2024-09-14T09:07:38.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269197+0000", 2024-09-14T09:07:38.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2980000000000003e-06 2024-09-14T09:07:38.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282219+0000", 2024-09-14T09:07:38.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013022687 2024-09-14T09:07:38.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:38.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282223+0000", 2024-09-14T09:07:38.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.241e-06 2024-09-14T09:07:38.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297470+0000", 2024-09-14T09:07:38.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.015247778 2024-09-14T09:07:38.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297704+0000", 2024-09-14T09:07:38.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000233367 2024-09-14T09:07:38.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297833+0000", 2024-09-14T09:07:38.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012888900000000001 2024-09-14T09:07:38.979 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.979 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.979 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.979 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73554 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:38.979 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:38.980 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.644581482000007, 2024-09-14T09:07:38.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.029005113, 2024-09-14T09:07:38.980 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.980 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.980 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.980 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.980 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.980 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.980 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73554 2024-09-14T09:07:38.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.980 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:38.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:38.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269207+0000", 2024-09-14T09:07:38.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.68e-07 2024-09-14T09:07:38.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269210+0000", 2024-09-14T09:07:38.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8100000000000002e-06 2024-09-14T09:07:38.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269219+0000", 2024-09-14T09:07:38.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:07:38.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269227+0000", 2024-09-14T09:07:38.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3839999999999999e-06 2024-09-14T09:07:38.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282234+0000", 2024-09-14T09:07:38.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013007187 2024-09-14T09:07:38.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:38.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282237+0000", 2024-09-14T09:07:38.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1200000000000002e-06 2024-09-14T09:07:38.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297942+0000", 2024-09-14T09:07:38.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0157048040000001 2024-09-14T09:07:38.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298087+0000", 2024-09-14T09:07:38.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014499099999999999 2024-09-14T09:07:38.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298211+0000", 2024-09-14T09:07:38.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012461899999999999 2024-09-14T09:07:38.985 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.985 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.985 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.985 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73555 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:38.985 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:38.985 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.644551633999995, 2024-09-14T09:07:38.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.02941558, 2024-09-14T09:07:38.985 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.986 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.986 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.986 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.986 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.986 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.986 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73555 2024-09-14T09:07:38.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.986 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:38.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:38.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269237+0000", 2024-09-14T09:07:38.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3399999999999999e-07 2024-09-14T09:07:38.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.987 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269240+0000", 2024-09-14T09:07:38.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8480000000000001e-06 2024-09-14T09:07:38.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.987 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269249+0000", 2024-09-14T09:07:38.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1880000000000008e-06 2024-09-14T09:07:38.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269256+0000", 2024-09-14T09:07:38.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:07:38.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282248+0000", 2024-09-14T09:07:38.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012992327 2024-09-14T09:07:38.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:38.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282251+0000", 2024-09-14T09:07:38.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2160000000000002e-06 2024-09-14T09:07:38.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298299+0000", 2024-09-14T09:07:38.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0160479330000001 2024-09-14T09:07:38.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298446+0000", 2024-09-14T09:07:38.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000146525 2024-09-14T09:07:38.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298652+0000", 2024-09-14T09:07:38.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020584100000000001 2024-09-14T09:07:38.990 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.990 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.990 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.990 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73556 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:38.990 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:38.990 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.644521888, 2024-09-14T09:07:38.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0299854239999999, 2024-09-14T09:07:38.991 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.991 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.991 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.991 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.991 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.991 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.991 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73556 2024-09-14T09:07:38.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.991 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:38.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:38.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:38.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:38.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.992 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.992 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269270+0000", 2024-09-14T09:07:38.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0920000000000002e-06 2024-09-14T09:07:38.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269280+0000", 2024-09-14T09:07:38.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0013999999999999e-05 2024-09-14T09:07:38.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269288+0000", 2024-09-14T09:07:38.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:07:38.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282263+0000", 2024-09-14T09:07:38.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012974703000000001 2024-09-14T09:07:38.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:38.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282266+0000", 2024-09-14T09:07:38.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1520000000000001e-06 2024-09-14T09:07:38.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298729+0000", 2024-09-14T09:07:38.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.016463506 2024-09-14T09:07:38.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298884+0000", 2024-09-14T09:07:38.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015450800000000001 2024-09-14T09:07:38.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:38.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299251+0000", 2024-09-14T09:07:38.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036759899999999997 2024-09-14T09:07:38.995 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.995 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:38.995 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:38.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.995 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73557 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16547840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:38.995 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:38.995 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.644475237999998, 2024-09-14T09:07:38.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0307969480000001, 2024-09-14T09:07:38.995 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:38.995 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:38.995 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:38.996 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:38.996 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:38.996 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:38.996 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73557 2024-09-14T09:07:38.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.996 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:38.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:38.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:38.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:38.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:38.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:38.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:38.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269313+0000", 2024-09-14T09:07:38.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.92e-07 2024-09-14T09:07:38.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.997 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:38.997 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269336+0000", 2024-09-14T09:07:38.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3118000000000001e-05 2024-09-14T09:07:38.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:38.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269346+0000", 2024-09-14T09:07:38.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4069999999999999e-06 2024-09-14T09:07:38.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:38.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269352+0000", 2024-09-14T09:07:38.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5810000000000002e-06 2024-09-14T09:07:38.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282277+0000", 2024-09-14T09:07:38.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012924658 2024-09-14T09:07:38.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:38.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282280+0000", 2024-09-14T09:07:38.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2880000000000001e-06 2024-09-14T09:07:38.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:38.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299343+0000", 2024-09-14T09:07:38.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0170630190000001 2024-09-14T09:07:38.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:38.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:38.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:38.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299554+0000", 2024-09-14T09:07:38.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021085099999999999 2024-09-14T09:07:39.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:39.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300109+0000", 2024-09-14T09:07:39.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00055533399999999999 2024-09-14T09:07:39.000 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:39.000 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:39.000 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:39.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.000 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73558 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:39.000 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:39.000 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.644423614999994, 2024-09-14T09:07:39.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0314849340000001, 2024-09-14T09:07:39.001 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:39.001 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:39.001 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:39.001 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:39.001 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:39.001 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:39.001 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73558 2024-09-14T09:07:39.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.001 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:39.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:39.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:39.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:39.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:39.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:39.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:39.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:39.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269365+0000", 2024-09-14T09:07:39.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7400000000000003e-07 2024-09-14T09:07:39.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.002 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.002 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:39.002 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269367+0000", 2024-09-14T09:07:39.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4930000000000001e-06 2024-09-14T09:07:39.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:39.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269374+0000", 2024-09-14T09:07:39.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9519999999999997e-06 2024-09-14T09:07:39.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:39.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269379+0000", 2024-09-14T09:07:39.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9710000000000003e-06 2024-09-14T09:07:39.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:39.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282292+0000", 2024-09-14T09:07:39.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012912704000000001 2024-09-14T09:07:39.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:39.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282295+0000", 2024-09-14T09:07:39.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.179e-06 2024-09-14T09:07:39.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:39.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300195+0000", 2024-09-14T09:07:39.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.017899975 2024-09-14T09:07:39.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:39.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300345+0000", 2024-09-14T09:07:39.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014957499999999999 2024-09-14T09:07:39.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:39.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300849+0000", 2024-09-14T09:07:39.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050451100000000002 2024-09-14T09:07:39.005 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:39.005 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:39.005 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:39.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.005 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73559 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:39.005 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:39.006 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.644401205999998, 2024-09-14T09:07:39.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.031916343, 2024-09-14T09:07:39.006 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:39.006 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:39.006 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:39.006 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:39.006 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:39.006 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:39.006 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73559 2024-09-14T09:07:39.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.006 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:39.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:39.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:39.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:39.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:39.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:39.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:39.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:39.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:39.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3500000000000002e-07 2024-09-14T09:07:39.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:39.007 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269390+0000", 2024-09-14T09:07:39.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5890000000000001e-06 2024-09-14T09:07:39.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:39.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269396+0000", 2024-09-14T09:07:39.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3300000000000004e-06 2024-09-14T09:07:39.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:39.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269401+0000", 2024-09-14T09:07:39.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0340000000000004e-06 2024-09-14T09:07:39.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:39.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282311+0000", 2024-09-14T09:07:39.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012910204 2024-09-14T09:07:39.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:39.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282317+0000", 2024-09-14T09:07:39.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2639999999999997e-06 2024-09-14T09:07:39.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:39.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300933+0000", 2024-09-14T09:07:39.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0186160710000001 2024-09-14T09:07:39.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:39.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301158+0000", 2024-09-14T09:07:39.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225021 2024-09-14T09:07:39.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:39.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301303+0000", 2024-09-14T09:07:39.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014439499999999999 2024-09-14T09:07:39.010 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:39.010 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:39.010 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:39.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.010 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106964 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:39.010 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:07:39.011 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.798473372, 2024-09-14T09:07:39.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8802924409999999, 2024-09-14T09:07:39.011 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:39.011 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:39.011 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:39.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:39.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:39.011 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:39.011 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106964 2024-09-14T09:07:39.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.011 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:39.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:39.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:07:39.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:39.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:39.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:07:39.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:39.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:39.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115317+0000", 2024-09-14T09:07:39.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6429999999999999e-06 2024-09-14T09:07:39.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:39.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115326+0000", 2024-09-14T09:07:39.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2140000000000002e-06 2024-09-14T09:07:39.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:39.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115346+0000", 2024-09-14T09:07:39.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9419000000000002e-05 2024-09-14T09:07:39.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:39.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115357+0000", 2024-09-14T09:07:39.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1014e-05 2024-09-14T09:07:39.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:39.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115431+0000", 2024-09-14T09:07:39.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4040000000000003e-05 2024-09-14T09:07:39.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:39.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115438+0000", 2024-09-14T09:07:39.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9920000000000001e-06 2024-09-14T09:07:39.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:39.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844944+0000", 2024-09-14T09:07:39.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72950628399999995 2024-09-14T09:07:39.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:39.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844961+0000", 2024-09-14T09:07:39.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7156e-05 2024-09-14T09:07:39.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:39.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988004+0000", 2024-09-14T09:07:39.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.14304297 2024-09-14T09:07:39.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:39.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988511+0000", 2024-09-14T09:07:39.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050694100000000001 2024-09-14T09:07:39.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:39.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.989802+0000", 2024-09-14T09:07:39.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001291195 2024-09-14T09:07:39.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:39.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994866+0000", 2024-09-14T09:07:39.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050632760000000002 2024-09-14T09:07:39.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:39.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994988+0000", 2024-09-14T09:07:39.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012252899999999999 2024-09-14T09:07:39.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:39.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995198+0000", 2024-09-14T09:07:39.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021010899999999999 2024-09-14T09:07:39.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:39.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995607+0000", 2024-09-14T09:07:39.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040865900000000002 2024-09-14T09:07:39.020 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:39.020 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:39.020 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:39.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.020 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106965 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:39.020 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:07:39.021 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 17.797270995000002, 2024-09-14T09:07:39.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.879730162, 2024-09-14T09:07:39.021 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:39.021 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:39.021 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:39.021 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:39.021 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:39.021 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:39.021 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106965 2024-09-14T09:07:39.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.021 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:39.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.021 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:39.021 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:07:39.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:39.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:39.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:07:39.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:39.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:39.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:07:39.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:07:39.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:39.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116522+0000", 2024-09-14T09:07:39.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3989999999999997e-06 2024-09-14T09:07:39.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:39.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116533+0000", 2024-09-14T09:07:39.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0669000000000001e-05 2024-09-14T09:07:39.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:39.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116541+0000", 2024-09-14T09:07:39.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2730000000000002e-06 2024-09-14T09:07:39.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:39.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116604+0000", 2024-09-14T09:07:39.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2706999999999994e-05 2024-09-14T09:07:39.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:39.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116610+0000", 2024-09-14T09:07:39.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8630000000000002e-06 2024-09-14T09:07:39.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:39.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845060+0000", 2024-09-14T09:07:39.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72845003699999999 2024-09-14T09:07:39.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:39.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845079+0000", 2024-09-14T09:07:39.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8287999999999998e-05 2024-09-14T09:07:39.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:39.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.991866+0000", 2024-09-14T09:07:39.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1467871519999999 2024-09-14T09:07:39.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:39.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.992274+0000", 2024-09-14T09:07:39.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040813500000000001 2024-09-14T09:07:39.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:39.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.993538+0000", 2024-09-14T09:07:39.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001263655 2024-09-14T09:07:39.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:39.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995770+0000", 2024-09-14T09:07:39.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002231949 2024-09-14T09:07:39.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:39.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995810+0000", 2024-09-14T09:07:39.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0852999999999997e-05 2024-09-14T09:07:39.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.026 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:39.026 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995979+0000", 2024-09-14T09:07:39.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016866 2024-09-14T09:07:39.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:39.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:39.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:39.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996247+0000", 2024-09-14T09:07:39.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000267769 2024-09-14T09:07:39.027 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:39.027 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:39.027 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:39.027 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:39.027 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:39.027 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:39.028 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T09:07:39.029 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:39.030+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:39.030 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:07:39.031 INFO:teuthology.orchestra.run.smithi176.stdout:5647 write 0x11e6a8f thru 0x11f2aa8 (0xc01a bytes) 2024-09-14T09:07:39.031 INFO:teuthology.orchestra.run.smithi176.stdout:5648 read 0x15dd75a thru 0x15e11df (0x3a86 bytes) 2024-09-14T09:07:39.031 INFO:teuthology.orchestra.run.smithi176.stdout:5649 read 0x13547e1 thru 0x136104b (0xc86b bytes) 2024-09-14T09:07:39.031 INFO:teuthology.orchestra.run.smithi176.stdout:5650 read 0x841061 thru 0x849348 (0x82e8 bytes) 2024-09-14T09:07:39.031 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:39.031 INFO:teuthology.orchestra.run.smithi176.stdout:5652 punch from 0x1c81a19 to 0x1c8b6ec, (0x9cd3 bytes) 2024-09-14T09:07:39.031 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:39.031 INFO:teuthology.orchestra.run.smithi176.stdout:5654 punch from 0x14fdb67 to 0x1505356, (0x77ef bytes) 2024-09-14T09:07:39.031 INFO:teuthology.orchestra.run.smithi176.stdout:5655 write 0xd5fab7a thru 0xd60a261 (0xf6e8 bytes) 2024-09-14T09:07:39.042 INFO:teuthology.orchestra.run.smithi176.stdout:5656 read 0x57a0848 thru 0x57a6483 (0x5c3c bytes) 2024-09-14T09:07:39.042 INFO:teuthology.orchestra.run.smithi176.stdout:5657 read 0x680f170 thru 0x6816a06 (0x7897 bytes) 2024-09-14T09:07:39.042 INFO:teuthology.orchestra.run.smithi176.stdout:5659 read 0xcad1eb8 thru 0xcad7959 (0x5aa2 bytes) 2024-09-14T09:07:39.054 INFO:teuthology.orchestra.run.smithi176.stdout:5661 write 0xe156e45 thru 0xe15999a (0x2b56 bytes) 2024-09-14T09:07:39.084 INFO:teuthology.orchestra.run.smithi176.stdout:5663 trunc from 0xe15999b to 0x189755e 2024-09-14T09:07:39.130 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:39.134+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:39.148 INFO:teuthology.orchestra.run.smithi176.stdout:5664 write 0xdffeca7 thru 0xe007f5f (0x92b9 bytes) 2024-09-14T09:07:39.162 INFO:teuthology.orchestra.run.smithi176.stdout:5665 write 0xd61007 thru 0xd65f28 (0x4f22 bytes) 2024-09-14T09:07:39.178 INFO:teuthology.orchestra.run.smithi176.stdout:5666 trunc from 0xe007f60 to 0x967f545 2024-09-14T09:07:39.183 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:39.184 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:39.184 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:39.184 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:39.201 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:07:39.202 INFO:teuthology.orchestra.run.smithi176.stdout:5668 write 0x92dc127 thru 0x92eaf17 (0xedf1 bytes) 2024-09-14T09:07:39.216 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:07:39.221 INFO:teuthology.orchestra.run.smithi176.stdout:5669 write 0x3d132b8 thru 0x3d1f5fe (0xc347 bytes) 2024-09-14T09:07:39.231 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:39.234+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:39.234 INFO:teuthology.orchestra.run.smithi176.stdout:5671 punch from 0x7ffd0b9 to 0x7ffe7c8, (0x170f bytes) 2024-09-14T09:07:39.247 INFO:teuthology.orchestra.run.smithi176.stdout:5674 trunc from 0x967f545 to 0xa2caa1 2024-09-14T09:07:39.290 INFO:teuthology.orchestra.run.smithi176.stdout:5675 trunc from 0xa2caa1 to 0x39150af 2024-09-14T09:07:39.326 INFO:teuthology.orchestra.run.smithi176.stdout:5676 write 0xdc2f3a5 thru 0xdc3068d (0x12e9 bytes) 2024-09-14T09:07:39.332 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:39.334+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:39.335 INFO:teuthology.orchestra.run.smithi176.stdout:5677 write 0x5222f59 thru 0x52262e2 (0x338a bytes) 2024-09-14T09:07:39.353 INFO:teuthology.orchestra.run.smithi176.stdout:5678 trunc from 0xdc3068e to 0x8e28db9 2024-09-14T09:07:39.367 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:39.367 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:39.367 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:39.367 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:39.367 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:39.370 INFO:teuthology.orchestra.run.smithi176.stdout:5679 write 0x2b31167 thru 0x2b34aa1 (0x393b bytes) 2024-09-14T09:07:39.379 INFO:teuthology.orchestra.run.smithi176.stdout:5680 writesame 0x5f38ea1 thru 0x5f3cf78 data_size 0xa(0x40d8 bytes) 2024-09-14T09:07:39.380 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:39.380 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:39.389 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:07:39.389 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:39.389 INFO:teuthology.orchestra.run.smithi176.stdout:5682 writesame 0x51445be thru 0x5151299 data_size 0x2(0xccdc bytes) 2024-09-14T09:07:39.403 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:39.403 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:39.421 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:39.421 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:39.432 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:39.431+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:39.436 INFO:teuthology.orchestra.run.smithi176.stdout:5683 read 0x6563a73 thru 0x6573250 (0xf7de bytes) 2024-09-14T09:07:39.436 INFO:teuthology.orchestra.run.smithi176.stdout:5684 punch from 0xd792cc to 0xd8674c, (0xd480 bytes) 2024-09-14T09:07:39.436 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:39.436 INFO:teuthology.orchestra.run.smithi176.stdout:5687 punch from 0x6b60c49 to 0x6b6d274, (0xc62b bytes) 2024-09-14T09:07:39.436 INFO:teuthology.orchestra.run.smithi176.stdout:5688 read 0x8037d38 thru 0x80423d2 (0xa69b bytes) 2024-09-14T09:07:39.440 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:39.441 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:39.449 INFO:teuthology.orchestra.run.smithi176.stdout:5689 write 0xbc19ee7 thru 0xbc20480 (0x659a bytes) 2024-09-14T09:07:39.464 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:07:39.465 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:07:39.465 INFO:teuthology.orchestra.run.smithi176.stdout:5690 write 0x53fed30 thru 0x5408f83 (0xa254 bytes) 2024-09-14T09:07:39.479 INFO:teuthology.orchestra.run.smithi176.stdout:5691 write 0xa838d07 thru 0xa83a356 (0x1650 bytes) 2024-09-14T09:07:39.487 INFO:teuthology.orchestra.run.smithi176.stdout:5693 writesame 0x2026eb5 thru 0x2030c0b data_size 0x1(0x9d57 bytes) 2024-09-14T09:07:39.489 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:39.489 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:39.514 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:39.514 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:39.532 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:39.532 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:39.533 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:39.534+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:39.536 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:39.536 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:39.545 INFO:teuthology.orchestra.run.smithi176.stdout:5694 read 0x63601b0 thru 0x636c6b3 (0xc504 bytes) 2024-09-14T09:07:39.545 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:39.548 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:07:39.557 INFO:teuthology.orchestra.run.smithi176.stdout:5696 write 0xd5fc1f9 thru 0xd606f52 (0xad5a bytes) 2024-09-14T09:07:39.574 INFO:teuthology.orchestra.run.smithi176.stdout:5697 punch from 0x6dc37b5 to 0x6dd2c48, (0xf493 bytes) 2024-09-14T09:07:39.574 INFO:teuthology.orchestra.run.smithi176.stdout:5698 punch from 0x9e3ce1f to 0x9e4a585, (0xd766 bytes) 2024-09-14T09:07:39.574 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:39.587 INFO:teuthology.orchestra.run.smithi176.stdout:5700 trunc from 0xd606f53 to 0x4295fe5 2024-09-14T09:07:39.618 INFO:teuthology.orchestra.run.smithi176.stdout:5701 read 0x3954de2 thru 0x395d1cd (0x83ec bytes) 2024-09-14T09:07:39.632 INFO:teuthology.orchestra.run.smithi176.stdout:5702 write 0x815df33 thru 0x8163603 (0x56d1 bytes) 2024-09-14T09:07:39.634 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:39.635+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:39.645 INFO:teuthology.orchestra.run.smithi176.stdout:5704 read 0x532e6e7 thru 0x533029f (0x1bb9 bytes) 2024-09-14T09:07:39.645 INFO:teuthology.orchestra.run.smithi176.stdout:5705 write 0x150c1d1 thru 0x1514a73 (0x88a3 bytes) 2024-09-14T09:07:39.659 INFO:teuthology.orchestra.run.smithi176.stdout:5706 read 0x5180634 thru 0x518d241 (0xcc0e bytes) 2024-09-14T09:07:39.660 INFO:teuthology.orchestra.run.smithi176.stdout:5707 read 0x2c1ea22 thru 0x2c1fb1c (0x10fb bytes) 2024-09-14T09:07:39.660 INFO:teuthology.orchestra.run.smithi176.stdout:5708 punch from 0x5238fbb to 0x523fd1e, (0x6d63 bytes) 2024-09-14T09:07:39.660 INFO:teuthology.orchestra.run.smithi176.stdout:5709 read 0x21c0b8d thru 0x21d0639 (0xfaad bytes) 2024-09-14T09:07:39.660 INFO:teuthology.orchestra.run.smithi176.stdout:5710 punch from 0x54cba90 to 0x54d67fa, (0xad6a bytes) 2024-09-14T09:07:39.661 INFO:teuthology.orchestra.run.smithi176.stdout:5711 trunc from 0x8163604 to 0x7c5bb19 2024-09-14T09:07:39.686 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:39.686 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:39.686 INFO:teuthology.orchestra.run.smithi176.stdout:5712 write 0x8c01336 thru 0x8c0dd0f (0xc9da bytes) 2024-09-14T09:07:39.701 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:39.702 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:39.705 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:07:39.715 INFO:teuthology.orchestra.run.smithi176.stdout:5716 write 0xabe2ae8 thru 0xabee901 (0xbe1a bytes) 2024-09-14T09:07:39.731 INFO:teuthology.orchestra.run.smithi176.stdout:5717 write 0x404e639 thru 0x4059a39 (0xb401 bytes) 2024-09-14T09:07:39.735 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:39.738+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:39.749 INFO:teuthology.orchestra.run.smithi176.stdout:5719 writesame 0x7960f51 thru 0x796cda0 data_size 0x69(0xbe50 bytes) 2024-09-14T09:07:39.765 INFO:teuthology.orchestra.run.smithi176.stdout:5720 write 0x479942 thru 0x47dbc7 (0x4286 bytes) 2024-09-14T09:07:39.780 INFO:teuthology.orchestra.run.smithi176.stdout:5721 trunc from 0xabee902 to 0x3554b5e 2024-09-14T09:07:39.814 INFO:teuthology.orchestra.run.smithi176.stdout:5722 read 0x340c92a thru 0x3414e54 (0x852b bytes) 2024-09-14T09:07:39.814 INFO:teuthology.orchestra.run.smithi176.stdout:5723 write 0x1dc46b thru 0x1e7f55 (0xbaeb bytes) 2024-09-14T09:07:39.836 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:39.835+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:39.840 INFO:teuthology.orchestra.run.smithi176.stdout:5724 write 0x91e6c9f thru 0x91f6b21 (0xfe83 bytes) 2024-09-14T09:07:39.856 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:39.856 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:39.871 INFO:teuthology.orchestra.run.smithi176.stdout:5725 trunc from 0x91f6b22 to 0x2a7e808 2024-09-14T09:07:39.878 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:07:39.900 INFO:teuthology.orchestra.run.smithi176.stdout:5726 write 0x10f382a thru 0x10f7bfc (0x43d3 bytes) 2024-09-14T09:07:39.912 INFO:teuthology.orchestra.run.smithi176.stdout:5727 punch from 0x2a6ac4d to 0x2a79d99, (0xf14c bytes) 2024-09-14T09:07:39.913 INFO:teuthology.orchestra.run.smithi176.stdout:5728 read 0x1b5fac7 thru 0x1b6bbbd (0xc0f7 bytes) 2024-09-14T09:07:39.913 INFO:teuthology.orchestra.run.smithi176.stdout:5729 read 0xa53239 thru 0xa5578b (0x2553 bytes) 2024-09-14T09:07:39.916 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:07:39.916 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:07:39.916 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-14T09:07:39.916 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-14T09:07:39.937 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is set 2024-09-14T09:07:39.938 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:39.938+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:39.948 INFO:teuthology.orchestra.run.smithi176.stdout:5730 writesame 0xe4d27d4 thru 0xe4d81ec data_size 0x7(0x5a19 bytes) 2024-09-14T09:07:39.949 INFO:teuthology.orchestra.run.smithi176.stdout:5731 read 0x9b16224 thru 0x9b1e548 (0x8325 bytes) 2024-09-14T09:07:39.949 INFO:teuthology.orchestra.run.smithi176.stdout:5733 read 0xaf7e43d thru 0xaf89aca (0xb68e bytes) 2024-09-14T09:07:39.949 INFO:teuthology.orchestra.run.smithi176.stdout:5734 write 0x1fb704a thru 0x1fc5502 (0xe4b9 bytes) 2024-09-14T09:07:39.966 INFO:teuthology.orchestra.run.smithi176.stdout:5735 trunc from 0xe4d81ed to 0xd306cc7 2024-09-14T09:07:39.975 INFO:teuthology.orchestra.run.smithi176.stdout:5737 read 0xc99687c thru 0xc99ea32 (0x81b7 bytes) 2024-09-14T09:07:39.975 INFO:teuthology.orchestra.run.smithi176.stdout:5740 read 0x8033337 thru 0x803f911 (0xc5db bytes) 2024-09-14T09:07:39.975 INFO:teuthology.orchestra.run.smithi176.stdout:5741 write 0xb9cc8e1 thru 0xb9d7648 (0xad68 bytes) 2024-09-14T09:07:39.989 INFO:teuthology.orchestra.run.smithi176.stdout:5742 write 0x3a76a6b thru 0x3a84d2c (0xe2c2 bytes) 2024-09-14T09:07:40.007 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:40.007 INFO:teuthology.orchestra.run.smithi176.stdout:5744 writesame 0x66b3f1e thru 0x66c27a3 data_size 0x9(0xe886 bytes) 2024-09-14T09:07:40.025 INFO:teuthology.orchestra.run.smithi176.stdout:5746 writesame 0x1ce36c0 thru 0x1ce51f9 data_size 0x52(0x1b3a bytes) 2024-09-14T09:07:40.029 INFO:teuthology.orchestra.run.smithi176.stdout:5747 read 0x3368875 thru 0x33705b9 (0x7d45 bytes) 2024-09-14T09:07:40.029 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:40.029 INFO:teuthology.orchestra.run.smithi176.stdout:5749 clone 42 order 23 su 32768 sc 7 2024-09-14T09:07:40.038 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:40.042+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:40.040 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:40.040 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:40.055 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:07:40.089 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-14T09:07:40.178 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:40.179+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:40.183 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:40.183 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:40.197 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:07:40.240 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:40.239+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:40.290 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-14T09:07:40.321 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:40.321 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:40.336 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:07:40.382 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:40.383+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:40.441 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:40.443+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:40.453 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:40.453 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:40.468 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:07:40.485 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-14T09:07:40.542 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:40.546+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:40.585 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:40.585 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:40.600 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:07:40.642 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:40.643+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:40.683 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-14T09:07:40.744 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:40.746+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:40.766 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:40.766 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:40.781 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:07:40.866 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:40.870+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:40.883 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-14T09:07:40.946 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:40.947+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:40.946 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:40.946 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:40.961 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:07:41.047 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-14T09:07:41.047 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:41.050+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:41.124 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:41.125 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:41.139 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:07:41.186 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:41.190+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:41.242 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-14T09:07:41.249 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:41.250+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:41.311 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:41.311 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:41.325 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:07:41.366 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:41.370+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:41.450 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:41.454+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:41.472 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-14T09:07:41.494 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:41.494 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:41.512 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:07:41.551 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:41.551+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:41.559 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:07:41.636 INFO:teuthology.orchestra.run.smithi176.stdout:5751 write 0x2ba33e0 thru 0x2ba620c (0x2e2d bytes) 2024-09-14T09:07:41.653 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:41.654+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:41.697 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:41.697 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:41.706 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:41.706 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:41.712 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:07:41.713 INFO:teuthology.orchestra.run.smithi176.stdout:5752 write 0x8b43fcb thru 0x8b4ad4f (0x6d85 bytes) 2024-09-14T09:07:41.723 INFO:teuthology.orchestra.run.smithi176.stdout:5753 read 0x5654f25 thru 0x565c124 (0x7200 bytes) 2024-09-14T09:07:41.723 INFO:teuthology.orchestra.run.smithi176.stdout:5754 write 0xc53d5d2 thru 0xc54808e (0xaabd bytes) 2024-09-14T09:07:41.724 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:41.724 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:41.740 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:41.740 INFO:teuthology.orchestra.run.smithi176.stdout:5757 read 0xb5e45a3 thru 0xb5ead6e (0x67cc bytes) 2024-09-14T09:07:41.741 INFO:teuthology.orchestra.run.smithi176.stdout:5758 read 0x6fa0e00 thru 0x6fa4282 (0x3483 bytes) 2024-09-14T09:07:41.741 INFO:teuthology.orchestra.run.smithi176.stdout:5759 punch from 0x2297b89 to 0x229c216, (0x468d bytes) 2024-09-14T09:07:41.741 INFO:teuthology.orchestra.run.smithi176.stdout:5760 trunc from 0xd306cc7 to 0xd2ee553 2024-09-14T09:07:41.751 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:41.751 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:41.754 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:41.758+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:41.769 INFO:teuthology.orchestra.run.smithi176.stdout:5761 write 0x1ae2d6f thru 0x1aed2f8 (0xa58a bytes) 2024-09-14T09:07:41.771 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:41.771 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:41.798 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:07:41.798 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:07:41.821 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:41.821 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:41.838 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:41.838 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:41.850 INFO:teuthology.orchestra.run.smithi176.stdout:5762 write 0xb1dd3a6 thru 0xb1e7eb3 (0xab0e bytes) 2024-09-14T09:07:41.854 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:41.855+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:41.856 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:41.856 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:41.858 INFO:teuthology.orchestra.run.smithi176.stdout:5763 read 0xbc8b4fe thru 0xbc8d7a9 (0x22ac bytes) 2024-09-14T09:07:41.859 INFO:teuthology.orchestra.run.smithi176.stdout:5765 writesame 0x731ef3c thru 0x7321b8c data_size 0x1(0x2c51 bytes) 2024-09-14T09:07:41.889 INFO:teuthology.orchestra.run.smithi176.stdout:5766 trunc from 0xd2ee553 to 0x59d5a3 2024-09-14T09:07:41.915 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:41.915 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:41.915 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:07:41.916 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:07:41.916 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:41.941 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:07:41.955 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:41.955+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:41.962 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T09:07:42.038 INFO:teuthology.orchestra.run.smithi176.stdout:5767 writesame 0x3384aa5 thru 0x338eb68 data_size 0x2(0xa0c4 bytes) 2024-09-14T09:07:42.040 INFO:teuthology.orchestra.run.smithi176.stdout:5769 punch from 0x32466f3 to 0x32468e4, (0x1f1 bytes) 2024-09-14T09:07:42.040 INFO:teuthology.orchestra.run.smithi176.stdout:5770 read 0xa46309 thru 0xa521d2 (0xbeca bytes) 2024-09-14T09:07:42.040 INFO:teuthology.orchestra.run.smithi176.stdout:5771 clone 43 order 21 su 65536 sc 5 2024-09-14T09:07:42.055 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:42.055+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:42.080 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:42.081 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:07:42.081 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:07:42.081 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:07:42.081 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.081 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38593 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [watch reconnect cookie 139718844451680 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:07:42.081 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:07:42.081 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 100.261933387, 2024-09-14T09:07:42.081 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0218296630000001, 2024-09-14T09:07:42.081 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.081 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.081 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:42.081 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.082 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.082 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.082 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38593 2024-09-14T09:07:42.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.082 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.082 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.082 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.082 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:07:42.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.082 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.082 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.082 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:07:42.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818420+0000", 2024-09-14T09:07:42.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8399999999999999e-06 2024-09-14T09:07:42.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818431+0000", 2024-09-14T09:07:42.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0859999999999999e-05 2024-09-14T09:07:42.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818466+0000", 2024-09-14T09:07:42.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4486999999999997e-05 2024-09-14T09:07:42.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.084 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818481+0000", 2024-09-14T09:07:42.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5362999999999998e-05 2024-09-14T09:07:42.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.084 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818659+0000", 2024-09-14T09:07:42.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017807500000000001 2024-09-14T09:07:42.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op must wait for map", 2024-09-14T09:07:42.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818681+0000", 2024-09-14T09:07:42.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2096999999999999e-05 2024-09-14T09:07:42.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825472+0000", 2024-09-14T09:07:42.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067905170000000003 2024-09-14T09:07:42.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:42.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825491+0000", 2024-09-14T09:07:42.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9372999999999999e-05 2024-09-14T09:07:42.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.086 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.086 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835402+0000", 2024-09-14T09:07:42.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0099111620000001 2024-09-14T09:07:42.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.086 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:42.086 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835548+0000", 2024-09-14T09:07:42.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014607300000000001 2024-09-14T09:07:42.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.086 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.086 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.840247+0000", 2024-09-14T09:07:42.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046988159999999998 2024-09-14T09:07:42.087 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.087 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.087 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.087 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38615 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:07:42.087 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:07:42.087 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 100.21356817900001, 2024-09-14T09:07:42.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.97014128899999996, 2024-09-14T09:07:42.087 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.087 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.087 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:42.087 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.088 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.088 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.088 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38615 2024-09-14T09:07:42.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.088 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:07:42.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:07:42.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866787+0000", 2024-09-14T09:07:42.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.476e-06 2024-09-14T09:07:42.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866806+0000", 2024-09-14T09:07:42.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8606000000000001e-05 2024-09-14T09:07:42.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866839+0000", 2024-09-14T09:07:42.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3510000000000003e-05 2024-09-14T09:07:42.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866867+0000", 2024-09-14T09:07:42.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7610000000000002e-05 2024-09-14T09:07:42.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867031+0000", 2024-09-14T09:07:42.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016414199999999999 2024-09-14T09:07:42.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867054+0000", 2024-09-14T09:07:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2693000000000001e-05 2024-09-14T09:07:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836462+0000", 2024-09-14T09:07:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.96940797700000003 2024-09-14T09:07:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836547+0000", 2024-09-14T09:07:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5294e-05 2024-09-14T09:07:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836924+0000", 2024-09-14T09:07:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037698099999999997 2024-09-14T09:07:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105723 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:07:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.985134134999999, 2024-09-14T09:07:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77512799700000001, 2024-09-14T09:07:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105723 2024-09-14T09:07:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:07:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:07:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:07:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1399999999999997e-07 2024-09-14T09:07:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095220+0000", 2024-09-14T09:07:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0510000000000001e-06 2024-09-14T09:07:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095228+0000", 2024-09-14T09:07:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4159999999999998e-06 2024-09-14T09:07:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095234+0000", 2024-09-14T09:07:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3010000000000003e-06 2024-09-14T09:07:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103902+0000", 2024-09-14T09:07:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086678299999999996 2024-09-14T09:07:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103905+0000", 2024-09-14T09:07:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5020000000000001e-06 2024-09-14T09:07:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870332+0000", 2024-09-14T09:07:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76642700900000005 2024-09-14T09:07:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870345+0000", 2024-09-14T09:07:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2274e-05 2024-09-14T09:07:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105724 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:07:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.985109622, 2024-09-14T09:07:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77515698099999997, 2024-09-14T09:07:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105724 2024-09-14T09:07:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:07:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:07:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095242+0000", 2024-09-14T09:07:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9400000000000001e-07 2024-09-14T09:07:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095243+0000", 2024-09-14T09:07:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7540000000000001e-06 2024-09-14T09:07:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095248+0000", 2024-09-14T09:07:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1320000000000004e-06 2024-09-14T09:07:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095251+0000", 2024-09-14T09:07:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2770000000000001e-06 2024-09-14T09:07:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103947+0000", 2024-09-14T09:07:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086960089999999993 2024-09-14T09:07:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103955+0000", 2024-09-14T09:07:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9519999999999994e-06 2024-09-14T09:07:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870386+0000", 2024-09-14T09:07:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76643141100000001 2024-09-14T09:07:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870398+0000", 2024-09-14T09:07:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2052000000000001e-05 2024-09-14T09:07:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105727 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:07:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.985069916, 2024-09-14T09:07:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517292900000001, 2024-09-14T09:07:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105727 2024-09-14T09:07:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:07:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:07:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:07:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3300000000000002e-07 2024-09-14T09:07:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095284+0000", 2024-09-14T09:07:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9790000000000002e-06 2024-09-14T09:07:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095292+0000", 2024-09-14T09:07:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3810000000000001e-06 2024-09-14T09:07:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095298+0000", 2024-09-14T09:07:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9710000000000001e-06 2024-09-14T09:07:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103987+0000", 2024-09-14T09:07:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086893820000000007 2024-09-14T09:07:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103991+0000", 2024-09-14T09:07:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5480000000000002e-06 2024-09-14T09:07:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870442+0000", 2024-09-14T09:07:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.766451145 2024-09-14T09:07:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870454+0000", 2024-09-14T09:07:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.199e-05 2024-09-14T09:07:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105729 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:07:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.985021559, 2024-09-14T09:07:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517686500000005, 2024-09-14T09:07:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105729 2024-09-14T09:07:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:07:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:07:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095330+0000", 2024-09-14T09:07:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-09-14T09:07:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095332+0000", 2024-09-14T09:07:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1560000000000002e-06 2024-09-14T09:07:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095338+0000", 2024-09-14T09:07:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.592e-06 2024-09-14T09:07:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095341+0000", 2024-09-14T09:07:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8139999999999998e-06 2024-09-14T09:07:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104031+0000", 2024-09-14T09:07:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086896330000000004 2024-09-14T09:07:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104039+0000", 2024-09-14T09:07:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0900000000000005e-06 2024-09-14T09:07:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870495+0000", 2024-09-14T09:07:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76645611199999997 2024-09-14T09:07:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870506+0000", 2024-09-14T09:07:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0986e-05 2024-09-14T09:07:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105731 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:07:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.985003292000002, 2024-09-14T09:07:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77520598600000001, 2024-09-14T09:07:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105731 2024-09-14T09:07:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:07:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:07:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:07:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8599999999999999e-07 2024-09-14T09:07:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095350+0000", 2024-09-14T09:07:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1110000000000002e-06 2024-09-14T09:07:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095357+0000", 2024-09-14T09:07:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9020000000000002e-06 2024-09-14T09:07:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095360+0000", 2024-09-14T09:07:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1939999999999998e-06 2024-09-14T09:07:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104073+0000", 2024-09-14T09:07:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087131210000000008 2024-09-14T09:07:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104077+0000", 2024-09-14T09:07:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.501e-06 2024-09-14T09:07:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870541+0000", 2024-09-14T09:07:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76646461700000001 2024-09-14T09:07:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870554+0000", 2024-09-14T09:07:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2153999999999999e-05 2024-09-14T09:07:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105733 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:07:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.984983261, 2024-09-14T09:07:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.775237277, 2024-09-14T09:07:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105733 2024-09-14T09:07:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:07:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:07:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:07:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8699999999999995e-07 2024-09-14T09:07:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095371+0000", 2024-09-14T09:07:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5450000000000002e-06 2024-09-14T09:07:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095377+0000", 2024-09-14T09:07:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.776e-06 2024-09-14T09:07:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095383+0000", 2024-09-14T09:07:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3129999999999998e-06 2024-09-14T09:07:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104124+0000", 2024-09-14T09:07:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087409840000000003 2024-09-14T09:07:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104132+0000", 2024-09-14T09:07:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3159999999999997e-06 2024-09-14T09:07:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870593+0000", 2024-09-14T09:07:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76646064800000002 2024-09-14T09:07:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870605+0000", 2024-09-14T09:07:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2208000000000001e-05 2024-09-14T09:07:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105734 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:07:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.984960179000002, 2024-09-14T09:07:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77526569599999995, 2024-09-14T09:07:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105734 2024-09-14T09:07:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:07:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:07:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:07:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7399999999999998e-07 2024-09-14T09:07:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095394+0000", 2024-09-14T09:07:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8030000000000001e-06 2024-09-14T09:07:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095401+0000", 2024-09-14T09:07:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.046e-06 2024-09-14T09:07:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095407+0000", 2024-09-14T09:07:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6060000000000002e-06 2024-09-14T09:07:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104167+0000", 2024-09-14T09:07:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087602530000000008 2024-09-14T09:07:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104171+0000", 2024-09-14T09:07:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7620000000000001e-06 2024-09-14T09:07:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870645+0000", 2024-09-14T09:07:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76647400600000004 2024-09-14T09:07:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870656+0000", 2024-09-14T09:07:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1746000000000001e-05 2024-09-14T09:07:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105738 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:07:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.984920452000001, 2024-09-14T09:07:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77527530700000002, 2024-09-14T09:07:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105738 2024-09-14T09:07:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:07:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:07:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095431+0000", 2024-09-14T09:07:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0999999999999999e-07 2024-09-14T09:07:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095434+0000", 2024-09-14T09:07:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.757e-06 2024-09-14T09:07:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095442+0000", 2024-09-14T09:07:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9929999999999995e-06 2024-09-14T09:07:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095448+0000", 2024-09-14T09:07:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1340000000000001e-06 2024-09-14T09:07:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104213+0000", 2024-09-14T09:07:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087650000000000002 2024-09-14T09:07:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104221+0000", 2024-09-14T09:07:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4589999999999998e-06 2024-09-14T09:07:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870694+0000", 2024-09-14T09:07:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76647303200000005 2024-09-14T09:07:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870706+0000", 2024-09-14T09:07:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1522000000000001e-05 2024-09-14T09:07:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105739 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:07:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.984895923, 2024-09-14T09:07:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77530221700000002, 2024-09-14T09:07:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105739 2024-09-14T09:07:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:07:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:07:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:07:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.15e-07 2024-09-14T09:07:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095457+0000", 2024-09-14T09:07:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6470000000000001e-06 2024-09-14T09:07:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095461+0000", 2024-09-14T09:07:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0910000000000003e-06 2024-09-14T09:07:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095464+0000", 2024-09-14T09:07:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2059999999999999e-06 2024-09-14T09:07:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104253+0000", 2024-09-14T09:07:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087884120000000007 2024-09-14T09:07:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104256+0000", 2024-09-14T09:07:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5439999999999999e-06 2024-09-14T09:07:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870745+0000", 2024-09-14T09:07:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76648908000000004 2024-09-14T09:07:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870757+0000", 2024-09-14T09:07:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1822e-05 2024-09-14T09:07:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105741 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:07:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.984826797, 2024-09-14T09:07:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77528469899999997, 2024-09-14T09:07:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105741 2024-09-14T09:07:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:07:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:07:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095525+0000", 2024-09-14T09:07:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3740000000000001e-06 2024-09-14T09:07:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095533+0000", 2024-09-14T09:07:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6380000000000004e-06 2024-09-14T09:07:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095549+0000", 2024-09-14T09:07:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5554999999999999e-05 2024-09-14T09:07:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095566+0000", 2024-09-14T09:07:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7146000000000002e-05 2024-09-14T09:07:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104297+0000", 2024-09-14T09:07:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087315750000000001 2024-09-14T09:07:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104305+0000", 2024-09-14T09:07:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1389999999999995e-06 2024-09-14T09:07:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870797+0000", 2024-09-14T09:07:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76649129599999999 2024-09-14T09:07:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870809+0000", 2024-09-14T09:07:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1976e-05 2024-09-14T09:07:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105742 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:07:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.984762013000001, 2024-09-14T09:07:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.775271873, 2024-09-14T09:07:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105742 2024-09-14T09:07:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:07:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:07:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095590+0000", 2024-09-14T09:07:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0400000000000005e-07 2024-09-14T09:07:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095594+0000", 2024-09-14T09:07:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5639999999999998e-06 2024-09-14T09:07:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095620+0000", 2024-09-14T09:07:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5494999999999999e-05 2024-09-14T09:07:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095639+0000", 2024-09-14T09:07:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9639999999999998e-05 2024-09-14T09:07:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104340+0000", 2024-09-14T09:07:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087004950000000008 2024-09-14T09:07:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104343+0000", 2024-09-14T09:07:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4020000000000002e-06 2024-09-14T09:07:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870850+0000", 2024-09-14T09:07:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76650648499999996 2024-09-14T09:07:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870861+0000", 2024-09-14T09:07:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0888e-05 2024-09-14T09:07:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105745 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:07:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.984663397999999, 2024-09-14T09:07:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77521998700000005, 2024-09-14T09:07:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105745 2024-09-14T09:07:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:07:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:07:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095688+0000", 2024-09-14T09:07:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2699999999999998e-07 2024-09-14T09:07:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095693+0000", 2024-09-14T09:07:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0479999999999998e-06 2024-09-14T09:07:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095707+0000", 2024-09-14T09:07:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.342e-05 2024-09-14T09:07:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095716+0000", 2024-09-14T09:07:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9800000000000004e-06 2024-09-14T09:07:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104388+0000", 2024-09-14T09:07:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086717349999999999 2024-09-14T09:07:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104396+0000", 2024-09-14T09:07:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1769999999999998e-06 2024-09-14T09:07:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870897+0000", 2024-09-14T09:07:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76650102600000003 2024-09-14T09:07:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870907+0000", 2024-09-14T09:07:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0674e-05 2024-09-14T09:07:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105747 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:07:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.984617178000001, 2024-09-14T09:07:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77522190300000005, 2024-09-14T09:07:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105747 2024-09-14T09:07:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:07:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:07:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:07:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8999999999999995e-07 2024-09-14T09:07:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095738+0000", 2024-09-14T09:07:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8430000000000003e-06 2024-09-14T09:07:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095750+0000", 2024-09-14T09:07:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1229999999999999e-05 2024-09-14T09:07:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095758+0000", 2024-09-14T09:07:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.337e-06 2024-09-14T09:07:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104431+0000", 2024-09-14T09:07:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086728780000000002 2024-09-14T09:07:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104434+0000", 2024-09-14T09:07:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5049999999999999e-06 2024-09-14T09:07:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870945+0000", 2024-09-14T09:07:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651031800000002 2024-09-14T09:07:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870956+0000", 2024-09-14T09:07:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1002000000000001e-05 2024-09-14T09:07:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105750 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:07:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.984547232000001, 2024-09-14T09:07:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77519961800000003, 2024-09-14T09:07:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105750 2024-09-14T09:07:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:07:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:07:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:07:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3600000000000002e-07 2024-09-14T09:07:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095809+0000", 2024-09-14T09:07:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4000000000000002e-06 2024-09-14T09:07:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095822+0000", 2024-09-14T09:07:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2704000000000001e-05 2024-09-14T09:07:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095831+0000", 2024-09-14T09:07:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9530000000000005e-06 2024-09-14T09:07:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104477+0000", 2024-09-14T09:07:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086459989999999997 2024-09-14T09:07:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104488+0000", 2024-09-14T09:07:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:07:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870992+0000", 2024-09-14T09:07:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.766504097 2024-09-14T09:07:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871003+0000", 2024-09-14T09:07:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1358999999999999e-05 2024-09-14T09:07:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105751 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:07:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.984489347, 2024-09-14T09:07:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77518966700000003, 2024-09-14T09:07:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105751 2024-09-14T09:07:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:07:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:07:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:07:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7300000000000005e-07 2024-09-14T09:07:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095867+0000", 2024-09-14T09:07:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2729999999999996e-06 2024-09-14T09:07:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095879+0000", 2024-09-14T09:07:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2917e-05 2024-09-14T09:07:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095888+0000", 2024-09-14T09:07:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2609999999999993e-06 2024-09-14T09:07:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104520+0000", 2024-09-14T09:07:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086322970000000006 2024-09-14T09:07:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104524+0000", 2024-09-14T09:07:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.54e-06 2024-09-14T09:07:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871040+0000", 2024-09-14T09:07:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651615399999995 2024-09-14T09:07:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871051+0000", 2024-09-14T09:07:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1452e-05 2024-09-14T09:07:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105752 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:07:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.984428583, 2024-09-14T09:07:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517684899999995, 2024-09-14T09:07:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105752 2024-09-14T09:07:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:07:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:07:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095923+0000", 2024-09-14T09:07:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3300000000000001e-07 2024-09-14T09:07:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095927+0000", 2024-09-14T09:07:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:07:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095940+0000", 2024-09-14T09:07:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2309e-05 2024-09-14T09:07:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095948+0000", 2024-09-14T09:07:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9199999999999993e-06 2024-09-14T09:07:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104566+0000", 2024-09-14T09:07:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086171449999999997 2024-09-14T09:07:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104574+0000", 2024-09-14T09:07:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0690000000000002e-06 2024-09-14T09:07:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871087+0000", 2024-09-14T09:07:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651370699999999 2024-09-14T09:07:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871099+0000", 2024-09-14T09:07:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1718e-05 2024-09-14T09:07:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105755 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:07:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.984370640000002, 2024-09-14T09:07:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77516787899999995, 2024-09-14T09:07:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105755 2024-09-14T09:07:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:07:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:07:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095981+0000", 2024-09-14T09:07:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2699999999999998e-07 2024-09-14T09:07:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095985+0000", 2024-09-14T09:07:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4349999999999999e-06 2024-09-14T09:07:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095999+0000", 2024-09-14T09:07:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3139000000000001e-05 2024-09-14T09:07:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096007+0000", 2024-09-14T09:07:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6610000000000006e-06 2024-09-14T09:07:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104591+0000", 2024-09-14T09:07:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0085841709999999998 2024-09-14T09:07:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104595+0000", 2024-09-14T09:07:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5590000000000001e-06 2024-09-14T09:07:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871137+0000", 2024-09-14T09:07:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76654202100000002 2024-09-14T09:07:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871148+0000", 2024-09-14T09:07:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1066e-05 2024-09-14T09:07:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105757 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:07:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 20.984312060000001, 2024-09-14T09:07:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77516401300000004, 2024-09-14T09:07:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105757 2024-09-14T09:07:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:07:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:07:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856228+0000", 2024-09-14T09:07:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76018875100000005 2024-09-14T09:07:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856240+0000", 2024-09-14T09:07:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-05 2024-09-14T09:07:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856268+0000", 2024-09-14T09:07:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7628999999999999e-05 2024-09-14T09:07:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856307+0000", 2024-09-14T09:07:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8967000000000001e-05 2024-09-14T09:07:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871192+0000", 2024-09-14T09:07:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.01488449 2024-09-14T09:07:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871203+0000", 2024-09-14T09:07:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1266e-05 2024-09-14T09:07:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.175 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.175 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.175 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:42.175 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:42.175 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:42.176 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:42.178+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:42.177 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:07:42.277 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:42.275+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:42.292 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:42.292 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:42.292 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:42.292 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:42.304 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:07:42.378 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:42.382+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:42.468 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:42.468 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:42.468 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:42.469 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:42.469 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:42.479 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:42.482+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:42.483 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:07:42.581 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:42.582+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:42.610 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:42.610 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:42.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:42.610 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:42.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.610 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:71052 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:07:42.610 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:07:42.610 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 72.871181209, 2024-09-14T09:07:42.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14854635599999999, 2024-09-14T09:07:42.610 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.611 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.611 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:42.611 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:42.611 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:42.611 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.611 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 71052 2024-09-14T09:07:42.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.611 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:07:42.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:07:42.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737570+0000", 2024-09-14T09:07:42.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4770000000000002e-06 2024-09-14T09:07:42.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737584+0000", 2024-09-14T09:07:42.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3457e-05 2024-09-14T09:07:42.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737608+0000", 2024-09-14T09:07:42.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3893999999999999e-05 2024-09-14T09:07:42.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737620+0000", 2024-09-14T09:07:42.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2157000000000001e-05 2024-09-14T09:07:42.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737738+0000", 2024-09-14T09:07:42.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118409 2024-09-14T09:07:42.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737892+0000", 2024-09-14T09:07:42.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154222 2024-09-14T09:07:42.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:07:42.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.749622+0000", 2024-09-14T09:07:42.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011729537 2024-09-14T09:07:42.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:42.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885934+0000", 2024-09-14T09:07:42.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13631222200000001 2024-09-14T09:07:42.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:42.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885996+0000", 2024-09-14T09:07:42.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1555000000000003e-05 2024-09-14T09:07:42.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:42.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886060+0000", 2024-09-14T09:07:42.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3980999999999996e-05 2024-09-14T09:07:42.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886114+0000", 2024-09-14T09:07:42.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4444999999999998e-05 2024-09-14T09:07:42.616 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.616 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.616 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.616 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:71052 3.1 e71/34 ondisk, result = 0)", 2024-09-14T09:07:42.617 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:07:42.617 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 72.828277830999994, 2024-09-14T09:07:42.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10565334799999999, 2024-09-14T09:07:42.617 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.617 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.617 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:42.617 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:07:42.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:07:42.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780473+0000", 2024-09-14T09:07:42.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8980000000000001e-06 2024-09-14T09:07:42.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780482+0000", 2024-09-14T09:07:42.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9080000000000001e-06 2024-09-14T09:07:42.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780511+0000", 2024-09-14T09:07:42.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9490999999999999e-05 2024-09-14T09:07:42.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780527+0000", 2024-09-14T09:07:42.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.508e-05 2024-09-14T09:07:42.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885978+0000", 2024-09-14T09:07:42.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.105451328 2024-09-14T09:07:42.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885989+0000", 2024-09-14T09:07:42.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0875e-05 2024-09-14T09:07:42.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886125+0000", 2024-09-14T09:07:42.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013576800000000001 2024-09-14T09:07:42.620 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.620 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.620 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.621 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:98007 3.12 3:4faf1f2a:::rbd_data.115e113dc8ca.000000000000004f:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:07:42.621 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:07:42.621 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.751305606999999, 2024-09-14T09:07:42.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.098233707000000003, 2024-09-14T09:07:42.621 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.621 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.621 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:42.621 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:42.621 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:42.621 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.621 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 98007 2024-09-14T09:07:42.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.622 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:07:42.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:07:42.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857446+0000", 2024-09-14T09:07:42.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7599999999999998e-06 2024-09-14T09:07:42.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857459+0000", 2024-09-14T09:07:42.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3123e-05 2024-09-14T09:07:42.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857491+0000", 2024-09-14T09:07:42.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1603000000000002e-05 2024-09-14T09:07:42.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857506+0000", 2024-09-14T09:07:42.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.537e-05 2024-09-14T09:07:42.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857694+0000", 2024-09-14T09:07:42.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018766599999999999 2024-09-14T09:07:42.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857955+0000", 2024-09-14T09:07:42.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000260979 2024-09-14T09:07:42.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:07:42.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.863125+0000", 2024-09-14T09:07:42.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.005170582 2024-09-14T09:07:42.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:42.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955268+0000", 2024-09-14T09:07:42.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092142659000000002 2024-09-14T09:07:42.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:42.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955385+0000", 2024-09-14T09:07:42.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116723 2024-09-14T09:07:42.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:42.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955553+0000", 2024-09-14T09:07:42.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016809299999999999 2024-09-14T09:07:42.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955677+0000", 2024-09-14T09:07:42.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012414899999999999 2024-09-14T09:07:42.626 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.626 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.626 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.627 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118153 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 445440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:42.627 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:07:42.627 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.770748232000001, 2024-09-14T09:07:42.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095358268999999996, 2024-09-14T09:07:42.627 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.627 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.627 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:42.627 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:42.627 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:42.627 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.627 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118153 2024-09-14T09:07:42.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.628 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:07:42.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:07:42.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:07:42.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0999999999999999e-07 2024-09-14T09:07:42.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838003+0000", 2024-09-14T09:07:42.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2000000000000001e-06 2024-09-14T09:07:42.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838009+0000", 2024-09-14T09:07:42.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.105e-06 2024-09-14T09:07:42.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838015+0000", 2024-09-14T09:07:42.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9030000000000001e-06 2024-09-14T09:07:42.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933049+0000", 2024-09-14T09:07:42.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095033513999999999 2024-09-14T09:07:42.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933113+0000", 2024-09-14T09:07:42.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3616999999999998e-05 2024-09-14T09:07:42.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933359+0000", 2024-09-14T09:07:42.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024652000000000002 2024-09-14T09:07:42.631 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.631 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.631 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.631 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118155 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 449536~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:42.632 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:07:42.632 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.770725707, 2024-09-14T09:07:42.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095668000000000003, 2024-09-14T09:07:42.632 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.632 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.632 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:42.632 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:42.632 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:42.632 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.632 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118155 2024-09-14T09:07:42.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.632 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:07:42.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:07:42.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838024+0000", 2024-09-14T09:07:42.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0400000000000002e-07 2024-09-14T09:07:42.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838026+0000", 2024-09-14T09:07:42.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.508e-06 2024-09-14T09:07:42.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838033+0000", 2024-09-14T09:07:42.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.392e-06 2024-09-14T09:07:42.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838038+0000", 2024-09-14T09:07:42.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1050000000000003e-06 2024-09-14T09:07:42.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933400+0000", 2024-09-14T09:07:42.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095362279999999994 2024-09-14T09:07:42.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933451+0000", 2024-09-14T09:07:42.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0927000000000002e-05 2024-09-14T09:07:42.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933691+0000", 2024-09-14T09:07:42.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024038400000000001 2024-09-14T09:07:42.636 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.636 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.636 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.636 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118158 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 453632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:42.636 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:07:42.636 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.770704685, 2024-09-14T09:07:42.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096024231000000002, 2024-09-14T09:07:42.637 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.637 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.637 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:42.637 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:42.637 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:42.637 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.637 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118158 2024-09-14T09:07:42.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.637 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:07:42.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:07:42.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838045+0000", 2024-09-14T09:07:42.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2199999999999999e-07 2024-09-14T09:07:42.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838047+0000", 2024-09-14T09:07:42.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1059999999999998e-06 2024-09-14T09:07:42.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838054+0000", 2024-09-14T09:07:42.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4089999999999997e-06 2024-09-14T09:07:42.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838059+0000", 2024-09-14T09:07:42.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8590000000000004e-06 2024-09-14T09:07:42.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933729+0000", 2024-09-14T09:07:42.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095670274999999999 2024-09-14T09:07:42.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933794+0000", 2024-09-14T09:07:42.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4846999999999998e-05 2024-09-14T09:07:42.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934069+0000", 2024-09-14T09:07:42.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027431299999999999 2024-09-14T09:07:42.640 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.640 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.640 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.641 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118160 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 457728~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:42.641 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:07:42.641 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.770682565, 2024-09-14T09:07:42.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096394812999999996, 2024-09-14T09:07:42.641 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.641 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.641 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:42.641 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:42.641 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:42.641 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.641 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118160 2024-09-14T09:07:42.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.642 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:07:42.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:07:42.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838067+0000", 2024-09-14T09:07:42.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9399999999999995e-07 2024-09-14T09:07:42.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838069+0000", 2024-09-14T09:07:42.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1579999999999999e-06 2024-09-14T09:07:42.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838077+0000", 2024-09-14T09:07:42.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6939999999999995e-06 2024-09-14T09:07:42.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838082+0000", 2024-09-14T09:07:42.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2290000000000002e-06 2024-09-14T09:07:42.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934106+0000", 2024-09-14T09:07:42.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096023874999999995 2024-09-14T09:07:42.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934158+0000", 2024-09-14T09:07:42.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2052000000000002e-05 2024-09-14T09:07:42.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934461+0000", 2024-09-14T09:07:42.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030331100000000001 2024-09-14T09:07:42.645 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.645 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.645 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.645 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118162 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 461824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:42.645 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:07:42.645 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.770659985, 2024-09-14T09:07:42.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096615294000000004, 2024-09-14T09:07:42.645 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.646 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.646 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:42.646 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:42.646 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:42.646 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.646 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118162 2024-09-14T09:07:42.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.646 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.646 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:07:42.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:07:42.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838090+0000", 2024-09-14T09:07:42.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9699999999999996e-07 2024-09-14T09:07:42.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838092+0000", 2024-09-14T09:07:42.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1119999999999999e-06 2024-09-14T09:07:42.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838098+0000", 2024-09-14T09:07:42.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4359999999999999e-06 2024-09-14T09:07:42.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838103+0000", 2024-09-14T09:07:42.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1200000000000001e-06 2024-09-14T09:07:42.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934504+0000", 2024-09-14T09:07:42.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096400885000000006 2024-09-14T09:07:42.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934596+0000", 2024-09-14T09:07:42.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1775999999999997e-05 2024-09-14T09:07:42.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934704+0000", 2024-09-14T09:07:42.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010846800000000001 2024-09-14T09:07:42.649 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.649 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.649 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.649 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118164 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 465920~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:42.649 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:07:42.649 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.770638725, 2024-09-14T09:07:42.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096951372999999993, 2024-09-14T09:07:42.650 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.650 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.650 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:42.650 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:42.650 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:42.650 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.650 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118164 2024-09-14T09:07:42.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.650 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:07:42.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:07:42.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838111+0000", 2024-09-14T09:07:42.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0100000000000005e-07 2024-09-14T09:07:42.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838113+0000", 2024-09-14T09:07:42.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0339999999999999e-06 2024-09-14T09:07:42.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838119+0000", 2024-09-14T09:07:42.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.409e-06 2024-09-14T09:07:42.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838124+0000", 2024-09-14T09:07:42.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9030000000000003e-06 2024-09-14T09:07:42.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934739+0000", 2024-09-14T09:07:42.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096615124999999996 2024-09-14T09:07:42.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934789+0000", 2024-09-14T09:07:42.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9554e-05 2024-09-14T09:07:42.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935062+0000", 2024-09-14T09:07:42.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000272847 2024-09-14T09:07:42.653 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.653 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.653 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.653 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118166 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 470016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:42.653 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:07:42.654 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.770617971, 2024-09-14T09:07:42.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097306410999999995, 2024-09-14T09:07:42.654 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.654 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.654 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:42.654 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:42.654 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:42.654 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.654 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118166 2024-09-14T09:07:42.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.654 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:07:42.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:07:42.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:07:42.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.03e-07 2024-09-14T09:07:42.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838133+0000", 2024-09-14T09:07:42.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9479999999999998e-06 2024-09-14T09:07:42.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838141+0000", 2024-09-14T09:07:42.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5739999999999998e-06 2024-09-14T09:07:42.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838146+0000", 2024-09-14T09:07:42.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3750000000000002e-06 2024-09-14T09:07:42.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935095+0000", 2024-09-14T09:07:42.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096948435999999999 2024-09-14T09:07:42.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935152+0000", 2024-09-14T09:07:42.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.694e-05 2024-09-14T09:07:42.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935437+0000", 2024-09-14T09:07:42.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028573500000000001 2024-09-14T09:07:42.657 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.657 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.657 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.658 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118168 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 474112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:42.658 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:07:42.658 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.77059575, 2024-09-14T09:07:42.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097658024999999996, 2024-09-14T09:07:42.658 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.658 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.658 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:42.658 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:42.658 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:42.658 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.658 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118168 2024-09-14T09:07:42.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.658 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.659 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:07:42.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.659 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:07:42.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.659 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838154+0000", 2024-09-14T09:07:42.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4200000000000001e-07 2024-09-14T09:07:42.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838160+0000", 2024-09-14T09:07:42.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8510000000000004e-06 2024-09-14T09:07:42.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838168+0000", 2024-09-14T09:07:42.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9899999999999997e-06 2024-09-14T09:07:42.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838172+0000", 2024-09-14T09:07:42.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.887e-06 2024-09-14T09:07:42.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935473+0000", 2024-09-14T09:07:42.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097300132999999997 2024-09-14T09:07:42.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935523+0000", 2024-09-14T09:07:42.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0862000000000002e-05 2024-09-14T09:07:42.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935811+0000", 2024-09-14T09:07:42.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028786000000000003 2024-09-14T09:07:42.662 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.662 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.662 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.662 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118170 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 478208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:42.662 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:07:42.662 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.770568939, 2024-09-14T09:07:42.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.098077529999999996, 2024-09-14T09:07:42.662 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.662 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.662 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:42.662 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:42.663 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:42.663 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.663 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118170 2024-09-14T09:07:42.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.663 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:07:42.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:07:42.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842901+0000", 2024-09-14T09:07:42.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004721416 2024-09-14T09:07:42.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842906+0000", 2024-09-14T09:07:42.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7199999999999997e-06 2024-09-14T09:07:42.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842917+0000", 2024-09-14T09:07:42.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0671e-05 2024-09-14T09:07:42.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842923+0000", 2024-09-14T09:07:42.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.088e-06 2024-09-14T09:07:42.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935857+0000", 2024-09-14T09:07:42.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092933624000000006 2024-09-14T09:07:42.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935958+0000", 2024-09-14T09:07:42.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100933 2024-09-14T09:07:42.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.936258+0000", 2024-09-14T09:07:42.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030007800000000001 2024-09-14T09:07:42.666 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.667 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.667 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.667 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118181 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 498688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:42.667 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:07:42.667 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.765762670000001, 2024-09-14T09:07:42.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095212643999999999, 2024-09-14T09:07:42.667 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.667 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.667 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:42.667 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:42.667 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:42.667 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.668 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118181 2024-09-14T09:07:42.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.668 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:07:42.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:07:42.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842987+0000", 2024-09-14T09:07:42.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.29e-07 2024-09-14T09:07:42.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842988+0000", 2024-09-14T09:07:42.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.229e-06 2024-09-14T09:07:42.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842993+0000", 2024-09-14T09:07:42.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.792e-06 2024-09-14T09:07:42.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842996+0000", 2024-09-14T09:07:42.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3349999999999998e-06 2024-09-14T09:07:42.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.937807+0000", 2024-09-14T09:07:42.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.094811585000000004 2024-09-14T09:07:42.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.937899+0000", 2024-09-14T09:07:42.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1719999999999996e-05 2024-09-14T09:07:42.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938199+0000", 2024-09-14T09:07:42.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000299754 2024-09-14T09:07:42.671 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.671 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.671 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.671 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118182 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 502784~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:42.671 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:07:42.671 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.765749493, 2024-09-14T09:07:42.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095557953000000001, 2024-09-14T09:07:42.671 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.671 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.671 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:42.671 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:42.672 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:42.672 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.672 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118182 2024-09-14T09:07:42.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.672 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:07:42.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:07:42.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:07:42.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2999999999999999e-07 2024-09-14T09:07:42.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843001+0000", 2024-09-14T09:07:42.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.652e-06 2024-09-14T09:07:42.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843005+0000", 2024-09-14T09:07:42.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2780000000000002e-06 2024-09-14T09:07:42.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843007+0000", 2024-09-14T09:07:42.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.553e-06 2024-09-14T09:07:42.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938240+0000", 2024-09-14T09:07:42.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095232864 2024-09-14T09:07:42.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938295+0000", 2024-09-14T09:07:42.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5095e-05 2024-09-14T09:07:42.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938557+0000", 2024-09-14T09:07:42.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026228100000000001 2024-09-14T09:07:42.675 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.675 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.675 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.675 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118184 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 506880~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:42.675 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:07:42.675 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.765738293, 2024-09-14T09:07:42.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095950246000000003, 2024-09-14T09:07:42.676 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.676 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.676 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:42.676 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:42.676 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:42.676 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.676 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118184 2024-09-14T09:07:42.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.676 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:07:42.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:07:42.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:07:42.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2399999999999999e-07 2024-09-14T09:07:42.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843012+0000", 2024-09-14T09:07:42.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.269e-06 2024-09-14T09:07:42.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843015+0000", 2024-09-14T09:07:42.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0469999999999998e-06 2024-09-14T09:07:42.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843018+0000", 2024-09-14T09:07:42.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.044e-06 2024-09-14T09:07:42.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938597+0000", 2024-09-14T09:07:42.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095578688999999994 2024-09-14T09:07:42.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938687+0000", 2024-09-14T09:07:42.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0284000000000005e-05 2024-09-14T09:07:42.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938961+0000", 2024-09-14T09:07:42.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027368899999999997 2024-09-14T09:07:42.679 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.679 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.679 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.679 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118186 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 510976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:42.679 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:07:42.679 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.765727332999999, 2024-09-14T09:07:42.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096286497999999998, 2024-09-14T09:07:42.680 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.680 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.680 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:42.680 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:42.680 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:42.680 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.680 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118186 2024-09-14T09:07:42.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.680 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:07:42.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:07:42.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:07:42.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.35e-07 2024-09-14T09:07:42.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843023+0000", 2024-09-14T09:07:42.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.195e-06 2024-09-14T09:07:42.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843026+0000", 2024-09-14T09:07:42.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.055e-06 2024-09-14T09:07:42.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843029+0000", 2024-09-14T09:07:42.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6539999999999999e-06 2024-09-14T09:07:42.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939000+0000", 2024-09-14T09:07:42.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095971476 2024-09-14T09:07:42.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939049+0000", 2024-09-14T09:07:42.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8986999999999997e-05 2024-09-14T09:07:42.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939308+0000", 2024-09-14T09:07:42.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000258896 2024-09-14T09:07:42.683 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.683 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.683 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.683 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118188 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 515072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:42.683 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:07:42.684 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.765716943999999, 2024-09-14T09:07:42.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096722717999999999, 2024-09-14T09:07:42.684 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.684 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.684 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:42.684 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:42.684 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:42.684 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.684 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118188 2024-09-14T09:07:42.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.684 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:07:42.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:07:42.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:07:42.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2999999999999999e-07 2024-09-14T09:07:42.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843034+0000", 2024-09-14T09:07:42.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T09:07:42.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843037+0000", 2024-09-14T09:07:42.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0989999999999999e-06 2024-09-14T09:07:42.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843039+0000", 2024-09-14T09:07:42.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.666e-06 2024-09-14T09:07:42.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939351+0000", 2024-09-14T09:07:42.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096311789999999994 2024-09-14T09:07:42.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939440+0000", 2024-09-14T09:07:42.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8843000000000003e-05 2024-09-14T09:07:42.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939755+0000", 2024-09-14T09:07:42.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031486399999999999 2024-09-14T09:07:42.688 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.688 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.688 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.688 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118191 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 519168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:42.688 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:07:42.688 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.765706250999999, 2024-09-14T09:07:42.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097080607999999999, 2024-09-14T09:07:42.688 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.688 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.689 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:42.689 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:42.689 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:42.689 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.689 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118191 2024-09-14T09:07:42.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.689 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:07:42.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:07:42.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:07:42.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2100000000000001e-07 2024-09-14T09:07:42.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843044+0000", 2024-09-14T09:07:42.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2249999999999999e-06 2024-09-14T09:07:42.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843047+0000", 2024-09-14T09:07:42.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0240000000000002e-06 2024-09-14T09:07:42.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843050+0000", 2024-09-14T09:07:42.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.508e-06 2024-09-14T09:07:42.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939791+0000", 2024-09-14T09:07:42.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096741399000000006 2024-09-14T09:07:42.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939849+0000", 2024-09-14T09:07:42.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7912999999999999e-05 2024-09-14T09:07:42.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940123+0000", 2024-09-14T09:07:42.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027431800000000001 2024-09-14T09:07:42.692 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.692 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.692 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.692 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118192 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 523264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:42.692 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:07:42.692 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.765695191000001, 2024-09-14T09:07:42.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097587224, 2024-09-14T09:07:42.693 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.693 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.693 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:42.693 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:42.693 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:42.693 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.693 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118192 2024-09-14T09:07:42.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.693 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:07:42.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:07:42.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844132+0000", 2024-09-14T09:07:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010777810000000001 2024-09-14T09:07:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844140+0000", 2024-09-14T09:07:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4470000000000006e-06 2024-09-14T09:07:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844155+0000", 2024-09-14T09:07:42.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4924e-05 2024-09-14T09:07:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844163+0000", 2024-09-14T09:07:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9980000000000003e-06 2024-09-14T09:07:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940199+0000", 2024-09-14T09:07:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096036047999999999 2024-09-14T09:07:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940282+0000", 2024-09-14T09:07:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3073999999999998e-05 2024-09-14T09:07:42.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940641+0000", 2024-09-14T09:07:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000358952 2024-09-14T09:07:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118197 3.17 3:ec5ec63a:::rbd_data.115e96ceebb2.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 3718221] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:07:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 10.760915776000001, 2024-09-14T09:07:42.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.141131069, 2024-09-14T09:07:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118197 2024-09-14T09:07:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:07:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:07:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847834+0000", 2024-09-14T09:07:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2500000000000004e-07 2024-09-14T09:07:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849227+0000", 2024-09-14T09:07:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0013932149999999999 2024-09-14T09:07:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849246+0000", 2024-09-14T09:07:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8958e-05 2024-09-14T09:07:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849254+0000", 2024-09-14T09:07:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2360000000000004e-06 2024-09-14T09:07:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940154+0000", 2024-09-14T09:07:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.090899468999999997 2024-09-14T09:07:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:42.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940312+0000", 2024-09-14T09:07:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000158326 2024-09-14T09:07:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:07:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940728+0000", 2024-09-14T09:07:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00041583100000000002 2024-09-14T09:07:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.962731+0000", 2024-09-14T09:07:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022002608999999999 2024-09-14T09:07:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:42.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988881+0000", 2024-09-14T09:07:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026150620999999999 2024-09-14T09:07:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988919+0000", 2024-09-14T09:07:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8108999999999997e-05 2024-09-14T09:07:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988964+0000", 2024-09-14T09:07:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4870000000000002e-05 2024-09-14T09:07:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.704 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:42.705 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:42.705 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:42.706 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:07:42.706 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:42.710+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:42.807 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:42.810+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:42.836 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:42.836 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:42.836 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:42.836 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:42.850 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:07:42.908 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:42.907+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:42.949 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is unset 2024-09-14T09:07:42.974 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:42.974 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:42.974 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:42.974 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:42.974 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:42.989 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:07:43.009 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:43.010+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:43.045 INFO:teuthology.orchestra.run.smithi176.stdout:truncating image image_client.0-clone42 from 0x338eb69 (overlap 0x59d5a3) to 0x1af34a 2024-09-14T09:07:43.110 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:43.111+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:43.144 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-14T09:07:43.156 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73542 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15319040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.885850223999995, 2024-09-14T09:07:43.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0237091359999999, 2024-09-14T09:07:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73542 2024-09-14T09:07:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268762+0000", 2024-09-14T09:07:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268766+0000", 2024-09-14T09:07:43.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5880000000000002e-06 2024-09-14T09:07:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268776+0000", 2024-09-14T09:07:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0168e-05 2024-09-14T09:07:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268789+0000", 2024-09-14T09:07:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:07:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282053+0000", 2024-09-14T09:07:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013264496000000001 2024-09-14T09:07:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282057+0000", 2024-09-14T09:07:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6090000000000001e-06 2024-09-14T09:07:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292052+0000", 2024-09-14T09:07:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0099950070000001 2024-09-14T09:07:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292262+0000", 2024-09-14T09:07:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021009800000000001 2024-09-14T09:07:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292470+0000", 2024-09-14T09:07:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000208314 2024-09-14T09:07:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73543 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:43.161 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.885811434000004, 2024-09-14T09:07:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023985473, 2024-09-14T09:07:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73543 2024-09-14T09:07:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:43.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268801+0000", 2024-09-14T09:07:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268803+0000", 2024-09-14T09:07:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7439999999999999e-06 2024-09-14T09:07:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268812+0000", 2024-09-14T09:07:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7080000000000003e-06 2024-09-14T09:07:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268819+0000", 2024-09-14T09:07:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T09:07:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282068+0000", 2024-09-14T09:07:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013249540000000001 2024-09-14T09:07:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282072+0000", 2024-09-14T09:07:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0960000000000001e-06 2024-09-14T09:07:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292573+0000", 2024-09-14T09:07:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0105014560000001 2024-09-14T09:07:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292679+0000", 2024-09-14T09:07:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010545899999999999 2024-09-14T09:07:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292786+0000", 2024-09-14T09:07:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107052 2024-09-14T09:07:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.166 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73544 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.885783731999993, 2024-09-14T09:07:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0244830549999999, 2024-09-14T09:07:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73544 2024-09-14T09:07:43.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5599999999999995e-07 2024-09-14T09:07:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268831+0000", 2024-09-14T09:07:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.57e-06 2024-09-14T09:07:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268844+0000", 2024-09-14T09:07:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2811e-05 2024-09-14T09:07:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268851+0000", 2024-09-14T09:07:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9140000000000002e-06 2024-09-14T09:07:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282083+0000", 2024-09-14T09:07:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013232111 2024-09-14T09:07:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282086+0000", 2024-09-14T09:07:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2909999999999999e-06 2024-09-14T09:07:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292884+0000", 2024-09-14T09:07:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0107975069999999 2024-09-14T09:07:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293100+0000", 2024-09-14T09:07:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021639399999999999 2024-09-14T09:07:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293311+0000", 2024-09-14T09:07:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021090099999999999 2024-09-14T09:07:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.171 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73545 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15564800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.885732547000003, 2024-09-14T09:07:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0249746740000001, 2024-09-14T09:07:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.172 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73545 2024-09-14T09:07:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268880+0000", 2024-09-14T09:07:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5600000000000005e-07 2024-09-14T09:07:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268889+0000", 2024-09-14T09:07:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8559999999999996e-06 2024-09-14T09:07:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268913+0000", 2024-09-14T09:07:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3958e-05 2024-09-14T09:07:43.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268921+0000", 2024-09-14T09:07:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7479999999999999e-06 2024-09-14T09:07:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282098+0000", 2024-09-14T09:07:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013176297999999999 2024-09-14T09:07:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282101+0000", 2024-09-14T09:07:43.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2329999999999998e-06 2024-09-14T09:07:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293435+0000", 2024-09-14T09:07:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0113339459999999 2024-09-14T09:07:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293648+0000", 2024-09-14T09:07:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021307100000000001 2024-09-14T09:07:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293854+0000", 2024-09-14T09:07:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020580800000000001 2024-09-14T09:07:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73546 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15646720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.885678675999998, 2024-09-14T09:07:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0254846360000001, 2024-09-14T09:07:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.177 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73546 2024-09-14T09:07:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:43.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268934+0000", 2024-09-14T09:07:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7999999999999995e-07 2024-09-14T09:07:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268937+0000", 2024-09-14T09:07:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9840000000000001e-06 2024-09-14T09:07:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268947+0000", 2024-09-14T09:07:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0522e-05 2024-09-14T09:07:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268955+0000", 2024-09-14T09:07:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8059999999999995e-06 2024-09-14T09:07:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282112+0000", 2024-09-14T09:07:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013157189 2024-09-14T09:07:43.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282115+0000", 2024-09-14T09:07:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2949999999999998e-06 2024-09-14T09:07:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293968+0000", 2024-09-14T09:07:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0118530990000001 2024-09-14T09:07:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294197+0000", 2024-09-14T09:07:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228507 2024-09-14T09:07:43.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294417+0000", 2024-09-14T09:07:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000220554 2024-09-14T09:07:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73547 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.885646230000006, 2024-09-14T09:07:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.025970525, 2024-09-14T09:07:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.182 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73547 2024-09-14T09:07:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268966+0000", 2024-09-14T09:07:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3799999999999997e-07 2024-09-14T09:07:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268969+0000", 2024-09-14T09:07:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8739999999999999e-06 2024-09-14T09:07:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268979+0000", 2024-09-14T09:07:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9939999999999995e-06 2024-09-14T09:07:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268986+0000", 2024-09-14T09:07:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3459999999999996e-06 2024-09-14T09:07:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282127+0000", 2024-09-14T09:07:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01314039 2024-09-14T09:07:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282130+0000", 2024-09-14T09:07:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2219999999999998e-06 2024-09-14T09:07:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294540+0000", 2024-09-14T09:07:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124105880000001 2024-09-14T09:07:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294759+0000", 2024-09-14T09:07:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000218798 2024-09-14T09:07:43.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294936+0000", 2024-09-14T09:07:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000176675 2024-09-14T09:07:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73548 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15810560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.885615118000004, 2024-09-14T09:07:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0265045129999999, 2024-09-14T09:07:43.187 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73548 2024-09-14T09:07:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268997+0000", 2024-09-14T09:07:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7599999999999996e-07 2024-09-14T09:07:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269000+0000", 2024-09-14T09:07:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0580000000000002e-06 2024-09-14T09:07:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269010+0000", 2024-09-14T09:07:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124e-05 2024-09-14T09:07:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269018+0000", 2024-09-14T09:07:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5399999999999998e-06 2024-09-14T09:07:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282141+0000", 2024-09-14T09:07:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013123012 2024-09-14T09:07:43.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282144+0000", 2024-09-14T09:07:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.196e-06 2024-09-14T09:07:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295042+0000", 2024-09-14T09:07:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0128975689999999 2024-09-14T09:07:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295272+0000", 2024-09-14T09:07:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023048099999999999 2024-09-14T09:07:43.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295501+0000", 2024-09-14T09:07:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228657 2024-09-14T09:07:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73549 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15892480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.885569239000006, 2024-09-14T09:07:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0268354099999999, 2024-09-14T09:07:43.192 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73549 2024-09-14T09:07:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269043+0000", 2024-09-14T09:07:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6199999999999997e-07 2024-09-14T09:07:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269046+0000", 2024-09-14T09:07:43.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2480000000000001e-06 2024-09-14T09:07:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269057+0000", 2024-09-14T09:07:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0436e-05 2024-09-14T09:07:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269064+0000", 2024-09-14T09:07:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5179999999999998e-06 2024-09-14T09:07:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282155+0000", 2024-09-14T09:07:43.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013091190000000001 2024-09-14T09:07:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282159+0000", 2024-09-14T09:07:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.366e-06 2024-09-14T09:07:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295599+0000", 2024-09-14T09:07:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0134405559999999 2024-09-14T09:07:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295765+0000", 2024-09-14T09:07:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016575499999999999 2024-09-14T09:07:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295878+0000", 2024-09-14T09:07:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112679 2024-09-14T09:07:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.197 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73550 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15974400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:43.198 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:43.198 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.885536638999994, 2024-09-14T09:07:43.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027357657, 2024-09-14T09:07:43.198 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.198 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.198 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.198 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.198 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.198 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.198 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73550 2024-09-14T09:07:43.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.198 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:43.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:43.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269076+0000", 2024-09-14T09:07:43.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:07:43.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269079+0000", 2024-09-14T09:07:43.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:07:43.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269098+0000", 2024-09-14T09:07:43.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9698e-05 2024-09-14T09:07:43.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269106+0000", 2024-09-14T09:07:43.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6179999999999997e-06 2024-09-14T09:07:43.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282170+0000", 2024-09-14T09:07:43.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013063874 2024-09-14T09:07:43.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282173+0000", 2024-09-14T09:07:43.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1999999999999999e-06 2024-09-14T09:07:43.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295996+0000", 2024-09-14T09:07:43.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.013823151 2024-09-14T09:07:43.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296214+0000", 2024-09-14T09:07:43.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021817700000000001 2024-09-14T09:07:43.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296433+0000", 2024-09-14T09:07:43.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021829099999999999 2024-09-14T09:07:43.202 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.202 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.202 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.202 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73551 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:43.203 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:43.203 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.885495222000003, 2024-09-14T09:07:43.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027730276, 2024-09-14T09:07:43.203 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.203 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.203 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.203 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.203 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.203 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.203 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73551 2024-09-14T09:07:43.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.203 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:43.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:43.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269117+0000", 2024-09-14T09:07:43.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:07:43.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269120+0000", 2024-09-14T09:07:43.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.056e-06 2024-09-14T09:07:43.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269130+0000", 2024-09-14T09:07:43.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0084e-05 2024-09-14T09:07:43.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269137+0000", 2024-09-14T09:07:43.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1359999999999999e-06 2024-09-14T09:07:43.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282184+0000", 2024-09-14T09:07:43.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013046537 2024-09-14T09:07:43.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282188+0000", 2024-09-14T09:07:43.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3220000000000003e-06 2024-09-14T09:07:43.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296529+0000", 2024-09-14T09:07:43.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0143406109999999 2024-09-14T09:07:43.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296695+0000", 2024-09-14T09:07:43.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016604799999999999 2024-09-14T09:07:43.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296847+0000", 2024-09-14T09:07:43.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000151822 2024-09-14T09:07:43.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.207 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.207 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73552 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:43.208 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:43.208 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.885464642000002, 2024-09-14T09:07:43.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0281767500000001, 2024-09-14T09:07:43.208 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.208 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.208 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.208 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73552 2024-09-14T09:07:43.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.208 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:43.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:43.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:43.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:07:43.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269150+0000", 2024-09-14T09:07:43.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9119999999999998e-06 2024-09-14T09:07:43.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269160+0000", 2024-09-14T09:07:43.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6819999999999998e-06 2024-09-14T09:07:43.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269168+0000", 2024-09-14T09:07:43.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6079999999999998e-06 2024-09-14T09:07:43.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282205+0000", 2024-09-14T09:07:43.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013036982000000001 2024-09-14T09:07:43.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282208+0000", 2024-09-14T09:07:43.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4620000000000001e-06 2024-09-14T09:07:43.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296968+0000", 2024-09-14T09:07:43.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0147601209999999 2024-09-14T09:07:43.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297115+0000", 2024-09-14T09:07:43.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014673900000000001 2024-09-14T09:07:43.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297324+0000", 2024-09-14T09:07:43.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020866400000000001 2024-09-14T09:07:43.212 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.212 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.212 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.213 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73553 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:43.213 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:43.213 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.885434578000002, 2024-09-14T09:07:43.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0286556739999999, 2024-09-14T09:07:43.213 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.213 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.213 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.213 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.213 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.213 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.213 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73553 2024-09-14T09:07:43.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.213 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:43.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:43.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269178+0000", 2024-09-14T09:07:43.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:07:43.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269180+0000", 2024-09-14T09:07:43.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T09:07:43.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269189+0000", 2024-09-14T09:07:43.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2e-06 2024-09-14T09:07:43.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269197+0000", 2024-09-14T09:07:43.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2980000000000003e-06 2024-09-14T09:07:43.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282219+0000", 2024-09-14T09:07:43.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013022687 2024-09-14T09:07:43.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282223+0000", 2024-09-14T09:07:43.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.241e-06 2024-09-14T09:07:43.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297470+0000", 2024-09-14T09:07:43.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.015247778 2024-09-14T09:07:43.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297704+0000", 2024-09-14T09:07:43.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000233367 2024-09-14T09:07:43.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297833+0000", 2024-09-14T09:07:43.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012888900000000001 2024-09-14T09:07:43.217 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.217 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.217 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.217 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73554 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:43.218 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:43.218 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.885405261000002, 2024-09-14T09:07:43.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.029005113, 2024-09-14T09:07:43.218 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.218 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.218 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.218 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.218 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.218 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.218 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73554 2024-09-14T09:07:43.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.218 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:43.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:43.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269207+0000", 2024-09-14T09:07:43.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.68e-07 2024-09-14T09:07:43.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269210+0000", 2024-09-14T09:07:43.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8100000000000002e-06 2024-09-14T09:07:43.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269219+0000", 2024-09-14T09:07:43.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:07:43.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269227+0000", 2024-09-14T09:07:43.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3839999999999999e-06 2024-09-14T09:07:43.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282234+0000", 2024-09-14T09:07:43.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013007187 2024-09-14T09:07:43.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282237+0000", 2024-09-14T09:07:43.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1200000000000002e-06 2024-09-14T09:07:43.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297942+0000", 2024-09-14T09:07:43.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0157048040000001 2024-09-14T09:07:43.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298087+0000", 2024-09-14T09:07:43.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014499099999999999 2024-09-14T09:07:43.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298211+0000", 2024-09-14T09:07:43.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012461899999999999 2024-09-14T09:07:43.222 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.222 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.222 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.222 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73555 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:43.222 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:43.223 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.885375413000006, 2024-09-14T09:07:43.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.02941558, 2024-09-14T09:07:43.223 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.223 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.223 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.223 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73555 2024-09-14T09:07:43.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.223 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:43.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:43.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269237+0000", 2024-09-14T09:07:43.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3399999999999999e-07 2024-09-14T09:07:43.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269240+0000", 2024-09-14T09:07:43.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8480000000000001e-06 2024-09-14T09:07:43.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269249+0000", 2024-09-14T09:07:43.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1880000000000008e-06 2024-09-14T09:07:43.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269256+0000", 2024-09-14T09:07:43.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:07:43.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282248+0000", 2024-09-14T09:07:43.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012992327 2024-09-14T09:07:43.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282251+0000", 2024-09-14T09:07:43.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2160000000000002e-06 2024-09-14T09:07:43.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298299+0000", 2024-09-14T09:07:43.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0160479330000001 2024-09-14T09:07:43.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298446+0000", 2024-09-14T09:07:43.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000146525 2024-09-14T09:07:43.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298652+0000", 2024-09-14T09:07:43.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020584100000000001 2024-09-14T09:07:43.227 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.227 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.227 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.227 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73556 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:43.227 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:43.227 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.885345666999996, 2024-09-14T09:07:43.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0299854239999999, 2024-09-14T09:07:43.228 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.228 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.228 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.228 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.228 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.228 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.228 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73556 2024-09-14T09:07:43.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.228 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:43.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:43.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:43.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:43.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269270+0000", 2024-09-14T09:07:43.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0920000000000002e-06 2024-09-14T09:07:43.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269280+0000", 2024-09-14T09:07:43.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0013999999999999e-05 2024-09-14T09:07:43.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269288+0000", 2024-09-14T09:07:43.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:07:43.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282263+0000", 2024-09-14T09:07:43.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012974703000000001 2024-09-14T09:07:43.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282266+0000", 2024-09-14T09:07:43.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1520000000000001e-06 2024-09-14T09:07:43.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298729+0000", 2024-09-14T09:07:43.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.016463506 2024-09-14T09:07:43.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298884+0000", 2024-09-14T09:07:43.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015450800000000001 2024-09-14T09:07:43.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299251+0000", 2024-09-14T09:07:43.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036759899999999997 2024-09-14T09:07:43.232 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.232 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.232 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.232 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73557 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16547840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:43.232 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:43.232 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.885299016999994, 2024-09-14T09:07:43.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0307969480000001, 2024-09-14T09:07:43.233 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.233 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.233 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.233 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.233 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.233 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.233 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73557 2024-09-14T09:07:43.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.233 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:43.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:43.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269313+0000", 2024-09-14T09:07:43.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.92e-07 2024-09-14T09:07:43.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269336+0000", 2024-09-14T09:07:43.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3118000000000001e-05 2024-09-14T09:07:43.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269346+0000", 2024-09-14T09:07:43.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4069999999999999e-06 2024-09-14T09:07:43.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269352+0000", 2024-09-14T09:07:43.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5810000000000002e-06 2024-09-14T09:07:43.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282277+0000", 2024-09-14T09:07:43.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012924658 2024-09-14T09:07:43.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282280+0000", 2024-09-14T09:07:43.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2880000000000001e-06 2024-09-14T09:07:43.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299343+0000", 2024-09-14T09:07:43.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0170630190000001 2024-09-14T09:07:43.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299554+0000", 2024-09-14T09:07:43.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021085099999999999 2024-09-14T09:07:43.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300109+0000", 2024-09-14T09:07:43.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00055533399999999999 2024-09-14T09:07:43.237 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.237 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.237 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.237 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73558 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:43.237 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:43.238 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.885247394000004, 2024-09-14T09:07:43.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0314849340000001, 2024-09-14T09:07:43.238 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.238 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.238 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.238 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.238 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.238 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.238 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73558 2024-09-14T09:07:43.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.238 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:43.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:43.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269365+0000", 2024-09-14T09:07:43.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7400000000000003e-07 2024-09-14T09:07:43.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269367+0000", 2024-09-14T09:07:43.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4930000000000001e-06 2024-09-14T09:07:43.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269374+0000", 2024-09-14T09:07:43.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9519999999999997e-06 2024-09-14T09:07:43.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269379+0000", 2024-09-14T09:07:43.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9710000000000003e-06 2024-09-14T09:07:43.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282292+0000", 2024-09-14T09:07:43.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012912704000000001 2024-09-14T09:07:43.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282295+0000", 2024-09-14T09:07:43.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.179e-06 2024-09-14T09:07:43.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300195+0000", 2024-09-14T09:07:43.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.017899975 2024-09-14T09:07:43.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300345+0000", 2024-09-14T09:07:43.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014957499999999999 2024-09-14T09:07:43.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300849+0000", 2024-09-14T09:07:43.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050451100000000002 2024-09-14T09:07:43.242 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.242 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.242 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.242 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73559 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:43.242 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:43.243 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.885224984999994, 2024-09-14T09:07:43.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.031916343, 2024-09-14T09:07:43.243 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.243 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.243 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.243 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.243 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.243 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.243 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73559 2024-09-14T09:07:43.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.243 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:43.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:43.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:43.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3500000000000002e-07 2024-09-14T09:07:43.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269390+0000", 2024-09-14T09:07:43.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5890000000000001e-06 2024-09-14T09:07:43.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269396+0000", 2024-09-14T09:07:43.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3300000000000004e-06 2024-09-14T09:07:43.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269401+0000", 2024-09-14T09:07:43.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0340000000000004e-06 2024-09-14T09:07:43.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282311+0000", 2024-09-14T09:07:43.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012910204 2024-09-14T09:07:43.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282317+0000", 2024-09-14T09:07:43.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2639999999999997e-06 2024-09-14T09:07:43.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300933+0000", 2024-09-14T09:07:43.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0186160710000001 2024-09-14T09:07:43.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301158+0000", 2024-09-14T09:07:43.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225021 2024-09-14T09:07:43.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301303+0000", 2024-09-14T09:07:43.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014439499999999999 2024-09-14T09:07:43.247 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.247 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.247 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.247 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106964 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:43.247 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:07:43.247 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 22.039297151, 2024-09-14T09:07:43.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8802924409999999, 2024-09-14T09:07:43.248 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.248 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.248 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:43.248 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.248 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.248 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.248 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106964 2024-09-14T09:07:43.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.248 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:07:43.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:07:43.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115317+0000", 2024-09-14T09:07:43.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6429999999999999e-06 2024-09-14T09:07:43.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115326+0000", 2024-09-14T09:07:43.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2140000000000002e-06 2024-09-14T09:07:43.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115346+0000", 2024-09-14T09:07:43.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9419000000000002e-05 2024-09-14T09:07:43.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115357+0000", 2024-09-14T09:07:43.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1014e-05 2024-09-14T09:07:43.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115431+0000", 2024-09-14T09:07:43.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4040000000000003e-05 2024-09-14T09:07:43.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115438+0000", 2024-09-14T09:07:43.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9920000000000001e-06 2024-09-14T09:07:43.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844944+0000", 2024-09-14T09:07:43.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72950628399999995 2024-09-14T09:07:43.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844961+0000", 2024-09-14T09:07:43.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7156e-05 2024-09-14T09:07:43.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988004+0000", 2024-09-14T09:07:43.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.14304297 2024-09-14T09:07:43.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988511+0000", 2024-09-14T09:07:43.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050694100000000001 2024-09-14T09:07:43.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:43.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.989802+0000", 2024-09-14T09:07:43.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001291195 2024-09-14T09:07:43.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:43.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994866+0000", 2024-09-14T09:07:43.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050632760000000002 2024-09-14T09:07:43.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:43.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994988+0000", 2024-09-14T09:07:43.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012252899999999999 2024-09-14T09:07:43.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:43.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995198+0000", 2024-09-14T09:07:43.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021010899999999999 2024-09-14T09:07:43.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995607+0000", 2024-09-14T09:07:43.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040865900000000002 2024-09-14T09:07:43.254 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.254 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.254 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.254 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106965 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:43.254 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:07:43.254 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 22.038094774000001, 2024-09-14T09:07:43.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.879730162, 2024-09-14T09:07:43.255 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.255 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.255 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:43.255 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.255 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.255 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.255 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106965 2024-09-14T09:07:43.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.255 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:07:43.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:07:43.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:07:43.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:07:43.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.256 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.256 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116522+0000", 2024-09-14T09:07:43.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3989999999999997e-06 2024-09-14T09:07:43.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116533+0000", 2024-09-14T09:07:43.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0669000000000001e-05 2024-09-14T09:07:43.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116541+0000", 2024-09-14T09:07:43.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2730000000000002e-06 2024-09-14T09:07:43.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116604+0000", 2024-09-14T09:07:43.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2706999999999994e-05 2024-09-14T09:07:43.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116610+0000", 2024-09-14T09:07:43.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8630000000000002e-06 2024-09-14T09:07:43.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845060+0000", 2024-09-14T09:07:43.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72845003699999999 2024-09-14T09:07:43.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845079+0000", 2024-09-14T09:07:43.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8287999999999998e-05 2024-09-14T09:07:43.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.991866+0000", 2024-09-14T09:07:43.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1467871519999999 2024-09-14T09:07:43.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.992274+0000", 2024-09-14T09:07:43.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040813500000000001 2024-09-14T09:07:43.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:43.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.993538+0000", 2024-09-14T09:07:43.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001263655 2024-09-14T09:07:43.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:43.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995770+0000", 2024-09-14T09:07:43.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002231949 2024-09-14T09:07:43.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:43.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995810+0000", 2024-09-14T09:07:43.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0852999999999997e-05 2024-09-14T09:07:43.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:43.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995979+0000", 2024-09-14T09:07:43.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016866 2024-09-14T09:07:43.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996247+0000", 2024-09-14T09:07:43.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000267769 2024-09-14T09:07:43.261 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.261 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.261 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.261 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.261 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.261 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:43.263 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:07:43.263 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:43.263+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:43.364 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:43.363+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:43.428 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:43.428 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:43.428 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:43.428 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:43.445 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:07:43.464 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:43.466+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:43.566 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:43.567+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:43.589 INFO:teuthology.orchestra.run.smithi176.stdout:5773 trunc from 0x338eb69 to 0xa5b561a 2024-09-14T09:07:43.613 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:43.614 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:43.614 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:43.614 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:43.614 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:43.633 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:07:43.635 INFO:teuthology.orchestra.run.smithi176.stdout:5774 write 0xe9919b9 thru 0xe99ba6f (0xa0b7 bytes) 2024-09-14T09:07:43.643 INFO:teuthology.orchestra.run.smithi176.stdout:5775 writesame 0x31cd956 thru 0x31db2f3 data_size 0x2d(0xd99e bytes) 2024-09-14T09:07:43.660 INFO:teuthology.orchestra.run.smithi176.stdout:5776 writesame 0xdaebe5a thru 0xdaefcbe data_size 0x1(0x3e65 bytes) 2024-09-14T09:07:43.667 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:43.670+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:43.695 INFO:teuthology.orchestra.run.smithi176.stdout:5777 punch from 0x6c3a4f0 to 0x6c3c6c3, (0x21d3 bytes) 2024-09-14T09:07:43.695 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:43.695 INFO:teuthology.orchestra.run.smithi176.stdout:5779 writesame 0xdd794d5 thru 0xdd7a522 data_size 0x1(0x104e bytes) 2024-09-14T09:07:43.705 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:43.705 INFO:teuthology.orchestra.run.smithi176.stdout:5781 writesame 0xa30dbfb thru 0xa30eeca data_size 0x7(0x12d0 bytes) 2024-09-14T09:07:43.712 INFO:teuthology.orchestra.run.smithi176.stdout:5782 writesame 0x388f5f0 thru 0x389a68c data_size 0x7(0xb09d bytes) 2024-09-14T09:07:43.750 INFO:teuthology.orchestra.run.smithi176.stdout:5783 write 0x9c3eb45 thru 0x9c485ad (0x9a69 bytes) 2024-09-14T09:07:43.764 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:43.764 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:43.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:43.765 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:43.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.765 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106705 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 518144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.765 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:07:43.765 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914817723999999, 2024-09-14T09:07:43.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3001326799999999, 2024-09-14T09:07:43.765 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.765 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.765 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.765 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.765 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.765 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.765 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106705 2024-09-14T09:07:43.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.766 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:07:43.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:07:43.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:07:43.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4699999999999998e-07 2024-09-14T09:07:43.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847500+0000", 2024-09-14T09:07:43.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0860000000000001e-06 2024-09-14T09:07:43.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847503+0000", 2024-09-14T09:07:43.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1269999999999999e-06 2024-09-14T09:07:43.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847506+0000", 2024-09-14T09:07:43.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5780000000000001e-06 2024-09-14T09:07:43.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858911+0000", 2024-09-14T09:07:43.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011404925 2024-09-14T09:07:43.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858919+0000", 2024-09-14T09:07:43.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8620000000000003e-06 2024-09-14T09:07:43.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147254+0000", 2024-09-14T09:07:43.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2883353719999999 2024-09-14T09:07:43.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147408+0000", 2024-09-14T09:07:43.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015389200000000001 2024-09-14T09:07:43.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147631+0000", 2024-09-14T09:07:43.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022359100000000001 2024-09-14T09:07:43.769 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.769 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.769 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.770 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106707 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 522240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.770 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:07:43.770 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914807253999999, 2024-09-14T09:07:43.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.300471801, 2024-09-14T09:07:43.770 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.770 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.770 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.770 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.770 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.770 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.770 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106707 2024-09-14T09:07:43.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.770 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:07:43.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:07:43.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:07:43.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.23e-07 2024-09-14T09:07:43.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847514+0000", 2024-09-14T09:07:43.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9150000000000004e-06 2024-09-14T09:07:43.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847520+0000", 2024-09-14T09:07:43.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.203e-06 2024-09-14T09:07:43.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847532+0000", 2024-09-14T09:07:43.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2441e-05 2024-09-14T09:07:43.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858953+0000", 2024-09-14T09:07:43.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01142108 2024-09-14T09:07:43.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858957+0000", 2024-09-14T09:07:43.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2830000000000003e-06 2024-09-14T09:07:43.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147667+0000", 2024-09-14T09:07:43.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2887094619999999 2024-09-14T09:07:43.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147784+0000", 2024-09-14T09:07:43.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117465 2024-09-14T09:07:43.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147981+0000", 2024-09-14T09:07:43.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019672900000000001 2024-09-14T09:07:43.774 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.774 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.774 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.775 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106709 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 3072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.775 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:07:43.775 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914778963, 2024-09-14T09:07:43.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.301335192, 2024-09-14T09:07:43.775 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.775 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.775 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.775 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.775 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.775 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.775 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106709 2024-09-14T09:07:43.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.775 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:07:43.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:07:43.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847538+0000", 2024-09-14T09:07:43.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2800000000000003e-07 2024-09-14T09:07:43.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847539+0000", 2024-09-14T09:07:43.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2720000000000001e-06 2024-09-14T09:07:43.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847544+0000", 2024-09-14T09:07:43.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7219999999999999e-06 2024-09-14T09:07:43.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847547+0000", 2024-09-14T09:07:43.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9960000000000002e-06 2024-09-14T09:07:43.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858991+0000", 2024-09-14T09:07:43.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011444218000000001 2024-09-14T09:07:43.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858999+0000", 2024-09-14T09:07:43.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9419999999999995e-06 2024-09-14T09:07:43.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148053+0000", 2024-09-14T09:07:43.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2890544099999999 2024-09-14T09:07:43.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148203+0000", 2024-09-14T09:07:43.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149419 2024-09-14T09:07:43.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148873+0000", 2024-09-14T09:07:43.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00066988499999999995 2024-09-14T09:07:43.779 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.779 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.779 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.779 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106711 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 7168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.780 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:07:43.780 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914765705000001, 2024-09-14T09:07:43.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3022241219999999, 2024-09-14T09:07:43.780 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.780 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.780 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.780 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.780 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.780 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.780 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106711 2024-09-14T09:07:43.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.780 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:07:43.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:07:43.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:07:43.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8000000000000002e-07 2024-09-14T09:07:43.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847552+0000", 2024-09-14T09:07:43.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1340000000000001e-06 2024-09-14T09:07:43.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847557+0000", 2024-09-14T09:07:43.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6179999999999997e-06 2024-09-14T09:07:43.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847560+0000", 2024-09-14T09:07:43.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5899999999999999e-06 2024-09-14T09:07:43.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859050+0000", 2024-09-14T09:07:43.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01148914 2024-09-14T09:07:43.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859056+0000", 2024-09-14T09:07:43.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6630000000000004e-06 2024-09-14T09:07:43.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148948+0000", 2024-09-14T09:07:43.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2898914370000001 2024-09-14T09:07:43.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149079+0000", 2024-09-14T09:07:43.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013145400000000001 2024-09-14T09:07:43.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149775+0000", 2024-09-14T09:07:43.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00069580599999999999 2024-09-14T09:07:43.784 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.785 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.785 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.785 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106713 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 11264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.785 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:07:43.785 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914752239999999, 2024-09-14T09:07:43.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3035188959999999, 2024-09-14T09:07:43.785 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.785 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.785 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.785 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.785 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.785 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.786 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106713 2024-09-14T09:07:43.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.786 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:07:43.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:07:43.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:07:43.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6e-07 2024-09-14T09:07:43.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847566+0000", 2024-09-14T09:07:43.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1039999999999999e-06 2024-09-14T09:07:43.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847569+0000", 2024-09-14T09:07:43.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.191e-06 2024-09-14T09:07:43.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847571+0000", 2024-09-14T09:07:43.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6390000000000001e-06 2024-09-14T09:07:43.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859099+0000", 2024-09-14T09:07:43.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011527379000000001 2024-09-14T09:07:43.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859106+0000", 2024-09-14T09:07:43.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3679999999999996e-06 2024-09-14T09:07:43.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149878+0000", 2024-09-14T09:07:43.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.29077208 2024-09-14T09:07:43.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.150175+0000", 2024-09-14T09:07:43.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029685699999999998 2024-09-14T09:07:43.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151083+0000", 2024-09-14T09:07:43.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00090801800000000002 2024-09-14T09:07:43.790 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.790 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.790 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.790 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106715 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 15360~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.791 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:07:43.791 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914741231000001, 2024-09-14T09:07:43.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3047478800000001, 2024-09-14T09:07:43.791 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.791 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.791 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.791 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.791 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.791 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.791 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106715 2024-09-14T09:07:43.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.791 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:07:43.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:07:43.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:07:43.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6e-07 2024-09-14T09:07:43.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847580+0000", 2024-09-14T09:07:43.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9250000000000003e-06 2024-09-14T09:07:43.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847585+0000", 2024-09-14T09:07:43.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8409999999999999e-06 2024-09-14T09:07:43.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847588+0000", 2024-09-14T09:07:43.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6989999999999999e-06 2024-09-14T09:07:43.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859151+0000", 2024-09-14T09:07:43.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011562604000000001 2024-09-14T09:07:43.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859159+0000", 2024-09-14T09:07:43.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4179999999999997e-06 2024-09-14T09:07:43.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151155+0000", 2024-09-14T09:07:43.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.291996068 2024-09-14T09:07:43.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151383+0000", 2024-09-14T09:07:43.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022746100000000001 2024-09-14T09:07:43.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152323+0000", 2024-09-14T09:07:43.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00094060400000000005 2024-09-14T09:07:43.796 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.796 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.796 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.796 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106717 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 19456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.796 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:07:43.796 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914724542999998, 2024-09-14T09:07:43.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.305574579, 2024-09-14T09:07:43.796 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.796 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.796 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.796 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.796 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.796 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.797 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106717 2024-09-14T09:07:43.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.797 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:07:43.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:07:43.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:07:43.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1199999999999999e-07 2024-09-14T09:07:43.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847593+0000", 2024-09-14T09:07:43.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.257e-06 2024-09-14T09:07:43.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847598+0000", 2024-09-14T09:07:43.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4139999999999996e-06 2024-09-14T09:07:43.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847602+0000", 2024-09-14T09:07:43.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3440000000000003e-06 2024-09-14T09:07:43.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859203+0000", 2024-09-14T09:07:43.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011600572999999999 2024-09-14T09:07:43.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859211+0000", 2024-09-14T09:07:43.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-06 2024-09-14T09:07:43.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152398+0000", 2024-09-14T09:07:43.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.293187812 2024-09-14T09:07:43.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152543+0000", 2024-09-14T09:07:43.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014478000000000001 2024-09-14T09:07:43.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153166+0000", 2024-09-14T09:07:43.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000623257 2024-09-14T09:07:43.800 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.800 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.801 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.801 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106719 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 23552~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.801 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:07:43.801 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914710227, 2024-09-14T09:07:43.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.306394982, 2024-09-14T09:07:43.801 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.801 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.801 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.801 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.801 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.801 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.802 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106719 2024-09-14T09:07:43.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.802 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:07:43.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:07:43.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:07:43.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5100000000000001e-07 2024-09-14T09:07:43.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847608+0000", 2024-09-14T09:07:43.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.189e-06 2024-09-14T09:07:43.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847612+0000", 2024-09-14T09:07:43.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9519999999999996e-06 2024-09-14T09:07:43.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847614+0000", 2024-09-14T09:07:43.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6280000000000001e-06 2024-09-14T09:07:43.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859254+0000", 2024-09-14T09:07:43.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011640157 2024-09-14T09:07:43.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859261+0000", 2024-09-14T09:07:43.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0570000000000004e-06 2024-09-14T09:07:43.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153236+0000", 2024-09-14T09:07:43.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2939744040000001 2024-09-14T09:07:43.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153380+0000", 2024-09-14T09:07:43.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014410399999999999 2024-09-14T09:07:43.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154001+0000", 2024-09-14T09:07:43.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062124000000000001 2024-09-14T09:07:43.806 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.806 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.806 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.807 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106721 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 27648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.807 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:07:43.807 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914698576999999, 2024-09-14T09:07:43.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3074446659999999, 2024-09-14T09:07:43.807 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.807 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.807 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.807 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.807 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.807 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.807 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106721 2024-09-14T09:07:43.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.807 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:07:43.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:07:43.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:07:43.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4299999999999999e-07 2024-09-14T09:07:43.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847619+0000", 2024-09-14T09:07:43.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T09:07:43.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847624+0000", 2024-09-14T09:07:43.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4229999999999998e-06 2024-09-14T09:07:43.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847627+0000", 2024-09-14T09:07:43.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0120000000000001e-06 2024-09-14T09:07:43.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859305+0000", 2024-09-14T09:07:43.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01167812 2024-09-14T09:07:43.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859313+0000", 2024-09-14T09:07:43.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2409999999999995e-06 2024-09-14T09:07:43.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154081+0000", 2024-09-14T09:07:43.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.294768296 2024-09-14T09:07:43.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154280+0000", 2024-09-14T09:07:43.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019824 2024-09-14T09:07:43.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155063+0000", 2024-09-14T09:07:43.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00078303400000000005 2024-09-14T09:07:43.812 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.812 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.813 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.813 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106723 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 31744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.813 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:07:43.813 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914686147000001, 2024-09-14T09:07:43.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308293648, 2024-09-14T09:07:43.813 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.813 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.813 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.813 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.813 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.814 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.814 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106723 2024-09-14T09:07:43.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.814 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:07:43.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:07:43.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847631+0000", 2024-09-14T09:07:43.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6100000000000002e-07 2024-09-14T09:07:43.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847632+0000", 2024-09-14T09:07:43.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0589999999999999e-06 2024-09-14T09:07:43.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847635+0000", 2024-09-14T09:07:43.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.551e-06 2024-09-14T09:07:43.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847638+0000", 2024-09-14T09:07:43.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.644e-06 2024-09-14T09:07:43.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859344+0000", 2024-09-14T09:07:43.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011706397 2024-09-14T09:07:43.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859348+0000", 2024-09-14T09:07:43.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5779999999999999e-06 2024-09-14T09:07:43.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155135+0000", 2024-09-14T09:07:43.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.295787448 2024-09-14T09:07:43.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155279+0000", 2024-09-14T09:07:43.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014400700000000001 2024-09-14T09:07:43.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155924+0000", 2024-09-14T09:07:43.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00064470300000000005 2024-09-14T09:07:43.818 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.818 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.819 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.819 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106725 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 35840~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.819 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:07:43.819 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914675035999998, 2024-09-14T09:07:43.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.309103218, 2024-09-14T09:07:43.819 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.819 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.819 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.820 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.820 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.820 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.820 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106725 2024-09-14T09:07:43.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.820 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:07:43.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:07:43.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847642+0000", 2024-09-14T09:07:43.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4499999999999998e-07 2024-09-14T09:07:43.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847643+0000", 2024-09-14T09:07:43.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4e-07 2024-09-14T09:07:43.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847647+0000", 2024-09-14T09:07:43.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6190000000000002e-06 2024-09-14T09:07:43.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847655+0000", 2024-09-14T09:07:43.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:07:43.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859381+0000", 2024-09-14T09:07:43.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011725793 2024-09-14T09:07:43.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859391+0000", 2024-09-14T09:07:43.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4390000000000006e-06 2024-09-14T09:07:43.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155997+0000", 2024-09-14T09:07:43.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2966067830000001 2024-09-14T09:07:43.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156133+0000", 2024-09-14T09:07:43.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000135609 2024-09-14T09:07:43.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156745+0000", 2024-09-14T09:07:43.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000611592 2024-09-14T09:07:43.825 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.825 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.825 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.825 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106727 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 39936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.825 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:07:43.825 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914656736000001, 2024-09-14T09:07:43.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3099046560000001, 2024-09-14T09:07:43.826 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.826 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.826 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.826 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.826 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.826 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.826 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106727 2024-09-14T09:07:43.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.826 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:07:43.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:07:43.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:07:43.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1300000000000001e-07 2024-09-14T09:07:43.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847661+0000", 2024-09-14T09:07:43.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.192e-06 2024-09-14T09:07:43.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847665+0000", 2024-09-14T09:07:43.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0999999999999997e-06 2024-09-14T09:07:43.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847668+0000", 2024-09-14T09:07:43.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.644e-06 2024-09-14T09:07:43.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859422+0000", 2024-09-14T09:07:43.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011753559 2024-09-14T09:07:43.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859425+0000", 2024-09-14T09:07:43.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.343e-06 2024-09-14T09:07:43.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156817+0000", 2024-09-14T09:07:43.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2973924800000001 2024-09-14T09:07:43.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156956+0000", 2024-09-14T09:07:43.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013835 2024-09-14T09:07:43.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157564+0000", 2024-09-14T09:07:43.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060867500000000004 2024-09-14T09:07:43.831 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.831 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.831 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.831 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106729 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 44032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.831 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:07:43.831 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914644762999998, 2024-09-14T09:07:43.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3107195970000001, 2024-09-14T09:07:43.831 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.831 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.831 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.832 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.832 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.832 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.832 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106729 2024-09-14T09:07:43.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.832 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:07:43.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:07:43.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:07:43.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.29e-07 2024-09-14T09:07:43.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847673+0000", 2024-09-14T09:07:43.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.096e-06 2024-09-14T09:07:43.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847678+0000", 2024-09-14T09:07:43.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.527e-06 2024-09-14T09:07:43.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847681+0000", 2024-09-14T09:07:43.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.788e-06 2024-09-14T09:07:43.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859462+0000", 2024-09-14T09:07:43.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01178032 2024-09-14T09:07:43.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859471+0000", 2024-09-14T09:07:43.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3430000000000002e-06 2024-09-14T09:07:43.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157635+0000", 2024-09-14T09:07:43.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2981640320000001 2024-09-14T09:07:43.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157773+0000", 2024-09-14T09:07:43.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763 2024-09-14T09:07:43.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158391+0000", 2024-09-14T09:07:43.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061863200000000004 2024-09-14T09:07:43.837 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.837 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.837 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.837 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106731 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 48128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.837 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:07:43.837 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914629118000001, 2024-09-14T09:07:43.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3115142020000001, 2024-09-14T09:07:43.838 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.838 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.838 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.838 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.838 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.838 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.838 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106731 2024-09-14T09:07:43.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.838 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:07:43.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:07:43.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847688+0000", 2024-09-14T09:07:43.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0199999999999998e-07 2024-09-14T09:07:43.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847689+0000", 2024-09-14T09:07:43.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4529999999999999e-06 2024-09-14T09:07:43.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847694+0000", 2024-09-14T09:07:43.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9080000000000003e-06 2024-09-14T09:07:43.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847697+0000", 2024-09-14T09:07:43.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4920000000000002e-06 2024-09-14T09:07:43.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859499+0000", 2024-09-14T09:07:43.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011801676000000001 2024-09-14T09:07:43.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859502+0000", 2024-09-14T09:07:43.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2899999999999998e-06 2024-09-14T09:07:43.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158461+0000", 2024-09-14T09:07:43.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.298958796 2024-09-14T09:07:43.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158594+0000", 2024-09-14T09:07:43.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013321599999999999 2024-09-14T09:07:43.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159202+0000", 2024-09-14T09:07:43.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060706900000000001 2024-09-14T09:07:43.843 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.843 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.843 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.844 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106733 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 52224~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.844 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:07:43.844 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914614862000001, 2024-09-14T09:07:43.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3123343809999999, 2024-09-14T09:07:43.844 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.844 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.844 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.844 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.844 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.844 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.845 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106733 2024-09-14T09:07:43.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.845 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:07:43.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:07:43.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:07:43.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5800000000000001e-07 2024-09-14T09:07:43.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847703+0000", 2024-09-14T09:07:43.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.127e-06 2024-09-14T09:07:43.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847706+0000", 2024-09-14T09:07:43.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3210000000000001e-06 2024-09-14T09:07:43.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847709+0000", 2024-09-14T09:07:43.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7539999999999998e-06 2024-09-14T09:07:43.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859538+0000", 2024-09-14T09:07:43.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01182942 2024-09-14T09:07:43.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859547+0000", 2024-09-14T09:07:43.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T09:07:43.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159273+0000", 2024-09-14T09:07:43.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2997263700000001 2024-09-14T09:07:43.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159415+0000", 2024-09-14T09:07:43.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000141765 2024-09-14T09:07:43.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160036+0000", 2024-09-14T09:07:43.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062122100000000003 2024-09-14T09:07:43.849 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.850 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.850 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.850 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106735 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 56320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.850 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:07:43.850 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914603417999999, 2024-09-14T09:07:43.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3131339309999999, 2024-09-14T09:07:43.850 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.850 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106735 2024-09-14T09:07:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:07:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:07:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:07:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2100000000000001e-07 2024-09-14T09:07:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847714+0000", 2024-09-14T09:07:43.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.051e-06 2024-09-14T09:07:43.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847719+0000", 2024-09-14T09:07:43.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3640000000000001e-06 2024-09-14T09:07:43.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847723+0000", 2024-09-14T09:07:43.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8789999999999996e-06 2024-09-14T09:07:43.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859577+0000", 2024-09-14T09:07:43.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011854416 2024-09-14T09:07:43.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859580+0000", 2024-09-14T09:07:43.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.292e-06 2024-09-14T09:07:43.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160108+0000", 2024-09-14T09:07:43.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.30052758 2024-09-14T09:07:43.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160238+0000", 2024-09-14T09:07:43.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013043500000000001 2024-09-14T09:07:43.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160847+0000", 2024-09-14T09:07:43.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060869299999999999 2024-09-14T09:07:43.856 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.856 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.856 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.856 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106738 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 60416~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.856 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:07:43.856 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914590214, 2024-09-14T09:07:43.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3139482179999999, 2024-09-14T09:07:43.856 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.857 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.857 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.857 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.857 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.857 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.857 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106738 2024-09-14T09:07:43.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.857 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:07:43.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:07:43.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:07:43.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3999999999999998e-07 2024-09-14T09:07:43.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847733+0000", 2024-09-14T09:07:43.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8070000000000002e-06 2024-09-14T09:07:43.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847738+0000", 2024-09-14T09:07:43.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9740000000000001e-06 2024-09-14T09:07:43.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847742+0000", 2024-09-14T09:07:43.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6009999999999999e-06 2024-09-14T09:07:43.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859612+0000", 2024-09-14T09:07:43.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011870537 2024-09-14T09:07:43.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859621+0000", 2024-09-14T09:07:43.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3019999999999995e-06 2024-09-14T09:07:43.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160917+0000", 2024-09-14T09:07:43.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3012961080000001 2024-09-14T09:07:43.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161054+0000", 2024-09-14T09:07:43.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000137422 2024-09-14T09:07:43.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161674+0000", 2024-09-14T09:07:43.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000620227 2024-09-14T09:07:43.862 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.862 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.862 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.862 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106739 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 64512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.862 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:07:43.862 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914570374, 2024-09-14T09:07:43.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.314753777, 2024-09-14T09:07:43.862 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.862 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.862 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.863 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.863 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.863 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.863 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106739 2024-09-14T09:07:43.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.863 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:07:43.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:07:43.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:07:43.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9700000000000003e-07 2024-09-14T09:07:43.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847748+0000", 2024-09-14T09:07:43.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2589999999999999e-06 2024-09-14T09:07:43.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847752+0000", 2024-09-14T09:07:43.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6099999999999999e-06 2024-09-14T09:07:43.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847755+0000", 2024-09-14T09:07:43.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0520000000000002e-06 2024-09-14T09:07:43.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859653+0000", 2024-09-14T09:07:43.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011897575000000001 2024-09-14T09:07:43.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859658+0000", 2024-09-14T09:07:43.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8810000000000004e-06 2024-09-14T09:07:43.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161745+0000", 2024-09-14T09:07:43.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3020870819999999 2024-09-14T09:07:43.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161877+0000", 2024-09-14T09:07:43.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000132657 2024-09-14T09:07:43.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162500+0000", 2024-09-14T09:07:43.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062236400000000003 2024-09-14T09:07:43.868 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.868 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.868 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.868 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106741 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 68608~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.868 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:07:43.868 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914557582, 2024-09-14T09:07:43.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3155568449999999, 2024-09-14T09:07:43.868 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.868 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.868 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.869 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.869 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.869 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.869 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106741 2024-09-14T09:07:43.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.869 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:07:43.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:07:43.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:07:43.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5800000000000001e-07 2024-09-14T09:07:43.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847760+0000", 2024-09-14T09:07:43.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:07:43.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847764+0000", 2024-09-14T09:07:43.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3189999999999999e-06 2024-09-14T09:07:43.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847766+0000", 2024-09-14T09:07:43.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6429999999999999e-06 2024-09-14T09:07:43.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859693+0000", 2024-09-14T09:07:43.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011926724 2024-09-14T09:07:43.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:43.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859703+0000", 2024-09-14T09:07:43.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6160000000000008e-06 2024-09-14T09:07:43.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162575+0000", 2024-09-14T09:07:43.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.302872128 2024-09-14T09:07:43.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162709+0000", 2024-09-14T09:07:43.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013446000000000001 2024-09-14T09:07:43.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163316+0000", 2024-09-14T09:07:43.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060650700000000005 2024-09-14T09:07:43.873 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.874 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.874 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.874 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106744 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 76800~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:43.874 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:07:43.874 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 21.914546471000001, 2024-09-14T09:07:43.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3163901789999999, 2024-09-14T09:07:43.874 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:43.874 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:43.874 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:43.874 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:43.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:43.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:43.875 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106744 2024-09-14T09:07:43.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.875 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:43.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:43.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:07:43.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:43.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:07:43.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:43.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:43.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996947+0000", 2024-09-14T09:07:43.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.149177154 2024-09-14T09:07:43.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:43.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996964+0000", 2024-09-14T09:07:43.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.679e-05 2024-09-14T09:07:43.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:43.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996999+0000", 2024-09-14T09:07:43.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5463000000000002e-05 2024-09-14T09:07:43.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:43.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.997029+0000", 2024-09-14T09:07:43.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9567e-05 2024-09-14T09:07:43.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:43.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163390+0000", 2024-09-14T09:07:43.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.16636056799999999 2024-09-14T09:07:43.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:43.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163526+0000", 2024-09-14T09:07:43.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013655899999999999 2024-09-14T09:07:43.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:43.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:43.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:43.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.164160+0000", 2024-09-14T09:07:43.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00063407800000000005 2024-09-14T09:07:43.878 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.879 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.879 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.879 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:43.879 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:43.879 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:43.879 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:07:43.880 INFO:teuthology.orchestra.run.smithi176.stdout:5784 read 0x795991d thru 0x795bbb9 (0x229d bytes) 2024-09-14T09:07:43.881 INFO:teuthology.orchestra.run.smithi176.stdout:5787 writesame 0x8a2d35a thru 0x8a2edf5 data_size 0x1a(0x1a9c bytes) 2024-09-14T09:07:43.881 INFO:teuthology.orchestra.run.smithi176.stdout:5788 read 0x32e9363 thru 0x32ed2d4 (0x3f72 bytes) 2024-09-14T09:07:43.881 INFO:teuthology.orchestra.run.smithi176.stdout:5789 read 0x5f4f2b4 thru 0x5f58472 (0x91bf bytes) 2024-09-14T09:07:43.881 INFO:teuthology.orchestra.run.smithi176.stdout:5791 write 0x5b517c4 thru 0x5b5ff08 (0xe745 bytes) 2024-09-14T09:07:43.882 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:07:43.883 INFO:teuthology.orchestra.run.smithi176.stdout:5792 write 0x9946c0f thru 0x994be16 (0x5208 bytes) 2024-09-14T09:07:43.883 INFO:teuthology.orchestra.run.smithi176.stdout:5793 write 0xb7a153 thru 0xb85be1 (0xba8f bytes) 2024-09-14T09:07:43.883 INFO:teuthology.orchestra.run.smithi176.stdout:5794 write 0x2a13c7 thru 0x2ab80a (0xa444 bytes) 2024-09-14T09:07:43.883 INFO:teuthology.orchestra.run.smithi176.stdout:5795 read 0xd53142 thru 0xd5c8b1 (0x9770 bytes) 2024-09-14T09:07:43.883 INFO:teuthology.orchestra.run.smithi176.stdout:5796 punch from 0xa69c3c7 to 0xa6a9e85, (0xdabe bytes) 2024-09-14T09:07:43.883 INFO:teuthology.orchestra.run.smithi176.stdout:5797 read 0xa3702f2 thru 0xa377467 (0x7176 bytes) 2024-09-14T09:07:43.883 INFO:teuthology.orchestra.run.smithi176.stdout:5798 writesame 0xa383693 thru 0xa3917f4 data_size 0x2(0xe162 bytes) 2024-09-14T09:07:43.884 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:43.879+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:43.911 INFO:teuthology.orchestra.run.smithi176.stdout:5799 writesame 0xe22cb39 thru 0xe22f0cb data_size 0x1(0x2593 bytes) 2024-09-14T09:07:43.963 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:43.963 INFO:teuthology.orchestra.run.smithi176.stdout:5802 write 0x21c027a thru 0x21c9d71 (0x9af8 bytes) 2024-09-14T09:07:43.980 INFO:teuthology.orchestra.run.smithi176.stdout:5804 punch from 0x8157f70 to 0x8160b04, (0x8b94 bytes) 2024-09-14T09:07:43.980 INFO:teuthology.orchestra.run.smithi176.stdout:5805 read 0x48050e7 thru 0x4805a9a (0x9b4 bytes) 2024-09-14T09:07:43.980 INFO:teuthology.orchestra.run.smithi176.stdout:5807 clone 44 order 20 su 262144 sc 15 2024-09-14T09:07:43.981 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:43.979+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:44.043 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:44.043 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:44.043 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:44.043 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:44.057 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:07:44.062 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:44.062 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:44.079 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:44.080 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:44.081 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:44.082+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:44.110 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:44.111 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:44.128 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:44.129 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:44.149 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4: {} 2024-09-14T09:07:44.149 INFO:teuthology.orchestra.run.smithi176.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T09:07:44.170 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:44.170 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:44.181 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:44.182+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:44.187 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:44.187 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:44.190 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:44.190 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:44.202 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:07:44.212 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:44.212 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:44.282 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:44.283+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:44.325 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:44.326 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:44.339 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:07:44.386 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:44.390+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:44.472 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:44.473 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:44.483 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:44.483+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:44.486 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T09:07:44.585 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:44.583+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:44.653 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:44.653 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:44.667 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T09:07:44.714 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:44.718+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:44.786 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:44.790+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:44.788 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:44.788 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:44.802 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:07:44.890 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:44.894+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:44.970 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:44.970 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:44.973 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T09:07:44.986 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:07:45.030 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:45.034+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:45.088 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:45.090+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:45.112 INFO:teuthology.orchestra.run.smithi176.stdout:leaving image image_client.0-clone43 intact 2024-09-14T09:07:45.160 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:45.160 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:45.174 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:07:45.189 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:45.191+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:45.290 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:45.291+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:45.341 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:45.341 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:45.351 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-14T09:07:45.356 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:07:45.361 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:45.361 INFO:teuthology.orchestra.run.smithi176.stdout:5812 punch from 0xe8dd3d5 to 0xe8de772, (0x139d bytes) 2024-09-14T09:07:45.367 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:45.367 INFO:teuthology.orchestra.run.smithi176.stdout:5814 writesame 0x7686df thru 0x76aff8 data_size 0x1(0x291a bytes) 2024-09-14T09:07:45.391 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:45.391+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:45.404 INFO:teuthology.orchestra.run.smithi176.stdout:5815 read 0xa49e988 thru 0xa49f244 (0x8bd bytes) 2024-09-14T09:07:45.405 INFO:teuthology.orchestra.run.smithi176.stdout:5816 write 0xa839686 thru 0xa842311 (0x8c8c bytes) 2024-09-14T09:07:45.419 INFO:teuthology.orchestra.run.smithi176.stdout:5817 write 0x9f03d4e thru 0x9f12cfc (0xefaf bytes) 2024-09-14T09:07:45.435 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:45.436 INFO:teuthology.orchestra.run.smithi176.stdout:5821 write 0xe0ec474 thru 0xe0fac81 (0xe80e bytes) 2024-09-14T09:07:45.455 INFO:teuthology.orchestra.run.smithi176.stdout:5823 punch from 0xa7e45ee to 0xa7ed23d, (0x8c4f bytes) 2024-09-14T09:07:45.455 INFO:teuthology.orchestra.run.smithi176.stdout:5824 read 0xe944e77 thru 0xe94d9ec (0x8b76 bytes) 2024-09-14T09:07:45.455 INFO:teuthology.orchestra.run.smithi176.stdout:5825 read 0x2b5ce3f thru 0x2b5deb2 (0x1074 bytes) 2024-09-14T09:07:45.456 INFO:teuthology.orchestra.run.smithi176.stdout:5827 read 0xb947f32 thru 0xb950a8e (0x8b5d bytes) 2024-09-14T09:07:45.457 INFO:teuthology.orchestra.run.smithi176.stdout:5828 write 0x55184bc thru 0x552316d (0xacb2 bytes) 2024-09-14T09:07:45.470 INFO:teuthology.orchestra.run.smithi176.stdout:5829 write 0xe725ccf thru 0xe7280fb (0x242d bytes) 2024-09-14T09:07:45.478 INFO:teuthology.orchestra.run.smithi176.stdout:5830 punch from 0x49ee778 to 0x49fc8dc, (0xe164 bytes) 2024-09-14T09:07:45.481 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:45.481 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:45.493 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:45.491+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:45.495 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:07:45.508 INFO:teuthology.orchestra.run.smithi176.stdout:5831 trunc from 0xe99ba70 to 0x3f20cc7 2024-09-14T09:07:45.593 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:45.591+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:45.604 INFO:teuthology.orchestra.run.smithi176.stdout:5832 writesame 0xcc9c07f thru 0xcc9f061 data_size 0x17(0x2fe3 bytes) 2024-09-14T09:07:45.604 INFO:teuthology.orchestra.run.smithi176.stdout:5834 write 0x6b4ba29 thru 0x6b4e626 (0x2bfe bytes) 2024-09-14T09:07:45.613 INFO:teuthology.orchestra.run.smithi176.stdout:5835 flatten 2024-09-14T09:07:45.630 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:45.630 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:45.645 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:07:45.694 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:45.695+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:45.753 INFO:teuthology.orchestra.run.smithi176.stdout:5836 write 0x2b3a2ab thru 0x2b3a5aa (0x300 bytes) 2024-09-14T09:07:45.757 INFO:teuthology.orchestra.run.smithi176.stdout:5837 read 0x7cd85a3 thru 0x7cd9446 (0xea4 bytes) 2024-09-14T09:07:45.757 INFO:teuthology.orchestra.run.smithi176.stdout:5838 punch from 0x5c7c8d3 to 0x5c87719, (0xae46 bytes) 2024-09-14T09:07:45.757 INFO:teuthology.orchestra.run.smithi176.stdout:5839 read 0x52e9b93 thru 0x52f31d7 (0x9645 bytes) 2024-09-14T09:07:45.762 INFO:teuthology.orchestra.run.smithi176.stdout:5840 trunc from 0xcc9f062 to 0x92676d3 2024-09-14T09:07:45.782 INFO:teuthology.orchestra.run.smithi176.stdout:5841 trunc from 0x92676d3 to 0x1ff9ce8 2024-09-14T09:07:45.792 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:45.792 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:45.794 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:45.798+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:45.800 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:45.800 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:45.800 INFO:teuthology.orchestra.run.smithi176.stdout:5844 read 0x1b996d4 thru 0x1ba80f0 (0xea1d bytes) 2024-09-14T09:07:45.800 INFO:teuthology.orchestra.run.smithi176.stdout:5845 punch from 0x4d68d to 0x535bb, (0x5f2e bytes) 2024-09-14T09:07:45.809 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:07:45.809 INFO:teuthology.orchestra.run.smithi176.stdout:5846 write 0x31afd9c thru 0x31bcf32 (0xd197 bytes) 2024-09-14T09:07:45.813 INFO:teuthology.orchestra.run.smithi176.stdout:5847 read 0x25db04b thru 0x25db949 (0x8ff bytes) 2024-09-14T09:07:45.826 INFO:teuthology.orchestra.run.smithi176.stdout:5848 write 0x64fd4da thru 0x6503d36 (0x685d bytes) 2024-09-14T09:07:45.832 INFO:teuthology.orchestra.run.smithi176.stdout:5849 write 0x1f10cc0 thru 0x1f1fbb6 (0xeef7 bytes) 2024-09-14T09:07:45.842 INFO:teuthology.orchestra.run.smithi176.stdout:5850 read 0x9f8e21 thru 0xa03eb0 (0xb090 bytes) 2024-09-14T09:07:45.843 INFO:teuthology.orchestra.run.smithi176.stdout:5851 writesame 0x16ae9fd thru 0x16b55c2 data_size 0xa(0x6bc6 bytes) 2024-09-14T09:07:45.875 INFO:teuthology.orchestra.run.smithi176.stdout:5854 write 0xebd53fb thru 0xebdfd71 (0xa977 bytes) 2024-09-14T09:07:45.881 INFO:teuthology.orchestra.run.smithi176.stdout:5855 writesame 0x1e3785a thru 0x1e38ce7 data_size 0x3(0x148e bytes) 2024-09-14T09:07:45.883 INFO:teuthology.orchestra.run.smithi176.stdout:5856 read 0x145fc7d thru 0x146c53e (0xc8c2 bytes) 2024-09-14T09:07:45.884 INFO:teuthology.orchestra.run.smithi176.stdout:5858 write 0xd561cfd thru 0xd567230 (0x5534 bytes) 2024-09-14T09:07:45.890 INFO:teuthology.orchestra.run.smithi176.stdout:5860 write 0x62da814 thru 0x62e0164 (0x5951 bytes) 2024-09-14T09:07:45.895 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:45.898+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:45.896 INFO:teuthology.orchestra.run.smithi176.stdout:5862 write 0x925d1d6 thru 0x925f301 (0x212c bytes) 2024-09-14T09:07:45.902 INFO:teuthology.orchestra.run.smithi176.stdout:5863 punch from 0xd795aad to 0xd79c5ad, (0x6b00 bytes) 2024-09-14T09:07:45.902 INFO:teuthology.orchestra.run.smithi176.stdout:5864 write 0xe9070f8 thru 0xe914f45 (0xde4e bytes) 2024-09-14T09:07:45.914 INFO:teuthology.orchestra.run.smithi176.stdout:5867 write 0xcce78fa thru 0xccf565f (0xdd66 bytes) 2024-09-14T09:07:45.923 INFO:teuthology.orchestra.run.smithi176.stdout:5868 write 0x50b77e1 thru 0x50bca4b (0x526b bytes) 2024-09-14T09:07:45.930 INFO:teuthology.orchestra.run.smithi176.stdout:5869 write 0x73e4c5 thru 0x745990 (0x74cc bytes) 2024-09-14T09:07:45.934 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:45.934 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:45.934 INFO:teuthology.orchestra.run.smithi176.stdout:5872 write 0x24a223c thru 0x24b1ad3 (0xf898 bytes) 2024-09-14T09:07:45.944 INFO:teuthology.orchestra.run.smithi176.stdout:5873 punch from 0x3e29c1f to 0x3e2a11b, (0x4fc bytes) 2024-09-14T09:07:45.945 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:45.945 INFO:teuthology.orchestra.run.smithi176.stdout:5876 read 0xe7d8afb thru 0xe7e05df (0x7ae5 bytes) 2024-09-14T09:07:45.945 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:45.945 INFO:teuthology.orchestra.run.smithi176.stdout:5878 read 0x4b3e66b thru 0x4b4d4e2 (0xee78 bytes) 2024-09-14T09:07:45.945 INFO:teuthology.orchestra.run.smithi176.stdout:5879 write 0x40f63c1 thru 0x4105df7 (0xfa37 bytes) 2024-09-14T09:07:45.957 INFO:teuthology.orchestra.run.smithi176.stdout:5881 read 0x9f3e7f7 thru 0x9f49f57 (0xb761 bytes) 2024-09-14T09:07:45.958 INFO:teuthology.orchestra.run.smithi176.stdout:5882 writesame 0x3fe2308 thru 0x3ff1df2 data_size 0x1(0xfaeb bytes) 2024-09-14T09:07:45.971 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is unset 2024-09-14T09:07:45.972 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:45.972 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:45.989 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:07:46.030 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:46.034+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:46.097 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:07:46.098+0000 7f7a9ead5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26514) UID: 0 2024-09-14T09:07:46.132 INFO:teuthology.orchestra.run.smithi176.stdout:5883 write 0x991ef20 thru 0x9926403 (0x74e4 bytes) 2024-09-14T09:07:46.143 INFO:teuthology.orchestra.run.smithi176.stdout:5884 read 0xc3a674f thru 0xc3ab5c3 (0x4e75 bytes) 2024-09-14T09:07:46.143 INFO:teuthology.orchestra.run.smithi176.stdout:5885 punch from 0x32e8348 to 0x32edf9c, (0x5c54 bytes) 2024-09-14T09:07:46.143 INFO:teuthology.orchestra.run.smithi176.stdout:5886 write 0x67ab5b1 thru 0x67b5e5f (0xa8af bytes) 2024-09-14T09:07:46.154 INFO:teuthology.orchestra.run.smithi176.stdout:5887 trunc from 0xebdfd72 to 0xe5b75b7 2024-09-14T09:07:46.160 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:46.160 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:46.167 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:46.168 INFO:teuthology.orchestra.run.smithi176.stdout:5890 read 0x4101ab1 thru 0x410c34a (0xa89a bytes) 2024-09-14T09:07:46.175 INFO:teuthology.orchestra.run.smithi176.stdout:5891 trunc from 0xe5b75b7 to 0xc4a8c32 2024-09-14T09:07:46.176 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:07:46.198 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:46.199+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:46.210 INFO:teuthology.orchestra.run.smithi176.stdout:5893 write 0x6eea8f9 thru 0x6eecda7 (0x24af bytes) 2024-09-14T09:07:46.218 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:46.218 INFO:teuthology.orchestra.run.smithi176.stdout:5895 write 0xb2e3bc6 thru 0xb2f1f41 (0xe37c bytes) 2024-09-14T09:07:46.230 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:07:46.239 INFO:teuthology.orchestra.run.smithi176.stdout:5896 trunc from 0xc4a8c32 to 0xa2938bb 2024-09-14T09:07:46.262 INFO:teuthology.orchestra.run.smithi176.stdout:5897 punch from 0x13ffb43 to 0x14089b5, (0x8e72 bytes) 2024-09-14T09:07:46.268 INFO:teuthology.orchestra.run.smithi176.stdout:5899 trunc from 0xa2938bb to 0x6e33c72 2024-09-14T09:07:46.287 INFO:teuthology.orchestra.run.smithi176.stdout:5901 punch from 0x5cd8c9e to 0x5cdc5db, (0x393d bytes) 2024-09-14T09:07:46.288 INFO:teuthology.orchestra.run.smithi176.stdout:5902 read 0x2b8aa13 thru 0x2b8ab42 (0x130 bytes) 2024-09-14T09:07:46.288 INFO:teuthology.orchestra.run.smithi176.stdout:5903 read 0x603440e thru 0x6035e91 (0x1a84 bytes) 2024-09-14T09:07:46.288 INFO:teuthology.orchestra.run.smithi176.stdout:5904 read 0x3f6591a thru 0x3f73581 (0xdc68 bytes) 2024-09-14T09:07:46.288 INFO:teuthology.orchestra.run.smithi176.stdout:5905 punch from 0x6035fdb to 0x603c711, (0x6736 bytes) 2024-09-14T09:07:46.298 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:46.302+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:46.306 INFO:teuthology.orchestra.run.smithi176.stdout:5906 write 0xbdb3cb5 thru 0xbdc0054 (0xc3a0 bytes) 2024-09-14T09:07:46.324 INFO:teuthology.orchestra.run.smithi176.stdout:5907 write 0xc26e2b9 thru 0xc272eb6 (0x4bfe bytes) 2024-09-14T09:07:46.331 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:46.331 INFO:teuthology.orchestra.run.smithi176.stdout:5909 punch from 0x2793e6e to 0x279b414, (0x75a6 bytes) 2024-09-14T09:07:46.331 INFO:teuthology.orchestra.run.smithi176.stdout:5910 read 0x542c033 thru 0x5438575 (0xc543 bytes) 2024-09-14T09:07:46.338 INFO:teuthology.orchestra.run.smithi176.stdout:5911 trunc from 0xc272eb7 to 0x76f2d6b 2024-09-14T09:07:46.347 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:46.347 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:46.363 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:07:46.364 INFO:teuthology.orchestra.run.smithi176.stdout:5912 write 0x4738f68 thru 0x473d1c9 (0x4262 bytes) 2024-09-14T09:07:46.378 INFO:teuthology.orchestra.run.smithi176.stdout:5914 write 0x944bd85 thru 0x9459eca (0xe146 bytes) 2024-09-14T09:07:46.383 INFO:teuthology.orchestra.run.smithi176.stdout:5915 punch from 0x6cd9bd9 to 0x6ce6b1e, (0xcf45 bytes) 2024-09-14T09:07:46.392 INFO:teuthology.orchestra.run.smithi176.stdout:5916 trunc from 0x9459ecb to 0x2cf38ef 2024-09-14T09:07:46.397 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:46.397 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:46.399 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:46.399+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:46.403 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:07:46.403 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:07:46.403 INFO:tasks.thrashosds.thrasher:Killing osd 4, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T09:07:46.404 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-14T09:07:46.404 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T09:07:46.416 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:46.416 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:46.432 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:46.433 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:46.450 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:46.451 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:46.485 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:46.485 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:46.486 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:46.486 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:46.501 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:07:46.501 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:46.499+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:46.502 INFO:tasks.ceph.osd.4:Stopped 2024-09-14T09:07:46.502 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.4 2024-09-14T09:07:46.503 INFO:tasks.thrashosds.thrasher:remote for osd 4 is ubuntu@smithi183.front.sepia.ceph.com 2024-09-14T09:07:46.503 DEBUG:teuthology.orchestra.run.smithi183:> type ceph-objectstore-tool 2024-09-14T09:07:46.508 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T09:07:46.508 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op list-pgs 2024-09-14T09:07:46.602 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:46.606+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:46.663 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:46.664 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:46.664 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:46.664 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:46.664 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:46.678 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:07:46.702 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:46.706+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:46.803 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:46.803+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:46.804 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:46.804 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:46.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:71052 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:07:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:07:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 77.065926043000005, 2024-09-14T09:07:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14854635599999999, 2024-09-14T09:07:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 71052 2024-09-14T09:07:46.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:07:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:07:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737570+0000", 2024-09-14T09:07:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4770000000000002e-06 2024-09-14T09:07:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737584+0000", 2024-09-14T09:07:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3457e-05 2024-09-14T09:07:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737608+0000", 2024-09-14T09:07:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3893999999999999e-05 2024-09-14T09:07:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737620+0000", 2024-09-14T09:07:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2157000000000001e-05 2024-09-14T09:07:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737738+0000", 2024-09-14T09:07:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118409 2024-09-14T09:07:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737892+0000", 2024-09-14T09:07:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154222 2024-09-14T09:07:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:07:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.749622+0000", 2024-09-14T09:07:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011729537 2024-09-14T09:07:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885934+0000", 2024-09-14T09:07:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13631222200000001 2024-09-14T09:07:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885996+0000", 2024-09-14T09:07:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1555000000000003e-05 2024-09-14T09:07:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886060+0000", 2024-09-14T09:07:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3980999999999996e-05 2024-09-14T09:07:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886114+0000", 2024-09-14T09:07:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4444999999999998e-05 2024-09-14T09:07:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.810 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:71052 3.1 e71/34 ondisk, result = 0)", 2024-09-14T09:07:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:07:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 77.023022664999999, 2024-09-14T09:07:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10565334799999999, 2024-09-14T09:07:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:07:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:07:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780473+0000", 2024-09-14T09:07:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8980000000000001e-06 2024-09-14T09:07:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780482+0000", 2024-09-14T09:07:46.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9080000000000001e-06 2024-09-14T09:07:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780511+0000", 2024-09-14T09:07:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9490999999999999e-05 2024-09-14T09:07:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780527+0000", 2024-09-14T09:07:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.508e-05 2024-09-14T09:07:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885978+0000", 2024-09-14T09:07:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.105451328 2024-09-14T09:07:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885989+0000", 2024-09-14T09:07:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0875e-05 2024-09-14T09:07:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886125+0000", 2024-09-14T09:07:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013576800000000001 2024-09-14T09:07:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.814 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:98007 3.12 3:4faf1f2a:::rbd_data.115e113dc8ca.000000000000004f:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:07:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:07:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.946050440999997, 2024-09-14T09:07:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.098233707000000003, 2024-09-14T09:07:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 98007 2024-09-14T09:07:46.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:07:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:07:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857446+0000", 2024-09-14T09:07:46.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7599999999999998e-06 2024-09-14T09:07:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857459+0000", 2024-09-14T09:07:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3123e-05 2024-09-14T09:07:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857491+0000", 2024-09-14T09:07:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1603000000000002e-05 2024-09-14T09:07:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857506+0000", 2024-09-14T09:07:46.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.537e-05 2024-09-14T09:07:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857694+0000", 2024-09-14T09:07:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018766599999999999 2024-09-14T09:07:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857955+0000", 2024-09-14T09:07:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000260979 2024-09-14T09:07:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:07:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.863125+0000", 2024-09-14T09:07:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.005170582 2024-09-14T09:07:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955268+0000", 2024-09-14T09:07:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092142659000000002 2024-09-14T09:07:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955385+0000", 2024-09-14T09:07:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116723 2024-09-14T09:07:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955553+0000", 2024-09-14T09:07:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016809299999999999 2024-09-14T09:07:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955677+0000", 2024-09-14T09:07:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012414899999999999 2024-09-14T09:07:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118153 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 445440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:46.820 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:07:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.965493066000001, 2024-09-14T09:07:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095358268999999996, 2024-09-14T09:07:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118153 2024-09-14T09:07:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:07:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:07:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:07:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0999999999999999e-07 2024-09-14T09:07:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838003+0000", 2024-09-14T09:07:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2000000000000001e-06 2024-09-14T09:07:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838009+0000", 2024-09-14T09:07:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.105e-06 2024-09-14T09:07:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838015+0000", 2024-09-14T09:07:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9030000000000001e-06 2024-09-14T09:07:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933049+0000", 2024-09-14T09:07:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095033513999999999 2024-09-14T09:07:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933113+0000", 2024-09-14T09:07:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3616999999999998e-05 2024-09-14T09:07:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933359+0000", 2024-09-14T09:07:46.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024652000000000002 2024-09-14T09:07:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118155 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 449536~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:07:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.965470541, 2024-09-14T09:07:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095668000000000003, 2024-09-14T09:07:46.825 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118155 2024-09-14T09:07:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:07:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:07:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838024+0000", 2024-09-14T09:07:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0400000000000002e-07 2024-09-14T09:07:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838026+0000", 2024-09-14T09:07:46.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.508e-06 2024-09-14T09:07:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838033+0000", 2024-09-14T09:07:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.392e-06 2024-09-14T09:07:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838038+0000", 2024-09-14T09:07:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1050000000000003e-06 2024-09-14T09:07:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933400+0000", 2024-09-14T09:07:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095362279999999994 2024-09-14T09:07:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933451+0000", 2024-09-14T09:07:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0927000000000002e-05 2024-09-14T09:07:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933691+0000", 2024-09-14T09:07:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024038400000000001 2024-09-14T09:07:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.829 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118158 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 453632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:07:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.965449519, 2024-09-14T09:07:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096024231000000002, 2024-09-14T09:07:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118158 2024-09-14T09:07:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.830 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:07:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:07:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838045+0000", 2024-09-14T09:07:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2199999999999999e-07 2024-09-14T09:07:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838047+0000", 2024-09-14T09:07:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1059999999999998e-06 2024-09-14T09:07:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838054+0000", 2024-09-14T09:07:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4089999999999997e-06 2024-09-14T09:07:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838059+0000", 2024-09-14T09:07:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8590000000000004e-06 2024-09-14T09:07:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933729+0000", 2024-09-14T09:07:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095670274999999999 2024-09-14T09:07:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933794+0000", 2024-09-14T09:07:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4846999999999998e-05 2024-09-14T09:07:46.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934069+0000", 2024-09-14T09:07:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027431299999999999 2024-09-14T09:07:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118160 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 457728~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:07:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.965427398999999, 2024-09-14T09:07:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096394812999999996, 2024-09-14T09:07:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.834 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118160 2024-09-14T09:07:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:07:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:07:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838067+0000", 2024-09-14T09:07:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9399999999999995e-07 2024-09-14T09:07:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838069+0000", 2024-09-14T09:07:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1579999999999999e-06 2024-09-14T09:07:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838077+0000", 2024-09-14T09:07:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6939999999999995e-06 2024-09-14T09:07:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838082+0000", 2024-09-14T09:07:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2290000000000002e-06 2024-09-14T09:07:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934106+0000", 2024-09-14T09:07:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096023874999999995 2024-09-14T09:07:46.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934158+0000", 2024-09-14T09:07:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2052000000000002e-05 2024-09-14T09:07:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934461+0000", 2024-09-14T09:07:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030331100000000001 2024-09-14T09:07:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118162 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 461824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:07:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.965404819, 2024-09-14T09:07:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096615294000000004, 2024-09-14T09:07:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118162 2024-09-14T09:07:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.839 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:07:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:07:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838090+0000", 2024-09-14T09:07:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9699999999999996e-07 2024-09-14T09:07:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838092+0000", 2024-09-14T09:07:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1119999999999999e-06 2024-09-14T09:07:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838098+0000", 2024-09-14T09:07:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4359999999999999e-06 2024-09-14T09:07:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838103+0000", 2024-09-14T09:07:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1200000000000001e-06 2024-09-14T09:07:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934504+0000", 2024-09-14T09:07:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096400885000000006 2024-09-14T09:07:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934596+0000", 2024-09-14T09:07:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1775999999999997e-05 2024-09-14T09:07:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934704+0000", 2024-09-14T09:07:46.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010846800000000001 2024-09-14T09:07:46.843 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.843 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.843 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.843 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118164 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 465920~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:46.843 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:07:46.843 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.965383558999999, 2024-09-14T09:07:46.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096951372999999993, 2024-09-14T09:07:46.843 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.843 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.843 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:46.844 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:46.844 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:46.844 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:46.844 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118164 2024-09-14T09:07:46.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.844 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:07:46.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:07:46.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838111+0000", 2024-09-14T09:07:46.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0100000000000005e-07 2024-09-14T09:07:46.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838113+0000", 2024-09-14T09:07:46.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0339999999999999e-06 2024-09-14T09:07:46.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838119+0000", 2024-09-14T09:07:46.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.409e-06 2024-09-14T09:07:46.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838124+0000", 2024-09-14T09:07:46.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9030000000000003e-06 2024-09-14T09:07:46.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934739+0000", 2024-09-14T09:07:46.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096615124999999996 2024-09-14T09:07:46.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934789+0000", 2024-09-14T09:07:46.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9554e-05 2024-09-14T09:07:46.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935062+0000", 2024-09-14T09:07:46.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000272847 2024-09-14T09:07:46.847 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.847 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.847 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.847 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118166 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 470016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:46.847 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:07:46.847 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.965362805, 2024-09-14T09:07:46.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097306410999999995, 2024-09-14T09:07:46.848 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.848 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.848 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:46.848 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:46.848 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:46.848 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:46.848 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118166 2024-09-14T09:07:46.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.848 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:07:46.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:07:46.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:07:46.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.03e-07 2024-09-14T09:07:46.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838133+0000", 2024-09-14T09:07:46.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9479999999999998e-06 2024-09-14T09:07:46.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838141+0000", 2024-09-14T09:07:46.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5739999999999998e-06 2024-09-14T09:07:46.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838146+0000", 2024-09-14T09:07:46.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3750000000000002e-06 2024-09-14T09:07:46.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935095+0000", 2024-09-14T09:07:46.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096948435999999999 2024-09-14T09:07:46.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935152+0000", 2024-09-14T09:07:46.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.694e-05 2024-09-14T09:07:46.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935437+0000", 2024-09-14T09:07:46.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028573500000000001 2024-09-14T09:07:46.851 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.851 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.852 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.852 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118168 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 474112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:46.852 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:07:46.852 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.965340584, 2024-09-14T09:07:46.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097658024999999996, 2024-09-14T09:07:46.852 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.852 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.852 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:46.852 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:46.852 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:46.852 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:46.853 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118168 2024-09-14T09:07:46.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.853 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:07:46.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:07:46.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838154+0000", 2024-09-14T09:07:46.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4200000000000001e-07 2024-09-14T09:07:46.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838160+0000", 2024-09-14T09:07:46.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8510000000000004e-06 2024-09-14T09:07:46.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838168+0000", 2024-09-14T09:07:46.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9899999999999997e-06 2024-09-14T09:07:46.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838172+0000", 2024-09-14T09:07:46.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.887e-06 2024-09-14T09:07:46.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935473+0000", 2024-09-14T09:07:46.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097300132999999997 2024-09-14T09:07:46.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935523+0000", 2024-09-14T09:07:46.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0862000000000002e-05 2024-09-14T09:07:46.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935811+0000", 2024-09-14T09:07:46.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028786000000000003 2024-09-14T09:07:46.856 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.856 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.856 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.856 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118170 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 478208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:46.856 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:07:46.856 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.965313773, 2024-09-14T09:07:46.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.098077529999999996, 2024-09-14T09:07:46.857 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.857 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.857 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:46.857 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:46.857 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:46.857 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:46.857 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118170 2024-09-14T09:07:46.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.857 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:07:46.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:07:46.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842901+0000", 2024-09-14T09:07:46.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004721416 2024-09-14T09:07:46.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842906+0000", 2024-09-14T09:07:46.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7199999999999997e-06 2024-09-14T09:07:46.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842917+0000", 2024-09-14T09:07:46.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0671e-05 2024-09-14T09:07:46.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842923+0000", 2024-09-14T09:07:46.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.088e-06 2024-09-14T09:07:46.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935857+0000", 2024-09-14T09:07:46.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092933624000000006 2024-09-14T09:07:46.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935958+0000", 2024-09-14T09:07:46.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100933 2024-09-14T09:07:46.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.936258+0000", 2024-09-14T09:07:46.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030007800000000001 2024-09-14T09:07:46.860 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.860 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.860 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.861 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118181 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 498688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:46.861 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:07:46.861 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.960507504000001, 2024-09-14T09:07:46.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095212643999999999, 2024-09-14T09:07:46.861 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.861 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.861 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:46.861 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:46.861 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:46.861 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:46.861 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118181 2024-09-14T09:07:46.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.862 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:07:46.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:07:46.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842987+0000", 2024-09-14T09:07:46.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.29e-07 2024-09-14T09:07:46.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842988+0000", 2024-09-14T09:07:46.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.229e-06 2024-09-14T09:07:46.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842993+0000", 2024-09-14T09:07:46.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.792e-06 2024-09-14T09:07:46.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842996+0000", 2024-09-14T09:07:46.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3349999999999998e-06 2024-09-14T09:07:46.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.937807+0000", 2024-09-14T09:07:46.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.094811585000000004 2024-09-14T09:07:46.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.937899+0000", 2024-09-14T09:07:46.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1719999999999996e-05 2024-09-14T09:07:46.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938199+0000", 2024-09-14T09:07:46.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000299754 2024-09-14T09:07:46.865 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.865 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.865 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.865 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118182 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 502784~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:46.865 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:07:46.865 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.960494326999999, 2024-09-14T09:07:46.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095557953000000001, 2024-09-14T09:07:46.865 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.866 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.866 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:46.866 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:46.866 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:46.866 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:46.866 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118182 2024-09-14T09:07:46.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.866 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:07:46.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:07:46.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:07:46.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2999999999999999e-07 2024-09-14T09:07:46.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843001+0000", 2024-09-14T09:07:46.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.652e-06 2024-09-14T09:07:46.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843005+0000", 2024-09-14T09:07:46.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2780000000000002e-06 2024-09-14T09:07:46.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843007+0000", 2024-09-14T09:07:46.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.553e-06 2024-09-14T09:07:46.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938240+0000", 2024-09-14T09:07:46.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095232864 2024-09-14T09:07:46.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938295+0000", 2024-09-14T09:07:46.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5095e-05 2024-09-14T09:07:46.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938557+0000", 2024-09-14T09:07:46.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026228100000000001 2024-09-14T09:07:46.869 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.869 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.869 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.870 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118184 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 506880~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:46.870 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:07:46.870 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.960483127, 2024-09-14T09:07:46.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095950246000000003, 2024-09-14T09:07:46.870 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.870 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.870 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:46.870 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:46.870 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:46.870 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:46.870 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118184 2024-09-14T09:07:46.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.870 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:07:46.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:07:46.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:07:46.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2399999999999999e-07 2024-09-14T09:07:46.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843012+0000", 2024-09-14T09:07:46.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.269e-06 2024-09-14T09:07:46.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843015+0000", 2024-09-14T09:07:46.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0469999999999998e-06 2024-09-14T09:07:46.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843018+0000", 2024-09-14T09:07:46.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.044e-06 2024-09-14T09:07:46.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938597+0000", 2024-09-14T09:07:46.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095578688999999994 2024-09-14T09:07:46.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938687+0000", 2024-09-14T09:07:46.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0284000000000005e-05 2024-09-14T09:07:46.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938961+0000", 2024-09-14T09:07:46.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027368899999999997 2024-09-14T09:07:46.874 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.874 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.874 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.874 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118186 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 510976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:46.874 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:07:46.874 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.960472167000001, 2024-09-14T09:07:46.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096286497999999998, 2024-09-14T09:07:46.875 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.875 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.875 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:46.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:46.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:46.875 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:46.875 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118186 2024-09-14T09:07:46.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.875 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:07:46.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:07:46.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:07:46.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.35e-07 2024-09-14T09:07:46.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843023+0000", 2024-09-14T09:07:46.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.195e-06 2024-09-14T09:07:46.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843026+0000", 2024-09-14T09:07:46.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.055e-06 2024-09-14T09:07:46.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843029+0000", 2024-09-14T09:07:46.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6539999999999999e-06 2024-09-14T09:07:46.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939000+0000", 2024-09-14T09:07:46.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095971476 2024-09-14T09:07:46.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939049+0000", 2024-09-14T09:07:46.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8986999999999997e-05 2024-09-14T09:07:46.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939308+0000", 2024-09-14T09:07:46.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000258896 2024-09-14T09:07:46.878 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.878 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.878 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.879 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118188 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 515072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:46.879 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:07:46.879 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.960461777999999, 2024-09-14T09:07:46.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096722717999999999, 2024-09-14T09:07:46.879 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.879 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.879 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:46.879 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:46.879 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:46.879 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:46.879 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118188 2024-09-14T09:07:46.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.879 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:07:46.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:07:46.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:07:46.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2999999999999999e-07 2024-09-14T09:07:46.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843034+0000", 2024-09-14T09:07:46.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T09:07:46.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843037+0000", 2024-09-14T09:07:46.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0989999999999999e-06 2024-09-14T09:07:46.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843039+0000", 2024-09-14T09:07:46.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.666e-06 2024-09-14T09:07:46.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939351+0000", 2024-09-14T09:07:46.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096311789999999994 2024-09-14T09:07:46.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939440+0000", 2024-09-14T09:07:46.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8843000000000003e-05 2024-09-14T09:07:46.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939755+0000", 2024-09-14T09:07:46.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031486399999999999 2024-09-14T09:07:46.883 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.883 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.883 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.883 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118191 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 519168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:46.883 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:07:46.883 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.960451085000001, 2024-09-14T09:07:46.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097080607999999999, 2024-09-14T09:07:46.883 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.883 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.883 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:46.883 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:46.883 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:46.884 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:46.884 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118191 2024-09-14T09:07:46.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.884 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:07:46.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:07:46.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:07:46.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2100000000000001e-07 2024-09-14T09:07:46.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843044+0000", 2024-09-14T09:07:46.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2249999999999999e-06 2024-09-14T09:07:46.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843047+0000", 2024-09-14T09:07:46.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0240000000000002e-06 2024-09-14T09:07:46.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843050+0000", 2024-09-14T09:07:46.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.508e-06 2024-09-14T09:07:46.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939791+0000", 2024-09-14T09:07:46.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096741399000000006 2024-09-14T09:07:46.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939849+0000", 2024-09-14T09:07:46.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7912999999999999e-05 2024-09-14T09:07:46.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940123+0000", 2024-09-14T09:07:46.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027431800000000001 2024-09-14T09:07:46.887 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.887 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.887 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.887 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118192 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 523264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:46.887 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:07:46.888 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.960440025, 2024-09-14T09:07:46.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097587224, 2024-09-14T09:07:46.888 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.888 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.888 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:46.888 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:46.888 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:46.888 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:46.888 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118192 2024-09-14T09:07:46.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.888 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:07:46.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:07:46.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844132+0000", 2024-09-14T09:07:46.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010777810000000001 2024-09-14T09:07:46.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844140+0000", 2024-09-14T09:07:46.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4470000000000006e-06 2024-09-14T09:07:46.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844155+0000", 2024-09-14T09:07:46.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4924e-05 2024-09-14T09:07:46.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844163+0000", 2024-09-14T09:07:46.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9980000000000003e-06 2024-09-14T09:07:46.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940199+0000", 2024-09-14T09:07:46.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096036047999999999 2024-09-14T09:07:46.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940282+0000", 2024-09-14T09:07:46.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3073999999999998e-05 2024-09-14T09:07:46.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940641+0000", 2024-09-14T09:07:46.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000358952 2024-09-14T09:07:46.891 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.891 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.892 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.892 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118197 3.17 3:ec5ec63a:::rbd_data.115e96ceebb2.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 3718221] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:46.892 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:07:46.892 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 14.955660610000001, 2024-09-14T09:07:46.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.141131069, 2024-09-14T09:07:46.892 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:46.892 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:46.892 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:46.892 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:46.892 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:46.892 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:46.892 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118197 2024-09-14T09:07:46.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.893 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:46.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:46.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:07:46.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:46.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:07:46.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:46.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:46.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847834+0000", 2024-09-14T09:07:46.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2500000000000004e-07 2024-09-14T09:07:46.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:46.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849227+0000", 2024-09-14T09:07:46.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0013932149999999999 2024-09-14T09:07:46.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:46.894 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849246+0000", 2024-09-14T09:07:46.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8958e-05 2024-09-14T09:07:46.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.894 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:46.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849254+0000", 2024-09-14T09:07:46.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2360000000000004e-06 2024-09-14T09:07:46.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:46.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940154+0000", 2024-09-14T09:07:46.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.090899468999999997 2024-09-14T09:07:46.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:46.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940312+0000", 2024-09-14T09:07:46.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000158326 2024-09-14T09:07:46.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:07:46.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940728+0000", 2024-09-14T09:07:46.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00041583100000000002 2024-09-14T09:07:46.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:46.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.962731+0000", 2024-09-14T09:07:46.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022002608999999999 2024-09-14T09:07:46.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:46.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988881+0000", 2024-09-14T09:07:46.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026150620999999999 2024-09-14T09:07:46.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:46.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988919+0000", 2024-09-14T09:07:46.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8108999999999997e-05 2024-09-14T09:07:46.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:46.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:46.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:46.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988964+0000", 2024-09-14T09:07:46.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4870000000000002e-05 2024-09-14T09:07:46.897 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.897 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.897 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.897 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:46.897 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:46.898 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:46.899 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:07:46.942 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:46.943+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:47.008 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:47.006+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:47.047 INFO:teuthology.orchestra.run.smithi183.stdout:3.f 2024-09-14T09:07:47.047 INFO:teuthology.orchestra.run.smithi183.stdout:3.e 2024-09-14T09:07:47.047 INFO:teuthology.orchestra.run.smithi183.stdout:3.a 2024-09-14T09:07:47.047 INFO:teuthology.orchestra.run.smithi183.stdout:3.3 2024-09-14T09:07:47.047 INFO:teuthology.orchestra.run.smithi183.stdout:3.1d 2024-09-14T09:07:47.047 INFO:teuthology.orchestra.run.smithi183.stdout:3.1c 2024-09-14T09:07:47.047 INFO:teuthology.orchestra.run.smithi183.stdout:3.1a 2024-09-14T09:07:47.048 INFO:teuthology.orchestra.run.smithi183.stdout:3.1 2024-09-14T09:07:47.048 INFO:teuthology.orchestra.run.smithi183.stdout:3.14 2024-09-14T09:07:47.048 INFO:teuthology.orchestra.run.smithi183.stdout:3.13 2024-09-14T09:07:47.048 INFO:teuthology.orchestra.run.smithi183.stdout:3.11 2024-09-14T09:07:47.072 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:47.072 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:47.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.072 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:138298 3.1a 3.ac0e573a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e135)", 2024-09-14T09:07:47.072 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:46.991669+0000", 2024-09-14T09:07:47.072 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.082247472000000002, 2024-09-14T09:07:47.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.082292404999999999, 2024-09-14T09:07:47.073 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.073 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.073 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "waiting for peered", 2024-09-14T09:07:47.073 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.073 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.073 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.073 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 138298 2024-09-14T09:07:47.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.073 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:46.991669+0000", 2024-09-14T09:07:47.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:46.991669+0000", 2024-09-14T09:07:47.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:46.991672+0000", 2024-09-14T09:07:47.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1839999999999999e-06 2024-09-14T09:07:47.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:46.991684+0000", 2024-09-14T09:07:47.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2064e-05 2024-09-14T09:07:47.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:46.991710+0000", 2024-09-14T09:07:47.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5789e-05 2024-09-14T09:07:47.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:46.991724+0000", 2024-09-14T09:07:47.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4586999999999999e-05 2024-09-14T09:07:47.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:46.991841+0000", 2024-09-14T09:07:47.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117012 2024-09-14T09:07:47.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op must wait for map", 2024-09-14T09:07:47.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:46.991856+0000", 2024-09-14T09:07:47.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.5044e-05 2024-09-14T09:07:47.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:46.999746+0000", 2024-09-14T09:07:47.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0078896780000000007 2024-09-14T09:07:47.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:46.999760+0000", 2024-09-14T09:07:47.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3723000000000001e-05 2024-09-14T09:07:47.077 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.077 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.077 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.077 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.077 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:47.077 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 1 2024-09-14T09:07:47.077 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:47.087 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:07:47.134 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:47.135+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:47.206 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:47.210+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:47.208 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:47.208 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:47.208 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:47.208 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:47.208 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:47.222 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:07:47.314 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:47.315+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:47.333 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op export --pgid 3.3 --file /var/log/ceph/exp.3.3.4 2024-09-14T09:07:47.395 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:47.395 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:47.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:47.395 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:47.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.395 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73542 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15319040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:47.395 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:47.395 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 73.124472875999999, 2024-09-14T09:07:47.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0237091359999999, 2024-09-14T09:07:47.395 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.395 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.396 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:47.396 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.396 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.396 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.396 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73542 2024-09-14T09:07:47.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.396 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:47.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:47.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268762+0000", 2024-09-14T09:07:47.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:47.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268766+0000", 2024-09-14T09:07:47.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5880000000000002e-06 2024-09-14T09:07:47.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268776+0000", 2024-09-14T09:07:47.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0168e-05 2024-09-14T09:07:47.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268789+0000", 2024-09-14T09:07:47.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:07:47.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282053+0000", 2024-09-14T09:07:47.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013264496000000001 2024-09-14T09:07:47.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282057+0000", 2024-09-14T09:07:47.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6090000000000001e-06 2024-09-14T09:07:47.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292052+0000", 2024-09-14T09:07:47.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0099950070000001 2024-09-14T09:07:47.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292262+0000", 2024-09-14T09:07:47.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021009800000000001 2024-09-14T09:07:47.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292470+0000", 2024-09-14T09:07:47.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000208314 2024-09-14T09:07:47.401 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.401 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.401 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.402 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73543 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:47.402 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:47.402 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 73.124434085999994, 2024-09-14T09:07:47.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023985473, 2024-09-14T09:07:47.402 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.403 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.403 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:47.403 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.403 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.403 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.403 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73543 2024-09-14T09:07:47.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.404 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:47.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:47.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268801+0000", 2024-09-14T09:07:47.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:47.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268803+0000", 2024-09-14T09:07:47.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7439999999999999e-06 2024-09-14T09:07:47.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268812+0000", 2024-09-14T09:07:47.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7080000000000003e-06 2024-09-14T09:07:47.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268819+0000", 2024-09-14T09:07:47.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T09:07:47.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282068+0000", 2024-09-14T09:07:47.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013249540000000001 2024-09-14T09:07:47.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282072+0000", 2024-09-14T09:07:47.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0960000000000001e-06 2024-09-14T09:07:47.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292573+0000", 2024-09-14T09:07:47.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0105014560000001 2024-09-14T09:07:47.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292679+0000", 2024-09-14T09:07:47.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010545899999999999 2024-09-14T09:07:47.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292786+0000", 2024-09-14T09:07:47.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107052 2024-09-14T09:07:47.411 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.411 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.411 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.411 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73544 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:47.411 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:47.411 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 73.124406383999997, 2024-09-14T09:07:47.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0244830549999999, 2024-09-14T09:07:47.412 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.412 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.412 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:47.412 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.412 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.412 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.412 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73544 2024-09-14T09:07:47.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.412 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:47.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:47.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:47.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5599999999999995e-07 2024-09-14T09:07:47.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268831+0000", 2024-09-14T09:07:47.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.57e-06 2024-09-14T09:07:47.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268844+0000", 2024-09-14T09:07:47.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2811e-05 2024-09-14T09:07:47.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268851+0000", 2024-09-14T09:07:47.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9140000000000002e-06 2024-09-14T09:07:47.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282083+0000", 2024-09-14T09:07:47.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013232111 2024-09-14T09:07:47.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282086+0000", 2024-09-14T09:07:47.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2909999999999999e-06 2024-09-14T09:07:47.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292884+0000", 2024-09-14T09:07:47.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0107975069999999 2024-09-14T09:07:47.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293100+0000", 2024-09-14T09:07:47.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021639399999999999 2024-09-14T09:07:47.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293311+0000", 2024-09-14T09:07:47.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021090099999999999 2024-09-14T09:07:47.421 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.421 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.421 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.421 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73545 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15564800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:47.421 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:47.421 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 73.124355198999993, 2024-09-14T09:07:47.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0249746740000001, 2024-09-14T09:07:47.422 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.422 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.422 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:47.422 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.422 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.422 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.422 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73545 2024-09-14T09:07:47.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.422 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:47.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:47.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268880+0000", 2024-09-14T09:07:47.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5600000000000005e-07 2024-09-14T09:07:47.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268889+0000", 2024-09-14T09:07:47.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8559999999999996e-06 2024-09-14T09:07:47.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268913+0000", 2024-09-14T09:07:47.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3958e-05 2024-09-14T09:07:47.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268921+0000", 2024-09-14T09:07:47.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7479999999999999e-06 2024-09-14T09:07:47.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282098+0000", 2024-09-14T09:07:47.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013176297999999999 2024-09-14T09:07:47.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282101+0000", 2024-09-14T09:07:47.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2329999999999998e-06 2024-09-14T09:07:47.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293435+0000", 2024-09-14T09:07:47.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0113339459999999 2024-09-14T09:07:47.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293648+0000", 2024-09-14T09:07:47.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021307100000000001 2024-09-14T09:07:47.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293854+0000", 2024-09-14T09:07:47.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020580800000000001 2024-09-14T09:07:47.426 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.426 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.426 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.427 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73546 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15646720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:47.427 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:47.427 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 73.124301328000001, 2024-09-14T09:07:47.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0254846360000001, 2024-09-14T09:07:47.427 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.427 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.427 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:47.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.427 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.427 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73546 2024-09-14T09:07:47.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268934+0000", 2024-09-14T09:07:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7999999999999995e-07 2024-09-14T09:07:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268937+0000", 2024-09-14T09:07:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9840000000000001e-06 2024-09-14T09:07:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268947+0000", 2024-09-14T09:07:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0522e-05 2024-09-14T09:07:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268955+0000", 2024-09-14T09:07:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8059999999999995e-06 2024-09-14T09:07:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282112+0000", 2024-09-14T09:07:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013157189 2024-09-14T09:07:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282115+0000", 2024-09-14T09:07:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2949999999999998e-06 2024-09-14T09:07:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293968+0000", 2024-09-14T09:07:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0118530990000001 2024-09-14T09:07:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294197+0000", 2024-09-14T09:07:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228507 2024-09-14T09:07:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294417+0000", 2024-09-14T09:07:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000220554 2024-09-14T09:07:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.431 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73547 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 73.124268881999996, 2024-09-14T09:07:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.025970525, 2024-09-14T09:07:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.432 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73547 2024-09-14T09:07:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268966+0000", 2024-09-14T09:07:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3799999999999997e-07 2024-09-14T09:07:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268969+0000", 2024-09-14T09:07:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8739999999999999e-06 2024-09-14T09:07:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268979+0000", 2024-09-14T09:07:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9939999999999995e-06 2024-09-14T09:07:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268986+0000", 2024-09-14T09:07:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3459999999999996e-06 2024-09-14T09:07:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282127+0000", 2024-09-14T09:07:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01314039 2024-09-14T09:07:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282130+0000", 2024-09-14T09:07:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2219999999999998e-06 2024-09-14T09:07:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294540+0000", 2024-09-14T09:07:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124105880000001 2024-09-14T09:07:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294759+0000", 2024-09-14T09:07:47.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000218798 2024-09-14T09:07:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294936+0000", 2024-09-14T09:07:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000176675 2024-09-14T09:07:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73548 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15810560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 73.124237769999993, 2024-09-14T09:07:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0265045129999999, 2024-09-14T09:07:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73548 2024-09-14T09:07:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:47.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268997+0000", 2024-09-14T09:07:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7599999999999996e-07 2024-09-14T09:07:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269000+0000", 2024-09-14T09:07:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0580000000000002e-06 2024-09-14T09:07:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269010+0000", 2024-09-14T09:07:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124e-05 2024-09-14T09:07:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269018+0000", 2024-09-14T09:07:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5399999999999998e-06 2024-09-14T09:07:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282141+0000", 2024-09-14T09:07:47.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013123012 2024-09-14T09:07:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282144+0000", 2024-09-14T09:07:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.196e-06 2024-09-14T09:07:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295042+0000", 2024-09-14T09:07:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0128975689999999 2024-09-14T09:07:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295272+0000", 2024-09-14T09:07:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023048099999999999 2024-09-14T09:07:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295501+0000", 2024-09-14T09:07:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228657 2024-09-14T09:07:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.443 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73549 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15892480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 73.124191890999995, 2024-09-14T09:07:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0268354099999999, 2024-09-14T09:07:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.444 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73549 2024-09-14T09:07:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269043+0000", 2024-09-14T09:07:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6199999999999997e-07 2024-09-14T09:07:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269046+0000", 2024-09-14T09:07:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2480000000000001e-06 2024-09-14T09:07:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269057+0000", 2024-09-14T09:07:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0436e-05 2024-09-14T09:07:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269064+0000", 2024-09-14T09:07:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5179999999999998e-06 2024-09-14T09:07:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282155+0000", 2024-09-14T09:07:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013091190000000001 2024-09-14T09:07:47.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282159+0000", 2024-09-14T09:07:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.366e-06 2024-09-14T09:07:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295599+0000", 2024-09-14T09:07:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0134405559999999 2024-09-14T09:07:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295765+0000", 2024-09-14T09:07:47.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016575499999999999 2024-09-14T09:07:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295878+0000", 2024-09-14T09:07:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112679 2024-09-14T09:07:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73550 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15974400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:47.449 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 73.124159290999998, 2024-09-14T09:07:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027357657, 2024-09-14T09:07:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73550 2024-09-14T09:07:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269076+0000", 2024-09-14T09:07:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:07:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269079+0000", 2024-09-14T09:07:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:07:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269098+0000", 2024-09-14T09:07:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9698e-05 2024-09-14T09:07:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269106+0000", 2024-09-14T09:07:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6179999999999997e-06 2024-09-14T09:07:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282170+0000", 2024-09-14T09:07:47.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013063874 2024-09-14T09:07:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282173+0000", 2024-09-14T09:07:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1999999999999999e-06 2024-09-14T09:07:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295996+0000", 2024-09-14T09:07:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.013823151 2024-09-14T09:07:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296214+0000", 2024-09-14T09:07:47.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021817700000000001 2024-09-14T09:07:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296433+0000", 2024-09-14T09:07:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021829099999999999 2024-09-14T09:07:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73551 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 73.124117874000007, 2024-09-14T09:07:47.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027730276, 2024-09-14T09:07:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73551 2024-09-14T09:07:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.456 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269117+0000", 2024-09-14T09:07:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:07:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269120+0000", 2024-09-14T09:07:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.056e-06 2024-09-14T09:07:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269130+0000", 2024-09-14T09:07:47.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0084e-05 2024-09-14T09:07:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269137+0000", 2024-09-14T09:07:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1359999999999999e-06 2024-09-14T09:07:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282184+0000", 2024-09-14T09:07:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013046537 2024-09-14T09:07:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282188+0000", 2024-09-14T09:07:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3220000000000003e-06 2024-09-14T09:07:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296529+0000", 2024-09-14T09:07:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0143406109999999 2024-09-14T09:07:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296695+0000", 2024-09-14T09:07:47.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016604799999999999 2024-09-14T09:07:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296847+0000", 2024-09-14T09:07:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000151822 2024-09-14T09:07:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73552 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 73.124087294000006, 2024-09-14T09:07:47.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0281767500000001, 2024-09-14T09:07:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73552 2024-09-14T09:07:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:07:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269150+0000", 2024-09-14T09:07:47.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9119999999999998e-06 2024-09-14T09:07:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269160+0000", 2024-09-14T09:07:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6819999999999998e-06 2024-09-14T09:07:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269168+0000", 2024-09-14T09:07:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6079999999999998e-06 2024-09-14T09:07:47.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282205+0000", 2024-09-14T09:07:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013036982000000001 2024-09-14T09:07:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282208+0000", 2024-09-14T09:07:47.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4620000000000001e-06 2024-09-14T09:07:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296968+0000", 2024-09-14T09:07:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0147601209999999 2024-09-14T09:07:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297115+0000", 2024-09-14T09:07:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014673900000000001 2024-09-14T09:07:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297324+0000", 2024-09-14T09:07:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020866400000000001 2024-09-14T09:07:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73553 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 73.124057230000005, 2024-09-14T09:07:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0286556739999999, 2024-09-14T09:07:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:47.467 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73553 2024-09-14T09:07:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269178+0000", 2024-09-14T09:07:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:07:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269180+0000", 2024-09-14T09:07:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T09:07:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269189+0000", 2024-09-14T09:07:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2e-06 2024-09-14T09:07:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269197+0000", 2024-09-14T09:07:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2980000000000003e-06 2024-09-14T09:07:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282219+0000", 2024-09-14T09:07:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013022687 2024-09-14T09:07:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282223+0000", 2024-09-14T09:07:47.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.241e-06 2024-09-14T09:07:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297470+0000", 2024-09-14T09:07:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.015247778 2024-09-14T09:07:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297704+0000", 2024-09-14T09:07:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000233367 2024-09-14T09:07:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297833+0000", 2024-09-14T09:07:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012888900000000001 2024-09-14T09:07:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73554 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:47.473 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 73.124027913000006, 2024-09-14T09:07:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.029005113, 2024-09-14T09:07:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73554 2024-09-14T09:07:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269207+0000", 2024-09-14T09:07:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.68e-07 2024-09-14T09:07:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269210+0000", 2024-09-14T09:07:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8100000000000002e-06 2024-09-14T09:07:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269219+0000", 2024-09-14T09:07:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:07:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269227+0000", 2024-09-14T09:07:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3839999999999999e-06 2024-09-14T09:07:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282234+0000", 2024-09-14T09:07:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013007187 2024-09-14T09:07:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282237+0000", 2024-09-14T09:07:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1200000000000002e-06 2024-09-14T09:07:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297942+0000", 2024-09-14T09:07:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0157048040000001 2024-09-14T09:07:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298087+0000", 2024-09-14T09:07:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014499099999999999 2024-09-14T09:07:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298211+0000", 2024-09-14T09:07:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012461899999999999 2024-09-14T09:07:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73555 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 73.123998064999995, 2024-09-14T09:07:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.02941558, 2024-09-14T09:07:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.479 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73555 2024-09-14T09:07:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269237+0000", 2024-09-14T09:07:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3399999999999999e-07 2024-09-14T09:07:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269240+0000", 2024-09-14T09:07:47.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8480000000000001e-06 2024-09-14T09:07:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269249+0000", 2024-09-14T09:07:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1880000000000008e-06 2024-09-14T09:07:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269256+0000", 2024-09-14T09:07:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:07:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282248+0000", 2024-09-14T09:07:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012992327 2024-09-14T09:07:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282251+0000", 2024-09-14T09:07:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2160000000000002e-06 2024-09-14T09:07:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298299+0000", 2024-09-14T09:07:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0160479330000001 2024-09-14T09:07:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298446+0000", 2024-09-14T09:07:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000146525 2024-09-14T09:07:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298652+0000", 2024-09-14T09:07:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020584100000000001 2024-09-14T09:07:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.484 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73556 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 73.123968318999999, 2024-09-14T09:07:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0299854239999999, 2024-09-14T09:07:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73556 2024-09-14T09:07:47.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269270+0000", 2024-09-14T09:07:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0920000000000002e-06 2024-09-14T09:07:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269280+0000", 2024-09-14T09:07:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0013999999999999e-05 2024-09-14T09:07:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269288+0000", 2024-09-14T09:07:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:07:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282263+0000", 2024-09-14T09:07:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012974703000000001 2024-09-14T09:07:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282266+0000", 2024-09-14T09:07:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1520000000000001e-06 2024-09-14T09:07:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298729+0000", 2024-09-14T09:07:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.016463506 2024-09-14T09:07:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298884+0000", 2024-09-14T09:07:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015450800000000001 2024-09-14T09:07:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299251+0000", 2024-09-14T09:07:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036759899999999997 2024-09-14T09:07:47.489 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73557 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16547840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 73.123921668999998, 2024-09-14T09:07:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0307969480000001, 2024-09-14T09:07:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.490 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73557 2024-09-14T09:07:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269313+0000", 2024-09-14T09:07:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.92e-07 2024-09-14T09:07:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269336+0000", 2024-09-14T09:07:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3118000000000001e-05 2024-09-14T09:07:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269346+0000", 2024-09-14T09:07:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4069999999999999e-06 2024-09-14T09:07:47.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269352+0000", 2024-09-14T09:07:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5810000000000002e-06 2024-09-14T09:07:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282277+0000", 2024-09-14T09:07:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012924658 2024-09-14T09:07:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282280+0000", 2024-09-14T09:07:47.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2880000000000001e-06 2024-09-14T09:07:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299343+0000", 2024-09-14T09:07:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0170630190000001 2024-09-14T09:07:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299554+0000", 2024-09-14T09:07:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021085099999999999 2024-09-14T09:07:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300109+0000", 2024-09-14T09:07:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00055533399999999999 2024-09-14T09:07:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73558 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 73.123870045999993, 2024-09-14T09:07:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0314849340000001, 2024-09-14T09:07:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:47.495 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73558 2024-09-14T09:07:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:47.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269365+0000", 2024-09-14T09:07:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7400000000000003e-07 2024-09-14T09:07:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269367+0000", 2024-09-14T09:07:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4930000000000001e-06 2024-09-14T09:07:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269374+0000", 2024-09-14T09:07:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9519999999999997e-06 2024-09-14T09:07:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269379+0000", 2024-09-14T09:07:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9710000000000003e-06 2024-09-14T09:07:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282292+0000", 2024-09-14T09:07:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012912704000000001 2024-09-14T09:07:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282295+0000", 2024-09-14T09:07:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.179e-06 2024-09-14T09:07:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300195+0000", 2024-09-14T09:07:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.017899975 2024-09-14T09:07:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300345+0000", 2024-09-14T09:07:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014957499999999999 2024-09-14T09:07:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300849+0000", 2024-09-14T09:07:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050451100000000002 2024-09-14T09:07:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.500 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73559 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 73.123847636999997, 2024-09-14T09:07:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.031916343, 2024-09-14T09:07:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.501 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73559 2024-09-14T09:07:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3500000000000002e-07 2024-09-14T09:07:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269390+0000", 2024-09-14T09:07:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5890000000000001e-06 2024-09-14T09:07:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269396+0000", 2024-09-14T09:07:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3300000000000004e-06 2024-09-14T09:07:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269401+0000", 2024-09-14T09:07:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0340000000000004e-06 2024-09-14T09:07:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282311+0000", 2024-09-14T09:07:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012910204 2024-09-14T09:07:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282317+0000", 2024-09-14T09:07:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2639999999999997e-06 2024-09-14T09:07:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300933+0000", 2024-09-14T09:07:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0186160710000001 2024-09-14T09:07:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301158+0000", 2024-09-14T09:07:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225021 2024-09-14T09:07:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301303+0000", 2024-09-14T09:07:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014439499999999999 2024-09-14T09:07:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106964 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:07:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.277919803, 2024-09-14T09:07:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8802924409999999, 2024-09-14T09:07:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106964 2024-09-14T09:07:47.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:07:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:07:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115317+0000", 2024-09-14T09:07:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6429999999999999e-06 2024-09-14T09:07:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115326+0000", 2024-09-14T09:07:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2140000000000002e-06 2024-09-14T09:07:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115346+0000", 2024-09-14T09:07:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9419000000000002e-05 2024-09-14T09:07:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115357+0000", 2024-09-14T09:07:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1014e-05 2024-09-14T09:07:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115431+0000", 2024-09-14T09:07:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4040000000000003e-05 2024-09-14T09:07:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115438+0000", 2024-09-14T09:07:47.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9920000000000001e-06 2024-09-14T09:07:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844944+0000", 2024-09-14T09:07:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72950628399999995 2024-09-14T09:07:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844961+0000", 2024-09-14T09:07:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7156e-05 2024-09-14T09:07:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988004+0000", 2024-09-14T09:07:47.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.14304297 2024-09-14T09:07:47.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988511+0000", 2024-09-14T09:07:47.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050694100000000001 2024-09-14T09:07:47.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:47.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.989802+0000", 2024-09-14T09:07:47.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001291195 2024-09-14T09:07:47.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:47.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994866+0000", 2024-09-14T09:07:47.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050632760000000002 2024-09-14T09:07:47.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:47.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994988+0000", 2024-09-14T09:07:47.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012252899999999999 2024-09-14T09:07:47.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:47.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995198+0000", 2024-09-14T09:07:47.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021010899999999999 2024-09-14T09:07:47.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995607+0000", 2024-09-14T09:07:47.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040865900000000002 2024-09-14T09:07:47.514 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.514 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.514 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.514 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106965 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:47.514 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:07:47.514 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.276717426000001, 2024-09-14T09:07:47.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.879730162, 2024-09-14T09:07:47.515 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:47.515 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:47.515 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:47.515 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:47.515 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:47.515 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:47.515 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106965 2024-09-14T09:07:47.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.515 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:47.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:47.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:07:47.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:47.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:07:47.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:47.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:47.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:07:47.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:07:47.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:47.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116522+0000", 2024-09-14T09:07:47.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3989999999999997e-06 2024-09-14T09:07:47.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:47.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116533+0000", 2024-09-14T09:07:47.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0669000000000001e-05 2024-09-14T09:07:47.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:47.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116541+0000", 2024-09-14T09:07:47.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2730000000000002e-06 2024-09-14T09:07:47.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116604+0000", 2024-09-14T09:07:47.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2706999999999994e-05 2024-09-14T09:07:47.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116610+0000", 2024-09-14T09:07:47.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8630000000000002e-06 2024-09-14T09:07:47.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845060+0000", 2024-09-14T09:07:47.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72845003699999999 2024-09-14T09:07:47.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:47.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845079+0000", 2024-09-14T09:07:47.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8287999999999998e-05 2024-09-14T09:07:47.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:47.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.991866+0000", 2024-09-14T09:07:47.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1467871519999999 2024-09-14T09:07:47.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:47.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.992274+0000", 2024-09-14T09:07:47.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040813500000000001 2024-09-14T09:07:47.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:47.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.993538+0000", 2024-09-14T09:07:47.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001263655 2024-09-14T09:07:47.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:47.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995770+0000", 2024-09-14T09:07:47.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002231949 2024-09-14T09:07:47.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:47.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995810+0000", 2024-09-14T09:07:47.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0852999999999997e-05 2024-09-14T09:07:47.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:47.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995979+0000", 2024-09-14T09:07:47.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016866 2024-09-14T09:07:47.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:47.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:47.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:47.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996247+0000", 2024-09-14T09:07:47.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000267769 2024-09-14T09:07:47.522 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.522 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.522 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.522 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:47.522 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:47.522 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:47.523 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:07:47.524 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:47.523+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:47.624 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:47.626+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:47.695 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:47.695 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:47.695 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:47.695 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:47.709 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:07:47.754 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:47.755+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:47.826 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:47.827+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:47.828 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:47.829 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:47.829 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:47.829 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:47.829 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:47.843 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:07:47.848 INFO:teuthology.orchestra.run.smithi183.stderr:Exporting 3.3 info 3.3( v 134'569 (0'0,134'569] local-lis/les=86/87 n=72 ec=23/23 lis/c=86/86 les/c/f=87/87/0 sis=86) 2024-09-14T09:07:47.854 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c036442c:::rbd_data.115e77b799c3.000000000000000a:20# 2024-09-14T09:07:47.856 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c036442c:::rbd_data.115e77b799c3.000000000000000a:head# 2024-09-14T09:07:47.856 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c038bd58:::rbd_data.115e9a87c1b2.0000000000000017:21# 2024-09-14T09:07:47.856 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c038bd58:::rbd_data.115e9a87c1b2.0000000000000017:head# 2024-09-14T09:07:47.857 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c03e445b:::rbd_data.115ee10e61a5.0000000000000015:head# 2024-09-14T09:07:47.857 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c046d2dc:::rbd_object_map.115e63832e35.0000000000000023:head# 2024-09-14T09:07:47.857 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c0779ca8:::rbd_data.115e2588c85c.000000000000009c:head# 2024-09-14T09:07:47.857 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c085c20b:::rbd_object_map.115e5dc498ac.0000000000000014:head# 2024-09-14T09:07:47.858 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c0fdc971:::rbd_data.115e2eebe49a.000000000000014a:1d# 2024-09-14T09:07:47.858 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c0fdc971:::rbd_data.115e2eebe49a.000000000000014a:head# 2024-09-14T09:07:47.859 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c10f85e1:::rbd_data.115eb62c5c2b.0000000000000395:17# 2024-09-14T09:07:47.859 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c10f85e1:::rbd_data.115eb62c5c2b.0000000000000395:head# 2024-09-14T09:07:47.859 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c14df650:::rbd_data.115ed0fcd179.00000000000000e6:head# 2024-09-14T09:07:47.859 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c16c5059:::rbd_data.115ee10e61a5.0000000000000110:head# 2024-09-14T09:07:47.860 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c1881537:::rbd_data.115e2baf3d9a.000000000000003f:head# 2024-09-14T09:07:47.861 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c1e37c2b:::rbd_data.115e9d49ac1.0000000000000005:b# 2024-09-14T09:07:47.861 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c1e37c2b:::rbd_data.115e9d49ac1.0000000000000005:head# 2024-09-14T09:07:47.861 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27# 2024-09-14T09:07:47.863 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:head# 2024-09-14T09:07:47.863 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c2a58dd5:::rbd_data.115e3abd3b2c.0000000000000031:head# 2024-09-14T09:07:47.863 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c2d99c9b:::rbd_data.115eb62c5c2b.000000000000014f:17# 2024-09-14T09:07:47.864 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c2d99c9b:::rbd_data.115eb62c5c2b.000000000000014f:head# 2024-09-14T09:07:47.864 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c349b650:::rbd_data.115e9d49ac1.000000000000004e:b# 2024-09-14T09:07:47.864 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c349b650:::rbd_data.115e9d49ac1.000000000000004e:head# 2024-09-14T09:07:47.864 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c35ec4f9:::rbd_data.115e2588c85c.0000000000000050:29# 2024-09-14T09:07:47.865 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c35ec4f9:::rbd_data.115e2588c85c.0000000000000050:head# 2024-09-14T09:07:47.865 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c38815bf:::rbd_data.115e9a87c1b2.000000000000000c:21# 2024-09-14T09:07:47.866 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c38815bf:::rbd_data.115e9a87c1b2.000000000000000c:head# 2024-09-14T09:07:47.866 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c3b0db07:::rbd_data.115e5dc498ac.0000000000000049:14# 2024-09-14T09:07:47.868 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c3b0db07:::rbd_data.115e5dc498ac.0000000000000049:head# 2024-09-14T09:07:47.868 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c3ffd0ef:::rbd_data.115ec297f3f3.0000000000000052:head# 2024-09-14T09:07:47.868 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c40097e4:::rbd_id.image_client.0-clone37:head# 2024-09-14T09:07:47.869 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c415229f:::rbd_data.115e9a87c1b2.0000000000000016:21# 2024-09-14T09:07:47.869 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c415229f:::rbd_data.115e9a87c1b2.0000000000000016:head# 2024-09-14T09:07:47.869 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c44034d2:::rbd_data.115e2eebe49a.0000000000000021:1d# 2024-09-14T09:07:47.871 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c44034d2:::rbd_data.115e2eebe49a.0000000000000021:head# 2024-09-14T09:07:47.871 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c440a843:::rbd_data.115ef5361ff8.0000000000000069:16# 2024-09-14T09:07:47.871 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c440a843:::rbd_data.115ef5361ff8.0000000000000069:head# 2024-09-14T09:07:47.872 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c44430d4:::rbd_data.115e77b799c3.0000000000000109:20# 2024-09-14T09:07:47.872 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c44430d4:::rbd_data.115e77b799c3.0000000000000109:head# 2024-09-14T09:07:47.872 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c4594dd3:::rbd_data.115e21c31fcf.0000000000000025:head# 2024-09-14T09:07:47.873 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c4ebb483:::rbd_data.115e32e50606.0000000000000003:13# 2024-09-14T09:07:47.876 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c4ebb483:::rbd_data.115e32e50606.0000000000000003:head# 2024-09-14T09:07:47.877 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c4fc38e9:::rbd_data.115e2588c85c.0000000000000228:29# 2024-09-14T09:07:47.877 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c4fc38e9:::rbd_data.115e2588c85c.0000000000000228:head# 2024-09-14T09:07:47.878 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c4fd7e8e:::rbd_data.115e113dc8ca.000000000000000c:head# 2024-09-14T09:07:47.878 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c508885d:::rbd_data.115eb73c120f.0000000000000016:head# 2024-09-14T09:07:47.878 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c5320280:::rbd_object_map.115e17d15072:head# 2024-09-14T09:07:47.878 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c541298f:::rbd_data.115ebc3c67d1.0000000000000015:head# 2024-09-14T09:07:47.879 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c54429df:::rbd_data.115ee7a2dde0.0000000000000000:10# 2024-09-14T09:07:47.892 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c54429df:::rbd_data.115ee7a2dde0.0000000000000000:head# 2024-09-14T09:07:47.900 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c5644c39:::rbd_data.115e9d49ac1.000000000000000d:b# 2024-09-14T09:07:47.901 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c5644c39:::rbd_data.115e9d49ac1.000000000000000d:head# 2024-09-14T09:07:47.901 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c589307d:::rbd_data.115e96ceebb2.0000000000000004:2a# 2024-09-14T09:07:47.903 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c589307d:::rbd_data.115e96ceebb2.0000000000000004:head# 2024-09-14T09:07:47.903 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c5b07644:::rbd_data.115e2588c85c.000000000000004c:head# 2024-09-14T09:07:47.903 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c5caae6c:::rbd_data.115e2588c85c.000000000000008d:29# 2024-09-14T09:07:47.904 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c5caae6c:::rbd_data.115e2588c85c.000000000000008d:head# 2024-09-14T09:07:47.904 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c5cf7412:::rbd_object_map.115e4c71d6e8:head# 2024-09-14T09:07:47.904 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c5d5ef46:::rbd_data.115ebc3c67d1.000000000000000d:head# 2024-09-14T09:07:47.905 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c5e39d02:::rbd_data.115ec297f3f3.000000000000006a:15# 2024-09-14T09:07:47.905 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c5e39d02:::rbd_data.115ec297f3f3.000000000000006a:head# 2024-09-14T09:07:47.905 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c62e0486:::rbd_data.115e9e133b92.0000000000000000:head# 2024-09-14T09:07:47.918 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c67eb973:::rbd_data.115ee10e61a5.000000000000000a:head# 2024-09-14T09:07:47.919 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c696d26d:::rbd_data.115ec19646f4.0000000000000078:2b# 2024-09-14T09:07:47.919 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c696d26d:::rbd_data.115ec19646f4.0000000000000078:head# 2024-09-14T09:07:47.919 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c6d19d67:::rbd_data.115e889d4809.000000000000000c:a# 2024-09-14T09:07:47.919 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c6d19d67:::rbd_data.115e889d4809.000000000000000c:head# 2024-09-14T09:07:47.920 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c6d67aab:::rbd_header.115e7770ddc5:head# 2024-09-14T09:07:47.920 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c7113cc8:::rbd_header.115ea8d131ff:head# 2024-09-14T09:07:47.920 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c7408ca1:::rbd_data.115eb62c5c2b.0000000000000240:17# 2024-09-14T09:07:47.920 INFO:teuthology.orchestra.run.smithi183.stderr:Read #3:c7408ca1:::rbd_data.115eb62c5c2b.0000000000000240:head# 2024-09-14T09:07:47.920 INFO:teuthology.orchestra.run.smithi183.stderr:Export successful 2024-09-14T09:07:47.934 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:47.935+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:48.000 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T09:07:48.006 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:48.007 INFO:teuthology.orchestra.run.smithi176.stdout:5919 trunc from 0x2cf38ef to 0x33d55cd 2024-09-14T09:07:48.020 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:48.020 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:48.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:48.020 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:48.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.021 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106705 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 518144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.021 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:07:48.021 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.173168168, 2024-09-14T09:07:48.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3001326799999999, 2024-09-14T09:07:48.021 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.021 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.021 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.021 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.021 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.021 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.021 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106705 2024-09-14T09:07:48.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.021 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:07:48.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:07:48.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:07:48.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4699999999999998e-07 2024-09-14T09:07:48.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847500+0000", 2024-09-14T09:07:48.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0860000000000001e-06 2024-09-14T09:07:48.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847503+0000", 2024-09-14T09:07:48.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1269999999999999e-06 2024-09-14T09:07:48.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847506+0000", 2024-09-14T09:07:48.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5780000000000001e-06 2024-09-14T09:07:48.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858911+0000", 2024-09-14T09:07:48.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011404925 2024-09-14T09:07:48.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858919+0000", 2024-09-14T09:07:48.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8620000000000003e-06 2024-09-14T09:07:48.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147254+0000", 2024-09-14T09:07:48.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2883353719999999 2024-09-14T09:07:48.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147408+0000", 2024-09-14T09:07:48.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015389200000000001 2024-09-14T09:07:48.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147631+0000", 2024-09-14T09:07:48.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022359100000000001 2024-09-14T09:07:48.025 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.025 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.025 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.026 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106707 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 522240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.026 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:07:48.026 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.173157698000001, 2024-09-14T09:07:48.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.300471801, 2024-09-14T09:07:48.026 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.026 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.026 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.026 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.026 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.026 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.026 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106707 2024-09-14T09:07:48.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.026 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:07:48.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:07:48.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:07:48.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.23e-07 2024-09-14T09:07:48.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847514+0000", 2024-09-14T09:07:48.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9150000000000004e-06 2024-09-14T09:07:48.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847520+0000", 2024-09-14T09:07:48.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.203e-06 2024-09-14T09:07:48.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847532+0000", 2024-09-14T09:07:48.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2441e-05 2024-09-14T09:07:48.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858953+0000", 2024-09-14T09:07:48.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01142108 2024-09-14T09:07:48.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858957+0000", 2024-09-14T09:07:48.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2830000000000003e-06 2024-09-14T09:07:48.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147667+0000", 2024-09-14T09:07:48.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2887094619999999 2024-09-14T09:07:48.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147784+0000", 2024-09-14T09:07:48.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117465 2024-09-14T09:07:48.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147981+0000", 2024-09-14T09:07:48.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019672900000000001 2024-09-14T09:07:48.030 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.030 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.031 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.031 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106709 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 3072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.031 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:07:48.031 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.173129407000001, 2024-09-14T09:07:48.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.301335192, 2024-09-14T09:07:48.031 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.031 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.031 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.031 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.031 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.031 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.031 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106709 2024-09-14T09:07:48.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.032 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:07:48.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:07:48.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847538+0000", 2024-09-14T09:07:48.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2800000000000003e-07 2024-09-14T09:07:48.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847539+0000", 2024-09-14T09:07:48.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2720000000000001e-06 2024-09-14T09:07:48.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847544+0000", 2024-09-14T09:07:48.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7219999999999999e-06 2024-09-14T09:07:48.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847547+0000", 2024-09-14T09:07:48.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9960000000000002e-06 2024-09-14T09:07:48.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.034 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858991+0000", 2024-09-14T09:07:48.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011444218000000001 2024-09-14T09:07:48.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.034 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858999+0000", 2024-09-14T09:07:48.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9419999999999995e-06 2024-09-14T09:07:48.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148053+0000", 2024-09-14T09:07:48.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2890544099999999 2024-09-14T09:07:48.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148203+0000", 2024-09-14T09:07:48.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149419 2024-09-14T09:07:48.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148873+0000", 2024-09-14T09:07:48.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00066988499999999995 2024-09-14T09:07:48.035 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.035 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.036 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.036 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.036 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.036 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106711 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 7168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.036 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:07:48.036 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.173116148999998, 2024-09-14T09:07:48.036 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3022241219999999, 2024-09-14T09:07:48.036 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.036 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.036 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.036 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.036 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.036 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.037 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106711 2024-09-14T09:07:48.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.037 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:07:48.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.037 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:07:48.037 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.037 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.037 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.037 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:07:48.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8000000000000002e-07 2024-09-14T09:07:48.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847552+0000", 2024-09-14T09:07:48.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1340000000000001e-06 2024-09-14T09:07:48.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.038 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.038 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847557+0000", 2024-09-14T09:07:48.038 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6179999999999997e-06 2024-09-14T09:07:48.038 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.038 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847560+0000", 2024-09-14T09:07:48.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5899999999999999e-06 2024-09-14T09:07:48.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859050+0000", 2024-09-14T09:07:48.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01148914 2024-09-14T09:07:48.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.039 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.039 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859056+0000", 2024-09-14T09:07:48.039 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6630000000000004e-06 2024-09-14T09:07:48.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.039 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148948+0000", 2024-09-14T09:07:48.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2898914370000001 2024-09-14T09:07:48.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149079+0000", 2024-09-14T09:07:48.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013145400000000001 2024-09-14T09:07:48.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.040 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.040 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.040 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149775+0000", 2024-09-14T09:07:48.040 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00069580599999999999 2024-09-14T09:07:48.040 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.041 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.041 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.041 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.041 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.041 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106713 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 11264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.041 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:07:48.041 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.173102684, 2024-09-14T09:07:48.041 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3035188959999999, 2024-09-14T09:07:48.041 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.041 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.041 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.041 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.041 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.042 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.042 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106713 2024-09-14T09:07:48.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.042 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.042 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.042 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:07:48.042 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.042 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.042 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:07:48.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:07:48.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6e-07 2024-09-14T09:07:48.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.043 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.043 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.043 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847566+0000", 2024-09-14T09:07:48.043 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1039999999999999e-06 2024-09-14T09:07:48.043 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847569+0000", 2024-09-14T09:07:48.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.191e-06 2024-09-14T09:07:48.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847571+0000", 2024-09-14T09:07:48.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6390000000000001e-06 2024-09-14T09:07:48.044 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.044 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.044 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.044 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859099+0000", 2024-09-14T09:07:48.044 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011527379000000001 2024-09-14T09:07:48.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859106+0000", 2024-09-14T09:07:48.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3679999999999996e-06 2024-09-14T09:07:48.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149878+0000", 2024-09-14T09:07:48.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.29077208 2024-09-14T09:07:48.045 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.045 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.045 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.045 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.150175+0000", 2024-09-14T09:07:48.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029685699999999998 2024-09-14T09:07:48.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.046 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.046 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151083+0000", 2024-09-14T09:07:48.046 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00090801800000000002 2024-09-14T09:07:48.046 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.046 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.046 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.046 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.046 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106715 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 15360~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.046 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:07:48.046 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.173091674999998, 2024-09-14T09:07:48.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3047478800000001, 2024-09-14T09:07:48.047 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.047 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.047 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.047 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.047 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.047 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.047 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106715 2024-09-14T09:07:48.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.047 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.047 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.047 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.047 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:07:48.047 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.047 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:07:48.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.048 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:07:48.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6e-07 2024-09-14T09:07:48.048 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.048 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.048 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847580+0000", 2024-09-14T09:07:48.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9250000000000003e-06 2024-09-14T09:07:48.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847585+0000", 2024-09-14T09:07:48.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8409999999999999e-06 2024-09-14T09:07:48.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847588+0000", 2024-09-14T09:07:48.049 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6989999999999999e-06 2024-09-14T09:07:48.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859151+0000", 2024-09-14T09:07:48.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011562604000000001 2024-09-14T09:07:48.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859159+0000", 2024-09-14T09:07:48.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4179999999999997e-06 2024-09-14T09:07:48.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151155+0000", 2024-09-14T09:07:48.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.291996068 2024-09-14T09:07:48.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151383+0000", 2024-09-14T09:07:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022746100000000001 2024-09-14T09:07:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152323+0000", 2024-09-14T09:07:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00094060400000000005 2024-09-14T09:07:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106717 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 19456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:07:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.173074987, 2024-09-14T09:07:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.305574579, 2024-09-14T09:07:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106717 2024-09-14T09:07:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.052 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:07:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:07:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:07:48.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1199999999999999e-07 2024-09-14T09:07:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847593+0000", 2024-09-14T09:07:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.257e-06 2024-09-14T09:07:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847598+0000", 2024-09-14T09:07:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4139999999999996e-06 2024-09-14T09:07:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847602+0000", 2024-09-14T09:07:48.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3440000000000003e-06 2024-09-14T09:07:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859203+0000", 2024-09-14T09:07:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011600572999999999 2024-09-14T09:07:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859211+0000", 2024-09-14T09:07:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-06 2024-09-14T09:07:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152398+0000", 2024-09-14T09:07:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.293187812 2024-09-14T09:07:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152543+0000", 2024-09-14T09:07:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014478000000000001 2024-09-14T09:07:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153166+0000", 2024-09-14T09:07:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000623257 2024-09-14T09:07:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.056 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106719 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 23552~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:07:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.173060671000002, 2024-09-14T09:07:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.306394982, 2024-09-14T09:07:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.057 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106719 2024-09-14T09:07:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:07:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:07:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:07:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5100000000000001e-07 2024-09-14T09:07:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847608+0000", 2024-09-14T09:07:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.189e-06 2024-09-14T09:07:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847612+0000", 2024-09-14T09:07:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9519999999999996e-06 2024-09-14T09:07:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847614+0000", 2024-09-14T09:07:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6280000000000001e-06 2024-09-14T09:07:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859254+0000", 2024-09-14T09:07:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011640157 2024-09-14T09:07:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859261+0000", 2024-09-14T09:07:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0570000000000004e-06 2024-09-14T09:07:48.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153236+0000", 2024-09-14T09:07:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2939744040000001 2024-09-14T09:07:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153380+0000", 2024-09-14T09:07:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014410399999999999 2024-09-14T09:07:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154001+0000", 2024-09-14T09:07:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062124000000000001 2024-09-14T09:07:48.061 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106721 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 27648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:07:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.173049021000001, 2024-09-14T09:07:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3074446659999999, 2024-09-14T09:07:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.062 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106721 2024-09-14T09:07:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:07:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:07:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:07:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4299999999999999e-07 2024-09-14T09:07:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847619+0000", 2024-09-14T09:07:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T09:07:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847624+0000", 2024-09-14T09:07:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4229999999999998e-06 2024-09-14T09:07:48.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847627+0000", 2024-09-14T09:07:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0120000000000001e-06 2024-09-14T09:07:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859305+0000", 2024-09-14T09:07:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01167812 2024-09-14T09:07:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859313+0000", 2024-09-14T09:07:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2409999999999995e-06 2024-09-14T09:07:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154081+0000", 2024-09-14T09:07:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.294768296 2024-09-14T09:07:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154280+0000", 2024-09-14T09:07:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019824 2024-09-14T09:07:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155063+0000", 2024-09-14T09:07:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00078303400000000005 2024-09-14T09:07:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106723 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 31744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:07:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.173036590999999, 2024-09-14T09:07:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308293648, 2024-09-14T09:07:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.067 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106723 2024-09-14T09:07:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:07:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:07:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847631+0000", 2024-09-14T09:07:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6100000000000002e-07 2024-09-14T09:07:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847632+0000", 2024-09-14T09:07:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0589999999999999e-06 2024-09-14T09:07:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847635+0000", 2024-09-14T09:07:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.551e-06 2024-09-14T09:07:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847638+0000", 2024-09-14T09:07:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.644e-06 2024-09-14T09:07:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859344+0000", 2024-09-14T09:07:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011706397 2024-09-14T09:07:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859348+0000", 2024-09-14T09:07:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5779999999999999e-06 2024-09-14T09:07:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155135+0000", 2024-09-14T09:07:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.295787448 2024-09-14T09:07:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155279+0000", 2024-09-14T09:07:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014400700000000001 2024-09-14T09:07:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155924+0000", 2024-09-14T09:07:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00064470300000000005 2024-09-14T09:07:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106725 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 35840~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:07:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.17302548, 2024-09-14T09:07:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.309103218, 2024-09-14T09:07:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.072 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106725 2024-09-14T09:07:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:07:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:07:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847642+0000", 2024-09-14T09:07:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4499999999999998e-07 2024-09-14T09:07:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847643+0000", 2024-09-14T09:07:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4e-07 2024-09-14T09:07:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847647+0000", 2024-09-14T09:07:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6190000000000002e-06 2024-09-14T09:07:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847655+0000", 2024-09-14T09:07:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:07:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859381+0000", 2024-09-14T09:07:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011725793 2024-09-14T09:07:48.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859391+0000", 2024-09-14T09:07:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4390000000000006e-06 2024-09-14T09:07:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155997+0000", 2024-09-14T09:07:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2966067830000001 2024-09-14T09:07:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156133+0000", 2024-09-14T09:07:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000135609 2024-09-14T09:07:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156745+0000", 2024-09-14T09:07:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000611592 2024-09-14T09:07:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106727 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 39936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:07:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.173007179999999, 2024-09-14T09:07:48.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3099046560000001, 2024-09-14T09:07:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106727 2024-09-14T09:07:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:07:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:07:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:07:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1300000000000001e-07 2024-09-14T09:07:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847661+0000", 2024-09-14T09:07:48.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.192e-06 2024-09-14T09:07:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847665+0000", 2024-09-14T09:07:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0999999999999997e-06 2024-09-14T09:07:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847668+0000", 2024-09-14T09:07:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.644e-06 2024-09-14T09:07:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859422+0000", 2024-09-14T09:07:48.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011753559 2024-09-14T09:07:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859425+0000", 2024-09-14T09:07:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.343e-06 2024-09-14T09:07:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156817+0000", 2024-09-14T09:07:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2973924800000001 2024-09-14T09:07:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156956+0000", 2024-09-14T09:07:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013835 2024-09-14T09:07:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157564+0000", 2024-09-14T09:07:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060867500000000004 2024-09-14T09:07:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106729 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 44032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:07:48.082 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.172995207, 2024-09-14T09:07:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3107195970000001, 2024-09-14T09:07:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106729 2024-09-14T09:07:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:07:48.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:07:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:07:48.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.29e-07 2024-09-14T09:07:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847673+0000", 2024-09-14T09:07:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.096e-06 2024-09-14T09:07:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847678+0000", 2024-09-14T09:07:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.527e-06 2024-09-14T09:07:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847681+0000", 2024-09-14T09:07:48.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.788e-06 2024-09-14T09:07:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859462+0000", 2024-09-14T09:07:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01178032 2024-09-14T09:07:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859471+0000", 2024-09-14T09:07:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3430000000000002e-06 2024-09-14T09:07:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157635+0000", 2024-09-14T09:07:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2981640320000001 2024-09-14T09:07:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157773+0000", 2024-09-14T09:07:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763 2024-09-14T09:07:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158391+0000", 2024-09-14T09:07:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061863200000000004 2024-09-14T09:07:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.087 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106731 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 48128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:07:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.172979561999998, 2024-09-14T09:07:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3115142020000001, 2024-09-14T09:07:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106731 2024-09-14T09:07:48.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:07:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:07:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.089 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847688+0000", 2024-09-14T09:07:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0199999999999998e-07 2024-09-14T09:07:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847689+0000", 2024-09-14T09:07:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4529999999999999e-06 2024-09-14T09:07:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847694+0000", 2024-09-14T09:07:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9080000000000003e-06 2024-09-14T09:07:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847697+0000", 2024-09-14T09:07:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4920000000000002e-06 2024-09-14T09:07:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859499+0000", 2024-09-14T09:07:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011801676000000001 2024-09-14T09:07:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859502+0000", 2024-09-14T09:07:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2899999999999998e-06 2024-09-14T09:07:48.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158461+0000", 2024-09-14T09:07:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.298958796 2024-09-14T09:07:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158594+0000", 2024-09-14T09:07:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013321599999999999 2024-09-14T09:07:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159202+0000", 2024-09-14T09:07:48.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060706900000000001 2024-09-14T09:07:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106733 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 52224~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:07:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.172965305999998, 2024-09-14T09:07:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3123343809999999, 2024-09-14T09:07:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.093 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106733 2024-09-14T09:07:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:07:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:07:48.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:07:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5800000000000001e-07 2024-09-14T09:07:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847703+0000", 2024-09-14T09:07:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.127e-06 2024-09-14T09:07:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847706+0000", 2024-09-14T09:07:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3210000000000001e-06 2024-09-14T09:07:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847709+0000", 2024-09-14T09:07:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7539999999999998e-06 2024-09-14T09:07:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859538+0000", 2024-09-14T09:07:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01182942 2024-09-14T09:07:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859547+0000", 2024-09-14T09:07:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T09:07:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159273+0000", 2024-09-14T09:07:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2997263700000001 2024-09-14T09:07:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159415+0000", 2024-09-14T09:07:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000141765 2024-09-14T09:07:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160036+0000", 2024-09-14T09:07:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062122100000000003 2024-09-14T09:07:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106735 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 56320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:07:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.172953862, 2024-09-14T09:07:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3131339309999999, 2024-09-14T09:07:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.098 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106735 2024-09-14T09:07:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:07:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:07:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:07:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2100000000000001e-07 2024-09-14T09:07:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847714+0000", 2024-09-14T09:07:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.051e-06 2024-09-14T09:07:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847719+0000", 2024-09-14T09:07:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3640000000000001e-06 2024-09-14T09:07:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847723+0000", 2024-09-14T09:07:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8789999999999996e-06 2024-09-14T09:07:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859577+0000", 2024-09-14T09:07:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011854416 2024-09-14T09:07:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859580+0000", 2024-09-14T09:07:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.292e-06 2024-09-14T09:07:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160108+0000", 2024-09-14T09:07:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.30052758 2024-09-14T09:07:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160238+0000", 2024-09-14T09:07:48.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013043500000000001 2024-09-14T09:07:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160847+0000", 2024-09-14T09:07:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060869299999999999 2024-09-14T09:07:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106738 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 60416~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:07:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.172940658000002, 2024-09-14T09:07:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3139482179999999, 2024-09-14T09:07:48.103 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106738 2024-09-14T09:07:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:07:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:07:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:07:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3999999999999998e-07 2024-09-14T09:07:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847733+0000", 2024-09-14T09:07:48.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8070000000000002e-06 2024-09-14T09:07:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847738+0000", 2024-09-14T09:07:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9740000000000001e-06 2024-09-14T09:07:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847742+0000", 2024-09-14T09:07:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6009999999999999e-06 2024-09-14T09:07:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859612+0000", 2024-09-14T09:07:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011870537 2024-09-14T09:07:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859621+0000", 2024-09-14T09:07:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3019999999999995e-06 2024-09-14T09:07:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160917+0000", 2024-09-14T09:07:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3012961080000001 2024-09-14T09:07:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161054+0000", 2024-09-14T09:07:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000137422 2024-09-14T09:07:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161674+0000", 2024-09-14T09:07:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000620227 2024-09-14T09:07:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106739 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 64512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:07:48.108 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.172920818000001, 2024-09-14T09:07:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.314753777, 2024-09-14T09:07:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106739 2024-09-14T09:07:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:07:48.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:07:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:07:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9700000000000003e-07 2024-09-14T09:07:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847748+0000", 2024-09-14T09:07:48.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2589999999999999e-06 2024-09-14T09:07:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847752+0000", 2024-09-14T09:07:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6099999999999999e-06 2024-09-14T09:07:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847755+0000", 2024-09-14T09:07:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0520000000000002e-06 2024-09-14T09:07:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859653+0000", 2024-09-14T09:07:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011897575000000001 2024-09-14T09:07:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859658+0000", 2024-09-14T09:07:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8810000000000004e-06 2024-09-14T09:07:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161745+0000", 2024-09-14T09:07:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3020870819999999 2024-09-14T09:07:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161877+0000", 2024-09-14T09:07:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000132657 2024-09-14T09:07:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162500+0000", 2024-09-14T09:07:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062236400000000003 2024-09-14T09:07:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106741 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 68608~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.113 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:07:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.172908026000002, 2024-09-14T09:07:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3155568449999999, 2024-09-14T09:07:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106741 2024-09-14T09:07:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:07:48.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:07:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:07:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5800000000000001e-07 2024-09-14T09:07:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847760+0000", 2024-09-14T09:07:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:07:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847764+0000", 2024-09-14T09:07:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3189999999999999e-06 2024-09-14T09:07:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847766+0000", 2024-09-14T09:07:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6429999999999999e-06 2024-09-14T09:07:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859693+0000", 2024-09-14T09:07:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011926724 2024-09-14T09:07:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859703+0000", 2024-09-14T09:07:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6160000000000008e-06 2024-09-14T09:07:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162575+0000", 2024-09-14T09:07:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.302872128 2024-09-14T09:07:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162709+0000", 2024-09-14T09:07:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013446000000000001 2024-09-14T09:07:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163316+0000", 2024-09-14T09:07:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060650700000000005 2024-09-14T09:07:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106744 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 76800~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:48.118 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:07:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 26.172896914999999, 2024-09-14T09:07:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3163901789999999, 2024-09-14T09:07:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106744 2024-09-14T09:07:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:07:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:07:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996947+0000", 2024-09-14T09:07:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.149177154 2024-09-14T09:07:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996964+0000", 2024-09-14T09:07:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.679e-05 2024-09-14T09:07:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996999+0000", 2024-09-14T09:07:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5463000000000002e-05 2024-09-14T09:07:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.997029+0000", 2024-09-14T09:07:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9567e-05 2024-09-14T09:07:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163390+0000", 2024-09-14T09:07:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.16636056799999999 2024-09-14T09:07:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163526+0000", 2024-09-14T09:07:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013655899999999999 2024-09-14T09:07:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.164160+0000", 2024-09-14T09:07:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00063407800000000005 2024-09-14T09:07:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.122 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.123 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.123 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.123 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.123 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:48.124 INFO:teuthology.orchestra.run.smithi176.stdout:5920 trunc from 0x33d55cd to 0xdfa69e6 2024-09-14T09:07:48.125 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:07:48.125 INFO:teuthology.orchestra.run.smithi176.stdout:5921 writesame 0x9d6d795 thru 0x9d7671c data_size 0xc(0x8f88 bytes) 2024-09-14T09:07:48.126 INFO:teuthology.orchestra.run.smithi176.stdout:5924 read 0x14630d2 thru 0x146dd89 (0xacb8 bytes) 2024-09-14T09:07:48.126 INFO:teuthology.orchestra.run.smithi176.stdout:5925 write 0x35c6a92 thru 0x35d1705 (0xac74 bytes) 2024-09-14T09:07:48.126 INFO:teuthology.orchestra.run.smithi176.stdout:5929 writesame 0x6e29da8 thru 0x6e2bfaa data_size 0x1(0x2203 bytes) 2024-09-14T09:07:48.126 INFO:teuthology.orchestra.run.smithi176.stdout:5930 trunc from 0xdfa69e6 to 0x190fbcd 2024-09-14T09:07:48.126 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:48.126 INFO:teuthology.orchestra.run.smithi176.stdout:5932 read 0x11d7b81 thru 0x11dc945 (0x4dc5 bytes) 2024-09-14T09:07:48.126 INFO:teuthology.orchestra.run.smithi176.stdout:5933 read 0xc74100 thru 0xc7ac9f (0x6ba0 bytes) 2024-09-14T09:07:48.126 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:48.123+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:48.133 INFO:teuthology.orchestra.run.smithi176.stdout:5934 write 0x1ccc5ca thru 0x1cd8520 (0xbf57 bytes) 2024-09-14T09:07:48.141 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:48.141 INFO:teuthology.orchestra.run.smithi176.stdout:5937 punch from 0x3da82e to 0x3e944d, (0xec1f bytes) 2024-09-14T09:07:48.141 INFO:teuthology.orchestra.run.smithi176.stdout:5938 punch from 0x2fda69 to 0x2ffb23, (0x20ba bytes) 2024-09-14T09:07:48.141 INFO:teuthology.orchestra.run.smithi176.stdout:5941 writesame 0x979940 thru 0x98023d data_size 0x59(0x68fe bytes) 2024-09-14T09:07:48.144 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:48.144 INFO:teuthology.orchestra.run.smithi176.stdout:5943 punch from 0x87cf4e to 0x889fed, (0xd09f bytes) 2024-09-14T09:07:48.144 INFO:teuthology.orchestra.run.smithi176.stdout:5945 read 0x17ce00b thru 0x17db44d (0xd443 bytes) 2024-09-14T09:07:48.145 INFO:teuthology.orchestra.run.smithi176.stdout:5946 read 0x1ab49b2 thru 0x1ab75cc (0x2c1b bytes) 2024-09-14T09:07:48.145 INFO:teuthology.orchestra.run.smithi176.stdout:5947 read 0x189f01a thru 0x18aaea4 (0xbe8b bytes) 2024-09-14T09:07:48.177 INFO:teuthology.orchestra.run.smithi176.stdout:5948 writesame 0xe606351 thru 0xe613acc data_size 0x6(0xd77c bytes) 2024-09-14T09:07:48.177 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:48.177 INFO:teuthology.orchestra.run.smithi176.stdout:5951 punch from 0xb74c29e to 0xb74f6a0, (0x3402 bytes) 2024-09-14T09:07:48.177 INFO:teuthology.orchestra.run.smithi176.stdout:5952 read 0x8715480 thru 0x8725000 (0xfb81 bytes) 2024-09-14T09:07:48.177 INFO:teuthology.orchestra.run.smithi176.stdout:5953 read 0x92890ad thru 0x928d634 (0x4588 bytes) 2024-09-14T09:07:48.177 INFO:teuthology.orchestra.run.smithi176.stdout:5955 writesame 0x8cae041 thru 0x8cb4fe6 data_size 0x1f(0x6fa6 bytes) 2024-09-14T09:07:48.185 INFO:teuthology.orchestra.run.smithi176.stdout:5958 writesame 0x39bd25c thru 0x39c787e data_size 0x1(0xa623 bytes) 2024-09-14T09:07:48.225 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:48.226+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:48.228 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --force --op remove --pgid 3.3 2024-09-14T09:07:48.262 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:48.262 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:48.262 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:48.262 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:48.268 INFO:teuthology.orchestra.run.smithi176.stdout:5959 trunc from 0xe613acd to 0x43e1c44 2024-09-14T09:07:48.283 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:07:48.306 INFO:teuthology.orchestra.run.smithi176.stdout:5960 write 0xc42ebd9 thru 0xc432ceb (0x4113 bytes) 2024-09-14T09:07:48.324 INFO:teuthology.orchestra.run.smithi176.stdout:5961 trunc from 0xc432cec to 0x4b4f0f5 2024-09-14T09:07:48.326 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:48.327+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:48.338 INFO:teuthology.orchestra.run.smithi176.stdout:5962 read 0x5b8ec7 thru 0x5c33ef (0xa529 bytes) 2024-09-14T09:07:48.365 INFO:teuthology.orchestra.run.smithi176.stdout:5963 write 0xe067d34 thru 0xe06ac2a (0x2ef7 bytes) 2024-09-14T09:07:48.370 INFO:teuthology.orchestra.run.smithi176.stdout:5964 punch from 0x4326abd to 0x432872c, (0x1c6f bytes) 2024-09-14T09:07:48.387 INFO:teuthology.orchestra.run.smithi176.stdout:5965 trunc from 0xe06ac2b to 0x36e59a3 2024-09-14T09:07:48.405 INFO:teuthology.orchestra.run.smithi176.stdout:5966 punch from 0x1058e9c to 0x1061bf3, (0x8d57 bytes) 2024-09-14T09:07:48.418 INFO:teuthology.orchestra.run.smithi176.stdout:5967 trunc from 0x36e59a3 to 0xc47caae 2024-09-14T09:07:48.427 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:48.430+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:48.428 INFO:teuthology.orchestra.run.smithi176.stdout:5968 read 0x5b07fc thru 0x5b9125 (0x892a bytes) 2024-09-14T09:07:48.436 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:48.437 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:48.437 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:48.437 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:48.437 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:48.442 INFO:teuthology.orchestra.run.smithi176.stdout:5969 write 0xdee1ec8 thru 0xdee9e70 (0x7fa9 bytes) 2024-09-14T09:07:48.447 INFO:teuthology.orchestra.run.smithi176.stdout:5971 write 0x91f4a2c thru 0x91f60db (0x16b0 bytes) 2024-09-14T09:07:48.452 INFO:teuthology.orchestra.run.smithi176.stdout:5972 write 0x3d5af5f thru 0x3d62682 (0x7724 bytes) 2024-09-14T09:07:48.453 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:07:48.460 INFO:teuthology.orchestra.run.smithi176.stdout:5974 read 0xc890f0d thru 0xc89cdfc (0xbef0 bytes) 2024-09-14T09:07:48.461 INFO:teuthology.orchestra.run.smithi176.stdout:5976 write 0x4357538 thru 0x43613cb (0x9e94 bytes) 2024-09-14T09:07:48.469 INFO:teuthology.orchestra.run.smithi176.stdout:5977 writesame 0xc275b9 thru 0xc2e55f data_size 0x1(0x6fa7 bytes) 2024-09-14T09:07:48.528 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:48.527+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:48.580 INFO:teuthology.orchestra.run.smithi176.stdout:5978 read 0x7719d13 thru 0x7729339 (0xf627 bytes) 2024-09-14T09:07:48.583 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:48.583 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:48.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:48.583 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:48.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.584 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:48.584 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:48.584 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 135.49838704699999, 2024-09-14T09:07:48.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:07:48.584 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.584 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.584 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.584 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.584 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.584 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.584 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:07:48.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.585 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:48.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:48.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:07:48.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:07:48.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:07:48.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:07:48.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:07:48.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:07:48.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:07:48.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:07:48.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:07:48.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:07:48.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:07:48.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:07:48.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:07:48.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:07:48.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:07:48.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:07:48.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:07:48.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:07:48.590 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.590 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.590 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.591 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:48.591 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:48.591 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 135.49833238900001, 2024-09-14T09:07:48.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:07:48.591 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.591 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.591 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.591 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.591 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.591 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.591 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:07:48.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.591 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:48.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:48.592 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.592 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.592 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.592 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.592 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:48.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:07:48.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:07:48.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:07:48.593 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.593 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.593 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.593 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:07:48.593 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:07:48.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:07:48.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:07:48.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.594 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.594 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.594 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:07:48.594 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:07:48.594 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:07:48.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:07:48.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.595 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.595 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:07:48.595 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:07:48.595 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.595 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:07:48.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:07:48.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.596 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.596 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.596 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:07:48.596 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:07:48.596 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.596 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.596 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.596 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.597 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.597 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:48.597 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:48.597 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 135.498227015, 2024-09-14T09:07:48.597 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:07:48.597 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.597 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.597 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.597 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.597 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.597 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.598 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:07:48.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.598 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:48.598 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.598 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.598 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.598 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.598 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:48.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.599 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.599 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:07:48.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:07:48.599 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:07:48.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:07:48.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:07:48.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:07:48.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:07:48.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:07:48.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:07:48.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:07:48.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:07:48.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:07:48.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:07:48.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:07:48.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:07:48.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:07:48.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:07:48.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:07:48.603 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.603 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.603 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.604 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:48.604 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:48.604 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 135.49787025099999, 2024-09-14T09:07:48.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:07:48.604 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.604 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.604 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.605 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:07:48.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.605 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:48.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:48.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:07:48.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:07:48.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:07:48.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:07:48.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:07:48.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:07:48.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:07:48.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:07:48.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:07:48.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:07:48.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:07:48.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:07:48.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:07:48.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:07:48.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.609 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:07:48.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:07:48.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:07:48.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:07:48.610 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.610 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.610 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.610 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:48.610 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:48.610 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 135.497785513, 2024-09-14T09:07:48.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:07:48.610 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.610 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.611 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.611 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.611 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.611 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.611 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:07:48.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.611 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:48.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:48.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:07:48.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:07:48.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:07:48.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:07:48.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:07:48.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:07:48.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:07:48.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:07:48.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:07:48.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:07:48.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:07:48.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:07:48.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:07:48.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:07:48.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:07:48.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:07:48.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:07:48.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:07:48.615 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.615 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.615 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.616 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:48.616 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:48.616 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 135.497697827, 2024-09-14T09:07:48.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:07:48.616 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.616 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.616 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.616 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.616 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.616 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.616 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:07:48.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.617 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:48.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:48.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:07:48.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:07:48.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:07:48.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:07:48.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:07:48.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:07:48.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:07:48.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:07:48.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:07:48.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:07:48.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:48.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:07:48.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:07:48.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:07:48.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:07:48.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:07:48.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:07:48.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:07:48.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:07:48.621 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.621 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.621 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.621 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:07:48.621 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:48.621 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 102.42140584000001, 2024-09-14T09:07:48.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:07:48.621 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.621 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.621 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:48.621 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.621 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.622 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.622 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:07:48.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.622 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:48.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:48.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:07:48.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:07:48.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:07:48.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:07:48.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:07:48.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:07:48.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:07:48.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:07:48.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:07:48.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:07:48.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:07:48.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:07:48.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:48.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:07:48.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:07:48.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:48.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:07:48.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:07:48.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:48.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:07:48.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:07:48.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:48.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:07:48.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:07:48.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:07:48.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:07:48.626 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.626 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.627 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.627 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:97796 3.15 3:aaaa5f7f:::rbd_data.115e113dc8ca.0000000000000029:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:07:48.627 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:48.627 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.487536970000001, 2024-09-14T09:07:48.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14537731300000001, 2024-09-14T09:07:48.627 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.627 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.627 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:48.627 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.627 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.627 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.627 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97796 2024-09-14T09:07:48.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.628 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:48.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:48.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094482+0000", 2024-09-14T09:07:48.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6400000000000001e-07 2024-09-14T09:07:48.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094485+0000", 2024-09-14T09:07:48.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.693e-06 2024-09-14T09:07:48.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094494+0000", 2024-09-14T09:07:48.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9590000000000001e-06 2024-09-14T09:07:48.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094502+0000", 2024-09-14T09:07:48.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3180000000000001e-06 2024-09-14T09:07:48.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094567+0000", 2024-09-14T09:07:48.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5526999999999998e-05 2024-09-14T09:07:48.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.630 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094726+0000", 2024-09-14T09:07:48.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015842499999999999 2024-09-14T09:07:48.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:07:48.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.101940+0000", 2024-09-14T09:07:48.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0072146559999999998 2024-09-14T09:07:48.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:48.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239665+0000", 2024-09-14T09:07:48.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.137724501 2024-09-14T09:07:48.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:48.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239726+0000", 2024-09-14T09:07:48.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1550999999999994e-05 2024-09-14T09:07:48.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:48.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239800+0000", 2024-09-14T09:07:48.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3318000000000005e-05 2024-09-14T09:07:48.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239858+0000", 2024-09-14T09:07:48.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8600999999999997e-05 2024-09-14T09:07:48.632 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.632 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.633 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.633 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:97796 3.15 e106/103 ondisk, result = 0)", 2024-09-14T09:07:48.633 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:48.633 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 36.465191101000002, 2024-09-14T09:07:48.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123043465, 2024-09-14T09:07:48.633 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.633 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.633 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.633 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:48.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:48.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116828+0000", 2024-09-14T09:07:48.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8100000000000002e-07 2024-09-14T09:07:48.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116831+0000", 2024-09-14T09:07:48.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3009999999999998e-06 2024-09-14T09:07:48.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116841+0000", 2024-09-14T09:07:48.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7829999999999993e-06 2024-09-14T09:07:48.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116851+0000", 2024-09-14T09:07:48.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0135999999999999e-05 2024-09-14T09:07:48.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239707+0000", 2024-09-14T09:07:48.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12285581499999999 2024-09-14T09:07:48.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239718+0000", 2024-09-14T09:07:48.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1804000000000001e-05 2024-09-14T09:07:48.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239870+0000", 2024-09-14T09:07:48.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015194500000000001 2024-09-14T09:07:48.636 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.636 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.637 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.637 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117315 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 309248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:48.637 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:07:48.637 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.858555364000001, 2024-09-14T09:07:48.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11247383800000001, 2024-09-14T09:07:48.637 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.637 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.637 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.637 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.637 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.637 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.637 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117315 2024-09-14T09:07:48.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.638 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:07:48.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:07:48.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723899+0000", 2024-09-14T09:07:48.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043641699999999998 2024-09-14T09:07:48.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723903+0000", 2024-09-14T09:07:48.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5389999999999999e-06 2024-09-14T09:07:48.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723910+0000", 2024-09-14T09:07:48.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4229999999999999e-06 2024-09-14T09:07:48.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723917+0000", 2024-09-14T09:07:48.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4560000000000003e-06 2024-09-14T09:07:48.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835514+0000", 2024-09-14T09:07:48.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111596258 2024-09-14T09:07:48.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835571+0000", 2024-09-14T09:07:48.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7317e-05 2024-09-14T09:07:48.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835936+0000", 2024-09-14T09:07:48.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000365428 2024-09-14T09:07:48.641 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.641 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.641 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.641 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117317 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 313344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:48.641 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:07:48.641 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.858088347999999, 2024-09-14T09:07:48.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112450201, 2024-09-14T09:07:48.645 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.645 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.645 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.645 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.645 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.646 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.646 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117317 2024-09-14T09:07:48.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.646 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.646 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:07:48.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.646 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:07:48.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724396+0000", 2024-09-14T09:07:48.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000466028 2024-09-14T09:07:48.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724401+0000", 2024-09-14T09:07:48.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2299999999999999e-06 2024-09-14T09:07:48.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724412+0000", 2024-09-14T09:07:48.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0762e-05 2024-09-14T09:07:48.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724419+0000", 2024-09-14T09:07:48.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1010000000000003e-06 2024-09-14T09:07:48.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835981+0000", 2024-09-14T09:07:48.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111562174 2024-09-14T09:07:48.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836070+0000", 2024-09-14T09:07:48.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8839999999999996e-05 2024-09-14T09:07:48.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836380+0000", 2024-09-14T09:07:48.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031006600000000001 2024-09-14T09:07:48.649 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.649 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.649 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.649 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117319 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 317440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:48.649 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:07:48.650 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.857583712, 2024-09-14T09:07:48.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112467763, 2024-09-14T09:07:48.650 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.650 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.650 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.650 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.650 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.650 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.650 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117319 2024-09-14T09:07:48.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.650 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:07:48.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:07:48.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724846+0000", 2024-09-14T09:07:48.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00041155400000000001 2024-09-14T09:07:48.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724854+0000", 2024-09-14T09:07:48.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5469999999999997e-06 2024-09-14T09:07:48.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724862+0000", 2024-09-14T09:07:48.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1289999999999996e-06 2024-09-14T09:07:48.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724867+0000", 2024-09-14T09:07:48.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8720000000000001e-06 2024-09-14T09:07:48.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836486+0000", 2024-09-14T09:07:48.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111618357 2024-09-14T09:07:48.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836554+0000", 2024-09-14T09:07:48.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8378000000000002e-05 2024-09-14T09:07:48.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836902+0000", 2024-09-14T09:07:48.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034792599999999998 2024-09-14T09:07:48.654 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.654 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.654 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.654 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117321 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 321536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:48.654 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:07:48.654 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.857139388, 2024-09-14T09:07:48.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11249685600000001, 2024-09-14T09:07:48.654 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.654 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.654 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.654 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.655 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.655 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.655 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117321 2024-09-14T09:07:48.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.655 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:07:48.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:07:48.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725241+0000", 2024-09-14T09:07:48.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036211099999999998 2024-09-14T09:07:48.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725245+0000", 2024-09-14T09:07:48.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9179999999999996e-06 2024-09-14T09:07:48.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725254+0000", 2024-09-14T09:07:48.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0699999999999996e-06 2024-09-14T09:07:48.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725260+0000", 2024-09-14T09:07:48.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1099999999999999e-06 2024-09-14T09:07:48.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836935+0000", 2024-09-14T09:07:48.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111674997 2024-09-14T09:07:48.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837021+0000", 2024-09-14T09:07:48.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5931000000000001e-05 2024-09-14T09:07:48.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837375+0000", 2024-09-14T09:07:48.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035471899999999998 2024-09-14T09:07:48.659 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.659 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.659 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.659 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117323 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:48.659 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:07:48.659 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.856745162999999, 2024-09-14T09:07:48.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11250631, 2024-09-14T09:07:48.659 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.659 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.659 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.659 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.659 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117323 2024-09-14T09:07:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:07:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:07:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725941+0000", 2024-09-14T09:07:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00066866500000000004 2024-09-14T09:07:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725946+0000", 2024-09-14T09:07:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9060000000000001e-06 2024-09-14T09:07:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725959+0000", 2024-09-14T09:07:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2622e-05 2024-09-14T09:07:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725968+0000", 2024-09-14T09:07:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5660000000000007e-06 2024-09-14T09:07:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837408+0000", 2024-09-14T09:07:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111440626 2024-09-14T09:07:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837495+0000", 2024-09-14T09:07:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6654000000000001e-05 2024-09-14T09:07:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837779+0000", 2024-09-14T09:07:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000284271 2024-09-14T09:07:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117327 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 333824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:48.663 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:07:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.856008971000001, 2024-09-14T09:07:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112409908, 2024-09-14T09:07:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117327 2024-09-14T09:07:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:07:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:07:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726303+0000", 2024-09-14T09:07:48.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029388199999999998 2024-09-14T09:07:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726307+0000", 2024-09-14T09:07:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4320000000000001e-06 2024-09-14T09:07:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726317+0000", 2024-09-14T09:07:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4609999999999997e-06 2024-09-14T09:07:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726324+0000", 2024-09-14T09:07:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0480000000000001e-06 2024-09-14T09:07:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838051+0000", 2024-09-14T09:07:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111727534 2024-09-14T09:07:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838143+0000", 2024-09-14T09:07:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1359000000000004e-05 2024-09-14T09:07:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838419+0000", 2024-09-14T09:07:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027619199999999997 2024-09-14T09:07:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117329 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 337920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:07:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.855682212000001, 2024-09-14T09:07:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112411122, 2024-09-14T09:07:48.668 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117329 2024-09-14T09:07:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:07:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:07:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726742+0000", 2024-09-14T09:07:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040626099999999999 2024-09-14T09:07:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726748+0000", 2024-09-14T09:07:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6860000000000003e-06 2024-09-14T09:07:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726761+0000", 2024-09-14T09:07:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3599e-05 2024-09-14T09:07:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726770+0000", 2024-09-14T09:07:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5990000000000002e-06 2024-09-14T09:07:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838443+0000", 2024-09-14T09:07:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111672664 2024-09-14T09:07:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838487+0000", 2024-09-14T09:07:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4249000000000003e-05 2024-09-14T09:07:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838747+0000", 2024-09-14T09:07:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026006400000000001 2024-09-14T09:07:48.672 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117331 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 342016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:07:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.855214617000001, 2024-09-14T09:07:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112291139, 2024-09-14T09:07:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.673 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117331 2024-09-14T09:07:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:07:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:07:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727158+0000", 2024-09-14T09:07:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035434000000000003 2024-09-14T09:07:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727162+0000", 2024-09-14T09:07:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5449999999999997e-06 2024-09-14T09:07:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727174+0000", 2024-09-14T09:07:48.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2278e-05 2024-09-14T09:07:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727183+0000", 2024-09-14T09:07:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3820000000000004e-06 2024-09-14T09:07:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838788+0000", 2024-09-14T09:07:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111604748 2024-09-14T09:07:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838847+0000", 2024-09-14T09:07:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8983000000000001e-05 2024-09-14T09:07:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839094+0000", 2024-09-14T09:07:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024786300000000002 2024-09-14T09:07:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117333 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 346112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:07:48.677 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.854820805999999, 2024-09-14T09:07:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112319711, 2024-09-14T09:07:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117333 2024-09-14T09:07:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:07:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:07:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727785+0000", 2024-09-14T09:07:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000587512 2024-09-14T09:07:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727789+0000", 2024-09-14T09:07:48.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.352e-06 2024-09-14T09:07:48.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727800+0000", 2024-09-14T09:07:48.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1369000000000001e-05 2024-09-14T09:07:48.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727807+0000", 2024-09-14T09:07:48.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1300000000000003e-06 2024-09-14T09:07:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839137+0000", 2024-09-14T09:07:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111329898 2024-09-14T09:07:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839226+0000", 2024-09-14T09:07:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8337000000000005e-05 2024-09-14T09:07:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839517+0000", 2024-09-14T09:07:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029111300000000002 2024-09-14T09:07:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117363 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 407552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:07:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.848724706999999, 2024-09-14T09:07:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112191306, 2024-09-14T09:07:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.682 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117363 2024-09-14T09:07:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:07:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:07:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733666+0000", 2024-09-14T09:07:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00037284500000000001 2024-09-14T09:07:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733671+0000", 2024-09-14T09:07:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9269999999999996e-06 2024-09-14T09:07:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733681+0000", 2024-09-14T09:07:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5599999999999999e-06 2024-09-14T09:07:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733689+0000", 2024-09-14T09:07:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2649999999999996e-06 2024-09-14T09:07:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845028+0000", 2024-09-14T09:07:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111338933 2024-09-14T09:07:48.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845094+0000", 2024-09-14T09:07:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6749999999999996e-05 2024-09-14T09:07:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845485+0000", 2024-09-14T09:07:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00039002600000000002 2024-09-14T09:07:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.686 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117365 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 411648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:07:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 16.848314942000002, 2024-09-14T09:07:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112271685, 2024-09-14T09:07:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117365 2024-09-14T09:07:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:07:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:07:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734147+0000", 2024-09-14T09:07:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00044374399999999999 2024-09-14T09:07:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734152+0000", 2024-09-14T09:07:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9139999999999999e-06 2024-09-14T09:07:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734164+0000", 2024-09-14T09:07:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2299e-05 2024-09-14T09:07:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734172+0000", 2024-09-14T09:07:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0390000000000005e-06 2024-09-14T09:07:48.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845528+0000", 2024-09-14T09:07:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11135639899999999 2024-09-14T09:07:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845602+0000", 2024-09-14T09:07:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3957999999999999e-05 2024-09-14T09:07:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845975+0000", 2024-09-14T09:07:48.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00037233200000000001 2024-09-14T09:07:48.691 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.691 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.691 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.691 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:48.691 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:48.691 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:48.692 INFO:teuthology.orchestra.run.smithi176.stdout:5979 trunc from 0xdee9e71 to 0x4bdf5e9 2024-09-14T09:07:48.693 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:07:48.693 INFO:teuthology.orchestra.run.smithi176.stdout:5980 trunc from 0x4bdf5e9 to 0x8df4141 2024-09-14T09:07:48.693 INFO:teuthology.orchestra.run.smithi176.stdout:5981 trunc from 0x8df4141 to 0x6430c25 2024-09-14T09:07:48.694 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:48.691+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:48.703 INFO:teuthology.orchestra.run.smithi176.stdout:5983 write 0xe63e62d thru 0xe64dc82 (0xf656 bytes) 2024-09-14T09:07:48.716 INFO:teuthology.orchestra.run.smithi176.stdout:5984 write 0x4531086 thru 0x453f80e (0xe789 bytes) 2024-09-14T09:07:48.729 INFO:teuthology.orchestra.run.smithi176.stdout:5985 read 0x4f551a7 thru 0x4f5b38d (0x61e7 bytes) 2024-09-14T09:07:48.729 INFO:teuthology.orchestra.run.smithi176.stdout:5986 read 0x7ac597 thru 0x7b2b1a (0x6584 bytes) 2024-09-14T09:07:48.730 INFO:teuthology.orchestra.run.smithi176.stdout:5987 punch from 0x4f7bfab to 0x4f7fc22, (0x3c77 bytes) 2024-09-14T09:07:48.731 INFO:teuthology.orchestra.run.smithi176.stdout:5988 read 0x3c189d6 thru 0x3c1a11f (0x174a bytes) 2024-09-14T09:07:48.731 INFO:teuthology.orchestra.run.smithi176.stdout:5989 write 0xa7b6fa6 thru 0xa7c2827 (0xb882 bytes) 2024-09-14T09:07:48.743 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:48.765 INFO:teuthology.orchestra.run.smithi176.stdout:5991 trunc from 0xe64dc83 to 0x5e83c3d 2024-09-14T09:07:48.786 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:48.792 INFO:teuthology.orchestra.run.smithi176.stdout:5995 trunc from 0x5e83c3d to 0x370484a 2024-09-14T09:07:48.793 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:48.791+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:48.816 INFO:teuthology.orchestra.run.smithi176.stdout:5996 read 0x2d32474 thru 0x2d39ca7 (0x7834 bytes) 2024-09-14T09:07:48.825 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:48.825 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:48.832 INFO:teuthology.orchestra.run.smithi176.stdout:5997 trunc from 0x370484a to 0x9a0f460 2024-09-14T09:07:48.841 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:07:48.857 INFO:teuthology.orchestra.run.smithi176.stdout:5998 trunc from 0x9a0f460 to 0xe827cd0 2024-09-14T09:07:48.867 INFO:teuthology.orchestra.run.smithi176.stdout:compare and write disabled 2024-09-14T09:07:48.867 INFO:teuthology.orchestra.run.smithi176.stdout:6000 punch from 0x84e361b to 0x84e5d4b, (0x2730 bytes) 2024-09-14T09:07:48.894 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:48.898+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:48.928 INFO:teuthology.orchestra.run.smithi176.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-14T09:07:48.958 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:48.958 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:48.972 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:07:48.994 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is set 2024-09-14T09:07:48.995 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:48.995+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:49.096 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:49.095+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:49.151 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:49.151 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:49.168 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:07:49.218 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:49.222+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:49.290 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:49.291 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:49.298 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:49.299+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:49.304 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:07:49.399 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:49.399+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:49.475 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:49.475 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:49.489 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:07:49.534 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:49.538+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:49.601 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:49.599+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:49.617 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:49.617 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:49.633 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:07:49.702 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:49.703+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:49.803 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:49.803+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:49.810 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:49.811 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:49.826 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:07:49.904 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:49.906+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:49.951 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:49.951 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:49.966 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:07:50.010 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:50.014+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:50.107 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:50.107+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:50.147 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:50.147 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:50.165 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:07:50.210 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:50.211+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:50.244 INFO:teuthology.orchestra.run.smithi183.stdout: marking collection for removal 2024-09-14T09:07:50.249 INFO:teuthology.orchestra.run.smithi183.stdout:setting '_remove' omap key 2024-09-14T09:07:50.249 INFO:teuthology.orchestra.run.smithi183.stdout:finish_remove_pgs 3.3_head removing 3.3 2024-09-14T09:07:50.259 INFO:teuthology.orchestra.run.smithi183.stdout:Remove successful 2024-09-14T09:07:50.303 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:50.303 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:50.303 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:50.303 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:50.308 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:50.307+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:50.317 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:07:50.409 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:50.414+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:50.481 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:50.481 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:50.481 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:50.481 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:50.481 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:50.494 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:07:50.510 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:50.514+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:50.611 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:50.614+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:50.618 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:50.618 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:50.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:50.618 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:50.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.618 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:71052 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:07:50.619 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:07:50.619 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.879652218999993, 2024-09-14T09:07:50.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14854635599999999, 2024-09-14T09:07:50.619 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.619 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.619 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:50.619 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:50.619 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:50.619 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:50.619 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 71052 2024-09-14T09:07:50.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.619 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:07:50.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:07:50.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.620 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.620 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737570+0000", 2024-09-14T09:07:50.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4770000000000002e-06 2024-09-14T09:07:50.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737584+0000", 2024-09-14T09:07:50.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3457e-05 2024-09-14T09:07:50.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737608+0000", 2024-09-14T09:07:50.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3893999999999999e-05 2024-09-14T09:07:50.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737620+0000", 2024-09-14T09:07:50.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2157000000000001e-05 2024-09-14T09:07:50.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737738+0000", 2024-09-14T09:07:50.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118409 2024-09-14T09:07:50.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737892+0000", 2024-09-14T09:07:50.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154222 2024-09-14T09:07:50.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:07:50.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.749622+0000", 2024-09-14T09:07:50.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011729537 2024-09-14T09:07:50.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:50.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885934+0000", 2024-09-14T09:07:50.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13631222200000001 2024-09-14T09:07:50.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:50.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885996+0000", 2024-09-14T09:07:50.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1555000000000003e-05 2024-09-14T09:07:50.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:50.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886060+0000", 2024-09-14T09:07:50.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3980999999999996e-05 2024-09-14T09:07:50.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886114+0000", 2024-09-14T09:07:50.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4444999999999998e-05 2024-09-14T09:07:50.624 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.624 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.625 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.625 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:71052 3.1 e71/34 ondisk, result = 0)", 2024-09-14T09:07:50.625 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:07:50.625 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 80.836748841000002, 2024-09-14T09:07:50.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10565334799999999, 2024-09-14T09:07:50.625 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.625 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.625 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:50.625 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.625 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.625 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:07:50.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:07:50.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780473+0000", 2024-09-14T09:07:50.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8980000000000001e-06 2024-09-14T09:07:50.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780482+0000", 2024-09-14T09:07:50.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9080000000000001e-06 2024-09-14T09:07:50.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780511+0000", 2024-09-14T09:07:50.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9490999999999999e-05 2024-09-14T09:07:50.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780527+0000", 2024-09-14T09:07:50.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.508e-05 2024-09-14T09:07:50.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885978+0000", 2024-09-14T09:07:50.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.105451328 2024-09-14T09:07:50.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885989+0000", 2024-09-14T09:07:50.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0875e-05 2024-09-14T09:07:50.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886125+0000", 2024-09-14T09:07:50.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013576800000000001 2024-09-14T09:07:50.629 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.629 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.629 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.629 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:98007 3.12 3:4faf1f2a:::rbd_data.115e113dc8ca.000000000000004f:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:07:50.629 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:07:50.630 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 37.759776617, 2024-09-14T09:07:50.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.098233707000000003, 2024-09-14T09:07:50.630 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.630 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.630 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:50.630 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:50.630 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:50.630 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:50.630 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 98007 2024-09-14T09:07:50.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.630 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.630 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:07:50.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:07:50.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857446+0000", 2024-09-14T09:07:50.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7599999999999998e-06 2024-09-14T09:07:50.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857459+0000", 2024-09-14T09:07:50.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3123e-05 2024-09-14T09:07:50.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857491+0000", 2024-09-14T09:07:50.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1603000000000002e-05 2024-09-14T09:07:50.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857506+0000", 2024-09-14T09:07:50.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.537e-05 2024-09-14T09:07:50.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857694+0000", 2024-09-14T09:07:50.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018766599999999999 2024-09-14T09:07:50.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857955+0000", 2024-09-14T09:07:50.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000260979 2024-09-14T09:07:50.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:07:50.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.863125+0000", 2024-09-14T09:07:50.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.005170582 2024-09-14T09:07:50.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:50.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955268+0000", 2024-09-14T09:07:50.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092142659000000002 2024-09-14T09:07:50.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:50.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955385+0000", 2024-09-14T09:07:50.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116723 2024-09-14T09:07:50.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.635 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:50.635 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955553+0000", 2024-09-14T09:07:50.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016809299999999999 2024-09-14T09:07:50.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955677+0000", 2024-09-14T09:07:50.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012414899999999999 2024-09-14T09:07:50.636 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.636 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.636 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.636 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118153 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 445440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:50.636 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:07:50.637 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.779219242, 2024-09-14T09:07:50.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095358268999999996, 2024-09-14T09:07:50.637 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.637 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.637 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:50.637 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:50.637 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:50.637 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:50.637 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118153 2024-09-14T09:07:50.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.637 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:07:50.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:07:50.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:07:50.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0999999999999999e-07 2024-09-14T09:07:50.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838003+0000", 2024-09-14T09:07:50.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2000000000000001e-06 2024-09-14T09:07:50.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838009+0000", 2024-09-14T09:07:50.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.105e-06 2024-09-14T09:07:50.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838015+0000", 2024-09-14T09:07:50.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9030000000000001e-06 2024-09-14T09:07:50.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933049+0000", 2024-09-14T09:07:50.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095033513999999999 2024-09-14T09:07:50.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933113+0000", 2024-09-14T09:07:50.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3616999999999998e-05 2024-09-14T09:07:50.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.641 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.641 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933359+0000", 2024-09-14T09:07:50.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024652000000000002 2024-09-14T09:07:50.641 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.641 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.641 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.641 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118155 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 449536~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:50.642 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:07:50.642 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.779196717000001, 2024-09-14T09:07:50.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095668000000000003, 2024-09-14T09:07:50.642 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.642 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.642 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:50.642 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:50.642 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:50.642 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:50.642 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118155 2024-09-14T09:07:50.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.642 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:07:50.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:07:50.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838024+0000", 2024-09-14T09:07:50.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0400000000000002e-07 2024-09-14T09:07:50.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838026+0000", 2024-09-14T09:07:50.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.508e-06 2024-09-14T09:07:50.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838033+0000", 2024-09-14T09:07:50.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.392e-06 2024-09-14T09:07:50.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838038+0000", 2024-09-14T09:07:50.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1050000000000003e-06 2024-09-14T09:07:50.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933400+0000", 2024-09-14T09:07:50.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095362279999999994 2024-09-14T09:07:50.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933451+0000", 2024-09-14T09:07:50.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0927000000000002e-05 2024-09-14T09:07:50.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.646 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933691+0000", 2024-09-14T09:07:50.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024038400000000001 2024-09-14T09:07:50.646 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.646 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.646 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.646 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118158 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 453632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:50.646 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:07:50.646 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.779175694999999, 2024-09-14T09:07:50.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096024231000000002, 2024-09-14T09:07:50.646 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.647 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.647 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:50.647 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:50.647 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:50.647 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:50.647 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118158 2024-09-14T09:07:50.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.647 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:07:50.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:07:50.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838045+0000", 2024-09-14T09:07:50.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2199999999999999e-07 2024-09-14T09:07:50.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838047+0000", 2024-09-14T09:07:50.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1059999999999998e-06 2024-09-14T09:07:50.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838054+0000", 2024-09-14T09:07:50.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4089999999999997e-06 2024-09-14T09:07:50.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838059+0000", 2024-09-14T09:07:50.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8590000000000004e-06 2024-09-14T09:07:50.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933729+0000", 2024-09-14T09:07:50.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095670274999999999 2024-09-14T09:07:50.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933794+0000", 2024-09-14T09:07:50.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4846999999999998e-05 2024-09-14T09:07:50.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934069+0000", 2024-09-14T09:07:50.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027431299999999999 2024-09-14T09:07:50.650 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.650 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.651 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.651 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118160 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 457728~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:50.651 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:07:50.651 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.779153574999999, 2024-09-14T09:07:50.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096394812999999996, 2024-09-14T09:07:50.651 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.651 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.651 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:50.651 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:50.651 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:50.651 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:50.651 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118160 2024-09-14T09:07:50.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.652 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:07:50.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:07:50.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838067+0000", 2024-09-14T09:07:50.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9399999999999995e-07 2024-09-14T09:07:50.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838069+0000", 2024-09-14T09:07:50.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1579999999999999e-06 2024-09-14T09:07:50.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838077+0000", 2024-09-14T09:07:50.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6939999999999995e-06 2024-09-14T09:07:50.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838082+0000", 2024-09-14T09:07:50.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2290000000000002e-06 2024-09-14T09:07:50.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934106+0000", 2024-09-14T09:07:50.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096023874999999995 2024-09-14T09:07:50.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934158+0000", 2024-09-14T09:07:50.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2052000000000002e-05 2024-09-14T09:07:50.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934461+0000", 2024-09-14T09:07:50.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030331100000000001 2024-09-14T09:07:50.655 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.655 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.655 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.655 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118162 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 461824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:50.655 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:07:50.655 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.779130994999999, 2024-09-14T09:07:50.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096615294000000004, 2024-09-14T09:07:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118162 2024-09-14T09:07:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:07:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:07:50.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838090+0000", 2024-09-14T09:07:50.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9699999999999996e-07 2024-09-14T09:07:50.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838092+0000", 2024-09-14T09:07:50.657 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1119999999999999e-06 2024-09-14T09:07:50.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838098+0000", 2024-09-14T09:07:50.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4359999999999999e-06 2024-09-14T09:07:50.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838103+0000", 2024-09-14T09:07:50.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1200000000000001e-06 2024-09-14T09:07:50.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934504+0000", 2024-09-14T09:07:50.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096400885000000006 2024-09-14T09:07:50.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.659 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934596+0000", 2024-09-14T09:07:50.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1775999999999997e-05 2024-09-14T09:07:50.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.659 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.659 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.659 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934704+0000", 2024-09-14T09:07:50.659 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010846800000000001 2024-09-14T09:07:50.659 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.659 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.659 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.659 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.660 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118164 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 465920~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:50.660 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:07:50.660 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.779109734999999, 2024-09-14T09:07:50.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096951372999999993, 2024-09-14T09:07:50.660 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.660 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.660 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:50.660 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:50.660 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:50.660 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:50.660 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118164 2024-09-14T09:07:50.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.661 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:07:50.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:07:50.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838111+0000", 2024-09-14T09:07:50.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0100000000000005e-07 2024-09-14T09:07:50.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838113+0000", 2024-09-14T09:07:50.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0339999999999999e-06 2024-09-14T09:07:50.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838119+0000", 2024-09-14T09:07:50.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.409e-06 2024-09-14T09:07:50.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838124+0000", 2024-09-14T09:07:50.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9030000000000003e-06 2024-09-14T09:07:50.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934739+0000", 2024-09-14T09:07:50.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096615124999999996 2024-09-14T09:07:50.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934789+0000", 2024-09-14T09:07:50.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9554e-05 2024-09-14T09:07:50.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.663 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935062+0000", 2024-09-14T09:07:50.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000272847 2024-09-14T09:07:50.664 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.664 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.664 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.664 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118166 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 470016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:50.664 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:07:50.664 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.779088981000001, 2024-09-14T09:07:50.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097306410999999995, 2024-09-14T09:07:50.664 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.664 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.664 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:50.665 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:50.665 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:50.665 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:50.665 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118166 2024-09-14T09:07:50.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.665 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:07:50.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:07:50.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:07:50.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.03e-07 2024-09-14T09:07:50.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838133+0000", 2024-09-14T09:07:50.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9479999999999998e-06 2024-09-14T09:07:50.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838141+0000", 2024-09-14T09:07:50.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5739999999999998e-06 2024-09-14T09:07:50.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838146+0000", 2024-09-14T09:07:50.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3750000000000002e-06 2024-09-14T09:07:50.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935095+0000", 2024-09-14T09:07:50.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096948435999999999 2024-09-14T09:07:50.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935152+0000", 2024-09-14T09:07:50.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.694e-05 2024-09-14T09:07:50.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935437+0000", 2024-09-14T09:07:50.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028573500000000001 2024-09-14T09:07:50.668 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.668 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.668 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.668 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118168 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 474112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:50.669 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:07:50.669 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.779066759999999, 2024-09-14T09:07:50.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097658024999999996, 2024-09-14T09:07:50.669 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.669 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.669 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:50.669 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:50.669 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:50.669 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:50.669 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118168 2024-09-14T09:07:50.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.669 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.669 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:07:50.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:07:50.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838154+0000", 2024-09-14T09:07:50.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4200000000000001e-07 2024-09-14T09:07:50.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838160+0000", 2024-09-14T09:07:50.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8510000000000004e-06 2024-09-14T09:07:50.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838168+0000", 2024-09-14T09:07:50.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9899999999999997e-06 2024-09-14T09:07:50.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838172+0000", 2024-09-14T09:07:50.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.887e-06 2024-09-14T09:07:50.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935473+0000", 2024-09-14T09:07:50.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097300132999999997 2024-09-14T09:07:50.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935523+0000", 2024-09-14T09:07:50.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0862000000000002e-05 2024-09-14T09:07:50.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935811+0000", 2024-09-14T09:07:50.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028786000000000003 2024-09-14T09:07:50.672 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.672 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.673 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.673 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118170 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 478208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:50.673 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:07:50.673 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.779039949000001, 2024-09-14T09:07:50.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.098077529999999996, 2024-09-14T09:07:50.673 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.673 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.673 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:50.673 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:50.673 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:50.673 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:50.673 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118170 2024-09-14T09:07:50.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.674 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:07:50.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:07:50.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.674 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842901+0000", 2024-09-14T09:07:50.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004721416 2024-09-14T09:07:50.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842906+0000", 2024-09-14T09:07:50.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7199999999999997e-06 2024-09-14T09:07:50.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842917+0000", 2024-09-14T09:07:50.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0671e-05 2024-09-14T09:07:50.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842923+0000", 2024-09-14T09:07:50.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.088e-06 2024-09-14T09:07:50.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935857+0000", 2024-09-14T09:07:50.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092933624000000006 2024-09-14T09:07:50.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935958+0000", 2024-09-14T09:07:50.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100933 2024-09-14T09:07:50.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.936258+0000", 2024-09-14T09:07:50.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030007800000000001 2024-09-14T09:07:50.677 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.677 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.677 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.677 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118181 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 498688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:50.677 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:07:50.677 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.774233680000002, 2024-09-14T09:07:50.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095212643999999999, 2024-09-14T09:07:50.677 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.677 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.678 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:50.678 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:50.678 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:50.678 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:50.678 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118181 2024-09-14T09:07:50.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.678 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:07:50.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:07:50.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842987+0000", 2024-09-14T09:07:50.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.29e-07 2024-09-14T09:07:50.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842988+0000", 2024-09-14T09:07:50.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.229e-06 2024-09-14T09:07:50.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842993+0000", 2024-09-14T09:07:50.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.792e-06 2024-09-14T09:07:50.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842996+0000", 2024-09-14T09:07:50.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3349999999999998e-06 2024-09-14T09:07:50.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.680 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.680 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.937807+0000", 2024-09-14T09:07:50.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.094811585000000004 2024-09-14T09:07:50.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.937899+0000", 2024-09-14T09:07:50.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1719999999999996e-05 2024-09-14T09:07:50.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938199+0000", 2024-09-14T09:07:50.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000299754 2024-09-14T09:07:50.681 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.681 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.682 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.682 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118182 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 502784~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:50.682 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:07:50.682 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.774220502999999, 2024-09-14T09:07:50.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095557953000000001, 2024-09-14T09:07:50.682 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.682 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.682 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:50.683 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:50.683 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:50.683 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:50.683 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118182 2024-09-14T09:07:50.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.683 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:07:50.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:07:50.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:07:50.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2999999999999999e-07 2024-09-14T09:07:50.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843001+0000", 2024-09-14T09:07:50.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.652e-06 2024-09-14T09:07:50.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843005+0000", 2024-09-14T09:07:50.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2780000000000002e-06 2024-09-14T09:07:50.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843007+0000", 2024-09-14T09:07:50.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.553e-06 2024-09-14T09:07:50.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938240+0000", 2024-09-14T09:07:50.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095232864 2024-09-14T09:07:50.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938295+0000", 2024-09-14T09:07:50.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5095e-05 2024-09-14T09:07:50.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938557+0000", 2024-09-14T09:07:50.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026228100000000001 2024-09-14T09:07:50.686 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.686 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.686 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.686 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118184 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 506880~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:50.686 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:07:50.687 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.774209302999999, 2024-09-14T09:07:50.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095950246000000003, 2024-09-14T09:07:50.687 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.687 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.687 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:50.687 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:50.687 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:50.687 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:50.687 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118184 2024-09-14T09:07:50.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.687 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:07:50.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:07:50.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:07:50.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2399999999999999e-07 2024-09-14T09:07:50.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843012+0000", 2024-09-14T09:07:50.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.269e-06 2024-09-14T09:07:50.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843015+0000", 2024-09-14T09:07:50.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0469999999999998e-06 2024-09-14T09:07:50.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843018+0000", 2024-09-14T09:07:50.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.044e-06 2024-09-14T09:07:50.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938597+0000", 2024-09-14T09:07:50.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095578688999999994 2024-09-14T09:07:50.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938687+0000", 2024-09-14T09:07:50.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0284000000000005e-05 2024-09-14T09:07:50.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938961+0000", 2024-09-14T09:07:50.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027368899999999997 2024-09-14T09:07:50.691 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.691 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.691 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.691 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118186 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 510976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:50.691 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:07:50.691 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.774198342999998, 2024-09-14T09:07:50.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096286497999999998, 2024-09-14T09:07:50.691 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.691 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.691 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:50.692 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:50.692 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:50.692 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:50.692 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118186 2024-09-14T09:07:50.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.692 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:07:50.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:07:50.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:07:50.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.35e-07 2024-09-14T09:07:50.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843023+0000", 2024-09-14T09:07:50.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.195e-06 2024-09-14T09:07:50.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843026+0000", 2024-09-14T09:07:50.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.055e-06 2024-09-14T09:07:50.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843029+0000", 2024-09-14T09:07:50.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6539999999999999e-06 2024-09-14T09:07:50.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939000+0000", 2024-09-14T09:07:50.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095971476 2024-09-14T09:07:50.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939049+0000", 2024-09-14T09:07:50.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8986999999999997e-05 2024-09-14T09:07:50.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939308+0000", 2024-09-14T09:07:50.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000258896 2024-09-14T09:07:50.695 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.695 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.695 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.695 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118188 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 515072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:50.696 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:07:50.696 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.774187953999999, 2024-09-14T09:07:50.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096722717999999999, 2024-09-14T09:07:50.696 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.696 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.696 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:50.696 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:50.696 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:50.696 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:50.696 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118188 2024-09-14T09:07:50.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.696 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:07:50.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:07:50.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:07:50.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2999999999999999e-07 2024-09-14T09:07:50.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843034+0000", 2024-09-14T09:07:50.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T09:07:50.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843037+0000", 2024-09-14T09:07:50.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0989999999999999e-06 2024-09-14T09:07:50.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843039+0000", 2024-09-14T09:07:50.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.666e-06 2024-09-14T09:07:50.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939351+0000", 2024-09-14T09:07:50.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096311789999999994 2024-09-14T09:07:50.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939440+0000", 2024-09-14T09:07:50.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8843000000000003e-05 2024-09-14T09:07:50.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939755+0000", 2024-09-14T09:07:50.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031486399999999999 2024-09-14T09:07:50.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.700 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.700 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.700 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118191 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 519168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:50.700 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:07:50.700 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.774177260999998, 2024-09-14T09:07:50.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097080607999999999, 2024-09-14T09:07:50.700 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.700 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.700 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:50.700 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:50.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:50.701 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:50.701 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118191 2024-09-14T09:07:50.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.701 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:07:50.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:07:50.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:07:50.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2100000000000001e-07 2024-09-14T09:07:50.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843044+0000", 2024-09-14T09:07:50.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2249999999999999e-06 2024-09-14T09:07:50.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843047+0000", 2024-09-14T09:07:50.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0240000000000002e-06 2024-09-14T09:07:50.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843050+0000", 2024-09-14T09:07:50.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.508e-06 2024-09-14T09:07:50.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939791+0000", 2024-09-14T09:07:50.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096741399000000006 2024-09-14T09:07:50.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939849+0000", 2024-09-14T09:07:50.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7912999999999999e-05 2024-09-14T09:07:50.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940123+0000", 2024-09-14T09:07:50.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027431800000000001 2024-09-14T09:07:50.704 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.704 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.704 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.704 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118192 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 523264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:50.704 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:07:50.705 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.774166201, 2024-09-14T09:07:50.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097587224, 2024-09-14T09:07:50.705 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.705 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.705 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:50.705 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:50.705 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:50.705 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:50.705 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118192 2024-09-14T09:07:50.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.705 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:07:50.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:07:50.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844132+0000", 2024-09-14T09:07:50.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010777810000000001 2024-09-14T09:07:50.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844140+0000", 2024-09-14T09:07:50.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4470000000000006e-06 2024-09-14T09:07:50.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844155+0000", 2024-09-14T09:07:50.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4924e-05 2024-09-14T09:07:50.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844163+0000", 2024-09-14T09:07:50.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9980000000000003e-06 2024-09-14T09:07:50.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940199+0000", 2024-09-14T09:07:50.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096036047999999999 2024-09-14T09:07:50.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940282+0000", 2024-09-14T09:07:50.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3073999999999998e-05 2024-09-14T09:07:50.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940641+0000", 2024-09-14T09:07:50.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000358952 2024-09-14T09:07:50.709 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.709 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.709 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.709 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118197 3.17 3:ec5ec63a:::rbd_data.115e96ceebb2.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 3718221] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:50.709 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:07:50.709 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 18.769386785999998, 2024-09-14T09:07:50.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.141131069, 2024-09-14T09:07:50.709 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:50.709 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:50.709 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:50.710 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:50.710 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:50.710 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:50.710 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118197 2024-09-14T09:07:50.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.710 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:50.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:50.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:07:50.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:50.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:07:50.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:50.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:50.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847834+0000", 2024-09-14T09:07:50.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2500000000000004e-07 2024-09-14T09:07:50.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:50.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849227+0000", 2024-09-14T09:07:50.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0013932149999999999 2024-09-14T09:07:50.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:50.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849246+0000", 2024-09-14T09:07:50.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8958e-05 2024-09-14T09:07:50.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:50.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849254+0000", 2024-09-14T09:07:50.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2360000000000004e-06 2024-09-14T09:07:50.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:50.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940154+0000", 2024-09-14T09:07:50.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.090899468999999997 2024-09-14T09:07:50.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:50.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940312+0000", 2024-09-14T09:07:50.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000158326 2024-09-14T09:07:50.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:07:50.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940728+0000", 2024-09-14T09:07:50.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00041583100000000002 2024-09-14T09:07:50.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:50.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.962731+0000", 2024-09-14T09:07:50.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022002608999999999 2024-09-14T09:07:50.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:50.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988881+0000", 2024-09-14T09:07:50.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026150620999999999 2024-09-14T09:07:50.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:50.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988919+0000", 2024-09-14T09:07:50.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8108999999999997e-05 2024-09-14T09:07:50.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:50.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:50.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:50.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988964+0000", 2024-09-14T09:07:50.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4870000000000002e-05 2024-09-14T09:07:50.715 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.715 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.715 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.715 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:50.715 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:50.715 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:50.717 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:07:50.717 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:50.718+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:50.797 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.3.4 2024-09-14T09:07:50.842 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:50.846+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:50.844 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:50.844 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:50.845 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:50.845 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:50.859 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:07:50.918 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:50.919+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:50.981 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:50.981 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:50.981 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:50.981 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:50.981 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:50.995 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:07:51.038 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:51.039+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:51.038 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T09:07:51.120 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:51.119+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:51.180 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:51.180 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:51.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:51.180 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73542 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15319040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.910929211999999, 2024-09-14T09:07:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0237091359999999, 2024-09-14T09:07:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73542 2024-09-14T09:07:51.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:07:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268762+0000", 2024-09-14T09:07:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268766+0000", 2024-09-14T09:07:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5880000000000002e-06 2024-09-14T09:07:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268776+0000", 2024-09-14T09:07:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0168e-05 2024-09-14T09:07:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268789+0000", 2024-09-14T09:07:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:07:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282053+0000", 2024-09-14T09:07:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013264496000000001 2024-09-14T09:07:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282057+0000", 2024-09-14T09:07:51.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6090000000000001e-06 2024-09-14T09:07:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292052+0000", 2024-09-14T09:07:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0099950070000001 2024-09-14T09:07:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292262+0000", 2024-09-14T09:07:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021009800000000001 2024-09-14T09:07:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292470+0000", 2024-09-14T09:07:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000208314 2024-09-14T09:07:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73543 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.910890421999994, 2024-09-14T09:07:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023985473, 2024-09-14T09:07:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.186 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73543 2024-09-14T09:07:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:07:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268801+0000", 2024-09-14T09:07:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268803+0000", 2024-09-14T09:07:51.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7439999999999999e-06 2024-09-14T09:07:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268812+0000", 2024-09-14T09:07:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7080000000000003e-06 2024-09-14T09:07:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268819+0000", 2024-09-14T09:07:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T09:07:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282068+0000", 2024-09-14T09:07:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013249540000000001 2024-09-14T09:07:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282072+0000", 2024-09-14T09:07:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0960000000000001e-06 2024-09-14T09:07:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292573+0000", 2024-09-14T09:07:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0105014560000001 2024-09-14T09:07:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292679+0000", 2024-09-14T09:07:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010545899999999999 2024-09-14T09:07:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292786+0000", 2024-09-14T09:07:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107052 2024-09-14T09:07:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.191 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73544 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.910862719999997, 2024-09-14T09:07:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0244830549999999, 2024-09-14T09:07:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73544 2024-09-14T09:07:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:07:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5599999999999995e-07 2024-09-14T09:07:51.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268831+0000", 2024-09-14T09:07:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.57e-06 2024-09-14T09:07:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268844+0000", 2024-09-14T09:07:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2811e-05 2024-09-14T09:07:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268851+0000", 2024-09-14T09:07:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9140000000000002e-06 2024-09-14T09:07:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282083+0000", 2024-09-14T09:07:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013232111 2024-09-14T09:07:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282086+0000", 2024-09-14T09:07:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2909999999999999e-06 2024-09-14T09:07:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292884+0000", 2024-09-14T09:07:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0107975069999999 2024-09-14T09:07:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293100+0000", 2024-09-14T09:07:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021639399999999999 2024-09-14T09:07:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293311+0000", 2024-09-14T09:07:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021090099999999999 2024-09-14T09:07:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73545 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15564800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.910811534999993, 2024-09-14T09:07:51.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0249746740000001, 2024-09-14T09:07:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73545 2024-09-14T09:07:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:07:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268880+0000", 2024-09-14T09:07:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5600000000000005e-07 2024-09-14T09:07:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268889+0000", 2024-09-14T09:07:51.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8559999999999996e-06 2024-09-14T09:07:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268913+0000", 2024-09-14T09:07:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3958e-05 2024-09-14T09:07:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268921+0000", 2024-09-14T09:07:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7479999999999999e-06 2024-09-14T09:07:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282098+0000", 2024-09-14T09:07:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013176297999999999 2024-09-14T09:07:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282101+0000", 2024-09-14T09:07:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2329999999999998e-06 2024-09-14T09:07:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293435+0000", 2024-09-14T09:07:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0113339459999999 2024-09-14T09:07:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293648+0000", 2024-09-14T09:07:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021307100000000001 2024-09-14T09:07:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293854+0000", 2024-09-14T09:07:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020580800000000001 2024-09-14T09:07:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.202 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73546 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15646720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.910757664000002, 2024-09-14T09:07:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0254846360000001, 2024-09-14T09:07:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73546 2024-09-14T09:07:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:07:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268934+0000", 2024-09-14T09:07:51.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7999999999999995e-07 2024-09-14T09:07:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268937+0000", 2024-09-14T09:07:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9840000000000001e-06 2024-09-14T09:07:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268947+0000", 2024-09-14T09:07:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0522e-05 2024-09-14T09:07:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268955+0000", 2024-09-14T09:07:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8059999999999995e-06 2024-09-14T09:07:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282112+0000", 2024-09-14T09:07:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013157189 2024-09-14T09:07:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282115+0000", 2024-09-14T09:07:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2949999999999998e-06 2024-09-14T09:07:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293968+0000", 2024-09-14T09:07:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0118530990000001 2024-09-14T09:07:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294197+0000", 2024-09-14T09:07:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228507 2024-09-14T09:07:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294417+0000", 2024-09-14T09:07:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000220554 2024-09-14T09:07:51.207 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73547 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.910725217999996, 2024-09-14T09:07:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.025970525, 2024-09-14T09:07:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.208 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73547 2024-09-14T09:07:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:07:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268966+0000", 2024-09-14T09:07:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3799999999999997e-07 2024-09-14T09:07:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268969+0000", 2024-09-14T09:07:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8739999999999999e-06 2024-09-14T09:07:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268979+0000", 2024-09-14T09:07:51.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9939999999999995e-06 2024-09-14T09:07:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268986+0000", 2024-09-14T09:07:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3459999999999996e-06 2024-09-14T09:07:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282127+0000", 2024-09-14T09:07:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01314039 2024-09-14T09:07:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282130+0000", 2024-09-14T09:07:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2219999999999998e-06 2024-09-14T09:07:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294540+0000", 2024-09-14T09:07:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124105880000001 2024-09-14T09:07:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294759+0000", 2024-09-14T09:07:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000218798 2024-09-14T09:07:51.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294936+0000", 2024-09-14T09:07:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000176675 2024-09-14T09:07:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73548 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15810560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.910694105999994, 2024-09-14T09:07:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0265045129999999, 2024-09-14T09:07:51.213 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73548 2024-09-14T09:07:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:07:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268997+0000", 2024-09-14T09:07:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7599999999999996e-07 2024-09-14T09:07:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269000+0000", 2024-09-14T09:07:51.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0580000000000002e-06 2024-09-14T09:07:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269010+0000", 2024-09-14T09:07:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124e-05 2024-09-14T09:07:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269018+0000", 2024-09-14T09:07:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5399999999999998e-06 2024-09-14T09:07:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282141+0000", 2024-09-14T09:07:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013123012 2024-09-14T09:07:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282144+0000", 2024-09-14T09:07:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.196e-06 2024-09-14T09:07:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295042+0000", 2024-09-14T09:07:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0128975689999999 2024-09-14T09:07:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295272+0000", 2024-09-14T09:07:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023048099999999999 2024-09-14T09:07:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295501+0000", 2024-09-14T09:07:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228657 2024-09-14T09:07:51.218 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73549 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15892480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.910648226999996, 2024-09-14T09:07:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0268354099999999, 2024-09-14T09:07:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.219 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73549 2024-09-14T09:07:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:07:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269043+0000", 2024-09-14T09:07:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6199999999999997e-07 2024-09-14T09:07:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269046+0000", 2024-09-14T09:07:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2480000000000001e-06 2024-09-14T09:07:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269057+0000", 2024-09-14T09:07:51.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0436e-05 2024-09-14T09:07:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269064+0000", 2024-09-14T09:07:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5179999999999998e-06 2024-09-14T09:07:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282155+0000", 2024-09-14T09:07:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013091190000000001 2024-09-14T09:07:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282159+0000", 2024-09-14T09:07:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.366e-06 2024-09-14T09:07:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295599+0000", 2024-09-14T09:07:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0134405559999999 2024-09-14T09:07:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295765+0000", 2024-09-14T09:07:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016575499999999999 2024-09-14T09:07:51.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295878+0000", 2024-09-14T09:07:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112679 2024-09-14T09:07:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73550 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15974400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:51.224 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.910615626999999, 2024-09-14T09:07:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027357657, 2024-09-14T09:07:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73550 2024-09-14T09:07:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:07:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269076+0000", 2024-09-14T09:07:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:07:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269079+0000", 2024-09-14T09:07:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:07:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269098+0000", 2024-09-14T09:07:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9698e-05 2024-09-14T09:07:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269106+0000", 2024-09-14T09:07:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6179999999999997e-06 2024-09-14T09:07:51.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282170+0000", 2024-09-14T09:07:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013063874 2024-09-14T09:07:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282173+0000", 2024-09-14T09:07:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1999999999999999e-06 2024-09-14T09:07:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295996+0000", 2024-09-14T09:07:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.013823151 2024-09-14T09:07:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296214+0000", 2024-09-14T09:07:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021817700000000001 2024-09-14T09:07:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296433+0000", 2024-09-14T09:07:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021829099999999999 2024-09-14T09:07:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.229 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73551 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.910574209999993, 2024-09-14T09:07:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027730276, 2024-09-14T09:07:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73551 2024-09-14T09:07:51.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:07:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269117+0000", 2024-09-14T09:07:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:07:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269120+0000", 2024-09-14T09:07:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.056e-06 2024-09-14T09:07:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269130+0000", 2024-09-14T09:07:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0084e-05 2024-09-14T09:07:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269137+0000", 2024-09-14T09:07:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1359999999999999e-06 2024-09-14T09:07:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282184+0000", 2024-09-14T09:07:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013046537 2024-09-14T09:07:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282188+0000", 2024-09-14T09:07:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3220000000000003e-06 2024-09-14T09:07:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296529+0000", 2024-09-14T09:07:51.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0143406109999999 2024-09-14T09:07:51.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296695+0000", 2024-09-14T09:07:51.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016604799999999999 2024-09-14T09:07:51.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296847+0000", 2024-09-14T09:07:51.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000151822 2024-09-14T09:07:51.234 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.234 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.235 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.235 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73552 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:51.235 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:51.235 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.910543630000006, 2024-09-14T09:07:51.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0281767500000001, 2024-09-14T09:07:51.235 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.235 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.235 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.235 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.235 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.235 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73552 2024-09-14T09:07:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:07:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:07:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269150+0000", 2024-09-14T09:07:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9119999999999998e-06 2024-09-14T09:07:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269160+0000", 2024-09-14T09:07:51.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6819999999999998e-06 2024-09-14T09:07:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269168+0000", 2024-09-14T09:07:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6079999999999998e-06 2024-09-14T09:07:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282205+0000", 2024-09-14T09:07:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013036982000000001 2024-09-14T09:07:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282208+0000", 2024-09-14T09:07:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4620000000000001e-06 2024-09-14T09:07:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296968+0000", 2024-09-14T09:07:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0147601209999999 2024-09-14T09:07:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297115+0000", 2024-09-14T09:07:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014673900000000001 2024-09-14T09:07:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297324+0000", 2024-09-14T09:07:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020866400000000001 2024-09-14T09:07:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73553 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.910513566000006, 2024-09-14T09:07:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0286556739999999, 2024-09-14T09:07:51.240 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73553 2024-09-14T09:07:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:07:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269178+0000", 2024-09-14T09:07:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:07:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269180+0000", 2024-09-14T09:07:51.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T09:07:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269189+0000", 2024-09-14T09:07:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2e-06 2024-09-14T09:07:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269197+0000", 2024-09-14T09:07:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2980000000000003e-06 2024-09-14T09:07:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282219+0000", 2024-09-14T09:07:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013022687 2024-09-14T09:07:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282223+0000", 2024-09-14T09:07:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.241e-06 2024-09-14T09:07:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297470+0000", 2024-09-14T09:07:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.015247778 2024-09-14T09:07:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297704+0000", 2024-09-14T09:07:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000233367 2024-09-14T09:07:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297833+0000", 2024-09-14T09:07:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012888900000000001 2024-09-14T09:07:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.245 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73554 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.910484249000007, 2024-09-14T09:07:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.029005113, 2024-09-14T09:07:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73554 2024-09-14T09:07:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:07:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269207+0000", 2024-09-14T09:07:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.68e-07 2024-09-14T09:07:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269210+0000", 2024-09-14T09:07:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8100000000000002e-06 2024-09-14T09:07:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269219+0000", 2024-09-14T09:07:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:07:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269227+0000", 2024-09-14T09:07:51.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3839999999999999e-06 2024-09-14T09:07:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282234+0000", 2024-09-14T09:07:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013007187 2024-09-14T09:07:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282237+0000", 2024-09-14T09:07:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1200000000000002e-06 2024-09-14T09:07:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297942+0000", 2024-09-14T09:07:51.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0157048040000001 2024-09-14T09:07:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298087+0000", 2024-09-14T09:07:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014499099999999999 2024-09-14T09:07:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298211+0000", 2024-09-14T09:07:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012461899999999999 2024-09-14T09:07:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73555 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.910454400999996, 2024-09-14T09:07:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.02941558, 2024-09-14T09:07:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73555 2024-09-14T09:07:51.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:07:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269237+0000", 2024-09-14T09:07:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3399999999999999e-07 2024-09-14T09:07:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269240+0000", 2024-09-14T09:07:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8480000000000001e-06 2024-09-14T09:07:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269249+0000", 2024-09-14T09:07:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1880000000000008e-06 2024-09-14T09:07:51.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269256+0000", 2024-09-14T09:07:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:07:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282248+0000", 2024-09-14T09:07:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012992327 2024-09-14T09:07:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282251+0000", 2024-09-14T09:07:51.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2160000000000002e-06 2024-09-14T09:07:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298299+0000", 2024-09-14T09:07:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0160479330000001 2024-09-14T09:07:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298446+0000", 2024-09-14T09:07:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000146525 2024-09-14T09:07:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298652+0000", 2024-09-14T09:07:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020584100000000001 2024-09-14T09:07:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73556 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.910424655, 2024-09-14T09:07:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0299854239999999, 2024-09-14T09:07:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.256 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73556 2024-09-14T09:07:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:51.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:07:51.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:07:51.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269270+0000", 2024-09-14T09:07:51.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0920000000000002e-06 2024-09-14T09:07:51.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269280+0000", 2024-09-14T09:07:51.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0013999999999999e-05 2024-09-14T09:07:51.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269288+0000", 2024-09-14T09:07:51.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:07:51.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282263+0000", 2024-09-14T09:07:51.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012974703000000001 2024-09-14T09:07:51.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282266+0000", 2024-09-14T09:07:51.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1520000000000001e-06 2024-09-14T09:07:51.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298729+0000", 2024-09-14T09:07:51.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.016463506 2024-09-14T09:07:51.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298884+0000", 2024-09-14T09:07:51.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015450800000000001 2024-09-14T09:07:51.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299251+0000", 2024-09-14T09:07:51.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036759899999999997 2024-09-14T09:07:51.261 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.261 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.261 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.261 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73557 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16547840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:51.261 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:51.262 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.910378004999998, 2024-09-14T09:07:51.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0307969480000001, 2024-09-14T09:07:51.262 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.262 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.262 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.262 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.262 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.262 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.262 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73557 2024-09-14T09:07:51.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.262 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:51.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:07:51.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269313+0000", 2024-09-14T09:07:51.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.92e-07 2024-09-14T09:07:51.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269336+0000", 2024-09-14T09:07:51.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3118000000000001e-05 2024-09-14T09:07:51.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269346+0000", 2024-09-14T09:07:51.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4069999999999999e-06 2024-09-14T09:07:51.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.264 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.264 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269352+0000", 2024-09-14T09:07:51.264 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5810000000000002e-06 2024-09-14T09:07:51.264 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.264 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282277+0000", 2024-09-14T09:07:51.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012924658 2024-09-14T09:07:51.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282280+0000", 2024-09-14T09:07:51.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2880000000000001e-06 2024-09-14T09:07:51.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.265 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.265 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.265 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299343+0000", 2024-09-14T09:07:51.265 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0170630190000001 2024-09-14T09:07:51.265 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299554+0000", 2024-09-14T09:07:51.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021085099999999999 2024-09-14T09:07:51.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.266 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.266 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300109+0000", 2024-09-14T09:07:51.266 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00055533399999999999 2024-09-14T09:07:51.266 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.266 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.266 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.266 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.267 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73558 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:51.267 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:51.267 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.910326381999994, 2024-09-14T09:07:51.267 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0314849340000001, 2024-09-14T09:07:51.267 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.267 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.267 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.267 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.267 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.267 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.267 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73558 2024-09-14T09:07:51.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.267 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:51.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:07:51.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.268 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.268 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.268 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.268 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269365+0000", 2024-09-14T09:07:51.268 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7400000000000003e-07 2024-09-14T09:07:51.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269367+0000", 2024-09-14T09:07:51.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4930000000000001e-06 2024-09-14T09:07:51.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.269 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.269 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269374+0000", 2024-09-14T09:07:51.269 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9519999999999997e-06 2024-09-14T09:07:51.269 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.269 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269379+0000", 2024-09-14T09:07:51.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9710000000000003e-06 2024-09-14T09:07:51.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282292+0000", 2024-09-14T09:07:51.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012912704000000001 2024-09-14T09:07:51.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.270 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.270 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.270 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282295+0000", 2024-09-14T09:07:51.270 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.179e-06 2024-09-14T09:07:51.270 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300195+0000", 2024-09-14T09:07:51.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.017899975 2024-09-14T09:07:51.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300345+0000", 2024-09-14T09:07:51.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014957499999999999 2024-09-14T09:07:51.271 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.271 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.271 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.271 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300849+0000", 2024-09-14T09:07:51.271 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050451100000000002 2024-09-14T09:07:51.272 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.272 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.272 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.272 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.272 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.272 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73559 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:07:51.272 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:51.272 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 76.910303972999998, 2024-09-14T09:07:51.272 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.031916343, 2024-09-14T09:07:51.272 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.272 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.272 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.272 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.273 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.273 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.273 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73559 2024-09-14T09:07:51.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.273 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:51.273 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.273 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.273 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.273 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.273 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:51.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:07:51.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3500000000000002e-07 2024-09-14T09:07:51.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.274 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269390+0000", 2024-09-14T09:07:51.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5890000000000001e-06 2024-09-14T09:07:51.274 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.274 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.274 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269396+0000", 2024-09-14T09:07:51.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3300000000000004e-06 2024-09-14T09:07:51.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269401+0000", 2024-09-14T09:07:51.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0340000000000004e-06 2024-09-14T09:07:51.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.275 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.275 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282311+0000", 2024-09-14T09:07:51.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012910204 2024-09-14T09:07:51.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282317+0000", 2024-09-14T09:07:51.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2639999999999997e-06 2024-09-14T09:07:51.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300933+0000", 2024-09-14T09:07:51.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0186160710000001 2024-09-14T09:07:51.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301158+0000", 2024-09-14T09:07:51.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225021 2024-09-14T09:07:51.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301303+0000", 2024-09-14T09:07:51.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014439499999999999 2024-09-14T09:07:51.277 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.277 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.277 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.277 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106964 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:51.277 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:07:51.277 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.064376139, 2024-09-14T09:07:51.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8802924409999999, 2024-09-14T09:07:51.278 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.278 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.278 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:51.278 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.278 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.278 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.278 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106964 2024-09-14T09:07:51.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.278 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:07:51.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:07:51.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115317+0000", 2024-09-14T09:07:51.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6429999999999999e-06 2024-09-14T09:07:51.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115326+0000", 2024-09-14T09:07:51.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2140000000000002e-06 2024-09-14T09:07:51.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115346+0000", 2024-09-14T09:07:51.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9419000000000002e-05 2024-09-14T09:07:51.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.280 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115357+0000", 2024-09-14T09:07:51.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1014e-05 2024-09-14T09:07:51.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.280 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115431+0000", 2024-09-14T09:07:51.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4040000000000003e-05 2024-09-14T09:07:51.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115438+0000", 2024-09-14T09:07:51.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9920000000000001e-06 2024-09-14T09:07:51.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844944+0000", 2024-09-14T09:07:51.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72950628399999995 2024-09-14T09:07:51.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844961+0000", 2024-09-14T09:07:51.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7156e-05 2024-09-14T09:07:51.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988004+0000", 2024-09-14T09:07:51.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.14304297 2024-09-14T09:07:51.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988511+0000", 2024-09-14T09:07:51.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050694100000000001 2024-09-14T09:07:51.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:51.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.989802+0000", 2024-09-14T09:07:51.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001291195 2024-09-14T09:07:51.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:51.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994866+0000", 2024-09-14T09:07:51.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050632760000000002 2024-09-14T09:07:51.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:51.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994988+0000", 2024-09-14T09:07:51.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012252899999999999 2024-09-14T09:07:51.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:51.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995198+0000", 2024-09-14T09:07:51.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021010899999999999 2024-09-14T09:07:51.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995607+0000", 2024-09-14T09:07:51.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040865900000000002 2024-09-14T09:07:51.284 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.284 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.284 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.285 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106965 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:51.285 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:07:51.285 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.063173762000002, 2024-09-14T09:07:51.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.879730162, 2024-09-14T09:07:51.285 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.285 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.285 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:51.285 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.285 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.285 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.285 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106965 2024-09-14T09:07:51.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.286 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:07:51.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:07:51.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:07:51.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:07:51.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116522+0000", 2024-09-14T09:07:51.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3989999999999997e-06 2024-09-14T09:07:51.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116533+0000", 2024-09-14T09:07:51.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0669000000000001e-05 2024-09-14T09:07:51.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116541+0000", 2024-09-14T09:07:51.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2730000000000002e-06 2024-09-14T09:07:51.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116604+0000", 2024-09-14T09:07:51.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2706999999999994e-05 2024-09-14T09:07:51.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116610+0000", 2024-09-14T09:07:51.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8630000000000002e-06 2024-09-14T09:07:51.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845060+0000", 2024-09-14T09:07:51.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72845003699999999 2024-09-14T09:07:51.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845079+0000", 2024-09-14T09:07:51.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8287999999999998e-05 2024-09-14T09:07:51.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.991866+0000", 2024-09-14T09:07:51.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1467871519999999 2024-09-14T09:07:51.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.992274+0000", 2024-09-14T09:07:51.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040813500000000001 2024-09-14T09:07:51.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:51.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.993538+0000", 2024-09-14T09:07:51.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001263655 2024-09-14T09:07:51.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:51.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995770+0000", 2024-09-14T09:07:51.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002231949 2024-09-14T09:07:51.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:51.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995810+0000", 2024-09-14T09:07:51.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0852999999999997e-05 2024-09-14T09:07:51.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:51.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995979+0000", 2024-09-14T09:07:51.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016866 2024-09-14T09:07:51.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996247+0000", 2024-09-14T09:07:51.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000267769 2024-09-14T09:07:51.292 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.292 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.292 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.292 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.292 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.292 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:51.294 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:07:51.294 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:51.298+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:51.378 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:07:51.395 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:51.398+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:51.403 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:51.403 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:51.428 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:51.428 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:51.453 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:51.453 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:51.457 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:07:51.457 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T09:07:51.462 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:51.462 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:51.462 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:07:51.462 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:51.476 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:07:51.518 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:51.519+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:51.597 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:51.598+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:51.641 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:51.641 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:51.641 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:51.641 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:51.641 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:51.657 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:07:51.698 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:51.702+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:51.785 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106705 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 518144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.785 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:07:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.936172158000002, 2024-09-14T09:07:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3001326799999999, 2024-09-14T09:07:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106705 2024-09-14T09:07:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.786 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:07:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:07:51.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:07:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4699999999999998e-07 2024-09-14T09:07:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847500+0000", 2024-09-14T09:07:51.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0860000000000001e-06 2024-09-14T09:07:51.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847503+0000", 2024-09-14T09:07:51.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1269999999999999e-06 2024-09-14T09:07:51.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847506+0000", 2024-09-14T09:07:51.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5780000000000001e-06 2024-09-14T09:07:51.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858911+0000", 2024-09-14T09:07:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011404925 2024-09-14T09:07:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858919+0000", 2024-09-14T09:07:51.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8620000000000003e-06 2024-09-14T09:07:51.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147254+0000", 2024-09-14T09:07:51.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2883353719999999 2024-09-14T09:07:51.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147408+0000", 2024-09-14T09:07:51.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015389200000000001 2024-09-14T09:07:51.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147631+0000", 2024-09-14T09:07:51.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022359100000000001 2024-09-14T09:07:51.795 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.795 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106707 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 522240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:07:51.796 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.936161687999999, 2024-09-14T09:07:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.300471801, 2024-09-14T09:07:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.797 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106707 2024-09-14T09:07:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:07:51.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:07:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:07:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.23e-07 2024-09-14T09:07:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847514+0000", 2024-09-14T09:07:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9150000000000004e-06 2024-09-14T09:07:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847520+0000", 2024-09-14T09:07:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.203e-06 2024-09-14T09:07:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847532+0000", 2024-09-14T09:07:51.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2441e-05 2024-09-14T09:07:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858953+0000", 2024-09-14T09:07:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01142108 2024-09-14T09:07:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858957+0000", 2024-09-14T09:07:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2830000000000003e-06 2024-09-14T09:07:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147667+0000", 2024-09-14T09:07:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2887094619999999 2024-09-14T09:07:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147784+0000", 2024-09-14T09:07:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117465 2024-09-14T09:07:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147981+0000", 2024-09-14T09:07:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019672900000000001 2024-09-14T09:07:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.803 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106709 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 3072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:07:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.936133396999999, 2024-09-14T09:07:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.301335192, 2024-09-14T09:07:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106709 2024-09-14T09:07:51.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:07:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:07:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847538+0000", 2024-09-14T09:07:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2800000000000003e-07 2024-09-14T09:07:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847539+0000", 2024-09-14T09:07:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2720000000000001e-06 2024-09-14T09:07:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847544+0000", 2024-09-14T09:07:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7219999999999999e-06 2024-09-14T09:07:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847547+0000", 2024-09-14T09:07:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9960000000000002e-06 2024-09-14T09:07:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858991+0000", 2024-09-14T09:07:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011444218000000001 2024-09-14T09:07:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858999+0000", 2024-09-14T09:07:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9419999999999995e-06 2024-09-14T09:07:51.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148053+0000", 2024-09-14T09:07:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2890544099999999 2024-09-14T09:07:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148203+0000", 2024-09-14T09:07:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149419 2024-09-14T09:07:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148873+0000", 2024-09-14T09:07:51.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00066988499999999995 2024-09-14T09:07:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106711 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 7168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:07:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.936120139, 2024-09-14T09:07:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3022241219999999, 2024-09-14T09:07:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.809 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106711 2024-09-14T09:07:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:07:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:07:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:07:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8000000000000002e-07 2024-09-14T09:07:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847552+0000", 2024-09-14T09:07:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1340000000000001e-06 2024-09-14T09:07:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847557+0000", 2024-09-14T09:07:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6179999999999997e-06 2024-09-14T09:07:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847560+0000", 2024-09-14T09:07:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5899999999999999e-06 2024-09-14T09:07:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859050+0000", 2024-09-14T09:07:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01148914 2024-09-14T09:07:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859056+0000", 2024-09-14T09:07:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6630000000000004e-06 2024-09-14T09:07:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148948+0000", 2024-09-14T09:07:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2898914370000001 2024-09-14T09:07:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149079+0000", 2024-09-14T09:07:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013145400000000001 2024-09-14T09:07:51.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149775+0000", 2024-09-14T09:07:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00069580599999999999 2024-09-14T09:07:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106713 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 11264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.815 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:07:51.816 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.936106674000001, 2024-09-14T09:07:51.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3035188959999999, 2024-09-14T09:07:51.816 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.816 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.816 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.816 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.816 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.816 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106713 2024-09-14T09:07:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:07:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:07:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:07:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6e-07 2024-09-14T09:07:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847566+0000", 2024-09-14T09:07:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1039999999999999e-06 2024-09-14T09:07:51.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847569+0000", 2024-09-14T09:07:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.191e-06 2024-09-14T09:07:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847571+0000", 2024-09-14T09:07:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6390000000000001e-06 2024-09-14T09:07:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859099+0000", 2024-09-14T09:07:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011527379000000001 2024-09-14T09:07:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859106+0000", 2024-09-14T09:07:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3679999999999996e-06 2024-09-14T09:07:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149878+0000", 2024-09-14T09:07:51.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.29077208 2024-09-14T09:07:51.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.150175+0000", 2024-09-14T09:07:51.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029685699999999998 2024-09-14T09:07:51.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151083+0000", 2024-09-14T09:07:51.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00090801800000000002 2024-09-14T09:07:51.822 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106715 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 15360~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.823 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:07:51.824 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.936095665, 2024-09-14T09:07:51.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3047478800000001, 2024-09-14T09:07:51.824 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.824 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.824 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.825 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.825 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.825 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.825 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106715 2024-09-14T09:07:51.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.825 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:07:51.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:07:51.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:07:51.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6e-07 2024-09-14T09:07:51.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847580+0000", 2024-09-14T09:07:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9250000000000003e-06 2024-09-14T09:07:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847585+0000", 2024-09-14T09:07:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8409999999999999e-06 2024-09-14T09:07:51.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847588+0000", 2024-09-14T09:07:51.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6989999999999999e-06 2024-09-14T09:07:51.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859151+0000", 2024-09-14T09:07:51.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011562604000000001 2024-09-14T09:07:51.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859159+0000", 2024-09-14T09:07:51.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4179999999999997e-06 2024-09-14T09:07:51.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151155+0000", 2024-09-14T09:07:51.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.291996068 2024-09-14T09:07:51.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.834 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.834 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151383+0000", 2024-09-14T09:07:51.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022746100000000001 2024-09-14T09:07:51.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152323+0000", 2024-09-14T09:07:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00094060400000000005 2024-09-14T09:07:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.835 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106717 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 19456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:07:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.936078977000001, 2024-09-14T09:07:51.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.305574579, 2024-09-14T09:07:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106717 2024-09-14T09:07:51.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:07:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:07:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:07:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1199999999999999e-07 2024-09-14T09:07:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847593+0000", 2024-09-14T09:07:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.257e-06 2024-09-14T09:07:51.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847598+0000", 2024-09-14T09:07:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4139999999999996e-06 2024-09-14T09:07:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847602+0000", 2024-09-14T09:07:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3440000000000003e-06 2024-09-14T09:07:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859203+0000", 2024-09-14T09:07:51.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011600572999999999 2024-09-14T09:07:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859211+0000", 2024-09-14T09:07:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-06 2024-09-14T09:07:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152398+0000", 2024-09-14T09:07:51.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.293187812 2024-09-14T09:07:51.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152543+0000", 2024-09-14T09:07:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014478000000000001 2024-09-14T09:07:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153166+0000", 2024-09-14T09:07:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000623257 2024-09-14T09:07:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106719 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 23552~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:07:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.936064661, 2024-09-14T09:07:51.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.306394982, 2024-09-14T09:07:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.847 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106719 2024-09-14T09:07:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:07:51.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:07:51.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.850 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:07:51.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5100000000000001e-07 2024-09-14T09:07:51.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847608+0000", 2024-09-14T09:07:51.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.189e-06 2024-09-14T09:07:51.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847612+0000", 2024-09-14T09:07:51.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9519999999999996e-06 2024-09-14T09:07:51.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847614+0000", 2024-09-14T09:07:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6280000000000001e-06 2024-09-14T09:07:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859254+0000", 2024-09-14T09:07:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011640157 2024-09-14T09:07:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859261+0000", 2024-09-14T09:07:51.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0570000000000004e-06 2024-09-14T09:07:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153236+0000", 2024-09-14T09:07:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2939744040000001 2024-09-14T09:07:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153380+0000", 2024-09-14T09:07:51.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014410399999999999 2024-09-14T09:07:51.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.855 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.855 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154001+0000", 2024-09-14T09:07:51.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062124000000000001 2024-09-14T09:07:51.855 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.855 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.855 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.855 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106721 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 27648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.856 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:07:51.856 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.936053010999998, 2024-09-14T09:07:51.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3074446659999999, 2024-09-14T09:07:51.856 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.856 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.856 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.857 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.857 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.857 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.857 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106721 2024-09-14T09:07:51.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.857 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:07:51.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:07:51.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:07:51.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4299999999999999e-07 2024-09-14T09:07:51.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847619+0000", 2024-09-14T09:07:51.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T09:07:51.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847624+0000", 2024-09-14T09:07:51.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4229999999999998e-06 2024-09-14T09:07:51.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847627+0000", 2024-09-14T09:07:51.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0120000000000001e-06 2024-09-14T09:07:51.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859305+0000", 2024-09-14T09:07:51.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01167812 2024-09-14T09:07:51.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859313+0000", 2024-09-14T09:07:51.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2409999999999995e-06 2024-09-14T09:07:51.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154081+0000", 2024-09-14T09:07:51.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.294768296 2024-09-14T09:07:51.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154280+0000", 2024-09-14T09:07:51.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019824 2024-09-14T09:07:51.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.864 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.864 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155063+0000", 2024-09-14T09:07:51.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00078303400000000005 2024-09-14T09:07:51.864 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.864 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.864 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.865 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106723 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 31744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.865 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:07:51.865 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.936040581, 2024-09-14T09:07:51.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308293648, 2024-09-14T09:07:51.865 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.865 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.865 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.865 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.865 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.865 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.865 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106723 2024-09-14T09:07:51.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.865 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:07:51.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:07:51.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847631+0000", 2024-09-14T09:07:51.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6100000000000002e-07 2024-09-14T09:07:51.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847632+0000", 2024-09-14T09:07:51.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0589999999999999e-06 2024-09-14T09:07:51.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847635+0000", 2024-09-14T09:07:51.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.551e-06 2024-09-14T09:07:51.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847638+0000", 2024-09-14T09:07:51.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.644e-06 2024-09-14T09:07:51.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859344+0000", 2024-09-14T09:07:51.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011706397 2024-09-14T09:07:51.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859348+0000", 2024-09-14T09:07:51.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5779999999999999e-06 2024-09-14T09:07:51.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155135+0000", 2024-09-14T09:07:51.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.295787448 2024-09-14T09:07:51.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155279+0000", 2024-09-14T09:07:51.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014400700000000001 2024-09-14T09:07:51.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.869 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.869 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155924+0000", 2024-09-14T09:07:51.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00064470300000000005 2024-09-14T09:07:51.869 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.869 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.869 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.870 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106725 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 35840~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.870 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:07:51.870 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.936029470000001, 2024-09-14T09:07:51.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.309103218, 2024-09-14T09:07:51.870 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.870 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.870 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.870 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.870 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.870 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.870 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106725 2024-09-14T09:07:51.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.870 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:07:51.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:07:51.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847642+0000", 2024-09-14T09:07:51.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4499999999999998e-07 2024-09-14T09:07:51.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847643+0000", 2024-09-14T09:07:51.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4e-07 2024-09-14T09:07:51.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847647+0000", 2024-09-14T09:07:51.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6190000000000002e-06 2024-09-14T09:07:51.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847655+0000", 2024-09-14T09:07:51.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:07:51.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859381+0000", 2024-09-14T09:07:51.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011725793 2024-09-14T09:07:51.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859391+0000", 2024-09-14T09:07:51.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4390000000000006e-06 2024-09-14T09:07:51.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155997+0000", 2024-09-14T09:07:51.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2966067830000001 2024-09-14T09:07:51.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156133+0000", 2024-09-14T09:07:51.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000135609 2024-09-14T09:07:51.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.875 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156745+0000", 2024-09-14T09:07:51.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000611592 2024-09-14T09:07:51.875 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.875 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.875 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.875 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106727 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 39936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.875 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:07:51.875 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.93601117, 2024-09-14T09:07:51.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3099046560000001, 2024-09-14T09:07:51.875 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.876 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.876 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.876 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.876 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.876 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.876 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106727 2024-09-14T09:07:51.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.876 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:07:51.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:07:51.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:07:51.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1300000000000001e-07 2024-09-14T09:07:51.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847661+0000", 2024-09-14T09:07:51.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.192e-06 2024-09-14T09:07:51.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847665+0000", 2024-09-14T09:07:51.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0999999999999997e-06 2024-09-14T09:07:51.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847668+0000", 2024-09-14T09:07:51.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.644e-06 2024-09-14T09:07:51.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859422+0000", 2024-09-14T09:07:51.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011753559 2024-09-14T09:07:51.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859425+0000", 2024-09-14T09:07:51.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.343e-06 2024-09-14T09:07:51.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156817+0000", 2024-09-14T09:07:51.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2973924800000001 2024-09-14T09:07:51.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156956+0000", 2024-09-14T09:07:51.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013835 2024-09-14T09:07:51.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.881 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.881 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157564+0000", 2024-09-14T09:07:51.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060867500000000004 2024-09-14T09:07:51.881 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.881 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.881 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.881 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106729 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 44032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.881 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:07:51.881 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.935999197000001, 2024-09-14T09:07:51.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3107195970000001, 2024-09-14T09:07:51.882 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.882 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.882 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.882 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.882 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.882 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.882 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106729 2024-09-14T09:07:51.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.882 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:07:51.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:07:51.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:07:51.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.29e-07 2024-09-14T09:07:51.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847673+0000", 2024-09-14T09:07:51.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.096e-06 2024-09-14T09:07:51.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847678+0000", 2024-09-14T09:07:51.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.527e-06 2024-09-14T09:07:51.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847681+0000", 2024-09-14T09:07:51.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.788e-06 2024-09-14T09:07:51.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859462+0000", 2024-09-14T09:07:51.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01178032 2024-09-14T09:07:51.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859471+0000", 2024-09-14T09:07:51.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3430000000000002e-06 2024-09-14T09:07:51.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157635+0000", 2024-09-14T09:07:51.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2981640320000001 2024-09-14T09:07:51.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157773+0000", 2024-09-14T09:07:51.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763 2024-09-14T09:07:51.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158391+0000", 2024-09-14T09:07:51.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061863200000000004 2024-09-14T09:07:51.887 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.887 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.887 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.887 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106731 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 48128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.887 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:07:51.887 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.935983552, 2024-09-14T09:07:51.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3115142020000001, 2024-09-14T09:07:51.888 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.888 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.888 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.888 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.888 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.888 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.888 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106731 2024-09-14T09:07:51.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.888 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:07:51.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:07:51.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847688+0000", 2024-09-14T09:07:51.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0199999999999998e-07 2024-09-14T09:07:51.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847689+0000", 2024-09-14T09:07:51.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4529999999999999e-06 2024-09-14T09:07:51.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847694+0000", 2024-09-14T09:07:51.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9080000000000003e-06 2024-09-14T09:07:51.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847697+0000", 2024-09-14T09:07:51.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4920000000000002e-06 2024-09-14T09:07:51.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859499+0000", 2024-09-14T09:07:51.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011801676000000001 2024-09-14T09:07:51.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859502+0000", 2024-09-14T09:07:51.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2899999999999998e-06 2024-09-14T09:07:51.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158461+0000", 2024-09-14T09:07:51.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.298958796 2024-09-14T09:07:51.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158594+0000", 2024-09-14T09:07:51.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013321599999999999 2024-09-14T09:07:51.893 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.893 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159202+0000", 2024-09-14T09:07:51.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060706900000000001 2024-09-14T09:07:51.893 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.893 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.893 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.894 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106733 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 52224~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.894 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:07:51.894 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.935969296, 2024-09-14T09:07:51.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3123343809999999, 2024-09-14T09:07:51.894 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.894 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.894 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.894 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.894 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.895 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.895 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106733 2024-09-14T09:07:51.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.895 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:07:51.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:07:51.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:07:51.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5800000000000001e-07 2024-09-14T09:07:51.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847703+0000", 2024-09-14T09:07:51.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.127e-06 2024-09-14T09:07:51.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847706+0000", 2024-09-14T09:07:51.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3210000000000001e-06 2024-09-14T09:07:51.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847709+0000", 2024-09-14T09:07:51.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7539999999999998e-06 2024-09-14T09:07:51.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859538+0000", 2024-09-14T09:07:51.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01182942 2024-09-14T09:07:51.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859547+0000", 2024-09-14T09:07:51.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T09:07:51.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159273+0000", 2024-09-14T09:07:51.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2997263700000001 2024-09-14T09:07:51.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159415+0000", 2024-09-14T09:07:51.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000141765 2024-09-14T09:07:51.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160036+0000", 2024-09-14T09:07:51.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062122100000000003 2024-09-14T09:07:51.899 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.899 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.899 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.899 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106735 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 56320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.899 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:07:51.899 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.935957852000001, 2024-09-14T09:07:51.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3131339309999999, 2024-09-14T09:07:51.899 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.899 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.900 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.900 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.900 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.900 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.900 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106735 2024-09-14T09:07:51.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.900 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:07:51.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:07:51.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:07:51.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2100000000000001e-07 2024-09-14T09:07:51.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.901 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847714+0000", 2024-09-14T09:07:51.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.051e-06 2024-09-14T09:07:51.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.901 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847719+0000", 2024-09-14T09:07:51.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3640000000000001e-06 2024-09-14T09:07:51.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847723+0000", 2024-09-14T09:07:51.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8789999999999996e-06 2024-09-14T09:07:51.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859577+0000", 2024-09-14T09:07:51.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011854416 2024-09-14T09:07:51.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859580+0000", 2024-09-14T09:07:51.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.292e-06 2024-09-14T09:07:51.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160108+0000", 2024-09-14T09:07:51.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.30052758 2024-09-14T09:07:51.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160238+0000", 2024-09-14T09:07:51.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013043500000000001 2024-09-14T09:07:51.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160847+0000", 2024-09-14T09:07:51.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060869299999999999 2024-09-14T09:07:51.904 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.904 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.904 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.904 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106738 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 60416~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.904 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:07:51.904 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.935944648, 2024-09-14T09:07:51.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3139482179999999, 2024-09-14T09:07:51.904 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.905 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.905 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.905 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.905 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.905 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.905 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106738 2024-09-14T09:07:51.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.905 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:07:51.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:07:51.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:07:51.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3999999999999998e-07 2024-09-14T09:07:51.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847733+0000", 2024-09-14T09:07:51.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8070000000000002e-06 2024-09-14T09:07:51.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847738+0000", 2024-09-14T09:07:51.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9740000000000001e-06 2024-09-14T09:07:51.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847742+0000", 2024-09-14T09:07:51.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6009999999999999e-06 2024-09-14T09:07:51.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859612+0000", 2024-09-14T09:07:51.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011870537 2024-09-14T09:07:51.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859621+0000", 2024-09-14T09:07:51.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3019999999999995e-06 2024-09-14T09:07:51.908 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160917+0000", 2024-09-14T09:07:51.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3012961080000001 2024-09-14T09:07:51.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161054+0000", 2024-09-14T09:07:51.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000137422 2024-09-14T09:07:51.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161674+0000", 2024-09-14T09:07:51.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000620227 2024-09-14T09:07:51.910 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.910 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.910 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.910 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106739 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 64512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.910 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:07:51.910 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.935924807999999, 2024-09-14T09:07:51.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.314753777, 2024-09-14T09:07:51.910 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.910 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.910 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106739 2024-09-14T09:07:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:07:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:07:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:07:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9700000000000003e-07 2024-09-14T09:07:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847748+0000", 2024-09-14T09:07:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2589999999999999e-06 2024-09-14T09:07:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847752+0000", 2024-09-14T09:07:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6099999999999999e-06 2024-09-14T09:07:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847755+0000", 2024-09-14T09:07:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0520000000000002e-06 2024-09-14T09:07:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859653+0000", 2024-09-14T09:07:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011897575000000001 2024-09-14T09:07:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859658+0000", 2024-09-14T09:07:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8810000000000004e-06 2024-09-14T09:07:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161745+0000", 2024-09-14T09:07:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3020870819999999 2024-09-14T09:07:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161877+0000", 2024-09-14T09:07:51.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000132657 2024-09-14T09:07:51.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162500+0000", 2024-09-14T09:07:51.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062236400000000003 2024-09-14T09:07:51.916 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.916 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.916 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106741 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 68608~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:07:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.935912016, 2024-09-14T09:07:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3155568449999999, 2024-09-14T09:07:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.917 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106741 2024-09-14T09:07:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:07:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:07:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:07:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5800000000000001e-07 2024-09-14T09:07:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847760+0000", 2024-09-14T09:07:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:07:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847764+0000", 2024-09-14T09:07:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3189999999999999e-06 2024-09-14T09:07:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847766+0000", 2024-09-14T09:07:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6429999999999999e-06 2024-09-14T09:07:51.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859693+0000", 2024-09-14T09:07:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011926724 2024-09-14T09:07:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859703+0000", 2024-09-14T09:07:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6160000000000008e-06 2024-09-14T09:07:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162575+0000", 2024-09-14T09:07:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.302872128 2024-09-14T09:07:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162709+0000", 2024-09-14T09:07:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013446000000000001 2024-09-14T09:07:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163316+0000", 2024-09-14T09:07:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060650700000000005 2024-09-14T09:07:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106744 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 76800~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:07:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 29.935900905, 2024-09-14T09:07:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3163901789999999, 2024-09-14T09:07:51.923 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106744 2024-09-14T09:07:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:07:51.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:07:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996947+0000", 2024-09-14T09:07:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.149177154 2024-09-14T09:07:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:51.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996964+0000", 2024-09-14T09:07:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.679e-05 2024-09-14T09:07:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996999+0000", 2024-09-14T09:07:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5463000000000002e-05 2024-09-14T09:07:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.997029+0000", 2024-09-14T09:07:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9567e-05 2024-09-14T09:07:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:51.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163390+0000", 2024-09-14T09:07:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.16636056799999999 2024-09-14T09:07:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163526+0000", 2024-09-14T09:07:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013655899999999999 2024-09-14T09:07:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.164160+0000", 2024-09-14T09:07:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00063407800000000005 2024-09-14T09:07:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.927 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:51.928 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:51.928 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:51.929 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:07:51.929 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:51.927+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:52.030 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:52.031+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:52.049 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is set 2024-09-14T09:07:52.112 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:52.112 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:52.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.112 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:140554 3.7 3:e140c993:::rbd_header.115e2baf3d9a:head [watch unwatch cookie 139718911127200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-14T09:07:52.112 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:52.113123+0000", 2024-09-14T09:07:52.112 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.00048403699999999999, 2024-09-14T09:07:52.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000508834, 2024-09-14T09:07:52.112 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.112 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.112 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "waiting for sub ops", 2024-09-14T09:07:52.113 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.113 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.113 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.113 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 140554 2024-09-14T09:07:52.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.113 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:52.113123+0000", 2024-09-14T09:07:52.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:52.113123+0000", 2024-09-14T09:07:52.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:52.113123+0000", 2024-09-14T09:07:52.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0699999999999996e-07 2024-09-14T09:07:52.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:52.113127+0000", 2024-09-14T09:07:52.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0260000000000001e-06 2024-09-14T09:07:52.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:52.113139+0000", 2024-09-14T09:07:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1544e-05 2024-09-14T09:07:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:52.113147+0000", 2024-09-14T09:07:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0819999999999999e-06 2024-09-14T09:07:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:52.113273+0000", 2024-09-14T09:07:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000125998 2024-09-14T09:07:52.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:52.113321+0000", 2024-09-14T09:07:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8006000000000001e-05 2024-09-14T09:07:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:07:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:52.113408+0000", 2024-09-14T09:07:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7256999999999995e-05 2024-09-14T09:07:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:140555 3.7 3.c9930287 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e140)", 2024-09-14T09:07:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:52.113339+0000", 2024-09-14T09:07:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.000267311, 2024-09-14T09:07:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036421100000000003, 2024-09-14T09:07:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "queued for pg", 2024-09-14T09:07:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 140555 2024-09-14T09:07:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:52.113339+0000", 2024-09-14T09:07:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:52.113339+0000", 2024-09-14T09:07:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:52.113340+0000", 2024-09-14T09:07:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7000000000000003e-07 2024-09-14T09:07:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:52.113346+0000", 2024-09-14T09:07:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.818e-06 2024-09-14T09:07:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:52.113358+0000", 2024-09-14T09:07:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2391999999999999e-05 2024-09-14T09:07:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:52.113367+0000", 2024-09-14T09:07:52.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.799e-06 2024-09-14T09:07:52.120 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.120 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.120 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.120 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.120 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:52.120 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 2 2024-09-14T09:07:52.120 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:52.129 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:07:52.170 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:52.171+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:52.232 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:52.234+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:52.296 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:52.296 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:52.296 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:52.296 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:52.297 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:52.311 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:07:52.358 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:52.359+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:52.434 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:52.438+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:52.436 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:52.436 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:52.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:52.436 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:52.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.437 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:52.437 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:52.437 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 139.35165569700001, 2024-09-14T09:07:52.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:07:52.437 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.437 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.437 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:52.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.437 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.437 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:07:52.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.437 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:52.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:52.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:07:52.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:07:52.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:07:52.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:07:52.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:07:52.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:07:52.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:07:52.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:07:52.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:07:52.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:07:52.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:52.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:07:52.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:07:52.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:07:52.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:07:52.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:07:52.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:07:52.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:07:52.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:07:52.442 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.442 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.442 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.442 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:52.442 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:52.443 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 139.351601039, 2024-09-14T09:07:52.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:07:52.443 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.443 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.443 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:52.443 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.443 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.443 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.443 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:07:52.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.443 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:52.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:52.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:52.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:07:52.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:07:52.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:07:52.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:07:52.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:07:52.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:07:52.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:07:52.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:07:52.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:07:52.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:52.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:07:52.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:07:52.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:07:52.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:07:52.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:07:52.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:07:52.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:07:52.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:07:52.447 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.447 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.447 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.449 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:52.449 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:52.449 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 139.35149566499999, 2024-09-14T09:07:52.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:07:52.449 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.449 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.449 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:52.449 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.449 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.449 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.449 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:07:52.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.450 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:52.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:52.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:07:52.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:07:52.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:07:52.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:07:52.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:07:52.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:07:52.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:07:52.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:07:52.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:07:52.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:07:52.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:52.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:07:52.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:07:52.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:07:52.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:07:52.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:07:52.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:07:52.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:07:52.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:07:52.453 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.453 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.454 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.454 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:52.454 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:52.454 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 139.35113890100001, 2024-09-14T09:07:52.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:07:52.454 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.454 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.454 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:52.454 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.454 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.454 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.454 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:07:52.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.455 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:52.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:52.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:07:52.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:07:52.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:07:52.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:07:52.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:07:52.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:07:52.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:07:52.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:07:52.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:07:52.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:07:52.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:52.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:07:52.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:07:52.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:07:52.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:07:52.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:07:52.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:07:52.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:07:52.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:07:52.458 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.459 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.459 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.459 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:52.459 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:52.459 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 139.35105416299999, 2024-09-14T09:07:52.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:07:52.459 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.459 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.460 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:52.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.460 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:07:52.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.460 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:52.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:52.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:07:52.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:07:52.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:07:52.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:07:52.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:07:52.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:07:52.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:07:52.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:07:52.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:07:52.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:07:52.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:52.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:07:52.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:07:52.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:07:52.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:07:52.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:07:52.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:07:52.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:07:52.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:07:52.464 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.464 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.464 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.464 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:52.464 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:52.464 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 139.35096647699999, 2024-09-14T09:07:52.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:07:52.464 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.464 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.465 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:52.465 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.465 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.465 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.465 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:07:52.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.465 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:52.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:52.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:07:52.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:07:52.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:07:52.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:07:52.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:07:52.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:07:52.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:07:52.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:07:52.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:07:52.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:07:52.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:52.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:07:52.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:07:52.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:07:52.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:07:52.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:07:52.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:07:52.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:07:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:07:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:07:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 106.27467449, 2024-09-14T09:07:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:07:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.469 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:07:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:07:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:07:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:07:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:07:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:07:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:07:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:07:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:07:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:07:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:07:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:07:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:07:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:07:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:07:52.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:07:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:07:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:07:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:07:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:07:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:07:52.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:07:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:07:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:97796 3.15 3:aaaa5f7f:::rbd_data.115e113dc8ca.0000000000000029:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:07:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 40.340805619999998, 2024-09-14T09:07:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14537731300000001, 2024-09-14T09:07:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.475 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97796 2024-09-14T09:07:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094482+0000", 2024-09-14T09:07:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6400000000000001e-07 2024-09-14T09:07:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094485+0000", 2024-09-14T09:07:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.693e-06 2024-09-14T09:07:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094494+0000", 2024-09-14T09:07:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9590000000000001e-06 2024-09-14T09:07:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094502+0000", 2024-09-14T09:07:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3180000000000001e-06 2024-09-14T09:07:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094567+0000", 2024-09-14T09:07:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5526999999999998e-05 2024-09-14T09:07:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094726+0000", 2024-09-14T09:07:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015842499999999999 2024-09-14T09:07:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:07:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.101940+0000", 2024-09-14T09:07:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0072146559999999998 2024-09-14T09:07:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239665+0000", 2024-09-14T09:07:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.137724501 2024-09-14T09:07:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239726+0000", 2024-09-14T09:07:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1550999999999994e-05 2024-09-14T09:07:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239800+0000", 2024-09-14T09:07:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3318000000000005e-05 2024-09-14T09:07:52.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239858+0000", 2024-09-14T09:07:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8600999999999997e-05 2024-09-14T09:07:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:97796 3.15 e106/103 ondisk, result = 0)", 2024-09-14T09:07:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 40.318459750999999, 2024-09-14T09:07:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123043465, 2024-09-14T09:07:52.481 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116828+0000", 2024-09-14T09:07:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8100000000000002e-07 2024-09-14T09:07:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116831+0000", 2024-09-14T09:07:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3009999999999998e-06 2024-09-14T09:07:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116841+0000", 2024-09-14T09:07:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7829999999999993e-06 2024-09-14T09:07:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116851+0000", 2024-09-14T09:07:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0135999999999999e-05 2024-09-14T09:07:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239707+0000", 2024-09-14T09:07:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12285581499999999 2024-09-14T09:07:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239718+0000", 2024-09-14T09:07:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1804000000000001e-05 2024-09-14T09:07:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239870+0000", 2024-09-14T09:07:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015194500000000001 2024-09-14T09:07:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.485 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117315 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 309248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:07:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.711824014000001, 2024-09-14T09:07:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11247383800000001, 2024-09-14T09:07:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117315 2024-09-14T09:07:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:07:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:07:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723899+0000", 2024-09-14T09:07:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043641699999999998 2024-09-14T09:07:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723903+0000", 2024-09-14T09:07:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5389999999999999e-06 2024-09-14T09:07:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723910+0000", 2024-09-14T09:07:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4229999999999999e-06 2024-09-14T09:07:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723917+0000", 2024-09-14T09:07:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4560000000000003e-06 2024-09-14T09:07:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835514+0000", 2024-09-14T09:07:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111596258 2024-09-14T09:07:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835571+0000", 2024-09-14T09:07:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7317e-05 2024-09-14T09:07:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835936+0000", 2024-09-14T09:07:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000365428 2024-09-14T09:07:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.489 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117317 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 313344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:07:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.711356997999999, 2024-09-14T09:07:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112450201, 2024-09-14T09:07:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.490 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117317 2024-09-14T09:07:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:07:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:07:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724396+0000", 2024-09-14T09:07:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000466028 2024-09-14T09:07:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724401+0000", 2024-09-14T09:07:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2299999999999999e-06 2024-09-14T09:07:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724412+0000", 2024-09-14T09:07:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0762e-05 2024-09-14T09:07:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724419+0000", 2024-09-14T09:07:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1010000000000003e-06 2024-09-14T09:07:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835981+0000", 2024-09-14T09:07:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111562174 2024-09-14T09:07:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836070+0000", 2024-09-14T09:07:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8839999999999996e-05 2024-09-14T09:07:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836380+0000", 2024-09-14T09:07:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031006600000000001 2024-09-14T09:07:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117319 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 317440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:07:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.710852362000001, 2024-09-14T09:07:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112467763, 2024-09-14T09:07:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.494 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117319 2024-09-14T09:07:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:07:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:07:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724846+0000", 2024-09-14T09:07:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00041155400000000001 2024-09-14T09:07:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724854+0000", 2024-09-14T09:07:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5469999999999997e-06 2024-09-14T09:07:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724862+0000", 2024-09-14T09:07:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1289999999999996e-06 2024-09-14T09:07:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724867+0000", 2024-09-14T09:07:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8720000000000001e-06 2024-09-14T09:07:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836486+0000", 2024-09-14T09:07:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111618357 2024-09-14T09:07:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836554+0000", 2024-09-14T09:07:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8378000000000002e-05 2024-09-14T09:07:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836902+0000", 2024-09-14T09:07:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034792599999999998 2024-09-14T09:07:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117321 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 321536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:52.498 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:07:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.710408038000001, 2024-09-14T09:07:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11249685600000001, 2024-09-14T09:07:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117321 2024-09-14T09:07:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:07:52.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:07:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725241+0000", 2024-09-14T09:07:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036211099999999998 2024-09-14T09:07:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725245+0000", 2024-09-14T09:07:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9179999999999996e-06 2024-09-14T09:07:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725254+0000", 2024-09-14T09:07:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0699999999999996e-06 2024-09-14T09:07:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725260+0000", 2024-09-14T09:07:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1099999999999999e-06 2024-09-14T09:07:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836935+0000", 2024-09-14T09:07:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111674997 2024-09-14T09:07:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837021+0000", 2024-09-14T09:07:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5931000000000001e-05 2024-09-14T09:07:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837375+0000", 2024-09-14T09:07:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035471899999999998 2024-09-14T09:07:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.502 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117323 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:07:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.710013813, 2024-09-14T09:07:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11250631, 2024-09-14T09:07:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117323 2024-09-14T09:07:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.503 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:07:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:07:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725941+0000", 2024-09-14T09:07:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00066866500000000004 2024-09-14T09:07:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725946+0000", 2024-09-14T09:07:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9060000000000001e-06 2024-09-14T09:07:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725959+0000", 2024-09-14T09:07:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2622e-05 2024-09-14T09:07:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725968+0000", 2024-09-14T09:07:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5660000000000007e-06 2024-09-14T09:07:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837408+0000", 2024-09-14T09:07:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111440626 2024-09-14T09:07:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837495+0000", 2024-09-14T09:07:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6654000000000001e-05 2024-09-14T09:07:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837779+0000", 2024-09-14T09:07:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000284271 2024-09-14T09:07:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117327 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 333824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:07:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.709277620999998, 2024-09-14T09:07:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112409908, 2024-09-14T09:07:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.507 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117327 2024-09-14T09:07:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:07:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:07:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726303+0000", 2024-09-14T09:07:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029388199999999998 2024-09-14T09:07:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726307+0000", 2024-09-14T09:07:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4320000000000001e-06 2024-09-14T09:07:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726317+0000", 2024-09-14T09:07:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4609999999999997e-06 2024-09-14T09:07:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726324+0000", 2024-09-14T09:07:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0480000000000001e-06 2024-09-14T09:07:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838051+0000", 2024-09-14T09:07:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111727534 2024-09-14T09:07:52.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838143+0000", 2024-09-14T09:07:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1359000000000004e-05 2024-09-14T09:07:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838419+0000", 2024-09-14T09:07:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027619199999999997 2024-09-14T09:07:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.511 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117329 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 337920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:07:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.708950862000002, 2024-09-14T09:07:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112411122, 2024-09-14T09:07:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117329 2024-09-14T09:07:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:07:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:07:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726742+0000", 2024-09-14T09:07:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040626099999999999 2024-09-14T09:07:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726748+0000", 2024-09-14T09:07:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6860000000000003e-06 2024-09-14T09:07:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726761+0000", 2024-09-14T09:07:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3599e-05 2024-09-14T09:07:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726770+0000", 2024-09-14T09:07:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5990000000000002e-06 2024-09-14T09:07:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838443+0000", 2024-09-14T09:07:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111672664 2024-09-14T09:07:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838487+0000", 2024-09-14T09:07:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4249000000000003e-05 2024-09-14T09:07:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838747+0000", 2024-09-14T09:07:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026006400000000001 2024-09-14T09:07:52.515 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117331 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 342016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:07:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.708483266999998, 2024-09-14T09:07:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112291139, 2024-09-14T09:07:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.516 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117331 2024-09-14T09:07:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:07:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:07:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727158+0000", 2024-09-14T09:07:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035434000000000003 2024-09-14T09:07:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727162+0000", 2024-09-14T09:07:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5449999999999997e-06 2024-09-14T09:07:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727174+0000", 2024-09-14T09:07:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2278e-05 2024-09-14T09:07:52.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727183+0000", 2024-09-14T09:07:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3820000000000004e-06 2024-09-14T09:07:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838788+0000", 2024-09-14T09:07:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111604748 2024-09-14T09:07:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838847+0000", 2024-09-14T09:07:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8983000000000001e-05 2024-09-14T09:07:52.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839094+0000", 2024-09-14T09:07:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024786300000000002 2024-09-14T09:07:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117333 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 346112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:07:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.708089456, 2024-09-14T09:07:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112319711, 2024-09-14T09:07:52.520 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117333 2024-09-14T09:07:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:07:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:07:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727785+0000", 2024-09-14T09:07:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000587512 2024-09-14T09:07:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727789+0000", 2024-09-14T09:07:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.352e-06 2024-09-14T09:07:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727800+0000", 2024-09-14T09:07:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1369000000000001e-05 2024-09-14T09:07:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.523 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727807+0000", 2024-09-14T09:07:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1300000000000003e-06 2024-09-14T09:07:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839137+0000", 2024-09-14T09:07:52.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111329898 2024-09-14T09:07:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839226+0000", 2024-09-14T09:07:52.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8337000000000005e-05 2024-09-14T09:07:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839517+0000", 2024-09-14T09:07:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029111300000000002 2024-09-14T09:07:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117363 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 407552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:52.529 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:07:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.701993356999999, 2024-09-14T09:07:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112191306, 2024-09-14T09:07:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117363 2024-09-14T09:07:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:07:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:07:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733666+0000", 2024-09-14T09:07:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00037284500000000001 2024-09-14T09:07:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733671+0000", 2024-09-14T09:07:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9269999999999996e-06 2024-09-14T09:07:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733681+0000", 2024-09-14T09:07:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5599999999999999e-06 2024-09-14T09:07:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733689+0000", 2024-09-14T09:07:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2649999999999996e-06 2024-09-14T09:07:52.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845028+0000", 2024-09-14T09:07:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111338933 2024-09-14T09:07:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845094+0000", 2024-09-14T09:07:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6749999999999996e-05 2024-09-14T09:07:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845485+0000", 2024-09-14T09:07:52.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00039002600000000002 2024-09-14T09:07:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117365 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 411648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:07:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 20.701583591999999, 2024-09-14T09:07:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112271685, 2024-09-14T09:07:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:52.534 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117365 2024-09-14T09:07:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:07:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:07:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:52.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734147+0000", 2024-09-14T09:07:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00044374399999999999 2024-09-14T09:07:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734152+0000", 2024-09-14T09:07:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9139999999999999e-06 2024-09-14T09:07:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734164+0000", 2024-09-14T09:07:52.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2299e-05 2024-09-14T09:07:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734172+0000", 2024-09-14T09:07:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0390000000000005e-06 2024-09-14T09:07:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845528+0000", 2024-09-14T09:07:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11135639899999999 2024-09-14T09:07:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845602+0000", 2024-09-14T09:07:52.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3957999999999999e-05 2024-09-14T09:07:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845975+0000", 2024-09-14T09:07:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00037233200000000001 2024-09-14T09:07:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:52.538 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:52.538 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:52.540 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:07:52.545 INFO:teuthology.orchestra.run.smithi183.stdout:get_pg_num_history pg_num_history pg_num_history(e134 pg_nums {1={16=1},2={19=8},3={23=32}} deleted_pools ) 2024-09-14T09:07:52.545 INFO:teuthology.orchestra.run.smithi183.stdout:Importing pgid 3.3 2024-09-14T09:07:52.545 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c036442c:::rbd_data.115e77b799c3.000000000000000a:20# 2024-09-14T09:07:52.545 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c036442c:::rbd_data.115e77b799c3.000000000000000a:head# 2024-09-14T09:07:52.546 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 20=[]:{20=[20]} 2024-09-14T09:07:52.546 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c038bd58:::rbd_data.115e9a87c1b2.0000000000000017:21# 2024-09-14T09:07:52.546 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c038bd58:::rbd_data.115e9a87c1b2.0000000000000017:head# 2024-09-14T09:07:52.546 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 21=[]:{21=[21]} 2024-09-14T09:07:52.546 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c03e445b:::rbd_data.115ee10e61a5.0000000000000015:head# 2024-09-14T09:07:52.546 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 0=[]:{} 2024-09-14T09:07:52.546 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c046d2dc:::rbd_object_map.115e63832e35.0000000000000023:head# 2024-09-14T09:07:52.546 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 0=[]:{} 2024-09-14T09:07:52.546 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c0779ca8:::rbd_data.115e2588c85c.000000000000009c:head# 2024-09-14T09:07:52.546 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 29=[]:{} 2024-09-14T09:07:52.546 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c085c20b:::rbd_object_map.115e5dc498ac.0000000000000014:head# 2024-09-14T09:07:52.546 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 0=[]:{} 2024-09-14T09:07:52.546 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c0fdc971:::rbd_data.115e2eebe49a.000000000000014a:1d# 2024-09-14T09:07:52.546 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c0fdc971:::rbd_data.115e2eebe49a.000000000000014a:head# 2024-09-14T09:07:52.547 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 1d=[]:{1d=[1d]} 2024-09-14T09:07:52.547 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c10f85e1:::rbd_data.115eb62c5c2b.0000000000000395:17# 2024-09-14T09:07:52.547 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c10f85e1:::rbd_data.115eb62c5c2b.0000000000000395:head# 2024-09-14T09:07:52.547 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 17=[]:{17=[17]} 2024-09-14T09:07:52.547 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c14df650:::rbd_data.115ed0fcd179.00000000000000e6:head# 2024-09-14T09:07:52.547 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 5=[]:{} 2024-09-14T09:07:52.547 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c16c5059:::rbd_data.115ee10e61a5.0000000000000110:head# 2024-09-14T09:07:52.547 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 0=[]:{} 2024-09-14T09:07:52.547 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c1881537:::rbd_data.115e2baf3d9a.000000000000003f:head# 2024-09-14T09:07:52.547 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 0=[]:{} 2024-09-14T09:07:52.547 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c1e37c2b:::rbd_data.115e9d49ac1.0000000000000005:b# 2024-09-14T09:07:52.547 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c1e37c2b:::rbd_data.115e9d49ac1.0000000000000005:head# 2024-09-14T09:07:52.547 INFO:teuthology.orchestra.run.smithi183.stdout:snapset b=[]:{b=[b]} 2024-09-14T09:07:52.547 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:27# 2024-09-14T09:07:52.548 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c241d7ec:::rbd_data.115e500fbcd9.0000000000000004:head# 2024-09-14T09:07:52.548 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 27=[]:{27=[27]} 2024-09-14T09:07:52.548 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c2a58dd5:::rbd_data.115e3abd3b2c.0000000000000031:head# 2024-09-14T09:07:52.548 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 8=[]:{} 2024-09-14T09:07:52.548 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c2d99c9b:::rbd_data.115eb62c5c2b.000000000000014f:17# 2024-09-14T09:07:52.548 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c2d99c9b:::rbd_data.115eb62c5c2b.000000000000014f:head# 2024-09-14T09:07:52.548 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 17=[]:{17=[17]} 2024-09-14T09:07:52.548 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c349b650:::rbd_data.115e9d49ac1.000000000000004e:b# 2024-09-14T09:07:52.548 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c349b650:::rbd_data.115e9d49ac1.000000000000004e:head# 2024-09-14T09:07:52.548 INFO:teuthology.orchestra.run.smithi183.stdout:snapset b=[]:{b=[b]} 2024-09-14T09:07:52.548 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c35ec4f9:::rbd_data.115e2588c85c.0000000000000050:29# 2024-09-14T09:07:52.548 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c35ec4f9:::rbd_data.115e2588c85c.0000000000000050:head# 2024-09-14T09:07:52.548 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 29=[]:{29=[29]} 2024-09-14T09:07:52.548 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c38815bf:::rbd_data.115e9a87c1b2.000000000000000c:21# 2024-09-14T09:07:52.549 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c38815bf:::rbd_data.115e9a87c1b2.000000000000000c:head# 2024-09-14T09:07:52.549 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 21=[]:{21=[21]} 2024-09-14T09:07:52.549 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c3b0db07:::rbd_data.115e5dc498ac.0000000000000049:14# 2024-09-14T09:07:52.549 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c3b0db07:::rbd_data.115e5dc498ac.0000000000000049:head# 2024-09-14T09:07:52.549 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 14=[]:{14=[14]} 2024-09-14T09:07:52.549 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c3ffd0ef:::rbd_data.115ec297f3f3.0000000000000052:head# 2024-09-14T09:07:52.549 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 15=[]:{} 2024-09-14T09:07:52.549 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c40097e4:::rbd_id.image_client.0-clone37:head# 2024-09-14T09:07:52.549 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 0=[]:{} 2024-09-14T09:07:52.549 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c415229f:::rbd_data.115e9a87c1b2.0000000000000016:21# 2024-09-14T09:07:52.549 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c415229f:::rbd_data.115e9a87c1b2.0000000000000016:head# 2024-09-14T09:07:52.549 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 21=[]:{21=[21]} 2024-09-14T09:07:52.549 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c44034d2:::rbd_data.115e2eebe49a.0000000000000021:1d# 2024-09-14T09:07:52.549 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c44034d2:::rbd_data.115e2eebe49a.0000000000000021:head# 2024-09-14T09:07:52.550 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 1d=[]:{1d=[1d]} 2024-09-14T09:07:52.550 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c440a843:::rbd_data.115ef5361ff8.0000000000000069:16# 2024-09-14T09:07:52.550 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c440a843:::rbd_data.115ef5361ff8.0000000000000069:head# 2024-09-14T09:07:52.550 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 16=[]:{16=[16]} 2024-09-14T09:07:52.550 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c44430d4:::rbd_data.115e77b799c3.0000000000000109:20# 2024-09-14T09:07:52.550 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c44430d4:::rbd_data.115e77b799c3.0000000000000109:head# 2024-09-14T09:07:52.550 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 20=[]:{20=[20]} 2024-09-14T09:07:52.550 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c4594dd3:::rbd_data.115e21c31fcf.0000000000000025:head# 2024-09-14T09:07:52.550 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 0=[]:{} 2024-09-14T09:07:52.550 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c4ebb483:::rbd_data.115e32e50606.0000000000000003:13# 2024-09-14T09:07:52.550 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c4ebb483:::rbd_data.115e32e50606.0000000000000003:head# 2024-09-14T09:07:52.550 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 13=[]:{13=[13]} 2024-09-14T09:07:52.550 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c4fc38e9:::rbd_data.115e2588c85c.0000000000000228:29# 2024-09-14T09:07:52.550 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c4fc38e9:::rbd_data.115e2588c85c.0000000000000228:head# 2024-09-14T09:07:52.551 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 29=[]:{29=[29]} 2024-09-14T09:07:52.551 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:52.542+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:52.552 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c4fd7e8e:::rbd_data.115e113dc8ca.000000000000000c:head# 2024-09-14T09:07:52.552 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 0=[]:{} 2024-09-14T09:07:52.552 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c508885d:::rbd_data.115eb73c120f.0000000000000016:head# 2024-09-14T09:07:52.552 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 24=[]:{} 2024-09-14T09:07:52.552 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c5320280:::rbd_object_map.115e17d15072:head# 2024-09-14T09:07:52.552 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 0=[]:{} 2024-09-14T09:07:52.552 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c541298f:::rbd_data.115ebc3c67d1.0000000000000015:head# 2024-09-14T09:07:52.553 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 0=[]:{} 2024-09-14T09:07:52.553 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c54429df:::rbd_data.115ee7a2dde0.0000000000000000:10# 2024-09-14T09:07:52.595 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c54429df:::rbd_data.115ee7a2dde0.0000000000000000:head# 2024-09-14T09:07:52.596 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 10=[]:{10=[10]} 2024-09-14T09:07:52.617 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c5644c39:::rbd_data.115e9d49ac1.000000000000000d:b# 2024-09-14T09:07:52.618 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c5644c39:::rbd_data.115e9d49ac1.000000000000000d:head# 2024-09-14T09:07:52.618 INFO:teuthology.orchestra.run.smithi183.stdout:snapset b=[]:{b=[b]} 2024-09-14T09:07:52.619 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c589307d:::rbd_data.115e96ceebb2.0000000000000004:2a# 2024-09-14T09:07:52.636 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c589307d:::rbd_data.115e96ceebb2.0000000000000004:head# 2024-09-14T09:07:52.636 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 2a=[]:{2a=[2a]} 2024-09-14T09:07:52.637 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c5b07644:::rbd_data.115e2588c85c.000000000000004c:head# 2024-09-14T09:07:52.637 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 29=[]:{} 2024-09-14T09:07:52.637 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c5caae6c:::rbd_data.115e2588c85c.000000000000008d:29# 2024-09-14T09:07:52.638 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c5caae6c:::rbd_data.115e2588c85c.000000000000008d:head# 2024-09-14T09:07:52.638 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 29=[]:{29=[29]} 2024-09-14T09:07:52.639 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c5cf7412:::rbd_object_map.115e4c71d6e8:head# 2024-09-14T09:07:52.639 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 0=[]:{} 2024-09-14T09:07:52.640 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c5d5ef46:::rbd_data.115ebc3c67d1.000000000000000d:head# 2024-09-14T09:07:52.640 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:52.639+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:52.641 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 0=[]:{} 2024-09-14T09:07:52.647 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c5e39d02:::rbd_data.115ec297f3f3.000000000000006a:15# 2024-09-14T09:07:52.650 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c5e39d02:::rbd_data.115ec297f3f3.000000000000006a:head# 2024-09-14T09:07:52.650 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 15=[]:{15=[15]} 2024-09-14T09:07:52.651 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c62e0486:::rbd_data.115e9e133b92.0000000000000000:head# 2024-09-14T09:07:52.653 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 0=[]:{} 2024-09-14T09:07:52.694 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c67eb973:::rbd_data.115ee10e61a5.000000000000000a:head# 2024-09-14T09:07:52.694 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 0=[]:{} 2024-09-14T09:07:52.695 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:52.695 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:52.695 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:07:52.695 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:52.695 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c696d26d:::rbd_data.115ec19646f4.0000000000000078:2b# 2024-09-14T09:07:52.697 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c696d26d:::rbd_data.115ec19646f4.0000000000000078:head# 2024-09-14T09:07:52.697 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 2b=[]:{2b=[2b]} 2024-09-14T09:07:52.698 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c6d19d67:::rbd_data.115e889d4809.000000000000000c:a# 2024-09-14T09:07:52.699 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c6d19d67:::rbd_data.115e889d4809.000000000000000c:head# 2024-09-14T09:07:52.699 INFO:teuthology.orchestra.run.smithi183.stdout:snapset a=[]:{a=[a]} 2024-09-14T09:07:52.701 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c6d67aab:::rbd_header.115e7770ddc5:head# 2024-09-14T09:07:52.701 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 0=[]:{} 2024-09-14T09:07:52.701 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c7113cc8:::rbd_header.115ea8d131ff:head# 2024-09-14T09:07:52.702 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 0=[]:{} 2024-09-14T09:07:52.702 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c7408ca1:::rbd_data.115eb62c5c2b.0000000000000240:17# 2024-09-14T09:07:52.704 INFO:teuthology.orchestra.run.smithi183.stdout:Write #3:c7408ca1:::rbd_data.115eb62c5c2b.0000000000000240:head# 2024-09-14T09:07:52.704 INFO:teuthology.orchestra.run.smithi183.stdout:snapset 17=[]:{17=[17]} 2024-09-14T09:07:52.705 INFO:teuthology.orchestra.run.smithi183.stdout:write_pg epoch 134 info 3.3( v 134'569 (0'0,134'569] local-lis/les=86/87 n=72 ec=23/23 lis/c=86/86 les/c/f=87/87/0 sis=86) 2024-09-14T09:07:52.708 INFO:teuthology.orchestra.run.smithi183.stdout:Import successful 2024-09-14T09:07:52.710 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:07:52.741 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:52.739+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:52.843 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:52.843+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:52.878 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:52.878 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:52.878 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:07:52.878 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:07:52.878 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:52.892 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:07:52.943 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:52.943+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:53.019 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:53.019 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:07:53.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:07:53.019 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:07:53.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.019 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105615 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 382976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.019 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:07:53.020 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.157815706000001, 2024-09-14T09:07:53.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117270900000001, 2024-09-14T09:07:53.020 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.020 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.020 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.020 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.020 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.020 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.020 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105615 2024-09-14T09:07:53.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.020 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:07:53.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:07:53.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864368+0000", 2024-09-14T09:07:53.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.043e-06 2024-09-14T09:07:53.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864373+0000", 2024-09-14T09:07:53.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4669999999999998e-06 2024-09-14T09:07:53.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864387+0000", 2024-09-14T09:07:53.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3876000000000001e-05 2024-09-14T09:07:53.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864397+0000", 2024-09-14T09:07:53.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0592e-05 2024-09-14T09:07:53.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866069+0000", 2024-09-14T09:07:53.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016714410000000001 2024-09-14T09:07:53.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866076+0000", 2024-09-14T09:07:53.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:07:53.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.075854+0000", 2024-09-14T09:07:53.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.209778501 2024-09-14T09:07:53.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.075940+0000", 2024-09-14T09:07:53.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5357000000000003e-05 2024-09-14T09:07:53.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.076094+0000", 2024-09-14T09:07:53.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000154745 2024-09-14T09:07:53.024 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.024 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.025 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.025 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105629 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 411648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.025 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:07:53.025 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.155661984000002, 2024-09-14T09:07:53.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2115591910000001, 2024-09-14T09:07:53.025 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.025 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.025 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.025 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.026 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.026 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.026 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105629 2024-09-14T09:07:53.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.026 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:07:53.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:07:53.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866522+0000", 2024-09-14T09:07:53.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6899999999999996e-07 2024-09-14T09:07:53.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866526+0000", 2024-09-14T09:07:53.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0110000000000002e-06 2024-09-14T09:07:53.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866539+0000", 2024-09-14T09:07:53.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3013000000000001e-05 2024-09-14T09:07:53.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866549+0000", 2024-09-14T09:07:53.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8490000000000001e-06 2024-09-14T09:07:53.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866650+0000", 2024-09-14T09:07:53.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101054 2024-09-14T09:07:53.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866664+0000", 2024-09-14T09:07:53.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3805000000000001e-05 2024-09-14T09:07:53.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.077849+0000", 2024-09-14T09:07:53.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2111855419999999 2024-09-14T09:07:53.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.077945+0000", 2024-09-14T09:07:53.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6086000000000001e-05 2024-09-14T09:07:53.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078080+0000", 2024-09-14T09:07:53.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000134862 2024-09-14T09:07:53.030 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.030 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.030 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.031 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105631 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 415744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.031 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:07:53.031 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.155602765000001, 2024-09-14T09:07:53.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119463509999999, 2024-09-14T09:07:53.031 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.031 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.031 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.031 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.031 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.031 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.031 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105631 2024-09-14T09:07:53.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.031 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:07:53.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:07:53.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866581+0000", 2024-09-14T09:07:53.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.879999999999999e-07 2024-09-14T09:07:53.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866585+0000", 2024-09-14T09:07:53.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0239999999999999e-06 2024-09-14T09:07:53.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866598+0000", 2024-09-14T09:07:53.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2862e-05 2024-09-14T09:07:53.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866608+0000", 2024-09-14T09:07:53.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9890000000000004e-06 2024-09-14T09:07:53.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866708+0000", 2024-09-14T09:07:53.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9605999999999995e-05 2024-09-14T09:07:53.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866720+0000", 2024-09-14T09:07:53.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.261e-05 2024-09-14T09:07:53.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078137+0000", 2024-09-14T09:07:53.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211416571 2024-09-14T09:07:53.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078293+0000", 2024-09-14T09:07:53.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000156 2024-09-14T09:07:53.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078527+0000", 2024-09-14T09:07:53.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000233701 2024-09-14T09:07:53.036 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.036 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.036 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.036 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105633 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.036 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:07:53.036 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.155543718000001, 2024-09-14T09:07:53.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211941734, 2024-09-14T09:07:53.036 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.036 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.036 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:53.036 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.037 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.037 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.037 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105633 2024-09-14T09:07:53.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.037 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:07:53.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:07:53.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866640+0000", 2024-09-14T09:07:53.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9300000000000006e-07 2024-09-14T09:07:53.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866656+0000", 2024-09-14T09:07:53.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5291e-05 2024-09-14T09:07:53.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866669+0000", 2024-09-14T09:07:53.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2918000000000001e-05 2024-09-14T09:07:53.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866678+0000", 2024-09-14T09:07:53.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8369999999999992e-06 2024-09-14T09:07:53.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866771+0000", 2024-09-14T09:07:53.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2250999999999995e-05 2024-09-14T09:07:53.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866784+0000", 2024-09-14T09:07:53.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2991e-05 2024-09-14T09:07:53.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078566+0000", 2024-09-14T09:07:53.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117822570000001 2024-09-14T09:07:53.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078581+0000", 2024-09-14T09:07:53.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5196e-05 2024-09-14T09:07:53.041 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.041 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.041 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.041 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105635 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.041 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:07:53.041 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.155489890999998, 2024-09-14T09:07:53.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119313250000001, 2024-09-14T09:07:53.041 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.041 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.042 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:53.042 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.042 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.042 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.042 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105635 2024-09-14T09:07:53.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.042 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:07:53.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:07:53.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866694+0000", 2024-09-14T09:07:53.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0210000000000001e-06 2024-09-14T09:07:53.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866698+0000", 2024-09-14T09:07:53.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.765e-06 2024-09-14T09:07:53.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866710+0000", 2024-09-14T09:07:53.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2320000000000001e-05 2024-09-14T09:07:53.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866722+0000", 2024-09-14T09:07:53.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2133000000000001e-05 2024-09-14T09:07:53.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866830+0000", 2024-09-14T09:07:53.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000107668 2024-09-14T09:07:53.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866843+0000", 2024-09-14T09:07:53.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2991e-05 2024-09-14T09:07:53.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078616+0000", 2024-09-14T09:07:53.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117732459999999 2024-09-14T09:07:53.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078624+0000", 2024-09-14T09:07:53.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1810000000000001e-06 2024-09-14T09:07:53.047 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.047 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.047 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.047 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105636 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.047 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:07:53.047 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.155421995000001, 2024-09-14T09:07:53.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119085119999999, 2024-09-14T09:07:53.047 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.047 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.047 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:53.047 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.047 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.048 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.048 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105636 2024-09-14T09:07:53.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.048 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:07:53.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:07:53.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866762+0000", 2024-09-14T09:07:53.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4600000000000003e-07 2024-09-14T09:07:53.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866766+0000", 2024-09-14T09:07:53.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:07:53.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866779+0000", 2024-09-14T09:07:53.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3101999999999999e-05 2024-09-14T09:07:53.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866789+0000", 2024-09-14T09:07:53.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8039999999999996e-06 2024-09-14T09:07:53.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866889+0000", 2024-09-14T09:07:53.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100147 2024-09-14T09:07:53.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866902+0000", 2024-09-14T09:07:53.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3212e-05 2024-09-14T09:07:53.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078659+0000", 2024-09-14T09:07:53.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211756362 2024-09-14T09:07:53.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078670+0000", 2024-09-14T09:07:53.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0956e-05 2024-09-14T09:07:53.052 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.052 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.052 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.052 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105639 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.052 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:07:53.052 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.155363120000001, 2024-09-14T09:07:53.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118829259999999, 2024-09-14T09:07:53.052 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.052 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.052 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:53.052 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.053 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.053 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.053 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105639 2024-09-14T09:07:53.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.053 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:07:53.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:07:53.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866821+0000", 2024-09-14T09:07:53.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9800000000000002e-07 2024-09-14T09:07:53.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866825+0000", 2024-09-14T09:07:53.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9269999999999998e-06 2024-09-14T09:07:53.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866838+0000", 2024-09-14T09:07:53.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3040000000000001e-05 2024-09-14T09:07:53.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866848+0000", 2024-09-14T09:07:53.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7929999999999992e-06 2024-09-14T09:07:53.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866948+0000", 2024-09-14T09:07:53.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100337 2024-09-14T09:07:53.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866961+0000", 2024-09-14T09:07:53.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3106e-05 2024-09-14T09:07:53.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078693+0000", 2024-09-14T09:07:53.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117320380000001 2024-09-14T09:07:53.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078703+0000", 2024-09-14T09:07:53.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6870000000000006e-06 2024-09-14T09:07:53.056 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.057 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.057 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.057 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105641 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.057 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:07:53.057 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.155304255000001, 2024-09-14T09:07:53.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118727169999999, 2024-09-14T09:07:53.057 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.057 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.057 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:53.057 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.057 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.058 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.058 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105641 2024-09-14T09:07:53.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.058 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:07:53.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:07:53.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866880+0000", 2024-09-14T09:07:53.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7699999999999992e-07 2024-09-14T09:07:53.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866884+0000", 2024-09-14T09:07:53.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9480000000000001e-06 2024-09-14T09:07:53.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866897+0000", 2024-09-14T09:07:53.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2832e-05 2024-09-14T09:07:53.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866906+0000", 2024-09-14T09:07:53.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7869999999999996e-06 2024-09-14T09:07:53.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867007+0000", 2024-09-14T09:07:53.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010103900000000001 2024-09-14T09:07:53.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867020+0000", 2024-09-14T09:07:53.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2896e-05 2024-09-14T09:07:53.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078739+0000", 2024-09-14T09:07:53.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211718305 2024-09-14T09:07:53.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078752+0000", 2024-09-14T09:07:53.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2933e-05 2024-09-14T09:07:53.061 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.061 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.062 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.062 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105643 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.062 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:07:53.062 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.155245696000001, 2024-09-14T09:07:53.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211858353, 2024-09-14T09:07:53.062 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.062 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.062 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:53.062 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.062 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.062 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.063 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105643 2024-09-14T09:07:53.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.063 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:07:53.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:07:53.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866938+0000", 2024-09-14T09:07:53.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5600000000000004e-07 2024-09-14T09:07:53.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866942+0000", 2024-09-14T09:07:53.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0350000000000003e-06 2024-09-14T09:07:53.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866955+0000", 2024-09-14T09:07:53.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3097e-05 2024-09-14T09:07:53.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866965+0000", 2024-09-14T09:07:53.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8179999999999999e-06 2024-09-14T09:07:53.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867067+0000", 2024-09-14T09:07:53.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010128699999999999 2024-09-14T09:07:53.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867080+0000", 2024-09-14T09:07:53.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3089e-05 2024-09-14T09:07:53.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078786+0000", 2024-09-14T09:07:53.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117065250000001 2024-09-14T09:07:53.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078796+0000", 2024-09-14T09:07:53.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5459999999999997e-06 2024-09-14T09:07:53.066 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.067 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.067 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.067 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105645 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.067 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:07:53.067 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.155185300999999, 2024-09-14T09:07:53.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211848641, 2024-09-14T09:07:53.067 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.067 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.067 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:53.067 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.067 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.067 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.068 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105645 2024-09-14T09:07:53.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.068 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:07:53.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:07:53.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866999+0000", 2024-09-14T09:07:53.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7999999999999993e-07 2024-09-14T09:07:53.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867003+0000", 2024-09-14T09:07:53.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0090000000000001e-06 2024-09-14T09:07:53.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867016+0000", 2024-09-14T09:07:53.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3304e-05 2024-09-14T09:07:53.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867026+0000", 2024-09-14T09:07:53.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5889999999999992e-06 2024-09-14T09:07:53.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867128+0000", 2024-09-14T09:07:53.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000102101 2024-09-14T09:07:53.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867141+0000", 2024-09-14T09:07:53.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2903e-05 2024-09-14T09:07:53.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078833+0000", 2024-09-14T09:07:53.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116920959999999 2024-09-14T09:07:53.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078846+0000", 2024-09-14T09:07:53.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3658999999999999e-05 2024-09-14T09:07:53.071 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.071 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.071 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.072 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105647 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.072 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:07:53.072 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.155126783, 2024-09-14T09:07:53.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211830097, 2024-09-14T09:07:53.072 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.072 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.072 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:53.072 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.072 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.072 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.072 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105647 2024-09-14T09:07:53.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.073 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:07:53.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:07:53.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867057+0000", 2024-09-14T09:07:53.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.95e-07 2024-09-14T09:07:53.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867061+0000", 2024-09-14T09:07:53.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9530000000000001e-06 2024-09-14T09:07:53.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867074+0000", 2024-09-14T09:07:53.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3061000000000001e-05 2024-09-14T09:07:53.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867086+0000", 2024-09-14T09:07:53.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1460999999999999e-05 2024-09-14T09:07:53.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867186+0000", 2024-09-14T09:07:53.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100181 2024-09-14T09:07:53.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867199+0000", 2024-09-14T09:07:53.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-05 2024-09-14T09:07:53.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078874+0000", 2024-09-14T09:07:53.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211674683 2024-09-14T09:07:53.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078886+0000", 2024-09-14T09:07:53.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2853e-05 2024-09-14T09:07:53.076 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.076 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.076 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.077 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105648 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.077 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:07:53.077 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.155065297, 2024-09-14T09:07:53.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118073570000001, 2024-09-14T09:07:53.077 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.077 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.077 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:53.077 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.077 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.077 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.077 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105648 2024-09-14T09:07:53.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.077 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:07:53.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:07:53.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867119+0000", 2024-09-14T09:07:53.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8400000000000002e-07 2024-09-14T09:07:53.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.079 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.079 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.079 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867123+0000", 2024-09-14T09:07:53.079 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9639999999999997e-06 2024-09-14T09:07:53.079 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.079 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.079 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.079 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867136+0000", 2024-09-14T09:07:53.079 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.289e-05 2024-09-14T09:07:53.079 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.079 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.079 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.079 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867145+0000", 2024-09-14T09:07:53.080 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8919999999999995e-06 2024-09-14T09:07:53.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.080 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.080 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.080 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867245+0000", 2024-09-14T09:07:53.080 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9282000000000003e-05 2024-09-14T09:07:53.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.080 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.080 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.081 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867258+0000", 2024-09-14T09:07:53.081 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2935000000000001e-05 2024-09-14T09:07:53.081 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.081 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.081 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.081 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078910+0000", 2024-09-14T09:07:53.081 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211652352 2024-09-14T09:07:53.081 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.081 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.081 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.081 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078925+0000", 2024-09-14T09:07:53.081 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5058000000000001e-05 2024-09-14T09:07:53.081 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.081 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.082 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.082 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.082 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105652 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.082 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:07:53.082 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.155007277999999, 2024-09-14T09:07:53.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211775737, 2024-09-14T09:07:53.082 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.082 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.082 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:53.082 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.082 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.082 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.083 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105652 2024-09-14T09:07:53.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.083 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:07:53.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:07:53.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867177+0000", 2024-09-14T09:07:53.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9100000000000002e-07 2024-09-14T09:07:53.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.084 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867181+0000", 2024-09-14T09:07:53.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0029999999999996e-06 2024-09-14T09:07:53.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.084 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867193+0000", 2024-09-14T09:07:53.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2675e-05 2024-09-14T09:07:53.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867203+0000", 2024-09-14T09:07:53.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5599999999999999e-06 2024-09-14T09:07:53.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867306+0000", 2024-09-14T09:07:53.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010330100000000001 2024-09-14T09:07:53.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867319+0000", 2024-09-14T09:07:53.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3129000000000001e-05 2024-09-14T09:07:53.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.086 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.086 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078944+0000", 2024-09-14T09:07:53.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116247520000001 2024-09-14T09:07:53.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.086 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.086 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078952+0000", 2024-09-14T09:07:53.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4259999999999997e-06 2024-09-14T09:07:53.086 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.086 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.086 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.087 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105653 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.087 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:07:53.087 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.154948608000002, 2024-09-14T09:07:53.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211774895, 2024-09-14T09:07:53.087 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.087 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.087 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:53.087 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.087 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.087 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.087 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105653 2024-09-14T09:07:53.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.088 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:07:53.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:07:53.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867235+0000", 2024-09-14T09:07:53.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9400000000000004e-07 2024-09-14T09:07:53.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867239+0000", 2024-09-14T09:07:53.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9380000000000002e-06 2024-09-14T09:07:53.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867252+0000", 2024-09-14T09:07:53.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.31e-05 2024-09-14T09:07:53.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867264+0000", 2024-09-14T09:07:53.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1275e-05 2024-09-14T09:07:53.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867367+0000", 2024-09-14T09:07:53.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103755 2024-09-14T09:07:53.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867382+0000", 2024-09-14T09:07:53.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4724e-05 2024-09-14T09:07:53.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078995+0000", 2024-09-14T09:07:53.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116132669999999 2024-09-14T09:07:53.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079009+0000", 2024-09-14T09:07:53.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3942e-05 2024-09-14T09:07:53.091 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.091 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.091 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.092 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105654 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.092 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:07:53.092 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.154887552000002, 2024-09-14T09:07:53.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117364070000001, 2024-09-14T09:07:53.092 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.092 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.092 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:53.092 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.092 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.092 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.092 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105654 2024-09-14T09:07:53.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.093 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:07:53.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:07:53.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867296+0000", 2024-09-14T09:07:53.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2399999999999996e-07 2024-09-14T09:07:53.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867300+0000", 2024-09-14T09:07:53.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8689999999999997e-06 2024-09-14T09:07:53.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867313+0000", 2024-09-14T09:07:53.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2660000000000001e-05 2024-09-14T09:07:53.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867322+0000", 2024-09-14T09:07:53.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.533e-06 2024-09-14T09:07:53.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867429+0000", 2024-09-14T09:07:53.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106566 2024-09-14T09:07:53.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867442+0000", 2024-09-14T09:07:53.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3244000000000001e-05 2024-09-14T09:07:53.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079024+0000", 2024-09-14T09:07:53.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211581813 2024-09-14T09:07:53.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079032+0000", 2024-09-14T09:07:53.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7980000000000006e-06 2024-09-14T09:07:53.096 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.096 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.097 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.097 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105657 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.097 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:07:53.097 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.154826378999999, 2024-09-14T09:07:53.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211700657, 2024-09-14T09:07:53.097 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.097 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.097 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:53.097 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.097 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.097 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.098 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105657 2024-09-14T09:07:53.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.098 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:07:53.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:07:53.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867358+0000", 2024-09-14T09:07:53.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9700000000000005e-07 2024-09-14T09:07:53.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867367+0000", 2024-09-14T09:07:53.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8919999999999995e-06 2024-09-14T09:07:53.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867383+0000", 2024-09-14T09:07:53.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.579e-05 2024-09-14T09:07:53.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867393+0000", 2024-09-14T09:07:53.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3070000000000008e-06 2024-09-14T09:07:53.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867490+0000", 2024-09-14T09:07:53.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7703999999999998e-05 2024-09-14T09:07:53.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867503+0000", 2024-09-14T09:07:53.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2849e-05 2024-09-14T09:07:53.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079050+0000", 2024-09-14T09:07:53.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211546899 2024-09-14T09:07:53.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079057+0000", 2024-09-14T09:07:53.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3189999999999997e-06 2024-09-14T09:07:53.101 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.101 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.101 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.102 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105658 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.102 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:07:53.102 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.154763995, 2024-09-14T09:07:53.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211691217, 2024-09-14T09:07:53.102 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.102 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.102 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:53.102 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.102 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.102 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.102 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105658 2024-09-14T09:07:53.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.103 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:07:53.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:07:53.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867420+0000", 2024-09-14T09:07:53.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5000000000000001e-07 2024-09-14T09:07:53.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867424+0000", 2024-09-14T09:07:53.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8469999999999998e-06 2024-09-14T09:07:53.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867437+0000", 2024-09-14T09:07:53.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3156000000000001e-05 2024-09-14T09:07:53.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867448+0000", 2024-09-14T09:07:53.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1185e-05 2024-09-14T09:07:53.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867552+0000", 2024-09-14T09:07:53.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103362 2024-09-14T09:07:53.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867565+0000", 2024-09-14T09:07:53.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3018e-05 2024-09-14T09:07:53.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079092+0000", 2024-09-14T09:07:53.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211527013 2024-09-14T09:07:53.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079110+0000", 2024-09-14T09:07:53.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8686e-05 2024-09-14T09:07:53.106 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.106 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.106 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.106 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105662 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.107 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:07:53.107 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.154702929999999, 2024-09-14T09:07:53.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211652226, 2024-09-14T09:07:53.107 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.107 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.107 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:53.107 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.107 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.107 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.107 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105662 2024-09-14T09:07:53.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.107 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:07:53.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:07:53.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867481+0000", 2024-09-14T09:07:53.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1800000000000004e-07 2024-09-14T09:07:53.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867485+0000", 2024-09-14T09:07:53.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9199999999999997e-06 2024-09-14T09:07:53.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867498+0000", 2024-09-14T09:07:53.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2896e-05 2024-09-14T09:07:53.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867509+0000", 2024-09-14T09:07:53.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1202e-05 2024-09-14T09:07:53.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867611+0000", 2024-09-14T09:07:53.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010166999999999999 2024-09-14T09:07:53.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867624+0000", 2024-09-14T09:07:53.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2996000000000001e-05 2024-09-14T09:07:53.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079125+0000", 2024-09-14T09:07:53.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2115009299999999 2024-09-14T09:07:53.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079132+0000", 2024-09-14T09:07:53.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6939999999999995e-06 2024-09-14T09:07:53.111 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.111 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.111 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.111 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105663 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.111 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:07:53.112 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.154641879, 2024-09-14T09:07:53.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116218169999999, 2024-09-14T09:07:53.112 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.112 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.112 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:53.112 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.112 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.112 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.112 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105663 2024-09-14T09:07:53.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.112 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:07:53.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:07:53.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867542+0000", 2024-09-14T09:07:53.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1399999999999995e-07 2024-09-14T09:07:53.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867546+0000", 2024-09-14T09:07:53.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7900000000000001e-06 2024-09-14T09:07:53.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867559+0000", 2024-09-14T09:07:53.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2805e-05 2024-09-14T09:07:53.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867568+0000", 2024-09-14T09:07:53.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-09-14T09:07:53.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867669+0000", 2024-09-14T09:07:53.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100655 2024-09-14T09:07:53.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867682+0000", 2024-09-14T09:07:53.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3013000000000001e-05 2024-09-14T09:07:53.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079151+0000", 2024-09-14T09:07:53.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2114689089999999 2024-09-14T09:07:53.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079163+0000", 2024-09-14T09:07:53.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1962e-05 2024-09-14T09:07:53.116 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.116 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.116 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.117 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105665 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:53.117 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:07:53.117 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.154583248000002, 2024-09-14T09:07:53.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211584848, 2024-09-14T09:07:53.117 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.117 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.117 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:53.117 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.117 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.117 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.117 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105665 2024-09-14T09:07:53.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.117 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:07:53.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:07:53.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867601+0000", 2024-09-14T09:07:53.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.569999999999999e-07 2024-09-14T09:07:53.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867605+0000", 2024-09-14T09:07:53.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8589999999999998e-06 2024-09-14T09:07:53.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867617+0000", 2024-09-14T09:07:53.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2833000000000001e-05 2024-09-14T09:07:53.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867627+0000", 2024-09-14T09:07:53.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5219999999999996e-06 2024-09-14T09:07:53.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867728+0000", 2024-09-14T09:07:53.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010057500000000001 2024-09-14T09:07:53.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:53.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867740+0000", 2024-09-14T09:07:53.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2744e-05 2024-09-14T09:07:53.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079178+0000", 2024-09-14T09:07:53.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211437552 2024-09-14T09:07:53.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079185+0000", 2024-09-14T09:07:53.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8059999999999997e-06 2024-09-14T09:07:53.121 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.121 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.121 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.121 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.121 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.121 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:53.123 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:07:53.123 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:53.126+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:53.224 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:53.226+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:53.263 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /var/log/ceph/exp.3.3.4 2024-09-14T09:07:53.306 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:53.306 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:53.306 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:07:53.306 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:53.321 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:07:53.325 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:53.323+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:53.426 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:53.426+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:53.458 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:07:53.491 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:53.492 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:53.492 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:07:53.492 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:07:53.492 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:53.506 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:07:53.553 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:53.554+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:53.613 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:53.613 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:53.627 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:53.627+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:53.630 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:53.631 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:53.647 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:53.647 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:53.664 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:53.664 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:53.665 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:07:53.681 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:53.681 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:07:53.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:07:53.682 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:07:53.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.682 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65139 3.a 3:54c90b7d:::rbd_data.115e21c31fcf.0000000000000008:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:07:53.682 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:07:53.682 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 92.305119344000005, 2024-09-14T09:07:53.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14429895700000001, 2024-09-14T09:07:53.682 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.682 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.682 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:53.682 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.682 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.682 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.682 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65139 2024-09-14T09:07:53.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.683 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:07:53.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:07:53.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379715+0000", 2024-09-14T09:07:53.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.576e-06 2024-09-14T09:07:53.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379724+0000", 2024-09-14T09:07:53.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1779999999999992e-06 2024-09-14T09:07:53.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379746+0000", 2024-09-14T09:07:53.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1661000000000001e-05 2024-09-14T09:07:53.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379759+0000", 2024-09-14T09:07:53.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.324e-05 2024-09-14T09:07:53.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379882+0000", 2024-09-14T09:07:53.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012239400000000001 2024-09-14T09:07:53.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.380023+0000", 2024-09-14T09:07:53.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000141402 2024-09-14T09:07:53.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:53.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.391553+0000", 2024-09-14T09:07:53.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011530197000000001 2024-09-14T09:07:53.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:53.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523927+0000", 2024-09-14T09:07:53.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.132373935 2024-09-14T09:07:53.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:53.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523941+0000", 2024-09-14T09:07:53.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3326000000000001e-05 2024-09-14T09:07:53.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:53.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523977+0000", 2024-09-14T09:07:53.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6185e-05 2024-09-14T09:07:53.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.524012+0000", 2024-09-14T09:07:53.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4863000000000001e-05 2024-09-14T09:07:53.687 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.687 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.687 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.688 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107606 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 14336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:53.688 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:07:53.688 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.278662443000002, 2024-09-14T09:07:53.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14293513799999999, 2024-09-14T09:07:53.688 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.688 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.688 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.688 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.688 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.688 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.688 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107606 2024-09-14T09:07:53.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.689 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:07:53.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:07:53.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:07:53.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1600000000000001e-07 2024-09-14T09:07:53.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406174+0000", 2024-09-14T09:07:53.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4199999999999999e-06 2024-09-14T09:07:53.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406188+0000", 2024-09-14T09:07:53.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4114000000000001e-05 2024-09-14T09:07:53.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406198+0000", 2024-09-14T09:07:53.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.041e-05 2024-09-14T09:07:53.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.548700+0000", 2024-09-14T09:07:53.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14250141299999999 2024-09-14T09:07:53.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.548783+0000", 2024-09-14T09:07:53.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3578999999999994e-05 2024-09-14T09:07:53.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549105+0000", 2024-09-14T09:07:53.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032158600000000001 2024-09-14T09:07:53.692 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.692 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.692 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.692 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107608 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:53.692 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:07:53.692 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.278596406999998, 2024-09-14T09:07:53.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14603042799999999, 2024-09-14T09:07:53.692 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.692 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.693 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.693 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.693 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.693 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.693 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107608 2024-09-14T09:07:53.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.693 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:07:53.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:07:53.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:07:53.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1200000000000002e-07 2024-09-14T09:07:53.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406241+0000", 2024-09-14T09:07:53.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7539999999999997e-06 2024-09-14T09:07:53.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406275+0000", 2024-09-14T09:07:53.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3512e-05 2024-09-14T09:07:53.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406322+0000", 2024-09-14T09:07:53.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6764999999999997e-05 2024-09-14T09:07:53.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549149+0000", 2024-09-14T09:07:53.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14282725499999999 2024-09-14T09:07:53.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549198+0000", 2024-09-14T09:07:53.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9599999999999999e-05 2024-09-14T09:07:53.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552266+0000", 2024-09-14T09:07:53.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030677299999999998 2024-09-14T09:07:53.696 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.696 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.696 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.697 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107610 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 22528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:53.697 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:07:53.697 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.278474532000001, 2024-09-14T09:07:53.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146437919, 2024-09-14T09:07:53.697 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.697 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.697 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.697 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.697 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.697 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.697 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107610 2024-09-14T09:07:53.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.697 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:07:53.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:07:53.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:07:53.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.02e-07 2024-09-14T09:07:53.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406363+0000", 2024-09-14T09:07:53.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8099999999999997e-06 2024-09-14T09:07:53.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406390+0000", 2024-09-14T09:07:53.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7004000000000001e-05 2024-09-14T09:07:53.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406404+0000", 2024-09-14T09:07:53.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3682e-05 2024-09-14T09:07:53.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552335+0000", 2024-09-14T09:07:53.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14593077800000001 2024-09-14T09:07:53.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552431+0000", 2024-09-14T09:07:53.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6086000000000001e-05 2024-09-14T09:07:53.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552795+0000", 2024-09-14T09:07:53.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000364657 2024-09-14T09:07:53.701 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.701 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.701 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.701 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107612 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 26624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:53.701 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:07:53.701 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.278409269000001, 2024-09-14T09:07:53.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146762061, 2024-09-14T09:07:53.701 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.701 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.701 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.702 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.702 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.702 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.702 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107612 2024-09-14T09:07:53.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.702 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:07:53.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:07:53.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406424+0000", 2024-09-14T09:07:53.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2600000000000002e-07 2024-09-14T09:07:53.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406451+0000", 2024-09-14T09:07:53.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7250000000000002e-05 2024-09-14T09:07:53.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406487+0000", 2024-09-14T09:07:53.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6086e-05 2024-09-14T09:07:53.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406498+0000", 2024-09-14T09:07:53.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0858e-05 2024-09-14T09:07:53.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552821+0000", 2024-09-14T09:07:53.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14632319099999999 2024-09-14T09:07:53.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552875+0000", 2024-09-14T09:07:53.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4262999999999999e-05 2024-09-14T09:07:53.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553185+0000", 2024-09-14T09:07:53.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000309687 2024-09-14T09:07:53.705 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.705 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.705 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.705 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107614 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:53.706 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:07:53.706 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.278321734999999, 2024-09-14T09:07:53.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14737121, 2024-09-14T09:07:53.706 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.706 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.706 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.706 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.706 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.706 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.706 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107614 2024-09-14T09:07:53.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.706 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:07:53.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:07:53.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406511+0000", 2024-09-14T09:07:53.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1799999999999995e-07 2024-09-14T09:07:53.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406514+0000", 2024-09-14T09:07:53.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8660000000000002e-06 2024-09-14T09:07:53.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406523+0000", 2024-09-14T09:07:53.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1940000000000004e-06 2024-09-14T09:07:53.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406531+0000", 2024-09-14T09:07:53.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3159999999999997e-06 2024-09-14T09:07:53.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553233+0000", 2024-09-14T09:07:53.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14670117899999999 2024-09-14T09:07:53.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553334+0000", 2024-09-14T09:07:53.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100982 2024-09-14T09:07:53.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553882+0000", 2024-09-14T09:07:53.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00054805499999999998 2024-09-14T09:07:53.709 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.710 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.710 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.710 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107616 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 34816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:53.710 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:07:53.710 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.2782901, 2024-09-14T09:07:53.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14780390500000001, 2024-09-14T09:07:53.710 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.710 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.710 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.710 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.710 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.710 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.711 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107616 2024-09-14T09:07:53.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.711 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:07:53.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:07:53.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406543+0000", 2024-09-14T09:07:53.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0200000000000001e-07 2024-09-14T09:07:53.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406545+0000", 2024-09-14T09:07:53.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3360000000000002e-06 2024-09-14T09:07:53.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406552+0000", 2024-09-14T09:07:53.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9779999999999999e-06 2024-09-14T09:07:53.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406559+0000", 2024-09-14T09:07:53.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6919999999999996e-06 2024-09-14T09:07:53.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553917+0000", 2024-09-14T09:07:53.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14735791600000001 2024-09-14T09:07:53.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554033+0000", 2024-09-14T09:07:53.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116047 2024-09-14T09:07:53.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554346+0000", 2024-09-14T09:07:53.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031323400000000002 2024-09-14T09:07:53.714 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.714 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.714 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.714 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107618 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 38912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:53.714 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:07:53.714 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.2782613, 2024-09-14T09:07:53.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14816009599999999, 2024-09-14T09:07:53.714 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.715 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.715 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.715 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.715 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.715 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.715 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107618 2024-09-14T09:07:53.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.715 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:07:53.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:07:53.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:07:53.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9999999999999998e-07 2024-09-14T09:07:53.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406574+0000", 2024-09-14T09:07:53.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2560000000000001e-06 2024-09-14T09:07:53.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406581+0000", 2024-09-14T09:07:53.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1539999999999996e-06 2024-09-14T09:07:53.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406586+0000", 2024-09-14T09:07:53.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.904e-06 2024-09-14T09:07:53.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554385+0000", 2024-09-14T09:07:53.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.147799443 2024-09-14T09:07:53.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554441+0000", 2024-09-14T09:07:53.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5921000000000002e-05 2024-09-14T09:07:53.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554731+0000", 2024-09-14T09:07:53.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028991800000000001 2024-09-14T09:07:53.718 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.718 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.718 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.719 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107620 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:53.719 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:07:53.719 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.278239517999999, 2024-09-14T09:07:53.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.148512643, 2024-09-14T09:07:53.719 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.719 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.719 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.720 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107620 2024-09-14T09:07:53.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.720 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:07:53.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:07:53.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:07:53.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9999999999999998e-07 2024-09-14T09:07:53.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406595+0000", 2024-09-14T09:07:53.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9360000000000002e-06 2024-09-14T09:07:53.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406605+0000", 2024-09-14T09:07:53.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0268000000000001e-05 2024-09-14T09:07:53.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406614+0000", 2024-09-14T09:07:53.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7180000000000002e-06 2024-09-14T09:07:53.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554750+0000", 2024-09-14T09:07:53.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14813599699999999 2024-09-14T09:07:53.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554794+0000", 2024-09-14T09:07:53.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.401e-05 2024-09-14T09:07:53.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555105+0000", 2024-09-14T09:07:53.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031131399999999998 2024-09-14T09:07:53.723 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.723 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.723 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.723 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107622 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:53.724 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:07:53.724 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.278208262, 2024-09-14T09:07:53.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14902820899999999, 2024-09-14T09:07:53.724 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.724 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.724 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.724 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.724 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.724 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.724 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107622 2024-09-14T09:07:53.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.725 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:07:53.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:07:53.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:07:53.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9200000000000001e-07 2024-09-14T09:07:53.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406636+0000", 2024-09-14T09:07:53.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.189e-05 2024-09-14T09:07:53.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406649+0000", 2024-09-14T09:07:53.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2442e-05 2024-09-14T09:07:53.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406658+0000", 2024-09-14T09:07:53.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3239999999999992e-06 2024-09-14T09:07:53.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555287+0000", 2024-09-14T09:07:53.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14862868400000001 2024-09-14T09:07:53.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555348+0000", 2024-09-14T09:07:53.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1686999999999994e-05 2024-09-14T09:07:53.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555652+0000", 2024-09-14T09:07:53.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00030369000000000001 2024-09-14T09:07:53.728 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.728 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.728 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.728 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107624 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:53.728 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:07:53.728 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.278164569000001, 2024-09-14T09:07:53.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14934019900000001, 2024-09-14T09:07:53.729 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.729 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.729 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.729 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.729 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.729 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.729 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107624 2024-09-14T09:07:53.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.729 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.729 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.729 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:07:53.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:07:53.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:07:53.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1399999999999997e-07 2024-09-14T09:07:53.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406670+0000", 2024-09-14T09:07:53.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4099999999999998e-06 2024-09-14T09:07:53.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406680+0000", 2024-09-14T09:07:53.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6460000000000005e-06 2024-09-14T09:07:53.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406688+0000", 2024-09-14T09:07:53.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5859999999999999e-06 2024-09-14T09:07:53.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555693+0000", 2024-09-14T09:07:53.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14900544499999999 2024-09-14T09:07:53.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555743+0000", 2024-09-14T09:07:53.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9966999999999998e-05 2024-09-14T09:07:53.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556008+0000", 2024-09-14T09:07:53.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026463099999999998 2024-09-14T09:07:53.733 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.733 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.733 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.733 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107626 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:53.733 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:07:53.733 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.278132594999999, 2024-09-14T09:07:53.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14983149500000001, 2024-09-14T09:07:53.733 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.733 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.733 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.733 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.734 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.734 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.734 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107626 2024-09-14T09:07:53.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.734 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:07:53.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:07:53.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408087+0000", 2024-09-14T09:07:53.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001387735 2024-09-14T09:07:53.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408094+0000", 2024-09-14T09:07:53.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9399999999999996e-06 2024-09-14T09:07:53.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408122+0000", 2024-09-14T09:07:53.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8076e-05 2024-09-14T09:07:53.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408147+0000", 2024-09-14T09:07:53.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4726e-05 2024-09-14T09:07:53.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556048+0000", 2024-09-14T09:07:53.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14790144299999999 2024-09-14T09:07:53.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556175+0000", 2024-09-14T09:07:53.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012663 2024-09-14T09:07:53.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556531+0000", 2024-09-14T09:07:53.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035594499999999998 2024-09-14T09:07:53.737 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.737 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.737 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.737 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107628 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:53.738 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:07:53.738 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.276662984000001, 2024-09-14T09:07:53.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14882648900000001, 2024-09-14T09:07:53.738 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.738 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.738 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.738 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.738 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.738 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.738 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107628 2024-09-14T09:07:53.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.738 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:07:53.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:07:53.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408878+0000", 2024-09-14T09:07:53.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070858800000000001 2024-09-14T09:07:53.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408885+0000", 2024-09-14T09:07:53.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4839999999999998e-06 2024-09-14T09:07:53.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408900+0000", 2024-09-14T09:07:53.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5174000000000001e-05 2024-09-14T09:07:53.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408918+0000", 2024-09-14T09:07:53.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.808e-05 2024-09-14T09:07:53.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556572+0000", 2024-09-14T09:07:53.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14765399500000001 2024-09-14T09:07:53.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556651+0000", 2024-09-14T09:07:53.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8919999999999997e-05 2024-09-14T09:07:53.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556996+0000", 2024-09-14T09:07:53.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034424800000000001 2024-09-14T09:07:53.742 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.742 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.742 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.742 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107630 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 63488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:53.742 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:07:53.742 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.275894149999999, 2024-09-14T09:07:53.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14855159200000001, 2024-09-14T09:07:53.743 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.743 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.743 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.743 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.743 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.743 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.743 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107630 2024-09-14T09:07:53.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.743 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:07:53.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:07:53.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409510+0000", 2024-09-14T09:07:53.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00057189700000000003 2024-09-14T09:07:53.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409518+0000", 2024-09-14T09:07:53.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7619999999999995e-06 2024-09-14T09:07:53.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409534+0000", 2024-09-14T09:07:53.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.594e-05 2024-09-14T09:07:53.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409546+0000", 2024-09-14T09:07:53.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2604e-05 2024-09-14T09:07:53.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557048+0000", 2024-09-14T09:07:53.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14750137899999999 2024-09-14T09:07:53.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557117+0000", 2024-09-14T09:07:53.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8998999999999994e-05 2024-09-14T09:07:53.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557490+0000", 2024-09-14T09:07:53.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037301100000000002 2024-09-14T09:07:53.746 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.746 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.746 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.747 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107632 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:53.747 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:07:53.747 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.275263205000002, 2024-09-14T09:07:53.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14833944199999999, 2024-09-14T09:07:53.747 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.747 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.747 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.747 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.747 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.747 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.747 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107632 2024-09-14T09:07:53.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.748 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:07:53.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:07:53.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412609+0000", 2024-09-14T09:07:53.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003040105 2024-09-14T09:07:53.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412618+0000", 2024-09-14T09:07:53.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2960000000000004e-06 2024-09-14T09:07:53.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412640+0000", 2024-09-14T09:07:53.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1773999999999999e-05 2024-09-14T09:07:53.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412655+0000", 2024-09-14T09:07:53.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4538e-05 2024-09-14T09:07:53.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557538+0000", 2024-09-14T09:07:53.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14488311800000001 2024-09-14T09:07:53.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557594+0000", 2024-09-14T09:07:53.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6122999999999998e-05 2024-09-14T09:07:53.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557908+0000", 2024-09-14T09:07:53.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031448799999999997 2024-09-14T09:07:53.751 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.751 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.751 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.751 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107634 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:53.751 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:07:53.751 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.272145557999998, 2024-09-14T09:07:53.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.145666186, 2024-09-14T09:07:53.752 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.752 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.752 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.752 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.752 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.752 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.752 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107634 2024-09-14T09:07:53.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.752 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:07:53.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:07:53.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412688+0000", 2024-09-14T09:07:53.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9000000000000005e-07 2024-09-14T09:07:53.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412707+0000", 2024-09-14T09:07:53.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9232000000000002e-05 2024-09-14T09:07:53.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412722+0000", 2024-09-14T09:07:53.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5722e-05 2024-09-14T09:07:53.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412733+0000", 2024-09-14T09:07:53.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0909e-05 2024-09-14T09:07:53.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557946+0000", 2024-09-14T09:07:53.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14521263100000001 2024-09-14T09:07:53.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558020+0000", 2024-09-14T09:07:53.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4294000000000004e-05 2024-09-14T09:07:53.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558353+0000", 2024-09-14T09:07:53.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033240800000000002 2024-09-14T09:07:53.759 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.759 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.759 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.759 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107636 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:53.759 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:07:53.759 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.272081255, 2024-09-14T09:07:53.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14599377399999999, 2024-09-14T09:07:53.759 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.759 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.759 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.759 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.759 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.759 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.760 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107636 2024-09-14T09:07:53.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.760 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:07:53.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:07:53.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412752+0000", 2024-09-14T09:07:53.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.156e-06 2024-09-14T09:07:53.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412770+0000", 2024-09-14T09:07:53.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7521999999999999e-05 2024-09-14T09:07:53.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412790+0000", 2024-09-14T09:07:53.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0942e-05 2024-09-14T09:07:53.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412809+0000", 2024-09-14T09:07:53.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8887999999999999e-05 2024-09-14T09:07:53.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558387+0000", 2024-09-14T09:07:53.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14557798799999999 2024-09-14T09:07:53.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558448+0000", 2024-09-14T09:07:53.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0862000000000001e-05 2024-09-14T09:07:53.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558745+0000", 2024-09-14T09:07:53.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029641599999999998 2024-09-14T09:07:53.763 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.763 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.763 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.763 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107638 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:53.763 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:07:53.763 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.271993998999999, 2024-09-14T09:07:53.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146344799, 2024-09-14T09:07:53.763 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.763 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.763 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.763 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.764 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.764 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.764 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107638 2024-09-14T09:07:53.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.764 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:07:53.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:07:53.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412839+0000", 2024-09-14T09:07:53.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0720000000000001e-06 2024-09-14T09:07:53.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412844+0000", 2024-09-14T09:07:53.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7759999999999997e-06 2024-09-14T09:07:53.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412857+0000", 2024-09-14T09:07:53.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3138e-05 2024-09-14T09:07:53.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412867+0000", 2024-09-14T09:07:53.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7229999999999999e-06 2024-09-14T09:07:53.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558785+0000", 2024-09-14T09:07:53.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14591828700000001 2024-09-14T09:07:53.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558833+0000", 2024-09-14T09:07:53.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7790000000000002e-05 2024-09-14T09:07:53.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559183+0000", 2024-09-14T09:07:53.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035001299999999998 2024-09-14T09:07:53.767 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.767 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.767 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.767 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107640 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:53.767 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:07:53.768 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.271948848000001, 2024-09-14T09:07:53.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146686804, 2024-09-14T09:07:53.768 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.768 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.768 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.768 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.768 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.768 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.768 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107640 2024-09-14T09:07:53.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.768 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:07:53.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:07:53.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420446+0000", 2024-09-14T09:07:53.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075632529999999998 2024-09-14T09:07:53.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420456+0000", 2024-09-14T09:07:53.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0850000000000003e-06 2024-09-14T09:07:53.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420475+0000", 2024-09-14T09:07:53.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9406e-05 2024-09-14T09:07:53.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420488+0000", 2024-09-14T09:07:53.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3278000000000001e-05 2024-09-14T09:07:53.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559205+0000", 2024-09-14T09:07:53.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13871644999999999 2024-09-14T09:07:53.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559252+0000", 2024-09-14T09:07:53.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7666000000000001e-05 2024-09-14T09:07:53.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559570+0000", 2024-09-14T09:07:53.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031766599999999998 2024-09-14T09:07:53.771 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.772 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.772 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.772 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107660 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 124928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:53.772 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:07:53.772 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.263878605999999, 2024-09-14T09:07:53.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.142954628, 2024-09-14T09:07:53.772 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:53.772 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:53.772 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:53.772 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:53.772 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:53.772 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:53.772 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107660 2024-09-14T09:07:53.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.773 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:53.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:53.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:07:53.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:53.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:07:53.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:53.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:53.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428190+0000", 2024-09-14T09:07:53.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072360339999999997 2024-09-14T09:07:53.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:53.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428398+0000", 2024-09-14T09:07:53.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020898499999999999 2024-09-14T09:07:53.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:53.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428458+0000", 2024-09-14T09:07:53.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9985999999999998e-05 2024-09-14T09:07:53.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:53.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428480+0000", 2024-09-14T09:07:53.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1284000000000002e-05 2024-09-14T09:07:53.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:53.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563583+0000", 2024-09-14T09:07:53.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13510361800000001 2024-09-14T09:07:53.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:53.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563644+0000", 2024-09-14T09:07:53.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1092000000000004e-05 2024-09-14T09:07:53.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:53.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:53.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:53.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563908+0000", 2024-09-14T09:07:53.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026362899999999998 2024-09-14T09:07:53.776 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.776 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.776 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.776 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:53.776 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:53.776 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:53.777 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:53.777 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:53.778 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:53.778 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:53.778 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:07:53.779 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:53.779 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:07:53.780 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:53.780 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T09:07:53.780 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:53.775+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:53.879 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:53.879+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:53.908 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:53.908 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:53.922 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:07:53.980 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:53.979+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:54.045 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:54.045 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:54.059 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:07:54.075 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T09:07:54.106 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:54.110+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:54.182 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:54.186+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:54.186 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:54.186 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:54.201 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:07:54.282 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:54.283+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:54.349 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:54.350 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:54.365 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:07:54.383 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:54.386+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:54.485 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:54.483+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:54.533 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:54.533 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:54.548 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:07:54.585 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:54.586+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:54.686 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:54.687+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:54.715 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:54.715 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:54.729 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:07:54.787 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:54.790+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:54.888 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:54.890+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:54.899 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:54.899 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:54.913 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:07:54.989 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:54.990+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:55.033 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:55.033 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:55.047 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:07:55.077 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is unset 2024-09-14T09:07:55.089 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:55.087+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:55.190 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:55.194+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:55.195 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:55.195 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:55.210 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:07:55.291 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:55.294+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:55.347 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:55.347 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:55.362 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:07:55.406 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:55.407+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:55.480 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:55.481 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:55.493 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:55.491+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:55.496 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:07:55.594 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:55.595+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:55.660 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:55.660 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:55.675 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:07:55.722 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:55.723+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:55.781 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:07:55.796 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:55.798+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:55.849 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:55.849 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:55.864 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:07:55.897 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:55.898+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:55.923 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:55.924 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:55.940 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:55.941 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:55.957 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:55.957 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:55.973 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:55.974 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:55.974 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:07:55.983 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:55.983 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:07:55.984 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:07:55.984 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:07:55.984 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:55.994 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:55.994 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:55.998 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:07:55.998 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:56.002+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:56.015 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:56.016 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:56.036 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:56.036 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:56.039 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:07:56.040 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T09:07:56.098 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:56.099+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:56.119 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:56.119 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:07:56.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:07:56.119 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:56.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.120 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:56.120 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:56.120 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 143.03623063800001, 2024-09-14T09:07:56.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:07:56.120 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.120 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.120 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:56.121 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:56.121 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:56.121 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.121 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:07:56.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.121 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:56.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:07:56.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:07:56.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:07:56.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:07:56.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:07:56.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:07:56.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:07:56.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:07:56.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:07:56.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:07:56.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:07:56.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:07:56.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:07:56.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:07:56.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:07:56.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:07:56.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:07:56.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:07:56.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:07:56.125 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.125 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.125 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.125 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:56.125 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:56.126 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 143.03617598, 2024-09-14T09:07:56.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:07:56.126 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.126 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.126 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:56.126 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:56.126 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:56.126 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.126 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:07:56.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.126 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:56.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:56.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:07:56.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:07:56.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:07:56.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:07:56.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:07:56.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:07:56.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:07:56.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:07:56.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:07:56.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:07:56.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:07:56.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:07:56.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:07:56.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:07:56.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:07:56.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:07:56.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:07:56.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:07:56.131 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.131 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.131 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.132 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:56.132 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:56.132 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 143.03607060600001, 2024-09-14T09:07:56.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:07:56.132 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.132 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.132 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:56.132 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:56.132 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:56.132 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.132 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:07:56.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.132 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:56.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:07:56.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:07:56.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:07:56.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:07:56.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:07:56.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:07:56.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:07:56.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:07:56.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:07:56.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:07:56.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:07:56.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:07:56.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:07:56.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:07:56.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:07:56.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:07:56.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:07:56.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:07:56.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:07:56.138 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.138 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.138 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.138 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:56.138 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:56.138 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 143.03571384200001, 2024-09-14T09:07:56.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:07:56.138 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.139 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.139 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:56.139 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:56.139 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:56.139 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.139 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:07:56.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.139 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:56.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:07:56.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:07:56.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:07:56.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:07:56.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:07:56.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:07:56.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:07:56.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:07:56.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:07:56.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:07:56.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:07:56.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:07:56.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:07:56.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:07:56.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:07:56.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:07:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:07:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:07:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:07:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:56.143 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 143.03562910400001, 2024-09-14T09:07:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:07:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:07:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:07:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:07:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:07:56.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:07:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:07:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:07:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:07:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:07:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:07:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:07:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:07:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:07:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:07:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:07:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:07:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:07:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:07:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:07:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:07:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:07:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 143.03554141800001, 2024-09-14T09:07:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:07:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.149 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:07:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:07:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:07:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:07:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:07:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:07:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:07:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:07:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:07:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:07:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:07:56.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:07:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:07:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:07:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:07:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:07:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:07:56.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:07:56.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:07:56.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:07:56.154 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.154 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.154 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.154 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:07:56.154 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:56.154 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 109.959249431, 2024-09-14T09:07:56.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:07:56.155 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.155 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.155 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:56.155 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:56.155 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:56.155 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.155 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:07:56.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.155 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:07:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:07:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:07:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:07:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:07:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:07:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:07:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:07:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:07:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:07:56.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:07:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:07:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:07:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:07:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:07:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:07:56.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:07:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:07:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:07:56.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:07:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:07:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:07:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:07:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:07:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.161 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:97796 3.15 3:aaaa5f7f:::rbd_data.115e113dc8ca.0000000000000029:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:07:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 44.025380560999999, 2024-09-14T09:07:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14537731300000001, 2024-09-14T09:07:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:56.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97796 2024-09-14T09:07:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:07:56.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094482+0000", 2024-09-14T09:07:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6400000000000001e-07 2024-09-14T09:07:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094485+0000", 2024-09-14T09:07:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.693e-06 2024-09-14T09:07:56.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094494+0000", 2024-09-14T09:07:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9590000000000001e-06 2024-09-14T09:07:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094502+0000", 2024-09-14T09:07:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3180000000000001e-06 2024-09-14T09:07:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094567+0000", 2024-09-14T09:07:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5526999999999998e-05 2024-09-14T09:07:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094726+0000", 2024-09-14T09:07:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015842499999999999 2024-09-14T09:07:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:07:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.101940+0000", 2024-09-14T09:07:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0072146559999999998 2024-09-14T09:07:56.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:07:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239665+0000", 2024-09-14T09:07:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.137724501 2024-09-14T09:07:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239726+0000", 2024-09-14T09:07:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1550999999999994e-05 2024-09-14T09:07:56.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:07:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239800+0000", 2024-09-14T09:07:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3318000000000005e-05 2024-09-14T09:07:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239858+0000", 2024-09-14T09:07:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8600999999999997e-05 2024-09-14T09:07:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:97796 3.15 e106/103 ondisk, result = 0)", 2024-09-14T09:07:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 44.003034692, 2024-09-14T09:07:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123043465, 2024-09-14T09:07:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:07:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116828+0000", 2024-09-14T09:07:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8100000000000002e-07 2024-09-14T09:07:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116831+0000", 2024-09-14T09:07:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3009999999999998e-06 2024-09-14T09:07:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116841+0000", 2024-09-14T09:07:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7829999999999993e-06 2024-09-14T09:07:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116851+0000", 2024-09-14T09:07:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0135999999999999e-05 2024-09-14T09:07:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239707+0000", 2024-09-14T09:07:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12285581499999999 2024-09-14T09:07:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239718+0000", 2024-09-14T09:07:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1804000000000001e-05 2024-09-14T09:07:56.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239870+0000", 2024-09-14T09:07:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015194500000000001 2024-09-14T09:07:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117315 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 309248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:07:56.174 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.396398954999999, 2024-09-14T09:07:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11247383800000001, 2024-09-14T09:07:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117315 2024-09-14T09:07:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:07:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:07:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723899+0000", 2024-09-14T09:07:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043641699999999998 2024-09-14T09:07:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723903+0000", 2024-09-14T09:07:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5389999999999999e-06 2024-09-14T09:07:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723910+0000", 2024-09-14T09:07:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4229999999999999e-06 2024-09-14T09:07:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723917+0000", 2024-09-14T09:07:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4560000000000003e-06 2024-09-14T09:07:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835514+0000", 2024-09-14T09:07:56.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111596258 2024-09-14T09:07:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835571+0000", 2024-09-14T09:07:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7317e-05 2024-09-14T09:07:56.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835936+0000", 2024-09-14T09:07:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000365428 2024-09-14T09:07:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117317 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 313344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:56.180 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:07:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.395931939, 2024-09-14T09:07:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112450201, 2024-09-14T09:07:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117317 2024-09-14T09:07:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:07:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:07:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724396+0000", 2024-09-14T09:07:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000466028 2024-09-14T09:07:56.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724401+0000", 2024-09-14T09:07:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2299999999999999e-06 2024-09-14T09:07:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724412+0000", 2024-09-14T09:07:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0762e-05 2024-09-14T09:07:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724419+0000", 2024-09-14T09:07:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1010000000000003e-06 2024-09-14T09:07:56.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835981+0000", 2024-09-14T09:07:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111562174 2024-09-14T09:07:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836070+0000", 2024-09-14T09:07:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8839999999999996e-05 2024-09-14T09:07:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836380+0000", 2024-09-14T09:07:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031006600000000001 2024-09-14T09:07:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117319 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 317440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:07:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.395427303000002, 2024-09-14T09:07:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112467763, 2024-09-14T09:07:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:56.185 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117319 2024-09-14T09:07:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:07:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:07:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724846+0000", 2024-09-14T09:07:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00041155400000000001 2024-09-14T09:07:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724854+0000", 2024-09-14T09:07:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5469999999999997e-06 2024-09-14T09:07:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724862+0000", 2024-09-14T09:07:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1289999999999996e-06 2024-09-14T09:07:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724867+0000", 2024-09-14T09:07:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8720000000000001e-06 2024-09-14T09:07:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836486+0000", 2024-09-14T09:07:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111618357 2024-09-14T09:07:56.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836554+0000", 2024-09-14T09:07:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8378000000000002e-05 2024-09-14T09:07:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836902+0000", 2024-09-14T09:07:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034792599999999998 2024-09-14T09:07:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117321 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 321536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:07:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.394982979000002, 2024-09-14T09:07:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11249685600000001, 2024-09-14T09:07:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117321 2024-09-14T09:07:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.190 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:07:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:07:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725241+0000", 2024-09-14T09:07:56.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036211099999999998 2024-09-14T09:07:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725245+0000", 2024-09-14T09:07:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9179999999999996e-06 2024-09-14T09:07:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725254+0000", 2024-09-14T09:07:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0699999999999996e-06 2024-09-14T09:07:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725260+0000", 2024-09-14T09:07:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1099999999999999e-06 2024-09-14T09:07:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836935+0000", 2024-09-14T09:07:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111674997 2024-09-14T09:07:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837021+0000", 2024-09-14T09:07:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5931000000000001e-05 2024-09-14T09:07:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837375+0000", 2024-09-14T09:07:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035471899999999998 2024-09-14T09:07:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117323 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:07:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.394588754000001, 2024-09-14T09:07:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11250631, 2024-09-14T09:07:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.194 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117323 2024-09-14T09:07:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:07:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:07:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725941+0000", 2024-09-14T09:07:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00066866500000000004 2024-09-14T09:07:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725946+0000", 2024-09-14T09:07:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9060000000000001e-06 2024-09-14T09:07:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725959+0000", 2024-09-14T09:07:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2622e-05 2024-09-14T09:07:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725968+0000", 2024-09-14T09:07:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5660000000000007e-06 2024-09-14T09:07:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837408+0000", 2024-09-14T09:07:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111440626 2024-09-14T09:07:56.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837495+0000", 2024-09-14T09:07:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6654000000000001e-05 2024-09-14T09:07:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837779+0000", 2024-09-14T09:07:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000284271 2024-09-14T09:07:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117327 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 333824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:07:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.393852561999999, 2024-09-14T09:07:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112409908, 2024-09-14T09:07:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117327 2024-09-14T09:07:56.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:07:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:07:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726303+0000", 2024-09-14T09:07:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029388199999999998 2024-09-14T09:07:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726307+0000", 2024-09-14T09:07:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4320000000000001e-06 2024-09-14T09:07:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726317+0000", 2024-09-14T09:07:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4609999999999997e-06 2024-09-14T09:07:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726324+0000", 2024-09-14T09:07:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0480000000000001e-06 2024-09-14T09:07:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838051+0000", 2024-09-14T09:07:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111727534 2024-09-14T09:07:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838143+0000", 2024-09-14T09:07:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1359000000000004e-05 2024-09-14T09:07:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838419+0000", 2024-09-14T09:07:56.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027619199999999997 2024-09-14T09:07:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117329 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 337920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:07:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.393525802999999, 2024-09-14T09:07:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112411122, 2024-09-14T09:07:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.204 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117329 2024-09-14T09:07:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:07:56.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:07:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726742+0000", 2024-09-14T09:07:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040626099999999999 2024-09-14T09:07:56.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726748+0000", 2024-09-14T09:07:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6860000000000003e-06 2024-09-14T09:07:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726761+0000", 2024-09-14T09:07:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3599e-05 2024-09-14T09:07:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726770+0000", 2024-09-14T09:07:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5990000000000002e-06 2024-09-14T09:07:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838443+0000", 2024-09-14T09:07:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111672664 2024-09-14T09:07:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838487+0000", 2024-09-14T09:07:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4249000000000003e-05 2024-09-14T09:07:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838747+0000", 2024-09-14T09:07:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026006400000000001 2024-09-14T09:07:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.209 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.210 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117331 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 342016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:56.210 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:07:56.211 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.393058207999999, 2024-09-14T09:07:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112291139, 2024-09-14T09:07:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117331 2024-09-14T09:07:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:07:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:07:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727158+0000", 2024-09-14T09:07:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035434000000000003 2024-09-14T09:07:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727162+0000", 2024-09-14T09:07:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5449999999999997e-06 2024-09-14T09:07:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727174+0000", 2024-09-14T09:07:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2278e-05 2024-09-14T09:07:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727183+0000", 2024-09-14T09:07:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3820000000000004e-06 2024-09-14T09:07:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838788+0000", 2024-09-14T09:07:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111604748 2024-09-14T09:07:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838847+0000", 2024-09-14T09:07:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8983000000000001e-05 2024-09-14T09:07:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839094+0000", 2024-09-14T09:07:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024786300000000002 2024-09-14T09:07:56.215 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117333 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 346112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:07:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.392664397000001, 2024-09-14T09:07:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112319711, 2024-09-14T09:07:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:56.216 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117333 2024-09-14T09:07:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:07:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:07:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727785+0000", 2024-09-14T09:07:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000587512 2024-09-14T09:07:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727789+0000", 2024-09-14T09:07:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.352e-06 2024-09-14T09:07:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727800+0000", 2024-09-14T09:07:56.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1369000000000001e-05 2024-09-14T09:07:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727807+0000", 2024-09-14T09:07:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1300000000000003e-06 2024-09-14T09:07:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839137+0000", 2024-09-14T09:07:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111329898 2024-09-14T09:07:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839226+0000", 2024-09-14T09:07:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8337000000000005e-05 2024-09-14T09:07:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839517+0000", 2024-09-14T09:07:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029111300000000002 2024-09-14T09:07:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117363 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 407552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:56.220 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:07:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.386568298, 2024-09-14T09:07:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112191306, 2024-09-14T09:07:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117363 2024-09-14T09:07:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:07:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:07:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733666+0000", 2024-09-14T09:07:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00037284500000000001 2024-09-14T09:07:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733671+0000", 2024-09-14T09:07:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9269999999999996e-06 2024-09-14T09:07:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733681+0000", 2024-09-14T09:07:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5599999999999999e-06 2024-09-14T09:07:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733689+0000", 2024-09-14T09:07:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2649999999999996e-06 2024-09-14T09:07:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845028+0000", 2024-09-14T09:07:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111338933 2024-09-14T09:07:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845094+0000", 2024-09-14T09:07:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6749999999999996e-05 2024-09-14T09:07:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845485+0000", 2024-09-14T09:07:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00039002600000000002 2024-09-14T09:07:56.224 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117365 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 411648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:07:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:07:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 24.386158533, 2024-09-14T09:07:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112271685, 2024-09-14T09:07:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:56.225 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117365 2024-09-14T09:07:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:07:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:07:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734147+0000", 2024-09-14T09:07:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00044374399999999999 2024-09-14T09:07:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734152+0000", 2024-09-14T09:07:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9139999999999999e-06 2024-09-14T09:07:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734164+0000", 2024-09-14T09:07:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2299e-05 2024-09-14T09:07:56.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734172+0000", 2024-09-14T09:07:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0390000000000005e-06 2024-09-14T09:07:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845528+0000", 2024-09-14T09:07:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11135639899999999 2024-09-14T09:07:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:56.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845602+0000", 2024-09-14T09:07:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3957999999999999e-05 2024-09-14T09:07:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:07:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845975+0000", 2024-09-14T09:07:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00037233200000000001 2024-09-14T09:07:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:56.229 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:56.229 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:56.231 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:07:56.231 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:56.234+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:56.332 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:56.334+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:56.365 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:56.365 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:56.365 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:07:56.366 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:56.380 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:07:56.466 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:56.470+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:56.533 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:56.534+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:56.545 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:56.546 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:56.546 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:07:56.546 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:07:56.546 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:56.560 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:07:56.646 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:56.650+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:56.729 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:56.729 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:07:56.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:07:56.729 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:07:56.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.730 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105615 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 382976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.730 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:07:56.730 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.867956266, 2024-09-14T09:07:56.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117270900000001, 2024-09-14T09:07:56.730 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.730 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.730 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:56.730 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.730 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.730 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.730 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105615 2024-09-14T09:07:56.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.730 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:07:56.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:07:56.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864368+0000", 2024-09-14T09:07:56.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.043e-06 2024-09-14T09:07:56.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864373+0000", 2024-09-14T09:07:56.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4669999999999998e-06 2024-09-14T09:07:56.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864387+0000", 2024-09-14T09:07:56.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3876000000000001e-05 2024-09-14T09:07:56.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864397+0000", 2024-09-14T09:07:56.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0592e-05 2024-09-14T09:07:56.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866069+0000", 2024-09-14T09:07:56.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016714410000000001 2024-09-14T09:07:56.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866076+0000", 2024-09-14T09:07:56.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:07:56.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.075854+0000", 2024-09-14T09:07:56.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.209778501 2024-09-14T09:07:56.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:56.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.075940+0000", 2024-09-14T09:07:56.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5357000000000003e-05 2024-09-14T09:07:56.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.076094+0000", 2024-09-14T09:07:56.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000154745 2024-09-14T09:07:56.735 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.735 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.735 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.735 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105629 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 411648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.735 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:07:56.735 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.865802543999997, 2024-09-14T09:07:56.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2115591910000001, 2024-09-14T09:07:56.735 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.735 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.735 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:56.735 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.735 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.736 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.736 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105629 2024-09-14T09:07:56.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.736 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:07:56.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:07:56.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866522+0000", 2024-09-14T09:07:56.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6899999999999996e-07 2024-09-14T09:07:56.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866526+0000", 2024-09-14T09:07:56.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0110000000000002e-06 2024-09-14T09:07:56.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866539+0000", 2024-09-14T09:07:56.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3013000000000001e-05 2024-09-14T09:07:56.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866549+0000", 2024-09-14T09:07:56.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8490000000000001e-06 2024-09-14T09:07:56.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866650+0000", 2024-09-14T09:07:56.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101054 2024-09-14T09:07:56.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866664+0000", 2024-09-14T09:07:56.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3805000000000001e-05 2024-09-14T09:07:56.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.077849+0000", 2024-09-14T09:07:56.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2111855419999999 2024-09-14T09:07:56.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:56.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.077945+0000", 2024-09-14T09:07:56.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6086000000000001e-05 2024-09-14T09:07:56.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078080+0000", 2024-09-14T09:07:56.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000134862 2024-09-14T09:07:56.740 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.740 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.740 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.740 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105631 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 415744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.740 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:07:56.740 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.865743324999997, 2024-09-14T09:07:56.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119463509999999, 2024-09-14T09:07:56.740 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.740 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.741 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:56.741 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.741 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.741 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.741 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105631 2024-09-14T09:07:56.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.741 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:07:56.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:07:56.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866581+0000", 2024-09-14T09:07:56.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.879999999999999e-07 2024-09-14T09:07:56.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866585+0000", 2024-09-14T09:07:56.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0239999999999999e-06 2024-09-14T09:07:56.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866598+0000", 2024-09-14T09:07:56.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2862e-05 2024-09-14T09:07:56.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866608+0000", 2024-09-14T09:07:56.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9890000000000004e-06 2024-09-14T09:07:56.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866708+0000", 2024-09-14T09:07:56.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9605999999999995e-05 2024-09-14T09:07:56.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866720+0000", 2024-09-14T09:07:56.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.261e-05 2024-09-14T09:07:56.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078137+0000", 2024-09-14T09:07:56.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211416571 2024-09-14T09:07:56.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:56.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078293+0000", 2024-09-14T09:07:56.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000156 2024-09-14T09:07:56.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078527+0000", 2024-09-14T09:07:56.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000233701 2024-09-14T09:07:56.745 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.745 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.745 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.745 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105633 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.745 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:07:56.745 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.865684278000003, 2024-09-14T09:07:56.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211941734, 2024-09-14T09:07:56.746 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.746 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.746 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:56.746 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.746 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.746 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.746 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105633 2024-09-14T09:07:56.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.746 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:07:56.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:07:56.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866640+0000", 2024-09-14T09:07:56.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9300000000000006e-07 2024-09-14T09:07:56.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866656+0000", 2024-09-14T09:07:56.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5291e-05 2024-09-14T09:07:56.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866669+0000", 2024-09-14T09:07:56.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2918000000000001e-05 2024-09-14T09:07:56.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866678+0000", 2024-09-14T09:07:56.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8369999999999992e-06 2024-09-14T09:07:56.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866771+0000", 2024-09-14T09:07:56.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2250999999999995e-05 2024-09-14T09:07:56.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866784+0000", 2024-09-14T09:07:56.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2991e-05 2024-09-14T09:07:56.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078566+0000", 2024-09-14T09:07:56.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117822570000001 2024-09-14T09:07:56.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078581+0000", 2024-09-14T09:07:56.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5196e-05 2024-09-14T09:07:56.750 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.750 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.750 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.750 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105635 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.750 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:07:56.750 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.865630451000001, 2024-09-14T09:07:56.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119313250000001, 2024-09-14T09:07:56.750 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.750 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.750 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:56.750 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.751 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.751 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.751 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105635 2024-09-14T09:07:56.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.751 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:07:56.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:07:56.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866694+0000", 2024-09-14T09:07:56.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0210000000000001e-06 2024-09-14T09:07:56.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866698+0000", 2024-09-14T09:07:56.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.765e-06 2024-09-14T09:07:56.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866710+0000", 2024-09-14T09:07:56.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2320000000000001e-05 2024-09-14T09:07:56.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866722+0000", 2024-09-14T09:07:56.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2133000000000001e-05 2024-09-14T09:07:56.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866830+0000", 2024-09-14T09:07:56.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000107668 2024-09-14T09:07:56.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866843+0000", 2024-09-14T09:07:56.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2991e-05 2024-09-14T09:07:56.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078616+0000", 2024-09-14T09:07:56.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117732459999999 2024-09-14T09:07:56.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078624+0000", 2024-09-14T09:07:56.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1810000000000001e-06 2024-09-14T09:07:56.754 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.754 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.754 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.755 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105636 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.755 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:07:56.755 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.865562554999997, 2024-09-14T09:07:56.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119085119999999, 2024-09-14T09:07:56.755 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.755 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.755 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:56.755 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.755 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.755 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.755 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105636 2024-09-14T09:07:56.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.755 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:07:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:07:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866762+0000", 2024-09-14T09:07:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4600000000000003e-07 2024-09-14T09:07:56.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866766+0000", 2024-09-14T09:07:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:07:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866779+0000", 2024-09-14T09:07:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3101999999999999e-05 2024-09-14T09:07:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866789+0000", 2024-09-14T09:07:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8039999999999996e-06 2024-09-14T09:07:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866889+0000", 2024-09-14T09:07:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100147 2024-09-14T09:07:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866902+0000", 2024-09-14T09:07:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3212e-05 2024-09-14T09:07:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078659+0000", 2024-09-14T09:07:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211756362 2024-09-14T09:07:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078670+0000", 2024-09-14T09:07:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0956e-05 2024-09-14T09:07:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105639 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:07:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.865503680000003, 2024-09-14T09:07:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118829259999999, 2024-09-14T09:07:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105639 2024-09-14T09:07:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:07:56.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:07:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866821+0000", 2024-09-14T09:07:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9800000000000002e-07 2024-09-14T09:07:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866825+0000", 2024-09-14T09:07:56.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9269999999999998e-06 2024-09-14T09:07:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866838+0000", 2024-09-14T09:07:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3040000000000001e-05 2024-09-14T09:07:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866848+0000", 2024-09-14T09:07:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7929999999999992e-06 2024-09-14T09:07:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866948+0000", 2024-09-14T09:07:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100337 2024-09-14T09:07:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866961+0000", 2024-09-14T09:07:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3106e-05 2024-09-14T09:07:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078693+0000", 2024-09-14T09:07:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117320380000001 2024-09-14T09:07:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078703+0000", 2024-09-14T09:07:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6870000000000006e-06 2024-09-14T09:07:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105641 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:07:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.865444815000004, 2024-09-14T09:07:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118727169999999, 2024-09-14T09:07:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.764 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105641 2024-09-14T09:07:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:07:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:07:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866880+0000", 2024-09-14T09:07:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7699999999999992e-07 2024-09-14T09:07:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866884+0000", 2024-09-14T09:07:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9480000000000001e-06 2024-09-14T09:07:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866897+0000", 2024-09-14T09:07:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2832e-05 2024-09-14T09:07:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866906+0000", 2024-09-14T09:07:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7869999999999996e-06 2024-09-14T09:07:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867007+0000", 2024-09-14T09:07:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010103900000000001 2024-09-14T09:07:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867020+0000", 2024-09-14T09:07:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2896e-05 2024-09-14T09:07:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078739+0000", 2024-09-14T09:07:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211718305 2024-09-14T09:07:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078752+0000", 2024-09-14T09:07:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2933e-05 2024-09-14T09:07:56.768 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105643 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:07:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.865386256000001, 2024-09-14T09:07:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211858353, 2024-09-14T09:07:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.769 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105643 2024-09-14T09:07:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:07:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:07:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866938+0000", 2024-09-14T09:07:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5600000000000004e-07 2024-09-14T09:07:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866942+0000", 2024-09-14T09:07:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0350000000000003e-06 2024-09-14T09:07:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866955+0000", 2024-09-14T09:07:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3097e-05 2024-09-14T09:07:56.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866965+0000", 2024-09-14T09:07:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8179999999999999e-06 2024-09-14T09:07:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867067+0000", 2024-09-14T09:07:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010128699999999999 2024-09-14T09:07:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867080+0000", 2024-09-14T09:07:56.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3089e-05 2024-09-14T09:07:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078786+0000", 2024-09-14T09:07:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117065250000001 2024-09-14T09:07:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078796+0000", 2024-09-14T09:07:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5459999999999997e-06 2024-09-14T09:07:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105645 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:07:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.865325861000002, 2024-09-14T09:07:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211848641, 2024-09-14T09:07:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105645 2024-09-14T09:07:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:07:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:07:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866999+0000", 2024-09-14T09:07:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7999999999999993e-07 2024-09-14T09:07:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867003+0000", 2024-09-14T09:07:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0090000000000001e-06 2024-09-14T09:07:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867016+0000", 2024-09-14T09:07:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3304e-05 2024-09-14T09:07:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867026+0000", 2024-09-14T09:07:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5889999999999992e-06 2024-09-14T09:07:56.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867128+0000", 2024-09-14T09:07:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000102101 2024-09-14T09:07:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867141+0000", 2024-09-14T09:07:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2903e-05 2024-09-14T09:07:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078833+0000", 2024-09-14T09:07:56.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116920959999999 2024-09-14T09:07:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078846+0000", 2024-09-14T09:07:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3658999999999999e-05 2024-09-14T09:07:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105647 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:07:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.865267342999999, 2024-09-14T09:07:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211830097, 2024-09-14T09:07:56.778 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105647 2024-09-14T09:07:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:07:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:07:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867057+0000", 2024-09-14T09:07:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.95e-07 2024-09-14T09:07:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867061+0000", 2024-09-14T09:07:56.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9530000000000001e-06 2024-09-14T09:07:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867074+0000", 2024-09-14T09:07:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3061000000000001e-05 2024-09-14T09:07:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867086+0000", 2024-09-14T09:07:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1460999999999999e-05 2024-09-14T09:07:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867186+0000", 2024-09-14T09:07:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100181 2024-09-14T09:07:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867199+0000", 2024-09-14T09:07:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-05 2024-09-14T09:07:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078874+0000", 2024-09-14T09:07:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211674683 2024-09-14T09:07:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078886+0000", 2024-09-14T09:07:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2853e-05 2024-09-14T09:07:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105648 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.783 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:07:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.865205856999999, 2024-09-14T09:07:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118073570000001, 2024-09-14T09:07:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105648 2024-09-14T09:07:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:07:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:07:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867119+0000", 2024-09-14T09:07:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8400000000000002e-07 2024-09-14T09:07:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867123+0000", 2024-09-14T09:07:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9639999999999997e-06 2024-09-14T09:07:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867136+0000", 2024-09-14T09:07:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.289e-05 2024-09-14T09:07:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867145+0000", 2024-09-14T09:07:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8919999999999995e-06 2024-09-14T09:07:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867245+0000", 2024-09-14T09:07:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9282000000000003e-05 2024-09-14T09:07:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867258+0000", 2024-09-14T09:07:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2935000000000001e-05 2024-09-14T09:07:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078910+0000", 2024-09-14T09:07:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211652352 2024-09-14T09:07:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078925+0000", 2024-09-14T09:07:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5058000000000001e-05 2024-09-14T09:07:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105652 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:07:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.865147837999999, 2024-09-14T09:07:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211775737, 2024-09-14T09:07:56.788 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105652 2024-09-14T09:07:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:07:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:07:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867177+0000", 2024-09-14T09:07:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9100000000000002e-07 2024-09-14T09:07:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867181+0000", 2024-09-14T09:07:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0029999999999996e-06 2024-09-14T09:07:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867193+0000", 2024-09-14T09:07:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2675e-05 2024-09-14T09:07:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867203+0000", 2024-09-14T09:07:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5599999999999999e-06 2024-09-14T09:07:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867306+0000", 2024-09-14T09:07:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010330100000000001 2024-09-14T09:07:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867319+0000", 2024-09-14T09:07:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3129000000000001e-05 2024-09-14T09:07:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078944+0000", 2024-09-14T09:07:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116247520000001 2024-09-14T09:07:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078952+0000", 2024-09-14T09:07:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4259999999999997e-06 2024-09-14T09:07:56.792 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105653 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:07:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.865089167999997, 2024-09-14T09:07:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211774895, 2024-09-14T09:07:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.793 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105653 2024-09-14T09:07:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:07:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:07:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867235+0000", 2024-09-14T09:07:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9400000000000004e-07 2024-09-14T09:07:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867239+0000", 2024-09-14T09:07:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9380000000000002e-06 2024-09-14T09:07:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867252+0000", 2024-09-14T09:07:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.31e-05 2024-09-14T09:07:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867264+0000", 2024-09-14T09:07:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1275e-05 2024-09-14T09:07:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867367+0000", 2024-09-14T09:07:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103755 2024-09-14T09:07:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867382+0000", 2024-09-14T09:07:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4724e-05 2024-09-14T09:07:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078995+0000", 2024-09-14T09:07:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116132669999999 2024-09-14T09:07:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079009+0000", 2024-09-14T09:07:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3942e-05 2024-09-14T09:07:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105654 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.797 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:07:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.865028111999997, 2024-09-14T09:07:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117364070000001, 2024-09-14T09:07:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105654 2024-09-14T09:07:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:07:56.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:07:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867296+0000", 2024-09-14T09:07:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2399999999999996e-07 2024-09-14T09:07:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867300+0000", 2024-09-14T09:07:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8689999999999997e-06 2024-09-14T09:07:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867313+0000", 2024-09-14T09:07:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2660000000000001e-05 2024-09-14T09:07:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867322+0000", 2024-09-14T09:07:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.533e-06 2024-09-14T09:07:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867429+0000", 2024-09-14T09:07:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106566 2024-09-14T09:07:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867442+0000", 2024-09-14T09:07:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3244000000000001e-05 2024-09-14T09:07:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079024+0000", 2024-09-14T09:07:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211581813 2024-09-14T09:07:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079032+0000", 2024-09-14T09:07:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7980000000000006e-06 2024-09-14T09:07:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105657 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:07:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.864966938999999, 2024-09-14T09:07:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211700657, 2024-09-14T09:07:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.802 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105657 2024-09-14T09:07:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:07:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:07:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867358+0000", 2024-09-14T09:07:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9700000000000005e-07 2024-09-14T09:07:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867367+0000", 2024-09-14T09:07:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8919999999999995e-06 2024-09-14T09:07:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867383+0000", 2024-09-14T09:07:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.579e-05 2024-09-14T09:07:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867393+0000", 2024-09-14T09:07:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3070000000000008e-06 2024-09-14T09:07:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867490+0000", 2024-09-14T09:07:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7703999999999998e-05 2024-09-14T09:07:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867503+0000", 2024-09-14T09:07:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2849e-05 2024-09-14T09:07:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079050+0000", 2024-09-14T09:07:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211546899 2024-09-14T09:07:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079057+0000", 2024-09-14T09:07:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3189999999999997e-06 2024-09-14T09:07:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.806 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105658 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:07:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.864904555000003, 2024-09-14T09:07:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211691217, 2024-09-14T09:07:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105658 2024-09-14T09:07:56.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:07:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:07:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867420+0000", 2024-09-14T09:07:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5000000000000001e-07 2024-09-14T09:07:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867424+0000", 2024-09-14T09:07:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8469999999999998e-06 2024-09-14T09:07:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867437+0000", 2024-09-14T09:07:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3156000000000001e-05 2024-09-14T09:07:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867448+0000", 2024-09-14T09:07:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1185e-05 2024-09-14T09:07:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867552+0000", 2024-09-14T09:07:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103362 2024-09-14T09:07:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867565+0000", 2024-09-14T09:07:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3018e-05 2024-09-14T09:07:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079092+0000", 2024-09-14T09:07:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211527013 2024-09-14T09:07:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079110+0000", 2024-09-14T09:07:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8686e-05 2024-09-14T09:07:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105662 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:07:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.864843489999998, 2024-09-14T09:07:56.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211652226, 2024-09-14T09:07:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105662 2024-09-14T09:07:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:07:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:07:56.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867481+0000", 2024-09-14T09:07:56.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1800000000000004e-07 2024-09-14T09:07:56.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867485+0000", 2024-09-14T09:07:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9199999999999997e-06 2024-09-14T09:07:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867498+0000", 2024-09-14T09:07:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2896e-05 2024-09-14T09:07:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867509+0000", 2024-09-14T09:07:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1202e-05 2024-09-14T09:07:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867611+0000", 2024-09-14T09:07:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010166999999999999 2024-09-14T09:07:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867624+0000", 2024-09-14T09:07:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2996000000000001e-05 2024-09-14T09:07:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079125+0000", 2024-09-14T09:07:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2115009299999999 2024-09-14T09:07:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079132+0000", 2024-09-14T09:07:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6939999999999995e-06 2024-09-14T09:07:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105663 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:07:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.864782439000003, 2024-09-14T09:07:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116218169999999, 2024-09-14T09:07:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.816 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105663 2024-09-14T09:07:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:07:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:07:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867542+0000", 2024-09-14T09:07:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1399999999999995e-07 2024-09-14T09:07:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867546+0000", 2024-09-14T09:07:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7900000000000001e-06 2024-09-14T09:07:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867559+0000", 2024-09-14T09:07:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2805e-05 2024-09-14T09:07:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867568+0000", 2024-09-14T09:07:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-09-14T09:07:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867669+0000", 2024-09-14T09:07:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100655 2024-09-14T09:07:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867682+0000", 2024-09-14T09:07:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3013000000000001e-05 2024-09-14T09:07:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079151+0000", 2024-09-14T09:07:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2114689089999999 2024-09-14T09:07:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079163+0000", 2024-09-14T09:07:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1962e-05 2024-09-14T09:07:56.820 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105665 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:07:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:07:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.864723808000001, 2024-09-14T09:07:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211584848, 2024-09-14T09:07:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:56.821 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105665 2024-09-14T09:07:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:07:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:07:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867601+0000", 2024-09-14T09:07:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.569999999999999e-07 2024-09-14T09:07:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867605+0000", 2024-09-14T09:07:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8589999999999998e-06 2024-09-14T09:07:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867617+0000", 2024-09-14T09:07:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2833000000000001e-05 2024-09-14T09:07:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867627+0000", 2024-09-14T09:07:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5219999999999996e-06 2024-09-14T09:07:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867728+0000", 2024-09-14T09:07:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010057500000000001 2024-09-14T09:07:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867740+0000", 2024-09-14T09:07:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2744e-05 2024-09-14T09:07:56.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079178+0000", 2024-09-14T09:07:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211437552 2024-09-14T09:07:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079185+0000", 2024-09-14T09:07:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8059999999999997e-06 2024-09-14T09:07:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:56.825 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:56.826 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:56.827 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:07:56.827 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:56.830+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:56.928 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:56.930+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:57.002 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:57.002 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:57.002 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:07:57.002 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:57.016 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:07:57.029 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:57.027+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:57.111 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T09:07:57.129 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:57.130+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:57.191 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:57.191 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:57.191 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:07:57.191 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:07:57.191 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:57.207 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:07:57.230 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:57.231+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:57.332 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:57.331+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:57.378 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:57.378 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:07:57.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:07:57.378 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:07:57.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.378 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65139 3.a 3:54c90b7d:::rbd_data.115e21c31fcf.0000000000000008:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:07:57.378 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:07:57.379 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 96.001865566999996, 2024-09-14T09:07:57.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14429895700000001, 2024-09-14T09:07:57.379 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.379 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.379 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:07:57.379 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.379 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.379 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.379 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65139 2024-09-14T09:07:57.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.379 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:07:57.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:07:57.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379715+0000", 2024-09-14T09:07:57.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.576e-06 2024-09-14T09:07:57.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379724+0000", 2024-09-14T09:07:57.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1779999999999992e-06 2024-09-14T09:07:57.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379746+0000", 2024-09-14T09:07:57.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1661000000000001e-05 2024-09-14T09:07:57.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379759+0000", 2024-09-14T09:07:57.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.324e-05 2024-09-14T09:07:57.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379882+0000", 2024-09-14T09:07:57.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012239400000000001 2024-09-14T09:07:57.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.380023+0000", 2024-09-14T09:07:57.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000141402 2024-09-14T09:07:57.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:07:57.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.391553+0000", 2024-09-14T09:07:57.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011530197000000001 2024-09-14T09:07:57.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:07:57.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523927+0000", 2024-09-14T09:07:57.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.132373935 2024-09-14T09:07:57.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:07:57.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523941+0000", 2024-09-14T09:07:57.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3326000000000001e-05 2024-09-14T09:07:57.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:07:57.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523977+0000", 2024-09-14T09:07:57.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6185e-05 2024-09-14T09:07:57.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.524012+0000", 2024-09-14T09:07:57.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4863000000000001e-05 2024-09-14T09:07:57.384 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.384 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.384 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.385 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107606 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 14336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:57.385 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:07:57.385 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.975408666, 2024-09-14T09:07:57.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14293513799999999, 2024-09-14T09:07:57.385 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.385 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.385 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.385 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.385 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.385 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.385 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107606 2024-09-14T09:07:57.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.386 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:07:57.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:07:57.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:07:57.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1600000000000001e-07 2024-09-14T09:07:57.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406174+0000", 2024-09-14T09:07:57.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4199999999999999e-06 2024-09-14T09:07:57.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406188+0000", 2024-09-14T09:07:57.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4114000000000001e-05 2024-09-14T09:07:57.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406198+0000", 2024-09-14T09:07:57.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.041e-05 2024-09-14T09:07:57.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.548700+0000", 2024-09-14T09:07:57.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14250141299999999 2024-09-14T09:07:57.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.548783+0000", 2024-09-14T09:07:57.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3578999999999994e-05 2024-09-14T09:07:57.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549105+0000", 2024-09-14T09:07:57.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032158600000000001 2024-09-14T09:07:57.389 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.389 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.389 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.389 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107608 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:57.389 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:07:57.389 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.97534263, 2024-09-14T09:07:57.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14603042799999999, 2024-09-14T09:07:57.390 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.390 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.390 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.390 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.390 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.390 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.390 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107608 2024-09-14T09:07:57.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.390 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:07:57.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:07:57.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:07:57.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1200000000000002e-07 2024-09-14T09:07:57.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406241+0000", 2024-09-14T09:07:57.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7539999999999997e-06 2024-09-14T09:07:57.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406275+0000", 2024-09-14T09:07:57.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3512e-05 2024-09-14T09:07:57.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406322+0000", 2024-09-14T09:07:57.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6764999999999997e-05 2024-09-14T09:07:57.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549149+0000", 2024-09-14T09:07:57.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14282725499999999 2024-09-14T09:07:57.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549198+0000", 2024-09-14T09:07:57.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9599999999999999e-05 2024-09-14T09:07:57.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552266+0000", 2024-09-14T09:07:57.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030677299999999998 2024-09-14T09:07:57.394 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.394 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.394 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.394 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107610 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 22528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:57.394 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:07:57.394 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.975220755000002, 2024-09-14T09:07:57.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146437919, 2024-09-14T09:07:57.394 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.394 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.394 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.395 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.395 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.395 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.395 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107610 2024-09-14T09:07:57.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.395 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:07:57.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:07:57.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:07:57.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.02e-07 2024-09-14T09:07:57.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406363+0000", 2024-09-14T09:07:57.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8099999999999997e-06 2024-09-14T09:07:57.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406390+0000", 2024-09-14T09:07:57.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7004000000000001e-05 2024-09-14T09:07:57.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406404+0000", 2024-09-14T09:07:57.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3682e-05 2024-09-14T09:07:57.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552335+0000", 2024-09-14T09:07:57.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14593077800000001 2024-09-14T09:07:57.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552431+0000", 2024-09-14T09:07:57.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6086000000000001e-05 2024-09-14T09:07:57.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552795+0000", 2024-09-14T09:07:57.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000364657 2024-09-14T09:07:57.398 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.399 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.399 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.399 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107612 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 26624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:57.399 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:07:57.399 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.975155491999999, 2024-09-14T09:07:57.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146762061, 2024-09-14T09:07:57.399 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.399 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.399 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.399 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.399 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.400 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107612 2024-09-14T09:07:57.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.400 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.400 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.400 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:07:57.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.400 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.400 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:07:57.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406424+0000", 2024-09-14T09:07:57.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2600000000000002e-07 2024-09-14T09:07:57.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406451+0000", 2024-09-14T09:07:57.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7250000000000002e-05 2024-09-14T09:07:57.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406487+0000", 2024-09-14T09:07:57.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6086e-05 2024-09-14T09:07:57.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406498+0000", 2024-09-14T09:07:57.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0858e-05 2024-09-14T09:07:57.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552821+0000", 2024-09-14T09:07:57.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14632319099999999 2024-09-14T09:07:57.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552875+0000", 2024-09-14T09:07:57.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4262999999999999e-05 2024-09-14T09:07:57.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553185+0000", 2024-09-14T09:07:57.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000309687 2024-09-14T09:07:57.403 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.403 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.403 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.403 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107614 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:57.404 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:07:57.404 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.975067957999997, 2024-09-14T09:07:57.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14737121, 2024-09-14T09:07:57.404 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.404 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.404 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.404 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.404 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.404 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.404 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107614 2024-09-14T09:07:57.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.405 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:07:57.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:07:57.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406511+0000", 2024-09-14T09:07:57.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1799999999999995e-07 2024-09-14T09:07:57.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406514+0000", 2024-09-14T09:07:57.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8660000000000002e-06 2024-09-14T09:07:57.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406523+0000", 2024-09-14T09:07:57.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1940000000000004e-06 2024-09-14T09:07:57.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406531+0000", 2024-09-14T09:07:57.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3159999999999997e-06 2024-09-14T09:07:57.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553233+0000", 2024-09-14T09:07:57.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14670117899999999 2024-09-14T09:07:57.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553334+0000", 2024-09-14T09:07:57.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100982 2024-09-14T09:07:57.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553882+0000", 2024-09-14T09:07:57.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00054805499999999998 2024-09-14T09:07:57.409 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.409 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.409 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.409 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107616 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 34816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:57.409 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:07:57.409 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.975036322999998, 2024-09-14T09:07:57.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14780390500000001, 2024-09-14T09:07:57.409 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.410 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.410 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.410 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.410 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.410 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.410 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107616 2024-09-14T09:07:57.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.410 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:07:57.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:07:57.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406543+0000", 2024-09-14T09:07:57.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0200000000000001e-07 2024-09-14T09:07:57.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406545+0000", 2024-09-14T09:07:57.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3360000000000002e-06 2024-09-14T09:07:57.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406552+0000", 2024-09-14T09:07:57.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9779999999999999e-06 2024-09-14T09:07:57.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406559+0000", 2024-09-14T09:07:57.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6919999999999996e-06 2024-09-14T09:07:57.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553917+0000", 2024-09-14T09:07:57.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14735791600000001 2024-09-14T09:07:57.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554033+0000", 2024-09-14T09:07:57.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116047 2024-09-14T09:07:57.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554346+0000", 2024-09-14T09:07:57.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031323400000000002 2024-09-14T09:07:57.415 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.415 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.415 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.416 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107618 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 38912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:57.416 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:07:57.416 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.975007523000002, 2024-09-14T09:07:57.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14816009599999999, 2024-09-14T09:07:57.416 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.416 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.416 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.416 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.416 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.416 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.417 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107618 2024-09-14T09:07:57.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.417 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:07:57.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:07:57.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:07:57.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9999999999999998e-07 2024-09-14T09:07:57.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406574+0000", 2024-09-14T09:07:57.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2560000000000001e-06 2024-09-14T09:07:57.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406581+0000", 2024-09-14T09:07:57.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1539999999999996e-06 2024-09-14T09:07:57.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406586+0000", 2024-09-14T09:07:57.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.904e-06 2024-09-14T09:07:57.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554385+0000", 2024-09-14T09:07:57.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.147799443 2024-09-14T09:07:57.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554441+0000", 2024-09-14T09:07:57.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5921000000000002e-05 2024-09-14T09:07:57.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554731+0000", 2024-09-14T09:07:57.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028991800000000001 2024-09-14T09:07:57.421 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.421 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.421 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.422 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107620 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:57.422 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:07:57.422 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.974985740999998, 2024-09-14T09:07:57.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.148512643, 2024-09-14T09:07:57.422 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.423 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.423 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.423 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.423 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.423 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.423 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107620 2024-09-14T09:07:57.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.423 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:07:57.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:07:57.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:07:57.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9999999999999998e-07 2024-09-14T09:07:57.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406595+0000", 2024-09-14T09:07:57.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9360000000000002e-06 2024-09-14T09:07:57.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406605+0000", 2024-09-14T09:07:57.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0268000000000001e-05 2024-09-14T09:07:57.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406614+0000", 2024-09-14T09:07:57.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7180000000000002e-06 2024-09-14T09:07:57.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554750+0000", 2024-09-14T09:07:57.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14813599699999999 2024-09-14T09:07:57.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554794+0000", 2024-09-14T09:07:57.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.401e-05 2024-09-14T09:07:57.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555105+0000", 2024-09-14T09:07:57.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031131399999999998 2024-09-14T09:07:57.427 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.427 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.427 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.427 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107622 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:57.427 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:07:57.427 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.974954484999998, 2024-09-14T09:07:57.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14902820899999999, 2024-09-14T09:07:57.427 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.427 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.427 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.428 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.428 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.428 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.428 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107622 2024-09-14T09:07:57.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.428 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:07:57.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:07:57.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:07:57.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9200000000000001e-07 2024-09-14T09:07:57.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406636+0000", 2024-09-14T09:07:57.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.189e-05 2024-09-14T09:07:57.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406649+0000", 2024-09-14T09:07:57.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2442e-05 2024-09-14T09:07:57.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406658+0000", 2024-09-14T09:07:57.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3239999999999992e-06 2024-09-14T09:07:57.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555287+0000", 2024-09-14T09:07:57.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14862868400000001 2024-09-14T09:07:57.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555348+0000", 2024-09-14T09:07:57.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1686999999999994e-05 2024-09-14T09:07:57.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555652+0000", 2024-09-14T09:07:57.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00030369000000000001 2024-09-14T09:07:57.432 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.432 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.432 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.433 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107624 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:57.433 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:07:57.433 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.974910792000003, 2024-09-14T09:07:57.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14934019900000001, 2024-09-14T09:07:57.433 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.433 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.433 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.433 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.434 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.434 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.434 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107624 2024-09-14T09:07:57.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.434 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:07:57.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:07:57.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:07:57.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1399999999999997e-07 2024-09-14T09:07:57.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406670+0000", 2024-09-14T09:07:57.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4099999999999998e-06 2024-09-14T09:07:57.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406680+0000", 2024-09-14T09:07:57.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6460000000000005e-06 2024-09-14T09:07:57.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406688+0000", 2024-09-14T09:07:57.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5859999999999999e-06 2024-09-14T09:07:57.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555693+0000", 2024-09-14T09:07:57.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14900544499999999 2024-09-14T09:07:57.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.439 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555743+0000", 2024-09-14T09:07:57.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9966999999999998e-05 2024-09-14T09:07:57.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.439 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556008+0000", 2024-09-14T09:07:57.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026463099999999998 2024-09-14T09:07:57.439 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.439 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.440 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.440 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107626 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:57.440 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:07:57.440 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.974878818000001, 2024-09-14T09:07:57.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14983149500000001, 2024-09-14T09:07:57.440 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.441 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.441 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.441 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.441 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.441 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.441 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107626 2024-09-14T09:07:57.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.441 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:07:57.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:07:57.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408087+0000", 2024-09-14T09:07:57.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001387735 2024-09-14T09:07:57.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408094+0000", 2024-09-14T09:07:57.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9399999999999996e-06 2024-09-14T09:07:57.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408122+0000", 2024-09-14T09:07:57.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8076e-05 2024-09-14T09:07:57.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408147+0000", 2024-09-14T09:07:57.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4726e-05 2024-09-14T09:07:57.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556048+0000", 2024-09-14T09:07:57.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14790144299999999 2024-09-14T09:07:57.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556175+0000", 2024-09-14T09:07:57.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012663 2024-09-14T09:07:57.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556531+0000", 2024-09-14T09:07:57.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035594499999999998 2024-09-14T09:07:57.446 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.446 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.446 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.447 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107628 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:57.447 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:07:57.447 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.973409207000003, 2024-09-14T09:07:57.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14882648900000001, 2024-09-14T09:07:57.447 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.447 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.447 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.447 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.447 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.447 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.447 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107628 2024-09-14T09:07:57.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.448 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:07:57.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:07:57.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408878+0000", 2024-09-14T09:07:57.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070858800000000001 2024-09-14T09:07:57.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.449 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408885+0000", 2024-09-14T09:07:57.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4839999999999998e-06 2024-09-14T09:07:57.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.449 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408900+0000", 2024-09-14T09:07:57.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5174000000000001e-05 2024-09-14T09:07:57.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408918+0000", 2024-09-14T09:07:57.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.808e-05 2024-09-14T09:07:57.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556572+0000", 2024-09-14T09:07:57.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14765399500000001 2024-09-14T09:07:57.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556651+0000", 2024-09-14T09:07:57.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8919999999999997e-05 2024-09-14T09:07:57.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556996+0000", 2024-09-14T09:07:57.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034424800000000001 2024-09-14T09:07:57.451 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.451 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.451 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.451 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107630 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 63488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:57.451 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:07:57.452 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.972640372999997, 2024-09-14T09:07:57.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14855159200000001, 2024-09-14T09:07:57.452 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.452 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.452 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.452 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.452 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.452 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.452 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107630 2024-09-14T09:07:57.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.452 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:07:57.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:07:57.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409510+0000", 2024-09-14T09:07:57.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00057189700000000003 2024-09-14T09:07:57.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409518+0000", 2024-09-14T09:07:57.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7619999999999995e-06 2024-09-14T09:07:57.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409534+0000", 2024-09-14T09:07:57.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.594e-05 2024-09-14T09:07:57.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409546+0000", 2024-09-14T09:07:57.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2604e-05 2024-09-14T09:07:57.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557048+0000", 2024-09-14T09:07:57.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14750137899999999 2024-09-14T09:07:57.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557117+0000", 2024-09-14T09:07:57.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8998999999999994e-05 2024-09-14T09:07:57.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557490+0000", 2024-09-14T09:07:57.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037301100000000002 2024-09-14T09:07:57.456 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.456 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.456 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.456 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107632 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:57.456 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:07:57.456 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.972009428, 2024-09-14T09:07:57.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14833944199999999, 2024-09-14T09:07:57.456 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.456 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.456 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.456 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.456 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.457 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.457 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107632 2024-09-14T09:07:57.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.457 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:07:57.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:07:57.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412609+0000", 2024-09-14T09:07:57.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003040105 2024-09-14T09:07:57.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412618+0000", 2024-09-14T09:07:57.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2960000000000004e-06 2024-09-14T09:07:57.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412640+0000", 2024-09-14T09:07:57.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1773999999999999e-05 2024-09-14T09:07:57.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412655+0000", 2024-09-14T09:07:57.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4538e-05 2024-09-14T09:07:57.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557538+0000", 2024-09-14T09:07:57.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14488311800000001 2024-09-14T09:07:57.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557594+0000", 2024-09-14T09:07:57.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6122999999999998e-05 2024-09-14T09:07:57.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557908+0000", 2024-09-14T09:07:57.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031448799999999997 2024-09-14T09:07:57.460 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.460 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.460 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.461 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107634 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:57.461 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:07:57.461 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.968891780999996, 2024-09-14T09:07:57.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.145666186, 2024-09-14T09:07:57.461 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.461 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.461 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.461 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.461 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.462 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.462 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107634 2024-09-14T09:07:57.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.462 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:07:57.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:07:57.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412688+0000", 2024-09-14T09:07:57.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9000000000000005e-07 2024-09-14T09:07:57.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412707+0000", 2024-09-14T09:07:57.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9232000000000002e-05 2024-09-14T09:07:57.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412722+0000", 2024-09-14T09:07:57.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5722e-05 2024-09-14T09:07:57.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412733+0000", 2024-09-14T09:07:57.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0909e-05 2024-09-14T09:07:57.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557946+0000", 2024-09-14T09:07:57.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14521263100000001 2024-09-14T09:07:57.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558020+0000", 2024-09-14T09:07:57.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4294000000000004e-05 2024-09-14T09:07:57.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558353+0000", 2024-09-14T09:07:57.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033240800000000002 2024-09-14T09:07:57.465 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.465 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.465 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.466 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107636 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:57.466 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:07:57.466 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.968827478000001, 2024-09-14T09:07:57.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14599377399999999, 2024-09-14T09:07:57.466 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.466 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.466 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.466 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.466 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.466 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.466 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107636 2024-09-14T09:07:57.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.467 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:07:57.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:07:57.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412752+0000", 2024-09-14T09:07:57.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.156e-06 2024-09-14T09:07:57.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412770+0000", 2024-09-14T09:07:57.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7521999999999999e-05 2024-09-14T09:07:57.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412790+0000", 2024-09-14T09:07:57.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0942e-05 2024-09-14T09:07:57.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412809+0000", 2024-09-14T09:07:57.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8887999999999999e-05 2024-09-14T09:07:57.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558387+0000", 2024-09-14T09:07:57.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14557798799999999 2024-09-14T09:07:57.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558448+0000", 2024-09-14T09:07:57.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0862000000000001e-05 2024-09-14T09:07:57.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558745+0000", 2024-09-14T09:07:57.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029641599999999998 2024-09-14T09:07:57.470 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.470 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.470 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.470 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107638 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:57.470 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:07:57.470 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.968740222000001, 2024-09-14T09:07:57.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146344799, 2024-09-14T09:07:57.471 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.471 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.471 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.471 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.471 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.471 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.471 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107638 2024-09-14T09:07:57.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.471 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:07:57.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:07:57.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412839+0000", 2024-09-14T09:07:57.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0720000000000001e-06 2024-09-14T09:07:57.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412844+0000", 2024-09-14T09:07:57.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7759999999999997e-06 2024-09-14T09:07:57.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412857+0000", 2024-09-14T09:07:57.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3138e-05 2024-09-14T09:07:57.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412867+0000", 2024-09-14T09:07:57.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7229999999999999e-06 2024-09-14T09:07:57.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558785+0000", 2024-09-14T09:07:57.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14591828700000001 2024-09-14T09:07:57.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558833+0000", 2024-09-14T09:07:57.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7790000000000002e-05 2024-09-14T09:07:57.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559183+0000", 2024-09-14T09:07:57.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035001299999999998 2024-09-14T09:07:57.474 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.474 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.474 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.475 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107640 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:57.475 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:07:57.475 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.968695070999999, 2024-09-14T09:07:57.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146686804, 2024-09-14T09:07:57.475 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.475 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.475 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.475 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.475 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.475 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.475 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107640 2024-09-14T09:07:57.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.476 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:07:57.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:07:57.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420446+0000", 2024-09-14T09:07:57.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075632529999999998 2024-09-14T09:07:57.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420456+0000", 2024-09-14T09:07:57.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0850000000000003e-06 2024-09-14T09:07:57.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420475+0000", 2024-09-14T09:07:57.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9406e-05 2024-09-14T09:07:57.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420488+0000", 2024-09-14T09:07:57.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3278000000000001e-05 2024-09-14T09:07:57.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559205+0000", 2024-09-14T09:07:57.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13871644999999999 2024-09-14T09:07:57.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559252+0000", 2024-09-14T09:07:57.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7666000000000001e-05 2024-09-14T09:07:57.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559570+0000", 2024-09-14T09:07:57.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031766599999999998 2024-09-14T09:07:57.479 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.479 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.479 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.480 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107660 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 124928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:07:57.480 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:07:57.480 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.960624828999997, 2024-09-14T09:07:57.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.142954628, 2024-09-14T09:07:57.480 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.480 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.480 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.480 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.480 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.480 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.480 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107660 2024-09-14T09:07:57.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.481 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:07:57.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:07:57.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428190+0000", 2024-09-14T09:07:57.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072360339999999997 2024-09-14T09:07:57.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428398+0000", 2024-09-14T09:07:57.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020898499999999999 2024-09-14T09:07:57.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428458+0000", 2024-09-14T09:07:57.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9985999999999998e-05 2024-09-14T09:07:57.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428480+0000", 2024-09-14T09:07:57.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1284000000000002e-05 2024-09-14T09:07:57.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563583+0000", 2024-09-14T09:07:57.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13510361800000001 2024-09-14T09:07:57.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563644+0000", 2024-09-14T09:07:57.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1092000000000004e-05 2024-09-14T09:07:57.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563908+0000", 2024-09-14T09:07:57.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026362899999999998 2024-09-14T09:07:57.484 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.484 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.484 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.484 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.484 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.484 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:57.486 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:07:57.486 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:57.490+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:57.587 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:57.587+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:57.660 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:57.660 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:57.661 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:07:57.661 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:57.674 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:07:57.718 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:57.722+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:57.788 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:57.790+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:57.798 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:57.798 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:07:57.798 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:07:57.798 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:07:57.798 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:57.812 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:07:57.898 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:57.902+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:57.977 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:07:57.977 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:07:57.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:07:57.978 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:07:57.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.978 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38593 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [watch reconnect cookie 139718844451680 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:07:57.978 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:07:57.978 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 116.162361416, 2024-09-14T09:07:57.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0218296630000001, 2024-09-14T09:07:57.978 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.978 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.978 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.978 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.978 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.978 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.978 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38593 2024-09-14T09:07:57.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.979 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:07:57.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:07:57.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.980 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818420+0000", 2024-09-14T09:07:57.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8399999999999999e-06 2024-09-14T09:07:57.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.980 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.980 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818431+0000", 2024-09-14T09:07:57.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0859999999999999e-05 2024-09-14T09:07:57.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.980 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.980 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818466+0000", 2024-09-14T09:07:57.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4486999999999997e-05 2024-09-14T09:07:57.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.981 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818481+0000", 2024-09-14T09:07:57.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5362999999999998e-05 2024-09-14T09:07:57.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.981 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818659+0000", 2024-09-14T09:07:57.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017807500000000001 2024-09-14T09:07:57.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op must wait for map", 2024-09-14T09:07:57.981 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818681+0000", 2024-09-14T09:07:57.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2096999999999999e-05 2024-09-14T09:07:57.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825472+0000", 2024-09-14T09:07:57.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067905170000000003 2024-09-14T09:07:57.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:57.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825491+0000", 2024-09-14T09:07:57.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9372999999999999e-05 2024-09-14T09:07:57.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835402+0000", 2024-09-14T09:07:57.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0099111620000001 2024-09-14T09:07:57.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835548+0000", 2024-09-14T09:07:57.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014607300000000001 2024-09-14T09:07:57.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.840247+0000", 2024-09-14T09:07:57.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046988159999999998 2024-09-14T09:07:57.983 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.984 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.984 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.984 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38615 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:07:57.984 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:07:57.984 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 116.113996208, 2024-09-14T09:07:57.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.97014128899999996, 2024-09-14T09:07:57.984 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.984 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.984 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:07:57.984 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.984 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.985 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.985 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38615 2024-09-14T09:07:57.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.985 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:07:57.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:07:57.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.986 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866787+0000", 2024-09-14T09:07:57.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.476e-06 2024-09-14T09:07:57.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.986 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866806+0000", 2024-09-14T09:07:57.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8606000000000001e-05 2024-09-14T09:07:57.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866839+0000", 2024-09-14T09:07:57.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3510000000000003e-05 2024-09-14T09:07:57.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866867+0000", 2024-09-14T09:07:57.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7610000000000002e-05 2024-09-14T09:07:57.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867031+0000", 2024-09-14T09:07:57.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016414199999999999 2024-09-14T09:07:57.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:57.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867054+0000", 2024-09-14T09:07:57.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2693000000000001e-05 2024-09-14T09:07:57.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836462+0000", 2024-09-14T09:07:57.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.96940797700000003 2024-09-14T09:07:57.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:07:57.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836547+0000", 2024-09-14T09:07:57.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5294e-05 2024-09-14T09:07:57.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836924+0000", 2024-09-14T09:07:57.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037698099999999997 2024-09-14T09:07:57.989 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.989 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.989 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.989 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105723 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:57.989 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:07:57.989 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.885562164, 2024-09-14T09:07:57.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77512799700000001, 2024-09-14T09:07:57.990 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.990 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.990 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:57.990 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.990 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.990 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.990 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105723 2024-09-14T09:07:57.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.990 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:07:57.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:07:57.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:07:57.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1399999999999997e-07 2024-09-14T09:07:57.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095220+0000", 2024-09-14T09:07:57.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0510000000000001e-06 2024-09-14T09:07:57.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095228+0000", 2024-09-14T09:07:57.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4159999999999998e-06 2024-09-14T09:07:57.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095234+0000", 2024-09-14T09:07:57.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3010000000000003e-06 2024-09-14T09:07:57.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103902+0000", 2024-09-14T09:07:57.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086678299999999996 2024-09-14T09:07:57.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:57.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103905+0000", 2024-09-14T09:07:57.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5020000000000001e-06 2024-09-14T09:07:57.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870332+0000", 2024-09-14T09:07:57.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76642700900000005 2024-09-14T09:07:57.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870345+0000", 2024-09-14T09:07:57.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2274e-05 2024-09-14T09:07:57.994 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.994 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.994 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.994 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105724 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:57.994 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:07:57.994 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.885537651, 2024-09-14T09:07:57.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77515698099999997, 2024-09-14T09:07:57.994 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:57.995 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:57.995 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:57.995 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:57.995 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:57.995 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:57.995 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105724 2024-09-14T09:07:57.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.995 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:57.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:57.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:07:57.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:57.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:07:57.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:57.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:57.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095242+0000", 2024-09-14T09:07:57.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9400000000000001e-07 2024-09-14T09:07:57.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:57.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095243+0000", 2024-09-14T09:07:57.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7540000000000001e-06 2024-09-14T09:07:57.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:57.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095248+0000", 2024-09-14T09:07:57.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1320000000000004e-06 2024-09-14T09:07:57.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:57.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095251+0000", 2024-09-14T09:07:57.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2770000000000001e-06 2024-09-14T09:07:57.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103947+0000", 2024-09-14T09:07:57.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086960089999999993 2024-09-14T09:07:57.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:57.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103955+0000", 2024-09-14T09:07:57.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9519999999999994e-06 2024-09-14T09:07:57.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:57.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870386+0000", 2024-09-14T09:07:57.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76643141100000001 2024-09-14T09:07:57.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:57.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870398+0000", 2024-09-14T09:07:57.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2052000000000001e-05 2024-09-14T09:07:57.999 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.999 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:57.999 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:57.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:57.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:57.999 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105727 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:57.999 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:07:58.000 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.885497944999997, 2024-09-14T09:07:58.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517292900000001, 2024-09-14T09:07:58.000 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:58.000 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:58.000 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:58.000 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:58.000 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:58.000 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:58.000 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105727 2024-09-14T09:07:58.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.000 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:58.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:58.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:07:58.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:58.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:07:58.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:58.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:07:58.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3300000000000002e-07 2024-09-14T09:07:58.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:58.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095284+0000", 2024-09-14T09:07:58.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9790000000000002e-06 2024-09-14T09:07:58.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:58.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095292+0000", 2024-09-14T09:07:58.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3810000000000001e-06 2024-09-14T09:07:58.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:58.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095298+0000", 2024-09-14T09:07:58.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9710000000000001e-06 2024-09-14T09:07:58.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103987+0000", 2024-09-14T09:07:58.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086893820000000007 2024-09-14T09:07:58.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:58.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103991+0000", 2024-09-14T09:07:58.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5480000000000002e-06 2024-09-14T09:07:58.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870442+0000", 2024-09-14T09:07:58.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.766451145 2024-09-14T09:07:58.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:58.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870454+0000", 2024-09-14T09:07:58.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.199e-05 2024-09-14T09:07:58.005 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.005 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:58.005 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.005 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105729 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:58.005 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:07:58.005 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.885449588, 2024-09-14T09:07:58.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517686500000005, 2024-09-14T09:07:58.005 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:58.005 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:58.005 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105729 2024-09-14T09:07:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:07:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:07:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095330+0000", 2024-09-14T09:07:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-09-14T09:07:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095332+0000", 2024-09-14T09:07:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1560000000000002e-06 2024-09-14T09:07:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:58.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095338+0000", 2024-09-14T09:07:58.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.592e-06 2024-09-14T09:07:58.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:58.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095341+0000", 2024-09-14T09:07:58.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8139999999999998e-06 2024-09-14T09:07:58.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104031+0000", 2024-09-14T09:07:58.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086896330000000004 2024-09-14T09:07:58.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:58.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104039+0000", 2024-09-14T09:07:58.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0900000000000005e-06 2024-09-14T09:07:58.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870495+0000", 2024-09-14T09:07:58.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76645611199999997 2024-09-14T09:07:58.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:58.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870506+0000", 2024-09-14T09:07:58.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0986e-05 2024-09-14T09:07:58.010 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.010 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:58.010 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.010 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105731 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:58.010 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:07:58.010 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.885431320999999, 2024-09-14T09:07:58.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77520598600000001, 2024-09-14T09:07:58.010 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:58.010 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:58.010 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:58.010 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:58.011 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:58.011 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:58.011 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105731 2024-09-14T09:07:58.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.011 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:58.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:58.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:07:58.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:58.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:07:58.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:58.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:07:58.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8599999999999999e-07 2024-09-14T09:07:58.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:58.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095350+0000", 2024-09-14T09:07:58.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1110000000000002e-06 2024-09-14T09:07:58.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:58.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095357+0000", 2024-09-14T09:07:58.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9020000000000002e-06 2024-09-14T09:07:58.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:58.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095360+0000", 2024-09-14T09:07:58.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1939999999999998e-06 2024-09-14T09:07:58.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104073+0000", 2024-09-14T09:07:58.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087131210000000008 2024-09-14T09:07:58.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:58.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104077+0000", 2024-09-14T09:07:58.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.501e-06 2024-09-14T09:07:58.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870541+0000", 2024-09-14T09:07:58.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76646461700000001 2024-09-14T09:07:58.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:58.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870554+0000", 2024-09-14T09:07:58.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2153999999999999e-05 2024-09-14T09:07:58.014 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.014 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:58.015 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.015 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105733 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:58.015 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:07:58.015 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.88541129, 2024-09-14T09:07:58.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.775237277, 2024-09-14T09:07:58.015 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:58.015 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:58.015 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:58.015 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:58.015 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:58.015 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:58.016 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105733 2024-09-14T09:07:58.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.016 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:58.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:58.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:07:58.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:58.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:07:58.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:58.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:07:58.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8699999999999995e-07 2024-09-14T09:07:58.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:58.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095371+0000", 2024-09-14T09:07:58.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5450000000000002e-06 2024-09-14T09:07:58.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:58.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095377+0000", 2024-09-14T09:07:58.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.776e-06 2024-09-14T09:07:58.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:58.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095383+0000", 2024-09-14T09:07:58.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3129999999999998e-06 2024-09-14T09:07:58.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104124+0000", 2024-09-14T09:07:58.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087409840000000003 2024-09-14T09:07:58.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:58.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104132+0000", 2024-09-14T09:07:58.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3159999999999997e-06 2024-09-14T09:07:58.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870593+0000", 2024-09-14T09:07:58.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76646064800000002 2024-09-14T09:07:58.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:58.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870605+0000", 2024-09-14T09:07:58.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2208000000000001e-05 2024-09-14T09:07:58.019 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.019 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:58.019 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.020 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105734 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:58.020 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:07:58.020 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.885388208000002, 2024-09-14T09:07:58.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77526569599999995, 2024-09-14T09:07:58.020 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:58.020 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:58.020 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:58.020 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:58.020 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:58.020 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:58.020 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105734 2024-09-14T09:07:58.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.021 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:58.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:58.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:07:58.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:58.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:07:58.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:58.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:07:58.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7399999999999998e-07 2024-09-14T09:07:58.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:58.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095394+0000", 2024-09-14T09:07:58.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8030000000000001e-06 2024-09-14T09:07:58.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:58.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095401+0000", 2024-09-14T09:07:58.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.046e-06 2024-09-14T09:07:58.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:58.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095407+0000", 2024-09-14T09:07:58.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6060000000000002e-06 2024-09-14T09:07:58.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104167+0000", 2024-09-14T09:07:58.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087602530000000008 2024-09-14T09:07:58.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:58.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104171+0000", 2024-09-14T09:07:58.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7620000000000001e-06 2024-09-14T09:07:58.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870645+0000", 2024-09-14T09:07:58.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76647400600000004 2024-09-14T09:07:58.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:58.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870656+0000", 2024-09-14T09:07:58.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1746000000000001e-05 2024-09-14T09:07:58.024 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.025 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:58.025 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.025 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105738 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:58.025 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:07:58.025 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.885348481000001, 2024-09-14T09:07:58.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77527530700000002, 2024-09-14T09:07:58.025 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:58.025 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:58.025 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:58.025 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:58.026 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:58.026 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:58.026 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105738 2024-09-14T09:07:58.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.026 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:58.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:58.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:07:58.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:58.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:07:58.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:58.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095431+0000", 2024-09-14T09:07:58.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0999999999999999e-07 2024-09-14T09:07:58.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:58.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095434+0000", 2024-09-14T09:07:58.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.757e-06 2024-09-14T09:07:58.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:58.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095442+0000", 2024-09-14T09:07:58.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9929999999999995e-06 2024-09-14T09:07:58.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:58.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095448+0000", 2024-09-14T09:07:58.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1340000000000001e-06 2024-09-14T09:07:58.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104213+0000", 2024-09-14T09:07:58.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087650000000000002 2024-09-14T09:07:58.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:58.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104221+0000", 2024-09-14T09:07:58.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4589999999999998e-06 2024-09-14T09:07:58.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870694+0000", 2024-09-14T09:07:58.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76647303200000005 2024-09-14T09:07:58.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:58.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870706+0000", 2024-09-14T09:07:58.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1522000000000001e-05 2024-09-14T09:07:58.030 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.030 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:58.030 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.030 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105739 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:58.030 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:07:58.030 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.885323952, 2024-09-14T09:07:58.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77530221700000002, 2024-09-14T09:07:58.030 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:58.031 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:58.031 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:58.031 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:58.031 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:58.031 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:58.031 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105739 2024-09-14T09:07:58.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.031 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:58.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:58.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:07:58.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:58.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:07:58.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:58.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:07:58.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.15e-07 2024-09-14T09:07:58.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:58.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095457+0000", 2024-09-14T09:07:58.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6470000000000001e-06 2024-09-14T09:07:58.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:58.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095461+0000", 2024-09-14T09:07:58.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0910000000000003e-06 2024-09-14T09:07:58.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:58.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095464+0000", 2024-09-14T09:07:58.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2059999999999999e-06 2024-09-14T09:07:58.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104253+0000", 2024-09-14T09:07:58.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087884120000000007 2024-09-14T09:07:58.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:58.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104256+0000", 2024-09-14T09:07:58.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5439999999999999e-06 2024-09-14T09:07:58.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870745+0000", 2024-09-14T09:07:58.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76648908000000004 2024-09-14T09:07:58.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:58.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870757+0000", 2024-09-14T09:07:58.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1822e-05 2024-09-14T09:07:58.035 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.035 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:58.035 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.035 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105741 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:58.035 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:07:58.035 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.885254826000001, 2024-09-14T09:07:58.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77528469899999997, 2024-09-14T09:07:58.035 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:58.035 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:58.035 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:58.036 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:58.036 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:58.036 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:58.036 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105741 2024-09-14T09:07:58.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.036 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:58.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:58.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:07:58.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:58.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:07:58.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:58.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095525+0000", 2024-09-14T09:07:58.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3740000000000001e-06 2024-09-14T09:07:58.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:58.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095533+0000", 2024-09-14T09:07:58.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6380000000000004e-06 2024-09-14T09:07:58.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:58.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095549+0000", 2024-09-14T09:07:58.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5554999999999999e-05 2024-09-14T09:07:58.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:58.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095566+0000", 2024-09-14T09:07:58.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7146000000000002e-05 2024-09-14T09:07:58.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104297+0000", 2024-09-14T09:07:58.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087315750000000001 2024-09-14T09:07:58.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:58.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104305+0000", 2024-09-14T09:07:58.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1389999999999995e-06 2024-09-14T09:07:58.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870797+0000", 2024-09-14T09:07:58.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76649129599999999 2024-09-14T09:07:58.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:58.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870809+0000", 2024-09-14T09:07:58.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1976e-05 2024-09-14T09:07:58.039 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.039 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:58.039 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.040 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105742 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:58.040 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:07:58.040 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.885190041999998, 2024-09-14T09:07:58.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.775271873, 2024-09-14T09:07:58.040 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:58.040 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:58.040 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:58.040 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:58.040 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:58.040 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:58.040 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105742 2024-09-14T09:07:58.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.041 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:58.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:58.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:07:58.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:58.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:07:58.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:58.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095590+0000", 2024-09-14T09:07:58.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0400000000000005e-07 2024-09-14T09:07:58.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:58.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095594+0000", 2024-09-14T09:07:58.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5639999999999998e-06 2024-09-14T09:07:58.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:58.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095620+0000", 2024-09-14T09:07:58.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5494999999999999e-05 2024-09-14T09:07:58.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:58.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095639+0000", 2024-09-14T09:07:58.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9639999999999998e-05 2024-09-14T09:07:58.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104340+0000", 2024-09-14T09:07:58.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087004950000000008 2024-09-14T09:07:58.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:58.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104343+0000", 2024-09-14T09:07:58.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4020000000000002e-06 2024-09-14T09:07:58.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870850+0000", 2024-09-14T09:07:58.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76650648499999996 2024-09-14T09:07:58.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:58.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870861+0000", 2024-09-14T09:07:58.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0888e-05 2024-09-14T09:07:58.044 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.044 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:58.044 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.044 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105745 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:58.045 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:07:58.045 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.885091426999999, 2024-09-14T09:07:58.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77521998700000005, 2024-09-14T09:07:58.045 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:58.045 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:58.045 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:58.045 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:58.045 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:58.045 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:58.045 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105745 2024-09-14T09:07:58.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.045 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:58.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:58.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:07:58.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:58.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:07:58.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:58.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095688+0000", 2024-09-14T09:07:58.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2699999999999998e-07 2024-09-14T09:07:58.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:58.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095693+0000", 2024-09-14T09:07:58.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0479999999999998e-06 2024-09-14T09:07:58.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:58.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095707+0000", 2024-09-14T09:07:58.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.342e-05 2024-09-14T09:07:58.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:58.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095716+0000", 2024-09-14T09:07:58.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9800000000000004e-06 2024-09-14T09:07:58.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104388+0000", 2024-09-14T09:07:58.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086717349999999999 2024-09-14T09:07:58.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:58.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104396+0000", 2024-09-14T09:07:58.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1769999999999998e-06 2024-09-14T09:07:58.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870897+0000", 2024-09-14T09:07:58.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76650102600000003 2024-09-14T09:07:58.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:58.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870907+0000", 2024-09-14T09:07:58.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0674e-05 2024-09-14T09:07:58.049 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.049 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:58.049 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.049 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105747 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:58.049 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:07:58.049 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.885045206999997, 2024-09-14T09:07:58.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77522190300000005, 2024-09-14T09:07:58.050 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:58.050 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:58.050 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:58.050 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:58.050 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:58.050 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:58.050 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105747 2024-09-14T09:07:58.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.050 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:58.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:58.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:07:58.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:58.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:07:58.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:58.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:07:58.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8999999999999995e-07 2024-09-14T09:07:58.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:58.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095738+0000", 2024-09-14T09:07:58.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8430000000000003e-06 2024-09-14T09:07:58.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:58.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095750+0000", 2024-09-14T09:07:58.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1229999999999999e-05 2024-09-14T09:07:58.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:58.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095758+0000", 2024-09-14T09:07:58.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.337e-06 2024-09-14T09:07:58.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104431+0000", 2024-09-14T09:07:58.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086728780000000002 2024-09-14T09:07:58.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:58.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104434+0000", 2024-09-14T09:07:58.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5049999999999999e-06 2024-09-14T09:07:58.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870945+0000", 2024-09-14T09:07:58.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651031800000002 2024-09-14T09:07:58.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:58.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870956+0000", 2024-09-14T09:07:58.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1002000000000001e-05 2024-09-14T09:07:58.054 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.054 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:58.054 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.054 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105750 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:58.054 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:07:58.054 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.884975261000001, 2024-09-14T09:07:58.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77519961800000003, 2024-09-14T09:07:58.054 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:58.055 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:58.055 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:58.055 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:58.055 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:58.055 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:58.055 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105750 2024-09-14T09:07:58.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.055 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:58.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:58.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:07:58.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:58.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:07:58.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:58.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:07:58.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3600000000000002e-07 2024-09-14T09:07:58.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:58.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095809+0000", 2024-09-14T09:07:58.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4000000000000002e-06 2024-09-14T09:07:58.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:58.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095822+0000", 2024-09-14T09:07:58.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2704000000000001e-05 2024-09-14T09:07:58.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:58.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095831+0000", 2024-09-14T09:07:58.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9530000000000005e-06 2024-09-14T09:07:58.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104477+0000", 2024-09-14T09:07:58.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086459989999999997 2024-09-14T09:07:58.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:58.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104488+0000", 2024-09-14T09:07:58.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:07:58.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870992+0000", 2024-09-14T09:07:58.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.766504097 2024-09-14T09:07:58.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:58.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871003+0000", 2024-09-14T09:07:58.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1358999999999999e-05 2024-09-14T09:07:58.059 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.059 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:58.059 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.059 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105751 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:58.059 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:07:58.059 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.884917375999997, 2024-09-14T09:07:58.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77518966700000003, 2024-09-14T09:07:58.059 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:58.060 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:58.060 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:58.060 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:58.060 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:58.060 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:58.060 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105751 2024-09-14T09:07:58.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.060 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:58.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:58.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:07:58.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:58.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:07:58.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:58.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:07:58.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7300000000000005e-07 2024-09-14T09:07:58.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:58.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095867+0000", 2024-09-14T09:07:58.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2729999999999996e-06 2024-09-14T09:07:58.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:58.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095879+0000", 2024-09-14T09:07:58.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2917e-05 2024-09-14T09:07:58.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:58.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095888+0000", 2024-09-14T09:07:58.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2609999999999993e-06 2024-09-14T09:07:58.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104520+0000", 2024-09-14T09:07:58.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086322970000000006 2024-09-14T09:07:58.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:58.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104524+0000", 2024-09-14T09:07:58.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.54e-06 2024-09-14T09:07:58.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871040+0000", 2024-09-14T09:07:58.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651615399999995 2024-09-14T09:07:58.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:58.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871051+0000", 2024-09-14T09:07:58.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1452e-05 2024-09-14T09:07:58.064 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.064 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:58.064 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.064 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105752 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:58.064 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:07:58.064 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.884856612, 2024-09-14T09:07:58.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517684899999995, 2024-09-14T09:07:58.064 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:58.064 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:58.065 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:58.065 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:58.065 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:58.065 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:58.065 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105752 2024-09-14T09:07:58.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.065 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:58.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:58.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:07:58.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:58.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:07:58.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:58.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095923+0000", 2024-09-14T09:07:58.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3300000000000001e-07 2024-09-14T09:07:58.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:58.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095927+0000", 2024-09-14T09:07:58.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:07:58.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:58.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095940+0000", 2024-09-14T09:07:58.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2309e-05 2024-09-14T09:07:58.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:58.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095948+0000", 2024-09-14T09:07:58.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9199999999999993e-06 2024-09-14T09:07:58.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104566+0000", 2024-09-14T09:07:58.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086171449999999997 2024-09-14T09:07:58.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:58.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104574+0000", 2024-09-14T09:07:58.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0690000000000002e-06 2024-09-14T09:07:58.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871087+0000", 2024-09-14T09:07:58.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651370699999999 2024-09-14T09:07:58.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:58.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871099+0000", 2024-09-14T09:07:58.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1718e-05 2024-09-14T09:07:58.068 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.068 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:58.069 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.069 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105755 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:58.069 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:07:58.069 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.884798668999998, 2024-09-14T09:07:58.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77516787899999995, 2024-09-14T09:07:58.069 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:58.069 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:58.069 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:58.069 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:58.069 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:58.069 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:58.070 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105755 2024-09-14T09:07:58.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.070 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:58.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:58.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:07:58.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:58.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:07:58.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:58.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095981+0000", 2024-09-14T09:07:58.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2699999999999998e-07 2024-09-14T09:07:58.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:58.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095985+0000", 2024-09-14T09:07:58.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4349999999999999e-06 2024-09-14T09:07:58.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:58.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095999+0000", 2024-09-14T09:07:58.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3139000000000001e-05 2024-09-14T09:07:58.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:58.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096007+0000", 2024-09-14T09:07:58.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6610000000000006e-06 2024-09-14T09:07:58.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104591+0000", 2024-09-14T09:07:58.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0085841709999999998 2024-09-14T09:07:58.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:07:58.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104595+0000", 2024-09-14T09:07:58.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5590000000000001e-06 2024-09-14T09:07:58.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871137+0000", 2024-09-14T09:07:58.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76654202100000002 2024-09-14T09:07:58.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:58.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871148+0000", 2024-09-14T09:07:58.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1066e-05 2024-09-14T09:07:58.073 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.073 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:58.074 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.074 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105757 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:07:58.074 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:07:58.074 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.884740088999997, 2024-09-14T09:07:58.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77516401300000004, 2024-09-14T09:07:58.074 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:07:58.074 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:07:58.074 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:07:58.074 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:07:58.074 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:07:58.074 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:58.075 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105757 2024-09-14T09:07:58.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.075 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:07:58.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:07:58.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:07:58.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:07:58.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:07:58.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:07:58.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:07:58.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856228+0000", 2024-09-14T09:07:58.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76018875100000005 2024-09-14T09:07:58.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:07:58.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856240+0000", 2024-09-14T09:07:58.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-05 2024-09-14T09:07:58.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:07:58.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856268+0000", 2024-09-14T09:07:58.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7628999999999999e-05 2024-09-14T09:07:58.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:07:58.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856307+0000", 2024-09-14T09:07:58.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8967000000000001e-05 2024-09-14T09:07:58.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:07:58.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871192+0000", 2024-09-14T09:07:58.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.01488449 2024-09-14T09:07:58.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:07:58.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:07:58.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:07:58.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871203+0000", 2024-09-14T09:07:58.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1266e-05 2024-09-14T09:07:58.078 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.078 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:58.078 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.078 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:07:58.078 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:07:58.078 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:07:58.079 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:07:58.080 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:58.082+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:58.080 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:07:58.103 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is unset 2024-09-14T09:07:58.181 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:58.182+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:07:58.219 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:58.220 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:07:58.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:58.220 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:141970 3.12 3:486f9092:::rbd_object_map.115e6d20f54d:head [call lock.lock in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-14T09:07:58.220 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:58.221310+0000", 2024-09-14T09:07:58.220 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 0.00030262900000000001, 2024-09-14T09:07:58.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00032046199999999998, 2024-09-14T09:07:58.220 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:07:58.220 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:07:58.220 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:07:58.220 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:07:58.220 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:07:58.221 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:07:58.221 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 141970 2024-09-14T09:07:58.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:58.221 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:07:58.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:58.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:07:58.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:58.221310+0000", 2024-09-14T09:07:58.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:58.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:58.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:58.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:07:58.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:58.221310+0000", 2024-09-14T09:07:58.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:07:58.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:58.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:58.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:07:58.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:58.221311+0000", 2024-09-14T09:07:58.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0139999999999999e-06 2024-09-14T09:07:58.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:58.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:58.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:07:58.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:58.221315+0000", 2024-09-14T09:07:58.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7040000000000002e-06 2024-09-14T09:07:58.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:58.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:58.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:07:58.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:58.221325+0000", 2024-09-14T09:07:58.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3889999999999994e-06 2024-09-14T09:07:58.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:58.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:58.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:07:58.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:58.221330+0000", 2024-09-14T09:07:58.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0490000000000003e-06 2024-09-14T09:07:58.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:58.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:58.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:07:58.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:58.221384+0000", 2024-09-14T09:07:58.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4542000000000001e-05 2024-09-14T09:07:58.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:07:58.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:07:58.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:07:58.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:58.221479+0000", 2024-09-14T09:07:58.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4840000000000007e-05 2024-09-14T09:07:58.224 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:58.224 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:07:58.224 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:58.224 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:07:58.224 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:07:58.224 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 1 2024-09-14T09:07:58.224 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:58.239 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:07:58.249 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:07:58.249 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:07:58.267 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:07:58.267 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:07:58.282 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:58.283+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:58.286 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:07:58.286 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:07:58.307 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:07:58.308 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:07:58.308 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:07:58.326 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-14T09:07:58.327 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:07:58.327 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-14T09:07:58.327 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T09:07:58.327 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T09:07:58.327 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-14T09:07:58.330 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:07:58.330 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:07:58.353 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:07:58.353 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:07:58.359 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:58.359 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:58.374 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:07:58.374 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:07:58.375 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:07:58.377 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:07:58.378 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T09:07:58.427 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:58.427+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:58.445 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:07:58.445 INFO:teuthology.orchestra.run.smithi176.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T09:07:58.445 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:07:58.484 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:58.486+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:58.540 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:58.540 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:58.554 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:07:58.586 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:58.590+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:58.674 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:58.674 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:58.686 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:58.690+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:58.688 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:07:58.787 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:58.787+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:58.804 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:58.804 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:58.817 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:07:58.902 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:58.903+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:58.967 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:58.967 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:58.979 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:07:59.022 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:07:59.023+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:07:59.090 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:59.091+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:59.144 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:59.144 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:59.158 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:07:59.190 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:07:59.194+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:07:59.291 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:59.291+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:59.329 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:59.329 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:59.346 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:07:59.392 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:07:59.394+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:07:59.493 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:07:59.491+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:07:59.508 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:59.508 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:59.524 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:07:59.610 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:07:59.611+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:07:59.684 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:59.685 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:59.697 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:59.695+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:59.698 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:07:59.798 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:07:59.799+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:07:59.867 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:07:59.867 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:07:59.881 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:07:59.902 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:07:59.906+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:00.003 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:00.006+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:00.050 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:00.050 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:00.065 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:08:00.103 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:00.103+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:00.135 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T09:08:00.201 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:00.201 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:00.204 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:00.206+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:00.215 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:08:00.304 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:00.303+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:00.378 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:08:00.381 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:00.381 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:00.395 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:08:00.422 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:00.423+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:00.506 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:00.507+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:00.519 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:00.519 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:00.520 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:08:00.520 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:08:00.533 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:08:00.537 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:08:00.537 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:08:00.553 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:08:00.553 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:08:00.570 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:08:00.571 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:08:00.571 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:08:00.589 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:08:00.589 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:08:00.607 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:08:00.607 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:08:00.607 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:00.607+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:00.625 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:08:00.625 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:08:00.630 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:08:00.631 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T09:08:00.657 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:00.657 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:00.658 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:08:00.658 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:00.671 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:08:00.718 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:00.722+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:00.791 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:00.791 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:00.791 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:08:00.791 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:08:00.791 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:00.805 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:08:00.809 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:00.807+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:00.911 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:00.911+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:00.975 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:00.975 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:08:00.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:08:00.976 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:08:00.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.976 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65139 3.a 3:54c90b7d:::rbd_data.115e21c31fcf.0000000000000008:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:08:00.976 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:08:00.976 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 99.599074138000006, 2024-09-14T09:08:00.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14429895700000001, 2024-09-14T09:08:00.976 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:00.976 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:00.976 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:00.976 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:00.976 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:00.976 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:00.976 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65139 2024-09-14T09:08:00.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.977 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:00.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:00.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:08:00.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:00.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:00.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:08:00.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:00.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:00.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379715+0000", 2024-09-14T09:08:00.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.576e-06 2024-09-14T09:08:00.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:00.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379724+0000", 2024-09-14T09:08:00.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1779999999999992e-06 2024-09-14T09:08:00.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:00.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379746+0000", 2024-09-14T09:08:00.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1661000000000001e-05 2024-09-14T09:08:00.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:00.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379759+0000", 2024-09-14T09:08:00.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.324e-05 2024-09-14T09:08:00.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:00.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379882+0000", 2024-09-14T09:08:00.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012239400000000001 2024-09-14T09:08:00.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:00.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.380023+0000", 2024-09-14T09:08:00.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000141402 2024-09-14T09:08:00.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.980 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:08:00.980 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.391553+0000", 2024-09-14T09:08:00.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011530197000000001 2024-09-14T09:08:00.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.980 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:00.980 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523927+0000", 2024-09-14T09:08:00.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.132373935 2024-09-14T09:08:00.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.980 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:08:00.981 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523941+0000", 2024-09-14T09:08:00.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3326000000000001e-05 2024-09-14T09:08:00.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:08:00.981 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523977+0000", 2024-09-14T09:08:00.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6185e-05 2024-09-14T09:08:00.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:00.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.524012+0000", 2024-09-14T09:08:00.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4863000000000001e-05 2024-09-14T09:08:00.982 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:00.982 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:00.982 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:00.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.983 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107606 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 14336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:00.983 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:00.983 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.572617237000003, 2024-09-14T09:08:00.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14293513799999999, 2024-09-14T09:08:00.983 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:00.983 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:00.983 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:00.983 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:00.983 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:00.983 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:00.983 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107606 2024-09-14T09:08:00.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.983 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:00.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:00.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:00.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:00.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:00.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:00.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:00.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:00.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:00.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1600000000000001e-07 2024-09-14T09:08:00.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:00.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406174+0000", 2024-09-14T09:08:00.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4199999999999999e-06 2024-09-14T09:08:00.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:00.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406188+0000", 2024-09-14T09:08:00.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4114000000000001e-05 2024-09-14T09:08:00.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:00.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406198+0000", 2024-09-14T09:08:00.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.041e-05 2024-09-14T09:08:00.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:00.986 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.548700+0000", 2024-09-14T09:08:00.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14250141299999999 2024-09-14T09:08:00.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:00.986 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.548783+0000", 2024-09-14T09:08:00.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3578999999999994e-05 2024-09-14T09:08:00.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:00.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549105+0000", 2024-09-14T09:08:00.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032158600000000001 2024-09-14T09:08:00.987 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:00.987 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:00.987 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:00.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.987 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107608 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:00.987 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:00.987 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.572551201000003, 2024-09-14T09:08:00.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14603042799999999, 2024-09-14T09:08:00.987 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:00.987 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:00.987 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:00.988 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:00.988 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:00.988 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:00.988 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107608 2024-09-14T09:08:00.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.988 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:00.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:00.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:00.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:00.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:00.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:00.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:00.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:00.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:00.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1200000000000002e-07 2024-09-14T09:08:00.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:00.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406241+0000", 2024-09-14T09:08:00.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7539999999999997e-06 2024-09-14T09:08:00.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:00.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406275+0000", 2024-09-14T09:08:00.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3512e-05 2024-09-14T09:08:00.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:00.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406322+0000", 2024-09-14T09:08:00.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6764999999999997e-05 2024-09-14T09:08:00.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:00.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549149+0000", 2024-09-14T09:08:00.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14282725499999999 2024-09-14T09:08:00.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:00.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549198+0000", 2024-09-14T09:08:00.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9599999999999999e-05 2024-09-14T09:08:00.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:00.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552266+0000", 2024-09-14T09:08:00.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030677299999999998 2024-09-14T09:08:00.991 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:00.991 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:00.991 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:00.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.992 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107610 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 22528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:00.992 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:00.992 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.572429325999998, 2024-09-14T09:08:00.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146437919, 2024-09-14T09:08:00.992 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:00.992 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:00.992 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:00.992 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:00.992 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:00.992 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:00.992 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107610 2024-09-14T09:08:00.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.992 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:00.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:00.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:00.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:00.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:00.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:00.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:00.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:00.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:00.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.02e-07 2024-09-14T09:08:00.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:00.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406363+0000", 2024-09-14T09:08:00.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8099999999999997e-06 2024-09-14T09:08:00.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:00.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406390+0000", 2024-09-14T09:08:00.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7004000000000001e-05 2024-09-14T09:08:00.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:00.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406404+0000", 2024-09-14T09:08:00.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3682e-05 2024-09-14T09:08:00.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:00.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552335+0000", 2024-09-14T09:08:00.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14593077800000001 2024-09-14T09:08:00.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:00.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552431+0000", 2024-09-14T09:08:00.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6086000000000001e-05 2024-09-14T09:08:00.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:00.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552795+0000", 2024-09-14T09:08:00.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000364657 2024-09-14T09:08:00.996 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:00.996 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:00.996 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:00.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.996 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107612 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 26624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:00.996 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:08:00.996 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.572364063000002, 2024-09-14T09:08:00.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146762061, 2024-09-14T09:08:00.996 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:00.996 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:00.996 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:00.997 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:00.997 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:00.997 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:00.997 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107612 2024-09-14T09:08:00.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.997 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:00.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:00.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:08:00.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:00.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:00.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:08:00.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:00.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:00.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406424+0000", 2024-09-14T09:08:00.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2600000000000002e-07 2024-09-14T09:08:00.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:00.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406451+0000", 2024-09-14T09:08:00.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7250000000000002e-05 2024-09-14T09:08:00.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:00.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406487+0000", 2024-09-14T09:08:00.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6086e-05 2024-09-14T09:08:00.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:00.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406498+0000", 2024-09-14T09:08:00.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0858e-05 2024-09-14T09:08:00.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:00.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552821+0000", 2024-09-14T09:08:00.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14632319099999999 2024-09-14T09:08:00.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:00.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:00.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552875+0000", 2024-09-14T09:08:01.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4262999999999999e-05 2024-09-14T09:08:01.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553185+0000", 2024-09-14T09:08:01.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000309687 2024-09-14T09:08:01.000 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.000 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.000 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.000 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107614 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:01.001 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:08:01.001 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.572276529, 2024-09-14T09:08:01.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14737121, 2024-09-14T09:08:01.001 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.001 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.001 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:01.001 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.001 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.001 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.001 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107614 2024-09-14T09:08:01.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.001 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:08:01.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:08:01.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406511+0000", 2024-09-14T09:08:01.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1799999999999995e-07 2024-09-14T09:08:01.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406514+0000", 2024-09-14T09:08:01.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8660000000000002e-06 2024-09-14T09:08:01.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406523+0000", 2024-09-14T09:08:01.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1940000000000004e-06 2024-09-14T09:08:01.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406531+0000", 2024-09-14T09:08:01.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3159999999999997e-06 2024-09-14T09:08:01.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553233+0000", 2024-09-14T09:08:01.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14670117899999999 2024-09-14T09:08:01.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553334+0000", 2024-09-14T09:08:01.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100982 2024-09-14T09:08:01.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553882+0000", 2024-09-14T09:08:01.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00054805499999999998 2024-09-14T09:08:01.005 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.005 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.005 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.005 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107616 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 34816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:01.005 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:08:01.005 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.572244894000001, 2024-09-14T09:08:01.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14780390500000001, 2024-09-14T09:08:01.005 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.005 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.005 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:01.005 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.006 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.006 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.006 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107616 2024-09-14T09:08:01.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.006 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:08:01.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:08:01.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406543+0000", 2024-09-14T09:08:01.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0200000000000001e-07 2024-09-14T09:08:01.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406545+0000", 2024-09-14T09:08:01.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3360000000000002e-06 2024-09-14T09:08:01.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406552+0000", 2024-09-14T09:08:01.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9779999999999999e-06 2024-09-14T09:08:01.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406559+0000", 2024-09-14T09:08:01.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6919999999999996e-06 2024-09-14T09:08:01.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553917+0000", 2024-09-14T09:08:01.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14735791600000001 2024-09-14T09:08:01.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554033+0000", 2024-09-14T09:08:01.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116047 2024-09-14T09:08:01.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554346+0000", 2024-09-14T09:08:01.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031323400000000002 2024-09-14T09:08:01.009 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.009 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.009 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.009 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107618 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 38912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:01.009 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:01.010 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.572216093999998, 2024-09-14T09:08:01.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14816009599999999, 2024-09-14T09:08:01.010 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.010 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.010 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:01.010 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.010 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.010 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.010 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107618 2024-09-14T09:08:01.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.010 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:01.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:01.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:01.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9999999999999998e-07 2024-09-14T09:08:01.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406574+0000", 2024-09-14T09:08:01.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2560000000000001e-06 2024-09-14T09:08:01.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406581+0000", 2024-09-14T09:08:01.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1539999999999996e-06 2024-09-14T09:08:01.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406586+0000", 2024-09-14T09:08:01.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.904e-06 2024-09-14T09:08:01.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554385+0000", 2024-09-14T09:08:01.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.147799443 2024-09-14T09:08:01.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554441+0000", 2024-09-14T09:08:01.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5921000000000002e-05 2024-09-14T09:08:01.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554731+0000", 2024-09-14T09:08:01.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028991800000000001 2024-09-14T09:08:01.014 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.014 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.014 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.014 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107620 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:01.014 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:01.014 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.572194312000001, 2024-09-14T09:08:01.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.148512643, 2024-09-14T09:08:01.014 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.014 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.014 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:01.014 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.014 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.014 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.015 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107620 2024-09-14T09:08:01.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.015 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:01.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:01.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:01.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9999999999999998e-07 2024-09-14T09:08:01.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406595+0000", 2024-09-14T09:08:01.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9360000000000002e-06 2024-09-14T09:08:01.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406605+0000", 2024-09-14T09:08:01.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0268000000000001e-05 2024-09-14T09:08:01.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406614+0000", 2024-09-14T09:08:01.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7180000000000002e-06 2024-09-14T09:08:01.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554750+0000", 2024-09-14T09:08:01.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14813599699999999 2024-09-14T09:08:01.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554794+0000", 2024-09-14T09:08:01.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.401e-05 2024-09-14T09:08:01.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555105+0000", 2024-09-14T09:08:01.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031131399999999998 2024-09-14T09:08:01.018 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.018 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.018 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.018 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107622 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:01.018 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:01.019 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.572163056000001, 2024-09-14T09:08:01.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14902820899999999, 2024-09-14T09:08:01.019 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.019 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.019 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:01.019 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.019 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.019 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.019 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107622 2024-09-14T09:08:01.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.019 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:01.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.020 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:01.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.020 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:01.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9200000000000001e-07 2024-09-14T09:08:01.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406636+0000", 2024-09-14T09:08:01.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.189e-05 2024-09-14T09:08:01.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406649+0000", 2024-09-14T09:08:01.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2442e-05 2024-09-14T09:08:01.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406658+0000", 2024-09-14T09:08:01.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3239999999999992e-06 2024-09-14T09:08:01.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555287+0000", 2024-09-14T09:08:01.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14862868400000001 2024-09-14T09:08:01.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555348+0000", 2024-09-14T09:08:01.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1686999999999994e-05 2024-09-14T09:08:01.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555652+0000", 2024-09-14T09:08:01.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00030369000000000001 2024-09-14T09:08:01.023 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.023 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.023 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.023 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107624 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:01.023 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:01.023 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.572119362999999, 2024-09-14T09:08:01.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14934019900000001, 2024-09-14T09:08:01.023 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.023 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.023 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:01.023 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.023 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.024 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.024 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107624 2024-09-14T09:08:01.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.024 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:01.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:01.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:01.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1399999999999997e-07 2024-09-14T09:08:01.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406670+0000", 2024-09-14T09:08:01.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4099999999999998e-06 2024-09-14T09:08:01.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406680+0000", 2024-09-14T09:08:01.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6460000000000005e-06 2024-09-14T09:08:01.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406688+0000", 2024-09-14T09:08:01.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5859999999999999e-06 2024-09-14T09:08:01.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555693+0000", 2024-09-14T09:08:01.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14900544499999999 2024-09-14T09:08:01.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555743+0000", 2024-09-14T09:08:01.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9966999999999998e-05 2024-09-14T09:08:01.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556008+0000", 2024-09-14T09:08:01.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026463099999999998 2024-09-14T09:08:01.027 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.027 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.027 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.027 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107626 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:01.027 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:08:01.027 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.572087389000004, 2024-09-14T09:08:01.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14983149500000001, 2024-09-14T09:08:01.028 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.028 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.028 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:01.028 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.028 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.028 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.028 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107626 2024-09-14T09:08:01.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.028 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:08:01.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:08:01.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408087+0000", 2024-09-14T09:08:01.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001387735 2024-09-14T09:08:01.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408094+0000", 2024-09-14T09:08:01.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9399999999999996e-06 2024-09-14T09:08:01.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408122+0000", 2024-09-14T09:08:01.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8076e-05 2024-09-14T09:08:01.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408147+0000", 2024-09-14T09:08:01.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4726e-05 2024-09-14T09:08:01.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556048+0000", 2024-09-14T09:08:01.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14790144299999999 2024-09-14T09:08:01.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556175+0000", 2024-09-14T09:08:01.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012663 2024-09-14T09:08:01.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556531+0000", 2024-09-14T09:08:01.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035594499999999998 2024-09-14T09:08:01.031 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.031 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.031 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.032 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107628 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:01.032 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:08:01.032 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.570617777999999, 2024-09-14T09:08:01.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14882648900000001, 2024-09-14T09:08:01.032 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.032 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.032 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:01.032 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.032 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.032 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.032 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107628 2024-09-14T09:08:01.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.033 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:08:01.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:08:01.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408878+0000", 2024-09-14T09:08:01.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070858800000000001 2024-09-14T09:08:01.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408885+0000", 2024-09-14T09:08:01.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4839999999999998e-06 2024-09-14T09:08:01.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408900+0000", 2024-09-14T09:08:01.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5174000000000001e-05 2024-09-14T09:08:01.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408918+0000", 2024-09-14T09:08:01.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.808e-05 2024-09-14T09:08:01.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556572+0000", 2024-09-14T09:08:01.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14765399500000001 2024-09-14T09:08:01.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556651+0000", 2024-09-14T09:08:01.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8919999999999997e-05 2024-09-14T09:08:01.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556996+0000", 2024-09-14T09:08:01.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034424800000000001 2024-09-14T09:08:01.036 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.036 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.036 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.036 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107630 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 63488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:01.036 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:08:01.036 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.569848944, 2024-09-14T09:08:01.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14855159200000001, 2024-09-14T09:08:01.036 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.036 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.036 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:01.037 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.037 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.037 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.037 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107630 2024-09-14T09:08:01.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.037 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:08:01.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:08:01.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409510+0000", 2024-09-14T09:08:01.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00057189700000000003 2024-09-14T09:08:01.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409518+0000", 2024-09-14T09:08:01.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7619999999999995e-06 2024-09-14T09:08:01.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409534+0000", 2024-09-14T09:08:01.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.594e-05 2024-09-14T09:08:01.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409546+0000", 2024-09-14T09:08:01.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2604e-05 2024-09-14T09:08:01.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557048+0000", 2024-09-14T09:08:01.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14750137899999999 2024-09-14T09:08:01.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557117+0000", 2024-09-14T09:08:01.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8998999999999994e-05 2024-09-14T09:08:01.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557490+0000", 2024-09-14T09:08:01.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037301100000000002 2024-09-14T09:08:01.040 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.040 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.040 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.040 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107632 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:01.041 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:08:01.041 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.569217999000003, 2024-09-14T09:08:01.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14833944199999999, 2024-09-14T09:08:01.041 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.041 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.041 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:01.041 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.041 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.041 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.041 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107632 2024-09-14T09:08:01.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.041 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:08:01.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:08:01.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412609+0000", 2024-09-14T09:08:01.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003040105 2024-09-14T09:08:01.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412618+0000", 2024-09-14T09:08:01.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2960000000000004e-06 2024-09-14T09:08:01.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412640+0000", 2024-09-14T09:08:01.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1773999999999999e-05 2024-09-14T09:08:01.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412655+0000", 2024-09-14T09:08:01.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4538e-05 2024-09-14T09:08:01.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557538+0000", 2024-09-14T09:08:01.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14488311800000001 2024-09-14T09:08:01.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557594+0000", 2024-09-14T09:08:01.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6122999999999998e-05 2024-09-14T09:08:01.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557908+0000", 2024-09-14T09:08:01.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031448799999999997 2024-09-14T09:08:01.044 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.044 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.045 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.045 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107634 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:01.045 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:08:01.045 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.566100351999999, 2024-09-14T09:08:01.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.145666186, 2024-09-14T09:08:01.045 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.045 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.045 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:01.045 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.045 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.045 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.045 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107634 2024-09-14T09:08:01.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.046 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:08:01.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:08:01.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412688+0000", 2024-09-14T09:08:01.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9000000000000005e-07 2024-09-14T09:08:01.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412707+0000", 2024-09-14T09:08:01.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9232000000000002e-05 2024-09-14T09:08:01.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412722+0000", 2024-09-14T09:08:01.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5722e-05 2024-09-14T09:08:01.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412733+0000", 2024-09-14T09:08:01.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0909e-05 2024-09-14T09:08:01.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557946+0000", 2024-09-14T09:08:01.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14521263100000001 2024-09-14T09:08:01.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558020+0000", 2024-09-14T09:08:01.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4294000000000004e-05 2024-09-14T09:08:01.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558353+0000", 2024-09-14T09:08:01.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033240800000000002 2024-09-14T09:08:01.049 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.049 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.049 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.050 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107636 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:01.050 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:08:01.050 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.566036048999997, 2024-09-14T09:08:01.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14599377399999999, 2024-09-14T09:08:01.050 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.050 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.050 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:01.050 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.050 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.050 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.050 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107636 2024-09-14T09:08:01.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.050 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:08:01.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:08:01.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412752+0000", 2024-09-14T09:08:01.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.156e-06 2024-09-14T09:08:01.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412770+0000", 2024-09-14T09:08:01.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7521999999999999e-05 2024-09-14T09:08:01.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412790+0000", 2024-09-14T09:08:01.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0942e-05 2024-09-14T09:08:01.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412809+0000", 2024-09-14T09:08:01.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8887999999999999e-05 2024-09-14T09:08:01.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558387+0000", 2024-09-14T09:08:01.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14557798799999999 2024-09-14T09:08:01.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558448+0000", 2024-09-14T09:08:01.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0862000000000001e-05 2024-09-14T09:08:01.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558745+0000", 2024-09-14T09:08:01.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029641599999999998 2024-09-14T09:08:01.054 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.054 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.054 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.054 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107638 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:01.054 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:08:01.055 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.565948792999997, 2024-09-14T09:08:01.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146344799, 2024-09-14T09:08:01.055 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.055 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.055 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:01.055 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.055 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.055 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.055 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107638 2024-09-14T09:08:01.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.055 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:08:01.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:08:01.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412839+0000", 2024-09-14T09:08:01.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0720000000000001e-06 2024-09-14T09:08:01.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412844+0000", 2024-09-14T09:08:01.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7759999999999997e-06 2024-09-14T09:08:01.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412857+0000", 2024-09-14T09:08:01.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3138e-05 2024-09-14T09:08:01.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412867+0000", 2024-09-14T09:08:01.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7229999999999999e-06 2024-09-14T09:08:01.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558785+0000", 2024-09-14T09:08:01.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14591828700000001 2024-09-14T09:08:01.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558833+0000", 2024-09-14T09:08:01.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7790000000000002e-05 2024-09-14T09:08:01.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559183+0000", 2024-09-14T09:08:01.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035001299999999998 2024-09-14T09:08:01.058 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.059 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.059 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.059 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107640 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:01.059 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:08:01.059 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.565903642000002, 2024-09-14T09:08:01.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146686804, 2024-09-14T09:08:01.059 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.059 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.059 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:01.059 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.059 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.059 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.060 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107640 2024-09-14T09:08:01.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.060 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:08:01.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:08:01.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420446+0000", 2024-09-14T09:08:01.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075632529999999998 2024-09-14T09:08:01.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420456+0000", 2024-09-14T09:08:01.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0850000000000003e-06 2024-09-14T09:08:01.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420475+0000", 2024-09-14T09:08:01.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9406e-05 2024-09-14T09:08:01.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420488+0000", 2024-09-14T09:08:01.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3278000000000001e-05 2024-09-14T09:08:01.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559205+0000", 2024-09-14T09:08:01.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13871644999999999 2024-09-14T09:08:01.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559252+0000", 2024-09-14T09:08:01.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7666000000000001e-05 2024-09-14T09:08:01.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559570+0000", 2024-09-14T09:08:01.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031766599999999998 2024-09-14T09:08:01.063 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.063 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.063 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.063 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107660 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 124928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:01.063 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:08:01.063 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.5578334, 2024-09-14T09:08:01.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.142954628, 2024-09-14T09:08:01.064 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.064 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.064 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:01.064 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.064 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.064 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.064 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107660 2024-09-14T09:08:01.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.064 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:08:01.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:08:01.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428190+0000", 2024-09-14T09:08:01.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072360339999999997 2024-09-14T09:08:01.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428398+0000", 2024-09-14T09:08:01.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020898499999999999 2024-09-14T09:08:01.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428458+0000", 2024-09-14T09:08:01.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9985999999999998e-05 2024-09-14T09:08:01.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428480+0000", 2024-09-14T09:08:01.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1284000000000002e-05 2024-09-14T09:08:01.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563583+0000", 2024-09-14T09:08:01.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13510361800000001 2024-09-14T09:08:01.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563644+0000", 2024-09-14T09:08:01.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1092000000000004e-05 2024-09-14T09:08:01.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563908+0000", 2024-09-14T09:08:01.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026362899999999998 2024-09-14T09:08:01.067 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.068 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.068 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.068 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.068 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.068 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:01.069 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:08:01.070 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:01.070+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:01.125 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is set 2024-09-14T09:08:01.206 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:01.207+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:01.248 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:01.248 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:08:01.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.248 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:142495 3.d 3:b6eb827e:::rbd_data.115e63832e35.0000000000000018:head [delete] snapc 23=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-14T09:08:01.248 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:08:01.253003+0000", 2024-09-14T09:08:01.248 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.00079524000000000001, 2024-09-14T09:08:01.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00082069, 2024-09-14T09:08:01.248 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.248 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.248 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:01.248 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.249 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.249 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.249 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 142495 2024-09-14T09:08:01.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.249 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:08:01.253003+0000", 2024-09-14T09:08:01.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:08:01.253003+0000", 2024-09-14T09:08:01.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:08:01.253005+0000", 2024-09-14T09:08:01.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.64e-06 2024-09-14T09:08:01.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:08:01.253015+0000", 2024-09-14T09:08:01.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5659999999999996e-06 2024-09-14T09:08:01.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:08:01.253040+0000", 2024-09-14T09:08:01.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5020000000000001e-05 2024-09-14T09:08:01.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:08:01.253053+0000", 2024-09-14T09:08:01.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3217000000000001e-05 2024-09-14T09:08:01.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:08:01.253220+0000", 2024-09-14T09:08:01.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016746600000000001 2024-09-14T09:08:01.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:08:01.253655+0000", 2024-09-14T09:08:01.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00043488899999999999 2024-09-14T09:08:01.252 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.252 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.252 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.252 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:142499 3.11 3:8a7e3c58:::rbd_data.115e63832e35.000000000000001c:head [delete] snapc 23=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-14T09:08:01.252 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:08:01.253107+0000", 2024-09-14T09:08:01.252 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 0.00069181699999999995, 2024-09-14T09:08:01.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00079454999999999999, 2024-09-14T09:08:01.253 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.253 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.253 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:01.253 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.253 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.253 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.253 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 142499 2024-09-14T09:08:01.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.253 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:08:01.253107+0000", 2024-09-14T09:08:01.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:08:01.253107+0000", 2024-09-14T09:08:01.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:08:01.253108+0000", 2024-09-14T09:08:01.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0160000000000001e-06 2024-09-14T09:08:01.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:08:01.253112+0000", 2024-09-14T09:08:01.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5839999999999996e-06 2024-09-14T09:08:01.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:08:01.253125+0000", 2024-09-14T09:08:01.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2619000000000001e-05 2024-09-14T09:08:01.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:08:01.253133+0000", 2024-09-14T09:08:01.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1499999999999999e-06 2024-09-14T09:08:01.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:08:01.253279+0000", 2024-09-14T09:08:01.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000145456 2024-09-14T09:08:01.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:08:01.253601+0000", 2024-09-14T09:08:01.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000321844 2024-09-14T09:08:01.257 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.257 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.257 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.257 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.257 INFO:teuthology.orchestra.run.smithi183.stdout: ], 2024-09-14T09:08:01.257 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 2 2024-09-14T09:08:01.257 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:01.264 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:08:01.310 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:01.314+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:01.372 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:01.371+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:01.376 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:01.376 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:01.376 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:08:01.376 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:08:01.376 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:01.389 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:08:01.474 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:01.478+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:01.552 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:01.552 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:08:01.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:08:01.552 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:08:01.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.552 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38593 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [watch reconnect cookie 139718844451680 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:08:01.552 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:08:01.552 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 119.73638903, 2024-09-14T09:08:01.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0218296630000001, 2024-09-14T09:08:01.552 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.553 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.553 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:01.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.553 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38593 2024-09-14T09:08:01.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.553 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:08:01.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:08:01.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818420+0000", 2024-09-14T09:08:01.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8399999999999999e-06 2024-09-14T09:08:01.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818431+0000", 2024-09-14T09:08:01.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0859999999999999e-05 2024-09-14T09:08:01.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818466+0000", 2024-09-14T09:08:01.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4486999999999997e-05 2024-09-14T09:08:01.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818481+0000", 2024-09-14T09:08:01.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5362999999999998e-05 2024-09-14T09:08:01.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818659+0000", 2024-09-14T09:08:01.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017807500000000001 2024-09-14T09:08:01.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op must wait for map", 2024-09-14T09:08:01.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818681+0000", 2024-09-14T09:08:01.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2096999999999999e-05 2024-09-14T09:08:01.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825472+0000", 2024-09-14T09:08:01.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067905170000000003 2024-09-14T09:08:01.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:01.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825491+0000", 2024-09-14T09:08:01.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9372999999999999e-05 2024-09-14T09:08:01.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835402+0000", 2024-09-14T09:08:01.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0099111620000001 2024-09-14T09:08:01.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835548+0000", 2024-09-14T09:08:01.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014607300000000001 2024-09-14T09:08:01.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.840247+0000", 2024-09-14T09:08:01.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046988159999999998 2024-09-14T09:08:01.558 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.558 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.558 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.558 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38615 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:08:01.558 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:08:01.559 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 119.68802382200001, 2024-09-14T09:08:01.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.97014128899999996, 2024-09-14T09:08:01.559 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.559 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.559 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:01.559 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.559 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.559 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.559 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38615 2024-09-14T09:08:01.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.559 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:08:01.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:08:01.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866787+0000", 2024-09-14T09:08:01.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.476e-06 2024-09-14T09:08:01.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866806+0000", 2024-09-14T09:08:01.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8606000000000001e-05 2024-09-14T09:08:01.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866839+0000", 2024-09-14T09:08:01.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3510000000000003e-05 2024-09-14T09:08:01.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866867+0000", 2024-09-14T09:08:01.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7610000000000002e-05 2024-09-14T09:08:01.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867031+0000", 2024-09-14T09:08:01.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016414199999999999 2024-09-14T09:08:01.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:01.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867054+0000", 2024-09-14T09:08:01.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2693000000000001e-05 2024-09-14T09:08:01.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836462+0000", 2024-09-14T09:08:01.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.96940797700000003 2024-09-14T09:08:01.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:01.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836547+0000", 2024-09-14T09:08:01.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5294e-05 2024-09-14T09:08:01.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836924+0000", 2024-09-14T09:08:01.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037698099999999997 2024-09-14T09:08:01.563 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.563 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.563 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.563 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105723 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:01.564 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:01.564 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.459589778000002, 2024-09-14T09:08:01.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77512799700000001, 2024-09-14T09:08:01.564 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.564 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.564 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:01.564 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.564 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.564 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.564 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105723 2024-09-14T09:08:01.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.564 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:01.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:01.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:01.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1399999999999997e-07 2024-09-14T09:08:01.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095220+0000", 2024-09-14T09:08:01.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0510000000000001e-06 2024-09-14T09:08:01.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095228+0000", 2024-09-14T09:08:01.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4159999999999998e-06 2024-09-14T09:08:01.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095234+0000", 2024-09-14T09:08:01.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3010000000000003e-06 2024-09-14T09:08:01.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103902+0000", 2024-09-14T09:08:01.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086678299999999996 2024-09-14T09:08:01.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:01.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103905+0000", 2024-09-14T09:08:01.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5020000000000001e-06 2024-09-14T09:08:01.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870332+0000", 2024-09-14T09:08:01.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76642700900000005 2024-09-14T09:08:01.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870345+0000", 2024-09-14T09:08:01.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2274e-05 2024-09-14T09:08:01.568 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.568 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.568 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.568 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105724 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:01.568 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:08:01.568 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.459565265000002, 2024-09-14T09:08:01.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77515698099999997, 2024-09-14T09:08:01.569 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.569 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.569 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:01.569 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.569 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.569 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.569 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105724 2024-09-14T09:08:01.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.569 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:08:01.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:08:01.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095242+0000", 2024-09-14T09:08:01.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9400000000000001e-07 2024-09-14T09:08:01.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095243+0000", 2024-09-14T09:08:01.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7540000000000001e-06 2024-09-14T09:08:01.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095248+0000", 2024-09-14T09:08:01.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1320000000000004e-06 2024-09-14T09:08:01.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095251+0000", 2024-09-14T09:08:01.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2770000000000001e-06 2024-09-14T09:08:01.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103947+0000", 2024-09-14T09:08:01.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086960089999999993 2024-09-14T09:08:01.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:01.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103955+0000", 2024-09-14T09:08:01.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9519999999999994e-06 2024-09-14T09:08:01.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870386+0000", 2024-09-14T09:08:01.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76643141100000001 2024-09-14T09:08:01.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870398+0000", 2024-09-14T09:08:01.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2052000000000001e-05 2024-09-14T09:08:01.573 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.573 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.573 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.573 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105727 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:01.573 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:01.573 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.459525558999999, 2024-09-14T09:08:01.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517292900000001, 2024-09-14T09:08:01.574 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.574 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.574 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:01.574 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.574 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.574 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.574 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105727 2024-09-14T09:08:01.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.574 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:01.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:01.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:01.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3300000000000002e-07 2024-09-14T09:08:01.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095284+0000", 2024-09-14T09:08:01.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9790000000000002e-06 2024-09-14T09:08:01.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095292+0000", 2024-09-14T09:08:01.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3810000000000001e-06 2024-09-14T09:08:01.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095298+0000", 2024-09-14T09:08:01.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9710000000000001e-06 2024-09-14T09:08:01.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103987+0000", 2024-09-14T09:08:01.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086893820000000007 2024-09-14T09:08:01.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:01.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103991+0000", 2024-09-14T09:08:01.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5480000000000002e-06 2024-09-14T09:08:01.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870442+0000", 2024-09-14T09:08:01.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.766451145 2024-09-14T09:08:01.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870454+0000", 2024-09-14T09:08:01.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.199e-05 2024-09-14T09:08:01.578 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.578 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.578 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.578 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105729 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:01.578 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:08:01.578 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.459477202000002, 2024-09-14T09:08:01.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517686500000005, 2024-09-14T09:08:01.578 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.578 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.579 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:01.579 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.579 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.579 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.579 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105729 2024-09-14T09:08:01.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.579 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:08:01.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:08:01.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095330+0000", 2024-09-14T09:08:01.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-09-14T09:08:01.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095332+0000", 2024-09-14T09:08:01.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1560000000000002e-06 2024-09-14T09:08:01.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095338+0000", 2024-09-14T09:08:01.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.592e-06 2024-09-14T09:08:01.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095341+0000", 2024-09-14T09:08:01.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8139999999999998e-06 2024-09-14T09:08:01.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104031+0000", 2024-09-14T09:08:01.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086896330000000004 2024-09-14T09:08:01.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:01.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104039+0000", 2024-09-14T09:08:01.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0900000000000005e-06 2024-09-14T09:08:01.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870495+0000", 2024-09-14T09:08:01.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76645611199999997 2024-09-14T09:08:01.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870506+0000", 2024-09-14T09:08:01.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0986e-05 2024-09-14T09:08:01.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.583 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.583 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105731 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:01.583 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:01.583 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.459458935000001, 2024-09-14T09:08:01.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77520598600000001, 2024-09-14T09:08:01.583 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.583 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.583 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:01.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.584 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105731 2024-09-14T09:08:01.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.584 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:01.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:01.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:01.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8599999999999999e-07 2024-09-14T09:08:01.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095350+0000", 2024-09-14T09:08:01.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1110000000000002e-06 2024-09-14T09:08:01.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095357+0000", 2024-09-14T09:08:01.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9020000000000002e-06 2024-09-14T09:08:01.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095360+0000", 2024-09-14T09:08:01.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1939999999999998e-06 2024-09-14T09:08:01.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104073+0000", 2024-09-14T09:08:01.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087131210000000008 2024-09-14T09:08:01.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:01.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104077+0000", 2024-09-14T09:08:01.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.501e-06 2024-09-14T09:08:01.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870541+0000", 2024-09-14T09:08:01.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76646461700000001 2024-09-14T09:08:01.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870554+0000", 2024-09-14T09:08:01.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2153999999999999e-05 2024-09-14T09:08:01.588 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.588 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.588 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.588 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105733 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:01.588 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:01.588 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.459438904000002, 2024-09-14T09:08:01.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.775237277, 2024-09-14T09:08:01.588 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.588 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.588 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:01.588 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.589 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.589 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.589 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105733 2024-09-14T09:08:01.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.589 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:01.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:01.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:01.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8699999999999995e-07 2024-09-14T09:08:01.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095371+0000", 2024-09-14T09:08:01.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5450000000000002e-06 2024-09-14T09:08:01.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095377+0000", 2024-09-14T09:08:01.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.776e-06 2024-09-14T09:08:01.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095383+0000", 2024-09-14T09:08:01.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3129999999999998e-06 2024-09-14T09:08:01.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104124+0000", 2024-09-14T09:08:01.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087409840000000003 2024-09-14T09:08:01.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:01.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104132+0000", 2024-09-14T09:08:01.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3159999999999997e-06 2024-09-14T09:08:01.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870593+0000", 2024-09-14T09:08:01.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76646064800000002 2024-09-14T09:08:01.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870605+0000", 2024-09-14T09:08:01.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2208000000000001e-05 2024-09-14T09:08:01.592 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.592 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.593 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.593 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105734 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:01.593 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:01.593 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.459415821999997, 2024-09-14T09:08:01.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77526569599999995, 2024-09-14T09:08:01.593 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.593 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.593 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:01.593 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.593 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.593 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.593 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105734 2024-09-14T09:08:01.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.594 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:01.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:01.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:01.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7399999999999998e-07 2024-09-14T09:08:01.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095394+0000", 2024-09-14T09:08:01.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8030000000000001e-06 2024-09-14T09:08:01.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095401+0000", 2024-09-14T09:08:01.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.046e-06 2024-09-14T09:08:01.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095407+0000", 2024-09-14T09:08:01.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6060000000000002e-06 2024-09-14T09:08:01.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104167+0000", 2024-09-14T09:08:01.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087602530000000008 2024-09-14T09:08:01.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:01.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104171+0000", 2024-09-14T09:08:01.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7620000000000001e-06 2024-09-14T09:08:01.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870645+0000", 2024-09-14T09:08:01.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76647400600000004 2024-09-14T09:08:01.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870656+0000", 2024-09-14T09:08:01.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1746000000000001e-05 2024-09-14T09:08:01.597 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.597 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.597 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.598 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105738 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:01.598 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:08:01.598 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.459376095000003, 2024-09-14T09:08:01.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77527530700000002, 2024-09-14T09:08:01.598 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.598 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.598 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:01.598 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.598 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.598 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.598 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105738 2024-09-14T09:08:01.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.599 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:08:01.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:08:01.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095431+0000", 2024-09-14T09:08:01.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0999999999999999e-07 2024-09-14T09:08:01.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095434+0000", 2024-09-14T09:08:01.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.757e-06 2024-09-14T09:08:01.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095442+0000", 2024-09-14T09:08:01.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9929999999999995e-06 2024-09-14T09:08:01.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095448+0000", 2024-09-14T09:08:01.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1340000000000001e-06 2024-09-14T09:08:01.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104213+0000", 2024-09-14T09:08:01.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087650000000000002 2024-09-14T09:08:01.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:01.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104221+0000", 2024-09-14T09:08:01.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4589999999999998e-06 2024-09-14T09:08:01.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870694+0000", 2024-09-14T09:08:01.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76647303200000005 2024-09-14T09:08:01.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870706+0000", 2024-09-14T09:08:01.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1522000000000001e-05 2024-09-14T09:08:01.602 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.602 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.602 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.603 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105739 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:01.603 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:01.603 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.459351566000002, 2024-09-14T09:08:01.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77530221700000002, 2024-09-14T09:08:01.603 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.603 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.603 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:01.603 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.603 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.603 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.603 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105739 2024-09-14T09:08:01.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.603 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:01.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:01.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:01.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.15e-07 2024-09-14T09:08:01.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095457+0000", 2024-09-14T09:08:01.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6470000000000001e-06 2024-09-14T09:08:01.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095461+0000", 2024-09-14T09:08:01.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0910000000000003e-06 2024-09-14T09:08:01.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095464+0000", 2024-09-14T09:08:01.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2059999999999999e-06 2024-09-14T09:08:01.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104253+0000", 2024-09-14T09:08:01.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087884120000000007 2024-09-14T09:08:01.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:01.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104256+0000", 2024-09-14T09:08:01.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5439999999999999e-06 2024-09-14T09:08:01.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870745+0000", 2024-09-14T09:08:01.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76648908000000004 2024-09-14T09:08:01.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870757+0000", 2024-09-14T09:08:01.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1822e-05 2024-09-14T09:08:01.607 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.607 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.607 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.607 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105741 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:01.607 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:08:01.607 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.459282440000003, 2024-09-14T09:08:01.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77528469899999997, 2024-09-14T09:08:01.608 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.608 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.608 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:01.608 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.608 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.608 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.608 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105741 2024-09-14T09:08:01.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.608 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:08:01.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:08:01.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095525+0000", 2024-09-14T09:08:01.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3740000000000001e-06 2024-09-14T09:08:01.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095533+0000", 2024-09-14T09:08:01.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6380000000000004e-06 2024-09-14T09:08:01.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095549+0000", 2024-09-14T09:08:01.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5554999999999999e-05 2024-09-14T09:08:01.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095566+0000", 2024-09-14T09:08:01.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7146000000000002e-05 2024-09-14T09:08:01.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104297+0000", 2024-09-14T09:08:01.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087315750000000001 2024-09-14T09:08:01.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:01.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104305+0000", 2024-09-14T09:08:01.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1389999999999995e-06 2024-09-14T09:08:01.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870797+0000", 2024-09-14T09:08:01.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76649129599999999 2024-09-14T09:08:01.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870809+0000", 2024-09-14T09:08:01.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1976e-05 2024-09-14T09:08:01.612 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.612 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.612 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.612 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105742 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:01.612 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:08:01.612 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.459217656, 2024-09-14T09:08:01.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.775271873, 2024-09-14T09:08:01.612 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.613 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.613 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:01.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.613 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105742 2024-09-14T09:08:01.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.613 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:08:01.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:08:01.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095590+0000", 2024-09-14T09:08:01.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0400000000000005e-07 2024-09-14T09:08:01.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095594+0000", 2024-09-14T09:08:01.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5639999999999998e-06 2024-09-14T09:08:01.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095620+0000", 2024-09-14T09:08:01.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5494999999999999e-05 2024-09-14T09:08:01.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095639+0000", 2024-09-14T09:08:01.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9639999999999998e-05 2024-09-14T09:08:01.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104340+0000", 2024-09-14T09:08:01.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087004950000000008 2024-09-14T09:08:01.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:01.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104343+0000", 2024-09-14T09:08:01.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4020000000000002e-06 2024-09-14T09:08:01.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870850+0000", 2024-09-14T09:08:01.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76650648499999996 2024-09-14T09:08:01.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870861+0000", 2024-09-14T09:08:01.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0888e-05 2024-09-14T09:08:01.617 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.617 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.617 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.617 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105745 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:01.617 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:08:01.617 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.459119041000001, 2024-09-14T09:08:01.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77521998700000005, 2024-09-14T09:08:01.617 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.617 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.617 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:01.617 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.618 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.618 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.618 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105745 2024-09-14T09:08:01.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.618 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:08:01.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:08:01.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095688+0000", 2024-09-14T09:08:01.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2699999999999998e-07 2024-09-14T09:08:01.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095693+0000", 2024-09-14T09:08:01.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0479999999999998e-06 2024-09-14T09:08:01.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095707+0000", 2024-09-14T09:08:01.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.342e-05 2024-09-14T09:08:01.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095716+0000", 2024-09-14T09:08:01.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9800000000000004e-06 2024-09-14T09:08:01.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104388+0000", 2024-09-14T09:08:01.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086717349999999999 2024-09-14T09:08:01.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:01.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104396+0000", 2024-09-14T09:08:01.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1769999999999998e-06 2024-09-14T09:08:01.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870897+0000", 2024-09-14T09:08:01.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76650102600000003 2024-09-14T09:08:01.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870907+0000", 2024-09-14T09:08:01.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0674e-05 2024-09-14T09:08:01.621 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.621 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.621 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.622 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105747 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:01.622 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:01.622 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.459072820999999, 2024-09-14T09:08:01.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77522190300000005, 2024-09-14T09:08:01.622 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.622 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.622 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:01.622 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.622 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.622 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.622 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105747 2024-09-14T09:08:01.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.623 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:01.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:01.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:01.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8999999999999995e-07 2024-09-14T09:08:01.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095738+0000", 2024-09-14T09:08:01.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8430000000000003e-06 2024-09-14T09:08:01.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095750+0000", 2024-09-14T09:08:01.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1229999999999999e-05 2024-09-14T09:08:01.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095758+0000", 2024-09-14T09:08:01.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.337e-06 2024-09-14T09:08:01.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104431+0000", 2024-09-14T09:08:01.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086728780000000002 2024-09-14T09:08:01.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:01.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104434+0000", 2024-09-14T09:08:01.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5049999999999999e-06 2024-09-14T09:08:01.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870945+0000", 2024-09-14T09:08:01.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651031800000002 2024-09-14T09:08:01.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870956+0000", 2024-09-14T09:08:01.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1002000000000001e-05 2024-09-14T09:08:01.626 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.626 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.626 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.627 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105750 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:01.627 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:01.627 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.459002875000003, 2024-09-14T09:08:01.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77519961800000003, 2024-09-14T09:08:01.627 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.627 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.627 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:01.627 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.627 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.627 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.627 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105750 2024-09-14T09:08:01.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.627 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:01.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:01.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:01.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3600000000000002e-07 2024-09-14T09:08:01.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095809+0000", 2024-09-14T09:08:01.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4000000000000002e-06 2024-09-14T09:08:01.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095822+0000", 2024-09-14T09:08:01.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2704000000000001e-05 2024-09-14T09:08:01.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095831+0000", 2024-09-14T09:08:01.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9530000000000005e-06 2024-09-14T09:08:01.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104477+0000", 2024-09-14T09:08:01.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086459989999999997 2024-09-14T09:08:01.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:01.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104488+0000", 2024-09-14T09:08:01.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:08:01.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870992+0000", 2024-09-14T09:08:01.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.766504097 2024-09-14T09:08:01.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871003+0000", 2024-09-14T09:08:01.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1358999999999999e-05 2024-09-14T09:08:01.631 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.631 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.631 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.631 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105751 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:01.631 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:01.632 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.458944989999999, 2024-09-14T09:08:01.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77518966700000003, 2024-09-14T09:08:01.632 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.632 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.632 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:01.632 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.632 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.632 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.632 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105751 2024-09-14T09:08:01.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.632 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:01.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:01.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:01.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7300000000000005e-07 2024-09-14T09:08:01.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095867+0000", 2024-09-14T09:08:01.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2729999999999996e-06 2024-09-14T09:08:01.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095879+0000", 2024-09-14T09:08:01.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2917e-05 2024-09-14T09:08:01.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095888+0000", 2024-09-14T09:08:01.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2609999999999993e-06 2024-09-14T09:08:01.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104520+0000", 2024-09-14T09:08:01.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086322970000000006 2024-09-14T09:08:01.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:01.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104524+0000", 2024-09-14T09:08:01.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.54e-06 2024-09-14T09:08:01.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871040+0000", 2024-09-14T09:08:01.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651615399999995 2024-09-14T09:08:01.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871051+0000", 2024-09-14T09:08:01.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1452e-05 2024-09-14T09:08:01.636 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.636 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.636 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.636 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105752 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:01.636 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:08:01.636 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.458884226000002, 2024-09-14T09:08:01.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517684899999995, 2024-09-14T09:08:01.637 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.637 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.637 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:01.637 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.637 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.637 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.637 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105752 2024-09-14T09:08:01.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.637 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:08:01.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:08:01.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095923+0000", 2024-09-14T09:08:01.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3300000000000001e-07 2024-09-14T09:08:01.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095927+0000", 2024-09-14T09:08:01.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:08:01.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095940+0000", 2024-09-14T09:08:01.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2309e-05 2024-09-14T09:08:01.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095948+0000", 2024-09-14T09:08:01.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9199999999999993e-06 2024-09-14T09:08:01.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104566+0000", 2024-09-14T09:08:01.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086171449999999997 2024-09-14T09:08:01.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:01.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104574+0000", 2024-09-14T09:08:01.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0690000000000002e-06 2024-09-14T09:08:01.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871087+0000", 2024-09-14T09:08:01.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651370699999999 2024-09-14T09:08:01.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871099+0000", 2024-09-14T09:08:01.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1718e-05 2024-09-14T09:08:01.641 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.641 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.641 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.641 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105755 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:01.641 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:08:01.641 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.458826283000001, 2024-09-14T09:08:01.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77516787899999995, 2024-09-14T09:08:01.642 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.642 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.642 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:01.642 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.642 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.642 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.642 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105755 2024-09-14T09:08:01.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.642 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:08:01.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:08:01.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095981+0000", 2024-09-14T09:08:01.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2699999999999998e-07 2024-09-14T09:08:01.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095985+0000", 2024-09-14T09:08:01.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4349999999999999e-06 2024-09-14T09:08:01.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095999+0000", 2024-09-14T09:08:01.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3139000000000001e-05 2024-09-14T09:08:01.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096007+0000", 2024-09-14T09:08:01.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6610000000000006e-06 2024-09-14T09:08:01.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104591+0000", 2024-09-14T09:08:01.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0085841709999999998 2024-09-14T09:08:01.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:01.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104595+0000", 2024-09-14T09:08:01.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5590000000000001e-06 2024-09-14T09:08:01.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871137+0000", 2024-09-14T09:08:01.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76654202100000002 2024-09-14T09:08:01.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871148+0000", 2024-09-14T09:08:01.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1066e-05 2024-09-14T09:08:01.646 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.646 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.646 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.646 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105757 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:01.646 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:08:01.646 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.458767702999999, 2024-09-14T09:08:01.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77516401300000004, 2024-09-14T09:08:01.647 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:01.647 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:01.647 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:01.647 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:01.647 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:01.647 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:01.647 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105757 2024-09-14T09:08:01.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.647 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:01.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:01.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:08:01.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:01.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:08:01.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:01.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:01.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856228+0000", 2024-09-14T09:08:01.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76018875100000005 2024-09-14T09:08:01.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:01.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856240+0000", 2024-09-14T09:08:01.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-05 2024-09-14T09:08:01.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:01.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856268+0000", 2024-09-14T09:08:01.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7628999999999999e-05 2024-09-14T09:08:01.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:01.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856307+0000", 2024-09-14T09:08:01.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8967000000000001e-05 2024-09-14T09:08:01.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:01.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871192+0000", 2024-09-14T09:08:01.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.01488449 2024-09-14T09:08:01.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:01.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:01.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:01.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871203+0000", 2024-09-14T09:08:01.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1266e-05 2024-09-14T09:08:01.650 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.650 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.650 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.650 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:01.650 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:01.650 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:01.652 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:01.654+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:01.652 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:08:01.753 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:01.751+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:01.783 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:01.784 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:01.784 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:01.784 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:01.798 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:08:01.854 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:01.858+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:01.954 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:01.955+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:01.967 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:08:01.967 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:08:01.967 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:01.981 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:08:02.056 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:02.058+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:02.152 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:02.152 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:08:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:08:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:08:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:71052 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:08:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:08:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.413181882999993, 2024-09-14T09:08:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14854635599999999, 2024-09-14T09:08:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:02.153 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 71052 2024-09-14T09:08:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:08:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:08:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737570+0000", 2024-09-14T09:08:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4770000000000002e-06 2024-09-14T09:08:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737584+0000", 2024-09-14T09:08:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3457e-05 2024-09-14T09:08:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737608+0000", 2024-09-14T09:08:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3893999999999999e-05 2024-09-14T09:08:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737620+0000", 2024-09-14T09:08:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2157000000000001e-05 2024-09-14T09:08:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737738+0000", 2024-09-14T09:08:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118409 2024-09-14T09:08:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737892+0000", 2024-09-14T09:08:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154222 2024-09-14T09:08:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:08:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.749622+0000", 2024-09-14T09:08:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011729537 2024-09-14T09:08:02.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885934+0000", 2024-09-14T09:08:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13631222200000001 2024-09-14T09:08:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885996+0000", 2024-09-14T09:08:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1555000000000003e-05 2024-09-14T09:08:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:02.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886060+0000", 2024-09-14T09:08:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3980999999999996e-05 2024-09-14T09:08:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886114+0000", 2024-09-14T09:08:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4444999999999998e-05 2024-09-14T09:08:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:71052 3.1 e71/34 ondisk, result = 0)", 2024-09-14T09:08:02.159 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:08:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.370278505000002, 2024-09-14T09:08:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10565334799999999, 2024-09-14T09:08:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:08:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:08:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780473+0000", 2024-09-14T09:08:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8980000000000001e-06 2024-09-14T09:08:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780482+0000", 2024-09-14T09:08:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9080000000000001e-06 2024-09-14T09:08:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780511+0000", 2024-09-14T09:08:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9490999999999999e-05 2024-09-14T09:08:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780527+0000", 2024-09-14T09:08:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.508e-05 2024-09-14T09:08:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885978+0000", 2024-09-14T09:08:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.105451328 2024-09-14T09:08:02.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885989+0000", 2024-09-14T09:08:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0875e-05 2024-09-14T09:08:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886125+0000", 2024-09-14T09:08:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013576800000000001 2024-09-14T09:08:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:98007 3.12 3:4faf1f2a:::rbd_data.115e113dc8ca.000000000000004f:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:08:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:08:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 49.293306281, 2024-09-14T09:08:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.098233707000000003, 2024-09-14T09:08:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 98007 2024-09-14T09:08:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.164 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:08:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:08:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857446+0000", 2024-09-14T09:08:02.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7599999999999998e-06 2024-09-14T09:08:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857459+0000", 2024-09-14T09:08:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3123e-05 2024-09-14T09:08:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857491+0000", 2024-09-14T09:08:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1603000000000002e-05 2024-09-14T09:08:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857506+0000", 2024-09-14T09:08:02.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.537e-05 2024-09-14T09:08:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857694+0000", 2024-09-14T09:08:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018766599999999999 2024-09-14T09:08:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857955+0000", 2024-09-14T09:08:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000260979 2024-09-14T09:08:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:08:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.863125+0000", 2024-09-14T09:08:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.005170582 2024-09-14T09:08:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955268+0000", 2024-09-14T09:08:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092142659000000002 2024-09-14T09:08:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955385+0000", 2024-09-14T09:08:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116723 2024-09-14T09:08:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955553+0000", 2024-09-14T09:08:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016809299999999999 2024-09-14T09:08:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955677+0000", 2024-09-14T09:08:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012414899999999999 2024-09-14T09:08:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.169 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118153 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 445440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:08:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.312748905999999, 2024-09-14T09:08:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095358268999999996, 2024-09-14T09:08:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118153 2024-09-14T09:08:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:08:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:08:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:08:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0999999999999999e-07 2024-09-14T09:08:02.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838003+0000", 2024-09-14T09:08:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2000000000000001e-06 2024-09-14T09:08:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838009+0000", 2024-09-14T09:08:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.105e-06 2024-09-14T09:08:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838015+0000", 2024-09-14T09:08:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9030000000000001e-06 2024-09-14T09:08:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933049+0000", 2024-09-14T09:08:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095033513999999999 2024-09-14T09:08:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933113+0000", 2024-09-14T09:08:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3616999999999998e-05 2024-09-14T09:08:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933359+0000", 2024-09-14T09:08:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024652000000000002 2024-09-14T09:08:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118155 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 449536~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:08:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.312726381000001, 2024-09-14T09:08:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095668000000000003, 2024-09-14T09:08:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.174 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118155 2024-09-14T09:08:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:08:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:08:02.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838024+0000", 2024-09-14T09:08:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0400000000000002e-07 2024-09-14T09:08:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838026+0000", 2024-09-14T09:08:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.508e-06 2024-09-14T09:08:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838033+0000", 2024-09-14T09:08:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.392e-06 2024-09-14T09:08:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838038+0000", 2024-09-14T09:08:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1050000000000003e-06 2024-09-14T09:08:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933400+0000", 2024-09-14T09:08:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095362279999999994 2024-09-14T09:08:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933451+0000", 2024-09-14T09:08:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0927000000000002e-05 2024-09-14T09:08:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933691+0000", 2024-09-14T09:08:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024038400000000001 2024-09-14T09:08:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118158 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 453632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:02.178 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:08:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.312705358999999, 2024-09-14T09:08:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096024231000000002, 2024-09-14T09:08:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118158 2024-09-14T09:08:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:08:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:08:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838045+0000", 2024-09-14T09:08:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2199999999999999e-07 2024-09-14T09:08:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838047+0000", 2024-09-14T09:08:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1059999999999998e-06 2024-09-14T09:08:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838054+0000", 2024-09-14T09:08:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4089999999999997e-06 2024-09-14T09:08:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838059+0000", 2024-09-14T09:08:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8590000000000004e-06 2024-09-14T09:08:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933729+0000", 2024-09-14T09:08:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095670274999999999 2024-09-14T09:08:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933794+0000", 2024-09-14T09:08:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4846999999999998e-05 2024-09-14T09:08:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934069+0000", 2024-09-14T09:08:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027431299999999999 2024-09-14T09:08:02.182 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118160 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 457728~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:08:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.312683238999998, 2024-09-14T09:08:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096394812999999996, 2024-09-14T09:08:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.183 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118160 2024-09-14T09:08:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:08:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:08:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838067+0000", 2024-09-14T09:08:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9399999999999995e-07 2024-09-14T09:08:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838069+0000", 2024-09-14T09:08:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1579999999999999e-06 2024-09-14T09:08:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838077+0000", 2024-09-14T09:08:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6939999999999995e-06 2024-09-14T09:08:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838082+0000", 2024-09-14T09:08:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2290000000000002e-06 2024-09-14T09:08:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934106+0000", 2024-09-14T09:08:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096023874999999995 2024-09-14T09:08:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934158+0000", 2024-09-14T09:08:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2052000000000002e-05 2024-09-14T09:08:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934461+0000", 2024-09-14T09:08:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030331100000000001 2024-09-14T09:08:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118162 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 461824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:08:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.312660658999999, 2024-09-14T09:08:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096615294000000004, 2024-09-14T09:08:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.187 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118162 2024-09-14T09:08:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:08:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:08:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838090+0000", 2024-09-14T09:08:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9699999999999996e-07 2024-09-14T09:08:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838092+0000", 2024-09-14T09:08:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1119999999999999e-06 2024-09-14T09:08:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838098+0000", 2024-09-14T09:08:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4359999999999999e-06 2024-09-14T09:08:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838103+0000", 2024-09-14T09:08:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1200000000000001e-06 2024-09-14T09:08:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934504+0000", 2024-09-14T09:08:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096400885000000006 2024-09-14T09:08:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934596+0000", 2024-09-14T09:08:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1775999999999997e-05 2024-09-14T09:08:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934704+0000", 2024-09-14T09:08:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010846800000000001 2024-09-14T09:08:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118164 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 465920~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:02.191 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:08:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.312639398999998, 2024-09-14T09:08:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096951372999999993, 2024-09-14T09:08:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118164 2024-09-14T09:08:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:08:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:08:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838111+0000", 2024-09-14T09:08:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0100000000000005e-07 2024-09-14T09:08:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838113+0000", 2024-09-14T09:08:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0339999999999999e-06 2024-09-14T09:08:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838119+0000", 2024-09-14T09:08:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.409e-06 2024-09-14T09:08:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838124+0000", 2024-09-14T09:08:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9030000000000003e-06 2024-09-14T09:08:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934739+0000", 2024-09-14T09:08:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096615124999999996 2024-09-14T09:08:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934789+0000", 2024-09-14T09:08:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9554e-05 2024-09-14T09:08:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935062+0000", 2024-09-14T09:08:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000272847 2024-09-14T09:08:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.195 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118166 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 470016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:08:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.312618645000001, 2024-09-14T09:08:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097306410999999995, 2024-09-14T09:08:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.196 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118166 2024-09-14T09:08:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:08:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:08:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:08:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.03e-07 2024-09-14T09:08:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838133+0000", 2024-09-14T09:08:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9479999999999998e-06 2024-09-14T09:08:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838141+0000", 2024-09-14T09:08:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5739999999999998e-06 2024-09-14T09:08:02.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838146+0000", 2024-09-14T09:08:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3750000000000002e-06 2024-09-14T09:08:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935095+0000", 2024-09-14T09:08:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096948435999999999 2024-09-14T09:08:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935152+0000", 2024-09-14T09:08:02.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.694e-05 2024-09-14T09:08:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935437+0000", 2024-09-14T09:08:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028573500000000001 2024-09-14T09:08:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118168 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 474112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:08:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.312596423999999, 2024-09-14T09:08:02.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097658024999999996, 2024-09-14T09:08:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118168 2024-09-14T09:08:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:08:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:08:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838154+0000", 2024-09-14T09:08:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4200000000000001e-07 2024-09-14T09:08:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838160+0000", 2024-09-14T09:08:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8510000000000004e-06 2024-09-14T09:08:02.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838168+0000", 2024-09-14T09:08:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9899999999999997e-06 2024-09-14T09:08:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838172+0000", 2024-09-14T09:08:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.887e-06 2024-09-14T09:08:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935473+0000", 2024-09-14T09:08:02.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097300132999999997 2024-09-14T09:08:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935523+0000", 2024-09-14T09:08:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0862000000000002e-05 2024-09-14T09:08:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935811+0000", 2024-09-14T09:08:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028786000000000003 2024-09-14T09:08:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118170 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 478208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:08:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.312569613000001, 2024-09-14T09:08:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.098077529999999996, 2024-09-14T09:08:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118170 2024-09-14T09:08:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.205 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:08:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:08:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842901+0000", 2024-09-14T09:08:02.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004721416 2024-09-14T09:08:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842906+0000", 2024-09-14T09:08:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7199999999999997e-06 2024-09-14T09:08:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842917+0000", 2024-09-14T09:08:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0671e-05 2024-09-14T09:08:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842923+0000", 2024-09-14T09:08:02.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.088e-06 2024-09-14T09:08:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935857+0000", 2024-09-14T09:08:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092933624000000006 2024-09-14T09:08:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935958+0000", 2024-09-14T09:08:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100933 2024-09-14T09:08:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.936258+0000", 2024-09-14T09:08:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030007800000000001 2024-09-14T09:08:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118181 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 498688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:08:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.307763344000001, 2024-09-14T09:08:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095212643999999999, 2024-09-14T09:08:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.209 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118181 2024-09-14T09:08:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:08:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:08:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842987+0000", 2024-09-14T09:08:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.29e-07 2024-09-14T09:08:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842988+0000", 2024-09-14T09:08:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.229e-06 2024-09-14T09:08:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842993+0000", 2024-09-14T09:08:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.792e-06 2024-09-14T09:08:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842996+0000", 2024-09-14T09:08:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3349999999999998e-06 2024-09-14T09:08:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.937807+0000", 2024-09-14T09:08:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.094811585000000004 2024-09-14T09:08:02.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.937899+0000", 2024-09-14T09:08:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1719999999999996e-05 2024-09-14T09:08:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938199+0000", 2024-09-14T09:08:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000299754 2024-09-14T09:08:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118182 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 502784~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:08:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.307750167000002, 2024-09-14T09:08:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095557953000000001, 2024-09-14T09:08:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118182 2024-09-14T09:08:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:08:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:08:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:08:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2999999999999999e-07 2024-09-14T09:08:02.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843001+0000", 2024-09-14T09:08:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.652e-06 2024-09-14T09:08:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843005+0000", 2024-09-14T09:08:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2780000000000002e-06 2024-09-14T09:08:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843007+0000", 2024-09-14T09:08:02.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.553e-06 2024-09-14T09:08:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938240+0000", 2024-09-14T09:08:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095232864 2024-09-14T09:08:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938295+0000", 2024-09-14T09:08:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5095e-05 2024-09-14T09:08:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.217 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938557+0000", 2024-09-14T09:08:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026228100000000001 2024-09-14T09:08:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118184 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 506880~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:08:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.307738966999999, 2024-09-14T09:08:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095950246000000003, 2024-09-14T09:08:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.218 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118184 2024-09-14T09:08:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:08:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:08:02.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:08:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2399999999999999e-07 2024-09-14T09:08:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843012+0000", 2024-09-14T09:08:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.269e-06 2024-09-14T09:08:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843015+0000", 2024-09-14T09:08:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0469999999999998e-06 2024-09-14T09:08:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843018+0000", 2024-09-14T09:08:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.044e-06 2024-09-14T09:08:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938597+0000", 2024-09-14T09:08:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095578688999999994 2024-09-14T09:08:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938687+0000", 2024-09-14T09:08:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0284000000000005e-05 2024-09-14T09:08:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938961+0000", 2024-09-14T09:08:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027368899999999997 2024-09-14T09:08:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118186 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 510976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:08:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.307728007000001, 2024-09-14T09:08:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096286497999999998, 2024-09-14T09:08:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.223 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118186 2024-09-14T09:08:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:08:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:08:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:08:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.35e-07 2024-09-14T09:08:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843023+0000", 2024-09-14T09:08:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.195e-06 2024-09-14T09:08:02.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843026+0000", 2024-09-14T09:08:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.055e-06 2024-09-14T09:08:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843029+0000", 2024-09-14T09:08:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6539999999999999e-06 2024-09-14T09:08:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939000+0000", 2024-09-14T09:08:02.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095971476 2024-09-14T09:08:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939049+0000", 2024-09-14T09:08:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8986999999999997e-05 2024-09-14T09:08:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939308+0000", 2024-09-14T09:08:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000258896 2024-09-14T09:08:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118188 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 515072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:08:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.307717618000002, 2024-09-14T09:08:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096722717999999999, 2024-09-14T09:08:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118188 2024-09-14T09:08:02.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:08:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:08:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:08:02.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2999999999999999e-07 2024-09-14T09:08:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843034+0000", 2024-09-14T09:08:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T09:08:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843037+0000", 2024-09-14T09:08:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0989999999999999e-06 2024-09-14T09:08:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843039+0000", 2024-09-14T09:08:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.666e-06 2024-09-14T09:08:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939351+0000", 2024-09-14T09:08:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096311789999999994 2024-09-14T09:08:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939440+0000", 2024-09-14T09:08:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8843000000000003e-05 2024-09-14T09:08:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939755+0000", 2024-09-14T09:08:02.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031486399999999999 2024-09-14T09:08:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118191 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 519168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:08:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.307706925000002, 2024-09-14T09:08:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097080607999999999, 2024-09-14T09:08:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.232 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118191 2024-09-14T09:08:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:08:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:08:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.233 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:08:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2100000000000001e-07 2024-09-14T09:08:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843044+0000", 2024-09-14T09:08:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2249999999999999e-06 2024-09-14T09:08:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843047+0000", 2024-09-14T09:08:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0240000000000002e-06 2024-09-14T09:08:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843050+0000", 2024-09-14T09:08:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.508e-06 2024-09-14T09:08:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939791+0000", 2024-09-14T09:08:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096741399000000006 2024-09-14T09:08:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939849+0000", 2024-09-14T09:08:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7912999999999999e-05 2024-09-14T09:08:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940123+0000", 2024-09-14T09:08:02.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027431800000000001 2024-09-14T09:08:02.236 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.236 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.236 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.236 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118192 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 523264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:02.236 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:08:02.236 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.307695864999999, 2024-09-14T09:08:02.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097587224, 2024-09-14T09:08:02.236 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.236 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.237 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.237 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.237 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.237 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.237 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118192 2024-09-14T09:08:02.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.237 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:08:02.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:08:02.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844132+0000", 2024-09-14T09:08:02.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010777810000000001 2024-09-14T09:08:02.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844140+0000", 2024-09-14T09:08:02.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4470000000000006e-06 2024-09-14T09:08:02.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.238 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.238 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844155+0000", 2024-09-14T09:08:02.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4924e-05 2024-09-14T09:08:02.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844163+0000", 2024-09-14T09:08:02.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9980000000000003e-06 2024-09-14T09:08:02.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940199+0000", 2024-09-14T09:08:02.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096036047999999999 2024-09-14T09:08:02.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940282+0000", 2024-09-14T09:08:02.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3073999999999998e-05 2024-09-14T09:08:02.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940641+0000", 2024-09-14T09:08:02.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000358952 2024-09-14T09:08:02.240 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.240 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.240 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.240 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118197 3.17 3:ec5ec63a:::rbd_data.115e96ceebb2.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 3718221] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:02.240 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:08:02.241 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 30.302916450000001, 2024-09-14T09:08:02.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.141131069, 2024-09-14T09:08:02.241 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.241 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.241 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:02.241 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.241 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.241 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.241 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118197 2024-09-14T09:08:02.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.241 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:08:02.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:08:02.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847834+0000", 2024-09-14T09:08:02.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2500000000000004e-07 2024-09-14T09:08:02.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849227+0000", 2024-09-14T09:08:02.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0013932149999999999 2024-09-14T09:08:02.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849246+0000", 2024-09-14T09:08:02.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8958e-05 2024-09-14T09:08:02.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849254+0000", 2024-09-14T09:08:02.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2360000000000004e-06 2024-09-14T09:08:02.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940154+0000", 2024-09-14T09:08:02.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.090899468999999997 2024-09-14T09:08:02.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940312+0000", 2024-09-14T09:08:02.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000158326 2024-09-14T09:08:02.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:08:02.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940728+0000", 2024-09-14T09:08:02.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00041583100000000002 2024-09-14T09:08:02.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:02.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.962731+0000", 2024-09-14T09:08:02.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022002608999999999 2024-09-14T09:08:02.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:02.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988881+0000", 2024-09-14T09:08:02.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026150620999999999 2024-09-14T09:08:02.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:02.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988919+0000", 2024-09-14T09:08:02.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8108999999999997e-05 2024-09-14T09:08:02.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988964+0000", 2024-09-14T09:08:02.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4870000000000002e-05 2024-09-14T09:08:02.246 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.246 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.246 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.246 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.246 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.246 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:02.247 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:08:02.248 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:02.250+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:02.349 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:02.350+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:02.381 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:02.382 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:02.382 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:02.382 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:02.396 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:08:02.449 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:02.447+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:02.516 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:02.516 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:02.516 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:08:02.516 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:08:02.517 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:02.530 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:08:02.574 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:02.575+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:02.632 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:08:02.678 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:02.679+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:02.695 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:02.695 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:08:02.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:08:02.695 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:08:02.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.695 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73542 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15319040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:02.695 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:08:02.695 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.426138678000001, 2024-09-14T09:08:02.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0237091359999999, 2024-09-14T09:08:02.695 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.695 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.696 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.696 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.696 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.696 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.696 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73542 2024-09-14T09:08:02.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.696 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:08:02.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:08:02.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268762+0000", 2024-09-14T09:08:02.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:08:02.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268766+0000", 2024-09-14T09:08:02.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5880000000000002e-06 2024-09-14T09:08:02.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268776+0000", 2024-09-14T09:08:02.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0168e-05 2024-09-14T09:08:02.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268789+0000", 2024-09-14T09:08:02.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:08:02.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282053+0000", 2024-09-14T09:08:02.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013264496000000001 2024-09-14T09:08:02.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282057+0000", 2024-09-14T09:08:02.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6090000000000001e-06 2024-09-14T09:08:02.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292052+0000", 2024-09-14T09:08:02.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0099950070000001 2024-09-14T09:08:02.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292262+0000", 2024-09-14T09:08:02.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021009800000000001 2024-09-14T09:08:02.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292470+0000", 2024-09-14T09:08:02.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000208314 2024-09-14T09:08:02.701 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.701 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.701 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.702 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73543 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:02.702 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:08:02.702 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.426099887999996, 2024-09-14T09:08:02.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023985473, 2024-09-14T09:08:02.702 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.702 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.702 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.702 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.702 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.703 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.703 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73543 2024-09-14T09:08:02.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.703 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:08:02.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:08:02.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268801+0000", 2024-09-14T09:08:02.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:08:02.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268803+0000", 2024-09-14T09:08:02.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7439999999999999e-06 2024-09-14T09:08:02.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268812+0000", 2024-09-14T09:08:02.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7080000000000003e-06 2024-09-14T09:08:02.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268819+0000", 2024-09-14T09:08:02.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T09:08:02.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282068+0000", 2024-09-14T09:08:02.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013249540000000001 2024-09-14T09:08:02.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282072+0000", 2024-09-14T09:08:02.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0960000000000001e-06 2024-09-14T09:08:02.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292573+0000", 2024-09-14T09:08:02.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0105014560000001 2024-09-14T09:08:02.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292679+0000", 2024-09-14T09:08:02.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010545899999999999 2024-09-14T09:08:02.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292786+0000", 2024-09-14T09:08:02.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107052 2024-09-14T09:08:02.708 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.708 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.708 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.709 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73544 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:02.709 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:02.709 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.426072185999999, 2024-09-14T09:08:02.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0244830549999999, 2024-09-14T09:08:02.709 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.709 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.709 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.709 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.709 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.709 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.709 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73544 2024-09-14T09:08:02.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.710 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:02.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:02.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:02.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5599999999999995e-07 2024-09-14T09:08:02.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268831+0000", 2024-09-14T09:08:02.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.57e-06 2024-09-14T09:08:02.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268844+0000", 2024-09-14T09:08:02.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2811e-05 2024-09-14T09:08:02.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268851+0000", 2024-09-14T09:08:02.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9140000000000002e-06 2024-09-14T09:08:02.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282083+0000", 2024-09-14T09:08:02.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013232111 2024-09-14T09:08:02.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282086+0000", 2024-09-14T09:08:02.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2909999999999999e-06 2024-09-14T09:08:02.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292884+0000", 2024-09-14T09:08:02.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0107975069999999 2024-09-14T09:08:02.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293100+0000", 2024-09-14T09:08:02.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021639399999999999 2024-09-14T09:08:02.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293311+0000", 2024-09-14T09:08:02.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021090099999999999 2024-09-14T09:08:02.714 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.715 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.715 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.715 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73545 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15564800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:02.715 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:08:02.715 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.426021000999995, 2024-09-14T09:08:02.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0249746740000001, 2024-09-14T09:08:02.715 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.715 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.715 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.716 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.716 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.716 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.716 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73545 2024-09-14T09:08:02.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.716 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:08:02.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:08:02.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268880+0000", 2024-09-14T09:08:02.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5600000000000005e-07 2024-09-14T09:08:02.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268889+0000", 2024-09-14T09:08:02.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8559999999999996e-06 2024-09-14T09:08:02.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268913+0000", 2024-09-14T09:08:02.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3958e-05 2024-09-14T09:08:02.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268921+0000", 2024-09-14T09:08:02.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7479999999999999e-06 2024-09-14T09:08:02.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282098+0000", 2024-09-14T09:08:02.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013176297999999999 2024-09-14T09:08:02.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282101+0000", 2024-09-14T09:08:02.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2329999999999998e-06 2024-09-14T09:08:02.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293435+0000", 2024-09-14T09:08:02.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0113339459999999 2024-09-14T09:08:02.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293648+0000", 2024-09-14T09:08:02.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021307100000000001 2024-09-14T09:08:02.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293854+0000", 2024-09-14T09:08:02.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020580800000000001 2024-09-14T09:08:02.722 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.722 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.722 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.723 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73546 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15646720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:02.723 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:08:02.723 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.425967130000004, 2024-09-14T09:08:02.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0254846360000001, 2024-09-14T09:08:02.723 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.723 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.723 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.723 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.723 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.723 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.723 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73546 2024-09-14T09:08:02.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.723 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:08:02.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:08:02.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268934+0000", 2024-09-14T09:08:02.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7999999999999995e-07 2024-09-14T09:08:02.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268937+0000", 2024-09-14T09:08:02.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9840000000000001e-06 2024-09-14T09:08:02.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268947+0000", 2024-09-14T09:08:02.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0522e-05 2024-09-14T09:08:02.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268955+0000", 2024-09-14T09:08:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8059999999999995e-06 2024-09-14T09:08:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282112+0000", 2024-09-14T09:08:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013157189 2024-09-14T09:08:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282115+0000", 2024-09-14T09:08:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2949999999999998e-06 2024-09-14T09:08:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293968+0000", 2024-09-14T09:08:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0118530990000001 2024-09-14T09:08:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294197+0000", 2024-09-14T09:08:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228507 2024-09-14T09:08:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294417+0000", 2024-09-14T09:08:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000220554 2024-09-14T09:08:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.727 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73547 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:08:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.425934683999998, 2024-09-14T09:08:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.025970525, 2024-09-14T09:08:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.728 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73547 2024-09-14T09:08:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:08:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:08:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268966+0000", 2024-09-14T09:08:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3799999999999997e-07 2024-09-14T09:08:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268969+0000", 2024-09-14T09:08:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8739999999999999e-06 2024-09-14T09:08:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268979+0000", 2024-09-14T09:08:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9939999999999995e-06 2024-09-14T09:08:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268986+0000", 2024-09-14T09:08:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3459999999999996e-06 2024-09-14T09:08:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282127+0000", 2024-09-14T09:08:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01314039 2024-09-14T09:08:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282130+0000", 2024-09-14T09:08:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2219999999999998e-06 2024-09-14T09:08:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294540+0000", 2024-09-14T09:08:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124105880000001 2024-09-14T09:08:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294759+0000", 2024-09-14T09:08:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000218798 2024-09-14T09:08:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294936+0000", 2024-09-14T09:08:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000176675 2024-09-14T09:08:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73548 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15810560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:08:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.425903571999996, 2024-09-14T09:08:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0265045129999999, 2024-09-14T09:08:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.733 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73548 2024-09-14T09:08:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:08:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:08:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268997+0000", 2024-09-14T09:08:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7599999999999996e-07 2024-09-14T09:08:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269000+0000", 2024-09-14T09:08:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0580000000000002e-06 2024-09-14T09:08:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269010+0000", 2024-09-14T09:08:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124e-05 2024-09-14T09:08:02.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269018+0000", 2024-09-14T09:08:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5399999999999998e-06 2024-09-14T09:08:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282141+0000", 2024-09-14T09:08:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013123012 2024-09-14T09:08:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282144+0000", 2024-09-14T09:08:02.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.196e-06 2024-09-14T09:08:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295042+0000", 2024-09-14T09:08:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0128975689999999 2024-09-14T09:08:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295272+0000", 2024-09-14T09:08:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023048099999999999 2024-09-14T09:08:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295501+0000", 2024-09-14T09:08:02.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228657 2024-09-14T09:08:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73549 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15892480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:08:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.425857692999998, 2024-09-14T09:08:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0268354099999999, 2024-09-14T09:08:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.738 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73549 2024-09-14T09:08:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:08:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:08:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269043+0000", 2024-09-14T09:08:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6199999999999997e-07 2024-09-14T09:08:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269046+0000", 2024-09-14T09:08:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2480000000000001e-06 2024-09-14T09:08:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269057+0000", 2024-09-14T09:08:02.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0436e-05 2024-09-14T09:08:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269064+0000", 2024-09-14T09:08:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5179999999999998e-06 2024-09-14T09:08:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282155+0000", 2024-09-14T09:08:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013091190000000001 2024-09-14T09:08:02.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282159+0000", 2024-09-14T09:08:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.366e-06 2024-09-14T09:08:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295599+0000", 2024-09-14T09:08:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0134405559999999 2024-09-14T09:08:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295765+0000", 2024-09-14T09:08:02.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016575499999999999 2024-09-14T09:08:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295878+0000", 2024-09-14T09:08:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112679 2024-09-14T09:08:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73550 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15974400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:08:02.743 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.425825093, 2024-09-14T09:08:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027357657, 2024-09-14T09:08:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73550 2024-09-14T09:08:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:08:02.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:08:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269076+0000", 2024-09-14T09:08:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:08:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269079+0000", 2024-09-14T09:08:02.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:08:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269098+0000", 2024-09-14T09:08:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9698e-05 2024-09-14T09:08:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269106+0000", 2024-09-14T09:08:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6179999999999997e-06 2024-09-14T09:08:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282170+0000", 2024-09-14T09:08:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013063874 2024-09-14T09:08:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282173+0000", 2024-09-14T09:08:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1999999999999999e-06 2024-09-14T09:08:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295996+0000", 2024-09-14T09:08:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.013823151 2024-09-14T09:08:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296214+0000", 2024-09-14T09:08:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021817700000000001 2024-09-14T09:08:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296433+0000", 2024-09-14T09:08:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021829099999999999 2024-09-14T09:08:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73551 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:08:02.748 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.425783675999995, 2024-09-14T09:08:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027730276, 2024-09-14T09:08:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73551 2024-09-14T09:08:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:08:02.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:08:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269117+0000", 2024-09-14T09:08:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:08:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269120+0000", 2024-09-14T09:08:02.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.056e-06 2024-09-14T09:08:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269130+0000", 2024-09-14T09:08:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0084e-05 2024-09-14T09:08:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269137+0000", 2024-09-14T09:08:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1359999999999999e-06 2024-09-14T09:08:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282184+0000", 2024-09-14T09:08:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013046537 2024-09-14T09:08:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282188+0000", 2024-09-14T09:08:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3220000000000003e-06 2024-09-14T09:08:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296529+0000", 2024-09-14T09:08:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0143406109999999 2024-09-14T09:08:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296695+0000", 2024-09-14T09:08:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016604799999999999 2024-09-14T09:08:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296847+0000", 2024-09-14T09:08:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000151822 2024-09-14T09:08:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73552 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:02.753 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.425753095999994, 2024-09-14T09:08:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0281767500000001, 2024-09-14T09:08:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73552 2024-09-14T09:08:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:08:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269150+0000", 2024-09-14T09:08:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9119999999999998e-06 2024-09-14T09:08:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269160+0000", 2024-09-14T09:08:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6819999999999998e-06 2024-09-14T09:08:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269168+0000", 2024-09-14T09:08:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6079999999999998e-06 2024-09-14T09:08:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282205+0000", 2024-09-14T09:08:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013036982000000001 2024-09-14T09:08:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282208+0000", 2024-09-14T09:08:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4620000000000001e-06 2024-09-14T09:08:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296968+0000", 2024-09-14T09:08:02.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0147601209999999 2024-09-14T09:08:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297115+0000", 2024-09-14T09:08:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014673900000000001 2024-09-14T09:08:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297324+0000", 2024-09-14T09:08:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020866400000000001 2024-09-14T09:08:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73553 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:08:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.425723031999993, 2024-09-14T09:08:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0286556739999999, 2024-09-14T09:08:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.759 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73553 2024-09-14T09:08:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:08:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:08:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269178+0000", 2024-09-14T09:08:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:08:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269180+0000", 2024-09-14T09:08:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T09:08:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269189+0000", 2024-09-14T09:08:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2e-06 2024-09-14T09:08:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269197+0000", 2024-09-14T09:08:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2980000000000003e-06 2024-09-14T09:08:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282219+0000", 2024-09-14T09:08:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013022687 2024-09-14T09:08:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282223+0000", 2024-09-14T09:08:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.241e-06 2024-09-14T09:08:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297470+0000", 2024-09-14T09:08:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.015247778 2024-09-14T09:08:02.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297704+0000", 2024-09-14T09:08:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000233367 2024-09-14T09:08:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297833+0000", 2024-09-14T09:08:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012888900000000001 2024-09-14T09:08:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73554 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:08:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.425693714999994, 2024-09-14T09:08:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.029005113, 2024-09-14T09:08:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73554 2024-09-14T09:08:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:08:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:08:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269207+0000", 2024-09-14T09:08:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.68e-07 2024-09-14T09:08:02.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269210+0000", 2024-09-14T09:08:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8100000000000002e-06 2024-09-14T09:08:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269219+0000", 2024-09-14T09:08:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:08:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269227+0000", 2024-09-14T09:08:02.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3839999999999999e-06 2024-09-14T09:08:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282234+0000", 2024-09-14T09:08:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013007187 2024-09-14T09:08:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282237+0000", 2024-09-14T09:08:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1200000000000002e-06 2024-09-14T09:08:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297942+0000", 2024-09-14T09:08:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0157048040000001 2024-09-14T09:08:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298087+0000", 2024-09-14T09:08:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014499099999999999 2024-09-14T09:08:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298211+0000", 2024-09-14T09:08:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012461899999999999 2024-09-14T09:08:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.770 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73555 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:08:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.425663866999997, 2024-09-14T09:08:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.02941558, 2024-09-14T09:08:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73555 2024-09-14T09:08:02.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:08:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:08:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269237+0000", 2024-09-14T09:08:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3399999999999999e-07 2024-09-14T09:08:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269240+0000", 2024-09-14T09:08:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8480000000000001e-06 2024-09-14T09:08:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269249+0000", 2024-09-14T09:08:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1880000000000008e-06 2024-09-14T09:08:02.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269256+0000", 2024-09-14T09:08:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:08:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282248+0000", 2024-09-14T09:08:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012992327 2024-09-14T09:08:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282251+0000", 2024-09-14T09:08:02.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2160000000000002e-06 2024-09-14T09:08:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298299+0000", 2024-09-14T09:08:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0160479330000001 2024-09-14T09:08:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298446+0000", 2024-09-14T09:08:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000146525 2024-09-14T09:08:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298652+0000", 2024-09-14T09:08:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020584100000000001 2024-09-14T09:08:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73556 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.425634121000002, 2024-09-14T09:08:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0299854239999999, 2024-09-14T09:08:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.776 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73556 2024-09-14T09:08:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:02.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:08:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269270+0000", 2024-09-14T09:08:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0920000000000002e-06 2024-09-14T09:08:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269280+0000", 2024-09-14T09:08:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0013999999999999e-05 2024-09-14T09:08:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269288+0000", 2024-09-14T09:08:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:08:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282263+0000", 2024-09-14T09:08:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012974703000000001 2024-09-14T09:08:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282266+0000", 2024-09-14T09:08:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1520000000000001e-06 2024-09-14T09:08:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298729+0000", 2024-09-14T09:08:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.016463506 2024-09-14T09:08:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298884+0000", 2024-09-14T09:08:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015450800000000001 2024-09-14T09:08:02.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299251+0000", 2024-09-14T09:08:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036759899999999997 2024-09-14T09:08:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73557 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16547840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:08:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.425587471, 2024-09-14T09:08:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0307969480000001, 2024-09-14T09:08:02.781 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73557 2024-09-14T09:08:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:08:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:08:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269313+0000", 2024-09-14T09:08:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.92e-07 2024-09-14T09:08:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269336+0000", 2024-09-14T09:08:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3118000000000001e-05 2024-09-14T09:08:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269346+0000", 2024-09-14T09:08:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4069999999999999e-06 2024-09-14T09:08:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269352+0000", 2024-09-14T09:08:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5810000000000002e-06 2024-09-14T09:08:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282277+0000", 2024-09-14T09:08:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012924658 2024-09-14T09:08:02.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282280+0000", 2024-09-14T09:08:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2880000000000001e-06 2024-09-14T09:08:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299343+0000", 2024-09-14T09:08:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0170630190000001 2024-09-14T09:08:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299554+0000", 2024-09-14T09:08:02.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021085099999999999 2024-09-14T09:08:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300109+0000", 2024-09-14T09:08:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00055533399999999999 2024-09-14T09:08:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73558 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:08:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.425535847999996, 2024-09-14T09:08:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0314849340000001, 2024-09-14T09:08:02.786 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73558 2024-09-14T09:08:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:08:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:08:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269365+0000", 2024-09-14T09:08:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7400000000000003e-07 2024-09-14T09:08:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269367+0000", 2024-09-14T09:08:02.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4930000000000001e-06 2024-09-14T09:08:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269374+0000", 2024-09-14T09:08:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9519999999999997e-06 2024-09-14T09:08:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269379+0000", 2024-09-14T09:08:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9710000000000003e-06 2024-09-14T09:08:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282292+0000", 2024-09-14T09:08:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012912704000000001 2024-09-14T09:08:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282295+0000", 2024-09-14T09:08:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.179e-06 2024-09-14T09:08:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300195+0000", 2024-09-14T09:08:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.017899975 2024-09-14T09:08:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300345+0000", 2024-09-14T09:08:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014957499999999999 2024-09-14T09:08:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300849+0000", 2024-09-14T09:08:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050451100000000002 2024-09-14T09:08:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73559 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:02.791 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 88.425513438999999, 2024-09-14T09:08:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.031916343, 2024-09-14T09:08:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73559 2024-09-14T09:08:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3500000000000002e-07 2024-09-14T09:08:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269390+0000", 2024-09-14T09:08:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5890000000000001e-06 2024-09-14T09:08:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269396+0000", 2024-09-14T09:08:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3300000000000004e-06 2024-09-14T09:08:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269401+0000", 2024-09-14T09:08:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0340000000000004e-06 2024-09-14T09:08:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282311+0000", 2024-09-14T09:08:02.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012910204 2024-09-14T09:08:02.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282317+0000", 2024-09-14T09:08:02.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2639999999999997e-06 2024-09-14T09:08:02.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300933+0000", 2024-09-14T09:08:02.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0186160710000001 2024-09-14T09:08:02.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301158+0000", 2024-09-14T09:08:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225021 2024-09-14T09:08:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301303+0000", 2024-09-14T09:08:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014439499999999999 2024-09-14T09:08:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106964 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:08:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.579585604999998, 2024-09-14T09:08:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8802924409999999, 2024-09-14T09:08:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106964 2024-09-14T09:08:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:08:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:08:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115317+0000", 2024-09-14T09:08:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6429999999999999e-06 2024-09-14T09:08:02.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115326+0000", 2024-09-14T09:08:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2140000000000002e-06 2024-09-14T09:08:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115346+0000", 2024-09-14T09:08:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9419000000000002e-05 2024-09-14T09:08:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115357+0000", 2024-09-14T09:08:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1014e-05 2024-09-14T09:08:02.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115431+0000", 2024-09-14T09:08:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4040000000000003e-05 2024-09-14T09:08:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115438+0000", 2024-09-14T09:08:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9920000000000001e-06 2024-09-14T09:08:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844944+0000", 2024-09-14T09:08:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72950628399999995 2024-09-14T09:08:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844961+0000", 2024-09-14T09:08:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7156e-05 2024-09-14T09:08:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988004+0000", 2024-09-14T09:08:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.14304297 2024-09-14T09:08:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988511+0000", 2024-09-14T09:08:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050694100000000001 2024-09-14T09:08:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:08:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.989802+0000", 2024-09-14T09:08:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001291195 2024-09-14T09:08:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994866+0000", 2024-09-14T09:08:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050632760000000002 2024-09-14T09:08:02.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994988+0000", 2024-09-14T09:08:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012252899999999999 2024-09-14T09:08:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995198+0000", 2024-09-14T09:08:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021010899999999999 2024-09-14T09:08:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995607+0000", 2024-09-14T09:08:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040865900000000002 2024-09-14T09:08:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106965 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 41.578383228, 2024-09-14T09:08:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.879730162, 2024-09-14T09:08:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:02.804 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106965 2024-09-14T09:08:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:02.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:08:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116522+0000", 2024-09-14T09:08:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3989999999999997e-06 2024-09-14T09:08:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116533+0000", 2024-09-14T09:08:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0669000000000001e-05 2024-09-14T09:08:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116541+0000", 2024-09-14T09:08:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2730000000000002e-06 2024-09-14T09:08:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116604+0000", 2024-09-14T09:08:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2706999999999994e-05 2024-09-14T09:08:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116610+0000", 2024-09-14T09:08:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8630000000000002e-06 2024-09-14T09:08:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845060+0000", 2024-09-14T09:08:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72845003699999999 2024-09-14T09:08:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845079+0000", 2024-09-14T09:08:02.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8287999999999998e-05 2024-09-14T09:08:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.991866+0000", 2024-09-14T09:08:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1467871519999999 2024-09-14T09:08:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.992274+0000", 2024-09-14T09:08:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040813500000000001 2024-09-14T09:08:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:08:02.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.993538+0000", 2024-09-14T09:08:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001263655 2024-09-14T09:08:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995770+0000", 2024-09-14T09:08:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002231949 2024-09-14T09:08:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995810+0000", 2024-09-14T09:08:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0852999999999997e-05 2024-09-14T09:08:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995979+0000", 2024-09-14T09:08:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016866 2024-09-14T09:08:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996247+0000", 2024-09-14T09:08:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000267769 2024-09-14T09:08:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:02.811 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:02.811 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:02.813 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:08:02.813 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:08:02.814 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:08:02.814 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:08:02.814 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:08:02.814 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:08:02.815 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:08:02.815 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:02.818+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:02.827 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:08:02.828 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:08:02.828 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:08:02.849 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:08:02.849 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:08:02.870 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:08:02.870 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:08:02.891 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:08:02.891 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:08:02.894 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:08:02.894 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T09:08:02.942 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:02.943+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:02.942 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:02.943 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:02.956 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:08:03.042 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:03.043+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:03.116 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:03.118+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:03.119 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:03.119 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:03.133 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:08:03.162 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T09:08:03.216 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:03.218+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:03.312 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:03.312 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:03.317 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:03.315+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:03.330 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:08:03.419 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:03.422+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:03.460 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-14T09:08:03.460 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:08:03.461 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T09:08:03.461 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T09:08:03.488 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:03.488 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:03.503 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:08:03.520 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:03.522+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:03.620 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:03.619+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:03.668 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:03.668 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:03.682 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:08:03.726 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:03.727+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:03.769 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:08:03.770 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:03.770 INFO:teuthology.orchestra.run.smithi176.stdout: "epoch": 151, 2024-09-14T09:08:03.770 INFO:teuthology.orchestra.run.smithi176.stdout: "fsid": "c635667d-994e-454f-bcea-51b45637b2a2", 2024-09-14T09:08:03.770 INFO:teuthology.orchestra.run.smithi176.stdout: "created": "2024-09-14T09:04:57.077293+0000", 2024-09-14T09:08:03.770 INFO:teuthology.orchestra.run.smithi176.stdout: "modified": "2024-09-14T09:08:03.151334+0000", 2024-09-14T09:08:03.770 INFO:teuthology.orchestra.run.smithi176.stdout: "last_up_change": "2024-09-14T09:07:46.980109+0000", 2024-09-14T09:08:03.770 INFO:teuthology.orchestra.run.smithi176.stdout: "last_in_change": "2024-09-14T09:07:21.060249+0000", 2024-09-14T09:08:03.770 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T09:08:03.770 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_num": 5801984, 2024-09-14T09:08:03.770 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_set": [ 2024-09-14T09:08:03.770 INFO:teuthology.orchestra.run.smithi176.stdout: "noscrub", 2024-09-14T09:08:03.770 INFO:teuthology.orchestra.run.smithi176.stdout: "pglog_hardlimit", 2024-09-14T09:08:03.771 INFO:teuthology.orchestra.run.smithi176.stdout: "purged_snapdirs", 2024-09-14T09:08:03.771 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_deletes", 2024-09-14T09:08:03.771 INFO:teuthology.orchestra.run.smithi176.stdout: "sortbitwise" 2024-09-14T09:08:03.771 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:08:03.771 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_version": 5, 2024-09-14T09:08:03.771 INFO:teuthology.orchestra.run.smithi176.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T09:08:03.771 INFO:teuthology.orchestra.run.smithi176.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T09:08:03.771 INFO:teuthology.orchestra.run.smithi176.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T09:08:03.771 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_snapshot": "", 2024-09-14T09:08:03.771 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_max": 3, 2024-09-14T09:08:03.771 INFO:teuthology.orchestra.run.smithi176.stdout: "max_osd": 8, 2024-09-14T09:08:03.771 INFO:teuthology.orchestra.run.smithi176.stdout: "require_min_compat_client": "luminous", 2024-09-14T09:08:03.771 INFO:teuthology.orchestra.run.smithi176.stdout: "min_compat_client": "luminous", 2024-09-14T09:08:03.772 INFO:teuthology.orchestra.run.smithi176.stdout: "require_osd_release": "squid", 2024-09-14T09:08:03.772 INFO:teuthology.orchestra.run.smithi176.stdout: "allow_crimson": false, 2024-09-14T09:08:03.772 INFO:teuthology.orchestra.run.smithi176.stdout: "pools": [ 2024-09-14T09:08:03.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.772 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 1, 2024-09-14T09:08:03.772 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_name": ".mgr", 2024-09-14T09:08:03.772 INFO:teuthology.orchestra.run.smithi176.stdout: "create_time": "2024-09-14T09:05:11.997673+0000", 2024-09-14T09:08:03.772 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": 1, 2024-09-14T09:08:03.772 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_names": "hashpspool", 2024-09-14T09:08:03.772 INFO:teuthology.orchestra.run.smithi176.stdout: "type": 1, 2024-09-14T09:08:03.772 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 2, 2024-09-14T09:08:03.772 INFO:teuthology.orchestra.run.smithi176.stdout: "min_size": 1, 2024-09-14T09:08:03.773 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_rule": 0, 2024-09-14T09:08:03.773 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_count": 0, 2024-09-14T09:08:03.773 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_target": 0, 2024-09-14T09:08:03.773 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T09:08:03.773 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T09:08:03.773 INFO:teuthology.orchestra.run.smithi176.stdout: "is_stretch_pool": false, 2024-09-14T09:08:03.773 INFO:teuthology.orchestra.run.smithi176.stdout: "object_hash": 2, 2024-09-14T09:08:03.773 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_autoscale_mode": "off", 2024-09-14T09:08:03.773 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num": 1, 2024-09-14T09:08:03.773 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num": 1, 2024-09-14T09:08:03.773 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num_target": 1, 2024-09-14T09:08:03.774 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_target": 1, 2024-09-14T09:08:03.774 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_pending": 1, 2024-09-14T09:08:03.774 INFO:teuthology.orchestra.run.smithi176.stdout: "last_pg_merge_meta": { 2024-09-14T09:08:03.774 INFO:teuthology.orchestra.run.smithi176.stdout: "source_pgid": "0.0", 2024-09-14T09:08:03.774 INFO:teuthology.orchestra.run.smithi176.stdout: "ready_epoch": 0, 2024-09-14T09:08:03.774 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_started": 0, 2024-09-14T09:08:03.774 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_clean": 0, 2024-09-14T09:08:03.774 INFO:teuthology.orchestra.run.smithi176.stdout: "source_version": "0'0", 2024-09-14T09:08:03.774 INFO:teuthology.orchestra.run.smithi176.stdout: "target_version": "0'0" 2024-09-14T09:08:03.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.774 INFO:teuthology.orchestra.run.smithi176.stdout: "last_change": "18", 2024-09-14T09:08:03.774 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend": "0", 2024-09-14T09:08:03.774 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T09:08:03.775 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T09:08:03.775 INFO:teuthology.orchestra.run.smithi176.stdout: "auid": 0, 2024-09-14T09:08:03.775 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_mode": "selfmanaged", 2024-09-14T09:08:03.775 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_seq": 0, 2024-09-14T09:08:03.775 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_epoch": 0, 2024-09-14T09:08:03.775 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_snaps": [], 2024-09-14T09:08:03.775 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps": "[]", 2024-09-14T09:08:03.775 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_bytes": 0, 2024-09-14T09:08:03.775 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_objects": 0, 2024-09-14T09:08:03.775 INFO:teuthology.orchestra.run.smithi176.stdout: "tiers": [], 2024-09-14T09:08:03.775 INFO:teuthology.orchestra.run.smithi176.stdout: "tier_of": -1, 2024-09-14T09:08:03.775 INFO:teuthology.orchestra.run.smithi176.stdout: "read_tier": -1, 2024-09-14T09:08:03.775 INFO:teuthology.orchestra.run.smithi176.stdout: "write_tier": -1, 2024-09-14T09:08:03.775 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_mode": "none", 2024-09-14T09:08:03.776 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_bytes": 0, 2024-09-14T09:08:03.776 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_objects": 0, 2024-09-14T09:08:03.776 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T09:08:03.776 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T09:08:03.776 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T09:08:03.776 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_flush_age": 0, 2024-09-14T09:08:03.776 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_evict_age": 0, 2024-09-14T09:08:03.776 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profile": "", 2024-09-14T09:08:03.776 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_params": { 2024-09-14T09:08:03.776 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "none" 2024-09-14T09:08:03.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.776 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_period": 0, 2024-09-14T09:08:03.776 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_count": 0, 2024-09-14T09:08:03.777 INFO:teuthology.orchestra.run.smithi176.stdout: "use_gmt_hitset": true, 2024-09-14T09:08:03.777 INFO:teuthology.orchestra.run.smithi176.stdout: "min_read_recency_for_promote": 0, 2024-09-14T09:08:03.777 INFO:teuthology.orchestra.run.smithi176.stdout: "min_write_recency_for_promote": 0, 2024-09-14T09:08:03.777 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T09:08:03.777 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_search_last_n": 0, 2024-09-14T09:08:03.777 INFO:teuthology.orchestra.run.smithi176.stdout: "grade_table": [], 2024-09-14T09:08:03.777 INFO:teuthology.orchestra.run.smithi176.stdout: "stripe_width": 0, 2024-09-14T09:08:03.777 INFO:teuthology.orchestra.run.smithi176.stdout: "expected_num_objects": 0, 2024-09-14T09:08:03.777 INFO:teuthology.orchestra.run.smithi176.stdout: "fast_read": false, 2024-09-14T09:08:03.777 INFO:teuthology.orchestra.run.smithi176.stdout: "options": { 2024-09-14T09:08:03.778 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_max": 32, 2024-09-14T09:08:03.778 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_min": 1 2024-09-14T09:08:03.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.778 INFO:teuthology.orchestra.run.smithi176.stdout: "application_metadata": { 2024-09-14T09:08:03.778 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": {} 2024-09-14T09:08:03.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.778 INFO:teuthology.orchestra.run.smithi176.stdout: "read_balance": { 2024-09-14T09:08:03.779 INFO:teuthology.orchestra.run.smithi176.stdout: "score_type": "Fair distribution", 2024-09-14T09:08:03.779 INFO:teuthology.orchestra.run.smithi176.stdout: "score_acting": 8, 2024-09-14T09:08:03.780 INFO:teuthology.orchestra.run.smithi176.stdout: "score_stable": 8, 2024-09-14T09:08:03.780 INFO:teuthology.orchestra.run.smithi176.stdout: "optimal_score": 0.25, 2024-09-14T09:08:03.780 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_acting": 2, 2024-09-14T09:08:03.780 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_stable": 2, 2024-09-14T09:08:03.780 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity_weighted": 1, 2024-09-14T09:08:03.780 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity": 1, 2024-09-14T09:08:03.780 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity_weighted": 1 2024-09-14T09:08:03.780 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.780 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 2, 2024-09-14T09:08:03.780 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_name": "rbd", 2024-09-14T09:08:03.780 INFO:teuthology.orchestra.run.smithi176.stdout: "create_time": "2024-09-14T09:05:15.352551+0000", 2024-09-14T09:08:03.781 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": 8193, 2024-09-14T09:08:03.781 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T09:08:03.781 INFO:teuthology.orchestra.run.smithi176.stdout: "type": 1, 2024-09-14T09:08:03.781 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 2, 2024-09-14T09:08:03.781 INFO:teuthology.orchestra.run.smithi176.stdout: "min_size": 1, 2024-09-14T09:08:03.781 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_rule": 0, 2024-09-14T09:08:03.781 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_count": 0, 2024-09-14T09:08:03.781 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_target": 0, 2024-09-14T09:08:03.781 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T09:08:03.781 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T09:08:03.781 INFO:teuthology.orchestra.run.smithi176.stdout: "is_stretch_pool": false, 2024-09-14T09:08:03.781 INFO:teuthology.orchestra.run.smithi176.stdout: "object_hash": 2, 2024-09-14T09:08:03.781 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_autoscale_mode": "off", 2024-09-14T09:08:03.782 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num": 8, 2024-09-14T09:08:03.782 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num": 8, 2024-09-14T09:08:03.782 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num_target": 8, 2024-09-14T09:08:03.782 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_target": 8, 2024-09-14T09:08:03.782 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_pending": 8, 2024-09-14T09:08:03.782 INFO:teuthology.orchestra.run.smithi176.stdout: "last_pg_merge_meta": { 2024-09-14T09:08:03.782 INFO:teuthology.orchestra.run.smithi176.stdout: "source_pgid": "0.0", 2024-09-14T09:08:03.782 INFO:teuthology.orchestra.run.smithi176.stdout: "ready_epoch": 0, 2024-09-14T09:08:03.782 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_started": 0, 2024-09-14T09:08:03.782 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_clean": 0, 2024-09-14T09:08:03.782 INFO:teuthology.orchestra.run.smithi176.stdout: "source_version": "0'0", 2024-09-14T09:08:03.782 INFO:teuthology.orchestra.run.smithi176.stdout: "target_version": "0'0" 2024-09-14T09:08:03.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.782 INFO:teuthology.orchestra.run.smithi176.stdout: "last_change": "22", 2024-09-14T09:08:03.782 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend": "0", 2024-09-14T09:08:03.783 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T09:08:03.783 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T09:08:03.783 INFO:teuthology.orchestra.run.smithi176.stdout: "auid": 0, 2024-09-14T09:08:03.783 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_mode": "selfmanaged", 2024-09-14T09:08:03.783 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_seq": 2, 2024-09-14T09:08:03.783 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_epoch": 22, 2024-09-14T09:08:03.783 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_snaps": [], 2024-09-14T09:08:03.783 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps": "[]", 2024-09-14T09:08:03.783 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_bytes": 0, 2024-09-14T09:08:03.783 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_objects": 0, 2024-09-14T09:08:03.783 INFO:teuthology.orchestra.run.smithi176.stdout: "tiers": [], 2024-09-14T09:08:03.783 INFO:teuthology.orchestra.run.smithi176.stdout: "tier_of": -1, 2024-09-14T09:08:03.783 INFO:teuthology.orchestra.run.smithi176.stdout: "read_tier": -1, 2024-09-14T09:08:03.784 INFO:teuthology.orchestra.run.smithi176.stdout: "write_tier": -1, 2024-09-14T09:08:03.784 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_mode": "none", 2024-09-14T09:08:03.784 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_bytes": 0, 2024-09-14T09:08:03.784 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_objects": 0, 2024-09-14T09:08:03.784 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T09:08:03.784 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T09:08:03.784 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T09:08:03.784 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_flush_age": 0, 2024-09-14T09:08:03.784 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_evict_age": 0, 2024-09-14T09:08:03.784 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profile": "", 2024-09-14T09:08:03.784 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_params": { 2024-09-14T09:08:03.784 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "none" 2024-09-14T09:08:03.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.784 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_period": 0, 2024-09-14T09:08:03.785 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_count": 0, 2024-09-14T09:08:03.785 INFO:teuthology.orchestra.run.smithi176.stdout: "use_gmt_hitset": true, 2024-09-14T09:08:03.785 INFO:teuthology.orchestra.run.smithi176.stdout: "min_read_recency_for_promote": 0, 2024-09-14T09:08:03.785 INFO:teuthology.orchestra.run.smithi176.stdout: "min_write_recency_for_promote": 0, 2024-09-14T09:08:03.785 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T09:08:03.785 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_search_last_n": 0, 2024-09-14T09:08:03.785 INFO:teuthology.orchestra.run.smithi176.stdout: "grade_table": [], 2024-09-14T09:08:03.785 INFO:teuthology.orchestra.run.smithi176.stdout: "stripe_width": 0, 2024-09-14T09:08:03.785 INFO:teuthology.orchestra.run.smithi176.stdout: "expected_num_objects": 0, 2024-09-14T09:08:03.785 INFO:teuthology.orchestra.run.smithi176.stdout: "fast_read": false, 2024-09-14T09:08:03.785 INFO:teuthology.orchestra.run.smithi176.stdout: "options": {}, 2024-09-14T09:08:03.785 INFO:teuthology.orchestra.run.smithi176.stdout: "application_metadata": { 2024-09-14T09:08:03.785 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd": {} 2024-09-14T09:08:03.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.786 INFO:teuthology.orchestra.run.smithi176.stdout: "read_balance": { 2024-09-14T09:08:03.786 INFO:teuthology.orchestra.run.smithi176.stdout: "score_type": "Fair distribution", 2024-09-14T09:08:03.786 INFO:teuthology.orchestra.run.smithi176.stdout: "score_acting": 1.9900000095367432, 2024-09-14T09:08:03.786 INFO:teuthology.orchestra.run.smithi176.stdout: "score_stable": 1.9900000095367432, 2024-09-14T09:08:03.786 INFO:teuthology.orchestra.run.smithi176.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T09:08:03.786 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_acting": 1.75, 2024-09-14T09:08:03.786 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_stable": 1.75, 2024-09-14T09:08:03.786 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity_weighted": 1, 2024-09-14T09:08:03.786 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity": 1, 2024-09-14T09:08:03.786 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity_weighted": 1 2024-09-14T09:08:03.786 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.787 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 3, 2024-09-14T09:08:03.787 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_name": "pool_client.0", 2024-09-14T09:08:03.787 INFO:teuthology.orchestra.run.smithi176.stdout: "create_time": "2024-09-14T09:05:25.840963+0000", 2024-09-14T09:08:03.787 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": 8193, 2024-09-14T09:08:03.787 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T09:08:03.787 INFO:teuthology.orchestra.run.smithi176.stdout: "type": 1, 2024-09-14T09:08:03.787 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 2, 2024-09-14T09:08:03.787 INFO:teuthology.orchestra.run.smithi176.stdout: "min_size": 1, 2024-09-14T09:08:03.787 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_rule": 0, 2024-09-14T09:08:03.787 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_count": 0, 2024-09-14T09:08:03.787 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_target": 0, 2024-09-14T09:08:03.787 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T09:08:03.787 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T09:08:03.787 INFO:teuthology.orchestra.run.smithi176.stdout: "is_stretch_pool": false, 2024-09-14T09:08:03.788 INFO:teuthology.orchestra.run.smithi176.stdout: "object_hash": 2, 2024-09-14T09:08:03.788 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_autoscale_mode": "off", 2024-09-14T09:08:03.788 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num": 32, 2024-09-14T09:08:03.788 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num": 32, 2024-09-14T09:08:03.788 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num_target": 32, 2024-09-14T09:08:03.788 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_target": 32, 2024-09-14T09:08:03.788 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_pending": 32, 2024-09-14T09:08:03.788 INFO:teuthology.orchestra.run.smithi176.stdout: "last_pg_merge_meta": { 2024-09-14T09:08:03.788 INFO:teuthology.orchestra.run.smithi176.stdout: "source_pgid": "0.0", 2024-09-14T09:08:03.788 INFO:teuthology.orchestra.run.smithi176.stdout: "ready_epoch": 0, 2024-09-14T09:08:03.788 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_started": 0, 2024-09-14T09:08:03.788 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_clean": 0, 2024-09-14T09:08:03.788 INFO:teuthology.orchestra.run.smithi176.stdout: "source_version": "0'0", 2024-09-14T09:08:03.788 INFO:teuthology.orchestra.run.smithi176.stdout: "target_version": "0'0" 2024-09-14T09:08:03.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.789 INFO:teuthology.orchestra.run.smithi176.stdout: "last_change": "151", 2024-09-14T09:08:03.789 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend": "0", 2024-09-14T09:08:03.789 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T09:08:03.789 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T09:08:03.789 INFO:teuthology.orchestra.run.smithi176.stdout: "auid": 0, 2024-09-14T09:08:03.789 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_mode": "selfmanaged", 2024-09-14T09:08:03.789 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_seq": 46, 2024-09-14T09:08:03.789 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_epoch": 151, 2024-09-14T09:08:03.789 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_snaps": [], 2024-09-14T09:08:03.789 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps": "[]", 2024-09-14T09:08:03.789 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_bytes": 0, 2024-09-14T09:08:03.789 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_objects": 0, 2024-09-14T09:08:03.790 INFO:teuthology.orchestra.run.smithi176.stdout: "tiers": [], 2024-09-14T09:08:03.790 INFO:teuthology.orchestra.run.smithi176.stdout: "tier_of": -1, 2024-09-14T09:08:03.790 INFO:teuthology.orchestra.run.smithi176.stdout: "read_tier": -1, 2024-09-14T09:08:03.790 INFO:teuthology.orchestra.run.smithi176.stdout: "write_tier": -1, 2024-09-14T09:08:03.790 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_mode": "none", 2024-09-14T09:08:03.790 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_bytes": 0, 2024-09-14T09:08:03.790 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_objects": 0, 2024-09-14T09:08:03.790 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T09:08:03.790 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T09:08:03.790 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T09:08:03.790 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_flush_age": 0, 2024-09-14T09:08:03.790 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_evict_age": 0, 2024-09-14T09:08:03.790 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profile": "", 2024-09-14T09:08:03.790 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_params": { 2024-09-14T09:08:03.791 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "none" 2024-09-14T09:08:03.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.791 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_period": 0, 2024-09-14T09:08:03.791 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_count": 0, 2024-09-14T09:08:03.791 INFO:teuthology.orchestra.run.smithi176.stdout: "use_gmt_hitset": true, 2024-09-14T09:08:03.791 INFO:teuthology.orchestra.run.smithi176.stdout: "min_read_recency_for_promote": 0, 2024-09-14T09:08:03.791 INFO:teuthology.orchestra.run.smithi176.stdout: "min_write_recency_for_promote": 0, 2024-09-14T09:08:03.791 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T09:08:03.791 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_search_last_n": 0, 2024-09-14T09:08:03.791 INFO:teuthology.orchestra.run.smithi176.stdout: "grade_table": [], 2024-09-14T09:08:03.791 INFO:teuthology.orchestra.run.smithi176.stdout: "stripe_width": 0, 2024-09-14T09:08:03.791 INFO:teuthology.orchestra.run.smithi176.stdout: "expected_num_objects": 0, 2024-09-14T09:08:03.791 INFO:teuthology.orchestra.run.smithi176.stdout: "fast_read": false, 2024-09-14T09:08:03.791 INFO:teuthology.orchestra.run.smithi176.stdout: "options": {}, 2024-09-14T09:08:03.792 INFO:teuthology.orchestra.run.smithi176.stdout: "application_metadata": { 2024-09-14T09:08:03.792 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd": {} 2024-09-14T09:08:03.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.792 INFO:teuthology.orchestra.run.smithi176.stdout: "read_balance": { 2024-09-14T09:08:03.792 INFO:teuthology.orchestra.run.smithi176.stdout: "score_type": "Fair distribution", 2024-09-14T09:08:03.792 INFO:teuthology.orchestra.run.smithi176.stdout: "score_acting": 1.7400000095367432, 2024-09-14T09:08:03.792 INFO:teuthology.orchestra.run.smithi176.stdout: "score_stable": 1.7400000095367432, 2024-09-14T09:08:03.792 INFO:teuthology.orchestra.run.smithi176.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T09:08:03.792 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-14T09:08:03.792 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-14T09:08:03.792 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity_weighted": 1, 2024-09-14T09:08:03.792 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity": 1, 2024-09-14T09:08:03.792 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity_weighted": 1 2024-09-14T09:08:03.792 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.793 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.793 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:08:03.793 INFO:teuthology.orchestra.run.smithi176.stdout: "osds": [ 2024-09-14T09:08:03.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.793 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 0, 2024-09-14T09:08:03.793 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "4ceedb4f-6398-467f-9562-ccf27200860b", 2024-09-14T09:08:03.793 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:08:03.793 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:08:03.793 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:08:03.793 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:08:03.793 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:08:03.793 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:08:03.793 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:08:03.794 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 135, 2024-09-14T09:08:03.794 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:08:03.794 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:08:03.794 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:08:03.794 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.794 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.794 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6824", 2024-09-14T09:08:03.794 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:08:03.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.794 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.794 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6825", 2024-09-14T09:08:03.795 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:08:03.795 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.795 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.795 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:08:03.795 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.795 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.795 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6826", 2024-09-14T09:08:03.795 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:08:03.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.795 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.796 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6827", 2024-09-14T09:08:03.796 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:08:03.796 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.796 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.796 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:08:03.796 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.796 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.796 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6830", 2024-09-14T09:08:03.796 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:08:03.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.797 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.797 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6831", 2024-09-14T09:08:03.797 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:08:03.797 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.797 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.797 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:08:03.797 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.797 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.797 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6828", 2024-09-14T09:08:03.797 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:08:03.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.798 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.798 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6829", 2024-09-14T09:08:03.798 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:08:03.798 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.798 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.798 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6825/3384710819", 2024-09-14T09:08:03.798 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6827/3384710819", 2024-09-14T09:08:03.798 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6831/3384710819", 2024-09-14T09:08:03.798 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6829/3384710819", 2024-09-14T09:08:03.798 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:08:03.798 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:08:03.799 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:08:03.799 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.799 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 1, 2024-09-14T09:08:03.799 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "604e2f62-7c13-46be-a97f-029b28c44a89", 2024-09-14T09:08:03.799 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:08:03.799 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:08:03.799 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:08:03.799 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:08:03.799 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:08:03.799 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:08:03.799 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:08:03.800 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 135, 2024-09-14T09:08:03.800 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:08:03.800 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:08:03.800 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:08:03.800 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.800 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.800 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6800", 2024-09-14T09:08:03.800 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:08:03.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.800 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.800 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6801", 2024-09-14T09:08:03.800 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:08:03.801 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.801 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.801 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:08:03.801 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.801 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.801 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6802", 2024-09-14T09:08:03.801 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:08:03.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.801 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.801 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6803", 2024-09-14T09:08:03.801 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:08:03.802 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.802 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.802 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:08:03.802 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.802 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.802 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6806", 2024-09-14T09:08:03.802 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:08:03.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.802 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.802 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6807", 2024-09-14T09:08:03.802 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:08:03.803 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.803 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.803 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:08:03.803 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.803 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.803 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6804", 2024-09-14T09:08:03.803 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:08:03.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.803 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.803 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6805", 2024-09-14T09:08:03.803 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:08:03.804 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.804 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.804 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6801/612012679", 2024-09-14T09:08:03.804 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6803/612012679", 2024-09-14T09:08:03.804 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6807/612012679", 2024-09-14T09:08:03.804 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6805/612012679", 2024-09-14T09:08:03.804 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:08:03.804 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:08:03.804 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:08:03.804 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.804 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 2, 2024-09-14T09:08:03.804 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "1f65589e-8b0a-4033-94c0-4ab66531db2b", 2024-09-14T09:08:03.805 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:08:03.805 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:08:03.805 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:08:03.805 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:08:03.805 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:08:03.805 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:08:03.805 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:08:03.805 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 118, 2024-09-14T09:08:03.805 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:08:03.805 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:08:03.805 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:08:03.805 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.806 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.806 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6808", 2024-09-14T09:08:03.806 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:08:03.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.806 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.806 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6809", 2024-09-14T09:08:03.806 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:08:03.806 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.806 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.806 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:08:03.806 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.807 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.807 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6810", 2024-09-14T09:08:03.807 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:08:03.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.807 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.807 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6811", 2024-09-14T09:08:03.807 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:08:03.807 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.807 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.808 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:08:03.808 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.808 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.808 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6814", 2024-09-14T09:08:03.808 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:08:03.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.808 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.808 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6815", 2024-09-14T09:08:03.808 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:08:03.808 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.808 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.809 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:08:03.809 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.809 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.809 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6812", 2024-09-14T09:08:03.809 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:08:03.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.809 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.809 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6813", 2024-09-14T09:08:03.809 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:08:03.809 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.809 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.810 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6809/4167379279", 2024-09-14T09:08:03.810 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6811/4167379279", 2024-09-14T09:08:03.810 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6815/4167379279", 2024-09-14T09:08:03.810 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6813/4167379279", 2024-09-14T09:08:03.810 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:08:03.810 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:08:03.810 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:08:03.810 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.810 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 3, 2024-09-14T09:08:03.810 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "209d4f65-79f9-4a7d-9d23-832a4209507f", 2024-09-14T09:08:03.810 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:08:03.811 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:08:03.811 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:08:03.811 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:08:03.811 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:08:03.811 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:08:03.811 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:08:03.811 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 117, 2024-09-14T09:08:03.811 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:08:03.811 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:08:03.811 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:08:03.811 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.811 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.811 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6816", 2024-09-14T09:08:03.812 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:08:03.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.812 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.812 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6817", 2024-09-14T09:08:03.812 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:08:03.812 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.812 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.812 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:08:03.812 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.812 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.813 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6818", 2024-09-14T09:08:03.813 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:08:03.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.813 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.813 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6819", 2024-09-14T09:08:03.813 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:08:03.813 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.813 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.813 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:08:03.813 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.814 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.814 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6822", 2024-09-14T09:08:03.814 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:08:03.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.814 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.814 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6823", 2024-09-14T09:08:03.814 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:08:03.814 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.814 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.814 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:08:03.814 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.815 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.815 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6820", 2024-09-14T09:08:03.815 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:08:03.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.815 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.815 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6821", 2024-09-14T09:08:03.815 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:08:03.815 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.815 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.816 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6817/1658209015", 2024-09-14T09:08:03.816 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6819/1658209015", 2024-09-14T09:08:03.816 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6823/1658209015", 2024-09-14T09:08:03.816 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6821/1658209015", 2024-09-14T09:08:03.816 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:08:03.816 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:08:03.816 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:08:03.816 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.816 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 4, 2024-09-14T09:08:03.816 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "8f9a8c9b-282e-4523-9e62-5824f4aba401", 2024-09-14T09:08:03.817 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 0, 2024-09-14T09:08:03.817 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:08:03.817 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:08:03.817 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:08:03.817 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:08:03.817 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:08:03.817 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:08:03.817 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 116, 2024-09-14T09:08:03.817 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 135, 2024-09-14T09:08:03.817 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:08:03.817 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:08:03.817 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.817 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.818 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6808", 2024-09-14T09:08:03.818 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:08:03.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.818 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.818 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6809", 2024-09-14T09:08:03.818 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:08:03.818 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.818 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.818 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:08:03.818 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.819 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.819 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6810", 2024-09-14T09:08:03.819 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:08:03.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.819 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.819 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6811", 2024-09-14T09:08:03.819 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:08:03.819 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.819 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.819 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:08:03.819 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.820 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.820 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6814", 2024-09-14T09:08:03.820 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:08:03.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.820 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.820 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6815", 2024-09-14T09:08:03.820 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:08:03.820 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.820 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.820 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:08:03.820 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.821 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.821 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6812", 2024-09-14T09:08:03.821 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:08:03.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.821 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.821 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6813", 2024-09-14T09:08:03.821 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:08:03.821 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.821 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.821 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6809/1576054389", 2024-09-14T09:08:03.821 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6811/1576054389", 2024-09-14T09:08:03.822 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6815/1576054389", 2024-09-14T09:08:03.822 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6813/1576054389", 2024-09-14T09:08:03.822 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:08:03.822 INFO:teuthology.orchestra.run.smithi176.stdout: "exists" 2024-09-14T09:08:03.822 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.822 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 5, 2024-09-14T09:08:03.822 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80", 2024-09-14T09:08:03.822 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:08:03.822 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:08:03.822 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:08:03.822 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:08:03.823 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:08:03.823 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:08:03.823 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:08:03.823 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 135, 2024-09-14T09:08:03.823 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:08:03.823 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:08:03.823 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:08:03.823 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.823 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.823 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6824", 2024-09-14T09:08:03.823 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:08:03.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.824 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.824 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6825", 2024-09-14T09:08:03.824 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:08:03.824 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.824 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.824 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:08:03.824 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.824 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.824 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6826", 2024-09-14T09:08:03.824 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:08:03.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.825 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.825 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6827", 2024-09-14T09:08:03.825 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:08:03.825 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.825 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.825 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:08:03.825 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.825 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.825 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6830", 2024-09-14T09:08:03.825 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:08:03.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.826 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.826 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6831", 2024-09-14T09:08:03.826 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:08:03.826 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.826 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.826 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:08:03.826 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.826 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.826 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6828", 2024-09-14T09:08:03.827 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:08:03.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.827 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.827 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6829", 2024-09-14T09:08:03.827 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:08:03.827 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.827 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.827 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6825/1711292386", 2024-09-14T09:08:03.827 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6827/1711292386", 2024-09-14T09:08:03.827 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6831/1711292386", 2024-09-14T09:08:03.827 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6829/1711292386", 2024-09-14T09:08:03.828 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:08:03.828 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:08:03.828 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:08:03.828 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.828 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 6, 2024-09-14T09:08:03.828 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "59d9259d-b9dd-4326-8ab9-9d599a2bf57b", 2024-09-14T09:08:03.828 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:08:03.828 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:08:03.828 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:08:03.828 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:08:03.829 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:08:03.829 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:08:03.829 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:08:03.829 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 135, 2024-09-14T09:08:03.829 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:08:03.829 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:08:03.829 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:08:03.829 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.829 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.829 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6816", 2024-09-14T09:08:03.829 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:08:03.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.830 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.830 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6817", 2024-09-14T09:08:03.830 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:08:03.830 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.830 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.830 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:08:03.830 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.830 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.830 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6818", 2024-09-14T09:08:03.831 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:08:03.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.831 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.831 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6819", 2024-09-14T09:08:03.831 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:08:03.831 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.831 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.831 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:08:03.831 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.831 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.831 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6822", 2024-09-14T09:08:03.832 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:08:03.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.832 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.832 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6823", 2024-09-14T09:08:03.832 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:08:03.832 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.832 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.832 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:08:03.832 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.832 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.833 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6820", 2024-09-14T09:08:03.833 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:08:03.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.833 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.833 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6821", 2024-09-14T09:08:03.833 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:08:03.833 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.833 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.833 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6817/2721245607", 2024-09-14T09:08:03.833 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6819/2721245607", 2024-09-14T09:08:03.833 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6823/2721245607", 2024-09-14T09:08:03.834 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6821/2721245607", 2024-09-14T09:08:03.834 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:08:03.834 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:08:03.834 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:08:03.834 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.834 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 7, 2024-09-14T09:08:03.834 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "e2f27d42-786b-449b-a2fb-9dd28f4e9788", 2024-09-14T09:08:03.834 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:08:03.834 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:08:03.834 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:08:03.834 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:08:03.834 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:08:03.835 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:08:03.835 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:08:03.835 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 135, 2024-09-14T09:08:03.835 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:08:03.835 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:08:03.835 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:08:03.835 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.835 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.835 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6800", 2024-09-14T09:08:03.835 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:08:03.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.835 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.836 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6801", 2024-09-14T09:08:03.836 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:08:03.836 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.836 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.836 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:08:03.836 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.836 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.836 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6802", 2024-09-14T09:08:03.836 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:08:03.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.837 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.837 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6803", 2024-09-14T09:08:03.837 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:08:03.837 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.837 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.837 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:08:03.837 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.837 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.837 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6806", 2024-09-14T09:08:03.837 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:08:03.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.838 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.838 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6807", 2024-09-14T09:08:03.838 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:08:03.838 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.838 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.838 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:08:03.838 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:03.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.838 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:03.838 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6804", 2024-09-14T09:08:03.838 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:08:03.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.839 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:03.839 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6805", 2024-09-14T09:08:03.839 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:08:03.839 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.839 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.839 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6801/2122823465", 2024-09-14T09:08:03.839 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6803/2122823465", 2024-09-14T09:08:03.839 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6807/2122823465", 2024-09-14T09:08:03.839 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6805/2122823465", 2024-09-14T09:08:03.839 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:08:03.839 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:08:03.839 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:08:03.839 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.840 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.840 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:08:03.840 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_xinfo": [ 2024-09-14T09:08:03.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.840 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 0, 2024-09-14T09:08:03.840 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:08:03.840 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:08:03.840 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:08:03.840 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:08:03.840 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:08:03.840 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.411789+0000", 2024-09-14T09:08:03.840 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:08:03.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.841 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 1, 2024-09-14T09:08:03.841 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:08:03.841 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:08:03.841 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:08:03.841 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:08:03.841 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:08:03.841 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:05.801873+0000", 2024-09-14T09:08:03.841 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:08:03.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.841 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 2, 2024-09-14T09:08:03.841 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:08:03.841 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:08:03.842 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:08:03.842 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:08:03.842 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:08:03.842 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.206725+0000", 2024-09-14T09:08:03.842 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:08:03.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.842 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 3, 2024-09-14T09:08:03.842 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:08:03.842 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:08:03.842 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:08:03.842 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:08:03.842 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:08:03.842 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:05.882180+0000", 2024-09-14T09:08:03.842 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:08:03.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.843 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 4, 2024-09-14T09:08:03.843 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:08:03.843 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:08:03.843 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:08:03.843 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:08:03.843 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:08:03.843 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.013864+0000", 2024-09-14T09:08:03.843 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 135 2024-09-14T09:08:03.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.843 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 5, 2024-09-14T09:08:03.843 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:08:03.844 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:08:03.844 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:08:03.844 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:08:03.844 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:08:03.844 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.652975+0000", 2024-09-14T09:08:03.844 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:08:03.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.844 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 6, 2024-09-14T09:08:03.844 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:08:03.844 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:08:03.844 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:08:03.844 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:08:03.845 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:08:03.845 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.741359+0000", 2024-09-14T09:08:03.845 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:08:03.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.845 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 7, 2024-09-14T09:08:03.845 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:08:03.845 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:08:03.845 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:08:03.845 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:08:03.845 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:08:03.845 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:05.873675+0000", 2024-09-14T09:08:03.845 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:08:03.845 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.846 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:08:03.846 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_upmap": [ 2024-09-14T09:08:03.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.846 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "2.3", 2024-09-14T09:08:03.846 INFO:teuthology.orchestra.run.smithi176.stdout: "osds": [ 2024-09-14T09:08:03.846 INFO:teuthology.orchestra.run.smithi176.stdout: 3, 2024-09-14T09:08:03.846 INFO:teuthology.orchestra.run.smithi176.stdout: 2 2024-09-14T09:08:03.846 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.846 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "3.4", 2024-09-14T09:08:03.846 INFO:teuthology.orchestra.run.smithi176.stdout: "osds": [ 2024-09-14T09:08:03.846 INFO:teuthology.orchestra.run.smithi176.stdout: 1, 2024-09-14T09:08:03.846 INFO:teuthology.orchestra.run.smithi176.stdout: 7 2024-09-14T09:08:03.847 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.847 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.847 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:08:03.847 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_upmap_items": [ 2024-09-14T09:08:03.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.847 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "3.9", 2024-09-14T09:08:03.847 INFO:teuthology.orchestra.run.smithi176.stdout: "mappings": [ 2024-09-14T09:08:03.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.847 INFO:teuthology.orchestra.run.smithi176.stdout: "from": 4, 2024-09-14T09:08:03.847 INFO:teuthology.orchestra.run.smithi176.stdout: "to": 6 2024-09-14T09:08:03.847 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.847 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.847 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "3.19", 2024-09-14T09:08:03.848 INFO:teuthology.orchestra.run.smithi176.stdout: "mappings": [ 2024-09-14T09:08:03.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.848 INFO:teuthology.orchestra.run.smithi176.stdout: "from": 3, 2024-09-14T09:08:03.848 INFO:teuthology.orchestra.run.smithi176.stdout: "to": 0 2024-09-14T09:08:03.848 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.848 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.848 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "3.1b", 2024-09-14T09:08:03.848 INFO:teuthology.orchestra.run.smithi176.stdout: "mappings": [ 2024-09-14T09:08:03.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.848 INFO:teuthology.orchestra.run.smithi176.stdout: "from": 4, 2024-09-14T09:08:03.848 INFO:teuthology.orchestra.run.smithi176.stdout: "to": 6 2024-09-14T09:08:03.849 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.849 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.849 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.849 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:08:03.849 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_upmap_primaries": [], 2024-09-14T09:08:03.849 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_temp": [], 2024-09-14T09:08:03.849 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_temp": [], 2024-09-14T09:08:03.849 INFO:teuthology.orchestra.run.smithi176.stdout: "blocklist": {}, 2024-09-14T09:08:03.849 INFO:teuthology.orchestra.run.smithi176.stdout: "range_blocklist": {}, 2024-09-14T09:08:03.849 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profiles": { 2024-09-14T09:08:03.849 INFO:teuthology.orchestra.run.smithi176.stdout: "default": { 2024-09-14T09:08:03.849 INFO:teuthology.orchestra.run.smithi176.stdout: "crush-failure-domain": "osd", 2024-09-14T09:08:03.849 INFO:teuthology.orchestra.run.smithi176.stdout: "k": "2", 2024-09-14T09:08:03.849 INFO:teuthology.orchestra.run.smithi176.stdout: "m": "1", 2024-09-14T09:08:03.850 INFO:teuthology.orchestra.run.smithi176.stdout: "plugin": "jerasure", 2024-09-14T09:08:03.850 INFO:teuthology.orchestra.run.smithi176.stdout: "technique": "reed_sol_van" 2024-09-14T09:08:03.850 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:03.850 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps_queue": [ 2024-09-14T09:08:03.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.850 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 3, 2024-09-14T09:08:03.850 INFO:teuthology.orchestra.run.smithi176.stdout: "snaps": [ 2024-09-14T09:08:03.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.850 INFO:teuthology.orchestra.run.smithi176.stdout: "begin": 33, 2024-09-14T09:08:03.850 INFO:teuthology.orchestra.run.smithi176.stdout: "length": 14 2024-09-14T09:08:03.850 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.850 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.850 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.851 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:08:03.851 INFO:teuthology.orchestra.run.smithi176.stdout: "new_removed_snaps": [ 2024-09-14T09:08:03.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.851 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 3, 2024-09-14T09:08:03.851 INFO:teuthology.orchestra.run.smithi176.stdout: "snaps": [ 2024-09-14T09:08:03.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:03.851 INFO:teuthology.orchestra.run.smithi176.stdout: "begin": 33, 2024-09-14T09:08:03.851 INFO:teuthology.orchestra.run.smithi176.stdout: "length": 1 2024-09-14T09:08:03.851 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.851 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:03.851 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.851 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:08:03.851 INFO:teuthology.orchestra.run.smithi176.stdout: "new_purged_snaps": [], 2024-09-14T09:08:03.851 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_node_flags": {}, 2024-09-14T09:08:03.852 INFO:teuthology.orchestra.run.smithi176.stdout: "device_class_flags": {}, 2024-09-14T09:08:03.852 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_mode": { 2024-09-14T09:08:03.852 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_mode_enabled": false, 2024-09-14T09:08:03.852 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_bucket_count": 0, 2024-09-14T09:08:03.852 INFO:teuthology.orchestra.run.smithi176.stdout: "degraded_stretch_mode": 0, 2024-09-14T09:08:03.852 INFO:teuthology.orchestra.run.smithi176.stdout: "recovering_stretch_mode": 0, 2024-09-14T09:08:03.852 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_mode_bucket": 0 2024-09-14T09:08:03.852 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:03.852 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:03.854 INFO:tasks.thrashosds.thrasher:j is {'epoch': 151, 'fsid': 'c635667d-994e-454f-bcea-51b45637b2a2', 'created': '2024-09-14T09:04:57.077293+0000', 'modified': '2024-09-14T09:08:03.151334+0000', 'last_up_change': '2024-09-14T09:07:46.980109+0000', 'last_in_change': '2024-09-14T09:07:21.060249+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T09:05:11.997673+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T09:05:15.352551+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T09:05:25.840963+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 46, 'snap_epoch': 151, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '4ceedb4f-6398-467f-9562-ccf27200860b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 135, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6824', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6825', 'nonce': 3384710819}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6826', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6827', 'nonce': 3384710819}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6830', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6831', 'nonce': 3384710819}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6828', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6829', 'nonce': 3384710819}]}, 'public_addr': '172.21.15.176:6825/3384710819', 'cluster_addr': '172.21.15.176:6827/3384710819', 'heartbeat_back_addr': '172.21.15.176:6831/3384710819', 'heartbeat_front_addr': '172.21.15.176:6829/3384710819', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '604e2f62-7c13-46be-a97f-029b28c44a89', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 135, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6800', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6801', 'nonce': 612012679}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6802', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6803', 'nonce': 612012679}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6806', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6807', 'nonce': 612012679}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6804', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6805', 'nonce': 612012679}]}, 'public_addr': '172.21.15.176:6801/612012679', 'cluster_addr': '172.21.15.176:6803/612012679', 'heartbeat_back_addr': '172.21.15.176:6807/612012679', 'heartbeat_front_addr': '172.21.15.176:6805/612012679', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1f65589e-8b0a-4033-94c0-4ab66531db2b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 118, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6808', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6809', 'nonce': 4167379279}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6810', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6811', 'nonce': 4167379279}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6814', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6815', 'nonce': 4167379279}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6812', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6813', 'nonce': 4167379279}]}, 'public_addr': '172.21.15.176:6809/4167379279', 'cluster_addr': '172.21.15.176:6811/4167379279', 'heartbeat_back_addr': '172.21.15.176:6815/4167379279', 'heartbeat_front_addr': '172.21.15.176:6813/4167379279', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '209d4f65-79f9-4a7d-9d23-832a4209507f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6816', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6817', 'nonce': 1658209015}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6818', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6819', 'nonce': 1658209015}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6822', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6823', 'nonce': 1658209015}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6820', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6821', 'nonce': 1658209015}]}, 'public_addr': '172.21.15.176:6817/1658209015', 'cluster_addr': '172.21.15.176:6819/1658209015', 'heartbeat_back_addr': '172.21.15.176:6823/1658209015', 'heartbeat_front_addr': '172.21.15.176:6821/1658209015', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8f9a8c9b-282e-4523-9e62-5824f4aba401', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 116, 'down_at': 135, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6808', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6809', 'nonce': 1576054389}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6810', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6811', 'nonce': 1576054389}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6814', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6815', 'nonce': 1576054389}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6812', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6813', 'nonce': 1576054389}]}, 'public_addr': '172.21.15.183:6809/1576054389', 'cluster_addr': '172.21.15.183:6811/1576054389', 'heartbeat_back_addr': '172.21.15.183:6815/1576054389', 'heartbeat_front_addr': '172.21.15.183:6813/1576054389', 'state': ['exists']}, {'osd': 5, 'uuid': '1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 135, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6824', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6825', 'nonce': 1711292386}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6826', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6827', 'nonce': 1711292386}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6830', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6831', 'nonce': 1711292386}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6828', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6829', 'nonce': 1711292386}]}, 'public_addr': '172.21.15.183:6825/1711292386', 'cluster_addr': '172.21.15.183:6827/1711292386', 'heartbeat_back_addr': '172.21.15.183:6831/1711292386', 'heartbeat_front_addr': '172.21.15.183:6829/1711292386', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '59d9259d-b9dd-4326-8ab9-9d599a2bf57b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 135, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6816', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6817', 'nonce': 2721245607}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6818', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6819', 'nonce': 2721245607}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6822', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6823', 'nonce': 2721245607}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6820', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6821', 'nonce': 2721245607}]}, 'public_addr': '172.21.15.183:6817/2721245607', 'cluster_addr': '172.21.15.183:6819/2721245607', 'heartbeat_back_addr': '172.21.15.183:6823/2721245607', 'heartbeat_front_addr': '172.21.15.183:6821/2721245607', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'e2f27d42-786b-449b-a2fb-9dd28f4e9788', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 135, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6800', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6801', 'nonce': 2122823465}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6802', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6803', 'nonce': 2122823465}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6806', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6807', 'nonce': 2122823465}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6804', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6805', 'nonce': 2122823465}]}, 'public_addr': '172.21.15.183:6801/2122823465', 'cluster_addr': '172.21.15.183:6803/2122823465', 'heartbeat_back_addr': '172.21.15.183:6807/2122823465', 'heartbeat_front_addr': '172.21.15.183:6805/2122823465', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.411789+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:05.801873+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.206725+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:05.882180+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.013864+0000', 'dead_epoch': 135}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.652975+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.741359+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:05.873675+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.3', 'osds': [3, 2]}, {'pgid': '3.4', 'osds': [1, 7]}], 'pg_upmap_items': [{'pgid': '3.9', 'mappings': [{'from': 4, 'to': 6}]}, {'pgid': '3.19', 'mappings': [{'from': 3, 'to': 0}]}, {'pgid': '3.1b', 'mappings': [{'from': 4, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 33, 'length': 14}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 33, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T09:08:03.854 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 2.3 2024-09-14T09:08:03.855 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 2.3 2024-09-14T09:08:03.855 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:03.855 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:03.855 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:03.855+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:03.857 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:08:03.955 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:03.958+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:03.987 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:03.987 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:04.001 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:08:04.056 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:04.055+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:04.157 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:04.155+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:04.167 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:04.167 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:04.170 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is set 2024-09-14T09:08:04.186 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:08:04.257 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:04.258+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:04.310 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:04.310 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:04.324 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:08:04.358 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:04.359+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:04.460 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:04.459+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:04.491 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:04.491 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:04.505 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:08:04.561 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:04.559+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:04.629 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:04.629 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:04.644 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:08:04.662 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:04.663+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:04.763 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:04.766+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:04.766 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:04.766 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:04.780 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:08:04.864 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:04.863+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:04.895 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:08:04.941 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:04.941 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:04.955 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:08:04.965 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:04.963+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:05.049 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:08:05.049 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:08:05.065 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:05.063+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:05.067 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:08:05.067 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:08:05.075 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:05.075 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:05.084 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:08:05.084 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:08:05.089 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:08:05.101 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:08:05.101 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:08:05.101 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:08:05.119 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:08:05.119 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:08:05.137 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:08:05.137 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:08:05.156 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:08:05.156 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:08:05.160 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:08:05.160 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T09:08:05.171 INFO:teuthology.orchestra.run.smithi176.stderr:clear 2.3 pg_upmap mapping 2024-09-14T09:08:05.171 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:05.171+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:05.218 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:05.218 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:08:05.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:08:05.218 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:08:05.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.218 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38593 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [watch reconnect cookie 139718844451680 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:08:05.218 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:08:05.218 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 123.40386442499999, 2024-09-14T09:08:05.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0218296630000001, 2024-09-14T09:08:05.218 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.218 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.218 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:05.218 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.219 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.219 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.219 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38593 2024-09-14T09:08:05.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.219 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:08:05.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:08:05.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818420+0000", 2024-09-14T09:08:05.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8399999999999999e-06 2024-09-14T09:08:05.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818431+0000", 2024-09-14T09:08:05.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0859999999999999e-05 2024-09-14T09:08:05.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818466+0000", 2024-09-14T09:08:05.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4486999999999997e-05 2024-09-14T09:08:05.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818481+0000", 2024-09-14T09:08:05.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5362999999999998e-05 2024-09-14T09:08:05.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818659+0000", 2024-09-14T09:08:05.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017807500000000001 2024-09-14T09:08:05.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op must wait for map", 2024-09-14T09:08:05.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818681+0000", 2024-09-14T09:08:05.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2096999999999999e-05 2024-09-14T09:08:05.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825472+0000", 2024-09-14T09:08:05.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067905170000000003 2024-09-14T09:08:05.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:05.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825491+0000", 2024-09-14T09:08:05.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9372999999999999e-05 2024-09-14T09:08:05.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835402+0000", 2024-09-14T09:08:05.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0099111620000001 2024-09-14T09:08:05.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:05.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835548+0000", 2024-09-14T09:08:05.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014607300000000001 2024-09-14T09:08:05.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.840247+0000", 2024-09-14T09:08:05.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046988159999999998 2024-09-14T09:08:05.224 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.224 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.224 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.224 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38615 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:08:05.224 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:08:05.224 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 123.355499217, 2024-09-14T09:08:05.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.97014128899999996, 2024-09-14T09:08:05.224 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.224 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.224 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:05.224 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38615 2024-09-14T09:08:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:08:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:08:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866787+0000", 2024-09-14T09:08:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.476e-06 2024-09-14T09:08:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866806+0000", 2024-09-14T09:08:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8606000000000001e-05 2024-09-14T09:08:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866839+0000", 2024-09-14T09:08:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3510000000000003e-05 2024-09-14T09:08:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866867+0000", 2024-09-14T09:08:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7610000000000002e-05 2024-09-14T09:08:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867031+0000", 2024-09-14T09:08:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016414199999999999 2024-09-14T09:08:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867054+0000", 2024-09-14T09:08:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2693000000000001e-05 2024-09-14T09:08:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836462+0000", 2024-09-14T09:08:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.96940797700000003 2024-09-14T09:08:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836547+0000", 2024-09-14T09:08:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5294e-05 2024-09-14T09:08:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836924+0000", 2024-09-14T09:08:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037698099999999997 2024-09-14T09:08:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105723 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.127065172999998, 2024-09-14T09:08:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77512799700000001, 2024-09-14T09:08:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105723 2024-09-14T09:08:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1399999999999997e-07 2024-09-14T09:08:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095220+0000", 2024-09-14T09:08:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0510000000000001e-06 2024-09-14T09:08:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095228+0000", 2024-09-14T09:08:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4159999999999998e-06 2024-09-14T09:08:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095234+0000", 2024-09-14T09:08:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3010000000000003e-06 2024-09-14T09:08:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103902+0000", 2024-09-14T09:08:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086678299999999996 2024-09-14T09:08:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103905+0000", 2024-09-14T09:08:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5020000000000001e-06 2024-09-14T09:08:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870332+0000", 2024-09-14T09:08:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76642700900000005 2024-09-14T09:08:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870345+0000", 2024-09-14T09:08:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2274e-05 2024-09-14T09:08:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105724 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:08:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.127040659999999, 2024-09-14T09:08:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77515698099999997, 2024-09-14T09:08:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105724 2024-09-14T09:08:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:08:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:08:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095242+0000", 2024-09-14T09:08:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9400000000000001e-07 2024-09-14T09:08:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095243+0000", 2024-09-14T09:08:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7540000000000001e-06 2024-09-14T09:08:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095248+0000", 2024-09-14T09:08:05.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1320000000000004e-06 2024-09-14T09:08:05.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095251+0000", 2024-09-14T09:08:05.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2770000000000001e-06 2024-09-14T09:08:05.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103947+0000", 2024-09-14T09:08:05.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086960089999999993 2024-09-14T09:08:05.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:05.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103955+0000", 2024-09-14T09:08:05.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9519999999999994e-06 2024-09-14T09:08:05.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870386+0000", 2024-09-14T09:08:05.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76643141100000001 2024-09-14T09:08:05.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870398+0000", 2024-09-14T09:08:05.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2052000000000001e-05 2024-09-14T09:08:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105727 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.127000954000003, 2024-09-14T09:08:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517292900000001, 2024-09-14T09:08:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.240 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.240 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105727 2024-09-14T09:08:05.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.240 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:05.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:05.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3300000000000002e-07 2024-09-14T09:08:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095284+0000", 2024-09-14T09:08:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9790000000000002e-06 2024-09-14T09:08:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095292+0000", 2024-09-14T09:08:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3810000000000001e-06 2024-09-14T09:08:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095298+0000", 2024-09-14T09:08:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9710000000000001e-06 2024-09-14T09:08:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103987+0000", 2024-09-14T09:08:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086893820000000007 2024-09-14T09:08:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103991+0000", 2024-09-14T09:08:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5480000000000002e-06 2024-09-14T09:08:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870442+0000", 2024-09-14T09:08:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.766451145 2024-09-14T09:08:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870454+0000", 2024-09-14T09:08:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.199e-05 2024-09-14T09:08:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105729 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:08:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.126952596999999, 2024-09-14T09:08:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517686500000005, 2024-09-14T09:08:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105729 2024-09-14T09:08:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:08:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:08:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095330+0000", 2024-09-14T09:08:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-09-14T09:08:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095332+0000", 2024-09-14T09:08:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1560000000000002e-06 2024-09-14T09:08:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095338+0000", 2024-09-14T09:08:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.592e-06 2024-09-14T09:08:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095341+0000", 2024-09-14T09:08:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8139999999999998e-06 2024-09-14T09:08:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104031+0000", 2024-09-14T09:08:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086896330000000004 2024-09-14T09:08:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104039+0000", 2024-09-14T09:08:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0900000000000005e-06 2024-09-14T09:08:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870495+0000", 2024-09-14T09:08:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76645611199999997 2024-09-14T09:08:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870506+0000", 2024-09-14T09:08:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0986e-05 2024-09-14T09:08:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105731 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.126934329999997, 2024-09-14T09:08:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77520598600000001, 2024-09-14T09:08:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105731 2024-09-14T09:08:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8599999999999999e-07 2024-09-14T09:08:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095350+0000", 2024-09-14T09:08:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1110000000000002e-06 2024-09-14T09:08:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095357+0000", 2024-09-14T09:08:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9020000000000002e-06 2024-09-14T09:08:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095360+0000", 2024-09-14T09:08:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1939999999999998e-06 2024-09-14T09:08:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104073+0000", 2024-09-14T09:08:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087131210000000008 2024-09-14T09:08:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104077+0000", 2024-09-14T09:08:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.501e-06 2024-09-14T09:08:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870541+0000", 2024-09-14T09:08:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76646461700000001 2024-09-14T09:08:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870554+0000", 2024-09-14T09:08:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2153999999999999e-05 2024-09-14T09:08:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105733 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.126914298999999, 2024-09-14T09:08:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.775237277, 2024-09-14T09:08:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105733 2024-09-14T09:08:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8699999999999995e-07 2024-09-14T09:08:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095371+0000", 2024-09-14T09:08:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5450000000000002e-06 2024-09-14T09:08:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095377+0000", 2024-09-14T09:08:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.776e-06 2024-09-14T09:08:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095383+0000", 2024-09-14T09:08:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3129999999999998e-06 2024-09-14T09:08:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104124+0000", 2024-09-14T09:08:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087409840000000003 2024-09-14T09:08:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104132+0000", 2024-09-14T09:08:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3159999999999997e-06 2024-09-14T09:08:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870593+0000", 2024-09-14T09:08:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76646064800000002 2024-09-14T09:08:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870605+0000", 2024-09-14T09:08:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2208000000000001e-05 2024-09-14T09:08:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105734 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.126891217000001, 2024-09-14T09:08:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77526569599999995, 2024-09-14T09:08:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105734 2024-09-14T09:08:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7399999999999998e-07 2024-09-14T09:08:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095394+0000", 2024-09-14T09:08:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8030000000000001e-06 2024-09-14T09:08:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095401+0000", 2024-09-14T09:08:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.046e-06 2024-09-14T09:08:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095407+0000", 2024-09-14T09:08:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6060000000000002e-06 2024-09-14T09:08:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104167+0000", 2024-09-14T09:08:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087602530000000008 2024-09-14T09:08:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104171+0000", 2024-09-14T09:08:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7620000000000001e-06 2024-09-14T09:08:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870645+0000", 2024-09-14T09:08:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76647400600000004 2024-09-14T09:08:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870656+0000", 2024-09-14T09:08:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1746000000000001e-05 2024-09-14T09:08:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105738 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:08:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.12685149, 2024-09-14T09:08:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77527530700000002, 2024-09-14T09:08:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105738 2024-09-14T09:08:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:08:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:08:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095431+0000", 2024-09-14T09:08:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0999999999999999e-07 2024-09-14T09:08:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095434+0000", 2024-09-14T09:08:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.757e-06 2024-09-14T09:08:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095442+0000", 2024-09-14T09:08:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9929999999999995e-06 2024-09-14T09:08:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095448+0000", 2024-09-14T09:08:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1340000000000001e-06 2024-09-14T09:08:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104213+0000", 2024-09-14T09:08:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087650000000000002 2024-09-14T09:08:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104221+0000", 2024-09-14T09:08:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4589999999999998e-06 2024-09-14T09:08:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870694+0000", 2024-09-14T09:08:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76647303200000005 2024-09-14T09:08:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870706+0000", 2024-09-14T09:08:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1522000000000001e-05 2024-09-14T09:08:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105739 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.126826960999999, 2024-09-14T09:08:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77530221700000002, 2024-09-14T09:08:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105739 2024-09-14T09:08:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.15e-07 2024-09-14T09:08:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095457+0000", 2024-09-14T09:08:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6470000000000001e-06 2024-09-14T09:08:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095461+0000", 2024-09-14T09:08:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0910000000000003e-06 2024-09-14T09:08:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095464+0000", 2024-09-14T09:08:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2059999999999999e-06 2024-09-14T09:08:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104253+0000", 2024-09-14T09:08:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087884120000000007 2024-09-14T09:08:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104256+0000", 2024-09-14T09:08:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5439999999999999e-06 2024-09-14T09:08:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870745+0000", 2024-09-14T09:08:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76648908000000004 2024-09-14T09:08:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870757+0000", 2024-09-14T09:08:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1822e-05 2024-09-14T09:08:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105741 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:08:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.126757834999999, 2024-09-14T09:08:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77528469899999997, 2024-09-14T09:08:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105741 2024-09-14T09:08:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:08:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:08:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095525+0000", 2024-09-14T09:08:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3740000000000001e-06 2024-09-14T09:08:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095533+0000", 2024-09-14T09:08:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6380000000000004e-06 2024-09-14T09:08:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095549+0000", 2024-09-14T09:08:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5554999999999999e-05 2024-09-14T09:08:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095566+0000", 2024-09-14T09:08:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7146000000000002e-05 2024-09-14T09:08:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104297+0000", 2024-09-14T09:08:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087315750000000001 2024-09-14T09:08:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104305+0000", 2024-09-14T09:08:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1389999999999995e-06 2024-09-14T09:08:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870797+0000", 2024-09-14T09:08:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76649129599999999 2024-09-14T09:08:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870809+0000", 2024-09-14T09:08:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1976e-05 2024-09-14T09:08:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105742 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:08:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.126693050999997, 2024-09-14T09:08:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.775271873, 2024-09-14T09:08:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105742 2024-09-14T09:08:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:08:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:08:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095590+0000", 2024-09-14T09:08:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0400000000000005e-07 2024-09-14T09:08:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095594+0000", 2024-09-14T09:08:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5639999999999998e-06 2024-09-14T09:08:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095620+0000", 2024-09-14T09:08:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5494999999999999e-05 2024-09-14T09:08:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095639+0000", 2024-09-14T09:08:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9639999999999998e-05 2024-09-14T09:08:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104340+0000", 2024-09-14T09:08:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087004950000000008 2024-09-14T09:08:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104343+0000", 2024-09-14T09:08:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4020000000000002e-06 2024-09-14T09:08:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870850+0000", 2024-09-14T09:08:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76650648499999996 2024-09-14T09:08:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870861+0000", 2024-09-14T09:08:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0888e-05 2024-09-14T09:08:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105745 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:08:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.126594435999998, 2024-09-14T09:08:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77521998700000005, 2024-09-14T09:08:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105745 2024-09-14T09:08:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:08:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:08:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095688+0000", 2024-09-14T09:08:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2699999999999998e-07 2024-09-14T09:08:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095693+0000", 2024-09-14T09:08:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0479999999999998e-06 2024-09-14T09:08:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095707+0000", 2024-09-14T09:08:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.342e-05 2024-09-14T09:08:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095716+0000", 2024-09-14T09:08:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9800000000000004e-06 2024-09-14T09:08:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104388+0000", 2024-09-14T09:08:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086717349999999999 2024-09-14T09:08:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104396+0000", 2024-09-14T09:08:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1769999999999998e-06 2024-09-14T09:08:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870897+0000", 2024-09-14T09:08:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76650102600000003 2024-09-14T09:08:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870907+0000", 2024-09-14T09:08:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0674e-05 2024-09-14T09:08:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105747 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.126548216000003, 2024-09-14T09:08:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77522190300000005, 2024-09-14T09:08:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105747 2024-09-14T09:08:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8999999999999995e-07 2024-09-14T09:08:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095738+0000", 2024-09-14T09:08:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8430000000000003e-06 2024-09-14T09:08:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095750+0000", 2024-09-14T09:08:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1229999999999999e-05 2024-09-14T09:08:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095758+0000", 2024-09-14T09:08:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.337e-06 2024-09-14T09:08:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104431+0000", 2024-09-14T09:08:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086728780000000002 2024-09-14T09:08:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104434+0000", 2024-09-14T09:08:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5049999999999999e-06 2024-09-14T09:08:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870945+0000", 2024-09-14T09:08:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651031800000002 2024-09-14T09:08:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870956+0000", 2024-09-14T09:08:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1002000000000001e-05 2024-09-14T09:08:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105750 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.12647827, 2024-09-14T09:08:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77519961800000003, 2024-09-14T09:08:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105750 2024-09-14T09:08:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3600000000000002e-07 2024-09-14T09:08:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095809+0000", 2024-09-14T09:08:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4000000000000002e-06 2024-09-14T09:08:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095822+0000", 2024-09-14T09:08:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2704000000000001e-05 2024-09-14T09:08:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095831+0000", 2024-09-14T09:08:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9530000000000005e-06 2024-09-14T09:08:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104477+0000", 2024-09-14T09:08:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086459989999999997 2024-09-14T09:08:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104488+0000", 2024-09-14T09:08:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:08:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870992+0000", 2024-09-14T09:08:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.766504097 2024-09-14T09:08:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871003+0000", 2024-09-14T09:08:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1358999999999999e-05 2024-09-14T09:08:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105751 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.126420385000003, 2024-09-14T09:08:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77518966700000003, 2024-09-14T09:08:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105751 2024-09-14T09:08:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7300000000000005e-07 2024-09-14T09:08:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095867+0000", 2024-09-14T09:08:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2729999999999996e-06 2024-09-14T09:08:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095879+0000", 2024-09-14T09:08:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2917e-05 2024-09-14T09:08:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095888+0000", 2024-09-14T09:08:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2609999999999993e-06 2024-09-14T09:08:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104520+0000", 2024-09-14T09:08:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086322970000000006 2024-09-14T09:08:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104524+0000", 2024-09-14T09:08:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.54e-06 2024-09-14T09:08:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871040+0000", 2024-09-14T09:08:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651615399999995 2024-09-14T09:08:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871051+0000", 2024-09-14T09:08:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1452e-05 2024-09-14T09:08:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105752 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:08:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.126359620999999, 2024-09-14T09:08:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517684899999995, 2024-09-14T09:08:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105752 2024-09-14T09:08:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:08:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:08:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095923+0000", 2024-09-14T09:08:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3300000000000001e-07 2024-09-14T09:08:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095927+0000", 2024-09-14T09:08:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:08:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095940+0000", 2024-09-14T09:08:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2309e-05 2024-09-14T09:08:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095948+0000", 2024-09-14T09:08:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9199999999999993e-06 2024-09-14T09:08:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104566+0000", 2024-09-14T09:08:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086171449999999997 2024-09-14T09:08:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104574+0000", 2024-09-14T09:08:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0690000000000002e-06 2024-09-14T09:08:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871087+0000", 2024-09-14T09:08:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651370699999999 2024-09-14T09:08:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871099+0000", 2024-09-14T09:08:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1718e-05 2024-09-14T09:08:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105755 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:08:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.126301677999997, 2024-09-14T09:08:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77516787899999995, 2024-09-14T09:08:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105755 2024-09-14T09:08:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:08:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:08:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095981+0000", 2024-09-14T09:08:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2699999999999998e-07 2024-09-14T09:08:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095985+0000", 2024-09-14T09:08:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4349999999999999e-06 2024-09-14T09:08:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095999+0000", 2024-09-14T09:08:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3139000000000001e-05 2024-09-14T09:08:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096007+0000", 2024-09-14T09:08:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6610000000000006e-06 2024-09-14T09:08:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104591+0000", 2024-09-14T09:08:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0085841709999999998 2024-09-14T09:08:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104595+0000", 2024-09-14T09:08:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5590000000000001e-06 2024-09-14T09:08:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871137+0000", 2024-09-14T09:08:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76654202100000002 2024-09-14T09:08:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871148+0000", 2024-09-14T09:08:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1066e-05 2024-09-14T09:08:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105757 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:08:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.126243098000003, 2024-09-14T09:08:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77516401300000004, 2024-09-14T09:08:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105757 2024-09-14T09:08:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:08:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:08:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856228+0000", 2024-09-14T09:08:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76018875100000005 2024-09-14T09:08:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856240+0000", 2024-09-14T09:08:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-05 2024-09-14T09:08:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856268+0000", 2024-09-14T09:08:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7628999999999999e-05 2024-09-14T09:08:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856307+0000", 2024-09-14T09:08:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8967000000000001e-05 2024-09-14T09:08:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871192+0000", 2024-09-14T09:08:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.01488449 2024-09-14T09:08:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871203+0000", 2024-09-14T09:08:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1266e-05 2024-09-14T09:08:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.318 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:05.318 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:05.318 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:05.319 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:08:05.320 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:05.319+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:05.420 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:05.422+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:05.440 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:05.440 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:05.440 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:05.440 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:05.454 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:08:05.521 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:05.522+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:05.581 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:05.581 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:05.581 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:08:05.581 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:08:05.581 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:05.595 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:08:05.642 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:05.643+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:05.722 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:08:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:08:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:08:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:71052 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:08:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:08:05.722 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.983397424000003, 2024-09-14T09:08:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14854635599999999, 2024-09-14T09:08:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 71052 2024-09-14T09:08:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:08:05.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:08:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737570+0000", 2024-09-14T09:08:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4770000000000002e-06 2024-09-14T09:08:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737584+0000", 2024-09-14T09:08:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3457e-05 2024-09-14T09:08:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737608+0000", 2024-09-14T09:08:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3893999999999999e-05 2024-09-14T09:08:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737620+0000", 2024-09-14T09:08:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2157000000000001e-05 2024-09-14T09:08:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737738+0000", 2024-09-14T09:08:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118409 2024-09-14T09:08:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737892+0000", 2024-09-14T09:08:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154222 2024-09-14T09:08:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:08:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.749622+0000", 2024-09-14T09:08:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011729537 2024-09-14T09:08:05.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885934+0000", 2024-09-14T09:08:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13631222200000001 2024-09-14T09:08:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885996+0000", 2024-09-14T09:08:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1555000000000003e-05 2024-09-14T09:08:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886060+0000", 2024-09-14T09:08:05.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3980999999999996e-05 2024-09-14T09:08:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886114+0000", 2024-09-14T09:08:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4444999999999998e-05 2024-09-14T09:08:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:71052 3.1 e71/34 ondisk, result = 0)", 2024-09-14T09:08:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:08:05.728 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.940494045999998, 2024-09-14T09:08:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10565334799999999, 2024-09-14T09:08:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:08:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:08:05.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780473+0000", 2024-09-14T09:08:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8980000000000001e-06 2024-09-14T09:08:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780482+0000", 2024-09-14T09:08:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9080000000000001e-06 2024-09-14T09:08:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780511+0000", 2024-09-14T09:08:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9490999999999999e-05 2024-09-14T09:08:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780527+0000", 2024-09-14T09:08:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.508e-05 2024-09-14T09:08:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885978+0000", 2024-09-14T09:08:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.105451328 2024-09-14T09:08:05.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885989+0000", 2024-09-14T09:08:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0875e-05 2024-09-14T09:08:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886125+0000", 2024-09-14T09:08:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013576800000000001 2024-09-14T09:08:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:98007 3.12 3:4faf1f2a:::rbd_data.115e113dc8ca.000000000000004f:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:08:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:08:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.863521822000003, 2024-09-14T09:08:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.098233707000000003, 2024-09-14T09:08:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 98007 2024-09-14T09:08:05.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:08:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:08:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857446+0000", 2024-09-14T09:08:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7599999999999998e-06 2024-09-14T09:08:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857459+0000", 2024-09-14T09:08:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3123e-05 2024-09-14T09:08:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857491+0000", 2024-09-14T09:08:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1603000000000002e-05 2024-09-14T09:08:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857506+0000", 2024-09-14T09:08:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.537e-05 2024-09-14T09:08:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857694+0000", 2024-09-14T09:08:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018766599999999999 2024-09-14T09:08:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857955+0000", 2024-09-14T09:08:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000260979 2024-09-14T09:08:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:08:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.863125+0000", 2024-09-14T09:08:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.005170582 2024-09-14T09:08:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955268+0000", 2024-09-14T09:08:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092142659000000002 2024-09-14T09:08:05.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955385+0000", 2024-09-14T09:08:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116723 2024-09-14T09:08:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955553+0000", 2024-09-14T09:08:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016809299999999999 2024-09-14T09:08:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955677+0000", 2024-09-14T09:08:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012414899999999999 2024-09-14T09:08:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118153 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 445440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:08:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.882964446999999, 2024-09-14T09:08:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095358268999999996, 2024-09-14T09:08:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.739 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118153 2024-09-14T09:08:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:08:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:08:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:08:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0999999999999999e-07 2024-09-14T09:08:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838003+0000", 2024-09-14T09:08:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2000000000000001e-06 2024-09-14T09:08:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838009+0000", 2024-09-14T09:08:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.105e-06 2024-09-14T09:08:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838015+0000", 2024-09-14T09:08:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9030000000000001e-06 2024-09-14T09:08:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933049+0000", 2024-09-14T09:08:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095033513999999999 2024-09-14T09:08:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933113+0000", 2024-09-14T09:08:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3616999999999998e-05 2024-09-14T09:08:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933359+0000", 2024-09-14T09:08:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024652000000000002 2024-09-14T09:08:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.743 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118155 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 449536~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:08:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.882941922000001, 2024-09-14T09:08:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095668000000000003, 2024-09-14T09:08:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118155 2024-09-14T09:08:05.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:08:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:08:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838024+0000", 2024-09-14T09:08:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0400000000000002e-07 2024-09-14T09:08:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838026+0000", 2024-09-14T09:08:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.508e-06 2024-09-14T09:08:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838033+0000", 2024-09-14T09:08:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.392e-06 2024-09-14T09:08:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838038+0000", 2024-09-14T09:08:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1050000000000003e-06 2024-09-14T09:08:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933400+0000", 2024-09-14T09:08:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095362279999999994 2024-09-14T09:08:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933451+0000", 2024-09-14T09:08:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0927000000000002e-05 2024-09-14T09:08:05.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933691+0000", 2024-09-14T09:08:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024038400000000001 2024-09-14T09:08:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118158 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 453632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:08:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.882920900000002, 2024-09-14T09:08:05.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096024231000000002, 2024-09-14T09:08:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118158 2024-09-14T09:08:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:08:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:08:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838045+0000", 2024-09-14T09:08:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2199999999999999e-07 2024-09-14T09:08:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838047+0000", 2024-09-14T09:08:05.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1059999999999998e-06 2024-09-14T09:08:05.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838054+0000", 2024-09-14T09:08:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4089999999999997e-06 2024-09-14T09:08:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838059+0000", 2024-09-14T09:08:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8590000000000004e-06 2024-09-14T09:08:05.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933729+0000", 2024-09-14T09:08:05.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095670274999999999 2024-09-14T09:08:05.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933794+0000", 2024-09-14T09:08:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4846999999999998e-05 2024-09-14T09:08:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934069+0000", 2024-09-14T09:08:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027431299999999999 2024-09-14T09:08:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.754 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118160 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 457728~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:08:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.882898779999998, 2024-09-14T09:08:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096394812999999996, 2024-09-14T09:08:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.755 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118160 2024-09-14T09:08:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:08:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:08:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838067+0000", 2024-09-14T09:08:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9399999999999995e-07 2024-09-14T09:08:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838069+0000", 2024-09-14T09:08:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1579999999999999e-06 2024-09-14T09:08:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838077+0000", 2024-09-14T09:08:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6939999999999995e-06 2024-09-14T09:08:05.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838082+0000", 2024-09-14T09:08:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2290000000000002e-06 2024-09-14T09:08:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934106+0000", 2024-09-14T09:08:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096023874999999995 2024-09-14T09:08:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934158+0000", 2024-09-14T09:08:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2052000000000002e-05 2024-09-14T09:08:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934461+0000", 2024-09-14T09:08:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030331100000000001 2024-09-14T09:08:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118162 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 461824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:08:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.882876199999998, 2024-09-14T09:08:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096615294000000004, 2024-09-14T09:08:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118162 2024-09-14T09:08:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:08:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:08:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838090+0000", 2024-09-14T09:08:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9699999999999996e-07 2024-09-14T09:08:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838092+0000", 2024-09-14T09:08:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1119999999999999e-06 2024-09-14T09:08:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838098+0000", 2024-09-14T09:08:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4359999999999999e-06 2024-09-14T09:08:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838103+0000", 2024-09-14T09:08:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1200000000000001e-06 2024-09-14T09:08:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934504+0000", 2024-09-14T09:08:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096400885000000006 2024-09-14T09:08:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934596+0000", 2024-09-14T09:08:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1775999999999997e-05 2024-09-14T09:08:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934704+0000", 2024-09-14T09:08:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010846800000000001 2024-09-14T09:08:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118164 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 465920~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:08:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.882854940000001, 2024-09-14T09:08:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096951372999999993, 2024-09-14T09:08:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.764 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118164 2024-09-14T09:08:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:08:05.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:08:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838111+0000", 2024-09-14T09:08:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0100000000000005e-07 2024-09-14T09:08:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838113+0000", 2024-09-14T09:08:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0339999999999999e-06 2024-09-14T09:08:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838119+0000", 2024-09-14T09:08:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.409e-06 2024-09-14T09:08:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838124+0000", 2024-09-14T09:08:05.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9030000000000003e-06 2024-09-14T09:08:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934739+0000", 2024-09-14T09:08:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096615124999999996 2024-09-14T09:08:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934789+0000", 2024-09-14T09:08:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9554e-05 2024-09-14T09:08:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935062+0000", 2024-09-14T09:08:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000272847 2024-09-14T09:08:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118166 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 470016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:08:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.882834185999997, 2024-09-14T09:08:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097306410999999995, 2024-09-14T09:08:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.769 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118166 2024-09-14T09:08:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:08:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:08:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:08:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.03e-07 2024-09-14T09:08:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838133+0000", 2024-09-14T09:08:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9479999999999998e-06 2024-09-14T09:08:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838141+0000", 2024-09-14T09:08:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5739999999999998e-06 2024-09-14T09:08:05.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838146+0000", 2024-09-14T09:08:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3750000000000002e-06 2024-09-14T09:08:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935095+0000", 2024-09-14T09:08:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096948435999999999 2024-09-14T09:08:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935152+0000", 2024-09-14T09:08:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.694e-05 2024-09-14T09:08:05.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935437+0000", 2024-09-14T09:08:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028573500000000001 2024-09-14T09:08:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118168 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 474112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:08:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.882811965000002, 2024-09-14T09:08:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097658024999999996, 2024-09-14T09:08:05.774 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118168 2024-09-14T09:08:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:08:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:08:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838154+0000", 2024-09-14T09:08:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4200000000000001e-07 2024-09-14T09:08:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838160+0000", 2024-09-14T09:08:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8510000000000004e-06 2024-09-14T09:08:05.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838168+0000", 2024-09-14T09:08:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9899999999999997e-06 2024-09-14T09:08:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838172+0000", 2024-09-14T09:08:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.887e-06 2024-09-14T09:08:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935473+0000", 2024-09-14T09:08:05.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097300132999999997 2024-09-14T09:08:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935523+0000", 2024-09-14T09:08:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0862000000000002e-05 2024-09-14T09:08:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935811+0000", 2024-09-14T09:08:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028786000000000003 2024-09-14T09:08:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.778 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118170 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 478208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:08:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.882785153999997, 2024-09-14T09:08:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.098077529999999996, 2024-09-14T09:08:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.779 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118170 2024-09-14T09:08:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:08:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:08:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842901+0000", 2024-09-14T09:08:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004721416 2024-09-14T09:08:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842906+0000", 2024-09-14T09:08:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7199999999999997e-06 2024-09-14T09:08:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842917+0000", 2024-09-14T09:08:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0671e-05 2024-09-14T09:08:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842923+0000", 2024-09-14T09:08:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.088e-06 2024-09-14T09:08:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935857+0000", 2024-09-14T09:08:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092933624000000006 2024-09-14T09:08:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935958+0000", 2024-09-14T09:08:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100933 2024-09-14T09:08:05.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.936258+0000", 2024-09-14T09:08:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030007800000000001 2024-09-14T09:08:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118181 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 498688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:08:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.877978884999997, 2024-09-14T09:08:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095212643999999999, 2024-09-14T09:08:05.783 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118181 2024-09-14T09:08:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:08:05.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:08:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842987+0000", 2024-09-14T09:08:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.29e-07 2024-09-14T09:08:05.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842988+0000", 2024-09-14T09:08:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.229e-06 2024-09-14T09:08:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842993+0000", 2024-09-14T09:08:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.792e-06 2024-09-14T09:08:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842996+0000", 2024-09-14T09:08:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3349999999999998e-06 2024-09-14T09:08:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.937807+0000", 2024-09-14T09:08:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.094811585000000004 2024-09-14T09:08:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.937899+0000", 2024-09-14T09:08:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1719999999999996e-05 2024-09-14T09:08:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938199+0000", 2024-09-14T09:08:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000299754 2024-09-14T09:08:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118182 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 502784~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:08:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.877965707999998, 2024-09-14T09:08:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095557953000000001, 2024-09-14T09:08:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.788 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118182 2024-09-14T09:08:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:08:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:08:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:08:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2999999999999999e-07 2024-09-14T09:08:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843001+0000", 2024-09-14T09:08:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.652e-06 2024-09-14T09:08:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843005+0000", 2024-09-14T09:08:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2780000000000002e-06 2024-09-14T09:08:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843007+0000", 2024-09-14T09:08:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.553e-06 2024-09-14T09:08:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938240+0000", 2024-09-14T09:08:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095232864 2024-09-14T09:08:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938295+0000", 2024-09-14T09:08:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5095e-05 2024-09-14T09:08:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938557+0000", 2024-09-14T09:08:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026228100000000001 2024-09-14T09:08:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.792 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118184 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 506880~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:08:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.877954508000002, 2024-09-14T09:08:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095950246000000003, 2024-09-14T09:08:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118184 2024-09-14T09:08:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.793 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:08:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:08:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:08:05.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2399999999999999e-07 2024-09-14T09:08:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843012+0000", 2024-09-14T09:08:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.269e-06 2024-09-14T09:08:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843015+0000", 2024-09-14T09:08:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0469999999999998e-06 2024-09-14T09:08:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843018+0000", 2024-09-14T09:08:05.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.044e-06 2024-09-14T09:08:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938597+0000", 2024-09-14T09:08:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095578688999999994 2024-09-14T09:08:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938687+0000", 2024-09-14T09:08:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0284000000000005e-05 2024-09-14T09:08:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938961+0000", 2024-09-14T09:08:05.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027368899999999997 2024-09-14T09:08:05.797 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.797 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.797 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.797 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118186 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 510976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:05.797 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:08:05.797 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.877943547999998, 2024-09-14T09:08:05.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096286497999999998, 2024-09-14T09:08:05.798 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.798 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.798 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:05.798 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:05.798 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:05.798 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.798 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118186 2024-09-14T09:08:05.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.798 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:08:05.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:08:05.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:08:05.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.35e-07 2024-09-14T09:08:05.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843023+0000", 2024-09-14T09:08:05.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.195e-06 2024-09-14T09:08:05.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843026+0000", 2024-09-14T09:08:05.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.055e-06 2024-09-14T09:08:05.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843029+0000", 2024-09-14T09:08:05.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6539999999999999e-06 2024-09-14T09:08:05.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939000+0000", 2024-09-14T09:08:05.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095971476 2024-09-14T09:08:05.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939049+0000", 2024-09-14T09:08:05.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8986999999999997e-05 2024-09-14T09:08:05.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939308+0000", 2024-09-14T09:08:05.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000258896 2024-09-14T09:08:05.802 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.802 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.802 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.802 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118188 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 515072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:05.802 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:08:05.802 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.877933159000001, 2024-09-14T09:08:05.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096722717999999999, 2024-09-14T09:08:05.802 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.803 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.803 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:05.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:05.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:05.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.803 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118188 2024-09-14T09:08:05.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.803 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:08:05.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:08:05.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:08:05.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2999999999999999e-07 2024-09-14T09:08:05.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843034+0000", 2024-09-14T09:08:05.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T09:08:05.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843037+0000", 2024-09-14T09:08:05.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0989999999999999e-06 2024-09-14T09:08:05.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843039+0000", 2024-09-14T09:08:05.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.666e-06 2024-09-14T09:08:05.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939351+0000", 2024-09-14T09:08:05.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096311789999999994 2024-09-14T09:08:05.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939440+0000", 2024-09-14T09:08:05.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8843000000000003e-05 2024-09-14T09:08:05.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939755+0000", 2024-09-14T09:08:05.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031486399999999999 2024-09-14T09:08:05.807 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.807 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.807 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.807 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118191 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 519168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:05.807 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:08:05.807 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.877922466000001, 2024-09-14T09:08:05.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097080607999999999, 2024-09-14T09:08:05.807 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.807 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.807 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:05.807 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:05.807 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:05.808 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.808 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118191 2024-09-14T09:08:05.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.808 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:08:05.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:08:05.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:08:05.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2100000000000001e-07 2024-09-14T09:08:05.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843044+0000", 2024-09-14T09:08:05.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2249999999999999e-06 2024-09-14T09:08:05.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843047+0000", 2024-09-14T09:08:05.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0240000000000002e-06 2024-09-14T09:08:05.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843050+0000", 2024-09-14T09:08:05.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.508e-06 2024-09-14T09:08:05.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939791+0000", 2024-09-14T09:08:05.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096741399000000006 2024-09-14T09:08:05.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939849+0000", 2024-09-14T09:08:05.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7912999999999999e-05 2024-09-14T09:08:05.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940123+0000", 2024-09-14T09:08:05.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027431800000000001 2024-09-14T09:08:05.811 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.811 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.811 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.812 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118192 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 523264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:05.812 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:08:05.812 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.877911406000003, 2024-09-14T09:08:05.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097587224, 2024-09-14T09:08:05.812 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.812 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.812 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:05.812 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:05.813 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:05.813 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.813 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118192 2024-09-14T09:08:05.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.813 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:08:05.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:08:05.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844132+0000", 2024-09-14T09:08:05.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010777810000000001 2024-09-14T09:08:05.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844140+0000", 2024-09-14T09:08:05.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4470000000000006e-06 2024-09-14T09:08:05.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844155+0000", 2024-09-14T09:08:05.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4924e-05 2024-09-14T09:08:05.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844163+0000", 2024-09-14T09:08:05.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9980000000000003e-06 2024-09-14T09:08:05.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940199+0000", 2024-09-14T09:08:05.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096036047999999999 2024-09-14T09:08:05.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940282+0000", 2024-09-14T09:08:05.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3073999999999998e-05 2024-09-14T09:08:05.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940641+0000", 2024-09-14T09:08:05.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000358952 2024-09-14T09:08:05.816 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.816 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.816 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.817 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118197 3.17 3:ec5ec63a:::rbd_data.115e96ceebb2.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 3718221] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:05.817 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:08:05.817 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 33.873131991000001, 2024-09-14T09:08:05.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.141131069, 2024-09-14T09:08:05.817 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:05.817 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:05.817 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:05.817 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:05.818 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:05.818 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:05.818 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118197 2024-09-14T09:08:05.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.818 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:05.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:05.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:08:05.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:05.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:08:05.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:05.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:05.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847834+0000", 2024-09-14T09:08:05.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2500000000000004e-07 2024-09-14T09:08:05.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:05.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849227+0000", 2024-09-14T09:08:05.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0013932149999999999 2024-09-14T09:08:05.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:05.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849246+0000", 2024-09-14T09:08:05.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8958e-05 2024-09-14T09:08:05.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:05.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849254+0000", 2024-09-14T09:08:05.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2360000000000004e-06 2024-09-14T09:08:05.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:05.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940154+0000", 2024-09-14T09:08:05.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.090899468999999997 2024-09-14T09:08:05.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:05.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940312+0000", 2024-09-14T09:08:05.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000158326 2024-09-14T09:08:05.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:08:05.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940728+0000", 2024-09-14T09:08:05.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00041583100000000002 2024-09-14T09:08:05.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:05.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.962731+0000", 2024-09-14T09:08:05.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022002608999999999 2024-09-14T09:08:05.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:05.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988881+0000", 2024-09-14T09:08:05.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026150620999999999 2024-09-14T09:08:05.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:05.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988919+0000", 2024-09-14T09:08:05.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8108999999999997e-05 2024-09-14T09:08:05.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:05.824 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:05.824 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:05.824 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988964+0000", 2024-09-14T09:08:05.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4870000000000002e-05 2024-09-14T09:08:05.824 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.824 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.824 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.824 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:05.825 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:05.825 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:05.826 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:08:05.826 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:05.827+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:05.927 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:05.927+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:05.997 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:05.997 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:05.997 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:05.998 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:06.012 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:08:06.054 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:06.055+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:06.129 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:06.127+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:06.178 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:06.178 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:06.178 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:08:06.178 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:08:06.178 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:06.197 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:08:06.205 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T09:08:06.232 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:06.234+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:06.331 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:06.334+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:06.382 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:06.382 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:08:06.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:08:06.382 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:08:06.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.383 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73542 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15319040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:06.383 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:08:06.383 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.113224803999998, 2024-09-14T09:08:06.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0237091359999999, 2024-09-14T09:08:06.383 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.384 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.384 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:06.384 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.384 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.384 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.384 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73542 2024-09-14T09:08:06.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.384 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:08:06.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:08:06.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268762+0000", 2024-09-14T09:08:06.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:08:06.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268766+0000", 2024-09-14T09:08:06.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5880000000000002e-06 2024-09-14T09:08:06.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268776+0000", 2024-09-14T09:08:06.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0168e-05 2024-09-14T09:08:06.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268789+0000", 2024-09-14T09:08:06.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:08:06.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282053+0000", 2024-09-14T09:08:06.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013264496000000001 2024-09-14T09:08:06.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282057+0000", 2024-09-14T09:08:06.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6090000000000001e-06 2024-09-14T09:08:06.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292052+0000", 2024-09-14T09:08:06.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0099950070000001 2024-09-14T09:08:06.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292262+0000", 2024-09-14T09:08:06.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021009800000000001 2024-09-14T09:08:06.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292470+0000", 2024-09-14T09:08:06.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000208314 2024-09-14T09:08:06.393 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.393 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.393 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.394 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73543 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:06.394 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:08:06.394 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.113186013999993, 2024-09-14T09:08:06.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023985473, 2024-09-14T09:08:06.394 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.394 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.394 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:06.395 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.395 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.395 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.395 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73543 2024-09-14T09:08:06.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.395 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:08:06.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:08:06.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268801+0000", 2024-09-14T09:08:06.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:08:06.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268803+0000", 2024-09-14T09:08:06.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7439999999999999e-06 2024-09-14T09:08:06.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268812+0000", 2024-09-14T09:08:06.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7080000000000003e-06 2024-09-14T09:08:06.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268819+0000", 2024-09-14T09:08:06.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T09:08:06.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282068+0000", 2024-09-14T09:08:06.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013249540000000001 2024-09-14T09:08:06.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282072+0000", 2024-09-14T09:08:06.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0960000000000001e-06 2024-09-14T09:08:06.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292573+0000", 2024-09-14T09:08:06.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0105014560000001 2024-09-14T09:08:06.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292679+0000", 2024-09-14T09:08:06.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010545899999999999 2024-09-14T09:08:06.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292786+0000", 2024-09-14T09:08:06.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107052 2024-09-14T09:08:06.404 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.404 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.404 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.404 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73544 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:06.404 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:06.404 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.113158311999996, 2024-09-14T09:08:06.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0244830549999999, 2024-09-14T09:08:06.405 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.405 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.405 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:06.405 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.405 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.405 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.405 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73544 2024-09-14T09:08:06.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.405 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:06.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:06.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:06.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5599999999999995e-07 2024-09-14T09:08:06.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268831+0000", 2024-09-14T09:08:06.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.57e-06 2024-09-14T09:08:06.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268844+0000", 2024-09-14T09:08:06.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2811e-05 2024-09-14T09:08:06.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268851+0000", 2024-09-14T09:08:06.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9140000000000002e-06 2024-09-14T09:08:06.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282083+0000", 2024-09-14T09:08:06.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013232111 2024-09-14T09:08:06.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282086+0000", 2024-09-14T09:08:06.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2909999999999999e-06 2024-09-14T09:08:06.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292884+0000", 2024-09-14T09:08:06.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0107975069999999 2024-09-14T09:08:06.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293100+0000", 2024-09-14T09:08:06.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021639399999999999 2024-09-14T09:08:06.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293311+0000", 2024-09-14T09:08:06.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021090099999999999 2024-09-14T09:08:06.409 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.409 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.409 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.410 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73545 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15564800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:06.410 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:08:06.410 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.113107127000006, 2024-09-14T09:08:06.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0249746740000001, 2024-09-14T09:08:06.410 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.410 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.410 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:06.410 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.410 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.410 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.410 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73545 2024-09-14T09:08:06.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.410 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:08:06.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:08:06.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268880+0000", 2024-09-14T09:08:06.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5600000000000005e-07 2024-09-14T09:08:06.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268889+0000", 2024-09-14T09:08:06.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8559999999999996e-06 2024-09-14T09:08:06.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268913+0000", 2024-09-14T09:08:06.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3958e-05 2024-09-14T09:08:06.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268921+0000", 2024-09-14T09:08:06.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7479999999999999e-06 2024-09-14T09:08:06.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282098+0000", 2024-09-14T09:08:06.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013176297999999999 2024-09-14T09:08:06.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.413 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.413 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282101+0000", 2024-09-14T09:08:06.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2329999999999998e-06 2024-09-14T09:08:06.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293435+0000", 2024-09-14T09:08:06.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0113339459999999 2024-09-14T09:08:06.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293648+0000", 2024-09-14T09:08:06.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021307100000000001 2024-09-14T09:08:06.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293854+0000", 2024-09-14T09:08:06.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020580800000000001 2024-09-14T09:08:06.416 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.416 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.416 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.417 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73546 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15646720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:06.417 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:08:06.417 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.113053256000001, 2024-09-14T09:08:06.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0254846360000001, 2024-09-14T09:08:06.417 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.418 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.418 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:06.418 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.418 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.418 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.418 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73546 2024-09-14T09:08:06.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.418 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:08:06.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:08:06.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268934+0000", 2024-09-14T09:08:06.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7999999999999995e-07 2024-09-14T09:08:06.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268937+0000", 2024-09-14T09:08:06.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9840000000000001e-06 2024-09-14T09:08:06.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268947+0000", 2024-09-14T09:08:06.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0522e-05 2024-09-14T09:08:06.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268955+0000", 2024-09-14T09:08:06.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8059999999999995e-06 2024-09-14T09:08:06.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282112+0000", 2024-09-14T09:08:06.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013157189 2024-09-14T09:08:06.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282115+0000", 2024-09-14T09:08:06.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2949999999999998e-06 2024-09-14T09:08:06.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293968+0000", 2024-09-14T09:08:06.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0118530990000001 2024-09-14T09:08:06.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294197+0000", 2024-09-14T09:08:06.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228507 2024-09-14T09:08:06.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294417+0000", 2024-09-14T09:08:06.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000220554 2024-09-14T09:08:06.427 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.427 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.427 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.427 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73547 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:06.428 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:08:06.428 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.113020809999995, 2024-09-14T09:08:06.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.025970525, 2024-09-14T09:08:06.428 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.428 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.428 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:06.429 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.429 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.429 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.429 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73547 2024-09-14T09:08:06.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.430 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:08:06.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:08:06.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268966+0000", 2024-09-14T09:08:06.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3799999999999997e-07 2024-09-14T09:08:06.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268969+0000", 2024-09-14T09:08:06.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8739999999999999e-06 2024-09-14T09:08:06.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268979+0000", 2024-09-14T09:08:06.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9939999999999995e-06 2024-09-14T09:08:06.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268986+0000", 2024-09-14T09:08:06.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3459999999999996e-06 2024-09-14T09:08:06.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282127+0000", 2024-09-14T09:08:06.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01314039 2024-09-14T09:08:06.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282130+0000", 2024-09-14T09:08:06.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2219999999999998e-06 2024-09-14T09:08:06.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294540+0000", 2024-09-14T09:08:06.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124105880000001 2024-09-14T09:08:06.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294759+0000", 2024-09-14T09:08:06.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000218798 2024-09-14T09:08:06.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294936+0000", 2024-09-14T09:08:06.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000176675 2024-09-14T09:08:06.437 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.438 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.438 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.438 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73548 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15810560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:06.438 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:08:06.439 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.112989698000007, 2024-09-14T09:08:06.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0265045129999999, 2024-09-14T09:08:06.439 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.439 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.439 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:06.439 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.440 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.440 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.440 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73548 2024-09-14T09:08:06.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.440 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:08:06.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:08:06.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268997+0000", 2024-09-14T09:08:06.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7599999999999996e-07 2024-09-14T09:08:06.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269000+0000", 2024-09-14T09:08:06.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0580000000000002e-06 2024-09-14T09:08:06.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269010+0000", 2024-09-14T09:08:06.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124e-05 2024-09-14T09:08:06.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269018+0000", 2024-09-14T09:08:06.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5399999999999998e-06 2024-09-14T09:08:06.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282141+0000", 2024-09-14T09:08:06.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013123012 2024-09-14T09:08:06.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282144+0000", 2024-09-14T09:08:06.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.196e-06 2024-09-14T09:08:06.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295042+0000", 2024-09-14T09:08:06.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0128975689999999 2024-09-14T09:08:06.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295272+0000", 2024-09-14T09:08:06.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023048099999999999 2024-09-14T09:08:06.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295501+0000", 2024-09-14T09:08:06.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228657 2024-09-14T09:08:06.446 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.446 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.446 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.446 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73549 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15892480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:06.447 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:08:06.447 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.112943818999995, 2024-09-14T09:08:06.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0268354099999999, 2024-09-14T09:08:06.447 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.447 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.447 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:06.447 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.447 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.447 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.447 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73549 2024-09-14T09:08:06.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.447 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:08:06.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:08:06.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.448 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269043+0000", 2024-09-14T09:08:06.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6199999999999997e-07 2024-09-14T09:08:06.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.448 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269046+0000", 2024-09-14T09:08:06.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2480000000000001e-06 2024-09-14T09:08:06.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269057+0000", 2024-09-14T09:08:06.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0436e-05 2024-09-14T09:08:06.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269064+0000", 2024-09-14T09:08:06.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5179999999999998e-06 2024-09-14T09:08:06.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282155+0000", 2024-09-14T09:08:06.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013091190000000001 2024-09-14T09:08:06.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282159+0000", 2024-09-14T09:08:06.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.366e-06 2024-09-14T09:08:06.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295599+0000", 2024-09-14T09:08:06.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0134405559999999 2024-09-14T09:08:06.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295765+0000", 2024-09-14T09:08:06.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016575499999999999 2024-09-14T09:08:06.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295878+0000", 2024-09-14T09:08:06.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112679 2024-09-14T09:08:06.451 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.451 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.451 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.451 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73550 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15974400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:06.451 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:08:06.452 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.112911218999997, 2024-09-14T09:08:06.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027357657, 2024-09-14T09:08:06.452 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.452 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.452 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:06.452 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.452 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.452 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.452 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73550 2024-09-14T09:08:06.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.452 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:08:06.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:08:06.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269076+0000", 2024-09-14T09:08:06.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:08:06.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269079+0000", 2024-09-14T09:08:06.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:08:06.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269098+0000", 2024-09-14T09:08:06.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9698e-05 2024-09-14T09:08:06.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.454 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.454 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269106+0000", 2024-09-14T09:08:06.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6179999999999997e-06 2024-09-14T09:08:06.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.454 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282170+0000", 2024-09-14T09:08:06.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013063874 2024-09-14T09:08:06.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282173+0000", 2024-09-14T09:08:06.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1999999999999999e-06 2024-09-14T09:08:06.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295996+0000", 2024-09-14T09:08:06.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.013823151 2024-09-14T09:08:06.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296214+0000", 2024-09-14T09:08:06.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021817700000000001 2024-09-14T09:08:06.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296433+0000", 2024-09-14T09:08:06.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021829099999999999 2024-09-14T09:08:06.456 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.456 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.456 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.457 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73551 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:06.457 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:08:06.457 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.112869802000006, 2024-09-14T09:08:06.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027730276, 2024-09-14T09:08:06.457 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.457 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.457 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:06.457 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.457 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.457 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.457 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73551 2024-09-14T09:08:06.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.457 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:08:06.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:08:06.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269117+0000", 2024-09-14T09:08:06.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:08:06.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269120+0000", 2024-09-14T09:08:06.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.056e-06 2024-09-14T09:08:06.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269130+0000", 2024-09-14T09:08:06.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0084e-05 2024-09-14T09:08:06.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269137+0000", 2024-09-14T09:08:06.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1359999999999999e-06 2024-09-14T09:08:06.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282184+0000", 2024-09-14T09:08:06.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013046537 2024-09-14T09:08:06.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282188+0000", 2024-09-14T09:08:06.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3220000000000003e-06 2024-09-14T09:08:06.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.460 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.460 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.460 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296529+0000", 2024-09-14T09:08:06.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0143406109999999 2024-09-14T09:08:06.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296695+0000", 2024-09-14T09:08:06.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016604799999999999 2024-09-14T09:08:06.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296847+0000", 2024-09-14T09:08:06.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000151822 2024-09-14T09:08:06.461 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.461 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.461 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.462 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73552 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:06.462 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:06.462 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.112839222000005, 2024-09-14T09:08:06.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0281767500000001, 2024-09-14T09:08:06.462 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.462 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.462 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:06.462 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.462 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.462 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.462 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73552 2024-09-14T09:08:06.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.462 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:06.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:06.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:06.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:08:06.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269150+0000", 2024-09-14T09:08:06.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9119999999999998e-06 2024-09-14T09:08:06.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269160+0000", 2024-09-14T09:08:06.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6819999999999998e-06 2024-09-14T09:08:06.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269168+0000", 2024-09-14T09:08:06.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6079999999999998e-06 2024-09-14T09:08:06.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282205+0000", 2024-09-14T09:08:06.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013036982000000001 2024-09-14T09:08:06.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282208+0000", 2024-09-14T09:08:06.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4620000000000001e-06 2024-09-14T09:08:06.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.465 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.465 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296968+0000", 2024-09-14T09:08:06.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0147601209999999 2024-09-14T09:08:06.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297115+0000", 2024-09-14T09:08:06.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014673900000000001 2024-09-14T09:08:06.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297324+0000", 2024-09-14T09:08:06.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020866400000000001 2024-09-14T09:08:06.466 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.466 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.466 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.467 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73553 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:06.467 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:08:06.467 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.112809158000005, 2024-09-14T09:08:06.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0286556739999999, 2024-09-14T09:08:06.467 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.467 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.467 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:06.467 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.467 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.467 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.467 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73553 2024-09-14T09:08:06.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.467 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:08:06.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:08:06.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269178+0000", 2024-09-14T09:08:06.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:08:06.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269180+0000", 2024-09-14T09:08:06.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T09:08:06.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269189+0000", 2024-09-14T09:08:06.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2e-06 2024-09-14T09:08:06.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269197+0000", 2024-09-14T09:08:06.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2980000000000003e-06 2024-09-14T09:08:06.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282219+0000", 2024-09-14T09:08:06.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013022687 2024-09-14T09:08:06.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.470 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282223+0000", 2024-09-14T09:08:06.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.241e-06 2024-09-14T09:08:06.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.470 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297470+0000", 2024-09-14T09:08:06.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.015247778 2024-09-14T09:08:06.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297704+0000", 2024-09-14T09:08:06.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000233367 2024-09-14T09:08:06.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.471 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.471 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297833+0000", 2024-09-14T09:08:06.471 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012888900000000001 2024-09-14T09:08:06.471 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.471 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.472 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.472 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73554 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:06.472 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:08:06.472 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.112779841000005, 2024-09-14T09:08:06.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.029005113, 2024-09-14T09:08:06.472 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.472 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.472 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:06.472 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.472 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.472 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.472 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73554 2024-09-14T09:08:06.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.473 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:08:06.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:08:06.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269207+0000", 2024-09-14T09:08:06.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.68e-07 2024-09-14T09:08:06.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269210+0000", 2024-09-14T09:08:06.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8100000000000002e-06 2024-09-14T09:08:06.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269219+0000", 2024-09-14T09:08:06.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:08:06.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269227+0000", 2024-09-14T09:08:06.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3839999999999999e-06 2024-09-14T09:08:06.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282234+0000", 2024-09-14T09:08:06.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013007187 2024-09-14T09:08:06.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282237+0000", 2024-09-14T09:08:06.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1200000000000002e-06 2024-09-14T09:08:06.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297942+0000", 2024-09-14T09:08:06.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0157048040000001 2024-09-14T09:08:06.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298087+0000", 2024-09-14T09:08:06.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014499099999999999 2024-09-14T09:08:06.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.476 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.476 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298211+0000", 2024-09-14T09:08:06.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012461899999999999 2024-09-14T09:08:06.477 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.477 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.477 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.477 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73555 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:06.477 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:08:06.477 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.112749992999994, 2024-09-14T09:08:06.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.02941558, 2024-09-14T09:08:06.477 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.477 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.477 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:06.477 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.477 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.478 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.478 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73555 2024-09-14T09:08:06.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.478 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:08:06.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:08:06.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269237+0000", 2024-09-14T09:08:06.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3399999999999999e-07 2024-09-14T09:08:06.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269240+0000", 2024-09-14T09:08:06.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8480000000000001e-06 2024-09-14T09:08:06.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269249+0000", 2024-09-14T09:08:06.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1880000000000008e-06 2024-09-14T09:08:06.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269256+0000", 2024-09-14T09:08:06.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:08:06.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282248+0000", 2024-09-14T09:08:06.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012992327 2024-09-14T09:08:06.480 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.480 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.480 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.480 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282251+0000", 2024-09-14T09:08:06.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2160000000000002e-06 2024-09-14T09:08:06.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298299+0000", 2024-09-14T09:08:06.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0160479330000001 2024-09-14T09:08:06.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298446+0000", 2024-09-14T09:08:06.481 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000146525 2024-09-14T09:08:06.481 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.481 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.481 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.481 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298652+0000", 2024-09-14T09:08:06.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020584100000000001 2024-09-14T09:08:06.482 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.482 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.482 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.482 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.482 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.482 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73556 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:06.482 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:06.482 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.112720246999999, 2024-09-14T09:08:06.482 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0299854239999999, 2024-09-14T09:08:06.482 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.482 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.482 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:06.482 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.483 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.483 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.483 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73556 2024-09-14T09:08:06.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.483 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:06.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.483 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.483 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.483 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:06.483 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.483 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:06.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:08:06.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269270+0000", 2024-09-14T09:08:06.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0920000000000002e-06 2024-09-14T09:08:06.484 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.484 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.484 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.484 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269280+0000", 2024-09-14T09:08:06.484 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0013999999999999e-05 2024-09-14T09:08:06.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269288+0000", 2024-09-14T09:08:06.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:08:06.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282263+0000", 2024-09-14T09:08:06.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012974703000000001 2024-09-14T09:08:06.485 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.485 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.485 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282266+0000", 2024-09-14T09:08:06.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1520000000000001e-06 2024-09-14T09:08:06.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298729+0000", 2024-09-14T09:08:06.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.016463506 2024-09-14T09:08:06.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298884+0000", 2024-09-14T09:08:06.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015450800000000001 2024-09-14T09:08:06.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299251+0000", 2024-09-14T09:08:06.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036759899999999997 2024-09-14T09:08:06.487 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.487 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.487 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.487 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73557 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16547840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:06.487 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:08:06.487 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.112673596999997, 2024-09-14T09:08:06.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0307969480000001, 2024-09-14T09:08:06.487 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.487 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.487 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:06.487 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.487 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.488 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.488 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73557 2024-09-14T09:08:06.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.488 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:08:06.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:08:06.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269313+0000", 2024-09-14T09:08:06.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.92e-07 2024-09-14T09:08:06.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269336+0000", 2024-09-14T09:08:06.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3118000000000001e-05 2024-09-14T09:08:06.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269346+0000", 2024-09-14T09:08:06.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4069999999999999e-06 2024-09-14T09:08:06.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269352+0000", 2024-09-14T09:08:06.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5810000000000002e-06 2024-09-14T09:08:06.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282277+0000", 2024-09-14T09:08:06.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012924658 2024-09-14T09:08:06.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282280+0000", 2024-09-14T09:08:06.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2880000000000001e-06 2024-09-14T09:08:06.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299343+0000", 2024-09-14T09:08:06.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0170630190000001 2024-09-14T09:08:06.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299554+0000", 2024-09-14T09:08:06.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021085099999999999 2024-09-14T09:08:06.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300109+0000", 2024-09-14T09:08:06.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00055533399999999999 2024-09-14T09:08:06.492 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.492 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.492 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.492 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73558 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:06.492 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:08:06.492 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.112621974000007, 2024-09-14T09:08:06.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0314849340000001, 2024-09-14T09:08:06.492 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.492 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.492 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:06.493 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.493 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.493 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.493 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73558 2024-09-14T09:08:06.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.493 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:08:06.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:08:06.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269365+0000", 2024-09-14T09:08:06.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7400000000000003e-07 2024-09-14T09:08:06.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269367+0000", 2024-09-14T09:08:06.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4930000000000001e-06 2024-09-14T09:08:06.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269374+0000", 2024-09-14T09:08:06.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9519999999999997e-06 2024-09-14T09:08:06.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269379+0000", 2024-09-14T09:08:06.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9710000000000003e-06 2024-09-14T09:08:06.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282292+0000", 2024-09-14T09:08:06.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012912704000000001 2024-09-14T09:08:06.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282295+0000", 2024-09-14T09:08:06.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.179e-06 2024-09-14T09:08:06.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300195+0000", 2024-09-14T09:08:06.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.017899975 2024-09-14T09:08:06.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300345+0000", 2024-09-14T09:08:06.496 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014957499999999999 2024-09-14T09:08:06.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300849+0000", 2024-09-14T09:08:06.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050451100000000002 2024-09-14T09:08:06.497 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.497 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.497 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.497 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73559 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:06.497 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:06.497 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 92.112599564999996, 2024-09-14T09:08:06.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.031916343, 2024-09-14T09:08:06.498 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.498 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.498 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:06.498 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.498 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.498 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.498 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73559 2024-09-14T09:08:06.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.498 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:06.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:06.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:06.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3500000000000002e-07 2024-09-14T09:08:06.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269390+0000", 2024-09-14T09:08:06.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5890000000000001e-06 2024-09-14T09:08:06.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269396+0000", 2024-09-14T09:08:06.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3300000000000004e-06 2024-09-14T09:08:06.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269401+0000", 2024-09-14T09:08:06.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0340000000000004e-06 2024-09-14T09:08:06.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282311+0000", 2024-09-14T09:08:06.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012910204 2024-09-14T09:08:06.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282317+0000", 2024-09-14T09:08:06.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2639999999999997e-06 2024-09-14T09:08:06.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300933+0000", 2024-09-14T09:08:06.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0186160710000001 2024-09-14T09:08:06.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.501 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.501 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301158+0000", 2024-09-14T09:08:06.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225021 2024-09-14T09:08:06.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301303+0000", 2024-09-14T09:08:06.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014439499999999999 2024-09-14T09:08:06.502 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.502 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.502 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.502 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106964 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:06.502 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:08:06.502 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.266671731000002, 2024-09-14T09:08:06.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8802924409999999, 2024-09-14T09:08:06.503 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.503 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.503 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:06.503 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.503 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.503 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.503 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106964 2024-09-14T09:08:06.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.503 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:08:06.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:08:06.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115317+0000", 2024-09-14T09:08:06.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6429999999999999e-06 2024-09-14T09:08:06.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115326+0000", 2024-09-14T09:08:06.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2140000000000002e-06 2024-09-14T09:08:06.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115346+0000", 2024-09-14T09:08:06.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9419000000000002e-05 2024-09-14T09:08:06.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115357+0000", 2024-09-14T09:08:06.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1014e-05 2024-09-14T09:08:06.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115431+0000", 2024-09-14T09:08:06.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4040000000000003e-05 2024-09-14T09:08:06.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115438+0000", 2024-09-14T09:08:06.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9920000000000001e-06 2024-09-14T09:08:06.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844944+0000", 2024-09-14T09:08:06.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72950628399999995 2024-09-14T09:08:06.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844961+0000", 2024-09-14T09:08:06.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7156e-05 2024-09-14T09:08:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988004+0000", 2024-09-14T09:08:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.14304297 2024-09-14T09:08:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988511+0000", 2024-09-14T09:08:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050694100000000001 2024-09-14T09:08:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:08:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.989802+0000", 2024-09-14T09:08:06.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001291195 2024-09-14T09:08:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994866+0000", 2024-09-14T09:08:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050632760000000002 2024-09-14T09:08:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994988+0000", 2024-09-14T09:08:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012252899999999999 2024-09-14T09:08:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:06.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995198+0000", 2024-09-14T09:08:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021010899999999999 2024-09-14T09:08:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995607+0000", 2024-09-14T09:08:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040865900000000002 2024-09-14T09:08:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106965 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:06.509 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.265469353999997, 2024-09-14T09:08:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.879730162, 2024-09-14T09:08:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106965 2024-09-14T09:08:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:08:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116522+0000", 2024-09-14T09:08:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3989999999999997e-06 2024-09-14T09:08:06.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116533+0000", 2024-09-14T09:08:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0669000000000001e-05 2024-09-14T09:08:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116541+0000", 2024-09-14T09:08:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2730000000000002e-06 2024-09-14T09:08:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116604+0000", 2024-09-14T09:08:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2706999999999994e-05 2024-09-14T09:08:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116610+0000", 2024-09-14T09:08:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8630000000000002e-06 2024-09-14T09:08:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845060+0000", 2024-09-14T09:08:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72845003699999999 2024-09-14T09:08:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845079+0000", 2024-09-14T09:08:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8287999999999998e-05 2024-09-14T09:08:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.991866+0000", 2024-09-14T09:08:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1467871519999999 2024-09-14T09:08:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.992274+0000", 2024-09-14T09:08:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040813500000000001 2024-09-14T09:08:06.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:08:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.993538+0000", 2024-09-14T09:08:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001263655 2024-09-14T09:08:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995770+0000", 2024-09-14T09:08:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002231949 2024-09-14T09:08:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995810+0000", 2024-09-14T09:08:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0852999999999997e-05 2024-09-14T09:08:06.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995979+0000", 2024-09-14T09:08:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016866 2024-09-14T09:08:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996247+0000", 2024-09-14T09:08:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000267769 2024-09-14T09:08:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:06.516 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:06.516 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:06.518 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:08:06.518 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:06.519+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:06.619 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:06.619+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:06.691 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:06.691 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:06.691 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:06.691 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:06.705 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:08:06.720 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:06.722+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:06.821 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:06.819+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:06.872 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:06.872 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:06.872 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:08:06.872 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:08:06.872 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:06.886 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:08:06.930 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:06.931+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:07.023 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:07.023+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:07.058 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:07.058 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:08:07.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:08:07.058 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:08:07.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.059 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106705 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 518144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.059 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:07.059 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.209832849000001, 2024-09-14T09:08:07.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3001326799999999, 2024-09-14T09:08:07.059 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.059 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.059 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.059 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.059 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106705 2024-09-14T09:08:07.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.059 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:07.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:07.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:07.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4699999999999998e-07 2024-09-14T09:08:07.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847500+0000", 2024-09-14T09:08:07.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0860000000000001e-06 2024-09-14T09:08:07.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847503+0000", 2024-09-14T09:08:07.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1269999999999999e-06 2024-09-14T09:08:07.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847506+0000", 2024-09-14T09:08:07.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5780000000000001e-06 2024-09-14T09:08:07.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858911+0000", 2024-09-14T09:08:07.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011404925 2024-09-14T09:08:07.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:07.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858919+0000", 2024-09-14T09:08:07.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8620000000000003e-06 2024-09-14T09:08:07.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147254+0000", 2024-09-14T09:08:07.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2883353719999999 2024-09-14T09:08:07.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147408+0000", 2024-09-14T09:08:07.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015389200000000001 2024-09-14T09:08:07.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147631+0000", 2024-09-14T09:08:07.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022359100000000001 2024-09-14T09:08:07.063 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.063 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.064 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.064 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106707 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 522240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.064 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:07.064 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.209822379000002, 2024-09-14T09:08:07.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.300471801, 2024-09-14T09:08:07.064 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.064 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.064 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.064 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.064 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.064 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.065 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106707 2024-09-14T09:08:07.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.065 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:07.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:07.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:07.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.23e-07 2024-09-14T09:08:07.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847514+0000", 2024-09-14T09:08:07.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9150000000000004e-06 2024-09-14T09:08:07.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847520+0000", 2024-09-14T09:08:07.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.203e-06 2024-09-14T09:08:07.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847532+0000", 2024-09-14T09:08:07.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2441e-05 2024-09-14T09:08:07.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858953+0000", 2024-09-14T09:08:07.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01142108 2024-09-14T09:08:07.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:07.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858957+0000", 2024-09-14T09:08:07.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2830000000000003e-06 2024-09-14T09:08:07.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147667+0000", 2024-09-14T09:08:07.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2887094619999999 2024-09-14T09:08:07.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147784+0000", 2024-09-14T09:08:07.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117465 2024-09-14T09:08:07.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147981+0000", 2024-09-14T09:08:07.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019672900000000001 2024-09-14T09:08:07.069 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.069 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.069 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.069 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106709 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 3072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.069 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:08:07.069 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.209794088000002, 2024-09-14T09:08:07.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.301335192, 2024-09-14T09:08:07.069 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.069 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.069 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.069 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.069 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.069 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.070 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106709 2024-09-14T09:08:07.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.070 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:08:07.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:08:07.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847538+0000", 2024-09-14T09:08:07.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2800000000000003e-07 2024-09-14T09:08:07.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847539+0000", 2024-09-14T09:08:07.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2720000000000001e-06 2024-09-14T09:08:07.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847544+0000", 2024-09-14T09:08:07.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7219999999999999e-06 2024-09-14T09:08:07.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847547+0000", 2024-09-14T09:08:07.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9960000000000002e-06 2024-09-14T09:08:07.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858991+0000", 2024-09-14T09:08:07.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011444218000000001 2024-09-14T09:08:07.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:07.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858999+0000", 2024-09-14T09:08:07.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9419999999999995e-06 2024-09-14T09:08:07.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148053+0000", 2024-09-14T09:08:07.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2890544099999999 2024-09-14T09:08:07.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148203+0000", 2024-09-14T09:08:07.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149419 2024-09-14T09:08:07.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148873+0000", 2024-09-14T09:08:07.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00066988499999999995 2024-09-14T09:08:07.074 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.074 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.074 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.074 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106711 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 7168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.074 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:07.074 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.20978083, 2024-09-14T09:08:07.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3022241219999999, 2024-09-14T09:08:07.074 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.074 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.074 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.075 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.075 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.075 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.075 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106711 2024-09-14T09:08:07.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.075 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:07.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:07.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:07.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8000000000000002e-07 2024-09-14T09:08:07.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847552+0000", 2024-09-14T09:08:07.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1340000000000001e-06 2024-09-14T09:08:07.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847557+0000", 2024-09-14T09:08:07.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6179999999999997e-06 2024-09-14T09:08:07.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847560+0000", 2024-09-14T09:08:07.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5899999999999999e-06 2024-09-14T09:08:07.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859050+0000", 2024-09-14T09:08:07.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01148914 2024-09-14T09:08:07.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:07.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859056+0000", 2024-09-14T09:08:07.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6630000000000004e-06 2024-09-14T09:08:07.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148948+0000", 2024-09-14T09:08:07.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2898914370000001 2024-09-14T09:08:07.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149079+0000", 2024-09-14T09:08:07.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013145400000000001 2024-09-14T09:08:07.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149775+0000", 2024-09-14T09:08:07.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00069580599999999999 2024-09-14T09:08:07.079 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.079 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.079 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.079 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106713 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 11264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.079 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:07.079 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.209767364999998, 2024-09-14T09:08:07.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3035188959999999, 2024-09-14T09:08:07.079 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.080 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.080 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.080 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.080 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.080 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.080 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106713 2024-09-14T09:08:07.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.080 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:07.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:07.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:07.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6e-07 2024-09-14T09:08:07.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847566+0000", 2024-09-14T09:08:07.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1039999999999999e-06 2024-09-14T09:08:07.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847569+0000", 2024-09-14T09:08:07.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.191e-06 2024-09-14T09:08:07.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847571+0000", 2024-09-14T09:08:07.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6390000000000001e-06 2024-09-14T09:08:07.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859099+0000", 2024-09-14T09:08:07.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011527379000000001 2024-09-14T09:08:07.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:07.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859106+0000", 2024-09-14T09:08:07.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3679999999999996e-06 2024-09-14T09:08:07.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149878+0000", 2024-09-14T09:08:07.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.29077208 2024-09-14T09:08:07.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.150175+0000", 2024-09-14T09:08:07.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029685699999999998 2024-09-14T09:08:07.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151083+0000", 2024-09-14T09:08:07.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00090801800000000002 2024-09-14T09:08:07.084 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.084 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.084 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.084 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106715 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 15360~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.084 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:07.084 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.209756356, 2024-09-14T09:08:07.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3047478800000001, 2024-09-14T09:08:07.085 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.085 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.085 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.085 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.085 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.085 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.085 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106715 2024-09-14T09:08:07.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.085 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:07.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:07.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:07.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6e-07 2024-09-14T09:08:07.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847580+0000", 2024-09-14T09:08:07.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9250000000000003e-06 2024-09-14T09:08:07.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847585+0000", 2024-09-14T09:08:07.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8409999999999999e-06 2024-09-14T09:08:07.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847588+0000", 2024-09-14T09:08:07.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6989999999999999e-06 2024-09-14T09:08:07.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859151+0000", 2024-09-14T09:08:07.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011562604000000001 2024-09-14T09:08:07.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:07.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859159+0000", 2024-09-14T09:08:07.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4179999999999997e-06 2024-09-14T09:08:07.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151155+0000", 2024-09-14T09:08:07.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.291996068 2024-09-14T09:08:07.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.089 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.089 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151383+0000", 2024-09-14T09:08:07.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022746100000000001 2024-09-14T09:08:07.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.089 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.089 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152323+0000", 2024-09-14T09:08:07.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00094060400000000005 2024-09-14T09:08:07.089 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.089 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.089 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.089 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106717 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 19456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.090 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:07.090 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.209739667999997, 2024-09-14T09:08:07.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.305574579, 2024-09-14T09:08:07.090 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.090 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.090 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.090 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.090 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.090 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.090 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106717 2024-09-14T09:08:07.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.090 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:07.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:07.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:07.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1199999999999999e-07 2024-09-14T09:08:07.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847593+0000", 2024-09-14T09:08:07.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.257e-06 2024-09-14T09:08:07.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847598+0000", 2024-09-14T09:08:07.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4139999999999996e-06 2024-09-14T09:08:07.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847602+0000", 2024-09-14T09:08:07.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3440000000000003e-06 2024-09-14T09:08:07.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859203+0000", 2024-09-14T09:08:07.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011600572999999999 2024-09-14T09:08:07.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:07.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859211+0000", 2024-09-14T09:08:07.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-06 2024-09-14T09:08:07.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152398+0000", 2024-09-14T09:08:07.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.293187812 2024-09-14T09:08:07.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.094 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.094 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152543+0000", 2024-09-14T09:08:07.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014478000000000001 2024-09-14T09:08:07.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.094 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.094 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153166+0000", 2024-09-14T09:08:07.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000623257 2024-09-14T09:08:07.094 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.094 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.094 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.095 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106719 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 23552~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.095 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:07.095 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.209725352, 2024-09-14T09:08:07.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.306394982, 2024-09-14T09:08:07.095 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.095 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.095 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.095 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.095 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.095 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.095 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106719 2024-09-14T09:08:07.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.095 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:07.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:07.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:07.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5100000000000001e-07 2024-09-14T09:08:07.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847608+0000", 2024-09-14T09:08:07.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.189e-06 2024-09-14T09:08:07.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847612+0000", 2024-09-14T09:08:07.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9519999999999996e-06 2024-09-14T09:08:07.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847614+0000", 2024-09-14T09:08:07.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6280000000000001e-06 2024-09-14T09:08:07.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859254+0000", 2024-09-14T09:08:07.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011640157 2024-09-14T09:08:07.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:07.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859261+0000", 2024-09-14T09:08:07.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0570000000000004e-06 2024-09-14T09:08:07.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153236+0000", 2024-09-14T09:08:07.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2939744040000001 2024-09-14T09:08:07.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153380+0000", 2024-09-14T09:08:07.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014410399999999999 2024-09-14T09:08:07.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154001+0000", 2024-09-14T09:08:07.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062124000000000001 2024-09-14T09:08:07.099 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.099 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.099 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.099 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106721 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 27648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.099 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:07.099 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.209713702000002, 2024-09-14T09:08:07.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3074446659999999, 2024-09-14T09:08:07.100 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.100 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.100 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.100 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.100 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.100 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.100 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106721 2024-09-14T09:08:07.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.100 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:07.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:07.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:07.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4299999999999999e-07 2024-09-14T09:08:07.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847619+0000", 2024-09-14T09:08:07.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T09:08:07.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847624+0000", 2024-09-14T09:08:07.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4229999999999998e-06 2024-09-14T09:08:07.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847627+0000", 2024-09-14T09:08:07.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0120000000000001e-06 2024-09-14T09:08:07.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859305+0000", 2024-09-14T09:08:07.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01167812 2024-09-14T09:08:07.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:07.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859313+0000", 2024-09-14T09:08:07.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2409999999999995e-06 2024-09-14T09:08:07.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154081+0000", 2024-09-14T09:08:07.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.294768296 2024-09-14T09:08:07.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154280+0000", 2024-09-14T09:08:07.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019824 2024-09-14T09:08:07.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155063+0000", 2024-09-14T09:08:07.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00078303400000000005 2024-09-14T09:08:07.104 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.104 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.104 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.104 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106723 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 31744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.105 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:08:07.105 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.209701271999997, 2024-09-14T09:08:07.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308293648, 2024-09-14T09:08:07.105 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.105 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.105 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.105 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.105 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.105 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.105 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106723 2024-09-14T09:08:07.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.105 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:08:07.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:08:07.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847631+0000", 2024-09-14T09:08:07.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6100000000000002e-07 2024-09-14T09:08:07.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847632+0000", 2024-09-14T09:08:07.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0589999999999999e-06 2024-09-14T09:08:07.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847635+0000", 2024-09-14T09:08:07.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.551e-06 2024-09-14T09:08:07.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847638+0000", 2024-09-14T09:08:07.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.644e-06 2024-09-14T09:08:07.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859344+0000", 2024-09-14T09:08:07.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011706397 2024-09-14T09:08:07.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:07.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859348+0000", 2024-09-14T09:08:07.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5779999999999999e-06 2024-09-14T09:08:07.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155135+0000", 2024-09-14T09:08:07.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.295787448 2024-09-14T09:08:07.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155279+0000", 2024-09-14T09:08:07.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014400700000000001 2024-09-14T09:08:07.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155924+0000", 2024-09-14T09:08:07.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00064470300000000005 2024-09-14T09:08:07.109 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.109 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.109 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.109 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106725 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 35840~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.110 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:08:07.110 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.209690160999997, 2024-09-14T09:08:07.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.309103218, 2024-09-14T09:08:07.110 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.110 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.110 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.110 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.110 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.110 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.110 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106725 2024-09-14T09:08:07.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.110 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:08:07.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:08:07.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847642+0000", 2024-09-14T09:08:07.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4499999999999998e-07 2024-09-14T09:08:07.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847643+0000", 2024-09-14T09:08:07.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4e-07 2024-09-14T09:08:07.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847647+0000", 2024-09-14T09:08:07.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6190000000000002e-06 2024-09-14T09:08:07.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847655+0000", 2024-09-14T09:08:07.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:08:07.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859381+0000", 2024-09-14T09:08:07.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011725793 2024-09-14T09:08:07.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:07.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859391+0000", 2024-09-14T09:08:07.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4390000000000006e-06 2024-09-14T09:08:07.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155997+0000", 2024-09-14T09:08:07.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2966067830000001 2024-09-14T09:08:07.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156133+0000", 2024-09-14T09:08:07.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000135609 2024-09-14T09:08:07.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156745+0000", 2024-09-14T09:08:07.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000611592 2024-09-14T09:08:07.114 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.114 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.114 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.115 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106727 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 39936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.115 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:07.115 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.209671860999997, 2024-09-14T09:08:07.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3099046560000001, 2024-09-14T09:08:07.115 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.115 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.115 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.115 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.115 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.115 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.115 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106727 2024-09-14T09:08:07.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.115 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:07.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:07.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:07.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1300000000000001e-07 2024-09-14T09:08:07.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847661+0000", 2024-09-14T09:08:07.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.192e-06 2024-09-14T09:08:07.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847665+0000", 2024-09-14T09:08:07.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0999999999999997e-06 2024-09-14T09:08:07.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847668+0000", 2024-09-14T09:08:07.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.644e-06 2024-09-14T09:08:07.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859422+0000", 2024-09-14T09:08:07.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011753559 2024-09-14T09:08:07.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:07.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859425+0000", 2024-09-14T09:08:07.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.343e-06 2024-09-14T09:08:07.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156817+0000", 2024-09-14T09:08:07.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2973924800000001 2024-09-14T09:08:07.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156956+0000", 2024-09-14T09:08:07.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013835 2024-09-14T09:08:07.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157564+0000", 2024-09-14T09:08:07.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060867500000000004 2024-09-14T09:08:07.119 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.119 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.119 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.119 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106729 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 44032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.119 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:07.120 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.209659887999997, 2024-09-14T09:08:07.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3107195970000001, 2024-09-14T09:08:07.120 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.120 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.120 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.120 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.120 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.120 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.120 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106729 2024-09-14T09:08:07.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.120 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:07.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:07.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:07.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.29e-07 2024-09-14T09:08:07.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847673+0000", 2024-09-14T09:08:07.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.096e-06 2024-09-14T09:08:07.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847678+0000", 2024-09-14T09:08:07.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.527e-06 2024-09-14T09:08:07.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847681+0000", 2024-09-14T09:08:07.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.788e-06 2024-09-14T09:08:07.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859462+0000", 2024-09-14T09:08:07.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01178032 2024-09-14T09:08:07.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:07.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859471+0000", 2024-09-14T09:08:07.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3430000000000002e-06 2024-09-14T09:08:07.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157635+0000", 2024-09-14T09:08:07.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2981640320000001 2024-09-14T09:08:07.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157773+0000", 2024-09-14T09:08:07.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763 2024-09-14T09:08:07.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158391+0000", 2024-09-14T09:08:07.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061863200000000004 2024-09-14T09:08:07.124 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.124 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.124 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.124 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106731 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 48128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.125 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:08:07.125 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.209644243, 2024-09-14T09:08:07.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3115142020000001, 2024-09-14T09:08:07.125 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.125 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.125 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.125 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.125 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.125 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.125 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106731 2024-09-14T09:08:07.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.125 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:08:07.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:08:07.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847688+0000", 2024-09-14T09:08:07.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0199999999999998e-07 2024-09-14T09:08:07.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847689+0000", 2024-09-14T09:08:07.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4529999999999999e-06 2024-09-14T09:08:07.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847694+0000", 2024-09-14T09:08:07.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9080000000000003e-06 2024-09-14T09:08:07.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847697+0000", 2024-09-14T09:08:07.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4920000000000002e-06 2024-09-14T09:08:07.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859499+0000", 2024-09-14T09:08:07.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011801676000000001 2024-09-14T09:08:07.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:07.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859502+0000", 2024-09-14T09:08:07.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2899999999999998e-06 2024-09-14T09:08:07.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158461+0000", 2024-09-14T09:08:07.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.298958796 2024-09-14T09:08:07.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158594+0000", 2024-09-14T09:08:07.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013321599999999999 2024-09-14T09:08:07.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159202+0000", 2024-09-14T09:08:07.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060706900000000001 2024-09-14T09:08:07.129 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.129 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.129 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.129 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106733 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 52224~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.129 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:07.129 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.209629987, 2024-09-14T09:08:07.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3123343809999999, 2024-09-14T09:08:07.129 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.129 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.129 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.130 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.130 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.130 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.130 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106733 2024-09-14T09:08:07.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.130 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:07.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:07.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:07.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5800000000000001e-07 2024-09-14T09:08:07.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847703+0000", 2024-09-14T09:08:07.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.127e-06 2024-09-14T09:08:07.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847706+0000", 2024-09-14T09:08:07.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3210000000000001e-06 2024-09-14T09:08:07.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847709+0000", 2024-09-14T09:08:07.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7539999999999998e-06 2024-09-14T09:08:07.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859538+0000", 2024-09-14T09:08:07.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01182942 2024-09-14T09:08:07.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:07.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859547+0000", 2024-09-14T09:08:07.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T09:08:07.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159273+0000", 2024-09-14T09:08:07.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2997263700000001 2024-09-14T09:08:07.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159415+0000", 2024-09-14T09:08:07.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000141765 2024-09-14T09:08:07.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160036+0000", 2024-09-14T09:08:07.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062122100000000003 2024-09-14T09:08:07.134 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.134 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.134 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.134 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106735 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 56320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.134 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:07.134 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.209618542999998, 2024-09-14T09:08:07.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3131339309999999, 2024-09-14T09:08:07.134 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.134 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.134 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.135 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106735 2024-09-14T09:08:07.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.135 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:07.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:07.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:07.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2100000000000001e-07 2024-09-14T09:08:07.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847714+0000", 2024-09-14T09:08:07.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.051e-06 2024-09-14T09:08:07.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847719+0000", 2024-09-14T09:08:07.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3640000000000001e-06 2024-09-14T09:08:07.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847723+0000", 2024-09-14T09:08:07.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8789999999999996e-06 2024-09-14T09:08:07.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859577+0000", 2024-09-14T09:08:07.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011854416 2024-09-14T09:08:07.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:07.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859580+0000", 2024-09-14T09:08:07.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.292e-06 2024-09-14T09:08:07.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160108+0000", 2024-09-14T09:08:07.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.30052758 2024-09-14T09:08:07.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160238+0000", 2024-09-14T09:08:07.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013043500000000001 2024-09-14T09:08:07.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160847+0000", 2024-09-14T09:08:07.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060869299999999999 2024-09-14T09:08:07.139 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.139 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.139 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.139 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106738 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 60416~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.139 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:07.139 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.209605338999999, 2024-09-14T09:08:07.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3139482179999999, 2024-09-14T09:08:07.140 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.140 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.140 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.140 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.140 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.140 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.140 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106738 2024-09-14T09:08:07.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.140 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:07.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:07.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:07.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3999999999999998e-07 2024-09-14T09:08:07.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847733+0000", 2024-09-14T09:08:07.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8070000000000002e-06 2024-09-14T09:08:07.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847738+0000", 2024-09-14T09:08:07.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9740000000000001e-06 2024-09-14T09:08:07.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847742+0000", 2024-09-14T09:08:07.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6009999999999999e-06 2024-09-14T09:08:07.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859612+0000", 2024-09-14T09:08:07.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011870537 2024-09-14T09:08:07.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:07.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859621+0000", 2024-09-14T09:08:07.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3019999999999995e-06 2024-09-14T09:08:07.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160917+0000", 2024-09-14T09:08:07.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3012961080000001 2024-09-14T09:08:07.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161054+0000", 2024-09-14T09:08:07.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000137422 2024-09-14T09:08:07.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161674+0000", 2024-09-14T09:08:07.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000620227 2024-09-14T09:08:07.144 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.144 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.144 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.144 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106739 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 64512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.144 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:07.145 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.209585498999999, 2024-09-14T09:08:07.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.314753777, 2024-09-14T09:08:07.145 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.145 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.145 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.145 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.145 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.145 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.145 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106739 2024-09-14T09:08:07.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.145 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:07.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:07.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:07.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9700000000000003e-07 2024-09-14T09:08:07.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847748+0000", 2024-09-14T09:08:07.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2589999999999999e-06 2024-09-14T09:08:07.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847752+0000", 2024-09-14T09:08:07.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6099999999999999e-06 2024-09-14T09:08:07.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847755+0000", 2024-09-14T09:08:07.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0520000000000002e-06 2024-09-14T09:08:07.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859653+0000", 2024-09-14T09:08:07.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011897575000000001 2024-09-14T09:08:07.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:07.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859658+0000", 2024-09-14T09:08:07.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8810000000000004e-06 2024-09-14T09:08:07.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161745+0000", 2024-09-14T09:08:07.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3020870819999999 2024-09-14T09:08:07.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161877+0000", 2024-09-14T09:08:07.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000132657 2024-09-14T09:08:07.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162500+0000", 2024-09-14T09:08:07.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062236400000000003 2024-09-14T09:08:07.149 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.149 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.149 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.149 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106741 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 68608~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.150 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:07.150 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.209572707, 2024-09-14T09:08:07.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3155568449999999, 2024-09-14T09:08:07.150 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.150 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.150 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.150 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.150 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.150 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.150 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106741 2024-09-14T09:08:07.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.150 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:07.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:07.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:07.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5800000000000001e-07 2024-09-14T09:08:07.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847760+0000", 2024-09-14T09:08:07.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:08:07.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847764+0000", 2024-09-14T09:08:07.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3189999999999999e-06 2024-09-14T09:08:07.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847766+0000", 2024-09-14T09:08:07.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6429999999999999e-06 2024-09-14T09:08:07.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859693+0000", 2024-09-14T09:08:07.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011926724 2024-09-14T09:08:07.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:07.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859703+0000", 2024-09-14T09:08:07.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6160000000000008e-06 2024-09-14T09:08:07.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162575+0000", 2024-09-14T09:08:07.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.302872128 2024-09-14T09:08:07.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162709+0000", 2024-09-14T09:08:07.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013446000000000001 2024-09-14T09:08:07.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163316+0000", 2024-09-14T09:08:07.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060650700000000005 2024-09-14T09:08:07.154 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.154 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.154 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.155 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106744 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 76800~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:07.155 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:08:07.155 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 45.209561596, 2024-09-14T09:08:07.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3163901789999999, 2024-09-14T09:08:07.155 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:07.155 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:07.155 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:07.155 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:07.155 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:07.155 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:07.155 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106744 2024-09-14T09:08:07.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.155 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:07.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:07.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:08:07.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:07.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:08:07.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:07.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:07.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996947+0000", 2024-09-14T09:08:07.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.149177154 2024-09-14T09:08:07.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:07.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996964+0000", 2024-09-14T09:08:07.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.679e-05 2024-09-14T09:08:07.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:07.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996999+0000", 2024-09-14T09:08:07.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5463000000000002e-05 2024-09-14T09:08:07.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:07.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.997029+0000", 2024-09-14T09:08:07.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9567e-05 2024-09-14T09:08:07.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:07.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163390+0000", 2024-09-14T09:08:07.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.16636056799999999 2024-09-14T09:08:07.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:07.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163526+0000", 2024-09-14T09:08:07.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013655899999999999 2024-09-14T09:08:07.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:07.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:07.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:07.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.164160+0000", 2024-09-14T09:08:07.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00063407800000000005 2024-09-14T09:08:07.158 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.159 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.159 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.159 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:07.159 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:07.159 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:07.160 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:08:07.161 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:07.159+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:07.161 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:08:07.215 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is unset 2024-09-14T09:08:07.261 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:07.259+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:07.347 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:07.347 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:07.347 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:07.347 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:07.361 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:07.363+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:07.362 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:08:07.365 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:08:07.366 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:08:07.383 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:08:07.383 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:08:07.402 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:08:07.402 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:08:07.422 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:08:07.422 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:08:07.422 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:08:07.443 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:08:07.443 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:08:07.462 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:07.466+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:07.463 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:08:07.463 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:08:07.484 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:08:07.485 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:08:07.486 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:07.486 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:07.488 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:08:07.488 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T09:08:07.501 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:08:07.590 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:07.591+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:07.664 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:07.663+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:07.666 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:07.666 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:07.681 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:08:07.765 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:07.763+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:07.812 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:07.812 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:07.827 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:08:07.874 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:07.878+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:07.952 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:07.952 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:07.966 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:08:08.014 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:08.018+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:08.069 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:08.067+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:08.134 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:08.135 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:08.149 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:08:08.194 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:08.198+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:08.268 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:08.269 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:08.270 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:08.271+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:08.283 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:08:08.371 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:08.374+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:08.453 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:08.454 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:08.468 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:08:08.472 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:08.471+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:08.573 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:08.575+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:08.632 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:08.632 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:08.647 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:08:08.694 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:08.698+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:08.770 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:08.771 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:08.774 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:08.775+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:08.785 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:08:08.876 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:08.878+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:08.948 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:08.948 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:08.962 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:08:09.006 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:09.010+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:09.077 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:09.082+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:09.130 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:09.130 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:09.145 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:08:09.194 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:09.195+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:09.264 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T09:08:09.290 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:09.290 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:09.290 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:09.291+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:09.308 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:08:09.380 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:09.382+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:09.473 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:09.473 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:09.481 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:09.479+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:09.488 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:08:09.489 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:08:09.581 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:09.579+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:09.648 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:09.648 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:09.664 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:08:09.670 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:08:09.670 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:08:09.682 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:09.686+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:09.687 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:08:09.687 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:08:09.704 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:08:09.704 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:08:09.720 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:08:09.721 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:08:09.721 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:08:09.741 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:08:09.742 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:08:09.762 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:08:09.763 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:08:09.783 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:08:09.783 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:08:09.783 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:09.783+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:09.786 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:09.786 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:09.786 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:09.786 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:09.788 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:08:09.789 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T09:08:09.801 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:08:09.890 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:09.891+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:09.967 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:09.967 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:09.968 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:08:09.968 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:08:09.968 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:09.981 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:08:10.026 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:10.027+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:10.086 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:10.090+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:10.153 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:10.154 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:08:10.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:08:10.154 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:08:10.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.154 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73542 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15319040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:10.154 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:08:10.154 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.88341982, 2024-09-14T09:08:10.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0237091359999999, 2024-09-14T09:08:10.154 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.154 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.155 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.155 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.155 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.155 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.155 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73542 2024-09-14T09:08:10.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.155 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:08:10.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:08:10.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268762+0000", 2024-09-14T09:08:10.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:08:10.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268766+0000", 2024-09-14T09:08:10.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5880000000000002e-06 2024-09-14T09:08:10.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268776+0000", 2024-09-14T09:08:10.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0168e-05 2024-09-14T09:08:10.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268789+0000", 2024-09-14T09:08:10.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:08:10.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282053+0000", 2024-09-14T09:08:10.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013264496000000001 2024-09-14T09:08:10.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282057+0000", 2024-09-14T09:08:10.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6090000000000001e-06 2024-09-14T09:08:10.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292052+0000", 2024-09-14T09:08:10.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0099950070000001 2024-09-14T09:08:10.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292262+0000", 2024-09-14T09:08:10.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021009800000000001 2024-09-14T09:08:10.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292470+0000", 2024-09-14T09:08:10.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000208314 2024-09-14T09:08:10.159 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.159 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.159 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.159 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73543 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:10.159 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:08:10.160 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.883381029999995, 2024-09-14T09:08:10.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023985473, 2024-09-14T09:08:10.160 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.160 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.160 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.160 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.160 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.160 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.160 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73543 2024-09-14T09:08:10.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.160 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:08:10.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:08:10.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268801+0000", 2024-09-14T09:08:10.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:08:10.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268803+0000", 2024-09-14T09:08:10.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7439999999999999e-06 2024-09-14T09:08:10.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268812+0000", 2024-09-14T09:08:10.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7080000000000003e-06 2024-09-14T09:08:10.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268819+0000", 2024-09-14T09:08:10.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T09:08:10.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282068+0000", 2024-09-14T09:08:10.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013249540000000001 2024-09-14T09:08:10.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282072+0000", 2024-09-14T09:08:10.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0960000000000001e-06 2024-09-14T09:08:10.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292573+0000", 2024-09-14T09:08:10.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0105014560000001 2024-09-14T09:08:10.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292679+0000", 2024-09-14T09:08:10.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010545899999999999 2024-09-14T09:08:10.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292786+0000", 2024-09-14T09:08:10.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107052 2024-09-14T09:08:10.164 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.164 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.164 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.165 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73544 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:10.165 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:10.165 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.883353327999998, 2024-09-14T09:08:10.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0244830549999999, 2024-09-14T09:08:10.165 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.165 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.165 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.165 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.165 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.165 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.165 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73544 2024-09-14T09:08:10.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.166 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:10.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:10.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:10.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5599999999999995e-07 2024-09-14T09:08:10.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268831+0000", 2024-09-14T09:08:10.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.57e-06 2024-09-14T09:08:10.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268844+0000", 2024-09-14T09:08:10.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2811e-05 2024-09-14T09:08:10.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268851+0000", 2024-09-14T09:08:10.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9140000000000002e-06 2024-09-14T09:08:10.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282083+0000", 2024-09-14T09:08:10.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013232111 2024-09-14T09:08:10.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282086+0000", 2024-09-14T09:08:10.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2909999999999999e-06 2024-09-14T09:08:10.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292884+0000", 2024-09-14T09:08:10.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0107975069999999 2024-09-14T09:08:10.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293100+0000", 2024-09-14T09:08:10.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021639399999999999 2024-09-14T09:08:10.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293311+0000", 2024-09-14T09:08:10.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021090099999999999 2024-09-14T09:08:10.170 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.170 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.170 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.170 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73545 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15564800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:10.170 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:08:10.170 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.883302142999995, 2024-09-14T09:08:10.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0249746740000001, 2024-09-14T09:08:10.170 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.170 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.170 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.170 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.171 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.171 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.171 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73545 2024-09-14T09:08:10.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.171 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:08:10.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:08:10.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268880+0000", 2024-09-14T09:08:10.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5600000000000005e-07 2024-09-14T09:08:10.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268889+0000", 2024-09-14T09:08:10.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8559999999999996e-06 2024-09-14T09:08:10.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268913+0000", 2024-09-14T09:08:10.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3958e-05 2024-09-14T09:08:10.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268921+0000", 2024-09-14T09:08:10.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7479999999999999e-06 2024-09-14T09:08:10.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282098+0000", 2024-09-14T09:08:10.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013176297999999999 2024-09-14T09:08:10.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282101+0000", 2024-09-14T09:08:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2329999999999998e-06 2024-09-14T09:08:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293435+0000", 2024-09-14T09:08:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0113339459999999 2024-09-14T09:08:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293648+0000", 2024-09-14T09:08:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021307100000000001 2024-09-14T09:08:10.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293854+0000", 2024-09-14T09:08:10.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020580800000000001 2024-09-14T09:08:10.175 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.175 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.175 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.175 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73546 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15646720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:10.175 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:08:10.175 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.883248272000003, 2024-09-14T09:08:10.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0254846360000001, 2024-09-14T09:08:10.175 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.176 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.176 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.176 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.176 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.176 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.176 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73546 2024-09-14T09:08:10.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.176 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:08:10.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:08:10.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268934+0000", 2024-09-14T09:08:10.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7999999999999995e-07 2024-09-14T09:08:10.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268937+0000", 2024-09-14T09:08:10.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9840000000000001e-06 2024-09-14T09:08:10.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268947+0000", 2024-09-14T09:08:10.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0522e-05 2024-09-14T09:08:10.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268955+0000", 2024-09-14T09:08:10.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8059999999999995e-06 2024-09-14T09:08:10.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282112+0000", 2024-09-14T09:08:10.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013157189 2024-09-14T09:08:10.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282115+0000", 2024-09-14T09:08:10.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2949999999999998e-06 2024-09-14T09:08:10.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293968+0000", 2024-09-14T09:08:10.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0118530990000001 2024-09-14T09:08:10.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294197+0000", 2024-09-14T09:08:10.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228507 2024-09-14T09:08:10.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294417+0000", 2024-09-14T09:08:10.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000220554 2024-09-14T09:08:10.180 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.180 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.180 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.180 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73547 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:10.180 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:08:10.180 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.883215825999997, 2024-09-14T09:08:10.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.025970525, 2024-09-14T09:08:10.181 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.181 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.181 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.181 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.181 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.181 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.181 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73547 2024-09-14T09:08:10.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.181 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:08:10.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:08:10.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268966+0000", 2024-09-14T09:08:10.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3799999999999997e-07 2024-09-14T09:08:10.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268969+0000", 2024-09-14T09:08:10.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8739999999999999e-06 2024-09-14T09:08:10.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268979+0000", 2024-09-14T09:08:10.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9939999999999995e-06 2024-09-14T09:08:10.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268986+0000", 2024-09-14T09:08:10.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3459999999999996e-06 2024-09-14T09:08:10.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282127+0000", 2024-09-14T09:08:10.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01314039 2024-09-14T09:08:10.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282130+0000", 2024-09-14T09:08:10.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2219999999999998e-06 2024-09-14T09:08:10.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.184 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.184 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294540+0000", 2024-09-14T09:08:10.184 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124105880000001 2024-09-14T09:08:10.184 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.184 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294759+0000", 2024-09-14T09:08:10.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000218798 2024-09-14T09:08:10.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.185 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.185 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294936+0000", 2024-09-14T09:08:10.185 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000176675 2024-09-14T09:08:10.185 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.185 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.185 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.185 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.185 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.185 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73548 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15810560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:10.186 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:08:10.186 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.883184713999995, 2024-09-14T09:08:10.186 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0265045129999999, 2024-09-14T09:08:10.186 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.186 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.186 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.186 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.186 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73548 2024-09-14T09:08:10.186 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.186 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.186 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.186 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:08:10.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:08:10.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.187 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.187 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268997+0000", 2024-09-14T09:08:10.187 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7599999999999996e-07 2024-09-14T09:08:10.187 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.187 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269000+0000", 2024-09-14T09:08:10.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0580000000000002e-06 2024-09-14T09:08:10.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269010+0000", 2024-09-14T09:08:10.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124e-05 2024-09-14T09:08:10.188 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.188 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.188 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.188 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269018+0000", 2024-09-14T09:08:10.188 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5399999999999998e-06 2024-09-14T09:08:10.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282141+0000", 2024-09-14T09:08:10.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013123012 2024-09-14T09:08:10.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282144+0000", 2024-09-14T09:08:10.189 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.196e-06 2024-09-14T09:08:10.189 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.189 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.189 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.189 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295042+0000", 2024-09-14T09:08:10.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0128975689999999 2024-09-14T09:08:10.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295272+0000", 2024-09-14T09:08:10.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023048099999999999 2024-09-14T09:08:10.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.190 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.190 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.190 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295501+0000", 2024-09-14T09:08:10.190 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228657 2024-09-14T09:08:10.190 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.190 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.190 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.190 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.191 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.191 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73549 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15892480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:10.191 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:08:10.191 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.883138834999997, 2024-09-14T09:08:10.191 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0268354099999999, 2024-09-14T09:08:10.191 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.191 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.191 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.191 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.191 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73549 2024-09-14T09:08:10.191 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.191 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:08:10.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:08:10.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.192 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.192 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.192 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.192 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269043+0000", 2024-09-14T09:08:10.192 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6199999999999997e-07 2024-09-14T09:08:10.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269046+0000", 2024-09-14T09:08:10.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2480000000000001e-06 2024-09-14T09:08:10.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269057+0000", 2024-09-14T09:08:10.193 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0436e-05 2024-09-14T09:08:10.193 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.193 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.193 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.193 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269064+0000", 2024-09-14T09:08:10.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5179999999999998e-06 2024-09-14T09:08:10.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282155+0000", 2024-09-14T09:08:10.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013091190000000001 2024-09-14T09:08:10.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282159+0000", 2024-09-14T09:08:10.194 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.366e-06 2024-09-14T09:08:10.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.194 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.194 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.194 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295599+0000", 2024-09-14T09:08:10.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0134405559999999 2024-09-14T09:08:10.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295765+0000", 2024-09-14T09:08:10.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016575499999999999 2024-09-14T09:08:10.195 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.195 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.195 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.195 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295878+0000", 2024-09-14T09:08:10.195 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112679 2024-09-14T09:08:10.195 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.195 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.195 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.196 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.196 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.196 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73550 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15974400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:10.196 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:08:10.196 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.883106235, 2024-09-14T09:08:10.196 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027357657, 2024-09-14T09:08:10.196 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.196 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.196 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.196 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.196 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.196 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.196 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73550 2024-09-14T09:08:10.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.197 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:08:10.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.197 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:08:10.197 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.197 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.197 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.197 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269076+0000", 2024-09-14T09:08:10.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:08:10.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269079+0000", 2024-09-14T09:08:10.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:08:10.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.198 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.198 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269098+0000", 2024-09-14T09:08:10.198 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9698e-05 2024-09-14T09:08:10.198 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.198 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269106+0000", 2024-09-14T09:08:10.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6179999999999997e-06 2024-09-14T09:08:10.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282170+0000", 2024-09-14T09:08:10.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013063874 2024-09-14T09:08:10.199 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.199 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.199 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.199 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282173+0000", 2024-09-14T09:08:10.199 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1999999999999999e-06 2024-09-14T09:08:10.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295996+0000", 2024-09-14T09:08:10.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.013823151 2024-09-14T09:08:10.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296214+0000", 2024-09-14T09:08:10.200 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021817700000000001 2024-09-14T09:08:10.200 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.200 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.200 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.200 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296433+0000", 2024-09-14T09:08:10.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021829099999999999 2024-09-14T09:08:10.201 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.201 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.201 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.201 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.201 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.201 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73551 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:10.201 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:08:10.201 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.883064817999994, 2024-09-14T09:08:10.201 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027730276, 2024-09-14T09:08:10.201 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.201 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.201 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.202 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.202 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.202 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.202 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73551 2024-09-14T09:08:10.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.202 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:08:10.202 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.202 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.202 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.202 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.202 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:08:10.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269117+0000", 2024-09-14T09:08:10.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:08:10.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.203 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.203 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269120+0000", 2024-09-14T09:08:10.203 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.056e-06 2024-09-14T09:08:10.203 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.203 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269130+0000", 2024-09-14T09:08:10.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0084e-05 2024-09-14T09:08:10.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269137+0000", 2024-09-14T09:08:10.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1359999999999999e-06 2024-09-14T09:08:10.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.204 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.204 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.204 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282184+0000", 2024-09-14T09:08:10.204 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013046537 2024-09-14T09:08:10.204 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282188+0000", 2024-09-14T09:08:10.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3220000000000003e-06 2024-09-14T09:08:10.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296529+0000", 2024-09-14T09:08:10.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0143406109999999 2024-09-14T09:08:10.205 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.205 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.205 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.205 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296695+0000", 2024-09-14T09:08:10.205 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016604799999999999 2024-09-14T09:08:10.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.206 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.206 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296847+0000", 2024-09-14T09:08:10.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000151822 2024-09-14T09:08:10.206 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.206 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.206 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.206 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.206 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.206 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73552 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:10.206 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:10.206 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.883034237999993, 2024-09-14T09:08:10.206 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0281767500000001, 2024-09-14T09:08:10.207 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.207 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.207 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.207 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.207 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.207 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.207 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73552 2024-09-14T09:08:10.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.207 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.207 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.207 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.207 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:10.207 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.207 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:10.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:10.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:08:10.208 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.208 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.208 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.208 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269150+0000", 2024-09-14T09:08:10.208 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9119999999999998e-06 2024-09-14T09:08:10.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269160+0000", 2024-09-14T09:08:10.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6819999999999998e-06 2024-09-14T09:08:10.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.209 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269168+0000", 2024-09-14T09:08:10.209 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6079999999999998e-06 2024-09-14T09:08:10.209 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.209 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.209 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282205+0000", 2024-09-14T09:08:10.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013036982000000001 2024-09-14T09:08:10.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282208+0000", 2024-09-14T09:08:10.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4620000000000001e-06 2024-09-14T09:08:10.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.210 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.210 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296968+0000", 2024-09-14T09:08:10.210 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0147601209999999 2024-09-14T09:08:10.210 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.210 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297115+0000", 2024-09-14T09:08:10.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014673900000000001 2024-09-14T09:08:10.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.211 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.211 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297324+0000", 2024-09-14T09:08:10.211 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020866400000000001 2024-09-14T09:08:10.211 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.211 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.211 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.211 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.211 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.211 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73553 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:10.212 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:08:10.212 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.883004174000007, 2024-09-14T09:08:10.212 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0286556739999999, 2024-09-14T09:08:10.212 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.212 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.212 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.212 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.212 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73553 2024-09-14T09:08:10.212 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.212 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.212 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:08:10.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:08:10.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.213 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.213 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.213 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269178+0000", 2024-09-14T09:08:10.213 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:08:10.213 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269180+0000", 2024-09-14T09:08:10.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T09:08:10.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269189+0000", 2024-09-14T09:08:10.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2e-06 2024-09-14T09:08:10.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.214 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.214 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.214 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269197+0000", 2024-09-14T09:08:10.214 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2980000000000003e-06 2024-09-14T09:08:10.214 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282219+0000", 2024-09-14T09:08:10.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013022687 2024-09-14T09:08:10.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282223+0000", 2024-09-14T09:08:10.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.241e-06 2024-09-14T09:08:10.215 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.215 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.215 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.215 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297470+0000", 2024-09-14T09:08:10.215 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.015247778 2024-09-14T09:08:10.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297704+0000", 2024-09-14T09:08:10.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000233367 2024-09-14T09:08:10.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.216 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.216 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.216 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297833+0000", 2024-09-14T09:08:10.216 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012888900000000001 2024-09-14T09:08:10.216 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.216 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.216 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.216 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.217 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.217 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73554 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:10.217 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:08:10.217 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.882974856999994, 2024-09-14T09:08:10.217 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.029005113, 2024-09-14T09:08:10.217 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.217 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.217 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.217 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.217 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.217 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.217 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73554 2024-09-14T09:08:10.217 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.217 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:08:10.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:08:10.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.218 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.218 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.218 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269207+0000", 2024-09-14T09:08:10.218 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.68e-07 2024-09-14T09:08:10.218 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269210+0000", 2024-09-14T09:08:10.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8100000000000002e-06 2024-09-14T09:08:10.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269219+0000", 2024-09-14T09:08:10.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:08:10.219 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.219 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.219 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.219 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269227+0000", 2024-09-14T09:08:10.219 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3839999999999999e-06 2024-09-14T09:08:10.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282234+0000", 2024-09-14T09:08:10.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013007187 2024-09-14T09:08:10.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282237+0000", 2024-09-14T09:08:10.220 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1200000000000002e-06 2024-09-14T09:08:10.220 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.220 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.220 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.220 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297942+0000", 2024-09-14T09:08:10.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0157048040000001 2024-09-14T09:08:10.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298087+0000", 2024-09-14T09:08:10.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014499099999999999 2024-09-14T09:08:10.221 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.221 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.221 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.221 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298211+0000", 2024-09-14T09:08:10.221 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012461899999999999 2024-09-14T09:08:10.221 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.221 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.221 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.222 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.222 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.222 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73555 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:10.222 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:08:10.222 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.882945008999997, 2024-09-14T09:08:10.222 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.02941558, 2024-09-14T09:08:10.222 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.222 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.222 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.222 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.223 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.223 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73555 2024-09-14T09:08:10.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.223 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:08:10.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.223 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.223 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.223 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.223 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:08:10.223 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269237+0000", 2024-09-14T09:08:10.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3399999999999999e-07 2024-09-14T09:08:10.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.224 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.224 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269240+0000", 2024-09-14T09:08:10.224 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8480000000000001e-06 2024-09-14T09:08:10.224 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.224 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269249+0000", 2024-09-14T09:08:10.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1880000000000008e-06 2024-09-14T09:08:10.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269256+0000", 2024-09-14T09:08:10.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:08:10.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.225 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.225 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.225 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282248+0000", 2024-09-14T09:08:10.225 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012992327 2024-09-14T09:08:10.225 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282251+0000", 2024-09-14T09:08:10.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2160000000000002e-06 2024-09-14T09:08:10.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298299+0000", 2024-09-14T09:08:10.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0160479330000001 2024-09-14T09:08:10.226 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.226 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.226 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.226 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298446+0000", 2024-09-14T09:08:10.226 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000146525 2024-09-14T09:08:10.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.227 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.227 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298652+0000", 2024-09-14T09:08:10.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020584100000000001 2024-09-14T09:08:10.227 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.227 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.227 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.227 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.227 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.227 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73556 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:10.227 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:10.227 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.882915263000001, 2024-09-14T09:08:10.227 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0299854239999999, 2024-09-14T09:08:10.228 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.228 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.228 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.228 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.228 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.228 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.228 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73556 2024-09-14T09:08:10.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.228 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.228 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.228 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.228 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:10.228 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.228 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:10.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:10.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:08:10.229 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.229 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.229 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.229 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269270+0000", 2024-09-14T09:08:10.229 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0920000000000002e-06 2024-09-14T09:08:10.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269280+0000", 2024-09-14T09:08:10.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0013999999999999e-05 2024-09-14T09:08:10.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269288+0000", 2024-09-14T09:08:10.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:08:10.230 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.230 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.230 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.230 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282263+0000", 2024-09-14T09:08:10.230 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012974703000000001 2024-09-14T09:08:10.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282266+0000", 2024-09-14T09:08:10.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1520000000000001e-06 2024-09-14T09:08:10.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.231 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298729+0000", 2024-09-14T09:08:10.231 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.016463506 2024-09-14T09:08:10.231 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.231 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.231 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298884+0000", 2024-09-14T09:08:10.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015450800000000001 2024-09-14T09:08:10.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.232 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.232 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299251+0000", 2024-09-14T09:08:10.232 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036759899999999997 2024-09-14T09:08:10.232 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.232 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.232 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.232 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.232 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.233 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73557 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16547840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:10.233 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:08:10.233 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.882868612999999, 2024-09-14T09:08:10.233 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0307969480000001, 2024-09-14T09:08:10.233 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.233 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.233 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.233 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.233 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.233 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.233 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73557 2024-09-14T09:08:10.233 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.233 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:08:10.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:08:10.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.234 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.234 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.234 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.234 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269313+0000", 2024-09-14T09:08:10.234 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.92e-07 2024-09-14T09:08:10.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269336+0000", 2024-09-14T09:08:10.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3118000000000001e-05 2024-09-14T09:08:10.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269346+0000", 2024-09-14T09:08:10.235 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4069999999999999e-06 2024-09-14T09:08:10.235 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.235 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.235 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.235 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269352+0000", 2024-09-14T09:08:10.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5810000000000002e-06 2024-09-14T09:08:10.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282277+0000", 2024-09-14T09:08:10.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012924658 2024-09-14T09:08:10.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.236 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.236 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.236 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282280+0000", 2024-09-14T09:08:10.236 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2880000000000001e-06 2024-09-14T09:08:10.236 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299343+0000", 2024-09-14T09:08:10.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0170630190000001 2024-09-14T09:08:10.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299554+0000", 2024-09-14T09:08:10.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021085099999999999 2024-09-14T09:08:10.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.237 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.237 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.237 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300109+0000", 2024-09-14T09:08:10.237 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00055533399999999999 2024-09-14T09:08:10.238 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.238 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.238 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.238 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.238 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.238 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73558 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:10.238 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:08:10.238 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.882816989999995, 2024-09-14T09:08:10.238 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0314849340000001, 2024-09-14T09:08:10.238 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.238 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.238 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.238 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.238 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.239 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.239 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73558 2024-09-14T09:08:10.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.239 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:08:10.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.239 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.239 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.239 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.239 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:08:10.239 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269365+0000", 2024-09-14T09:08:10.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7400000000000003e-07 2024-09-14T09:08:10.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269367+0000", 2024-09-14T09:08:10.240 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4930000000000001e-06 2024-09-14T09:08:10.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.240 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.240 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.240 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269374+0000", 2024-09-14T09:08:10.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9519999999999997e-06 2024-09-14T09:08:10.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269379+0000", 2024-09-14T09:08:10.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9710000000000003e-06 2024-09-14T09:08:10.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.241 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282292+0000", 2024-09-14T09:08:10.241 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012912704000000001 2024-09-14T09:08:10.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.241 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.241 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282295+0000", 2024-09-14T09:08:10.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.179e-06 2024-09-14T09:08:10.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300195+0000", 2024-09-14T09:08:10.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.017899975 2024-09-14T09:08:10.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.242 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.242 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300345+0000", 2024-09-14T09:08:10.242 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014957499999999999 2024-09-14T09:08:10.242 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.242 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.243 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.243 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300849+0000", 2024-09-14T09:08:10.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050451100000000002 2024-09-14T09:08:10.243 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.243 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.243 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.243 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.243 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.243 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73559 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:10.243 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:10.243 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 95.882794580999999, 2024-09-14T09:08:10.243 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.031916343, 2024-09-14T09:08:10.243 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.243 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.243 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.244 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.244 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.244 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.244 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73559 2024-09-14T09:08:10.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.244 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:10.244 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.244 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.244 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.244 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.244 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:10.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:10.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3500000000000002e-07 2024-09-14T09:08:10.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.245 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269390+0000", 2024-09-14T09:08:10.245 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5890000000000001e-06 2024-09-14T09:08:10.245 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.245 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.245 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269396+0000", 2024-09-14T09:08:10.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3300000000000004e-06 2024-09-14T09:08:10.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269401+0000", 2024-09-14T09:08:10.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0340000000000004e-06 2024-09-14T09:08:10.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.246 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.246 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282311+0000", 2024-09-14T09:08:10.246 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012910204 2024-09-14T09:08:10.246 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.246 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282317+0000", 2024-09-14T09:08:10.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2639999999999997e-06 2024-09-14T09:08:10.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300933+0000", 2024-09-14T09:08:10.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0186160710000001 2024-09-14T09:08:10.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.247 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.247 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.247 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301158+0000", 2024-09-14T09:08:10.247 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225021 2024-09-14T09:08:10.247 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.248 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.248 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301303+0000", 2024-09-14T09:08:10.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014439499999999999 2024-09-14T09:08:10.248 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.248 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.248 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.248 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.248 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.248 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106964 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:10.248 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:08:10.248 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 49.036866746999998, 2024-09-14T09:08:10.248 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8802924409999999, 2024-09-14T09:08:10.248 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.249 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.249 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:10.249 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.249 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.249 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.249 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106964 2024-09-14T09:08:10.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.249 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.249 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:08:10.249 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.249 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.249 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.249 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:08:10.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115317+0000", 2024-09-14T09:08:10.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6429999999999999e-06 2024-09-14T09:08:10.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.250 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.250 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115326+0000", 2024-09-14T09:08:10.250 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2140000000000002e-06 2024-09-14T09:08:10.250 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.250 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115346+0000", 2024-09-14T09:08:10.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9419000000000002e-05 2024-09-14T09:08:10.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115357+0000", 2024-09-14T09:08:10.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1014e-05 2024-09-14T09:08:10.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.251 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.251 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115431+0000", 2024-09-14T09:08:10.251 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4040000000000003e-05 2024-09-14T09:08:10.251 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.251 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115438+0000", 2024-09-14T09:08:10.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9920000000000001e-06 2024-09-14T09:08:10.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844944+0000", 2024-09-14T09:08:10.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72950628399999995 2024-09-14T09:08:10.252 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.252 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.252 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.252 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844961+0000", 2024-09-14T09:08:10.252 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7156e-05 2024-09-14T09:08:10.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988004+0000", 2024-09-14T09:08:10.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.14304297 2024-09-14T09:08:10.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988511+0000", 2024-09-14T09:08:10.253 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050694100000000001 2024-09-14T09:08:10.253 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.253 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.253 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:08:10.253 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.989802+0000", 2024-09-14T09:08:10.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001291195 2024-09-14T09:08:10.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:10.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994866+0000", 2024-09-14T09:08:10.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050632760000000002 2024-09-14T09:08:10.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:10.254 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994988+0000", 2024-09-14T09:08:10.254 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012252899999999999 2024-09-14T09:08:10.254 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.254 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.254 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:10.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995198+0000", 2024-09-14T09:08:10.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021010899999999999 2024-09-14T09:08:10.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.255 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.255 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995607+0000", 2024-09-14T09:08:10.255 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040865900000000002 2024-09-14T09:08:10.255 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.255 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.255 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.255 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.255 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.256 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106965 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:10.256 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:10.256 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 49.035664369999999, 2024-09-14T09:08:10.256 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.879730162, 2024-09-14T09:08:10.256 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.256 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.256 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:10.256 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.256 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.256 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.257 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106965 2024-09-14T09:08:10.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.257 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:10.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.257 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.257 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.257 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:10.257 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.257 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:10.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:08:10.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116522+0000", 2024-09-14T09:08:10.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3989999999999997e-06 2024-09-14T09:08:10.258 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.258 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.258 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.258 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116533+0000", 2024-09-14T09:08:10.258 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0669000000000001e-05 2024-09-14T09:08:10.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116541+0000", 2024-09-14T09:08:10.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2730000000000002e-06 2024-09-14T09:08:10.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116604+0000", 2024-09-14T09:08:10.259 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2706999999999994e-05 2024-09-14T09:08:10.259 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.259 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.259 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.259 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116610+0000", 2024-09-14T09:08:10.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8630000000000002e-06 2024-09-14T09:08:10.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845060+0000", 2024-09-14T09:08:10.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72845003699999999 2024-09-14T09:08:10.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.260 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.260 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845079+0000", 2024-09-14T09:08:10.260 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8287999999999998e-05 2024-09-14T09:08:10.260 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.260 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.991866+0000", 2024-09-14T09:08:10.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1467871519999999 2024-09-14T09:08:10.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.992274+0000", 2024-09-14T09:08:10.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040813500000000001 2024-09-14T09:08:10.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.261 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.261 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:08:10.261 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.993538+0000", 2024-09-14T09:08:10.261 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001263655 2024-09-14T09:08:10.261 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:10.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995770+0000", 2024-09-14T09:08:10.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002231949 2024-09-14T09:08:10.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:10.262 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995810+0000", 2024-09-14T09:08:10.262 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0852999999999997e-05 2024-09-14T09:08:10.262 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.262 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.262 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:10.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995979+0000", 2024-09-14T09:08:10.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016866 2024-09-14T09:08:10.263 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.263 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.263 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.263 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996247+0000", 2024-09-14T09:08:10.263 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000267769 2024-09-14T09:08:10.263 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.263 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.263 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.263 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.263 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.263 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:10.264 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-14T09:08:10.264 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:08:10.264 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-14T09:08:10.264 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-14T09:08:10.265 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:10.266+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:10.266 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:08:10.267 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is unset 2024-09-14T09:08:10.366 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:10.370+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:10.448 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:10.448 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:10.448 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:10.449 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:10.465 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:08:10.466 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:10.467+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:10.482 INFO:teuthology.orchestra.run.smithi176.stderr:no valid command found; 10 closest matches: 2024-09-14T09:08:10.483 INFO:teuthology.orchestra.run.smithi176.stderr:0 2024-09-14T09:08:10.483 INFO:teuthology.orchestra.run.smithi176.stderr:1 2024-09-14T09:08:10.483 INFO:teuthology.orchestra.run.smithi176.stderr:2 2024-09-14T09:08:10.483 INFO:teuthology.orchestra.run.smithi176.stderr:abort 2024-09-14T09:08:10.483 INFO:teuthology.orchestra.run.smithi176.stderr:assert 2024-09-14T09:08:10.483 INFO:teuthology.orchestra.run.smithi176.stderr:bench [] [] [] [] 2024-09-14T09:08:10.483 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs debug_inject_read_zeros 2024-09-14T09:08:10.483 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs files list 2024-09-14T09:08:10.483 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs stats 2024-09-14T09:08:10.483 INFO:teuthology.orchestra.run.smithi176.stderr:bluestore allocator dump block 2024-09-14T09:08:10.483 INFO:teuthology.orchestra.run.smithi176.stderr:Error EINVAL: invalid command 2024-09-14T09:08:10.485 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T09:08:10.486 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T09:08:10.486 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-14T09:08:10.579 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:10.580 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:10.580 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:08:10.580 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:08:10.580 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:10.580 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:10.579+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:10.595 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:08:10.682 INFO:teuthology.orchestra.run.smithi176.stderr:no valid command found; 10 closest matches: 2024-09-14T09:08:10.683 INFO:teuthology.orchestra.run.smithi176.stderr:0 2024-09-14T09:08:10.683 INFO:teuthology.orchestra.run.smithi176.stderr:1 2024-09-14T09:08:10.683 INFO:teuthology.orchestra.run.smithi176.stderr:2 2024-09-14T09:08:10.683 INFO:teuthology.orchestra.run.smithi176.stderr:abort 2024-09-14T09:08:10.683 INFO:teuthology.orchestra.run.smithi176.stderr:assert 2024-09-14T09:08:10.683 INFO:teuthology.orchestra.run.smithi176.stderr:bench [] [] [] [] 2024-09-14T09:08:10.683 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs debug_inject_read_zeros 2024-09-14T09:08:10.683 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs files list 2024-09-14T09:08:10.683 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs stats 2024-09-14T09:08:10.683 INFO:teuthology.orchestra.run.smithi176.stderr:bluestore allocator dump block 2024-09-14T09:08:10.683 INFO:teuthology.orchestra.run.smithi176.stderr:Error EINVAL: invalid command 2024-09-14T09:08:10.684 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:10.683+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:10.686 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T09:08:10.686 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T09:08:10.686 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-14T09:08:10.754 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:08:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:08:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:08:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106705 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 518144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.907470406000002, 2024-09-14T09:08:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3001326799999999, 2024-09-14T09:08:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.755 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106705 2024-09-14T09:08:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:10.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4699999999999998e-07 2024-09-14T09:08:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847500+0000", 2024-09-14T09:08:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0860000000000001e-06 2024-09-14T09:08:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847503+0000", 2024-09-14T09:08:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1269999999999999e-06 2024-09-14T09:08:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847506+0000", 2024-09-14T09:08:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5780000000000001e-06 2024-09-14T09:08:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858911+0000", 2024-09-14T09:08:10.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011404925 2024-09-14T09:08:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858919+0000", 2024-09-14T09:08:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8620000000000003e-06 2024-09-14T09:08:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147254+0000", 2024-09-14T09:08:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2883353719999999 2024-09-14T09:08:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147408+0000", 2024-09-14T09:08:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015389200000000001 2024-09-14T09:08:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147631+0000", 2024-09-14T09:08:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022359100000000001 2024-09-14T09:08:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106707 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 522240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.760 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.907459936000002, 2024-09-14T09:08:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.300471801, 2024-09-14T09:08:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106707 2024-09-14T09:08:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.23e-07 2024-09-14T09:08:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847514+0000", 2024-09-14T09:08:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9150000000000004e-06 2024-09-14T09:08:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847520+0000", 2024-09-14T09:08:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.203e-06 2024-09-14T09:08:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847532+0000", 2024-09-14T09:08:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2441e-05 2024-09-14T09:08:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858953+0000", 2024-09-14T09:08:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01142108 2024-09-14T09:08:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858957+0000", 2024-09-14T09:08:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2830000000000003e-06 2024-09-14T09:08:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147667+0000", 2024-09-14T09:08:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2887094619999999 2024-09-14T09:08:10.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147784+0000", 2024-09-14T09:08:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117465 2024-09-14T09:08:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147981+0000", 2024-09-14T09:08:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019672900000000001 2024-09-14T09:08:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106709 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 3072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:08:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.907431645000003, 2024-09-14T09:08:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.301335192, 2024-09-14T09:08:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106709 2024-09-14T09:08:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.766 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:08:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:08:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847538+0000", 2024-09-14T09:08:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2800000000000003e-07 2024-09-14T09:08:10.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847539+0000", 2024-09-14T09:08:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2720000000000001e-06 2024-09-14T09:08:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847544+0000", 2024-09-14T09:08:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7219999999999999e-06 2024-09-14T09:08:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847547+0000", 2024-09-14T09:08:10.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9960000000000002e-06 2024-09-14T09:08:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858991+0000", 2024-09-14T09:08:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011444218000000001 2024-09-14T09:08:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858999+0000", 2024-09-14T09:08:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9419999999999995e-06 2024-09-14T09:08:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148053+0000", 2024-09-14T09:08:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2890544099999999 2024-09-14T09:08:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148203+0000", 2024-09-14T09:08:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149419 2024-09-14T09:08:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148873+0000", 2024-09-14T09:08:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00066988499999999995 2024-09-14T09:08:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.770 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106711 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 7168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.907418387, 2024-09-14T09:08:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3022241219999999, 2024-09-14T09:08:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106711 2024-09-14T09:08:10.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8000000000000002e-07 2024-09-14T09:08:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847552+0000", 2024-09-14T09:08:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1340000000000001e-06 2024-09-14T09:08:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847557+0000", 2024-09-14T09:08:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6179999999999997e-06 2024-09-14T09:08:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847560+0000", 2024-09-14T09:08:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5899999999999999e-06 2024-09-14T09:08:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859050+0000", 2024-09-14T09:08:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01148914 2024-09-14T09:08:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859056+0000", 2024-09-14T09:08:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6630000000000004e-06 2024-09-14T09:08:10.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148948+0000", 2024-09-14T09:08:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2898914370000001 2024-09-14T09:08:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149079+0000", 2024-09-14T09:08:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013145400000000001 2024-09-14T09:08:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149775+0000", 2024-09-14T09:08:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00069580599999999999 2024-09-14T09:08:10.775 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106713 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 11264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.907404921999998, 2024-09-14T09:08:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3035188959999999, 2024-09-14T09:08:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.776 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106713 2024-09-14T09:08:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6e-07 2024-09-14T09:08:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847566+0000", 2024-09-14T09:08:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1039999999999999e-06 2024-09-14T09:08:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847569+0000", 2024-09-14T09:08:10.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.191e-06 2024-09-14T09:08:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847571+0000", 2024-09-14T09:08:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6390000000000001e-06 2024-09-14T09:08:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859099+0000", 2024-09-14T09:08:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011527379000000001 2024-09-14T09:08:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859106+0000", 2024-09-14T09:08:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3679999999999996e-06 2024-09-14T09:08:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149878+0000", 2024-09-14T09:08:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.29077208 2024-09-14T09:08:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.150175+0000", 2024-09-14T09:08:10.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029685699999999998 2024-09-14T09:08:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151083+0000", 2024-09-14T09:08:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00090801800000000002 2024-09-14T09:08:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106715 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 15360~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.907393913, 2024-09-14T09:08:10.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3047478800000001, 2024-09-14T09:08:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106715 2024-09-14T09:08:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6e-07 2024-09-14T09:08:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847580+0000", 2024-09-14T09:08:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9250000000000003e-06 2024-09-14T09:08:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847585+0000", 2024-09-14T09:08:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8409999999999999e-06 2024-09-14T09:08:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847588+0000", 2024-09-14T09:08:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6989999999999999e-06 2024-09-14T09:08:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859151+0000", 2024-09-14T09:08:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011562604000000001 2024-09-14T09:08:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859159+0000", 2024-09-14T09:08:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4179999999999997e-06 2024-09-14T09:08:10.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151155+0000", 2024-09-14T09:08:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.291996068 2024-09-14T09:08:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151383+0000", 2024-09-14T09:08:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022746100000000001 2024-09-14T09:08:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152323+0000", 2024-09-14T09:08:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00094060400000000005 2024-09-14T09:08:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106717 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 19456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.907377224999998, 2024-09-14T09:08:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.305574579, 2024-09-14T09:08:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.787 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106717 2024-09-14T09:08:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1199999999999999e-07 2024-09-14T09:08:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847593+0000", 2024-09-14T09:08:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.257e-06 2024-09-14T09:08:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847598+0000", 2024-09-14T09:08:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4139999999999996e-06 2024-09-14T09:08:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847602+0000", 2024-09-14T09:08:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3440000000000003e-06 2024-09-14T09:08:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859203+0000", 2024-09-14T09:08:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011600572999999999 2024-09-14T09:08:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859211+0000", 2024-09-14T09:08:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-06 2024-09-14T09:08:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152398+0000", 2024-09-14T09:08:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.293187812 2024-09-14T09:08:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152543+0000", 2024-09-14T09:08:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014478000000000001 2024-09-14T09:08:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153166+0000", 2024-09-14T09:08:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000623257 2024-09-14T09:08:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.792 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106719 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 23552~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.907362909, 2024-09-14T09:08:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.306394982, 2024-09-14T09:08:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106719 2024-09-14T09:08:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5100000000000001e-07 2024-09-14T09:08:10.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847608+0000", 2024-09-14T09:08:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.189e-06 2024-09-14T09:08:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847612+0000", 2024-09-14T09:08:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9519999999999996e-06 2024-09-14T09:08:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847614+0000", 2024-09-14T09:08:10.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6280000000000001e-06 2024-09-14T09:08:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859254+0000", 2024-09-14T09:08:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011640157 2024-09-14T09:08:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859261+0000", 2024-09-14T09:08:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0570000000000004e-06 2024-09-14T09:08:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153236+0000", 2024-09-14T09:08:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2939744040000001 2024-09-14T09:08:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153380+0000", 2024-09-14T09:08:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014410399999999999 2024-09-14T09:08:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154001+0000", 2024-09-14T09:08:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062124000000000001 2024-09-14T09:08:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106721 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 27648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.907351259000002, 2024-09-14T09:08:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3074446659999999, 2024-09-14T09:08:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106721 2024-09-14T09:08:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4299999999999999e-07 2024-09-14T09:08:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847619+0000", 2024-09-14T09:08:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T09:08:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847624+0000", 2024-09-14T09:08:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4229999999999998e-06 2024-09-14T09:08:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847627+0000", 2024-09-14T09:08:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0120000000000001e-06 2024-09-14T09:08:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859305+0000", 2024-09-14T09:08:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01167812 2024-09-14T09:08:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859313+0000", 2024-09-14T09:08:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2409999999999995e-06 2024-09-14T09:08:10.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154081+0000", 2024-09-14T09:08:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.294768296 2024-09-14T09:08:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154280+0000", 2024-09-14T09:08:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019824 2024-09-14T09:08:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155063+0000", 2024-09-14T09:08:10.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00078303400000000005 2024-09-14T09:08:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106723 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 31744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:08:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.907338828999997, 2024-09-14T09:08:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308293648, 2024-09-14T09:08:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.803 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106723 2024-09-14T09:08:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:08:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:08:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847631+0000", 2024-09-14T09:08:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6100000000000002e-07 2024-09-14T09:08:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847632+0000", 2024-09-14T09:08:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0589999999999999e-06 2024-09-14T09:08:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847635+0000", 2024-09-14T09:08:10.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.551e-06 2024-09-14T09:08:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847638+0000", 2024-09-14T09:08:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.644e-06 2024-09-14T09:08:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859344+0000", 2024-09-14T09:08:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011706397 2024-09-14T09:08:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859348+0000", 2024-09-14T09:08:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5779999999999999e-06 2024-09-14T09:08:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155135+0000", 2024-09-14T09:08:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.295787448 2024-09-14T09:08:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155279+0000", 2024-09-14T09:08:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014400700000000001 2024-09-14T09:08:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155924+0000", 2024-09-14T09:08:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00064470300000000005 2024-09-14T09:08:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106725 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 35840~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:08:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.907327717999998, 2024-09-14T09:08:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.309103218, 2024-09-14T09:08:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.808 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106725 2024-09-14T09:08:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:08:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:08:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847642+0000", 2024-09-14T09:08:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4499999999999998e-07 2024-09-14T09:08:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847643+0000", 2024-09-14T09:08:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4e-07 2024-09-14T09:08:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847647+0000", 2024-09-14T09:08:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6190000000000002e-06 2024-09-14T09:08:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847655+0000", 2024-09-14T09:08:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:08:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859381+0000", 2024-09-14T09:08:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011725793 2024-09-14T09:08:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859391+0000", 2024-09-14T09:08:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4390000000000006e-06 2024-09-14T09:08:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155997+0000", 2024-09-14T09:08:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2966067830000001 2024-09-14T09:08:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156133+0000", 2024-09-14T09:08:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000135609 2024-09-14T09:08:10.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156745+0000", 2024-09-14T09:08:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000611592 2024-09-14T09:08:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106727 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 39936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.907309417999997, 2024-09-14T09:08:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3099046560000001, 2024-09-14T09:08:10.813 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.814 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.814 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.814 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.814 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.814 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.814 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106727 2024-09-14T09:08:10.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.814 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:10.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:10.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:10.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1300000000000001e-07 2024-09-14T09:08:10.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847661+0000", 2024-09-14T09:08:10.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.192e-06 2024-09-14T09:08:10.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847665+0000", 2024-09-14T09:08:10.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0999999999999997e-06 2024-09-14T09:08:10.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847668+0000", 2024-09-14T09:08:10.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.644e-06 2024-09-14T09:08:10.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859422+0000", 2024-09-14T09:08:10.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011753559 2024-09-14T09:08:10.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859425+0000", 2024-09-14T09:08:10.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.343e-06 2024-09-14T09:08:10.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156817+0000", 2024-09-14T09:08:10.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2973924800000001 2024-09-14T09:08:10.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156956+0000", 2024-09-14T09:08:10.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013835 2024-09-14T09:08:10.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157564+0000", 2024-09-14T09:08:10.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060867500000000004 2024-09-14T09:08:10.818 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.818 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.818 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.818 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106729 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 44032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.818 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:10.818 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.907297444999998, 2024-09-14T09:08:10.819 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3107195970000001, 2024-09-14T09:08:10.819 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.819 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.819 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.819 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.819 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.819 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.819 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106729 2024-09-14T09:08:10.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.819 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.819 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.819 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:10.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:10.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.820 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:10.820 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.29e-07 2024-09-14T09:08:10.820 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.820 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.820 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847673+0000", 2024-09-14T09:08:10.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.096e-06 2024-09-14T09:08:10.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847678+0000", 2024-09-14T09:08:10.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.527e-06 2024-09-14T09:08:10.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.821 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.821 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847681+0000", 2024-09-14T09:08:10.821 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.788e-06 2024-09-14T09:08:10.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.821 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859462+0000", 2024-09-14T09:08:10.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01178032 2024-09-14T09:08:10.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859471+0000", 2024-09-14T09:08:10.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3430000000000002e-06 2024-09-14T09:08:10.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.822 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.822 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.822 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157635+0000", 2024-09-14T09:08:10.822 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2981640320000001 2024-09-14T09:08:10.822 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157773+0000", 2024-09-14T09:08:10.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763 2024-09-14T09:08:10.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.823 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.823 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158391+0000", 2024-09-14T09:08:10.823 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061863200000000004 2024-09-14T09:08:10.823 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.823 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.823 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.823 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.824 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106731 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 48128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.824 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:08:10.824 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.9072818, 2024-09-14T09:08:10.824 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3115142020000001, 2024-09-14T09:08:10.824 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.824 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.824 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.824 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.824 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.824 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.824 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106731 2024-09-14T09:08:10.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.824 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:08:10.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:08:10.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.825 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.825 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.825 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847688+0000", 2024-09-14T09:08:10.825 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0199999999999998e-07 2024-09-14T09:08:10.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847689+0000", 2024-09-14T09:08:10.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4529999999999999e-06 2024-09-14T09:08:10.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847694+0000", 2024-09-14T09:08:10.826 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9080000000000003e-06 2024-09-14T09:08:10.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.826 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.826 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847697+0000", 2024-09-14T09:08:10.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4920000000000002e-06 2024-09-14T09:08:10.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859499+0000", 2024-09-14T09:08:10.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011801676000000001 2024-09-14T09:08:10.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.827 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859502+0000", 2024-09-14T09:08:10.827 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2899999999999998e-06 2024-09-14T09:08:10.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.827 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158461+0000", 2024-09-14T09:08:10.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.298958796 2024-09-14T09:08:10.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158594+0000", 2024-09-14T09:08:10.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013321599999999999 2024-09-14T09:08:10.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.828 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.828 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159202+0000", 2024-09-14T09:08:10.828 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060706900000000001 2024-09-14T09:08:10.828 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.829 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.829 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.829 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106733 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 52224~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.829 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:10.829 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.907267544, 2024-09-14T09:08:10.829 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3123343809999999, 2024-09-14T09:08:10.829 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.829 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.829 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.829 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.829 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.829 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.829 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106733 2024-09-14T09:08:10.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.830 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:10.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.830 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:10.830 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.830 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:10.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5800000000000001e-07 2024-09-14T09:08:10.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847703+0000", 2024-09-14T09:08:10.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.127e-06 2024-09-14T09:08:10.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.831 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.831 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847706+0000", 2024-09-14T09:08:10.831 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3210000000000001e-06 2024-09-14T09:08:10.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847709+0000", 2024-09-14T09:08:10.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7539999999999998e-06 2024-09-14T09:08:10.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859538+0000", 2024-09-14T09:08:10.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01182942 2024-09-14T09:08:10.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.832 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.832 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859547+0000", 2024-09-14T09:08:10.832 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T09:08:10.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159273+0000", 2024-09-14T09:08:10.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2997263700000001 2024-09-14T09:08:10.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159415+0000", 2024-09-14T09:08:10.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000141765 2024-09-14T09:08:10.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.833 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.833 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160036+0000", 2024-09-14T09:08:10.833 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062122100000000003 2024-09-14T09:08:10.834 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.834 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.834 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.834 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106735 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 56320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.834 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:10.834 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.907256099999998, 2024-09-14T09:08:10.834 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3131339309999999, 2024-09-14T09:08:10.834 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.834 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.834 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.834 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.834 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.835 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.835 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106735 2024-09-14T09:08:10.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.835 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:10.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.835 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.835 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:10.835 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:10.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2100000000000001e-07 2024-09-14T09:08:10.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847714+0000", 2024-09-14T09:08:10.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.051e-06 2024-09-14T09:08:10.836 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.836 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.836 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.836 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847719+0000", 2024-09-14T09:08:10.836 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3640000000000001e-06 2024-09-14T09:08:10.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847723+0000", 2024-09-14T09:08:10.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8789999999999996e-06 2024-09-14T09:08:10.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.837 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859577+0000", 2024-09-14T09:08:10.837 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011854416 2024-09-14T09:08:10.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.837 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859580+0000", 2024-09-14T09:08:10.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.292e-06 2024-09-14T09:08:10.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160108+0000", 2024-09-14T09:08:10.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.30052758 2024-09-14T09:08:10.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.838 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.838 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160238+0000", 2024-09-14T09:08:10.838 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013043500000000001 2024-09-14T09:08:10.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.839 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.839 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160847+0000", 2024-09-14T09:08:10.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060869299999999999 2024-09-14T09:08:10.839 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.839 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.839 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.839 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106738 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 60416~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.839 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:10.839 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.907242896, 2024-09-14T09:08:10.839 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3139482179999999, 2024-09-14T09:08:10.839 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.839 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.840 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.840 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.840 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.840 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.840 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106738 2024-09-14T09:08:10.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.840 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:10.840 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.840 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.840 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:10.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:10.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3999999999999998e-07 2024-09-14T09:08:10.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.841 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.841 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847733+0000", 2024-09-14T09:08:10.841 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8070000000000002e-06 2024-09-14T09:08:10.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847738+0000", 2024-09-14T09:08:10.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9740000000000001e-06 2024-09-14T09:08:10.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847742+0000", 2024-09-14T09:08:10.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6009999999999999e-06 2024-09-14T09:08:10.842 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.842 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.842 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859612+0000", 2024-09-14T09:08:10.842 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011870537 2024-09-14T09:08:10.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859621+0000", 2024-09-14T09:08:10.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3019999999999995e-06 2024-09-14T09:08:10.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160917+0000", 2024-09-14T09:08:10.843 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3012961080000001 2024-09-14T09:08:10.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.843 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.843 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161054+0000", 2024-09-14T09:08:10.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000137422 2024-09-14T09:08:10.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.844 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.844 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161674+0000", 2024-09-14T09:08:10.844 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000620227 2024-09-14T09:08:10.844 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.844 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.844 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.844 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106739 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 64512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.844 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:10.845 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.907223055999999, 2024-09-14T09:08:10.845 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.314753777, 2024-09-14T09:08:10.845 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.845 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.845 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.845 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.845 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.845 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.845 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106739 2024-09-14T09:08:10.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.845 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.845 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.845 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:10.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:10.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:10.846 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9700000000000003e-07 2024-09-14T09:08:10.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.846 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.846 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847748+0000", 2024-09-14T09:08:10.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2589999999999999e-06 2024-09-14T09:08:10.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847752+0000", 2024-09-14T09:08:10.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6099999999999999e-06 2024-09-14T09:08:10.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.847 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847755+0000", 2024-09-14T09:08:10.847 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0520000000000002e-06 2024-09-14T09:08:10.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.847 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859653+0000", 2024-09-14T09:08:10.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011897575000000001 2024-09-14T09:08:10.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859658+0000", 2024-09-14T09:08:10.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8810000000000004e-06 2024-09-14T09:08:10.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.848 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.848 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161745+0000", 2024-09-14T09:08:10.848 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3020870819999999 2024-09-14T09:08:10.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161877+0000", 2024-09-14T09:08:10.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000132657 2024-09-14T09:08:10.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.849 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.849 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162500+0000", 2024-09-14T09:08:10.849 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062236400000000003 2024-09-14T09:08:10.849 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.849 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.849 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.849 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106741 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 68608~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.850 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:10.850 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.907210264, 2024-09-14T09:08:10.850 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3155568449999999, 2024-09-14T09:08:10.850 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.850 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.850 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.850 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.850 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.850 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.850 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106741 2024-09-14T09:08:10.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.850 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.850 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:10.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:10.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.851 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.851 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:10.851 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5800000000000001e-07 2024-09-14T09:08:10.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847760+0000", 2024-09-14T09:08:10.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:08:10.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847764+0000", 2024-09-14T09:08:10.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3189999999999999e-06 2024-09-14T09:08:10.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.852 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.852 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847766+0000", 2024-09-14T09:08:10.852 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6429999999999999e-06 2024-09-14T09:08:10.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859693+0000", 2024-09-14T09:08:10.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011926724 2024-09-14T09:08:10.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:10.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859703+0000", 2024-09-14T09:08:10.853 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6160000000000008e-06 2024-09-14T09:08:10.853 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.853 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.853 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.853 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162575+0000", 2024-09-14T09:08:10.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.302872128 2024-09-14T09:08:10.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162709+0000", 2024-09-14T09:08:10.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013446000000000001 2024-09-14T09:08:10.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.854 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.854 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163316+0000", 2024-09-14T09:08:10.854 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060650700000000005 2024-09-14T09:08:10.854 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.854 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.854 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.855 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106744 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 76800~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:10.855 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:08:10.855 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 48.907199153000001, 2024-09-14T09:08:10.855 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3163901789999999, 2024-09-14T09:08:10.855 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:10.855 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:10.855 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:10.855 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:10.855 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:10.855 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:10.855 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106744 2024-09-14T09:08:10.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.855 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:10.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:10.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:08:10.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:10.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:08:10.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:10.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.856 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:10.856 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996947+0000", 2024-09-14T09:08:10.856 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.149177154 2024-09-14T09:08:10.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:10.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996964+0000", 2024-09-14T09:08:10.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.679e-05 2024-09-14T09:08:10.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:10.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996999+0000", 2024-09-14T09:08:10.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5463000000000002e-05 2024-09-14T09:08:10.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.857 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:10.857 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.997029+0000", 2024-09-14T09:08:10.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9567e-05 2024-09-14T09:08:10.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:10.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163390+0000", 2024-09-14T09:08:10.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.16636056799999999 2024-09-14T09:08:10.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:10.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163526+0000", 2024-09-14T09:08:10.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013655899999999999 2024-09-14T09:08:10.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:10.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:10.858 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:10.858 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.164160+0000", 2024-09-14T09:08:10.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00063407800000000005 2024-09-14T09:08:10.859 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.859 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.859 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.859 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:10.859 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:10.859 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:10.860 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:08:10.861 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:10.859+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:10.885 INFO:teuthology.orchestra.run.smithi176.stderr:no valid command found; 10 closest matches: 2024-09-14T09:08:10.885 INFO:teuthology.orchestra.run.smithi176.stderr:0 2024-09-14T09:08:10.886 INFO:teuthology.orchestra.run.smithi176.stderr:1 2024-09-14T09:08:10.886 INFO:teuthology.orchestra.run.smithi176.stderr:2 2024-09-14T09:08:10.886 INFO:teuthology.orchestra.run.smithi176.stderr:abort 2024-09-14T09:08:10.886 INFO:teuthology.orchestra.run.smithi176.stderr:assert 2024-09-14T09:08:10.886 INFO:teuthology.orchestra.run.smithi176.stderr:bench [] [] [] [] 2024-09-14T09:08:10.886 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs debug_inject_read_zeros 2024-09-14T09:08:10.886 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs files list 2024-09-14T09:08:10.886 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs stats 2024-09-14T09:08:10.886 INFO:teuthology.orchestra.run.smithi176.stderr:bluestore allocator dump block 2024-09-14T09:08:10.886 INFO:teuthology.orchestra.run.smithi176.stderr:Error EINVAL: invalid command 2024-09-14T09:08:10.891 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T09:08:10.891 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T09:08:10.892 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-14T09:08:10.982 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:10.983+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:11.002 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:11.002 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:11.002 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:11.002 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:11.017 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:08:11.066 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:11.067+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:11.082 INFO:teuthology.orchestra.run.smithi176.stderr:no valid command found; 10 closest matches: 2024-09-14T09:08:11.082 INFO:teuthology.orchestra.run.smithi176.stderr:0 2024-09-14T09:08:11.082 INFO:teuthology.orchestra.run.smithi176.stderr:1 2024-09-14T09:08:11.082 INFO:teuthology.orchestra.run.smithi176.stderr:2 2024-09-14T09:08:11.082 INFO:teuthology.orchestra.run.smithi176.stderr:abort 2024-09-14T09:08:11.083 INFO:teuthology.orchestra.run.smithi176.stderr:assert 2024-09-14T09:08:11.083 INFO:teuthology.orchestra.run.smithi176.stderr:bench [] [] [] [] 2024-09-14T09:08:11.083 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs debug_inject_read_zeros 2024-09-14T09:08:11.083 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs files list 2024-09-14T09:08:11.083 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs stats 2024-09-14T09:08:11.083 INFO:teuthology.orchestra.run.smithi176.stderr:bluestore allocator dump block 2024-09-14T09:08:11.083 INFO:teuthology.orchestra.run.smithi176.stderr:Error EINVAL: invalid command 2024-09-14T09:08:11.085 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T09:08:11.085 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T09:08:11.086 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-14T09:08:11.177 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:11.177 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:11.177 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:08:11.177 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:08:11.178 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:11.178 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:11.179+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:11.191 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:08:11.259 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:08:11.262 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:08:11.262 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T09:08:11.262 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-14T09:08:11.306 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:11.307+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:11.364 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:11.363+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:11.372 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:11.372 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:08:11.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:08:11.372 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:08:11.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.372 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:11.373 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:08:11.373 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 158.288825882, 2024-09-14T09:08:11.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:08:11.373 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.373 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.373 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:11.373 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:11.373 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:11.373 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.373 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:08:11.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.374 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:08:11.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:08:11.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:08:11.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:08:11.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:08:11.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:08:11.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:08:11.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:08:11.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:08:11.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:08:11.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:08:11.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:08:11.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:11.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:08:11.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:08:11.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:08:11.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:08:11.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:08:11.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:08:11.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:08:11.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:08:11.378 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.379 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.379 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.379 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:11.379 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:08:11.379 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 158.28877122399999, 2024-09-14T09:08:11.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:08:11.379 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.379 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.379 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:11.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:11.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:11.380 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.380 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:08:11.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.380 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:08:11.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:08:11.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:08:11.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:08:11.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:08:11.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:08:11.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:08:11.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:08:11.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:08:11.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:08:11.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:08:11.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:08:11.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.383 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:11.383 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:08:11.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:08:11.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:08:11.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:08:11.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:08:11.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:08:11.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:08:11.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:08:11.385 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.385 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.385 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.385 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:11.385 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:08:11.385 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 158.28866585, 2024-09-14T09:08:11.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:08:11.385 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.385 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.385 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:11.386 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:11.386 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:11.386 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.386 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:08:11.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.386 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:08:11.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:08:11.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:08:11.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:08:11.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:08:11.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:08:11.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:08:11.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:08:11.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.388 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.388 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:08:11.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:08:11.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:08:11.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:08:11.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:11.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:08:11.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:08:11.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:08:11.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:08:11.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:08:11.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:08:11.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:08:11.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:08:11.390 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.390 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.391 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.391 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:11.391 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:08:11.391 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 158.288309086, 2024-09-14T09:08:11.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:08:11.391 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.391 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.391 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:11.391 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:11.392 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:11.392 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.392 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:08:11.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.392 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:08:11.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:08:11.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:08:11.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:08:11.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:08:11.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:08:11.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.393 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:08:11.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:08:11.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:08:11.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:08:11.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:08:11.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:08:11.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:11.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:08:11.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:08:11.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:08:11.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:08:11.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:08:11.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:08:11.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:08:11.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:08:11.396 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.396 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.396 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.396 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:11.396 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:08:11.396 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 158.288224348, 2024-09-14T09:08:11.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:08:11.396 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.396 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.397 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:11.397 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:11.397 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:11.397 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.397 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:08:11.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.397 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:08:11.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:08:11.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:08:11.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:08:11.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.398 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:08:11.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:08:11.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:08:11.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:08:11.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:08:11.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:08:11.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:08:11.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:08:11.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:11.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:08:11.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:08:11.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:08:11.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:08:11.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:08:11.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:08:11.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:08:11.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:08:11.401 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.401 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.401 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.401 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:11.401 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:08:11.401 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 158.288136662, 2024-09-14T09:08:11.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:08:11.402 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.402 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.402 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:11.402 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:11.402 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:11.402 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.402 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:08:11.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.402 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:08:11.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:08:11.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:08:11.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:08:11.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.403 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:08:11.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:08:11.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:08:11.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:08:11.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:08:11.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:08:11.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:08:11.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:08:11.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:11.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:08:11.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:08:11.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:08:11.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:08:11.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:08:11.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:08:11.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:08:11.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:08:11.406 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.406 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.406 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.406 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:08:11.407 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:08:11.407 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 125.21184467499999, 2024-09-14T09:08:11.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:08:11.407 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.407 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.407 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:11.407 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:11.407 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:11.407 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.407 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:08:11.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.407 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:08:11.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.408 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.408 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:08:11.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:08:11.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:08:11.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:08:11.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:08:11.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:08:11.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:08:11.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:08:11.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:08:11.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:08:11.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:08:11.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:08:11.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:08:11.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:08:11.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:08:11.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:08:11.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:11.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:08:11.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:08:11.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:11.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:08:11.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:08:11.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:11.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:08:11.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:08:11.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:08:11.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:08:11.413 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.413 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.413 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.413 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:97796 3.15 3:aaaa5f7f:::rbd_data.115e113dc8ca.0000000000000029:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:08:11.413 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:08:11.413 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 59.277975804999997, 2024-09-14T09:08:11.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14537731300000001, 2024-09-14T09:08:11.413 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.413 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.413 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:11.413 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:11.413 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:11.414 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.414 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97796 2024-09-14T09:08:11.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.414 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:08:11.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:08:11.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094482+0000", 2024-09-14T09:08:11.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6400000000000001e-07 2024-09-14T09:08:11.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094485+0000", 2024-09-14T09:08:11.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.693e-06 2024-09-14T09:08:11.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094494+0000", 2024-09-14T09:08:11.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9590000000000001e-06 2024-09-14T09:08:11.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094502+0000", 2024-09-14T09:08:11.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3180000000000001e-06 2024-09-14T09:08:11.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094567+0000", 2024-09-14T09:08:11.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5526999999999998e-05 2024-09-14T09:08:11.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094726+0000", 2024-09-14T09:08:11.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015842499999999999 2024-09-14T09:08:11.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:08:11.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.101940+0000", 2024-09-14T09:08:11.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0072146559999999998 2024-09-14T09:08:11.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:11.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239665+0000", 2024-09-14T09:08:11.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.137724501 2024-09-14T09:08:11.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:11.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239726+0000", 2024-09-14T09:08:11.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1550999999999994e-05 2024-09-14T09:08:11.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:11.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239800+0000", 2024-09-14T09:08:11.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3318000000000005e-05 2024-09-14T09:08:11.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.418 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.418 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239858+0000", 2024-09-14T09:08:11.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8600999999999997e-05 2024-09-14T09:08:11.419 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.419 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.419 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.419 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:97796 3.15 e106/103 ondisk, result = 0)", 2024-09-14T09:08:11.419 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:08:11.419 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 59.255629935999998, 2024-09-14T09:08:11.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123043465, 2024-09-14T09:08:11.419 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.419 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.420 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:11.420 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:08:11.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:08:11.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116828+0000", 2024-09-14T09:08:11.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8100000000000002e-07 2024-09-14T09:08:11.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116831+0000", 2024-09-14T09:08:11.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3009999999999998e-06 2024-09-14T09:08:11.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116841+0000", 2024-09-14T09:08:11.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7829999999999993e-06 2024-09-14T09:08:11.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116851+0000", 2024-09-14T09:08:11.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0135999999999999e-05 2024-09-14T09:08:11.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239707+0000", 2024-09-14T09:08:11.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12285581499999999 2024-09-14T09:08:11.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239718+0000", 2024-09-14T09:08:11.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1804000000000001e-05 2024-09-14T09:08:11.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239870+0000", 2024-09-14T09:08:11.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015194500000000001 2024-09-14T09:08:11.424 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.424 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.424 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.424 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117315 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 309248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:11.424 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:08:11.424 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.648994199000001, 2024-09-14T09:08:11.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11247383800000001, 2024-09-14T09:08:11.424 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.424 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.424 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:11.425 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:11.425 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:11.425 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.425 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117315 2024-09-14T09:08:11.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.425 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:08:11.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:08:11.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723899+0000", 2024-09-14T09:08:11.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043641699999999998 2024-09-14T09:08:11.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723903+0000", 2024-09-14T09:08:11.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5389999999999999e-06 2024-09-14T09:08:11.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723910+0000", 2024-09-14T09:08:11.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4229999999999999e-06 2024-09-14T09:08:11.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723917+0000", 2024-09-14T09:08:11.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4560000000000003e-06 2024-09-14T09:08:11.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835514+0000", 2024-09-14T09:08:11.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111596258 2024-09-14T09:08:11.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835571+0000", 2024-09-14T09:08:11.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7317e-05 2024-09-14T09:08:11.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.428 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.428 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835936+0000", 2024-09-14T09:08:11.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000365428 2024-09-14T09:08:11.428 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.428 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.428 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.428 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117317 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 313344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:11.429 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:08:11.429 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.648527182999999, 2024-09-14T09:08:11.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112450201, 2024-09-14T09:08:11.429 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.429 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.429 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:11.429 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:11.429 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:11.429 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.429 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117317 2024-09-14T09:08:11.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.429 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:08:11.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:08:11.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724396+0000", 2024-09-14T09:08:11.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000466028 2024-09-14T09:08:11.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724401+0000", 2024-09-14T09:08:11.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2299999999999999e-06 2024-09-14T09:08:11.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724412+0000", 2024-09-14T09:08:11.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0762e-05 2024-09-14T09:08:11.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724419+0000", 2024-09-14T09:08:11.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1010000000000003e-06 2024-09-14T09:08:11.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835981+0000", 2024-09-14T09:08:11.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111562174 2024-09-14T09:08:11.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836070+0000", 2024-09-14T09:08:11.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8839999999999996e-05 2024-09-14T09:08:11.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836380+0000", 2024-09-14T09:08:11.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031006600000000001 2024-09-14T09:08:11.433 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.433 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.433 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.433 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117319 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 317440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:11.433 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:08:11.433 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.648022546999997, 2024-09-14T09:08:11.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112467763, 2024-09-14T09:08:11.433 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.433 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.434 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:11.434 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:11.434 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:11.434 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.434 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117319 2024-09-14T09:08:11.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.434 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:08:11.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:08:11.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724846+0000", 2024-09-14T09:08:11.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00041155400000000001 2024-09-14T09:08:11.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724854+0000", 2024-09-14T09:08:11.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5469999999999997e-06 2024-09-14T09:08:11.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724862+0000", 2024-09-14T09:08:11.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1289999999999996e-06 2024-09-14T09:08:11.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724867+0000", 2024-09-14T09:08:11.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8720000000000001e-06 2024-09-14T09:08:11.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836486+0000", 2024-09-14T09:08:11.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111618357 2024-09-14T09:08:11.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836554+0000", 2024-09-14T09:08:11.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8378000000000002e-05 2024-09-14T09:08:11.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836902+0000", 2024-09-14T09:08:11.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034792599999999998 2024-09-14T09:08:11.438 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.438 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.439 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.439 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117321 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 321536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:11.439 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:08:11.439 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.647578222999996, 2024-09-14T09:08:11.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11249685600000001, 2024-09-14T09:08:11.439 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.439 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.439 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:11.439 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:11.439 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:11.439 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.439 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117321 2024-09-14T09:08:11.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.440 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:08:11.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:08:11.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725241+0000", 2024-09-14T09:08:11.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036211099999999998 2024-09-14T09:08:11.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725245+0000", 2024-09-14T09:08:11.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9179999999999996e-06 2024-09-14T09:08:11.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725254+0000", 2024-09-14T09:08:11.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0699999999999996e-06 2024-09-14T09:08:11.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725260+0000", 2024-09-14T09:08:11.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1099999999999999e-06 2024-09-14T09:08:11.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836935+0000", 2024-09-14T09:08:11.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111674997 2024-09-14T09:08:11.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837021+0000", 2024-09-14T09:08:11.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5931000000000001e-05 2024-09-14T09:08:11.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.443 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.443 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837375+0000", 2024-09-14T09:08:11.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035471899999999998 2024-09-14T09:08:11.443 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.443 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.443 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.443 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.443 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.443 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117323 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:11.443 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:08:11.443 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.647183998000003, 2024-09-14T09:08:11.443 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11250631, 2024-09-14T09:08:11.444 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.444 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.444 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:11.444 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:11.444 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:11.444 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.444 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117323 2024-09-14T09:08:11.444 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.444 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.444 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.444 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.444 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:08:11.444 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:08:11.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725941+0000", 2024-09-14T09:08:11.445 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00066866500000000004 2024-09-14T09:08:11.445 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.445 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.445 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.445 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725946+0000", 2024-09-14T09:08:11.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9060000000000001e-06 2024-09-14T09:08:11.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725959+0000", 2024-09-14T09:08:11.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2622e-05 2024-09-14T09:08:11.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.446 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725968+0000", 2024-09-14T09:08:11.446 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5660000000000007e-06 2024-09-14T09:08:11.446 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.446 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.446 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837408+0000", 2024-09-14T09:08:11.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111440626 2024-09-14T09:08:11.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837495+0000", 2024-09-14T09:08:11.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6654000000000001e-05 2024-09-14T09:08:11.447 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.447 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.447 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.447 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837779+0000", 2024-09-14T09:08:11.447 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000284271 2024-09-14T09:08:11.447 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.447 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.448 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.448 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.448 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.448 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117327 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 333824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:11.448 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:08:11.448 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.646447805999998, 2024-09-14T09:08:11.448 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112409908, 2024-09-14T09:08:11.448 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.448 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.448 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:11.448 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:11.448 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:11.449 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.449 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117327 2024-09-14T09:08:11.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.449 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.449 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.449 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:08:11.449 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.449 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.449 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:08:11.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726303+0000", 2024-09-14T09:08:11.450 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029388199999999998 2024-09-14T09:08:11.450 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.450 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.450 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.450 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726307+0000", 2024-09-14T09:08:11.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4320000000000001e-06 2024-09-14T09:08:11.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726317+0000", 2024-09-14T09:08:11.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4609999999999997e-06 2024-09-14T09:08:11.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.451 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.451 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.451 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726324+0000", 2024-09-14T09:08:11.451 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0480000000000001e-06 2024-09-14T09:08:11.451 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838051+0000", 2024-09-14T09:08:11.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111727534 2024-09-14T09:08:11.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.452 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.452 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.452 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838143+0000", 2024-09-14T09:08:11.452 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1359000000000004e-05 2024-09-14T09:08:11.452 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.453 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.453 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838419+0000", 2024-09-14T09:08:11.453 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027619199999999997 2024-09-14T09:08:11.453 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.453 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.453 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.453 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.453 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.453 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117329 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 337920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:11.453 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:08:11.454 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.646121047000001, 2024-09-14T09:08:11.454 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112411122, 2024-09-14T09:08:11.454 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.454 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.454 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:11.454 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:11.454 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:11.454 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.454 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117329 2024-09-14T09:08:11.454 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.455 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:08:11.455 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.455 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.455 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.455 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:08:11.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726742+0000", 2024-09-14T09:08:11.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040626099999999999 2024-09-14T09:08:11.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.456 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.456 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726748+0000", 2024-09-14T09:08:11.456 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6860000000000003e-06 2024-09-14T09:08:11.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.456 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726761+0000", 2024-09-14T09:08:11.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3599e-05 2024-09-14T09:08:11.457 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.457 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.457 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.457 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726770+0000", 2024-09-14T09:08:11.457 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5990000000000002e-06 2024-09-14T09:08:11.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838443+0000", 2024-09-14T09:08:11.458 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111672664 2024-09-14T09:08:11.458 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.458 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.458 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.458 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838487+0000", 2024-09-14T09:08:11.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4249000000000003e-05 2024-09-14T09:08:11.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.459 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.459 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838747+0000", 2024-09-14T09:08:11.459 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026006400000000001 2024-09-14T09:08:11.459 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.459 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.459 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.459 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.459 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.459 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117331 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 342016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:08:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.645653451999998, 2024-09-14T09:08:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112291139, 2024-09-14T09:08:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117331 2024-09-14T09:08:11.460 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:08:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:08:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727158+0000", 2024-09-14T09:08:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035434000000000003 2024-09-14T09:08:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727162+0000", 2024-09-14T09:08:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5449999999999997e-06 2024-09-14T09:08:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.462 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727174+0000", 2024-09-14T09:08:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2278e-05 2024-09-14T09:08:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727183+0000", 2024-09-14T09:08:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3820000000000004e-06 2024-09-14T09:08:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.463 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838788+0000", 2024-09-14T09:08:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111604748 2024-09-14T09:08:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838847+0000", 2024-09-14T09:08:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8983000000000001e-05 2024-09-14T09:08:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.464 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839094+0000", 2024-09-14T09:08:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024786300000000002 2024-09-14T09:08:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117333 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 346112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:08:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.645259641000003, 2024-09-14T09:08:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112319711, 2024-09-14T09:08:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.465 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117333 2024-09-14T09:08:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:08:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.466 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:08:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727785+0000", 2024-09-14T09:08:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000587512 2024-09-14T09:08:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727789+0000", 2024-09-14T09:08:11.467 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.352e-06 2024-09-14T09:08:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727800+0000", 2024-09-14T09:08:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1369000000000001e-05 2024-09-14T09:08:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727807+0000", 2024-09-14T09:08:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1300000000000003e-06 2024-09-14T09:08:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.468 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839137+0000", 2024-09-14T09:08:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111329898 2024-09-14T09:08:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839226+0000", 2024-09-14T09:08:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8337000000000005e-05 2024-09-14T09:08:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.469 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839517+0000", 2024-09-14T09:08:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029111300000000002 2024-09-14T09:08:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117363 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 407552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:08:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.639163541999999, 2024-09-14T09:08:11.470 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112191306, 2024-09-14T09:08:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117363 2024-09-14T09:08:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.471 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:08:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:08:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733666+0000", 2024-09-14T09:08:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00037284500000000001 2024-09-14T09:08:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733671+0000", 2024-09-14T09:08:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9269999999999996e-06 2024-09-14T09:08:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.473 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733681+0000", 2024-09-14T09:08:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5599999999999999e-06 2024-09-14T09:08:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733689+0000", 2024-09-14T09:08:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2649999999999996e-06 2024-09-14T09:08:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845028+0000", 2024-09-14T09:08:11.474 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111338933 2024-09-14T09:08:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845094+0000", 2024-09-14T09:08:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6749999999999996e-05 2024-09-14T09:08:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845485+0000", 2024-09-14T09:08:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00039002600000000002 2024-09-14T09:08:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.475 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117365 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 411648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:08:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 39.638753776999998, 2024-09-14T09:08:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112271685, 2024-09-14T09:08:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117365 2024-09-14T09:08:11.476 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:08:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:08:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:11.477 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734147+0000", 2024-09-14T09:08:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00044374399999999999 2024-09-14T09:08:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734152+0000", 2024-09-14T09:08:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9139999999999999e-06 2024-09-14T09:08:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734164+0000", 2024-09-14T09:08:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2299e-05 2024-09-14T09:08:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.478 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734172+0000", 2024-09-14T09:08:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0390000000000005e-06 2024-09-14T09:08:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845528+0000", 2024-09-14T09:08:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11135639899999999 2024-09-14T09:08:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845602+0000", 2024-09-14T09:08:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3957999999999999e-05 2024-09-14T09:08:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:11.479 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845975+0000", 2024-09-14T09:08:11.480 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00037233200000000001 2024-09-14T09:08:11.480 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.480 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.480 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.480 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:11.480 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:11.480 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:11.482 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:08:11.482 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T09:08:11.483 INFO:teuthology.orchestra.run.smithi176.stderr:no valid command found; 10 closest matches: 2024-09-14T09:08:11.483 INFO:teuthology.orchestra.run.smithi176.stderr:0 2024-09-14T09:08:11.483 INFO:teuthology.orchestra.run.smithi176.stderr:1 2024-09-14T09:08:11.483 INFO:teuthology.orchestra.run.smithi176.stderr:2 2024-09-14T09:08:11.483 INFO:teuthology.orchestra.run.smithi176.stderr:abort 2024-09-14T09:08:11.483 INFO:teuthology.orchestra.run.smithi176.stderr:assert 2024-09-14T09:08:11.483 INFO:teuthology.orchestra.run.smithi176.stderr:bench [] [] [] [] 2024-09-14T09:08:11.484 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs debug_inject_read_zeros 2024-09-14T09:08:11.484 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs files list 2024-09-14T09:08:11.484 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs stats 2024-09-14T09:08:11.484 INFO:teuthology.orchestra.run.smithi176.stderr:bluestore allocator dump block 2024-09-14T09:08:11.484 INFO:teuthology.orchestra.run.smithi176.stderr:Error EINVAL: invalid command 2024-09-14T09:08:11.484 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T09:08:11.484 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-14T09:08:11.485 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:11.486+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:11.582 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:11.586+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:11.620 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:11.621 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:11.621 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:08:11.621 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:11.635 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:08:11.682 INFO:teuthology.orchestra.run.smithi176.stderr:no valid command found; 10 closest matches: 2024-09-14T09:08:11.682 INFO:teuthology.orchestra.run.smithi176.stderr:0 2024-09-14T09:08:11.682 INFO:teuthology.orchestra.run.smithi176.stderr:1 2024-09-14T09:08:11.682 INFO:teuthology.orchestra.run.smithi176.stderr:2 2024-09-14T09:08:11.682 INFO:teuthology.orchestra.run.smithi176.stderr:abort 2024-09-14T09:08:11.682 INFO:teuthology.orchestra.run.smithi176.stderr:assert 2024-09-14T09:08:11.682 INFO:teuthology.orchestra.run.smithi176.stderr:bench [] [] [] [] 2024-09-14T09:08:11.682 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs debug_inject_read_zeros 2024-09-14T09:08:11.682 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs files list 2024-09-14T09:08:11.682 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs stats 2024-09-14T09:08:11.682 INFO:teuthology.orchestra.run.smithi176.stderr:bluestore allocator dump block 2024-09-14T09:08:11.682 INFO:teuthology.orchestra.run.smithi176.stderr:Error EINVAL: invalid command 2024-09-14T09:08:11.684 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:11.683+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:11.685 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T09:08:11.685 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T09:08:11.685 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-14T09:08:11.785 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:11.786+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:11.789 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:08:11.802 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:11.802 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:11.802 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:08:11.802 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:08:11.802 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:11.816 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:08:11.880 INFO:teuthology.orchestra.run.smithi176.stderr:no valid command found; 10 closest matches: 2024-09-14T09:08:11.881 INFO:teuthology.orchestra.run.smithi176.stderr:0 2024-09-14T09:08:11.881 INFO:teuthology.orchestra.run.smithi176.stderr:1 2024-09-14T09:08:11.881 INFO:teuthology.orchestra.run.smithi176.stderr:2 2024-09-14T09:08:11.881 INFO:teuthology.orchestra.run.smithi176.stderr:abort 2024-09-14T09:08:11.881 INFO:teuthology.orchestra.run.smithi176.stderr:assert 2024-09-14T09:08:11.881 INFO:teuthology.orchestra.run.smithi176.stderr:bench [] [] [] [] 2024-09-14T09:08:11.881 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs debug_inject_read_zeros 2024-09-14T09:08:11.881 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs files list 2024-09-14T09:08:11.881 INFO:teuthology.orchestra.run.smithi176.stderr:bluefs stats 2024-09-14T09:08:11.881 INFO:teuthology.orchestra.run.smithi176.stderr:bluestore allocator dump block 2024-09-14T09:08:11.881 INFO:teuthology.orchestra.run.smithi176.stderr:Error EINVAL: invalid command 2024-09-14T09:08:11.883 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T09:08:11.884 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T09:08:11.886 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:11.887+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:11.935 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:08:11.936 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:08:11.952 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:08:11.952 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:08:11.968 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:08:11.968 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:08:11.983 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:08:11.984 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:08:11.984 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:08:11.985 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:11.985 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:08:11.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:08:11.986 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:08:11.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.986 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105615 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 382976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:11.986 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:08:11.986 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.124301410000001, 2024-09-14T09:08:11.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117270900000001, 2024-09-14T09:08:11.986 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:11.986 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:11.986 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:11.986 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:11.986 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:11.986 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.986 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105615 2024-09-14T09:08:11.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.987 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:11.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:11.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:08:11.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:11.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:11.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:08:11.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:11.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:11.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864368+0000", 2024-09-14T09:08:11.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.043e-06 2024-09-14T09:08:11.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:11.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864373+0000", 2024-09-14T09:08:11.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4669999999999998e-06 2024-09-14T09:08:11.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:11.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864387+0000", 2024-09-14T09:08:11.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3876000000000001e-05 2024-09-14T09:08:11.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864397+0000", 2024-09-14T09:08:11.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0592e-05 2024-09-14T09:08:11.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:11.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866069+0000", 2024-09-14T09:08:11.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016714410000000001 2024-09-14T09:08:11.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:11.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866076+0000", 2024-09-14T09:08:11.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:08:11.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:11.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.075854+0000", 2024-09-14T09:08:11.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.209778501 2024-09-14T09:08:11.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:11.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.075940+0000", 2024-09-14T09:08:11.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5357000000000003e-05 2024-09-14T09:08:11.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:11.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.076094+0000", 2024-09-14T09:08:11.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000154745 2024-09-14T09:08:11.991 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:11.991 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:11.991 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:11.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.991 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105629 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 411648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:11.991 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:08:11.991 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.122147687999998, 2024-09-14T09:08:11.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2115591910000001, 2024-09-14T09:08:11.991 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:11.991 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:11.992 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:11.992 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:11.992 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:11.992 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.992 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105629 2024-09-14T09:08:11.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.992 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:11.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:11.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:08:11.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:11.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:11.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:08:11.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:11.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:11.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866522+0000", 2024-09-14T09:08:11.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6899999999999996e-07 2024-09-14T09:08:11.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:11.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866526+0000", 2024-09-14T09:08:11.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0110000000000002e-06 2024-09-14T09:08:11.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:11.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866539+0000", 2024-09-14T09:08:11.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3013000000000001e-05 2024-09-14T09:08:11.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:11.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866549+0000", 2024-09-14T09:08:11.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8490000000000001e-06 2024-09-14T09:08:11.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:11.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866650+0000", 2024-09-14T09:08:11.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101054 2024-09-14T09:08:11.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:11.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866664+0000", 2024-09-14T09:08:11.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3805000000000001e-05 2024-09-14T09:08:11.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:11.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.077849+0000", 2024-09-14T09:08:11.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2111855419999999 2024-09-14T09:08:11.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:11.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.077945+0000", 2024-09-14T09:08:11.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6086000000000001e-05 2024-09-14T09:08:11.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:11.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078080+0000", 2024-09-14T09:08:11.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000134862 2024-09-14T09:08:11.996 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:11.996 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:11.996 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:11.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.996 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105631 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 415744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:11.997 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:08:11.997 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.122088468999998, 2024-09-14T09:08:11.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119463509999999, 2024-09-14T09:08:11.997 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:11.997 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:11.997 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:11.997 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:11.997 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:11.997 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:11.997 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105631 2024-09-14T09:08:11.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.998 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:11.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:11.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:08:11.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:11.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:11.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:08:11.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:11.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:11.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866581+0000", 2024-09-14T09:08:11.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.879999999999999e-07 2024-09-14T09:08:11.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:11.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866585+0000", 2024-09-14T09:08:11.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0239999999999999e-06 2024-09-14T09:08:11.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:11.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:11.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866598+0000", 2024-09-14T09:08:11.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2862e-05 2024-09-14T09:08:11.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:11.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:12.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866608+0000", 2024-09-14T09:08:12.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9890000000000004e-06 2024-09-14T09:08:12.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866708+0000", 2024-09-14T09:08:12.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9605999999999995e-05 2024-09-14T09:08:12.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:12.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866720+0000", 2024-09-14T09:08:12.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.261e-05 2024-09-14T09:08:12.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078137+0000", 2024-09-14T09:08:12.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211416571 2024-09-14T09:08:12.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:12.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078293+0000", 2024-09-14T09:08:12.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000156 2024-09-14T09:08:12.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:12.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078527+0000", 2024-09-14T09:08:12.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000233701 2024-09-14T09:08:12.002 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.002 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:12.002 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.002 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105633 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:12.002 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:08:12.002 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.122029421999997, 2024-09-14T09:08:12.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211941734, 2024-09-14T09:08:12.002 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:12.002 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:12.003 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:12.003 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:12.003 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:12.003 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:12.003 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105633 2024-09-14T09:08:12.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.003 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:12.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:12.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:08:12.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:12.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:08:12.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:12.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866640+0000", 2024-09-14T09:08:12.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9300000000000006e-07 2024-09-14T09:08:12.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:12.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866656+0000", 2024-09-14T09:08:12.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5291e-05 2024-09-14T09:08:12.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:12.005 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866669+0000", 2024-09-14T09:08:12.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2918000000000001e-05 2024-09-14T09:08:12.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:12.005 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866678+0000", 2024-09-14T09:08:12.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8369999999999992e-06 2024-09-14T09:08:12.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866771+0000", 2024-09-14T09:08:12.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2250999999999995e-05 2024-09-14T09:08:12.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:12.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866784+0000", 2024-09-14T09:08:12.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2991e-05 2024-09-14T09:08:12.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078566+0000", 2024-09-14T09:08:12.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117822570000001 2024-09-14T09:08:12.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:12.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078581+0000", 2024-09-14T09:08:12.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5196e-05 2024-09-14T09:08:12.007 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.007 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:12.007 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.008 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105635 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:12.008 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:08:12.008 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.121975595000002, 2024-09-14T09:08:12.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119313250000001, 2024-09-14T09:08:12.008 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:12.008 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:12.008 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:12.008 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:12.008 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:12.008 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:12.009 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105635 2024-09-14T09:08:12.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.009 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:12.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:12.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:08:12.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:12.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:08:12.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:12.010 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866694+0000", 2024-09-14T09:08:12.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0210000000000001e-06 2024-09-14T09:08:12.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:12.010 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866698+0000", 2024-09-14T09:08:12.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.765e-06 2024-09-14T09:08:12.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:12.010 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866710+0000", 2024-09-14T09:08:12.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2320000000000001e-05 2024-09-14T09:08:12.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:12.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866722+0000", 2024-09-14T09:08:12.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2133000000000001e-05 2024-09-14T09:08:12.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866830+0000", 2024-09-14T09:08:12.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000107668 2024-09-14T09:08:12.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:12.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866843+0000", 2024-09-14T09:08:12.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2991e-05 2024-09-14T09:08:12.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078616+0000", 2024-09-14T09:08:12.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117732459999999 2024-09-14T09:08:12.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:12.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078624+0000", 2024-09-14T09:08:12.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1810000000000001e-06 2024-09-14T09:08:12.012 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.012 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:12.012 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.013 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105636 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:12.013 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:08:12.013 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.121907698999998, 2024-09-14T09:08:12.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119085119999999, 2024-09-14T09:08:12.013 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:12.013 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:12.013 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:12.013 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:12.013 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:12.013 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:12.013 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105636 2024-09-14T09:08:12.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.013 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:12.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:12.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:08:12.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:12.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:08:12.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:12.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866762+0000", 2024-09-14T09:08:12.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4600000000000003e-07 2024-09-14T09:08:12.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:12.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866766+0000", 2024-09-14T09:08:12.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:08:12.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:12.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866779+0000", 2024-09-14T09:08:12.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3101999999999999e-05 2024-09-14T09:08:12.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:12.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866789+0000", 2024-09-14T09:08:12.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8039999999999996e-06 2024-09-14T09:08:12.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866889+0000", 2024-09-14T09:08:12.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100147 2024-09-14T09:08:12.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:12.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866902+0000", 2024-09-14T09:08:12.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3212e-05 2024-09-14T09:08:12.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078659+0000", 2024-09-14T09:08:12.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211756362 2024-09-14T09:08:12.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:12.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078670+0000", 2024-09-14T09:08:12.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0956e-05 2024-09-14T09:08:12.017 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.017 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:12.017 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.018 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105639 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:12.018 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:08:12.018 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.121848823999997, 2024-09-14T09:08:12.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118829259999999, 2024-09-14T09:08:12.018 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:12.018 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:12.018 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:12.018 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:12.018 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:12.018 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:12.018 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105639 2024-09-14T09:08:12.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.019 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:12.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:12.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:08:12.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:12.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:08:12.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:12.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866821+0000", 2024-09-14T09:08:12.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9800000000000002e-07 2024-09-14T09:08:12.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:12.020 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866825+0000", 2024-09-14T09:08:12.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9269999999999998e-06 2024-09-14T09:08:12.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:12.020 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866838+0000", 2024-09-14T09:08:12.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3040000000000001e-05 2024-09-14T09:08:12.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:12.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866848+0000", 2024-09-14T09:08:12.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7929999999999992e-06 2024-09-14T09:08:12.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866948+0000", 2024-09-14T09:08:12.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100337 2024-09-14T09:08:12.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:12.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866961+0000", 2024-09-14T09:08:12.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3106e-05 2024-09-14T09:08:12.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078693+0000", 2024-09-14T09:08:12.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117320380000001 2024-09-14T09:08:12.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:12.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078703+0000", 2024-09-14T09:08:12.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6870000000000006e-06 2024-09-14T09:08:12.022 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.022 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:12.022 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.023 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105641 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:12.023 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:08:12.023 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.121789958999997, 2024-09-14T09:08:12.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118727169999999, 2024-09-14T09:08:12.023 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:12.023 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:12.023 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:12.023 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:12.023 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:12.023 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:12.023 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105641 2024-09-14T09:08:12.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.024 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:12.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:12.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:08:12.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:12.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:08:12.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:12.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866880+0000", 2024-09-14T09:08:12.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7699999999999992e-07 2024-09-14T09:08:12.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:12.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866884+0000", 2024-09-14T09:08:12.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9480000000000001e-06 2024-09-14T09:08:12.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:12.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866897+0000", 2024-09-14T09:08:12.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2832e-05 2024-09-14T09:08:12.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:12.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866906+0000", 2024-09-14T09:08:12.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7869999999999996e-06 2024-09-14T09:08:12.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867007+0000", 2024-09-14T09:08:12.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010103900000000001 2024-09-14T09:08:12.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:12.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867020+0000", 2024-09-14T09:08:12.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2896e-05 2024-09-14T09:08:12.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078739+0000", 2024-09-14T09:08:12.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211718305 2024-09-14T09:08:12.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:12.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078752+0000", 2024-09-14T09:08:12.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2933e-05 2024-09-14T09:08:12.027 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.027 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:12.027 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.028 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105643 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:12.028 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:08:12.028 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.121731400000002, 2024-09-14T09:08:12.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211858353, 2024-09-14T09:08:12.028 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:12.028 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:12.028 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:12.028 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:12.028 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:12.028 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:12.028 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105643 2024-09-14T09:08:12.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.029 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:12.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:12.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:08:12.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:12.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:08:12.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:12.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866938+0000", 2024-09-14T09:08:12.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5600000000000004e-07 2024-09-14T09:08:12.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:12.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866942+0000", 2024-09-14T09:08:12.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0350000000000003e-06 2024-09-14T09:08:12.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:12.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866955+0000", 2024-09-14T09:08:12.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3097e-05 2024-09-14T09:08:12.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:12.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866965+0000", 2024-09-14T09:08:12.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8179999999999999e-06 2024-09-14T09:08:12.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867067+0000", 2024-09-14T09:08:12.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010128699999999999 2024-09-14T09:08:12.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:12.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867080+0000", 2024-09-14T09:08:12.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3089e-05 2024-09-14T09:08:12.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078786+0000", 2024-09-14T09:08:12.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117065250000001 2024-09-14T09:08:12.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:12.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078796+0000", 2024-09-14T09:08:12.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5459999999999997e-06 2024-09-14T09:08:12.032 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.032 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:12.032 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.032 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105645 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:12.033 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:08:12.033 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.121671005000003, 2024-09-14T09:08:12.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211848641, 2024-09-14T09:08:12.033 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:12.033 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:12.033 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:12.033 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:12.033 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:12.033 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:12.033 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105645 2024-09-14T09:08:12.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.033 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:12.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:12.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:08:12.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:12.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:08:12.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:12.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866999+0000", 2024-09-14T09:08:12.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7999999999999993e-07 2024-09-14T09:08:12.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:12.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867003+0000", 2024-09-14T09:08:12.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0090000000000001e-06 2024-09-14T09:08:12.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:12.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867016+0000", 2024-09-14T09:08:12.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3304e-05 2024-09-14T09:08:12.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:12.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867026+0000", 2024-09-14T09:08:12.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5889999999999992e-06 2024-09-14T09:08:12.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867128+0000", 2024-09-14T09:08:12.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000102101 2024-09-14T09:08:12.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:12.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867141+0000", 2024-09-14T09:08:12.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2903e-05 2024-09-14T09:08:12.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078833+0000", 2024-09-14T09:08:12.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116920959999999 2024-09-14T09:08:12.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:12.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078846+0000", 2024-09-14T09:08:12.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3658999999999999e-05 2024-09-14T09:08:12.037 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.037 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:12.037 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.037 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105647 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:12.037 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:08:12.037 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.121612487, 2024-09-14T09:08:12.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211830097, 2024-09-14T09:08:12.038 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:12.038 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:12.038 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:12.038 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:12.038 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:12.038 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:12.038 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105647 2024-09-14T09:08:12.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.038 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:12.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:12.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:08:12.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867057+0000", 2024-09-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.95e-07 2024-09-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867061+0000", 2024-09-14T09:08:12.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9530000000000001e-06 2024-09-14T09:08:12.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:12.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867074+0000", 2024-09-14T09:08:12.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3061000000000001e-05 2024-09-14T09:08:12.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:12.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867086+0000", 2024-09-14T09:08:12.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1460999999999999e-05 2024-09-14T09:08:12.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867186+0000", 2024-09-14T09:08:12.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100181 2024-09-14T09:08:12.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:12.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867199+0000", 2024-09-14T09:08:12.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-05 2024-09-14T09:08:12.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078874+0000", 2024-09-14T09:08:12.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211674683 2024-09-14T09:08:12.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:12.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078886+0000", 2024-09-14T09:08:12.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2853e-05 2024-09-14T09:08:12.042 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.042 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:12.042 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.042 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105648 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:12.042 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:08:12.042 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.121551001, 2024-09-14T09:08:12.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118073570000001, 2024-09-14T09:08:12.042 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:12.042 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:12.042 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:12.042 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:12.043 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:12.043 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:12.043 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105648 2024-09-14T09:08:12.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.043 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:12.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:12.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:08:12.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:12.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:08:12.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:12.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867119+0000", 2024-09-14T09:08:12.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8400000000000002e-07 2024-09-14T09:08:12.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:12.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867123+0000", 2024-09-14T09:08:12.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9639999999999997e-06 2024-09-14T09:08:12.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:12.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867136+0000", 2024-09-14T09:08:12.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.289e-05 2024-09-14T09:08:12.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:12.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867145+0000", 2024-09-14T09:08:12.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8919999999999995e-06 2024-09-14T09:08:12.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867245+0000", 2024-09-14T09:08:12.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9282000000000003e-05 2024-09-14T09:08:12.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:12.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867258+0000", 2024-09-14T09:08:12.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2935000000000001e-05 2024-09-14T09:08:12.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078910+0000", 2024-09-14T09:08:12.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211652352 2024-09-14T09:08:12.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:12.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078925+0000", 2024-09-14T09:08:12.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5058000000000001e-05 2024-09-14T09:08:12.046 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.047 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:12.047 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.047 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105652 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:12.047 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:08:12.047 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.121492981999999, 2024-09-14T09:08:12.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211775737, 2024-09-14T09:08:12.047 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:12.047 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:12.047 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:12.047 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:12.047 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:12.047 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:12.048 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105652 2024-09-14T09:08:12.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.048 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:12.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:12.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:08:12.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:12.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:08:12.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:12.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867177+0000", 2024-09-14T09:08:12.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9100000000000002e-07 2024-09-14T09:08:12.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:12.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867181+0000", 2024-09-14T09:08:12.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0029999999999996e-06 2024-09-14T09:08:12.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:12.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867193+0000", 2024-09-14T09:08:12.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2675e-05 2024-09-14T09:08:12.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:12.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867203+0000", 2024-09-14T09:08:12.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5599999999999999e-06 2024-09-14T09:08:12.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867306+0000", 2024-09-14T09:08:12.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010330100000000001 2024-09-14T09:08:12.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:12.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867319+0000", 2024-09-14T09:08:12.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3129000000000001e-05 2024-09-14T09:08:12.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078944+0000", 2024-09-14T09:08:12.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116247520000001 2024-09-14T09:08:12.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:12.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078952+0000", 2024-09-14T09:08:12.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4259999999999997e-06 2024-09-14T09:08:12.051 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.051 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:12.051 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.052 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105653 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:12.052 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:08:12.052 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.121434311999998, 2024-09-14T09:08:12.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211774895, 2024-09-14T09:08:12.052 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:12.052 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:12.052 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:12.052 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:12.052 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:12.052 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:12.052 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105653 2024-09-14T09:08:12.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.052 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:12.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:12.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:08:12.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:12.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:08:12.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:12.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867235+0000", 2024-09-14T09:08:12.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9400000000000004e-07 2024-09-14T09:08:12.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:12.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867239+0000", 2024-09-14T09:08:12.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9380000000000002e-06 2024-09-14T09:08:12.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:12.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867252+0000", 2024-09-14T09:08:12.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.31e-05 2024-09-14T09:08:12.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:12.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867264+0000", 2024-09-14T09:08:12.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1275e-05 2024-09-14T09:08:12.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867367+0000", 2024-09-14T09:08:12.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103755 2024-09-14T09:08:12.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:12.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867382+0000", 2024-09-14T09:08:12.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4724e-05 2024-09-14T09:08:12.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078995+0000", 2024-09-14T09:08:12.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116132669999999 2024-09-14T09:08:12.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:12.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079009+0000", 2024-09-14T09:08:12.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3942e-05 2024-09-14T09:08:12.056 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.056 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:12.056 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.056 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105654 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:12.057 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:08:12.057 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.121373255999998, 2024-09-14T09:08:12.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117364070000001, 2024-09-14T09:08:12.057 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:12.057 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:12.057 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:12.057 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:12.057 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:12.057 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:12.057 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105654 2024-09-14T09:08:12.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.057 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:12.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:12.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:08:12.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:12.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:08:12.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:12.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867296+0000", 2024-09-14T09:08:12.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2399999999999996e-07 2024-09-14T09:08:12.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:12.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867300+0000", 2024-09-14T09:08:12.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8689999999999997e-06 2024-09-14T09:08:12.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:12.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867313+0000", 2024-09-14T09:08:12.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2660000000000001e-05 2024-09-14T09:08:12.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:12.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867322+0000", 2024-09-14T09:08:12.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.533e-06 2024-09-14T09:08:12.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867429+0000", 2024-09-14T09:08:12.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106566 2024-09-14T09:08:12.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:12.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867442+0000", 2024-09-14T09:08:12.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3244000000000001e-05 2024-09-14T09:08:12.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079024+0000", 2024-09-14T09:08:12.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211581813 2024-09-14T09:08:12.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:12.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079032+0000", 2024-09-14T09:08:12.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7980000000000006e-06 2024-09-14T09:08:12.061 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.061 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:12.061 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.061 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105657 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:12.062 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:08:12.062 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.121312082999999, 2024-09-14T09:08:12.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211700657, 2024-09-14T09:08:12.062 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:12.062 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:12.062 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:12.062 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:12.062 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:12.062 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:12.062 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105657 2024-09-14T09:08:12.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.062 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:12.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:12.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:08:12.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:12.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:08:12.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:12.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867358+0000", 2024-09-14T09:08:12.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9700000000000005e-07 2024-09-14T09:08:12.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:12.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867367+0000", 2024-09-14T09:08:12.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8919999999999995e-06 2024-09-14T09:08:12.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:12.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867383+0000", 2024-09-14T09:08:12.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.579e-05 2024-09-14T09:08:12.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:12.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867393+0000", 2024-09-14T09:08:12.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3070000000000008e-06 2024-09-14T09:08:12.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867490+0000", 2024-09-14T09:08:12.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7703999999999998e-05 2024-09-14T09:08:12.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:12.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867503+0000", 2024-09-14T09:08:12.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2849e-05 2024-09-14T09:08:12.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079050+0000", 2024-09-14T09:08:12.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211546899 2024-09-14T09:08:12.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:12.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079057+0000", 2024-09-14T09:08:12.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3189999999999997e-06 2024-09-14T09:08:12.066 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.066 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:12.066 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.066 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105658 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:12.066 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:08:12.067 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.121249699000003, 2024-09-14T09:08:12.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211691217, 2024-09-14T09:08:12.067 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:12.067 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:12.067 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:12.067 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:12.067 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:12.067 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:12.067 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105658 2024-09-14T09:08:12.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.067 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:12.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:12.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:08:12.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:12.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:08:12.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:12.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867420+0000", 2024-09-14T09:08:12.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5000000000000001e-07 2024-09-14T09:08:12.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:12.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867424+0000", 2024-09-14T09:08:12.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8469999999999998e-06 2024-09-14T09:08:12.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:12.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867437+0000", 2024-09-14T09:08:12.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3156000000000001e-05 2024-09-14T09:08:12.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:12.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867448+0000", 2024-09-14T09:08:12.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1185e-05 2024-09-14T09:08:12.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867552+0000", 2024-09-14T09:08:12.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103362 2024-09-14T09:08:12.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:12.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867565+0000", 2024-09-14T09:08:12.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3018e-05 2024-09-14T09:08:12.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079092+0000", 2024-09-14T09:08:12.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211527013 2024-09-14T09:08:12.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:12.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079110+0000", 2024-09-14T09:08:12.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8686e-05 2024-09-14T09:08:12.071 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.071 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:12.071 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.071 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105662 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:12.071 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:08:12.071 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.121188633999999, 2024-09-14T09:08:12.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211652226, 2024-09-14T09:08:12.072 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:12.072 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:12.072 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:12.072 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:12.072 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:12.072 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:12.072 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105662 2024-09-14T09:08:12.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.072 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:12.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:12.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:08:12.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:12.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:08:12.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:12.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867481+0000", 2024-09-14T09:08:12.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1800000000000004e-07 2024-09-14T09:08:12.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:12.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867485+0000", 2024-09-14T09:08:12.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9199999999999997e-06 2024-09-14T09:08:12.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:12.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867498+0000", 2024-09-14T09:08:12.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2896e-05 2024-09-14T09:08:12.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:12.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867509+0000", 2024-09-14T09:08:12.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1202e-05 2024-09-14T09:08:12.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867611+0000", 2024-09-14T09:08:12.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010166999999999999 2024-09-14T09:08:12.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:12.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867624+0000", 2024-09-14T09:08:12.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2996000000000001e-05 2024-09-14T09:08:12.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079125+0000", 2024-09-14T09:08:12.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2115009299999999 2024-09-14T09:08:12.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:12.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079132+0000", 2024-09-14T09:08:12.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6939999999999995e-06 2024-09-14T09:08:12.076 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.076 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:12.076 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.076 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105663 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:12.076 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:08:12.077 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.121127583000003, 2024-09-14T09:08:12.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116218169999999, 2024-09-14T09:08:12.077 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:12.077 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:12.077 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:12.077 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:12.077 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:12.077 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:12.077 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105663 2024-09-14T09:08:12.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.077 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:12.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:12.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:08:12.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:12.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:08:12.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:12.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867542+0000", 2024-09-14T09:08:12.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1399999999999995e-07 2024-09-14T09:08:12.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.079 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:12.079 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867546+0000", 2024-09-14T09:08:12.079 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7900000000000001e-06 2024-09-14T09:08:12.079 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.079 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.079 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:12.079 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867559+0000", 2024-09-14T09:08:12.079 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2805e-05 2024-09-14T09:08:12.079 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.079 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.079 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:12.079 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867568+0000", 2024-09-14T09:08:12.079 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-09-14T09:08:12.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.080 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.080 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.080 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867669+0000", 2024-09-14T09:08:12.080 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100655 2024-09-14T09:08:12.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.080 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.080 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:12.080 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867682+0000", 2024-09-14T09:08:12.080 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3013000000000001e-05 2024-09-14T09:08:12.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.080 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.080 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.080 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079151+0000", 2024-09-14T09:08:12.081 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2114689089999999 2024-09-14T09:08:12.081 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.081 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.081 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:12.081 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079163+0000", 2024-09-14T09:08:12.081 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1962e-05 2024-09-14T09:08:12.081 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.081 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:12.081 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.081 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.081 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.081 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105665 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:12.081 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:08:12.081 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.121068952000002, 2024-09-14T09:08:12.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211584848, 2024-09-14T09:08:12.082 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:12.082 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:12.082 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:12.082 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:12.082 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:12.082 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:12.082 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105665 2024-09-14T09:08:12.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.082 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:12.082 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.082 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:12.082 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:08:12.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:12.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:08:12.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:12.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:12.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867601+0000", 2024-09-14T09:08:12.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.569999999999999e-07 2024-09-14T09:08:12.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:12.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867605+0000", 2024-09-14T09:08:12.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8589999999999998e-06 2024-09-14T09:08:12.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.084 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:12.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867617+0000", 2024-09-14T09:08:12.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2833000000000001e-05 2024-09-14T09:08:12.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.084 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:12.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867627+0000", 2024-09-14T09:08:12.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5219999999999996e-06 2024-09-14T09:08:12.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867728+0000", 2024-09-14T09:08:12.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010057500000000001 2024-09-14T09:08:12.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:12.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867740+0000", 2024-09-14T09:08:12.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2744e-05 2024-09-14T09:08:12.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:12.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079178+0000", 2024-09-14T09:08:12.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211437552 2024-09-14T09:08:12.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:12.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:12.086 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:12.086 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079185+0000", 2024-09-14T09:08:12.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8059999999999997e-06 2024-09-14T09:08:12.086 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.086 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:12.086 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.086 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:12.086 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:12.086 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:12.087 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:08:12.088 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:08:12.088 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:08:12.089 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:08:12.089 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:08:12.089 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:12.090+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:12.090 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:08:12.090 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:08:12.090 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:08:12.091 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T09:08:12.188 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:12.187+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:12.263 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:12.263 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:12.269 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T09:08:12.280 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:08:12.330 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:12.334+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:12.390 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:12.391+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:12.407 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:12.407 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:12.424 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:08:12.491 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:12.491+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:12.583 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:12.583 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:12.592 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:12.591+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:12.597 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:08:12.694 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:12.695+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:12.719 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:12.719 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:12.733 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:08:12.794 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:12.795+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:12.849 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:12.849 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:12.864 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:08:12.910 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:12.914+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:12.987 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:12.987 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:12.995 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:12.995+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:13.001 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:08:13.097 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:13.095+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:13.132 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:13.132 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:13.146 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:08:13.234 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:13.235+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:13.249 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is set 2024-09-14T09:08:13.299 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:13.302+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:13.338 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:13.338 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:13.353 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:08:13.402 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:13.403+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:13.481 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:13.481 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:13.495 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:08:13.542 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:13.543+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:13.601 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:13.606+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:13.661 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:13.661 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:13.676 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:08:13.702 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:13.706+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:13.797 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:13.797 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:13.803 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:13.803+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:13.811 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:08:13.903 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:13.903+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:13.979 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:13.979 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:13.993 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:08:14.004 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:14.006+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:14.091 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:08:14.138 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:14.139+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:14.158 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:14.158 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:14.173 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:08:14.218 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:14.219+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:14.234 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:08:14.234 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:08:14.252 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:08:14.253 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:08:14.272 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:08:14.273 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:08:14.291 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:08:14.291 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:08:14.291 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:08:14.307 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:14.310+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:14.316 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:08:14.316 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:08:14.338 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:08:14.338 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:08:14.340 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:14.340 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:14.341 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:08:14.341 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:08:14.341 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:14.356 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:08:14.360 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:08:14.360 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:08:14.365 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:08:14.366 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T09:08:14.410 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:14.411+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:14.484 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:14.485 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:08:14.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:08:14.485 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:08:14.485 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.485 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106705 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 518144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.485 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:14.485 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637427868000003, 2024-09-14T09:08:14.485 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3001326799999999, 2024-09-14T09:08:14.485 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.485 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.485 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.485 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.485 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.486 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.486 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106705 2024-09-14T09:08:14.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.486 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:14.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.486 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.486 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.486 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:14.486 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.486 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:14.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4699999999999998e-07 2024-09-14T09:08:14.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847500+0000", 2024-09-14T09:08:14.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0860000000000001e-06 2024-09-14T09:08:14.487 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.487 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.487 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.487 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847503+0000", 2024-09-14T09:08:14.487 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1269999999999999e-06 2024-09-14T09:08:14.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847506+0000", 2024-09-14T09:08:14.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5780000000000001e-06 2024-09-14T09:08:14.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.488 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858911+0000", 2024-09-14T09:08:14.488 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011404925 2024-09-14T09:08:14.488 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.488 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.488 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:14.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858919+0000", 2024-09-14T09:08:14.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8620000000000003e-06 2024-09-14T09:08:14.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147254+0000", 2024-09-14T09:08:14.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2883353719999999 2024-09-14T09:08:14.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.489 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.489 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147408+0000", 2024-09-14T09:08:14.489 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015389200000000001 2024-09-14T09:08:14.489 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.489 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.490 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.490 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147631+0000", 2024-09-14T09:08:14.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022359100000000001 2024-09-14T09:08:14.490 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.490 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.490 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.490 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.490 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.490 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106707 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 522240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.490 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:14.490 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637417397999997, 2024-09-14T09:08:14.490 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.300471801, 2024-09-14T09:08:14.490 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.490 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.491 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.491 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.491 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.491 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.491 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106707 2024-09-14T09:08:14.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.491 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:14.491 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.491 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.491 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.491 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.491 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:14.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:14.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.23e-07 2024-09-14T09:08:14.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.492 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.492 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847514+0000", 2024-09-14T09:08:14.492 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9150000000000004e-06 2024-09-14T09:08:14.492 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.492 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847520+0000", 2024-09-14T09:08:14.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.203e-06 2024-09-14T09:08:14.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847532+0000", 2024-09-14T09:08:14.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2441e-05 2024-09-14T09:08:14.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.493 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.493 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858953+0000", 2024-09-14T09:08:14.493 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01142108 2024-09-14T09:08:14.493 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.493 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:14.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858957+0000", 2024-09-14T09:08:14.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2830000000000003e-06 2024-09-14T09:08:14.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147667+0000", 2024-09-14T09:08:14.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2887094619999999 2024-09-14T09:08:14.494 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.494 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.494 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.494 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147784+0000", 2024-09-14T09:08:14.494 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117465 2024-09-14T09:08:14.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.495 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.495 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147981+0000", 2024-09-14T09:08:14.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019672900000000001 2024-09-14T09:08:14.495 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.495 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.495 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.495 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.495 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.495 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106709 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 3072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.495 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:08:14.495 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637389106999997, 2024-09-14T09:08:14.495 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.301335192, 2024-09-14T09:08:14.496 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.496 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.496 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.496 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.496 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.496 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.496 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106709 2024-09-14T09:08:14.496 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.496 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.496 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.496 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.496 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:08:14.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:08:14.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.497 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.497 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847538+0000", 2024-09-14T09:08:14.497 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2800000000000003e-07 2024-09-14T09:08:14.497 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.497 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847539+0000", 2024-09-14T09:08:14.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2720000000000001e-06 2024-09-14T09:08:14.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847544+0000", 2024-09-14T09:08:14.498 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7219999999999999e-06 2024-09-14T09:08:14.498 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.498 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.498 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.498 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847547+0000", 2024-09-14T09:08:14.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9960000000000002e-06 2024-09-14T09:08:14.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858991+0000", 2024-09-14T09:08:14.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011444218000000001 2024-09-14T09:08:14.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.499 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:14.499 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858999+0000", 2024-09-14T09:08:14.499 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9419999999999995e-06 2024-09-14T09:08:14.499 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.499 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148053+0000", 2024-09-14T09:08:14.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2890544099999999 2024-09-14T09:08:14.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148203+0000", 2024-09-14T09:08:14.500 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149419 2024-09-14T09:08:14.500 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.500 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.500 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.500 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148873+0000", 2024-09-14T09:08:14.501 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00066988499999999995 2024-09-14T09:08:14.501 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.501 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.501 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.501 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.501 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.501 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106711 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 7168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.501 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:14.501 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637375849000001, 2024-09-14T09:08:14.502 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3022241219999999, 2024-09-14T09:08:14.502 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.502 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.502 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.502 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.502 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.502 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.502 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106711 2024-09-14T09:08:14.502 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.502 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.502 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.502 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.502 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:14.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:14.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.503 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.503 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:14.503 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8000000000000002e-07 2024-09-14T09:08:14.503 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.503 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847552+0000", 2024-09-14T09:08:14.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1340000000000001e-06 2024-09-14T09:08:14.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847557+0000", 2024-09-14T09:08:14.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6179999999999997e-06 2024-09-14T09:08:14.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.504 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.504 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.504 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847560+0000", 2024-09-14T09:08:14.504 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5899999999999999e-06 2024-09-14T09:08:14.504 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859050+0000", 2024-09-14T09:08:14.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01148914 2024-09-14T09:08:14.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.505 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:14.505 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859056+0000", 2024-09-14T09:08:14.505 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6630000000000004e-06 2024-09-14T09:08:14.505 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.505 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148948+0000", 2024-09-14T09:08:14.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2898914370000001 2024-09-14T09:08:14.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149079+0000", 2024-09-14T09:08:14.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013145400000000001 2024-09-14T09:08:14.506 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.506 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.506 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.506 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149775+0000", 2024-09-14T09:08:14.506 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00069580599999999999 2024-09-14T09:08:14.507 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.507 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.507 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.507 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.507 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.507 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106713 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 11264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.507 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:14.507 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637362383999999, 2024-09-14T09:08:14.507 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3035188959999999, 2024-09-14T09:08:14.507 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.507 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.507 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.507 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.507 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.507 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.507 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106713 2024-09-14T09:08:14.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.508 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:14.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:14.508 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.508 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.508 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.508 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.508 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:14.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6e-07 2024-09-14T09:08:14.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847566+0000", 2024-09-14T09:08:14.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1039999999999999e-06 2024-09-14T09:08:14.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.509 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847569+0000", 2024-09-14T09:08:14.509 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.191e-06 2024-09-14T09:08:14.509 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.509 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.509 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847571+0000", 2024-09-14T09:08:14.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6390000000000001e-06 2024-09-14T09:08:14.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859099+0000", 2024-09-14T09:08:14.510 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011527379000000001 2024-09-14T09:08:14.510 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.510 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.510 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:14.510 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859106+0000", 2024-09-14T09:08:14.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3679999999999996e-06 2024-09-14T09:08:14.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149878+0000", 2024-09-14T09:08:14.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.29077208 2024-09-14T09:08:14.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.150175+0000", 2024-09-14T09:08:14.511 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029685699999999998 2024-09-14T09:08:14.511 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.511 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.511 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.511 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151083+0000", 2024-09-14T09:08:14.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00090801800000000002 2024-09-14T09:08:14.512 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.512 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.512 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.512 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.512 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.512 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106715 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 15360~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.512 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:14.512 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637351375000001, 2024-09-14T09:08:14.512 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3047478800000001, 2024-09-14T09:08:14.512 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.512 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.512 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.512 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.513 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.513 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.513 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106715 2024-09-14T09:08:14.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.513 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.513 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:14.513 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.513 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.513 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.513 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:14.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:14.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6e-07 2024-09-14T09:08:14.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.514 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.514 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.514 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847580+0000", 2024-09-14T09:08:14.514 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9250000000000003e-06 2024-09-14T09:08:14.514 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847585+0000", 2024-09-14T09:08:14.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8409999999999999e-06 2024-09-14T09:08:14.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847588+0000", 2024-09-14T09:08:14.515 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6989999999999999e-06 2024-09-14T09:08:14.515 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.515 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.515 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.515 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859151+0000", 2024-09-14T09:08:14.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011562604000000001 2024-09-14T09:08:14.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:14.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859159+0000", 2024-09-14T09:08:14.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4179999999999997e-06 2024-09-14T09:08:14.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.516 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.516 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.516 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151155+0000", 2024-09-14T09:08:14.516 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.291996068 2024-09-14T09:08:14.516 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151383+0000", 2024-09-14T09:08:14.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022746100000000001 2024-09-14T09:08:14.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.517 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.517 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.517 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152323+0000", 2024-09-14T09:08:14.517 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00094060400000000005 2024-09-14T09:08:14.517 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.517 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.517 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.517 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.518 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.518 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106717 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 19456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.518 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:14.518 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637334686999999, 2024-09-14T09:08:14.518 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.305574579, 2024-09-14T09:08:14.518 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.518 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.518 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.518 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.518 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.518 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.518 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106717 2024-09-14T09:08:14.518 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.519 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:14.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:14.519 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.519 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.519 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.519 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.519 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:14.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1199999999999999e-07 2024-09-14T09:08:14.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847593+0000", 2024-09-14T09:08:14.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.257e-06 2024-09-14T09:08:14.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.520 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847598+0000", 2024-09-14T09:08:14.520 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4139999999999996e-06 2024-09-14T09:08:14.520 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.520 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.520 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847602+0000", 2024-09-14T09:08:14.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3440000000000003e-06 2024-09-14T09:08:14.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859203+0000", 2024-09-14T09:08:14.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011600572999999999 2024-09-14T09:08:14.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.521 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.521 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:14.521 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859211+0000", 2024-09-14T09:08:14.521 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-06 2024-09-14T09:08:14.521 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152398+0000", 2024-09-14T09:08:14.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.293187812 2024-09-14T09:08:14.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152543+0000", 2024-09-14T09:08:14.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014478000000000001 2024-09-14T09:08:14.522 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.522 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.522 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.522 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153166+0000", 2024-09-14T09:08:14.522 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000623257 2024-09-14T09:08:14.523 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.523 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.523 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.523 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.523 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.523 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106719 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 23552~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.523 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:14.523 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637320371000001, 2024-09-14T09:08:14.523 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.306394982, 2024-09-14T09:08:14.523 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.523 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.524 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.524 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.524 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.524 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.524 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106719 2024-09-14T09:08:14.524 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.524 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.524 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.524 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.524 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:14.524 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:14.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.525 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:14.525 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5100000000000001e-07 2024-09-14T09:08:14.525 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.525 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.525 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847608+0000", 2024-09-14T09:08:14.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.189e-06 2024-09-14T09:08:14.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847612+0000", 2024-09-14T09:08:14.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9519999999999996e-06 2024-09-14T09:08:14.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.526 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.526 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.526 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847614+0000", 2024-09-14T09:08:14.526 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6280000000000001e-06 2024-09-14T09:08:14.526 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859254+0000", 2024-09-14T09:08:14.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011640157 2024-09-14T09:08:14.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:14.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859261+0000", 2024-09-14T09:08:14.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0570000000000004e-06 2024-09-14T09:08:14.527 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.527 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.527 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.527 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153236+0000", 2024-09-14T09:08:14.527 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2939744040000001 2024-09-14T09:08:14.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153380+0000", 2024-09-14T09:08:14.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014410399999999999 2024-09-14T09:08:14.528 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.528 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.528 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.528 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154001+0000", 2024-09-14T09:08:14.528 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062124000000000001 2024-09-14T09:08:14.528 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.528 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.529 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.529 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.529 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.529 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106721 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 27648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.529 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:14.529 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637308720999997, 2024-09-14T09:08:14.529 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3074446659999999, 2024-09-14T09:08:14.529 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.529 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.529 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.529 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.529 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.530 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.530 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106721 2024-09-14T09:08:14.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.530 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:14.530 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.530 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.530 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.530 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.530 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:14.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:14.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4299999999999999e-07 2024-09-14T09:08:14.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847619+0000", 2024-09-14T09:08:14.531 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T09:08:14.531 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.531 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.531 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.531 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847624+0000", 2024-09-14T09:08:14.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4229999999999998e-06 2024-09-14T09:08:14.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847627+0000", 2024-09-14T09:08:14.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0120000000000001e-06 2024-09-14T09:08:14.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.532 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.532 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859305+0000", 2024-09-14T09:08:14.532 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01167812 2024-09-14T09:08:14.532 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.532 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:14.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859313+0000", 2024-09-14T09:08:14.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2409999999999995e-06 2024-09-14T09:08:14.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154081+0000", 2024-09-14T09:08:14.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.294768296 2024-09-14T09:08:14.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.533 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.533 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.533 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154280+0000", 2024-09-14T09:08:14.533 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019824 2024-09-14T09:08:14.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.534 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.534 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155063+0000", 2024-09-14T09:08:14.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00078303400000000005 2024-09-14T09:08:14.534 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.534 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.534 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.534 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.534 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106723 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 31744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.534 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:08:14.534 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637296290999998, 2024-09-14T09:08:14.534 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308293648, 2024-09-14T09:08:14.534 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.535 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.535 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.535 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.535 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.535 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.535 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106723 2024-09-14T09:08:14.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.535 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.535 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.535 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:08:14.535 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.535 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.535 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:08:14.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847631+0000", 2024-09-14T09:08:14.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6100000000000002e-07 2024-09-14T09:08:14.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.536 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.536 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847632+0000", 2024-09-14T09:08:14.536 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0589999999999999e-06 2024-09-14T09:08:14.536 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.536 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847635+0000", 2024-09-14T09:08:14.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.551e-06 2024-09-14T09:08:14.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847638+0000", 2024-09-14T09:08:14.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.644e-06 2024-09-14T09:08:14.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.537 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.537 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859344+0000", 2024-09-14T09:08:14.537 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011706397 2024-09-14T09:08:14.537 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.537 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:14.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859348+0000", 2024-09-14T09:08:14.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5779999999999999e-06 2024-09-14T09:08:14.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155135+0000", 2024-09-14T09:08:14.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.295787448 2024-09-14T09:08:14.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.538 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.538 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155279+0000", 2024-09-14T09:08:14.538 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014400700000000001 2024-09-14T09:08:14.538 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.538 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.539 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.539 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155924+0000", 2024-09-14T09:08:14.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00064470300000000005 2024-09-14T09:08:14.539 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.539 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.539 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.539 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.539 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.539 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106725 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 35840~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.539 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:08:14.539 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637285179999999, 2024-09-14T09:08:14.539 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.309103218, 2024-09-14T09:08:14.539 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.539 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.540 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.540 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.540 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.540 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.540 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106725 2024-09-14T09:08:14.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.540 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:08:14.540 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.540 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.540 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.540 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.540 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:08:14.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847642+0000", 2024-09-14T09:08:14.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4499999999999998e-07 2024-09-14T09:08:14.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.541 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.541 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847643+0000", 2024-09-14T09:08:14.541 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4e-07 2024-09-14T09:08:14.541 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.541 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847647+0000", 2024-09-14T09:08:14.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6190000000000002e-06 2024-09-14T09:08:14.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847655+0000", 2024-09-14T09:08:14.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:08:14.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.542 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.542 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859381+0000", 2024-09-14T09:08:14.542 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011725793 2024-09-14T09:08:14.542 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.542 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:14.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859391+0000", 2024-09-14T09:08:14.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4390000000000006e-06 2024-09-14T09:08:14.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155997+0000", 2024-09-14T09:08:14.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2966067830000001 2024-09-14T09:08:14.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.543 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.543 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.543 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156133+0000", 2024-09-14T09:08:14.543 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000135609 2024-09-14T09:08:14.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.544 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.544 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156745+0000", 2024-09-14T09:08:14.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000611592 2024-09-14T09:08:14.544 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.544 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.544 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.544 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.544 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106727 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 39936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.544 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:14.544 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637266879999999, 2024-09-14T09:08:14.544 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3099046560000001, 2024-09-14T09:08:14.545 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.545 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.545 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.545 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.545 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.545 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.545 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106727 2024-09-14T09:08:14.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.545 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.545 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.545 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.545 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:14.545 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:14.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:14.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1300000000000001e-07 2024-09-14T09:08:14.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.546 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.546 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.546 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847661+0000", 2024-09-14T09:08:14.546 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.192e-06 2024-09-14T09:08:14.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847665+0000", 2024-09-14T09:08:14.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0999999999999997e-06 2024-09-14T09:08:14.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847668+0000", 2024-09-14T09:08:14.547 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.644e-06 2024-09-14T09:08:14.547 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.547 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.547 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.547 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859422+0000", 2024-09-14T09:08:14.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011753559 2024-09-14T09:08:14.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:14.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859425+0000", 2024-09-14T09:08:14.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.343e-06 2024-09-14T09:08:14.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.548 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156817+0000", 2024-09-14T09:08:14.548 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2973924800000001 2024-09-14T09:08:14.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.548 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.548 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156956+0000", 2024-09-14T09:08:14.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013835 2024-09-14T09:08:14.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.549 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.549 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157564+0000", 2024-09-14T09:08:14.549 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060867500000000004 2024-09-14T09:08:14.549 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.549 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.549 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.549 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.549 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106729 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 44032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.549 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:14.550 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637254906999999, 2024-09-14T09:08:14.550 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3107195970000001, 2024-09-14T09:08:14.550 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.550 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.550 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.550 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.550 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.550 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.550 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106729 2024-09-14T09:08:14.550 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.550 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.550 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.550 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.550 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:14.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:14.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.551 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.551 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.551 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.551 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:14.551 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.29e-07 2024-09-14T09:08:14.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847673+0000", 2024-09-14T09:08:14.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.096e-06 2024-09-14T09:08:14.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.552 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847678+0000", 2024-09-14T09:08:14.552 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.527e-06 2024-09-14T09:08:14.552 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.552 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.552 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847681+0000", 2024-09-14T09:08:14.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.788e-06 2024-09-14T09:08:14.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859462+0000", 2024-09-14T09:08:14.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01178032 2024-09-14T09:08:14.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.553 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.553 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:14.553 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859471+0000", 2024-09-14T09:08:14.553 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3430000000000002e-06 2024-09-14T09:08:14.553 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157635+0000", 2024-09-14T09:08:14.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2981640320000001 2024-09-14T09:08:14.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157773+0000", 2024-09-14T09:08:14.554 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763 2024-09-14T09:08:14.554 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.554 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.554 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.554 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158391+0000", 2024-09-14T09:08:14.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061863200000000004 2024-09-14T09:08:14.555 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.555 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.555 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.555 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.555 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106731 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 48128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.555 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:08:14.555 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637239262000001, 2024-09-14T09:08:14.555 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3115142020000001, 2024-09-14T09:08:14.555 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.556 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.556 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.556 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.556 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.556 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.556 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106731 2024-09-14T09:08:14.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.556 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.556 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.556 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.556 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:08:14.556 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.556 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:08:14.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.557 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847688+0000", 2024-09-14T09:08:14.557 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0199999999999998e-07 2024-09-14T09:08:14.557 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.557 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.557 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847689+0000", 2024-09-14T09:08:14.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4529999999999999e-06 2024-09-14T09:08:14.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847694+0000", 2024-09-14T09:08:14.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9080000000000003e-06 2024-09-14T09:08:14.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.558 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.558 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847697+0000", 2024-09-14T09:08:14.558 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4920000000000002e-06 2024-09-14T09:08:14.558 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.558 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859499+0000", 2024-09-14T09:08:14.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011801676000000001 2024-09-14T09:08:14.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:14.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859502+0000", 2024-09-14T09:08:14.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2899999999999998e-06 2024-09-14T09:08:14.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.559 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.559 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.559 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158461+0000", 2024-09-14T09:08:14.559 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.298958796 2024-09-14T09:08:14.559 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158594+0000", 2024-09-14T09:08:14.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013321599999999999 2024-09-14T09:08:14.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.560 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.560 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.560 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159202+0000", 2024-09-14T09:08:14.560 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060706900000000001 2024-09-14T09:08:14.560 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.560 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.560 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.560 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.561 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.561 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106733 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 52224~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.561 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:14.561 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637225006000001, 2024-09-14T09:08:14.561 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3123343809999999, 2024-09-14T09:08:14.561 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.561 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.561 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.561 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.561 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.562 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.562 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106733 2024-09-14T09:08:14.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.562 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:14.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.562 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.562 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.562 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:14.562 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.562 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:14.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5800000000000001e-07 2024-09-14T09:08:14.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847703+0000", 2024-09-14T09:08:14.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.127e-06 2024-09-14T09:08:14.563 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.563 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.563 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.563 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847706+0000", 2024-09-14T09:08:14.563 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3210000000000001e-06 2024-09-14T09:08:14.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847709+0000", 2024-09-14T09:08:14.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7539999999999998e-06 2024-09-14T09:08:14.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.564 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859538+0000", 2024-09-14T09:08:14.564 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01182942 2024-09-14T09:08:14.564 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.564 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.564 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:14.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859547+0000", 2024-09-14T09:08:14.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T09:08:14.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159273+0000", 2024-09-14T09:08:14.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2997263700000001 2024-09-14T09:08:14.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.565 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.565 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.565 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159415+0000", 2024-09-14T09:08:14.565 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000141765 2024-09-14T09:08:14.565 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.566 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.566 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160036+0000", 2024-09-14T09:08:14.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062122100000000003 2024-09-14T09:08:14.566 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.566 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.566 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.566 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.566 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.566 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106735 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 56320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.566 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:14.566 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637213561999999, 2024-09-14T09:08:14.566 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3131339309999999, 2024-09-14T09:08:14.567 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.567 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.567 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.567 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.567 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.567 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.567 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106735 2024-09-14T09:08:14.567 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.567 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.567 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.567 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.567 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:14.567 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:14.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.568 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:14.568 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2100000000000001e-07 2024-09-14T09:08:14.568 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.568 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.568 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847714+0000", 2024-09-14T09:08:14.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.051e-06 2024-09-14T09:08:14.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847719+0000", 2024-09-14T09:08:14.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3640000000000001e-06 2024-09-14T09:08:14.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.569 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.569 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.569 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847723+0000", 2024-09-14T09:08:14.569 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8789999999999996e-06 2024-09-14T09:08:14.569 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859577+0000", 2024-09-14T09:08:14.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011854416 2024-09-14T09:08:14.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:14.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859580+0000", 2024-09-14T09:08:14.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.292e-06 2024-09-14T09:08:14.570 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.570 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.570 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.570 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160108+0000", 2024-09-14T09:08:14.570 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.30052758 2024-09-14T09:08:14.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160238+0000", 2024-09-14T09:08:14.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013043500000000001 2024-09-14T09:08:14.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.571 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.571 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.571 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160847+0000", 2024-09-14T09:08:14.571 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060869299999999999 2024-09-14T09:08:14.571 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.571 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.571 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.571 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.572 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.572 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106738 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 60416~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.572 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:14.572 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637200358000001, 2024-09-14T09:08:14.572 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3139482179999999, 2024-09-14T09:08:14.572 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.572 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.572 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.572 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.572 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.572 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.572 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106738 2024-09-14T09:08:14.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.573 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:14.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.573 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:14.573 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.573 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.573 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.573 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:14.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3999999999999998e-07 2024-09-14T09:08:14.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847733+0000", 2024-09-14T09:08:14.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8070000000000002e-06 2024-09-14T09:08:14.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.574 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.574 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847738+0000", 2024-09-14T09:08:14.574 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9740000000000001e-06 2024-09-14T09:08:14.574 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.574 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847742+0000", 2024-09-14T09:08:14.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6009999999999999e-06 2024-09-14T09:08:14.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859612+0000", 2024-09-14T09:08:14.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011870537 2024-09-14T09:08:14.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.575 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:14.575 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859621+0000", 2024-09-14T09:08:14.575 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3019999999999995e-06 2024-09-14T09:08:14.575 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.575 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160917+0000", 2024-09-14T09:08:14.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3012961080000001 2024-09-14T09:08:14.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161054+0000", 2024-09-14T09:08:14.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000137422 2024-09-14T09:08:14.576 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.576 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.576 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.576 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161674+0000", 2024-09-14T09:08:14.576 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000620227 2024-09-14T09:08:14.576 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.577 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.577 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.577 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.577 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.577 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106739 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 64512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.577 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:14.577 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637180518000001, 2024-09-14T09:08:14.577 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.314753777, 2024-09-14T09:08:14.577 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.577 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.577 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.577 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.577 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.578 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.578 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106739 2024-09-14T09:08:14.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.578 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:14.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.578 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.578 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.578 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.578 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:14.578 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:14.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9700000000000003e-07 2024-09-14T09:08:14.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847748+0000", 2024-09-14T09:08:14.579 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2589999999999999e-06 2024-09-14T09:08:14.579 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.579 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.579 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.579 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847752+0000", 2024-09-14T09:08:14.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6099999999999999e-06 2024-09-14T09:08:14.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847755+0000", 2024-09-14T09:08:14.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0520000000000002e-06 2024-09-14T09:08:14.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.580 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859653+0000", 2024-09-14T09:08:14.580 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011897575000000001 2024-09-14T09:08:14.580 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.580 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.580 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:14.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859658+0000", 2024-09-14T09:08:14.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8810000000000004e-06 2024-09-14T09:08:14.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161745+0000", 2024-09-14T09:08:14.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3020870819999999 2024-09-14T09:08:14.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.581 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.581 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.581 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161877+0000", 2024-09-14T09:08:14.581 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000132657 2024-09-14T09:08:14.581 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.582 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.582 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162500+0000", 2024-09-14T09:08:14.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062236400000000003 2024-09-14T09:08:14.582 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.582 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.582 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.582 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.582 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.582 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106741 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 68608~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.582 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:14.582 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637167726000001, 2024-09-14T09:08:14.582 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3155568449999999, 2024-09-14T09:08:14.583 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.583 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.583 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.583 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.583 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106741 2024-09-14T09:08:14.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.583 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.583 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.583 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.583 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:14.583 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.583 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:14.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:14.584 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5800000000000001e-07 2024-09-14T09:08:14.584 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.584 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.584 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.584 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847760+0000", 2024-09-14T09:08:14.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:08:14.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847764+0000", 2024-09-14T09:08:14.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3189999999999999e-06 2024-09-14T09:08:14.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.585 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847766+0000", 2024-09-14T09:08:14.585 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6429999999999999e-06 2024-09-14T09:08:14.585 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.585 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.585 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859693+0000", 2024-09-14T09:08:14.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011926724 2024-09-14T09:08:14.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:14.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859703+0000", 2024-09-14T09:08:14.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6160000000000008e-06 2024-09-14T09:08:14.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.586 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.586 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162575+0000", 2024-09-14T09:08:14.586 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.302872128 2024-09-14T09:08:14.586 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.586 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162709+0000", 2024-09-14T09:08:14.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013446000000000001 2024-09-14T09:08:14.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.587 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.587 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163316+0000", 2024-09-14T09:08:14.587 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060650700000000005 2024-09-14T09:08:14.587 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.587 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.587 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.587 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.587 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.587 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106744 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 76800~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:14.587 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:08:14.587 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 52.637156615000002, 2024-09-14T09:08:14.588 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3163901789999999, 2024-09-14T09:08:14.588 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:14.588 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:14.588 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:14.588 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:14.588 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:14.588 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:14.588 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106744 2024-09-14T09:08:14.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.588 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:14.588 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.588 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:14.588 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:08:14.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:14.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:08:14.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:14.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:14.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996947+0000", 2024-09-14T09:08:14.589 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.149177154 2024-09-14T09:08:14.589 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.589 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.589 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:14.589 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996964+0000", 2024-09-14T09:08:14.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.679e-05 2024-09-14T09:08:14.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:14.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996999+0000", 2024-09-14T09:08:14.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5463000000000002e-05 2024-09-14T09:08:14.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:14.590 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.997029+0000", 2024-09-14T09:08:14.590 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9567e-05 2024-09-14T09:08:14.590 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.590 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.590 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:14.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163390+0000", 2024-09-14T09:08:14.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.16636056799999999 2024-09-14T09:08:14.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:14.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163526+0000", 2024-09-14T09:08:14.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013655899999999999 2024-09-14T09:08:14.591 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:14.591 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:14.591 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:14.591 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.164160+0000", 2024-09-14T09:08:14.591 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00063407800000000005 2024-09-14T09:08:14.591 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.591 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.591 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.591 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:14.592 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:14.592 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:14.593 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:08:14.594 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:14.591+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:14.694 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:14.695+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:14.763 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:14.763 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:14.763 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:14.763 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:14.777 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:08:14.795 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:14.798+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:14.896 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:14.898+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:14.942 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:14.943 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:14.943 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:08:14.943 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:08:14.943 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:14.957 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:08:14.997 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:14.998+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:15.083 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:08:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:08:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:08:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:08:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 161.99828018100001, 2024-09-14T09:08:15.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:08:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:08:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:08:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:08:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:08:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:08:15.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:08:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:08:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:08:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:08:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:08:15.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:08:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:08:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:08:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:08:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:08:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:08:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:08:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:08:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:08:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:08:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:08:15.088 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:08:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 161.998225523, 2024-09-14T09:08:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:08:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:15.089 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:08:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:08:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:08:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:08:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:08:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:08:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:08:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:08:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:08:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:08:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:08:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:08:15.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:08:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:08:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:08:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:08:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:08:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:08:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:08:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:08:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:08:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:08:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 161.99812014899999, 2024-09-14T09:08:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:08:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:08:15.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:08:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:08:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:08:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:08:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:08:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:08:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:08:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:08:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:08:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:08:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:08:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:08:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:08:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:08:15.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:08:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:08:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:08:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:08:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:08:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:08:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:08:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 161.99776338500001, 2024-09-14T09:08:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:08:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.100 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:08:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:08:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:08:15.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:08:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:08:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:08:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:08:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:08:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:08:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:08:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:08:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:08:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:08:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:08:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:08:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:08:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:08:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:08:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:08:15.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:08:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:08:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:08:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 161.99767864699999, 2024-09-14T09:08:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:08:15.105 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:08:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:08:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:08:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:08:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:08:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:08:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:08:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:08:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:08:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:08:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:08:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:08:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:08:15.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:08:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:08:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:08:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:08:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:08:15.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:08:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:08:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:08:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:08:15.110 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 161.99759096099999, 2024-09-14T09:08:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:08:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:08:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:08:15.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:08:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:08:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:08:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:08:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:08:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:08:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:08:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:08:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:08:15.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:08:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:08:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:08:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:08:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:08:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:08:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:08:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:08:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:08:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:08:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:08:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:08:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 128.921298974, 2024-09-14T09:08:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:08:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.116 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:08:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:08:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:08:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:08:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:08:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:08:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:08:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:08:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:08:15.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:08:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:08:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:08:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:08:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:08:15.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:08:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:08:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:08:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:08:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:08:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:08:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:15.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:08:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:08:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:08:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:08:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:08:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:08:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:97796 3.15 3:aaaa5f7f:::rbd_data.115e113dc8ca.0000000000000029:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:08:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:08:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.987430103999998, 2024-09-14T09:08:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14537731300000001, 2024-09-14T09:08:15.122 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97796 2024-09-14T09:08:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:08:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:08:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094482+0000", 2024-09-14T09:08:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6400000000000001e-07 2024-09-14T09:08:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094485+0000", 2024-09-14T09:08:15.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.693e-06 2024-09-14T09:08:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094494+0000", 2024-09-14T09:08:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9590000000000001e-06 2024-09-14T09:08:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094502+0000", 2024-09-14T09:08:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3180000000000001e-06 2024-09-14T09:08:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094567+0000", 2024-09-14T09:08:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5526999999999998e-05 2024-09-14T09:08:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094726+0000", 2024-09-14T09:08:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015842499999999999 2024-09-14T09:08:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:08:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.101940+0000", 2024-09-14T09:08:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0072146559999999998 2024-09-14T09:08:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239665+0000", 2024-09-14T09:08:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.137724501 2024-09-14T09:08:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239726+0000", 2024-09-14T09:08:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1550999999999994e-05 2024-09-14T09:08:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239800+0000", 2024-09-14T09:08:15.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3318000000000005e-05 2024-09-14T09:08:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239858+0000", 2024-09-14T09:08:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8600999999999997e-05 2024-09-14T09:08:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:97796 3.15 e106/103 ondisk, result = 0)", 2024-09-14T09:08:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:08:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.965084234999999, 2024-09-14T09:08:15.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123043465, 2024-09-14T09:08:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:08:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:08:15.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116828+0000", 2024-09-14T09:08:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8100000000000002e-07 2024-09-14T09:08:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116831+0000", 2024-09-14T09:08:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3009999999999998e-06 2024-09-14T09:08:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.130 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116841+0000", 2024-09-14T09:08:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7829999999999993e-06 2024-09-14T09:08:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116851+0000", 2024-09-14T09:08:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0135999999999999e-05 2024-09-14T09:08:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239707+0000", 2024-09-14T09:08:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12285581499999999 2024-09-14T09:08:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239718+0000", 2024-09-14T09:08:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1804000000000001e-05 2024-09-14T09:08:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239870+0000", 2024-09-14T09:08:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015194500000000001 2024-09-14T09:08:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117315 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 309248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:08:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.358448498000001, 2024-09-14T09:08:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11247383800000001, 2024-09-14T09:08:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117315 2024-09-14T09:08:15.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:08:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:08:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723899+0000", 2024-09-14T09:08:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043641699999999998 2024-09-14T09:08:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723903+0000", 2024-09-14T09:08:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5389999999999999e-06 2024-09-14T09:08:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723910+0000", 2024-09-14T09:08:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4229999999999999e-06 2024-09-14T09:08:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723917+0000", 2024-09-14T09:08:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4560000000000003e-06 2024-09-14T09:08:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835514+0000", 2024-09-14T09:08:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111596258 2024-09-14T09:08:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835571+0000", 2024-09-14T09:08:15.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7317e-05 2024-09-14T09:08:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835936+0000", 2024-09-14T09:08:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000365428 2024-09-14T09:08:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117317 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 313344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:08:15.137 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.357981482, 2024-09-14T09:08:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112450201, 2024-09-14T09:08:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117317 2024-09-14T09:08:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:08:15.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:08:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724396+0000", 2024-09-14T09:08:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000466028 2024-09-14T09:08:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724401+0000", 2024-09-14T09:08:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2299999999999999e-06 2024-09-14T09:08:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724412+0000", 2024-09-14T09:08:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0762e-05 2024-09-14T09:08:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724419+0000", 2024-09-14T09:08:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1010000000000003e-06 2024-09-14T09:08:15.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835981+0000", 2024-09-14T09:08:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111562174 2024-09-14T09:08:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836070+0000", 2024-09-14T09:08:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8839999999999996e-05 2024-09-14T09:08:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836380+0000", 2024-09-14T09:08:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031006600000000001 2024-09-14T09:08:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117319 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 317440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:08:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.357476845999997, 2024-09-14T09:08:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112467763, 2024-09-14T09:08:15.142 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117319 2024-09-14T09:08:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:08:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:08:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724846+0000", 2024-09-14T09:08:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00041155400000000001 2024-09-14T09:08:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724854+0000", 2024-09-14T09:08:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5469999999999997e-06 2024-09-14T09:08:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724862+0000", 2024-09-14T09:08:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1289999999999996e-06 2024-09-14T09:08:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724867+0000", 2024-09-14T09:08:15.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8720000000000001e-06 2024-09-14T09:08:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836486+0000", 2024-09-14T09:08:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111618357 2024-09-14T09:08:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836554+0000", 2024-09-14T09:08:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8378000000000002e-05 2024-09-14T09:08:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836902+0000", 2024-09-14T09:08:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034792599999999998 2024-09-14T09:08:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117321 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 321536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:08:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.357032521999997, 2024-09-14T09:08:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11249685600000001, 2024-09-14T09:08:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.147 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117321 2024-09-14T09:08:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:08:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:08:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725241+0000", 2024-09-14T09:08:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036211099999999998 2024-09-14T09:08:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725245+0000", 2024-09-14T09:08:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9179999999999996e-06 2024-09-14T09:08:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725254+0000", 2024-09-14T09:08:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0699999999999996e-06 2024-09-14T09:08:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725260+0000", 2024-09-14T09:08:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1099999999999999e-06 2024-09-14T09:08:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836935+0000", 2024-09-14T09:08:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111674997 2024-09-14T09:08:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837021+0000", 2024-09-14T09:08:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5931000000000001e-05 2024-09-14T09:08:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837375+0000", 2024-09-14T09:08:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035471899999999998 2024-09-14T09:08:15.151 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117323 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:08:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.356638297000003, 2024-09-14T09:08:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11250631, 2024-09-14T09:08:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:15.152 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117323 2024-09-14T09:08:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:08:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:08:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725941+0000", 2024-09-14T09:08:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00066866500000000004 2024-09-14T09:08:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725946+0000", 2024-09-14T09:08:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9060000000000001e-06 2024-09-14T09:08:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725959+0000", 2024-09-14T09:08:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2622e-05 2024-09-14T09:08:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725968+0000", 2024-09-14T09:08:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5660000000000007e-06 2024-09-14T09:08:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837408+0000", 2024-09-14T09:08:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111440626 2024-09-14T09:08:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837495+0000", 2024-09-14T09:08:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6654000000000001e-05 2024-09-14T09:08:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837779+0000", 2024-09-14T09:08:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000284271 2024-09-14T09:08:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117327 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 333824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:08:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.355902104999998, 2024-09-14T09:08:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112409908, 2024-09-14T09:08:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117327 2024-09-14T09:08:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.157 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:08:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:08:15.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726303+0000", 2024-09-14T09:08:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029388199999999998 2024-09-14T09:08:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726307+0000", 2024-09-14T09:08:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4320000000000001e-06 2024-09-14T09:08:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726317+0000", 2024-09-14T09:08:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4609999999999997e-06 2024-09-14T09:08:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726324+0000", 2024-09-14T09:08:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0480000000000001e-06 2024-09-14T09:08:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838051+0000", 2024-09-14T09:08:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111727534 2024-09-14T09:08:15.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838143+0000", 2024-09-14T09:08:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1359000000000004e-05 2024-09-14T09:08:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838419+0000", 2024-09-14T09:08:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027619199999999997 2024-09-14T09:08:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117329 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 337920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:08:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.355575346000002, 2024-09-14T09:08:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112411122, 2024-09-14T09:08:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117329 2024-09-14T09:08:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:08:15.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:08:15.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726742+0000", 2024-09-14T09:08:15.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040626099999999999 2024-09-14T09:08:15.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726748+0000", 2024-09-14T09:08:15.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6860000000000003e-06 2024-09-14T09:08:15.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726761+0000", 2024-09-14T09:08:15.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3599e-05 2024-09-14T09:08:15.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726770+0000", 2024-09-14T09:08:15.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5990000000000002e-06 2024-09-14T09:08:15.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838443+0000", 2024-09-14T09:08:15.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111672664 2024-09-14T09:08:15.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838487+0000", 2024-09-14T09:08:15.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4249000000000003e-05 2024-09-14T09:08:15.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838747+0000", 2024-09-14T09:08:15.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026006400000000001 2024-09-14T09:08:15.166 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.166 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.166 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.166 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117331 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 342016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:15.166 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:08:15.166 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.355107750999998, 2024-09-14T09:08:15.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112291139, 2024-09-14T09:08:15.166 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.166 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.166 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:15.167 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:15.167 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:15.167 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.167 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117331 2024-09-14T09:08:15.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.167 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:08:15.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:08:15.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727158+0000", 2024-09-14T09:08:15.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035434000000000003 2024-09-14T09:08:15.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.168 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727162+0000", 2024-09-14T09:08:15.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5449999999999997e-06 2024-09-14T09:08:15.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727174+0000", 2024-09-14T09:08:15.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2278e-05 2024-09-14T09:08:15.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727183+0000", 2024-09-14T09:08:15.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3820000000000004e-06 2024-09-14T09:08:15.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838788+0000", 2024-09-14T09:08:15.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111604748 2024-09-14T09:08:15.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838847+0000", 2024-09-14T09:08:15.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8983000000000001e-05 2024-09-14T09:08:15.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839094+0000", 2024-09-14T09:08:15.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024786300000000002 2024-09-14T09:08:15.170 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.170 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.170 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.171 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117333 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 346112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:15.171 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:08:15.171 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.354713940000003, 2024-09-14T09:08:15.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112319711, 2024-09-14T09:08:15.171 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.171 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.171 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:15.171 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:15.171 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:15.171 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.171 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117333 2024-09-14T09:08:15.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.171 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:08:15.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:08:15.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727785+0000", 2024-09-14T09:08:15.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000587512 2024-09-14T09:08:15.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727789+0000", 2024-09-14T09:08:15.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.352e-06 2024-09-14T09:08:15.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727800+0000", 2024-09-14T09:08:15.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1369000000000001e-05 2024-09-14T09:08:15.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727807+0000", 2024-09-14T09:08:15.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1300000000000003e-06 2024-09-14T09:08:15.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839137+0000", 2024-09-14T09:08:15.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111329898 2024-09-14T09:08:15.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839226+0000", 2024-09-14T09:08:15.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8337000000000005e-05 2024-09-14T09:08:15.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839517+0000", 2024-09-14T09:08:15.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029111300000000002 2024-09-14T09:08:15.175 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.175 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.175 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.175 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117363 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 407552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:15.175 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:08:15.175 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.348617840999999, 2024-09-14T09:08:15.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112191306, 2024-09-14T09:08:15.175 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.175 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.175 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:15.176 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:15.176 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:15.176 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.176 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117363 2024-09-14T09:08:15.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.176 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:08:15.176 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.176 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.176 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.176 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.176 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:08:15.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733666+0000", 2024-09-14T09:08:15.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00037284500000000001 2024-09-14T09:08:15.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.177 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.177 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733671+0000", 2024-09-14T09:08:15.177 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9269999999999996e-06 2024-09-14T09:08:15.177 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.177 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733681+0000", 2024-09-14T09:08:15.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5599999999999999e-06 2024-09-14T09:08:15.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733689+0000", 2024-09-14T09:08:15.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2649999999999996e-06 2024-09-14T09:08:15.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.178 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.178 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845028+0000", 2024-09-14T09:08:15.178 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111338933 2024-09-14T09:08:15.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.178 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845094+0000", 2024-09-14T09:08:15.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6749999999999996e-05 2024-09-14T09:08:15.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.179 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.179 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845485+0000", 2024-09-14T09:08:15.179 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00039002600000000002 2024-09-14T09:08:15.179 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.179 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.179 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.179 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.179 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117365 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 411648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:15.180 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:08:15.180 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 43.348208075999999, 2024-09-14T09:08:15.180 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112271685, 2024-09-14T09:08:15.180 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:15.180 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:15.180 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:15.180 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:15.180 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:15.180 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.180 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117365 2024-09-14T09:08:15.180 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.180 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:15.180 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.180 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:15.180 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:08:15.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:15.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:08:15.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:15.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:15.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734147+0000", 2024-09-14T09:08:15.181 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00044374399999999999 2024-09-14T09:08:15.181 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.181 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.181 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:15.181 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734152+0000", 2024-09-14T09:08:15.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9139999999999999e-06 2024-09-14T09:08:15.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:15.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734164+0000", 2024-09-14T09:08:15.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2299e-05 2024-09-14T09:08:15.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.182 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734172+0000", 2024-09-14T09:08:15.182 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0390000000000005e-06 2024-09-14T09:08:15.182 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.182 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.182 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:15.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845528+0000", 2024-09-14T09:08:15.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11135639899999999 2024-09-14T09:08:15.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:15.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845602+0000", 2024-09-14T09:08:15.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3957999999999999e-05 2024-09-14T09:08:15.183 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:15.183 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:15.183 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:15.183 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845975+0000", 2024-09-14T09:08:15.183 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00037233200000000001 2024-09-14T09:08:15.183 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.183 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.184 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.184 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:15.184 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:15.184 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:15.185 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:08:15.186 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:15.183+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:15.286 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:15.287+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:15.294 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T09:08:15.327 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:15.327 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:15.327 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:08:15.327 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:15.341 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:08:15.387 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:15.387+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:15.468 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:15.468 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:15.468 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:08:15.468 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:08:15.468 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:15.483 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:08:15.488 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:15.487+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:15.590 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:15.594+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:15.612 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:15.612 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:08:15.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:08:15.612 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:08:15.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.612 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105615 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 382976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.612 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:08:15.612 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.750882867000001, 2024-09-14T09:08:15.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117270900000001, 2024-09-14T09:08:15.613 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.613 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.613 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:15.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.613 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105615 2024-09-14T09:08:15.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.613 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:08:15.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:08:15.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864368+0000", 2024-09-14T09:08:15.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.043e-06 2024-09-14T09:08:15.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864373+0000", 2024-09-14T09:08:15.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4669999999999998e-06 2024-09-14T09:08:15.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864387+0000", 2024-09-14T09:08:15.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3876000000000001e-05 2024-09-14T09:08:15.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864397+0000", 2024-09-14T09:08:15.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0592e-05 2024-09-14T09:08:15.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866069+0000", 2024-09-14T09:08:15.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016714410000000001 2024-09-14T09:08:15.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866076+0000", 2024-09-14T09:08:15.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:08:15.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.075854+0000", 2024-09-14T09:08:15.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.209778501 2024-09-14T09:08:15.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:15.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.075940+0000", 2024-09-14T09:08:15.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5357000000000003e-05 2024-09-14T09:08:15.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.076094+0000", 2024-09-14T09:08:15.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000154745 2024-09-14T09:08:15.617 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.617 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.617 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.618 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105629 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 411648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.618 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:08:15.618 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.748729144999999, 2024-09-14T09:08:15.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2115591910000001, 2024-09-14T09:08:15.618 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.618 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.618 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:15.618 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.618 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.618 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.618 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105629 2024-09-14T09:08:15.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.618 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:08:15.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:08:15.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866522+0000", 2024-09-14T09:08:15.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6899999999999996e-07 2024-09-14T09:08:15.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866526+0000", 2024-09-14T09:08:15.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0110000000000002e-06 2024-09-14T09:08:15.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866539+0000", 2024-09-14T09:08:15.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3013000000000001e-05 2024-09-14T09:08:15.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866549+0000", 2024-09-14T09:08:15.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8490000000000001e-06 2024-09-14T09:08:15.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866650+0000", 2024-09-14T09:08:15.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101054 2024-09-14T09:08:15.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866664+0000", 2024-09-14T09:08:15.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3805000000000001e-05 2024-09-14T09:08:15.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.077849+0000", 2024-09-14T09:08:15.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2111855419999999 2024-09-14T09:08:15.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:15.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.077945+0000", 2024-09-14T09:08:15.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6086000000000001e-05 2024-09-14T09:08:15.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078080+0000", 2024-09-14T09:08:15.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000134862 2024-09-14T09:08:15.622 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.622 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.623 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.623 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105631 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 415744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.623 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:08:15.623 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.748669925999998, 2024-09-14T09:08:15.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119463509999999, 2024-09-14T09:08:15.623 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.623 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.623 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:15.623 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.623 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.623 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.624 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105631 2024-09-14T09:08:15.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.624 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:08:15.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:08:15.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866581+0000", 2024-09-14T09:08:15.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.879999999999999e-07 2024-09-14T09:08:15.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866585+0000", 2024-09-14T09:08:15.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0239999999999999e-06 2024-09-14T09:08:15.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866598+0000", 2024-09-14T09:08:15.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2862e-05 2024-09-14T09:08:15.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866608+0000", 2024-09-14T09:08:15.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9890000000000004e-06 2024-09-14T09:08:15.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866708+0000", 2024-09-14T09:08:15.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9605999999999995e-05 2024-09-14T09:08:15.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866720+0000", 2024-09-14T09:08:15.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.261e-05 2024-09-14T09:08:15.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078137+0000", 2024-09-14T09:08:15.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211416571 2024-09-14T09:08:15.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:15.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078293+0000", 2024-09-14T09:08:15.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000156 2024-09-14T09:08:15.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078527+0000", 2024-09-14T09:08:15.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000233701 2024-09-14T09:08:15.628 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.628 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.628 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.628 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105633 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.628 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:08:15.628 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.748610878999997, 2024-09-14T09:08:15.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211941734, 2024-09-14T09:08:15.628 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.628 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.628 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:15.628 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.628 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.628 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.629 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105633 2024-09-14T09:08:15.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.629 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:08:15.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:08:15.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866640+0000", 2024-09-14T09:08:15.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9300000000000006e-07 2024-09-14T09:08:15.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866656+0000", 2024-09-14T09:08:15.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5291e-05 2024-09-14T09:08:15.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866669+0000", 2024-09-14T09:08:15.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2918000000000001e-05 2024-09-14T09:08:15.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866678+0000", 2024-09-14T09:08:15.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8369999999999992e-06 2024-09-14T09:08:15.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866771+0000", 2024-09-14T09:08:15.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2250999999999995e-05 2024-09-14T09:08:15.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866784+0000", 2024-09-14T09:08:15.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2991e-05 2024-09-14T09:08:15.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078566+0000", 2024-09-14T09:08:15.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117822570000001 2024-09-14T09:08:15.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078581+0000", 2024-09-14T09:08:15.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5196e-05 2024-09-14T09:08:15.632 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.632 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.632 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.632 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105635 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.633 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:08:15.633 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.748557052000002, 2024-09-14T09:08:15.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119313250000001, 2024-09-14T09:08:15.633 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.633 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.633 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:15.633 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.633 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.633 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.633 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105635 2024-09-14T09:08:15.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.636 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:08:15.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:08:15.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866694+0000", 2024-09-14T09:08:15.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0210000000000001e-06 2024-09-14T09:08:15.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866698+0000", 2024-09-14T09:08:15.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.765e-06 2024-09-14T09:08:15.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866710+0000", 2024-09-14T09:08:15.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2320000000000001e-05 2024-09-14T09:08:15.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866722+0000", 2024-09-14T09:08:15.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2133000000000001e-05 2024-09-14T09:08:15.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866830+0000", 2024-09-14T09:08:15.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000107668 2024-09-14T09:08:15.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866843+0000", 2024-09-14T09:08:15.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2991e-05 2024-09-14T09:08:15.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078616+0000", 2024-09-14T09:08:15.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117732459999999 2024-09-14T09:08:15.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078624+0000", 2024-09-14T09:08:15.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1810000000000001e-06 2024-09-14T09:08:15.639 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.639 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.639 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.639 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105636 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.639 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:08:15.639 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.748489155999998, 2024-09-14T09:08:15.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119085119999999, 2024-09-14T09:08:15.640 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.640 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.640 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:15.640 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.640 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.640 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.640 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105636 2024-09-14T09:08:15.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.640 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:08:15.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:08:15.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866762+0000", 2024-09-14T09:08:15.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4600000000000003e-07 2024-09-14T09:08:15.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866766+0000", 2024-09-14T09:08:15.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:08:15.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866779+0000", 2024-09-14T09:08:15.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3101999999999999e-05 2024-09-14T09:08:15.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866789+0000", 2024-09-14T09:08:15.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8039999999999996e-06 2024-09-14T09:08:15.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866889+0000", 2024-09-14T09:08:15.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100147 2024-09-14T09:08:15.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866902+0000", 2024-09-14T09:08:15.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3212e-05 2024-09-14T09:08:15.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078659+0000", 2024-09-14T09:08:15.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211756362 2024-09-14T09:08:15.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078670+0000", 2024-09-14T09:08:15.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0956e-05 2024-09-14T09:08:15.643 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.644 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.644 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.644 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105639 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.644 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:08:15.644 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.748430280999997, 2024-09-14T09:08:15.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118829259999999, 2024-09-14T09:08:15.644 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.644 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.644 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:15.644 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.644 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.644 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.645 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105639 2024-09-14T09:08:15.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.645 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:08:15.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:08:15.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866821+0000", 2024-09-14T09:08:15.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9800000000000002e-07 2024-09-14T09:08:15.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866825+0000", 2024-09-14T09:08:15.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9269999999999998e-06 2024-09-14T09:08:15.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866838+0000", 2024-09-14T09:08:15.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3040000000000001e-05 2024-09-14T09:08:15.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866848+0000", 2024-09-14T09:08:15.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7929999999999992e-06 2024-09-14T09:08:15.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866948+0000", 2024-09-14T09:08:15.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100337 2024-09-14T09:08:15.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866961+0000", 2024-09-14T09:08:15.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3106e-05 2024-09-14T09:08:15.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078693+0000", 2024-09-14T09:08:15.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117320380000001 2024-09-14T09:08:15.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078703+0000", 2024-09-14T09:08:15.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6870000000000006e-06 2024-09-14T09:08:15.648 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.648 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.648 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.649 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105641 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.649 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:08:15.649 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.748371415999998, 2024-09-14T09:08:15.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118727169999999, 2024-09-14T09:08:15.649 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.649 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.649 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:15.649 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.649 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.649 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.649 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105641 2024-09-14T09:08:15.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.650 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:08:15.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:08:15.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866880+0000", 2024-09-14T09:08:15.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7699999999999992e-07 2024-09-14T09:08:15.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866884+0000", 2024-09-14T09:08:15.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9480000000000001e-06 2024-09-14T09:08:15.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866897+0000", 2024-09-14T09:08:15.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2832e-05 2024-09-14T09:08:15.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866906+0000", 2024-09-14T09:08:15.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7869999999999996e-06 2024-09-14T09:08:15.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867007+0000", 2024-09-14T09:08:15.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010103900000000001 2024-09-14T09:08:15.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867020+0000", 2024-09-14T09:08:15.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2896e-05 2024-09-14T09:08:15.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078739+0000", 2024-09-14T09:08:15.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211718305 2024-09-14T09:08:15.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078752+0000", 2024-09-14T09:08:15.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2933e-05 2024-09-14T09:08:15.653 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.653 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.654 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.654 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105643 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.654 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:08:15.654 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.748312857000002, 2024-09-14T09:08:15.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211858353, 2024-09-14T09:08:15.654 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.654 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.654 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:15.654 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.654 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.654 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.655 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105643 2024-09-14T09:08:15.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.655 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:08:15.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:08:15.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866938+0000", 2024-09-14T09:08:15.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5600000000000004e-07 2024-09-14T09:08:15.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866942+0000", 2024-09-14T09:08:15.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0350000000000003e-06 2024-09-14T09:08:15.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866955+0000", 2024-09-14T09:08:15.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3097e-05 2024-09-14T09:08:15.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866965+0000", 2024-09-14T09:08:15.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8179999999999999e-06 2024-09-14T09:08:15.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867067+0000", 2024-09-14T09:08:15.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010128699999999999 2024-09-14T09:08:15.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867080+0000", 2024-09-14T09:08:15.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3089e-05 2024-09-14T09:08:15.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078786+0000", 2024-09-14T09:08:15.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117065250000001 2024-09-14T09:08:15.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078796+0000", 2024-09-14T09:08:15.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5459999999999997e-06 2024-09-14T09:08:15.658 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.658 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.658 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.658 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105645 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.659 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:08:15.659 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.748252461999996, 2024-09-14T09:08:15.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211848641, 2024-09-14T09:08:15.659 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.659 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.659 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:15.659 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.659 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.659 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.659 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105645 2024-09-14T09:08:15.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.659 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:08:15.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:08:15.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866999+0000", 2024-09-14T09:08:15.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7999999999999993e-07 2024-09-14T09:08:15.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867003+0000", 2024-09-14T09:08:15.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0090000000000001e-06 2024-09-14T09:08:15.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867016+0000", 2024-09-14T09:08:15.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3304e-05 2024-09-14T09:08:15.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867026+0000", 2024-09-14T09:08:15.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5889999999999992e-06 2024-09-14T09:08:15.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867128+0000", 2024-09-14T09:08:15.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000102101 2024-09-14T09:08:15.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867141+0000", 2024-09-14T09:08:15.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2903e-05 2024-09-14T09:08:15.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078833+0000", 2024-09-14T09:08:15.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116920959999999 2024-09-14T09:08:15.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078846+0000", 2024-09-14T09:08:15.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3658999999999999e-05 2024-09-14T09:08:15.663 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.663 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.663 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.664 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105647 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.664 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:08:15.664 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.748193944, 2024-09-14T09:08:15.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211830097, 2024-09-14T09:08:15.664 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.664 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.664 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:15.664 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.664 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.664 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.664 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105647 2024-09-14T09:08:15.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.665 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:08:15.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:08:15.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867057+0000", 2024-09-14T09:08:15.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.95e-07 2024-09-14T09:08:15.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867061+0000", 2024-09-14T09:08:15.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9530000000000001e-06 2024-09-14T09:08:15.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867074+0000", 2024-09-14T09:08:15.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3061000000000001e-05 2024-09-14T09:08:15.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867086+0000", 2024-09-14T09:08:15.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1460999999999999e-05 2024-09-14T09:08:15.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867186+0000", 2024-09-14T09:08:15.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100181 2024-09-14T09:08:15.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867199+0000", 2024-09-14T09:08:15.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-05 2024-09-14T09:08:15.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078874+0000", 2024-09-14T09:08:15.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211674683 2024-09-14T09:08:15.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078886+0000", 2024-09-14T09:08:15.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2853e-05 2024-09-14T09:08:15.668 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.668 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.668 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.668 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105648 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.669 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:08:15.669 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.748132458000001, 2024-09-14T09:08:15.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118073570000001, 2024-09-14T09:08:15.669 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.669 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.669 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:15.669 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.669 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.669 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.669 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105648 2024-09-14T09:08:15.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.669 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:08:15.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:08:15.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867119+0000", 2024-09-14T09:08:15.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8400000000000002e-07 2024-09-14T09:08:15.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867123+0000", 2024-09-14T09:08:15.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9639999999999997e-06 2024-09-14T09:08:15.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867136+0000", 2024-09-14T09:08:15.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.289e-05 2024-09-14T09:08:15.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867145+0000", 2024-09-14T09:08:15.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8919999999999995e-06 2024-09-14T09:08:15.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867245+0000", 2024-09-14T09:08:15.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9282000000000003e-05 2024-09-14T09:08:15.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867258+0000", 2024-09-14T09:08:15.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2935000000000001e-05 2024-09-14T09:08:15.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078910+0000", 2024-09-14T09:08:15.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211652352 2024-09-14T09:08:15.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078925+0000", 2024-09-14T09:08:15.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5058000000000001e-05 2024-09-14T09:08:15.673 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.673 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.673 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.673 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105652 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.673 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:08:15.673 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.748074439, 2024-09-14T09:08:15.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211775737, 2024-09-14T09:08:15.674 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.674 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.674 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:15.674 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.674 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.674 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.674 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105652 2024-09-14T09:08:15.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.674 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:08:15.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:08:15.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867177+0000", 2024-09-14T09:08:15.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9100000000000002e-07 2024-09-14T09:08:15.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867181+0000", 2024-09-14T09:08:15.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0029999999999996e-06 2024-09-14T09:08:15.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867193+0000", 2024-09-14T09:08:15.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2675e-05 2024-09-14T09:08:15.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867203+0000", 2024-09-14T09:08:15.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5599999999999999e-06 2024-09-14T09:08:15.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867306+0000", 2024-09-14T09:08:15.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010330100000000001 2024-09-14T09:08:15.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867319+0000", 2024-09-14T09:08:15.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3129000000000001e-05 2024-09-14T09:08:15.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078944+0000", 2024-09-14T09:08:15.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116247520000001 2024-09-14T09:08:15.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078952+0000", 2024-09-14T09:08:15.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4259999999999997e-06 2024-09-14T09:08:15.678 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.678 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.678 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.678 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105653 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.678 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:08:15.679 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.748015768999998, 2024-09-14T09:08:15.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211774895, 2024-09-14T09:08:15.679 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.679 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.679 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:15.679 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.679 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.679 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.679 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105653 2024-09-14T09:08:15.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.679 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:08:15.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:08:15.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867235+0000", 2024-09-14T09:08:15.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9400000000000004e-07 2024-09-14T09:08:15.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867239+0000", 2024-09-14T09:08:15.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9380000000000002e-06 2024-09-14T09:08:15.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867252+0000", 2024-09-14T09:08:15.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.31e-05 2024-09-14T09:08:15.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867264+0000", 2024-09-14T09:08:15.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1275e-05 2024-09-14T09:08:15.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867367+0000", 2024-09-14T09:08:15.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103755 2024-09-14T09:08:15.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867382+0000", 2024-09-14T09:08:15.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4724e-05 2024-09-14T09:08:15.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078995+0000", 2024-09-14T09:08:15.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116132669999999 2024-09-14T09:08:15.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079009+0000", 2024-09-14T09:08:15.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3942e-05 2024-09-14T09:08:15.683 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.683 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.683 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.683 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105654 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.683 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:08:15.683 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.747954712999999, 2024-09-14T09:08:15.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117364070000001, 2024-09-14T09:08:15.683 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.683 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.683 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:15.684 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.684 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.684 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.684 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105654 2024-09-14T09:08:15.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.684 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:08:15.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:08:15.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867296+0000", 2024-09-14T09:08:15.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2399999999999996e-07 2024-09-14T09:08:15.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867300+0000", 2024-09-14T09:08:15.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8689999999999997e-06 2024-09-14T09:08:15.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867313+0000", 2024-09-14T09:08:15.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2660000000000001e-05 2024-09-14T09:08:15.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867322+0000", 2024-09-14T09:08:15.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.533e-06 2024-09-14T09:08:15.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867429+0000", 2024-09-14T09:08:15.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106566 2024-09-14T09:08:15.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867442+0000", 2024-09-14T09:08:15.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3244000000000001e-05 2024-09-14T09:08:15.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079024+0000", 2024-09-14T09:08:15.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211581813 2024-09-14T09:08:15.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079032+0000", 2024-09-14T09:08:15.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7980000000000006e-06 2024-09-14T09:08:15.688 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.688 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.688 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.688 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105657 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.688 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:08:15.688 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.74789354, 2024-09-14T09:08:15.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211700657, 2024-09-14T09:08:15.689 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.689 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.689 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:15.689 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.689 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.689 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.689 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105657 2024-09-14T09:08:15.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.689 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:08:15.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:08:15.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867358+0000", 2024-09-14T09:08:15.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9700000000000005e-07 2024-09-14T09:08:15.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867367+0000", 2024-09-14T09:08:15.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8919999999999995e-06 2024-09-14T09:08:15.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867383+0000", 2024-09-14T09:08:15.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.579e-05 2024-09-14T09:08:15.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867393+0000", 2024-09-14T09:08:15.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3070000000000008e-06 2024-09-14T09:08:15.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867490+0000", 2024-09-14T09:08:15.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7703999999999998e-05 2024-09-14T09:08:15.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867503+0000", 2024-09-14T09:08:15.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2849e-05 2024-09-14T09:08:15.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079050+0000", 2024-09-14T09:08:15.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211546899 2024-09-14T09:08:15.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079057+0000", 2024-09-14T09:08:15.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3189999999999997e-06 2024-09-14T09:08:15.693 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.693 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.693 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.693 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105658 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.693 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:08:15.693 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.747831155999997, 2024-09-14T09:08:15.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211691217, 2024-09-14T09:08:15.694 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.694 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.694 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:15.694 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.694 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.694 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.694 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105658 2024-09-14T09:08:15.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.694 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:08:15.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:08:15.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867420+0000", 2024-09-14T09:08:15.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5000000000000001e-07 2024-09-14T09:08:15.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867424+0000", 2024-09-14T09:08:15.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8469999999999998e-06 2024-09-14T09:08:15.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867437+0000", 2024-09-14T09:08:15.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3156000000000001e-05 2024-09-14T09:08:15.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867448+0000", 2024-09-14T09:08:15.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1185e-05 2024-09-14T09:08:15.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867552+0000", 2024-09-14T09:08:15.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103362 2024-09-14T09:08:15.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867565+0000", 2024-09-14T09:08:15.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3018e-05 2024-09-14T09:08:15.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079092+0000", 2024-09-14T09:08:15.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211527013 2024-09-14T09:08:15.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079110+0000", 2024-09-14T09:08:15.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8686e-05 2024-09-14T09:08:15.698 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.698 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.698 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.698 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105662 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.698 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:08:15.698 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.747770091, 2024-09-14T09:08:15.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211652226, 2024-09-14T09:08:15.698 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.698 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.698 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:15.698 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.699 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.699 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.699 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105662 2024-09-14T09:08:15.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.699 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:08:15.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:08:15.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867481+0000", 2024-09-14T09:08:15.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1800000000000004e-07 2024-09-14T09:08:15.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867485+0000", 2024-09-14T09:08:15.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9199999999999997e-06 2024-09-14T09:08:15.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867498+0000", 2024-09-14T09:08:15.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2896e-05 2024-09-14T09:08:15.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867509+0000", 2024-09-14T09:08:15.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1202e-05 2024-09-14T09:08:15.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867611+0000", 2024-09-14T09:08:15.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010166999999999999 2024-09-14T09:08:15.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867624+0000", 2024-09-14T09:08:15.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2996000000000001e-05 2024-09-14T09:08:15.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079125+0000", 2024-09-14T09:08:15.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2115009299999999 2024-09-14T09:08:15.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079132+0000", 2024-09-14T09:08:15.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6939999999999995e-06 2024-09-14T09:08:15.703 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.703 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.703 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.703 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105663 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.703 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:08:15.703 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.747709039999997, 2024-09-14T09:08:15.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116218169999999, 2024-09-14T09:08:15.703 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.703 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.703 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:15.703 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.704 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.704 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.704 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105663 2024-09-14T09:08:15.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.704 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:08:15.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:08:15.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867542+0000", 2024-09-14T09:08:15.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1399999999999995e-07 2024-09-14T09:08:15.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867546+0000", 2024-09-14T09:08:15.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7900000000000001e-06 2024-09-14T09:08:15.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867559+0000", 2024-09-14T09:08:15.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2805e-05 2024-09-14T09:08:15.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867568+0000", 2024-09-14T09:08:15.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-09-14T09:08:15.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867669+0000", 2024-09-14T09:08:15.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100655 2024-09-14T09:08:15.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867682+0000", 2024-09-14T09:08:15.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3013000000000001e-05 2024-09-14T09:08:15.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079151+0000", 2024-09-14T09:08:15.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2114689089999999 2024-09-14T09:08:15.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079163+0000", 2024-09-14T09:08:15.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1962e-05 2024-09-14T09:08:15.708 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.708 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.708 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.709 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105665 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:15.709 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:08:15.709 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.747650409000002, 2024-09-14T09:08:15.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211584848, 2024-09-14T09:08:15.709 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:15.709 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:15.709 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:15.709 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:15.709 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:15.709 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:15.709 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105665 2024-09-14T09:08:15.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.709 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:15.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:15.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:08:15.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:15.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:08:15.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:15.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:15.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867601+0000", 2024-09-14T09:08:15.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.569999999999999e-07 2024-09-14T09:08:15.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:15.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867605+0000", 2024-09-14T09:08:15.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8589999999999998e-06 2024-09-14T09:08:15.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:15.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867617+0000", 2024-09-14T09:08:15.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2833000000000001e-05 2024-09-14T09:08:15.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:15.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867627+0000", 2024-09-14T09:08:15.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5219999999999996e-06 2024-09-14T09:08:15.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867728+0000", 2024-09-14T09:08:15.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010057500000000001 2024-09-14T09:08:15.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:15.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867740+0000", 2024-09-14T09:08:15.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2744e-05 2024-09-14T09:08:15.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:15.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079178+0000", 2024-09-14T09:08:15.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211437552 2024-09-14T09:08:15.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:15.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:15.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:15.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079185+0000", 2024-09-14T09:08:15.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8059999999999997e-06 2024-09-14T09:08:15.713 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.713 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.713 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.713 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:15.714 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:15.714 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:15.715 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:08:15.716 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:15.718+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:15.816 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:15.818+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:15.886 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:15.886 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:15.886 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:08:15.886 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:15.900 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:08:15.916 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:15.915+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:16.018 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:16.022+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:16.066 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:16.066 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:16.066 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:08:16.066 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:08:16.066 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:16.080 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:08:16.122 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:16.126+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:16.220 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:16.222+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:16.247 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:08:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:08:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:08:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65139 3.a 3:54c90b7d:::rbd_data.115e21c31fcf.0000000000000008:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:08:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:08:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 114.871044764, 2024-09-14T09:08:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14429895700000001, 2024-09-14T09:08:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.247 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65139 2024-09-14T09:08:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:08:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:08:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379715+0000", 2024-09-14T09:08:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.576e-06 2024-09-14T09:08:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379724+0000", 2024-09-14T09:08:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1779999999999992e-06 2024-09-14T09:08:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379746+0000", 2024-09-14T09:08:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1661000000000001e-05 2024-09-14T09:08:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379759+0000", 2024-09-14T09:08:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.324e-05 2024-09-14T09:08:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379882+0000", 2024-09-14T09:08:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012239400000000001 2024-09-14T09:08:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.380023+0000", 2024-09-14T09:08:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000141402 2024-09-14T09:08:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:08:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.391553+0000", 2024-09-14T09:08:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011530197000000001 2024-09-14T09:08:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523927+0000", 2024-09-14T09:08:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.132373935 2024-09-14T09:08:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:08:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523941+0000", 2024-09-14T09:08:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3326000000000001e-05 2024-09-14T09:08:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:08:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523977+0000", 2024-09-14T09:08:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6185e-05 2024-09-14T09:08:16.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.524012+0000", 2024-09-14T09:08:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4863000000000001e-05 2024-09-14T09:08:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107606 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 14336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.844587863000001, 2024-09-14T09:08:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14293513799999999, 2024-09-14T09:08:16.253 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107606 2024-09-14T09:08:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1600000000000001e-07 2024-09-14T09:08:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406174+0000", 2024-09-14T09:08:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4199999999999999e-06 2024-09-14T09:08:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406188+0000", 2024-09-14T09:08:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4114000000000001e-05 2024-09-14T09:08:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406198+0000", 2024-09-14T09:08:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.041e-05 2024-09-14T09:08:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.548700+0000", 2024-09-14T09:08:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14250141299999999 2024-09-14T09:08:16.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.548783+0000", 2024-09-14T09:08:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3578999999999994e-05 2024-09-14T09:08:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549105+0000", 2024-09-14T09:08:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032158600000000001 2024-09-14T09:08:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107608 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.844521827000001, 2024-09-14T09:08:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14603042799999999, 2024-09-14T09:08:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107608 2024-09-14T09:08:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1200000000000002e-07 2024-09-14T09:08:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406241+0000", 2024-09-14T09:08:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7539999999999997e-06 2024-09-14T09:08:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406275+0000", 2024-09-14T09:08:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3512e-05 2024-09-14T09:08:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406322+0000", 2024-09-14T09:08:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6764999999999997e-05 2024-09-14T09:08:16.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549149+0000", 2024-09-14T09:08:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14282725499999999 2024-09-14T09:08:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549198+0000", 2024-09-14T09:08:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9599999999999999e-05 2024-09-14T09:08:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552266+0000", 2024-09-14T09:08:16.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030677299999999998 2024-09-14T09:08:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107610 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 22528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.844399952000003, 2024-09-14T09:08:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146437919, 2024-09-14T09:08:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.262 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107610 2024-09-14T09:08:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.02e-07 2024-09-14T09:08:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406363+0000", 2024-09-14T09:08:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8099999999999997e-06 2024-09-14T09:08:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406390+0000", 2024-09-14T09:08:16.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7004000000000001e-05 2024-09-14T09:08:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406404+0000", 2024-09-14T09:08:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3682e-05 2024-09-14T09:08:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552335+0000", 2024-09-14T09:08:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14593077800000001 2024-09-14T09:08:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552431+0000", 2024-09-14T09:08:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6086000000000001e-05 2024-09-14T09:08:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552795+0000", 2024-09-14T09:08:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000364657 2024-09-14T09:08:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107612 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 26624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:16.266 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:08:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.844334689, 2024-09-14T09:08:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146762061, 2024-09-14T09:08:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107612 2024-09-14T09:08:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:08:16.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:08:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406424+0000", 2024-09-14T09:08:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2600000000000002e-07 2024-09-14T09:08:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406451+0000", 2024-09-14T09:08:16.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7250000000000002e-05 2024-09-14T09:08:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406487+0000", 2024-09-14T09:08:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6086e-05 2024-09-14T09:08:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406498+0000", 2024-09-14T09:08:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0858e-05 2024-09-14T09:08:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552821+0000", 2024-09-14T09:08:16.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14632319099999999 2024-09-14T09:08:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552875+0000", 2024-09-14T09:08:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4262999999999999e-05 2024-09-14T09:08:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553185+0000", 2024-09-14T09:08:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000309687 2024-09-14T09:08:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.270 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107614 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:08:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.844247154999998, 2024-09-14T09:08:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14737121, 2024-09-14T09:08:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107614 2024-09-14T09:08:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.271 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:08:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:08:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406511+0000", 2024-09-14T09:08:16.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1799999999999995e-07 2024-09-14T09:08:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406514+0000", 2024-09-14T09:08:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8660000000000002e-06 2024-09-14T09:08:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406523+0000", 2024-09-14T09:08:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1940000000000004e-06 2024-09-14T09:08:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406531+0000", 2024-09-14T09:08:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3159999999999997e-06 2024-09-14T09:08:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553233+0000", 2024-09-14T09:08:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14670117899999999 2024-09-14T09:08:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553334+0000", 2024-09-14T09:08:16.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100982 2024-09-14T09:08:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553882+0000", 2024-09-14T09:08:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00054805499999999998 2024-09-14T09:08:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107616 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 34816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:08:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.844215519999999, 2024-09-14T09:08:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14780390500000001, 2024-09-14T09:08:16.275 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107616 2024-09-14T09:08:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:08:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:08:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406543+0000", 2024-09-14T09:08:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0200000000000001e-07 2024-09-14T09:08:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406545+0000", 2024-09-14T09:08:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3360000000000002e-06 2024-09-14T09:08:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406552+0000", 2024-09-14T09:08:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9779999999999999e-06 2024-09-14T09:08:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406559+0000", 2024-09-14T09:08:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6919999999999996e-06 2024-09-14T09:08:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553917+0000", 2024-09-14T09:08:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14735791600000001 2024-09-14T09:08:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554033+0000", 2024-09-14T09:08:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116047 2024-09-14T09:08:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554346+0000", 2024-09-14T09:08:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031323400000000002 2024-09-14T09:08:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107618 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 38912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:16.279 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.844186720000003, 2024-09-14T09:08:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14816009599999999, 2024-09-14T09:08:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107618 2024-09-14T09:08:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9999999999999998e-07 2024-09-14T09:08:16.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406574+0000", 2024-09-14T09:08:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2560000000000001e-06 2024-09-14T09:08:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406581+0000", 2024-09-14T09:08:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1539999999999996e-06 2024-09-14T09:08:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406586+0000", 2024-09-14T09:08:16.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.904e-06 2024-09-14T09:08:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554385+0000", 2024-09-14T09:08:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.147799443 2024-09-14T09:08:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554441+0000", 2024-09-14T09:08:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5921000000000002e-05 2024-09-14T09:08:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554731+0000", 2024-09-14T09:08:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028991800000000001 2024-09-14T09:08:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107620 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.844164937999999, 2024-09-14T09:08:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.148512643, 2024-09-14T09:08:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.284 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107620 2024-09-14T09:08:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9999999999999998e-07 2024-09-14T09:08:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406595+0000", 2024-09-14T09:08:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9360000000000002e-06 2024-09-14T09:08:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406605+0000", 2024-09-14T09:08:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0268000000000001e-05 2024-09-14T09:08:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406614+0000", 2024-09-14T09:08:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7180000000000002e-06 2024-09-14T09:08:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554750+0000", 2024-09-14T09:08:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14813599699999999 2024-09-14T09:08:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554794+0000", 2024-09-14T09:08:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.401e-05 2024-09-14T09:08:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555105+0000", 2024-09-14T09:08:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031131399999999998 2024-09-14T09:08:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.288 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107622 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.844133681999999, 2024-09-14T09:08:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14902820899999999, 2024-09-14T09:08:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107622 2024-09-14T09:08:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9200000000000001e-07 2024-09-14T09:08:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406636+0000", 2024-09-14T09:08:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.189e-05 2024-09-14T09:08:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406649+0000", 2024-09-14T09:08:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2442e-05 2024-09-14T09:08:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406658+0000", 2024-09-14T09:08:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3239999999999992e-06 2024-09-14T09:08:16.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555287+0000", 2024-09-14T09:08:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14862868400000001 2024-09-14T09:08:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555348+0000", 2024-09-14T09:08:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1686999999999994e-05 2024-09-14T09:08:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555652+0000", 2024-09-14T09:08:16.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00030369000000000001 2024-09-14T09:08:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107624 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.844089988999997, 2024-09-14T09:08:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14934019900000001, 2024-09-14T09:08:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.293 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107624 2024-09-14T09:08:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1399999999999997e-07 2024-09-14T09:08:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406670+0000", 2024-09-14T09:08:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4099999999999998e-06 2024-09-14T09:08:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406680+0000", 2024-09-14T09:08:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6460000000000005e-06 2024-09-14T09:08:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406688+0000", 2024-09-14T09:08:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5859999999999999e-06 2024-09-14T09:08:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555693+0000", 2024-09-14T09:08:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14900544499999999 2024-09-14T09:08:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555743+0000", 2024-09-14T09:08:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9966999999999998e-05 2024-09-14T09:08:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556008+0000", 2024-09-14T09:08:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026463099999999998 2024-09-14T09:08:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.297 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107626 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:08:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.844058015000002, 2024-09-14T09:08:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14983149500000001, 2024-09-14T09:08:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107626 2024-09-14T09:08:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:08:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:08:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408087+0000", 2024-09-14T09:08:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001387735 2024-09-14T09:08:16.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408094+0000", 2024-09-14T09:08:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9399999999999996e-06 2024-09-14T09:08:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408122+0000", 2024-09-14T09:08:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8076e-05 2024-09-14T09:08:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408147+0000", 2024-09-14T09:08:16.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4726e-05 2024-09-14T09:08:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556048+0000", 2024-09-14T09:08:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14790144299999999 2024-09-14T09:08:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556175+0000", 2024-09-14T09:08:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012663 2024-09-14T09:08:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556531+0000", 2024-09-14T09:08:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035594499999999998 2024-09-14T09:08:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107628 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:08:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.842588403999997, 2024-09-14T09:08:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14882648900000001, 2024-09-14T09:08:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.302 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107628 2024-09-14T09:08:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:08:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:08:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408878+0000", 2024-09-14T09:08:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070858800000000001 2024-09-14T09:08:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408885+0000", 2024-09-14T09:08:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4839999999999998e-06 2024-09-14T09:08:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408900+0000", 2024-09-14T09:08:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5174000000000001e-05 2024-09-14T09:08:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408918+0000", 2024-09-14T09:08:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.808e-05 2024-09-14T09:08:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556572+0000", 2024-09-14T09:08:16.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14765399500000001 2024-09-14T09:08:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556651+0000", 2024-09-14T09:08:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8919999999999997e-05 2024-09-14T09:08:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556996+0000", 2024-09-14T09:08:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034424800000000001 2024-09-14T09:08:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107630 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 63488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:08:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.841819569999998, 2024-09-14T09:08:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14855159200000001, 2024-09-14T09:08:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107630 2024-09-14T09:08:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.307 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:08:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:08:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409510+0000", 2024-09-14T09:08:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00057189700000000003 2024-09-14T09:08:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409518+0000", 2024-09-14T09:08:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7619999999999995e-06 2024-09-14T09:08:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409534+0000", 2024-09-14T09:08:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.594e-05 2024-09-14T09:08:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409546+0000", 2024-09-14T09:08:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2604e-05 2024-09-14T09:08:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557048+0000", 2024-09-14T09:08:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14750137899999999 2024-09-14T09:08:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557117+0000", 2024-09-14T09:08:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8998999999999994e-05 2024-09-14T09:08:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557490+0000", 2024-09-14T09:08:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037301100000000002 2024-09-14T09:08:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107632 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:08:16.311 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.841188625000001, 2024-09-14T09:08:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14833944199999999, 2024-09-14T09:08:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107632 2024-09-14T09:08:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:08:16.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:08:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412609+0000", 2024-09-14T09:08:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003040105 2024-09-14T09:08:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412618+0000", 2024-09-14T09:08:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2960000000000004e-06 2024-09-14T09:08:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412640+0000", 2024-09-14T09:08:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1773999999999999e-05 2024-09-14T09:08:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412655+0000", 2024-09-14T09:08:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4538e-05 2024-09-14T09:08:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557538+0000", 2024-09-14T09:08:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14488311800000001 2024-09-14T09:08:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557594+0000", 2024-09-14T09:08:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6122999999999998e-05 2024-09-14T09:08:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557908+0000", 2024-09-14T09:08:16.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031448799999999997 2024-09-14T09:08:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107634 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:08:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.838070977999998, 2024-09-14T09:08:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.145666186, 2024-09-14T09:08:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.316 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107634 2024-09-14T09:08:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:08:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:08:16.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412688+0000", 2024-09-14T09:08:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9000000000000005e-07 2024-09-14T09:08:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412707+0000", 2024-09-14T09:08:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9232000000000002e-05 2024-09-14T09:08:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412722+0000", 2024-09-14T09:08:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5722e-05 2024-09-14T09:08:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412733+0000", 2024-09-14T09:08:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0909e-05 2024-09-14T09:08:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557946+0000", 2024-09-14T09:08:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14521263100000001 2024-09-14T09:08:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558020+0000", 2024-09-14T09:08:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4294000000000004e-05 2024-09-14T09:08:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558353+0000", 2024-09-14T09:08:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033240800000000002 2024-09-14T09:08:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.320 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107636 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:08:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.838006675000003, 2024-09-14T09:08:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14599377399999999, 2024-09-14T09:08:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107636 2024-09-14T09:08:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:08:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:08:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412752+0000", 2024-09-14T09:08:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.156e-06 2024-09-14T09:08:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412770+0000", 2024-09-14T09:08:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7521999999999999e-05 2024-09-14T09:08:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412790+0000", 2024-09-14T09:08:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0942e-05 2024-09-14T09:08:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412809+0000", 2024-09-14T09:08:16.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8887999999999999e-05 2024-09-14T09:08:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558387+0000", 2024-09-14T09:08:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14557798799999999 2024-09-14T09:08:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558448+0000", 2024-09-14T09:08:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0862000000000001e-05 2024-09-14T09:08:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558745+0000", 2024-09-14T09:08:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029641599999999998 2024-09-14T09:08:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107638 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:08:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.837919419000002, 2024-09-14T09:08:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146344799, 2024-09-14T09:08:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:16.325 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107638 2024-09-14T09:08:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:08:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:08:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412839+0000", 2024-09-14T09:08:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0720000000000001e-06 2024-09-14T09:08:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412844+0000", 2024-09-14T09:08:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7759999999999997e-06 2024-09-14T09:08:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412857+0000", 2024-09-14T09:08:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3138e-05 2024-09-14T09:08:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412867+0000", 2024-09-14T09:08:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7229999999999999e-06 2024-09-14T09:08:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558785+0000", 2024-09-14T09:08:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14591828700000001 2024-09-14T09:08:16.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558833+0000", 2024-09-14T09:08:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7790000000000002e-05 2024-09-14T09:08:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559183+0000", 2024-09-14T09:08:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035001299999999998 2024-09-14T09:08:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107640 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:08:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.837874268, 2024-09-14T09:08:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146686804, 2024-09-14T09:08:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107640 2024-09-14T09:08:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:08:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:08:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420446+0000", 2024-09-14T09:08:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075632529999999998 2024-09-14T09:08:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420456+0000", 2024-09-14T09:08:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0850000000000003e-06 2024-09-14T09:08:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420475+0000", 2024-09-14T09:08:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9406e-05 2024-09-14T09:08:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420488+0000", 2024-09-14T09:08:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3278000000000001e-05 2024-09-14T09:08:16.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559205+0000", 2024-09-14T09:08:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13871644999999999 2024-09-14T09:08:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559252+0000", 2024-09-14T09:08:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7666000000000001e-05 2024-09-14T09:08:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559570+0000", 2024-09-14T09:08:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031766599999999998 2024-09-14T09:08:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107660 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 124928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:08:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.829804025999998, 2024-09-14T09:08:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.142954628, 2024-09-14T09:08:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:16.334 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107660 2024-09-14T09:08:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:08:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:16.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:08:16.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:16.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:16.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428190+0000", 2024-09-14T09:08:16.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072360339999999997 2024-09-14T09:08:16.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:16.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428398+0000", 2024-09-14T09:08:16.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020898499999999999 2024-09-14T09:08:16.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:16.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428458+0000", 2024-09-14T09:08:16.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9985999999999998e-05 2024-09-14T09:08:16.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:16.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428480+0000", 2024-09-14T09:08:16.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1284000000000002e-05 2024-09-14T09:08:16.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:16.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563583+0000", 2024-09-14T09:08:16.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13510361800000001 2024-09-14T09:08:16.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:16.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563644+0000", 2024-09-14T09:08:16.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1092000000000004e-05 2024-09-14T09:08:16.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:16.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:16.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:16.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563908+0000", 2024-09-14T09:08:16.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026362899999999998 2024-09-14T09:08:16.338 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.338 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.338 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.338 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:16.339 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:16.339 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:16.341 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is set 2024-09-14T09:08:16.341 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:08:16.341 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:16.342+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:16.367 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:08:16.441 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:16.442+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:16.508 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:16.508 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:16.508 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:08:16.508 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:16.522 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:08:16.533 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:08:16.534 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:08:16.542 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:16.543+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:16.550 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:08:16.550 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:08:16.566 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:08:16.566 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:08:16.584 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:08:16.584 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:08:16.585 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:08:16.603 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:08:16.603 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:08:16.622 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:08:16.622 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:08:16.642 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:08:16.643 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:08:16.643 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:16.643+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:16.646 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:08:16.647 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T09:08:16.697 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:16.697 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:16.711 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:08:16.744 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:16.743+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:16.832 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:16.832 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:16.845 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:16.843+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:16.846 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:08:16.884 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-14T09:08:16.884 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:08:16.885 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-14T09:08:16.885 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T09:08:16.885 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T09:08:16.885 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-14T09:08:16.946 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:16.950+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:16.977 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:16.978 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:16.991 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:08:17.009 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:17.010 INFO:teuthology.orchestra.run.smithi183.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T09:08:17.010 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:17.046 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:17.047+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:17.115 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:17.115 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:17.129 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:08:17.148 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:17.150+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:17.249 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:17.247+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:17.300 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:17.300 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:17.319 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:08:17.351 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:17.354+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:17.451 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:17.451+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:17.498 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:17.499 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:17.513 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:08:17.553 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:17.554+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:17.654 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:17.655+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:17.681 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:17.681 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:17.695 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:08:17.754 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:17.758+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:17.855 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:17.855+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:17.862 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:17.862 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:17.876 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:08:17.956 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:17.958+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:18.041 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:18.041 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:18.056 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:08:18.057 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:18.062+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:18.158 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:18.159+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:18.181 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:18.181 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:18.195 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:08:18.259 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:18.262+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:18.343 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T09:08:18.365 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:18.365 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:18.366 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:18.367+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:18.382 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:08:18.470 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:18.471+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:18.549 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:18.549 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:18.561 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:18.563+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:18.565 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:08:18.647 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:08:18.662 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:18.666+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:18.722 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:18.722 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:18.737 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:08:18.763 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:18.766+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:18.786 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:08:18.786 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:08:18.804 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:08:18.804 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:08:18.820 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:08:18.821 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:08:18.837 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:08:18.837 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:08:18.837 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:08:18.857 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:18.857 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:08:18.857 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:08:18.858 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:08:18.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.858 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:18.858 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:08:18.858 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 165.77420862700001, 2024-09-14T09:08:18.858 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:08:18.858 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.858 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.858 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:18.858 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:18.858 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:18.859 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:18.859 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:08:18.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.859 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.859 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.859 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:08:18.859 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.859 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:08:18.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:08:18.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:08:18.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.860 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.860 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:08:18.860 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:08:18.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:08:18.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:08:18.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:08:18.861 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:08:18.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.861 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.861 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:08:18.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:08:18.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:18.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:08:18.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:08:18.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.862 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.862 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:08:18.862 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:08:18.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:08:18.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:08:18.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.863 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.863 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:08:18.863 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:08:18.863 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.863 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.863 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.863 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:18.864 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:08:18.864 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 165.774153969, 2024-09-14T09:08:18.864 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:08:18.864 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.864 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.864 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:18.864 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:18.864 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:18.864 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:18.864 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:08:18.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.864 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:08:18.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:08:18.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.865 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.865 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:08:18.865 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:08:18.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:08:18.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:08:18.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.866 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.866 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:08:18.866 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:08:18.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:08:18.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:08:18.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:08:18.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:08:18.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.867 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:18.867 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:08:18.867 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:08:18.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:08:18.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:08:18.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:08:18.868 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:08:18.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.868 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.868 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:08:18.869 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:08:18.869 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.869 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.869 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.869 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.869 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:18.869 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:08:18.869 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 165.77404859500001, 2024-09-14T09:08:18.870 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:08:18.870 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.870 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.870 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:18.870 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:18.870 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:18.870 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:18.870 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:08:18.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.870 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.870 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.870 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:08:18.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:08:18.871 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.871 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.871 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:08:18.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:08:18.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:08:18.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:08:18.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.872 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.872 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:08:18.872 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:08:18.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:08:18.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:08:18.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:08:18.873 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:08:18.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.873 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:18.873 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:08:18.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:08:18.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:08:18.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:08:18.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.874 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:08:18.874 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:08:18.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.874 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.875 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:08:18.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:08:18.875 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.875 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.875 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.875 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:18.875 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:08:18.875 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 165.77369183100001, 2024-09-14T09:08:18.875 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:08:18.875 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.875 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.875 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:18.876 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:18.876 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:18.876 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:18.876 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:08:18.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.876 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:08:18.876 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.876 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.876 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.876 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:08:18.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:08:18.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:08:18.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.877 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.877 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:08:18.877 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:08:18.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:08:18.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:08:18.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.878 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:08:18.878 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:08:18.878 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.878 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:08:18.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:08:18.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:18.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:08:18.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:08:18.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.879 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.879 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:08:18.879 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:08:18.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:08:18.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:08:18.880 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.880 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.880 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:08:18.880 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:08:18.881 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.881 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.881 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.881 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.881 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.881 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:18.881 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:08:18.881 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 165.77360709300001, 2024-09-14T09:08:18.881 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:08:18.881 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.881 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.881 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:18.882 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:18.882 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:18.882 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:18.882 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:08:18.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.882 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:08:18.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.882 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.882 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.882 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.882 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:08:18.882 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:08:18.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:08:18.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:08:18.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:08:18.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.883 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.883 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.883 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:08:18.883 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:08:18.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:08:18.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:08:18.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.884 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:08:18.884 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:08:18.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.884 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.884 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:18.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:08:18.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:08:18.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:08:18.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:08:18.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.885 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.885 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.885 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:08:18.885 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:08:18.885 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.886 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.886 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:08:18.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:08:18.886 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.886 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.886 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.886 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.886 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.886 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:18.886 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:08:18.886 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 165.77351940700001, 2024-09-14T09:08:18.886 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:08:18.887 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.887 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.887 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:18.887 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:18.887 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:18.887 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:18.887 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:08:18.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.887 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.887 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.887 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.887 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:08:18.887 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.887 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:08:18.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:08:18.888 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:08:18.888 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.888 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.888 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.888 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:08:18.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:08:18.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.889 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.889 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:08:18.889 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:08:18.889 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.889 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.890 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.890 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:08:18.890 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:08:18.890 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.890 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:08:18.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:08:18.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.891 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.891 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:18.891 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:08:18.891 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:08:18.891 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:08:18.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:08:18.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.892 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.892 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:08:18.892 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:08:18.892 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.892 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.893 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.893 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:08:18.893 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:08:18.893 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.893 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.894 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.894 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.894 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.894 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:08:18.894 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:08:18.894 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 132.69722741999999, 2024-09-14T09:08:18.894 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:08:18.894 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.894 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.894 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:18.894 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:18.895 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:18.895 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:18.895 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:08:18.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.895 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:08:18.895 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.895 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.895 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.895 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.895 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:08:18.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:08:18.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:08:18.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.896 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:08:18.896 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:08:18.896 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.896 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.896 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:08:18.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:08:18.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:08:18.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:08:18.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.897 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.897 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.897 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:08:18.897 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:08:18.897 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:08:18.898 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:08:18.898 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.898 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.898 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:08:18.898 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:08:18.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:08:18.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:18.899 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:08:18.899 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:08:18.899 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.899 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.899 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:18.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:08:18.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:08:18.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:18.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:08:18.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:08:18.900 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.900 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.900 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.900 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:08:18.900 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:08:18.900 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.901 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.901 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.901 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.901 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.901 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:97796 3.15 3:aaaa5f7f:::rbd_data.115e113dc8ca.0000000000000029:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:08:18.901 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:08:18.901 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.763358550000007, 2024-09-14T09:08:18.901 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14537731300000001, 2024-09-14T09:08:18.901 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.902 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.902 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:18.902 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:18.902 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:18.902 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:18.902 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97796 2024-09-14T09:08:18.902 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.902 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.902 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.902 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.902 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:08:18.902 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:08:18.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.903 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094482+0000", 2024-09-14T09:08:18.903 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6400000000000001e-07 2024-09-14T09:08:18.903 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.903 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.903 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094485+0000", 2024-09-14T09:08:18.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.693e-06 2024-09-14T09:08:18.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094494+0000", 2024-09-14T09:08:18.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9590000000000001e-06 2024-09-14T09:08:18.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.904 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.904 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094502+0000", 2024-09-14T09:08:18.904 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3180000000000001e-06 2024-09-14T09:08:18.904 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.904 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094567+0000", 2024-09-14T09:08:18.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5526999999999998e-05 2024-09-14T09:08:18.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.905 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.905 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094726+0000", 2024-09-14T09:08:18.905 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015842499999999999 2024-09-14T09:08:18.905 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.905 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:08:18.906 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.101940+0000", 2024-09-14T09:08:18.906 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0072146559999999998 2024-09-14T09:08:18.906 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.906 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.906 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:18.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239665+0000", 2024-09-14T09:08:18.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.137724501 2024-09-14T09:08:18.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.907 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.907 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:18.907 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239726+0000", 2024-09-14T09:08:18.907 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1550999999999994e-05 2024-09-14T09:08:18.907 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.908 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.908 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:18.908 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239800+0000", 2024-09-14T09:08:18.908 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3318000000000005e-05 2024-09-14T09:08:18.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.909 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.909 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239858+0000", 2024-09-14T09:08:18.909 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8600999999999997e-05 2024-09-14T09:08:18.909 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.909 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.909 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.909 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.909 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.909 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:97796 3.15 e106/103 ondisk, result = 0)", 2024-09-14T09:08:18.909 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:08:18.909 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 66.741012681000001, 2024-09-14T09:08:18.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123043465, 2024-09-14T09:08:18.910 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.910 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.910 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:18.910 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:08:18.910 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.910 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.910 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.910 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.910 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:08:18.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116828+0000", 2024-09-14T09:08:18.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8100000000000002e-07 2024-09-14T09:08:18.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.911 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116831+0000", 2024-09-14T09:08:18.911 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3009999999999998e-06 2024-09-14T09:08:18.911 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.911 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.911 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116841+0000", 2024-09-14T09:08:18.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7829999999999993e-06 2024-09-14T09:08:18.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116851+0000", 2024-09-14T09:08:18.912 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0135999999999999e-05 2024-09-14T09:08:18.912 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.912 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.912 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.912 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239707+0000", 2024-09-14T09:08:18.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12285581499999999 2024-09-14T09:08:18.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.913 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.913 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239718+0000", 2024-09-14T09:08:18.913 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1804000000000001e-05 2024-09-14T09:08:18.913 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.913 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.914 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.914 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239870+0000", 2024-09-14T09:08:18.914 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015194500000000001 2024-09-14T09:08:18.914 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.914 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.914 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.914 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.914 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.914 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117315 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 309248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:18.914 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:08:18.914 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.134376944000003, 2024-09-14T09:08:18.915 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11247383800000001, 2024-09-14T09:08:18.915 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.915 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.915 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:18.915 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:18.915 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:18.915 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:18.915 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117315 2024-09-14T09:08:18.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.916 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:08:18.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.916 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:08:18.916 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.916 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.916 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.916 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723899+0000", 2024-09-14T09:08:18.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043641699999999998 2024-09-14T09:08:18.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723903+0000", 2024-09-14T09:08:18.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5389999999999999e-06 2024-09-14T09:08:18.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.917 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.917 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.917 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723910+0000", 2024-09-14T09:08:18.917 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4229999999999999e-06 2024-09-14T09:08:18.917 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723917+0000", 2024-09-14T09:08:18.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4560000000000003e-06 2024-09-14T09:08:18.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.918 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.918 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835514+0000", 2024-09-14T09:08:18.918 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111596258 2024-09-14T09:08:18.918 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.918 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835571+0000", 2024-09-14T09:08:18.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7317e-05 2024-09-14T09:08:18.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.919 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.919 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.919 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835936+0000", 2024-09-14T09:08:18.919 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000365428 2024-09-14T09:08:18.919 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.919 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.919 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.919 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.920 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.920 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117317 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 313344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:18.920 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:08:18.920 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.133909928000001, 2024-09-14T09:08:18.920 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112450201, 2024-09-14T09:08:18.920 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.920 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.920 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:18.920 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:18.920 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:18.920 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:18.920 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117317 2024-09-14T09:08:18.920 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.920 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:08:18.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:08:18.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.921 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.921 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.921 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.921 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724396+0000", 2024-09-14T09:08:18.921 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000466028 2024-09-14T09:08:18.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724401+0000", 2024-09-14T09:08:18.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2299999999999999e-06 2024-09-14T09:08:18.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724412+0000", 2024-09-14T09:08:18.922 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0762e-05 2024-09-14T09:08:18.922 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.922 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.922 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.922 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724419+0000", 2024-09-14T09:08:18.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1010000000000003e-06 2024-09-14T09:08:18.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835981+0000", 2024-09-14T09:08:18.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111562174 2024-09-14T09:08:18.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.923 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836070+0000", 2024-09-14T09:08:18.923 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8839999999999996e-05 2024-09-14T09:08:18.923 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.923 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.923 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.924 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836380+0000", 2024-09-14T09:08:18.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031006600000000001 2024-09-14T09:08:18.924 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.924 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.924 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.924 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.924 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.924 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117319 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 317440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:18.924 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:08:18.924 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.133405291999999, 2024-09-14T09:08:18.924 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112467763, 2024-09-14T09:08:18.924 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.924 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.925 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:18.925 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:18.925 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:18.925 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:18.925 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117319 2024-09-14T09:08:18.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.925 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.925 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:08:18.925 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.925 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.925 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.925 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:08:18.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724846+0000", 2024-09-14T09:08:18.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00041155400000000001 2024-09-14T09:08:18.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.926 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.926 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724854+0000", 2024-09-14T09:08:18.926 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5469999999999997e-06 2024-09-14T09:08:18.926 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.926 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724862+0000", 2024-09-14T09:08:18.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1289999999999996e-06 2024-09-14T09:08:18.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724867+0000", 2024-09-14T09:08:18.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8720000000000001e-06 2024-09-14T09:08:18.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.927 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.927 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.927 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836486+0000", 2024-09-14T09:08:18.927 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111618357 2024-09-14T09:08:18.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836554+0000", 2024-09-14T09:08:18.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8378000000000002e-05 2024-09-14T09:08:18.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.928 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.928 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.928 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836902+0000", 2024-09-14T09:08:18.928 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034792599999999998 2024-09-14T09:08:18.928 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.928 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.928 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.929 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.929 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117321 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 321536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:18.929 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:08:18.929 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.132960967999999, 2024-09-14T09:08:18.929 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11249685600000001, 2024-09-14T09:08:18.929 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.929 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.929 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:18.929 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:18.929 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:18.929 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:18.929 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117321 2024-09-14T09:08:18.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.930 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:08:18.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:08:18.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.930 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.930 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.930 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725241+0000", 2024-09-14T09:08:18.930 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036211099999999998 2024-09-14T09:08:18.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725245+0000", 2024-09-14T09:08:18.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9179999999999996e-06 2024-09-14T09:08:18.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725254+0000", 2024-09-14T09:08:18.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0699999999999996e-06 2024-09-14T09:08:18.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.931 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.931 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.931 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725260+0000", 2024-09-14T09:08:18.931 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1099999999999999e-06 2024-09-14T09:08:18.931 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836935+0000", 2024-09-14T09:08:18.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111674997 2024-09-14T09:08:18.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.932 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837021+0000", 2024-09-14T09:08:18.932 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5931000000000001e-05 2024-09-14T09:08:18.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.932 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.932 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.933 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837375+0000", 2024-09-14T09:08:18.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035471899999999998 2024-09-14T09:08:18.933 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.933 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.933 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.933 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.933 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117323 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:18.933 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:08:18.933 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.132566742999998, 2024-09-14T09:08:18.933 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11250631, 2024-09-14T09:08:18.933 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.933 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.933 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:18.934 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:18.934 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:18.934 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:18.934 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117323 2024-09-14T09:08:18.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.934 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:08:18.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.934 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.934 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.934 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:08:18.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725941+0000", 2024-09-14T09:08:18.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00066866500000000004 2024-09-14T09:08:18.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725946+0000", 2024-09-14T09:08:18.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9060000000000001e-06 2024-09-14T09:08:18.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.935 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.935 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725959+0000", 2024-09-14T09:08:18.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2622e-05 2024-09-14T09:08:18.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725968+0000", 2024-09-14T09:08:18.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5660000000000007e-06 2024-09-14T09:08:18.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837408+0000", 2024-09-14T09:08:18.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111440626 2024-09-14T09:08:18.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837495+0000", 2024-09-14T09:08:18.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6654000000000001e-05 2024-09-14T09:08:18.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837779+0000", 2024-09-14T09:08:18.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000284271 2024-09-14T09:08:18.937 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.937 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.937 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.937 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117327 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 333824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:18.938 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:08:18.938 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.131830551, 2024-09-14T09:08:18.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112409908, 2024-09-14T09:08:18.938 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.938 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.938 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:18.938 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:18.938 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:18.938 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:18.938 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117327 2024-09-14T09:08:18.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.938 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:08:18.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:08:18.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726303+0000", 2024-09-14T09:08:18.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029388199999999998 2024-09-14T09:08:18.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726307+0000", 2024-09-14T09:08:18.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4320000000000001e-06 2024-09-14T09:08:18.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.940 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.940 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726317+0000", 2024-09-14T09:08:18.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4609999999999997e-06 2024-09-14T09:08:18.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726324+0000", 2024-09-14T09:08:18.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0480000000000001e-06 2024-09-14T09:08:18.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838051+0000", 2024-09-14T09:08:18.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111727534 2024-09-14T09:08:18.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838143+0000", 2024-09-14T09:08:18.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1359000000000004e-05 2024-09-14T09:08:18.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838419+0000", 2024-09-14T09:08:18.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027619199999999997 2024-09-14T09:08:18.942 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.942 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.942 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.943 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117329 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 337920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:18.943 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:08:18.943 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.131503791999997, 2024-09-14T09:08:18.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112411122, 2024-09-14T09:08:18.943 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.943 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.943 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:18.943 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:18.943 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:18.943 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:18.943 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117329 2024-09-14T09:08:18.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.944 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:08:18.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:08:18.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726742+0000", 2024-09-14T09:08:18.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040626099999999999 2024-09-14T09:08:18.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726748+0000", 2024-09-14T09:08:18.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6860000000000003e-06 2024-09-14T09:08:18.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726761+0000", 2024-09-14T09:08:18.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3599e-05 2024-09-14T09:08:18.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726770+0000", 2024-09-14T09:08:18.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5990000000000002e-06 2024-09-14T09:08:18.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838443+0000", 2024-09-14T09:08:18.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111672664 2024-09-14T09:08:18.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838487+0000", 2024-09-14T09:08:18.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4249000000000003e-05 2024-09-14T09:08:18.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838747+0000", 2024-09-14T09:08:18.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026006400000000001 2024-09-14T09:08:18.947 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.947 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.947 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.947 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117331 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 342016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:18.947 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:08:18.948 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.131036197, 2024-09-14T09:08:18.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112291139, 2024-09-14T09:08:18.948 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.948 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.948 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:18.948 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:18.948 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:18.948 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:18.948 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117331 2024-09-14T09:08:18.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.948 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:08:18.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:08:18.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727158+0000", 2024-09-14T09:08:18.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035434000000000003 2024-09-14T09:08:18.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727162+0000", 2024-09-14T09:08:18.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5449999999999997e-06 2024-09-14T09:08:18.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727174+0000", 2024-09-14T09:08:18.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2278e-05 2024-09-14T09:08:18.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727183+0000", 2024-09-14T09:08:18.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3820000000000004e-06 2024-09-14T09:08:18.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838788+0000", 2024-09-14T09:08:18.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111604748 2024-09-14T09:08:18.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838847+0000", 2024-09-14T09:08:18.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8983000000000001e-05 2024-09-14T09:08:18.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839094+0000", 2024-09-14T09:08:18.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024786300000000002 2024-09-14T09:08:18.952 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.952 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.952 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.953 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117333 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 346112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:18.953 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:08:18.953 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.130642385999998, 2024-09-14T09:08:18.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112319711, 2024-09-14T09:08:18.953 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.953 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.953 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:18.953 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:18.953 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:18.953 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:18.954 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117333 2024-09-14T09:08:18.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.954 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:08:18.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:08:18.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727785+0000", 2024-09-14T09:08:18.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000587512 2024-09-14T09:08:18.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727789+0000", 2024-09-14T09:08:18.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.352e-06 2024-09-14T09:08:18.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727800+0000", 2024-09-14T09:08:18.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1369000000000001e-05 2024-09-14T09:08:18.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.956 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.956 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727807+0000", 2024-09-14T09:08:18.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1300000000000003e-06 2024-09-14T09:08:18.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839137+0000", 2024-09-14T09:08:18.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111329898 2024-09-14T09:08:18.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839226+0000", 2024-09-14T09:08:18.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8337000000000005e-05 2024-09-14T09:08:18.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839517+0000", 2024-09-14T09:08:18.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029111300000000002 2024-09-14T09:08:18.958 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.958 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.958 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.958 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117363 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 407552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:18.958 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:08:18.958 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.124546287000001, 2024-09-14T09:08:18.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112191306, 2024-09-14T09:08:18.958 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.958 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.958 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:18.958 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:18.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:18.959 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:18.959 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117363 2024-09-14T09:08:18.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.959 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:08:18.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:08:18.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733666+0000", 2024-09-14T09:08:18.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00037284500000000001 2024-09-14T09:08:18.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733671+0000", 2024-09-14T09:08:18.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9269999999999996e-06 2024-09-14T09:08:18.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733681+0000", 2024-09-14T09:08:18.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5599999999999999e-06 2024-09-14T09:08:18.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.961 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.961 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733689+0000", 2024-09-14T09:08:18.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2649999999999996e-06 2024-09-14T09:08:18.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845028+0000", 2024-09-14T09:08:18.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111338933 2024-09-14T09:08:18.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845094+0000", 2024-09-14T09:08:18.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6749999999999996e-05 2024-09-14T09:08:18.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845485+0000", 2024-09-14T09:08:18.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00039002600000000002 2024-09-14T09:08:18.963 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.963 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.963 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.963 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117365 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 411648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:18.963 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:08:18.963 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 47.124136522000001, 2024-09-14T09:08:18.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112271685, 2024-09-14T09:08:18.963 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:18.963 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:18.963 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:18.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:18.963 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:18.964 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:18.964 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117365 2024-09-14T09:08:18.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.964 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:18.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:18.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:08:18.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:18.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:08:18.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:18.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:18.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734147+0000", 2024-09-14T09:08:18.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00044374399999999999 2024-09-14T09:08:18.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:18.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734152+0000", 2024-09-14T09:08:18.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9139999999999999e-06 2024-09-14T09:08:18.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:18.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734164+0000", 2024-09-14T09:08:18.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2299e-05 2024-09-14T09:08:18.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:18.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734172+0000", 2024-09-14T09:08:18.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0390000000000005e-06 2024-09-14T09:08:18.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:18.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845528+0000", 2024-09-14T09:08:18.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11135639899999999 2024-09-14T09:08:18.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.966 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:18.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845602+0000", 2024-09-14T09:08:18.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3957999999999999e-05 2024-09-14T09:08:18.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:18.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:18.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:18.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845975+0000", 2024-09-14T09:08:18.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00037233200000000001 2024-09-14T09:08:18.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.967 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.967 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:18.967 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:18.968 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:18.968 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:08:18.968 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:08:18.969 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:08:18.970 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:08:18.970 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:08:18.970 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:08:18.971 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:08:18.971 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:08:18.971 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T09:08:18.972 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:18.971+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:19.073 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:19.078+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:19.105 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:19.105 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:19.105 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:08:19.105 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:19.119 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:08:19.206 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:19.210+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:19.275 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:19.275+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:19.285 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:19.286 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:19.286 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:08:19.286 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:08:19.286 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:19.300 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:08:19.333 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is unset 2024-09-14T09:08:19.406 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:19.407+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:19.472 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:19.473 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:08:19.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:08:19.473 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:08:19.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.473 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105615 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 382976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.473 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:08:19.473 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.612979052, 2024-09-14T09:08:19.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117270900000001, 2024-09-14T09:08:19.473 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.473 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.473 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:19.473 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.474 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.474 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.474 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105615 2024-09-14T09:08:19.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.474 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:08:19.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:08:19.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864368+0000", 2024-09-14T09:08:19.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.043e-06 2024-09-14T09:08:19.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864373+0000", 2024-09-14T09:08:19.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4669999999999998e-06 2024-09-14T09:08:19.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864387+0000", 2024-09-14T09:08:19.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3876000000000001e-05 2024-09-14T09:08:19.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864397+0000", 2024-09-14T09:08:19.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0592e-05 2024-09-14T09:08:19.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866069+0000", 2024-09-14T09:08:19.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016714410000000001 2024-09-14T09:08:19.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866076+0000", 2024-09-14T09:08:19.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:08:19.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.075854+0000", 2024-09-14T09:08:19.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.209778501 2024-09-14T09:08:19.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:19.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.075940+0000", 2024-09-14T09:08:19.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5357000000000003e-05 2024-09-14T09:08:19.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.076094+0000", 2024-09-14T09:08:19.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000154745 2024-09-14T09:08:19.478 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.478 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.478 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.478 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105629 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 411648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.478 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:08:19.479 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.610825329999997, 2024-09-14T09:08:19.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2115591910000001, 2024-09-14T09:08:19.479 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.479 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.479 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:19.479 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.479 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.479 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.479 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105629 2024-09-14T09:08:19.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.479 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:08:19.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:08:19.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866522+0000", 2024-09-14T09:08:19.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6899999999999996e-07 2024-09-14T09:08:19.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866526+0000", 2024-09-14T09:08:19.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0110000000000002e-06 2024-09-14T09:08:19.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866539+0000", 2024-09-14T09:08:19.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3013000000000001e-05 2024-09-14T09:08:19.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866549+0000", 2024-09-14T09:08:19.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8490000000000001e-06 2024-09-14T09:08:19.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866650+0000", 2024-09-14T09:08:19.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101054 2024-09-14T09:08:19.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866664+0000", 2024-09-14T09:08:19.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3805000000000001e-05 2024-09-14T09:08:19.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.077849+0000", 2024-09-14T09:08:19.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2111855419999999 2024-09-14T09:08:19.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:19.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.077945+0000", 2024-09-14T09:08:19.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6086000000000001e-05 2024-09-14T09:08:19.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078080+0000", 2024-09-14T09:08:19.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000134862 2024-09-14T09:08:19.483 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.483 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.484 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.484 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105631 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 415744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.484 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:08:19.484 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.610766110999997, 2024-09-14T09:08:19.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119463509999999, 2024-09-14T09:08:19.484 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.484 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.484 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:19.484 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.484 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.484 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.485 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105631 2024-09-14T09:08:19.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.485 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:08:19.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:08:19.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866581+0000", 2024-09-14T09:08:19.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.879999999999999e-07 2024-09-14T09:08:19.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866585+0000", 2024-09-14T09:08:19.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0239999999999999e-06 2024-09-14T09:08:19.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866598+0000", 2024-09-14T09:08:19.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2862e-05 2024-09-14T09:08:19.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866608+0000", 2024-09-14T09:08:19.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9890000000000004e-06 2024-09-14T09:08:19.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866708+0000", 2024-09-14T09:08:19.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9605999999999995e-05 2024-09-14T09:08:19.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866720+0000", 2024-09-14T09:08:19.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.261e-05 2024-09-14T09:08:19.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078137+0000", 2024-09-14T09:08:19.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211416571 2024-09-14T09:08:19.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:19.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078293+0000", 2024-09-14T09:08:19.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000156 2024-09-14T09:08:19.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078527+0000", 2024-09-14T09:08:19.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000233701 2024-09-14T09:08:19.489 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.489 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.489 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.489 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105633 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.489 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:08:19.489 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.610707064000003, 2024-09-14T09:08:19.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211941734, 2024-09-14T09:08:19.489 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.490 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.490 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:19.490 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.490 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.490 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.490 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105633 2024-09-14T09:08:19.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.490 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:08:19.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:08:19.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866640+0000", 2024-09-14T09:08:19.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9300000000000006e-07 2024-09-14T09:08:19.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866656+0000", 2024-09-14T09:08:19.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5291e-05 2024-09-14T09:08:19.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866669+0000", 2024-09-14T09:08:19.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2918000000000001e-05 2024-09-14T09:08:19.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866678+0000", 2024-09-14T09:08:19.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8369999999999992e-06 2024-09-14T09:08:19.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866771+0000", 2024-09-14T09:08:19.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2250999999999995e-05 2024-09-14T09:08:19.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866784+0000", 2024-09-14T09:08:19.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2991e-05 2024-09-14T09:08:19.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078566+0000", 2024-09-14T09:08:19.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117822570000001 2024-09-14T09:08:19.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078581+0000", 2024-09-14T09:08:19.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5196e-05 2024-09-14T09:08:19.494 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.494 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.494 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.494 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105635 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.494 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:08:19.494 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.610653237000001, 2024-09-14T09:08:19.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119313250000001, 2024-09-14T09:08:19.494 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.494 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.495 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:19.495 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.495 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.495 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.495 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105635 2024-09-14T09:08:19.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.495 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:08:19.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:08:19.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866694+0000", 2024-09-14T09:08:19.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0210000000000001e-06 2024-09-14T09:08:19.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866698+0000", 2024-09-14T09:08:19.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.765e-06 2024-09-14T09:08:19.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866710+0000", 2024-09-14T09:08:19.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2320000000000001e-05 2024-09-14T09:08:19.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866722+0000", 2024-09-14T09:08:19.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2133000000000001e-05 2024-09-14T09:08:19.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866830+0000", 2024-09-14T09:08:19.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000107668 2024-09-14T09:08:19.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866843+0000", 2024-09-14T09:08:19.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2991e-05 2024-09-14T09:08:19.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078616+0000", 2024-09-14T09:08:19.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117732459999999 2024-09-14T09:08:19.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078624+0000", 2024-09-14T09:08:19.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1810000000000001e-06 2024-09-14T09:08:19.499 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.499 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.499 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.499 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105636 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.499 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:08:19.500 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.610585340999997, 2024-09-14T09:08:19.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119085119999999, 2024-09-14T09:08:19.500 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.500 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.500 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:19.500 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.500 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.500 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.500 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105636 2024-09-14T09:08:19.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.500 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:08:19.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:08:19.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866762+0000", 2024-09-14T09:08:19.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4600000000000003e-07 2024-09-14T09:08:19.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866766+0000", 2024-09-14T09:08:19.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:08:19.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866779+0000", 2024-09-14T09:08:19.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3101999999999999e-05 2024-09-14T09:08:19.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866789+0000", 2024-09-14T09:08:19.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8039999999999996e-06 2024-09-14T09:08:19.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866889+0000", 2024-09-14T09:08:19.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100147 2024-09-14T09:08:19.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866902+0000", 2024-09-14T09:08:19.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3212e-05 2024-09-14T09:08:19.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078659+0000", 2024-09-14T09:08:19.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211756362 2024-09-14T09:08:19.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078670+0000", 2024-09-14T09:08:19.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0956e-05 2024-09-14T09:08:19.504 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.504 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.504 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.504 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105639 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.504 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:08:19.505 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.610526466000003, 2024-09-14T09:08:19.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118829259999999, 2024-09-14T09:08:19.505 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.505 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.505 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:19.505 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.505 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.505 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.505 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105639 2024-09-14T09:08:19.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.505 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:08:19.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:08:19.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866821+0000", 2024-09-14T09:08:19.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9800000000000002e-07 2024-09-14T09:08:19.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866825+0000", 2024-09-14T09:08:19.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9269999999999998e-06 2024-09-14T09:08:19.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866838+0000", 2024-09-14T09:08:19.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3040000000000001e-05 2024-09-14T09:08:19.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866848+0000", 2024-09-14T09:08:19.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7929999999999992e-06 2024-09-14T09:08:19.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866948+0000", 2024-09-14T09:08:19.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100337 2024-09-14T09:08:19.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866961+0000", 2024-09-14T09:08:19.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3106e-05 2024-09-14T09:08:19.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078693+0000", 2024-09-14T09:08:19.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117320380000001 2024-09-14T09:08:19.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078703+0000", 2024-09-14T09:08:19.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6870000000000006e-06 2024-09-14T09:08:19.509 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.509 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.509 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.509 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105641 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.509 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:08:19.509 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.610467601000003, 2024-09-14T09:08:19.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118727169999999, 2024-09-14T09:08:19.510 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.510 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.510 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:19.510 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.510 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.510 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.510 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105641 2024-09-14T09:08:19.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.510 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:08:19.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:08:19.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866880+0000", 2024-09-14T09:08:19.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7699999999999992e-07 2024-09-14T09:08:19.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866884+0000", 2024-09-14T09:08:19.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9480000000000001e-06 2024-09-14T09:08:19.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866897+0000", 2024-09-14T09:08:19.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2832e-05 2024-09-14T09:08:19.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866906+0000", 2024-09-14T09:08:19.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7869999999999996e-06 2024-09-14T09:08:19.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867007+0000", 2024-09-14T09:08:19.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010103900000000001 2024-09-14T09:08:19.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867020+0000", 2024-09-14T09:08:19.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2896e-05 2024-09-14T09:08:19.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078739+0000", 2024-09-14T09:08:19.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211718305 2024-09-14T09:08:19.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078752+0000", 2024-09-14T09:08:19.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2933e-05 2024-09-14T09:08:19.514 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.514 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.514 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.514 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105643 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.514 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:08:19.514 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.610409042000001, 2024-09-14T09:08:19.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211858353, 2024-09-14T09:08:19.515 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.515 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.515 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:19.515 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.515 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.515 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.515 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105643 2024-09-14T09:08:19.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.515 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:08:19.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:08:19.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866938+0000", 2024-09-14T09:08:19.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5600000000000004e-07 2024-09-14T09:08:19.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866942+0000", 2024-09-14T09:08:19.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0350000000000003e-06 2024-09-14T09:08:19.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866955+0000", 2024-09-14T09:08:19.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3097e-05 2024-09-14T09:08:19.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866965+0000", 2024-09-14T09:08:19.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8179999999999999e-06 2024-09-14T09:08:19.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867067+0000", 2024-09-14T09:08:19.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010128699999999999 2024-09-14T09:08:19.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867080+0000", 2024-09-14T09:08:19.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3089e-05 2024-09-14T09:08:19.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078786+0000", 2024-09-14T09:08:19.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117065250000001 2024-09-14T09:08:19.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078796+0000", 2024-09-14T09:08:19.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5459999999999997e-06 2024-09-14T09:08:19.521 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.521 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.521 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.521 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105645 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.521 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:08:19.522 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.610348647000002, 2024-09-14T09:08:19.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211848641, 2024-09-14T09:08:19.522 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.522 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.522 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:19.522 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.522 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.522 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.522 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105645 2024-09-14T09:08:19.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.522 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:08:19.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:08:19.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866999+0000", 2024-09-14T09:08:19.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7999999999999993e-07 2024-09-14T09:08:19.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867003+0000", 2024-09-14T09:08:19.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0090000000000001e-06 2024-09-14T09:08:19.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867016+0000", 2024-09-14T09:08:19.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3304e-05 2024-09-14T09:08:19.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867026+0000", 2024-09-14T09:08:19.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5889999999999992e-06 2024-09-14T09:08:19.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867128+0000", 2024-09-14T09:08:19.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000102101 2024-09-14T09:08:19.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867141+0000", 2024-09-14T09:08:19.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2903e-05 2024-09-14T09:08:19.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078833+0000", 2024-09-14T09:08:19.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116920959999999 2024-09-14T09:08:19.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078846+0000", 2024-09-14T09:08:19.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3658999999999999e-05 2024-09-14T09:08:19.526 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.526 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.526 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.526 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105647 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.526 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:08:19.526 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.610290128999999, 2024-09-14T09:08:19.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211830097, 2024-09-14T09:08:19.526 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.527 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.527 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:19.527 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.527 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.527 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.527 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105647 2024-09-14T09:08:19.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.527 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:08:19.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:08:19.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867057+0000", 2024-09-14T09:08:19.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.95e-07 2024-09-14T09:08:19.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867061+0000", 2024-09-14T09:08:19.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9530000000000001e-06 2024-09-14T09:08:19.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867074+0000", 2024-09-14T09:08:19.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3061000000000001e-05 2024-09-14T09:08:19.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867086+0000", 2024-09-14T09:08:19.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1460999999999999e-05 2024-09-14T09:08:19.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867186+0000", 2024-09-14T09:08:19.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100181 2024-09-14T09:08:19.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867199+0000", 2024-09-14T09:08:19.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-05 2024-09-14T09:08:19.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078874+0000", 2024-09-14T09:08:19.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211674683 2024-09-14T09:08:19.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078886+0000", 2024-09-14T09:08:19.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2853e-05 2024-09-14T09:08:19.531 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.531 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.531 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.531 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105648 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.531 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:08:19.531 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.610228642999999, 2024-09-14T09:08:19.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118073570000001, 2024-09-14T09:08:19.531 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.531 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.532 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:19.532 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.532 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.532 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.532 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105648 2024-09-14T09:08:19.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.532 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:08:19.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:08:19.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867119+0000", 2024-09-14T09:08:19.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8400000000000002e-07 2024-09-14T09:08:19.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867123+0000", 2024-09-14T09:08:19.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9639999999999997e-06 2024-09-14T09:08:19.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867136+0000", 2024-09-14T09:08:19.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.289e-05 2024-09-14T09:08:19.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867145+0000", 2024-09-14T09:08:19.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8919999999999995e-06 2024-09-14T09:08:19.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867245+0000", 2024-09-14T09:08:19.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9282000000000003e-05 2024-09-14T09:08:19.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867258+0000", 2024-09-14T09:08:19.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2935000000000001e-05 2024-09-14T09:08:19.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078910+0000", 2024-09-14T09:08:19.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211652352 2024-09-14T09:08:19.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078925+0000", 2024-09-14T09:08:19.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5058000000000001e-05 2024-09-14T09:08:19.536 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.536 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.536 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.536 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105652 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.536 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:08:19.536 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.610170623999998, 2024-09-14T09:08:19.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211775737, 2024-09-14T09:08:19.536 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.536 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.537 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:19.537 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.537 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.537 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.537 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105652 2024-09-14T09:08:19.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.537 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:08:19.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:08:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867177+0000", 2024-09-14T09:08:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9100000000000002e-07 2024-09-14T09:08:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867181+0000", 2024-09-14T09:08:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0029999999999996e-06 2024-09-14T09:08:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867193+0000", 2024-09-14T09:08:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2675e-05 2024-09-14T09:08:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867203+0000", 2024-09-14T09:08:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5599999999999999e-06 2024-09-14T09:08:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867306+0000", 2024-09-14T09:08:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010330100000000001 2024-09-14T09:08:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867319+0000", 2024-09-14T09:08:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3129000000000001e-05 2024-09-14T09:08:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078944+0000", 2024-09-14T09:08:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116247520000001 2024-09-14T09:08:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078952+0000", 2024-09-14T09:08:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4259999999999997e-06 2024-09-14T09:08:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.540 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105653 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:08:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.610111953999997, 2024-09-14T09:08:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211774895, 2024-09-14T09:08:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.541 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105653 2024-09-14T09:08:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:08:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:08:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867235+0000", 2024-09-14T09:08:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9400000000000004e-07 2024-09-14T09:08:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867239+0000", 2024-09-14T09:08:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9380000000000002e-06 2024-09-14T09:08:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867252+0000", 2024-09-14T09:08:19.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.31e-05 2024-09-14T09:08:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867264+0000", 2024-09-14T09:08:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1275e-05 2024-09-14T09:08:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867367+0000", 2024-09-14T09:08:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103755 2024-09-14T09:08:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867382+0000", 2024-09-14T09:08:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4724e-05 2024-09-14T09:08:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078995+0000", 2024-09-14T09:08:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116132669999999 2024-09-14T09:08:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079009+0000", 2024-09-14T09:08:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3942e-05 2024-09-14T09:08:19.545 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105654 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:08:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.610050897999997, 2024-09-14T09:08:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117364070000001, 2024-09-14T09:08:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105654 2024-09-14T09:08:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:08:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:08:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867296+0000", 2024-09-14T09:08:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2399999999999996e-07 2024-09-14T09:08:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867300+0000", 2024-09-14T09:08:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8689999999999997e-06 2024-09-14T09:08:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867313+0000", 2024-09-14T09:08:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2660000000000001e-05 2024-09-14T09:08:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867322+0000", 2024-09-14T09:08:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.533e-06 2024-09-14T09:08:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867429+0000", 2024-09-14T09:08:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106566 2024-09-14T09:08:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867442+0000", 2024-09-14T09:08:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3244000000000001e-05 2024-09-14T09:08:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079024+0000", 2024-09-14T09:08:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211581813 2024-09-14T09:08:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079032+0000", 2024-09-14T09:08:19.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7980000000000006e-06 2024-09-14T09:08:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105657 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:08:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.609989724999998, 2024-09-14T09:08:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211700657, 2024-09-14T09:08:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.551 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.552 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105657 2024-09-14T09:08:19.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.552 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:08:19.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:08:19.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867358+0000", 2024-09-14T09:08:19.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9700000000000005e-07 2024-09-14T09:08:19.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867367+0000", 2024-09-14T09:08:19.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8919999999999995e-06 2024-09-14T09:08:19.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867383+0000", 2024-09-14T09:08:19.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.579e-05 2024-09-14T09:08:19.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867393+0000", 2024-09-14T09:08:19.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3070000000000008e-06 2024-09-14T09:08:19.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867490+0000", 2024-09-14T09:08:19.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7703999999999998e-05 2024-09-14T09:08:19.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867503+0000", 2024-09-14T09:08:19.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2849e-05 2024-09-14T09:08:19.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079050+0000", 2024-09-14T09:08:19.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211546899 2024-09-14T09:08:19.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079057+0000", 2024-09-14T09:08:19.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3189999999999997e-06 2024-09-14T09:08:19.555 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.555 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.555 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.555 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105658 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.556 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:08:19.556 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.609927341000002, 2024-09-14T09:08:19.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211691217, 2024-09-14T09:08:19.556 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.556 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.556 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:19.556 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.556 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.556 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.556 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105658 2024-09-14T09:08:19.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.556 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:08:19.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:08:19.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867420+0000", 2024-09-14T09:08:19.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5000000000000001e-07 2024-09-14T09:08:19.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867424+0000", 2024-09-14T09:08:19.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8469999999999998e-06 2024-09-14T09:08:19.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867437+0000", 2024-09-14T09:08:19.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3156000000000001e-05 2024-09-14T09:08:19.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867448+0000", 2024-09-14T09:08:19.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1185e-05 2024-09-14T09:08:19.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867552+0000", 2024-09-14T09:08:19.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103362 2024-09-14T09:08:19.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867565+0000", 2024-09-14T09:08:19.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3018e-05 2024-09-14T09:08:19.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079092+0000", 2024-09-14T09:08:19.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211527013 2024-09-14T09:08:19.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079110+0000", 2024-09-14T09:08:19.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8686e-05 2024-09-14T09:08:19.560 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.560 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.560 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.560 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105662 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.561 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:08:19.561 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.609866275999998, 2024-09-14T09:08:19.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211652226, 2024-09-14T09:08:19.561 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.561 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.561 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:19.561 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.561 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.561 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.561 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105662 2024-09-14T09:08:19.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.561 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:08:19.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:08:19.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867481+0000", 2024-09-14T09:08:19.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1800000000000004e-07 2024-09-14T09:08:19.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867485+0000", 2024-09-14T09:08:19.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9199999999999997e-06 2024-09-14T09:08:19.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867498+0000", 2024-09-14T09:08:19.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2896e-05 2024-09-14T09:08:19.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867509+0000", 2024-09-14T09:08:19.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1202e-05 2024-09-14T09:08:19.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867611+0000", 2024-09-14T09:08:19.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010166999999999999 2024-09-14T09:08:19.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867624+0000", 2024-09-14T09:08:19.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2996000000000001e-05 2024-09-14T09:08:19.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079125+0000", 2024-09-14T09:08:19.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2115009299999999 2024-09-14T09:08:19.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079132+0000", 2024-09-14T09:08:19.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6939999999999995e-06 2024-09-14T09:08:19.565 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.565 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.565 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.566 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105663 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.566 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:08:19.566 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.609805225000002, 2024-09-14T09:08:19.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116218169999999, 2024-09-14T09:08:19.566 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.566 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.566 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:19.566 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.566 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.566 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.566 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105663 2024-09-14T09:08:19.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.566 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:08:19.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:08:19.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867542+0000", 2024-09-14T09:08:19.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1399999999999995e-07 2024-09-14T09:08:19.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867546+0000", 2024-09-14T09:08:19.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7900000000000001e-06 2024-09-14T09:08:19.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867559+0000", 2024-09-14T09:08:19.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2805e-05 2024-09-14T09:08:19.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867568+0000", 2024-09-14T09:08:19.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-09-14T09:08:19.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867669+0000", 2024-09-14T09:08:19.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100655 2024-09-14T09:08:19.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867682+0000", 2024-09-14T09:08:19.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3013000000000001e-05 2024-09-14T09:08:19.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079151+0000", 2024-09-14T09:08:19.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2114689089999999 2024-09-14T09:08:19.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079163+0000", 2024-09-14T09:08:19.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1962e-05 2024-09-14T09:08:19.570 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.570 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.570 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.570 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105665 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:19.570 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:08:19.570 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.609746594000001, 2024-09-14T09:08:19.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211584848, 2024-09-14T09:08:19.571 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:19.571 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:19.571 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:19.571 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:19.571 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:19.571 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:19.571 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105665 2024-09-14T09:08:19.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.571 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:19.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:19.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:08:19.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:19.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:08:19.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:19.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:19.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867601+0000", 2024-09-14T09:08:19.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.569999999999999e-07 2024-09-14T09:08:19.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:19.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867605+0000", 2024-09-14T09:08:19.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8589999999999998e-06 2024-09-14T09:08:19.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:19.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867617+0000", 2024-09-14T09:08:19.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2833000000000001e-05 2024-09-14T09:08:19.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:19.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867627+0000", 2024-09-14T09:08:19.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5219999999999996e-06 2024-09-14T09:08:19.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867728+0000", 2024-09-14T09:08:19.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010057500000000001 2024-09-14T09:08:19.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:19.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867740+0000", 2024-09-14T09:08:19.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2744e-05 2024-09-14T09:08:19.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:19.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079178+0000", 2024-09-14T09:08:19.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211437552 2024-09-14T09:08:19.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:19.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:19.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:19.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079185+0000", 2024-09-14T09:08:19.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8059999999999997e-06 2024-09-14T09:08:19.575 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.575 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.575 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.575 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:19.575 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:19.575 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:19.577 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:08:19.577 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:19.575+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:19.678 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:19.679+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:19.705 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:19.705 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:19.705 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:08:19.705 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:19.719 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:08:19.779 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:19.782+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:19.836 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:19.836 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:19.836 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:08:19.836 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:08:19.836 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:19.850 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:08:19.880 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:19.879+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:19.980 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:19.982+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:20.019 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:20.019 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:08:20.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:08:20.019 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:08:20.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.019 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65139 3.a 3:54c90b7d:::rbd_data.115e21c31fcf.0000000000000008:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:08:20.019 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:08:20.019 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 118.643175841, 2024-09-14T09:08:20.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14429895700000001, 2024-09-14T09:08:20.020 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.020 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.020 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:20.020 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.020 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.020 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.020 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65139 2024-09-14T09:08:20.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.020 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.020 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:08:20.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:08:20.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379715+0000", 2024-09-14T09:08:20.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.576e-06 2024-09-14T09:08:20.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379724+0000", 2024-09-14T09:08:20.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1779999999999992e-06 2024-09-14T09:08:20.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379746+0000", 2024-09-14T09:08:20.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1661000000000001e-05 2024-09-14T09:08:20.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379759+0000", 2024-09-14T09:08:20.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.324e-05 2024-09-14T09:08:20.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379882+0000", 2024-09-14T09:08:20.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012239400000000001 2024-09-14T09:08:20.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.380023+0000", 2024-09-14T09:08:20.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000141402 2024-09-14T09:08:20.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:08:20.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.391553+0000", 2024-09-14T09:08:20.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011530197000000001 2024-09-14T09:08:20.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:20.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523927+0000", 2024-09-14T09:08:20.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.132373935 2024-09-14T09:08:20.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:08:20.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523941+0000", 2024-09-14T09:08:20.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3326000000000001e-05 2024-09-14T09:08:20.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:08:20.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523977+0000", 2024-09-14T09:08:20.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6185e-05 2024-09-14T09:08:20.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.524012+0000", 2024-09-14T09:08:20.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4863000000000001e-05 2024-09-14T09:08:20.027 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.027 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.027 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.027 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107606 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 14336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:20.027 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:20.027 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.616718939999998, 2024-09-14T09:08:20.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14293513799999999, 2024-09-14T09:08:20.027 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.027 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.028 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.028 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.028 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.028 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.028 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107606 2024-09-14T09:08:20.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.028 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:20.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:20.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:20.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1600000000000001e-07 2024-09-14T09:08:20.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406174+0000", 2024-09-14T09:08:20.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4199999999999999e-06 2024-09-14T09:08:20.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406188+0000", 2024-09-14T09:08:20.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4114000000000001e-05 2024-09-14T09:08:20.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406198+0000", 2024-09-14T09:08:20.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.041e-05 2024-09-14T09:08:20.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.548700+0000", 2024-09-14T09:08:20.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14250141299999999 2024-09-14T09:08:20.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.548783+0000", 2024-09-14T09:08:20.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3578999999999994e-05 2024-09-14T09:08:20.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549105+0000", 2024-09-14T09:08:20.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032158600000000001 2024-09-14T09:08:20.032 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.032 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.032 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.033 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107608 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:20.033 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:20.033 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.616652903999999, 2024-09-14T09:08:20.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14603042799999999, 2024-09-14T09:08:20.033 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.033 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.033 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.033 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.033 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.033 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.033 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107608 2024-09-14T09:08:20.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.033 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:20.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:20.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:20.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1200000000000002e-07 2024-09-14T09:08:20.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406241+0000", 2024-09-14T09:08:20.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7539999999999997e-06 2024-09-14T09:08:20.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406275+0000", 2024-09-14T09:08:20.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3512e-05 2024-09-14T09:08:20.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406322+0000", 2024-09-14T09:08:20.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6764999999999997e-05 2024-09-14T09:08:20.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549149+0000", 2024-09-14T09:08:20.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14282725499999999 2024-09-14T09:08:20.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549198+0000", 2024-09-14T09:08:20.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9599999999999999e-05 2024-09-14T09:08:20.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552266+0000", 2024-09-14T09:08:20.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030677299999999998 2024-09-14T09:08:20.037 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.037 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.038 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.038 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107610 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 22528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:20.038 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:20.038 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.616531029000001, 2024-09-14T09:08:20.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146437919, 2024-09-14T09:08:20.038 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.039 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.039 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.039 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.039 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.039 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.039 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107610 2024-09-14T09:08:20.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.039 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:20.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:20.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:20.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.02e-07 2024-09-14T09:08:20.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406363+0000", 2024-09-14T09:08:20.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8099999999999997e-06 2024-09-14T09:08:20.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406390+0000", 2024-09-14T09:08:20.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7004000000000001e-05 2024-09-14T09:08:20.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406404+0000", 2024-09-14T09:08:20.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3682e-05 2024-09-14T09:08:20.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552335+0000", 2024-09-14T09:08:20.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14593077800000001 2024-09-14T09:08:20.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552431+0000", 2024-09-14T09:08:20.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6086000000000001e-05 2024-09-14T09:08:20.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552795+0000", 2024-09-14T09:08:20.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000364657 2024-09-14T09:08:20.044 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.044 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.045 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.045 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107612 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 26624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:20.045 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:08:20.045 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.616465765999997, 2024-09-14T09:08:20.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146762061, 2024-09-14T09:08:20.045 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.045 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.045 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.045 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.045 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.045 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.045 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107612 2024-09-14T09:08:20.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.046 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:08:20.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:08:20.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406424+0000", 2024-09-14T09:08:20.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2600000000000002e-07 2024-09-14T09:08:20.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406451+0000", 2024-09-14T09:08:20.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7250000000000002e-05 2024-09-14T09:08:20.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406487+0000", 2024-09-14T09:08:20.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6086e-05 2024-09-14T09:08:20.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406498+0000", 2024-09-14T09:08:20.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0858e-05 2024-09-14T09:08:20.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552821+0000", 2024-09-14T09:08:20.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14632319099999999 2024-09-14T09:08:20.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552875+0000", 2024-09-14T09:08:20.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4262999999999999e-05 2024-09-14T09:08:20.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553185+0000", 2024-09-14T09:08:20.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000309687 2024-09-14T09:08:20.049 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.049 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.049 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.050 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107614 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:20.050 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:08:20.050 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.616378232000002, 2024-09-14T09:08:20.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14737121, 2024-09-14T09:08:20.050 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.050 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.050 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.050 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.050 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.050 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.050 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107614 2024-09-14T09:08:20.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.051 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:08:20.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:08:20.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406511+0000", 2024-09-14T09:08:20.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1799999999999995e-07 2024-09-14T09:08:20.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406514+0000", 2024-09-14T09:08:20.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8660000000000002e-06 2024-09-14T09:08:20.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406523+0000", 2024-09-14T09:08:20.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1940000000000004e-06 2024-09-14T09:08:20.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406531+0000", 2024-09-14T09:08:20.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3159999999999997e-06 2024-09-14T09:08:20.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553233+0000", 2024-09-14T09:08:20.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14670117899999999 2024-09-14T09:08:20.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553334+0000", 2024-09-14T09:08:20.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100982 2024-09-14T09:08:20.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553882+0000", 2024-09-14T09:08:20.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00054805499999999998 2024-09-14T09:08:20.055 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.055 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.056 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.056 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107616 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 34816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:20.056 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:08:20.056 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.616346597000003, 2024-09-14T09:08:20.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14780390500000001, 2024-09-14T09:08:20.057 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.057 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.057 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.057 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.057 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.058 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.058 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107616 2024-09-14T09:08:20.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.058 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:08:20.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:08:20.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406543+0000", 2024-09-14T09:08:20.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0200000000000001e-07 2024-09-14T09:08:20.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406545+0000", 2024-09-14T09:08:20.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3360000000000002e-06 2024-09-14T09:08:20.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406552+0000", 2024-09-14T09:08:20.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9779999999999999e-06 2024-09-14T09:08:20.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406559+0000", 2024-09-14T09:08:20.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6919999999999996e-06 2024-09-14T09:08:20.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553917+0000", 2024-09-14T09:08:20.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14735791600000001 2024-09-14T09:08:20.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554033+0000", 2024-09-14T09:08:20.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116047 2024-09-14T09:08:20.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554346+0000", 2024-09-14T09:08:20.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031323400000000002 2024-09-14T09:08:20.064 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.064 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.064 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.065 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107618 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 38912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:20.065 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:20.065 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.616317797000001, 2024-09-14T09:08:20.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14816009599999999, 2024-09-14T09:08:20.065 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.065 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.065 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.066 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.066 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.066 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.066 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107618 2024-09-14T09:08:20.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.066 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:20.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:20.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:20.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9999999999999998e-07 2024-09-14T09:08:20.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406574+0000", 2024-09-14T09:08:20.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2560000000000001e-06 2024-09-14T09:08:20.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406581+0000", 2024-09-14T09:08:20.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1539999999999996e-06 2024-09-14T09:08:20.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406586+0000", 2024-09-14T09:08:20.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.904e-06 2024-09-14T09:08:20.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554385+0000", 2024-09-14T09:08:20.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.147799443 2024-09-14T09:08:20.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554441+0000", 2024-09-14T09:08:20.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5921000000000002e-05 2024-09-14T09:08:20.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554731+0000", 2024-09-14T09:08:20.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028991800000000001 2024-09-14T09:08:20.069 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.069 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.069 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.069 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107620 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:20.069 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:20.070 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.616296015000003, 2024-09-14T09:08:20.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.148512643, 2024-09-14T09:08:20.070 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.070 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.070 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.070 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.070 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.070 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.070 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107620 2024-09-14T09:08:20.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.071 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:20.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:20.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:20.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9999999999999998e-07 2024-09-14T09:08:20.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406595+0000", 2024-09-14T09:08:20.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9360000000000002e-06 2024-09-14T09:08:20.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406605+0000", 2024-09-14T09:08:20.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0268000000000001e-05 2024-09-14T09:08:20.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406614+0000", 2024-09-14T09:08:20.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7180000000000002e-06 2024-09-14T09:08:20.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554750+0000", 2024-09-14T09:08:20.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14813599699999999 2024-09-14T09:08:20.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554794+0000", 2024-09-14T09:08:20.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.401e-05 2024-09-14T09:08:20.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555105+0000", 2024-09-14T09:08:20.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031131399999999998 2024-09-14T09:08:20.074 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.074 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.074 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.074 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107622 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:20.074 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:20.074 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.616264759000003, 2024-09-14T09:08:20.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14902820899999999, 2024-09-14T09:08:20.074 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.074 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.075 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.075 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.075 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.075 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.075 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107622 2024-09-14T09:08:20.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.075 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:20.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:20.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:20.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9200000000000001e-07 2024-09-14T09:08:20.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406636+0000", 2024-09-14T09:08:20.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.189e-05 2024-09-14T09:08:20.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406649+0000", 2024-09-14T09:08:20.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2442e-05 2024-09-14T09:08:20.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406658+0000", 2024-09-14T09:08:20.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3239999999999992e-06 2024-09-14T09:08:20.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555287+0000", 2024-09-14T09:08:20.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14862868400000001 2024-09-14T09:08:20.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555348+0000", 2024-09-14T09:08:20.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1686999999999994e-05 2024-09-14T09:08:20.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.079 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.079 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555652+0000", 2024-09-14T09:08:20.079 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00030369000000000001 2024-09-14T09:08:20.079 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.079 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.079 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.080 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.080 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107624 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:20.080 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:20.080 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.616221066000001, 2024-09-14T09:08:20.080 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14934019900000001, 2024-09-14T09:08:20.081 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.081 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.081 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.081 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.081 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.081 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.082 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107624 2024-09-14T09:08:20.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.082 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.082 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.082 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.082 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:20.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:20.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.084 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:20.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1399999999999997e-07 2024-09-14T09:08:20.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406670+0000", 2024-09-14T09:08:20.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4099999999999998e-06 2024-09-14T09:08:20.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.086 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.086 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406680+0000", 2024-09-14T09:08:20.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6460000000000005e-06 2024-09-14T09:08:20.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.086 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.087 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406688+0000", 2024-09-14T09:08:20.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5859999999999999e-06 2024-09-14T09:08:20.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.087 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.087 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555693+0000", 2024-09-14T09:08:20.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14900544499999999 2024-09-14T09:08:20.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555743+0000", 2024-09-14T09:08:20.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9966999999999998e-05 2024-09-14T09:08:20.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556008+0000", 2024-09-14T09:08:20.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026463099999999998 2024-09-14T09:08:20.090 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.090 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.090 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.091 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107626 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:20.091 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:08:20.091 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.616189091999999, 2024-09-14T09:08:20.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14983149500000001, 2024-09-14T09:08:20.091 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.091 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.092 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.092 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.092 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.092 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.092 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107626 2024-09-14T09:08:20.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.093 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:08:20.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:08:20.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408087+0000", 2024-09-14T09:08:20.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001387735 2024-09-14T09:08:20.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408094+0000", 2024-09-14T09:08:20.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9399999999999996e-06 2024-09-14T09:08:20.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408122+0000", 2024-09-14T09:08:20.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8076e-05 2024-09-14T09:08:20.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408147+0000", 2024-09-14T09:08:20.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4726e-05 2024-09-14T09:08:20.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556048+0000", 2024-09-14T09:08:20.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14790144299999999 2024-09-14T09:08:20.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556175+0000", 2024-09-14T09:08:20.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012663 2024-09-14T09:08:20.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556531+0000", 2024-09-14T09:08:20.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035594499999999998 2024-09-14T09:08:20.101 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.101 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.101 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.102 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107628 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:20.102 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:08:20.102 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.614719481000002, 2024-09-14T09:08:20.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14882648900000001, 2024-09-14T09:08:20.102 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.102 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.103 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.103 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.103 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.103 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.103 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107628 2024-09-14T09:08:20.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.104 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:08:20.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:08:20.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408878+0000", 2024-09-14T09:08:20.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070858800000000001 2024-09-14T09:08:20.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408885+0000", 2024-09-14T09:08:20.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4839999999999998e-06 2024-09-14T09:08:20.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408900+0000", 2024-09-14T09:08:20.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5174000000000001e-05 2024-09-14T09:08:20.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408918+0000", 2024-09-14T09:08:20.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.808e-05 2024-09-14T09:08:20.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556572+0000", 2024-09-14T09:08:20.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14765399500000001 2024-09-14T09:08:20.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556651+0000", 2024-09-14T09:08:20.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8919999999999997e-05 2024-09-14T09:08:20.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556996+0000", 2024-09-14T09:08:20.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034424800000000001 2024-09-14T09:08:20.109 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.109 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.109 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.110 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107630 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 63488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:20.110 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:08:20.110 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.613950647000003, 2024-09-14T09:08:20.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14855159200000001, 2024-09-14T09:08:20.110 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.110 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.110 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.110 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.110 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.110 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.110 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107630 2024-09-14T09:08:20.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.110 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:08:20.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:08:20.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409510+0000", 2024-09-14T09:08:20.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00057189700000000003 2024-09-14T09:08:20.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409518+0000", 2024-09-14T09:08:20.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7619999999999995e-06 2024-09-14T09:08:20.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409534+0000", 2024-09-14T09:08:20.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.594e-05 2024-09-14T09:08:20.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409546+0000", 2024-09-14T09:08:20.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2604e-05 2024-09-14T09:08:20.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557048+0000", 2024-09-14T09:08:20.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14750137899999999 2024-09-14T09:08:20.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557117+0000", 2024-09-14T09:08:20.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8998999999999994e-05 2024-09-14T09:08:20.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557490+0000", 2024-09-14T09:08:20.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037301100000000002 2024-09-14T09:08:20.115 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.115 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.115 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.115 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107632 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:20.115 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:08:20.115 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.613319701999998, 2024-09-14T09:08:20.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14833944199999999, 2024-09-14T09:08:20.115 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.115 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.115 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.115 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.116 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.116 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.116 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107632 2024-09-14T09:08:20.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.116 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:08:20.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:08:20.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412609+0000", 2024-09-14T09:08:20.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003040105 2024-09-14T09:08:20.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412618+0000", 2024-09-14T09:08:20.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2960000000000004e-06 2024-09-14T09:08:20.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412640+0000", 2024-09-14T09:08:20.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1773999999999999e-05 2024-09-14T09:08:20.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412655+0000", 2024-09-14T09:08:20.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4538e-05 2024-09-14T09:08:20.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557538+0000", 2024-09-14T09:08:20.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14488311800000001 2024-09-14T09:08:20.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557594+0000", 2024-09-14T09:08:20.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6122999999999998e-05 2024-09-14T09:08:20.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557908+0000", 2024-09-14T09:08:20.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031448799999999997 2024-09-14T09:08:20.119 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.119 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.119 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.120 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107634 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:20.120 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:08:20.120 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.610202055000002, 2024-09-14T09:08:20.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.145666186, 2024-09-14T09:08:20.120 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.120 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.120 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.120 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.120 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.120 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.120 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107634 2024-09-14T09:08:20.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.120 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:08:20.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:08:20.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412688+0000", 2024-09-14T09:08:20.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9000000000000005e-07 2024-09-14T09:08:20.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412707+0000", 2024-09-14T09:08:20.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9232000000000002e-05 2024-09-14T09:08:20.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412722+0000", 2024-09-14T09:08:20.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5722e-05 2024-09-14T09:08:20.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412733+0000", 2024-09-14T09:08:20.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0909e-05 2024-09-14T09:08:20.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557946+0000", 2024-09-14T09:08:20.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14521263100000001 2024-09-14T09:08:20.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558020+0000", 2024-09-14T09:08:20.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4294000000000004e-05 2024-09-14T09:08:20.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558353+0000", 2024-09-14T09:08:20.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033240800000000002 2024-09-14T09:08:20.124 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.124 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.124 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.125 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107636 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:20.125 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:08:20.125 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.610137752, 2024-09-14T09:08:20.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14599377399999999, 2024-09-14T09:08:20.125 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.125 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.125 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.125 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.125 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.125 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.125 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107636 2024-09-14T09:08:20.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.126 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:08:20.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:08:20.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412752+0000", 2024-09-14T09:08:20.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.156e-06 2024-09-14T09:08:20.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412770+0000", 2024-09-14T09:08:20.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7521999999999999e-05 2024-09-14T09:08:20.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412790+0000", 2024-09-14T09:08:20.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0942e-05 2024-09-14T09:08:20.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412809+0000", 2024-09-14T09:08:20.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8887999999999999e-05 2024-09-14T09:08:20.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558387+0000", 2024-09-14T09:08:20.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14557798799999999 2024-09-14T09:08:20.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558448+0000", 2024-09-14T09:08:20.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0862000000000001e-05 2024-09-14T09:08:20.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558745+0000", 2024-09-14T09:08:20.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029641599999999998 2024-09-14T09:08:20.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.129 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.129 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107638 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:20.129 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:08:20.129 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.610050495999999, 2024-09-14T09:08:20.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146344799, 2024-09-14T09:08:20.130 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.130 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.130 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.130 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.130 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.130 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.130 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107638 2024-09-14T09:08:20.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.130 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:08:20.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:08:20.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412839+0000", 2024-09-14T09:08:20.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0720000000000001e-06 2024-09-14T09:08:20.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412844+0000", 2024-09-14T09:08:20.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7759999999999997e-06 2024-09-14T09:08:20.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412857+0000", 2024-09-14T09:08:20.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3138e-05 2024-09-14T09:08:20.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412867+0000", 2024-09-14T09:08:20.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7229999999999999e-06 2024-09-14T09:08:20.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558785+0000", 2024-09-14T09:08:20.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14591828700000001 2024-09-14T09:08:20.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558833+0000", 2024-09-14T09:08:20.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7790000000000002e-05 2024-09-14T09:08:20.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559183+0000", 2024-09-14T09:08:20.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035001299999999998 2024-09-14T09:08:20.133 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.134 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.134 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.134 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107640 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:20.134 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:08:20.134 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.610005344999998, 2024-09-14T09:08:20.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146686804, 2024-09-14T09:08:20.134 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.134 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.134 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.134 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.135 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.135 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.135 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107640 2024-09-14T09:08:20.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.135 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:08:20.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:08:20.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420446+0000", 2024-09-14T09:08:20.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075632529999999998 2024-09-14T09:08:20.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420456+0000", 2024-09-14T09:08:20.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0850000000000003e-06 2024-09-14T09:08:20.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420475+0000", 2024-09-14T09:08:20.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9406e-05 2024-09-14T09:08:20.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420488+0000", 2024-09-14T09:08:20.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3278000000000001e-05 2024-09-14T09:08:20.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559205+0000", 2024-09-14T09:08:20.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13871644999999999 2024-09-14T09:08:20.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559252+0000", 2024-09-14T09:08:20.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7666000000000001e-05 2024-09-14T09:08:20.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559570+0000", 2024-09-14T09:08:20.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031766599999999998 2024-09-14T09:08:20.138 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.138 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.138 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.139 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107660 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 124928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:20.139 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:08:20.139 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.601935103000002, 2024-09-14T09:08:20.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.142954628, 2024-09-14T09:08:20.139 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.139 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.139 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.139 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.139 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.139 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.139 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107660 2024-09-14T09:08:20.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.139 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:08:20.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:08:20.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428190+0000", 2024-09-14T09:08:20.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072360339999999997 2024-09-14T09:08:20.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428398+0000", 2024-09-14T09:08:20.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020898499999999999 2024-09-14T09:08:20.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428458+0000", 2024-09-14T09:08:20.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9985999999999998e-05 2024-09-14T09:08:20.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428480+0000", 2024-09-14T09:08:20.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1284000000000002e-05 2024-09-14T09:08:20.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563583+0000", 2024-09-14T09:08:20.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13510361800000001 2024-09-14T09:08:20.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563644+0000", 2024-09-14T09:08:20.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1092000000000004e-05 2024-09-14T09:08:20.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563908+0000", 2024-09-14T09:08:20.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026362899999999998 2024-09-14T09:08:20.142 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.142 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.143 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.143 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.143 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.143 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:20.145 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:08:20.145 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:20.143+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:20.246 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:20.247+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:20.313 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:20.313 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:20.314 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:08:20.314 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:20.328 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:08:20.373 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:20.378+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:20.447 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:20.450+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:20.453 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:20.453 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:20.453 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:08:20.454 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:08:20.454 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:20.468 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:08:20.548 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:20.547+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:20.642 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:20.642 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:08:20.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:08:20.642 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:08:20.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.642 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38593 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [watch reconnect cookie 139718844451680 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:08:20.642 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:08:20.642 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 138.82708519799999, 2024-09-14T09:08:20.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0218296630000001, 2024-09-14T09:08:20.642 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.642 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.642 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.643 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.643 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.643 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.643 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38593 2024-09-14T09:08:20.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.643 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:08:20.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:08:20.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818420+0000", 2024-09-14T09:08:20.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8399999999999999e-06 2024-09-14T09:08:20.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818431+0000", 2024-09-14T09:08:20.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0859999999999999e-05 2024-09-14T09:08:20.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818466+0000", 2024-09-14T09:08:20.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4486999999999997e-05 2024-09-14T09:08:20.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818481+0000", 2024-09-14T09:08:20.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5362999999999998e-05 2024-09-14T09:08:20.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818659+0000", 2024-09-14T09:08:20.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017807500000000001 2024-09-14T09:08:20.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op must wait for map", 2024-09-14T09:08:20.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818681+0000", 2024-09-14T09:08:20.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2096999999999999e-05 2024-09-14T09:08:20.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825472+0000", 2024-09-14T09:08:20.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067905170000000003 2024-09-14T09:08:20.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:20.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825491+0000", 2024-09-14T09:08:20.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9372999999999999e-05 2024-09-14T09:08:20.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835402+0000", 2024-09-14T09:08:20.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0099111620000001 2024-09-14T09:08:20.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835548+0000", 2024-09-14T09:08:20.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014607300000000001 2024-09-14T09:08:20.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.840247+0000", 2024-09-14T09:08:20.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046988159999999998 2024-09-14T09:08:20.647 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.647 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.647 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.648 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38615 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:08:20.648 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:08:20.648 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 138.77871999000001, 2024-09-14T09:08:20.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.97014128899999996, 2024-09-14T09:08:20.648 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.648 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.648 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:20.648 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.648 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.648 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.648 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38615 2024-09-14T09:08:20.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.648 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:08:20.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:08:20.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866787+0000", 2024-09-14T09:08:20.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.476e-06 2024-09-14T09:08:20.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866806+0000", 2024-09-14T09:08:20.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8606000000000001e-05 2024-09-14T09:08:20.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866839+0000", 2024-09-14T09:08:20.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3510000000000003e-05 2024-09-14T09:08:20.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866867+0000", 2024-09-14T09:08:20.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7610000000000002e-05 2024-09-14T09:08:20.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867031+0000", 2024-09-14T09:08:20.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016414199999999999 2024-09-14T09:08:20.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:20.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867054+0000", 2024-09-14T09:08:20.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2693000000000001e-05 2024-09-14T09:08:20.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836462+0000", 2024-09-14T09:08:20.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.96940797700000003 2024-09-14T09:08:20.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:20.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836547+0000", 2024-09-14T09:08:20.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5294e-05 2024-09-14T09:08:20.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836924+0000", 2024-09-14T09:08:20.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037698099999999997 2024-09-14T09:08:20.652 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.652 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.652 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.652 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105723 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:20.652 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:20.653 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.550285946000002, 2024-09-14T09:08:20.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77512799700000001, 2024-09-14T09:08:20.653 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.653 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.653 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:20.653 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.653 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.653 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.653 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105723 2024-09-14T09:08:20.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.653 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:20.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:20.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:20.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1399999999999997e-07 2024-09-14T09:08:20.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095220+0000", 2024-09-14T09:08:20.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0510000000000001e-06 2024-09-14T09:08:20.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095228+0000", 2024-09-14T09:08:20.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4159999999999998e-06 2024-09-14T09:08:20.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095234+0000", 2024-09-14T09:08:20.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3010000000000003e-06 2024-09-14T09:08:20.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103902+0000", 2024-09-14T09:08:20.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086678299999999996 2024-09-14T09:08:20.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:20.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103905+0000", 2024-09-14T09:08:20.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5020000000000001e-06 2024-09-14T09:08:20.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870332+0000", 2024-09-14T09:08:20.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76642700900000005 2024-09-14T09:08:20.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870345+0000", 2024-09-14T09:08:20.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2274e-05 2024-09-14T09:08:20.657 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.657 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.657 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.657 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105724 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:20.657 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:08:20.657 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.550261433000003, 2024-09-14T09:08:20.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77515698099999997, 2024-09-14T09:08:20.657 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.657 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.657 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:20.657 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.658 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.658 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.658 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105724 2024-09-14T09:08:20.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.658 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:08:20.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:08:20.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095242+0000", 2024-09-14T09:08:20.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9400000000000001e-07 2024-09-14T09:08:20.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095243+0000", 2024-09-14T09:08:20.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7540000000000001e-06 2024-09-14T09:08:20.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095248+0000", 2024-09-14T09:08:20.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1320000000000004e-06 2024-09-14T09:08:20.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095251+0000", 2024-09-14T09:08:20.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2770000000000001e-06 2024-09-14T09:08:20.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103947+0000", 2024-09-14T09:08:20.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086960089999999993 2024-09-14T09:08:20.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:20.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103955+0000", 2024-09-14T09:08:20.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9519999999999994e-06 2024-09-14T09:08:20.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870386+0000", 2024-09-14T09:08:20.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76643141100000001 2024-09-14T09:08:20.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870398+0000", 2024-09-14T09:08:20.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2052000000000001e-05 2024-09-14T09:08:20.661 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.661 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.661 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.661 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105727 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:20.662 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:20.662 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.550221727, 2024-09-14T09:08:20.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517292900000001, 2024-09-14T09:08:20.662 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.662 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.662 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:20.662 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.662 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.662 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.662 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105727 2024-09-14T09:08:20.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.662 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:20.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:20.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:20.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3300000000000002e-07 2024-09-14T09:08:20.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095284+0000", 2024-09-14T09:08:20.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9790000000000002e-06 2024-09-14T09:08:20.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095292+0000", 2024-09-14T09:08:20.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3810000000000001e-06 2024-09-14T09:08:20.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095298+0000", 2024-09-14T09:08:20.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9710000000000001e-06 2024-09-14T09:08:20.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103987+0000", 2024-09-14T09:08:20.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086893820000000007 2024-09-14T09:08:20.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:20.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103991+0000", 2024-09-14T09:08:20.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5480000000000002e-06 2024-09-14T09:08:20.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870442+0000", 2024-09-14T09:08:20.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.766451145 2024-09-14T09:08:20.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870454+0000", 2024-09-14T09:08:20.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.199e-05 2024-09-14T09:08:20.666 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.666 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.667 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.667 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105729 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:20.667 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:08:20.667 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.550173370000003, 2024-09-14T09:08:20.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517686500000005, 2024-09-14T09:08:20.667 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.667 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.667 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:20.667 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.667 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.667 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.667 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105729 2024-09-14T09:08:20.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.667 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:08:20.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:08:20.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095330+0000", 2024-09-14T09:08:20.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-09-14T09:08:20.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095332+0000", 2024-09-14T09:08:20.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1560000000000002e-06 2024-09-14T09:08:20.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095338+0000", 2024-09-14T09:08:20.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.592e-06 2024-09-14T09:08:20.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095341+0000", 2024-09-14T09:08:20.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8139999999999998e-06 2024-09-14T09:08:20.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104031+0000", 2024-09-14T09:08:20.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086896330000000004 2024-09-14T09:08:20.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:20.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104039+0000", 2024-09-14T09:08:20.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0900000000000005e-06 2024-09-14T09:08:20.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870495+0000", 2024-09-14T09:08:20.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76645611199999997 2024-09-14T09:08:20.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870506+0000", 2024-09-14T09:08:20.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0986e-05 2024-09-14T09:08:20.671 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.671 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.671 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.671 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105731 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:20.671 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:20.671 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.550155103000002, 2024-09-14T09:08:20.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77520598600000001, 2024-09-14T09:08:20.672 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.672 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.672 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:20.672 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.672 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.672 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.672 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105731 2024-09-14T09:08:20.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.672 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:20.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:20.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:20.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8599999999999999e-07 2024-09-14T09:08:20.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095350+0000", 2024-09-14T09:08:20.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1110000000000002e-06 2024-09-14T09:08:20.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095357+0000", 2024-09-14T09:08:20.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9020000000000002e-06 2024-09-14T09:08:20.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095360+0000", 2024-09-14T09:08:20.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1939999999999998e-06 2024-09-14T09:08:20.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104073+0000", 2024-09-14T09:08:20.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087131210000000008 2024-09-14T09:08:20.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:20.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104077+0000", 2024-09-14T09:08:20.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.501e-06 2024-09-14T09:08:20.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870541+0000", 2024-09-14T09:08:20.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76646461700000001 2024-09-14T09:08:20.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870554+0000", 2024-09-14T09:08:20.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2153999999999999e-05 2024-09-14T09:08:20.676 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.676 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.676 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.676 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105733 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:20.676 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:20.676 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.550135072000003, 2024-09-14T09:08:20.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.775237277, 2024-09-14T09:08:20.676 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.676 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.676 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:20.676 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.676 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.676 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.676 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105733 2024-09-14T09:08:20.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.677 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:20.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:20.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:20.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8699999999999995e-07 2024-09-14T09:08:20.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095371+0000", 2024-09-14T09:08:20.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5450000000000002e-06 2024-09-14T09:08:20.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095377+0000", 2024-09-14T09:08:20.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.776e-06 2024-09-14T09:08:20.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095383+0000", 2024-09-14T09:08:20.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3129999999999998e-06 2024-09-14T09:08:20.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104124+0000", 2024-09-14T09:08:20.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087409840000000003 2024-09-14T09:08:20.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:20.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104132+0000", 2024-09-14T09:08:20.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3159999999999997e-06 2024-09-14T09:08:20.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870593+0000", 2024-09-14T09:08:20.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76646064800000002 2024-09-14T09:08:20.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870605+0000", 2024-09-14T09:08:20.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2208000000000001e-05 2024-09-14T09:08:20.680 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.680 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.680 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.680 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105734 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:20.680 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:20.680 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.550111989999998, 2024-09-14T09:08:20.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77526569599999995, 2024-09-14T09:08:20.681 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.681 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.681 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:20.681 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.681 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.681 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.681 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105734 2024-09-14T09:08:20.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.681 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:20.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:20.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:20.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7399999999999998e-07 2024-09-14T09:08:20.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095394+0000", 2024-09-14T09:08:20.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8030000000000001e-06 2024-09-14T09:08:20.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095401+0000", 2024-09-14T09:08:20.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.046e-06 2024-09-14T09:08:20.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095407+0000", 2024-09-14T09:08:20.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6060000000000002e-06 2024-09-14T09:08:20.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104167+0000", 2024-09-14T09:08:20.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087602530000000008 2024-09-14T09:08:20.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:20.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104171+0000", 2024-09-14T09:08:20.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7620000000000001e-06 2024-09-14T09:08:20.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870645+0000", 2024-09-14T09:08:20.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76647400600000004 2024-09-14T09:08:20.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870656+0000", 2024-09-14T09:08:20.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1746000000000001e-05 2024-09-14T09:08:20.685 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.685 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.685 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.685 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105738 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:20.685 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:08:20.685 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.550072262999997, 2024-09-14T09:08:20.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77527530700000002, 2024-09-14T09:08:20.685 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.686 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.686 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:20.686 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.686 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.686 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.686 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105738 2024-09-14T09:08:20.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.686 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:08:20.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:08:20.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095431+0000", 2024-09-14T09:08:20.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0999999999999999e-07 2024-09-14T09:08:20.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095434+0000", 2024-09-14T09:08:20.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.757e-06 2024-09-14T09:08:20.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095442+0000", 2024-09-14T09:08:20.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9929999999999995e-06 2024-09-14T09:08:20.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095448+0000", 2024-09-14T09:08:20.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1340000000000001e-06 2024-09-14T09:08:20.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104213+0000", 2024-09-14T09:08:20.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087650000000000002 2024-09-14T09:08:20.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:20.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104221+0000", 2024-09-14T09:08:20.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4589999999999998e-06 2024-09-14T09:08:20.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870694+0000", 2024-09-14T09:08:20.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76647303200000005 2024-09-14T09:08:20.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870706+0000", 2024-09-14T09:08:20.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1522000000000001e-05 2024-09-14T09:08:20.689 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.689 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.689 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105739 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.550047734000003, 2024-09-14T09:08:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77530221700000002, 2024-09-14T09:08:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105739 2024-09-14T09:08:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:20.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:20.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:20.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.15e-07 2024-09-14T09:08:20.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095457+0000", 2024-09-14T09:08:20.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6470000000000001e-06 2024-09-14T09:08:20.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095461+0000", 2024-09-14T09:08:20.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0910000000000003e-06 2024-09-14T09:08:20.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095464+0000", 2024-09-14T09:08:20.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2059999999999999e-06 2024-09-14T09:08:20.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104253+0000", 2024-09-14T09:08:20.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087884120000000007 2024-09-14T09:08:20.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:20.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104256+0000", 2024-09-14T09:08:20.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5439999999999999e-06 2024-09-14T09:08:20.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870745+0000", 2024-09-14T09:08:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76648908000000004 2024-09-14T09:08:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870757+0000", 2024-09-14T09:08:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1822e-05 2024-09-14T09:08:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105741 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:20.694 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:08:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.549978608000004, 2024-09-14T09:08:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77528469899999997, 2024-09-14T09:08:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105741 2024-09-14T09:08:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:08:20.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:08:20.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095525+0000", 2024-09-14T09:08:20.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3740000000000001e-06 2024-09-14T09:08:20.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095533+0000", 2024-09-14T09:08:20.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6380000000000004e-06 2024-09-14T09:08:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095549+0000", 2024-09-14T09:08:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5554999999999999e-05 2024-09-14T09:08:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095566+0000", 2024-09-14T09:08:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7146000000000002e-05 2024-09-14T09:08:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104297+0000", 2024-09-14T09:08:20.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087315750000000001 2024-09-14T09:08:20.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:20.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104305+0000", 2024-09-14T09:08:20.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1389999999999995e-06 2024-09-14T09:08:20.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870797+0000", 2024-09-14T09:08:20.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76649129599999999 2024-09-14T09:08:20.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870809+0000", 2024-09-14T09:08:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1976e-05 2024-09-14T09:08:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105742 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:08:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.549913824000001, 2024-09-14T09:08:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.775271873, 2024-09-14T09:08:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.699 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.700 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.700 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105742 2024-09-14T09:08:20.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.700 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:08:20.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:08:20.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095590+0000", 2024-09-14T09:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0400000000000005e-07 2024-09-14T09:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095594+0000", 2024-09-14T09:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5639999999999998e-06 2024-09-14T09:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095620+0000", 2024-09-14T09:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5494999999999999e-05 2024-09-14T09:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095639+0000", 2024-09-14T09:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9639999999999998e-05 2024-09-14T09:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104340+0000", 2024-09-14T09:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087004950000000008 2024-09-14T09:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104343+0000", 2024-09-14T09:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4020000000000002e-06 2024-09-14T09:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870850+0000", 2024-09-14T09:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76650648499999996 2024-09-14T09:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870861+0000", 2024-09-14T09:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0888e-05 2024-09-14T09:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105745 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:08:20.703 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.549815209000002, 2024-09-14T09:08:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77521998700000005, 2024-09-14T09:08:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105745 2024-09-14T09:08:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:08:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:08:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095688+0000", 2024-09-14T09:08:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2699999999999998e-07 2024-09-14T09:08:20.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095693+0000", 2024-09-14T09:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0479999999999998e-06 2024-09-14T09:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095707+0000", 2024-09-14T09:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.342e-05 2024-09-14T09:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095716+0000", 2024-09-14T09:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9800000000000004e-06 2024-09-14T09:08:20.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104388+0000", 2024-09-14T09:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086717349999999999 2024-09-14T09:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104396+0000", 2024-09-14T09:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1769999999999998e-06 2024-09-14T09:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870897+0000", 2024-09-14T09:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76650102600000003 2024-09-14T09:08:20.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870907+0000", 2024-09-14T09:08:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0674e-05 2024-09-14T09:08:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105747 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:20.708 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.549768989, 2024-09-14T09:08:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77522190300000005, 2024-09-14T09:08:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105747 2024-09-14T09:08:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:20.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:20.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8999999999999995e-07 2024-09-14T09:08:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095738+0000", 2024-09-14T09:08:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8430000000000003e-06 2024-09-14T09:08:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095750+0000", 2024-09-14T09:08:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1229999999999999e-05 2024-09-14T09:08:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095758+0000", 2024-09-14T09:08:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.337e-06 2024-09-14T09:08:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104431+0000", 2024-09-14T09:08:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086728780000000002 2024-09-14T09:08:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:20.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104434+0000", 2024-09-14T09:08:20.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5049999999999999e-06 2024-09-14T09:08:20.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870945+0000", 2024-09-14T09:08:20.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651031800000002 2024-09-14T09:08:20.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870956+0000", 2024-09-14T09:08:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1002000000000001e-05 2024-09-14T09:08:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105750 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.549699042999997, 2024-09-14T09:08:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77519961800000003, 2024-09-14T09:08:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.714 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105750 2024-09-14T09:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:20.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3600000000000002e-07 2024-09-14T09:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095809+0000", 2024-09-14T09:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4000000000000002e-06 2024-09-14T09:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095822+0000", 2024-09-14T09:08:20.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2704000000000001e-05 2024-09-14T09:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095831+0000", 2024-09-14T09:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9530000000000005e-06 2024-09-14T09:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104477+0000", 2024-09-14T09:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086459989999999997 2024-09-14T09:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104488+0000", 2024-09-14T09:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:08:20.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870992+0000", 2024-09-14T09:08:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.766504097 2024-09-14T09:08:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871003+0000", 2024-09-14T09:08:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1358999999999999e-05 2024-09-14T09:08:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.718 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105751 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.549641158, 2024-09-14T09:08:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77518966700000003, 2024-09-14T09:08:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.719 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105751 2024-09-14T09:08:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7300000000000005e-07 2024-09-14T09:08:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095867+0000", 2024-09-14T09:08:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2729999999999996e-06 2024-09-14T09:08:20.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095879+0000", 2024-09-14T09:08:20.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2917e-05 2024-09-14T09:08:20.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095888+0000", 2024-09-14T09:08:20.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2609999999999993e-06 2024-09-14T09:08:20.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104520+0000", 2024-09-14T09:08:20.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086322970000000006 2024-09-14T09:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104524+0000", 2024-09-14T09:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.54e-06 2024-09-14T09:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871040+0000", 2024-09-14T09:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651615399999995 2024-09-14T09:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871051+0000", 2024-09-14T09:08:20.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1452e-05 2024-09-14T09:08:20.724 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.724 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.724 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.724 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105752 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:20.724 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:08:20.724 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.549580394000003, 2024-09-14T09:08:20.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517684899999995, 2024-09-14T09:08:20.724 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.724 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.724 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:20.725 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.725 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.725 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.725 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105752 2024-09-14T09:08:20.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.725 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:08:20.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:08:20.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095923+0000", 2024-09-14T09:08:20.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3300000000000001e-07 2024-09-14T09:08:20.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095927+0000", 2024-09-14T09:08:20.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:08:20.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095940+0000", 2024-09-14T09:08:20.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2309e-05 2024-09-14T09:08:20.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095948+0000", 2024-09-14T09:08:20.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9199999999999993e-06 2024-09-14T09:08:20.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104566+0000", 2024-09-14T09:08:20.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086171449999999997 2024-09-14T09:08:20.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:20.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104574+0000", 2024-09-14T09:08:20.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0690000000000002e-06 2024-09-14T09:08:20.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871087+0000", 2024-09-14T09:08:20.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651370699999999 2024-09-14T09:08:20.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871099+0000", 2024-09-14T09:08:20.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1718e-05 2024-09-14T09:08:20.729 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.729 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.729 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.729 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105755 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:20.729 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:08:20.729 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.549522451000001, 2024-09-14T09:08:20.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77516787899999995, 2024-09-14T09:08:20.729 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.729 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.729 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:20.729 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.730 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.730 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.730 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105755 2024-09-14T09:08:20.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.730 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:08:20.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:08:20.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095981+0000", 2024-09-14T09:08:20.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2699999999999998e-07 2024-09-14T09:08:20.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095985+0000", 2024-09-14T09:08:20.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4349999999999999e-06 2024-09-14T09:08:20.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095999+0000", 2024-09-14T09:08:20.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3139000000000001e-05 2024-09-14T09:08:20.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096007+0000", 2024-09-14T09:08:20.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6610000000000006e-06 2024-09-14T09:08:20.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104591+0000", 2024-09-14T09:08:20.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0085841709999999998 2024-09-14T09:08:20.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:20.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104595+0000", 2024-09-14T09:08:20.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5590000000000001e-06 2024-09-14T09:08:20.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871137+0000", 2024-09-14T09:08:20.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76654202100000002 2024-09-14T09:08:20.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871148+0000", 2024-09-14T09:08:20.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1066e-05 2024-09-14T09:08:20.733 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.733 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.733 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.734 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105757 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:20.734 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:08:20.734 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.549463871, 2024-09-14T09:08:20.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77516401300000004, 2024-09-14T09:08:20.734 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:20.734 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:20.734 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:20.734 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:20.734 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:20.734 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:20.734 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105757 2024-09-14T09:08:20.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.734 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:20.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:20.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:08:20.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:20.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:08:20.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:20.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:20.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856228+0000", 2024-09-14T09:08:20.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76018875100000005 2024-09-14T09:08:20.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:20.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856240+0000", 2024-09-14T09:08:20.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-05 2024-09-14T09:08:20.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:20.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856268+0000", 2024-09-14T09:08:20.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7628999999999999e-05 2024-09-14T09:08:20.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:20.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856307+0000", 2024-09-14T09:08:20.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8967000000000001e-05 2024-09-14T09:08:20.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:20.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871192+0000", 2024-09-14T09:08:20.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.01488449 2024-09-14T09:08:20.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:20.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:20.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:20.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871203+0000", 2024-09-14T09:08:20.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1266e-05 2024-09-14T09:08:20.737 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.737 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.737 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.737 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:20.737 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:20.737 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:20.739 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:20.742+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:20.739 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:08:20.839 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:20.839+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:20.871 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:20.871 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:20.871 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:20.871 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:20.885 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:08:20.973 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:08:20.974 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:20.975+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:21.041 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:21.039+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:21.057 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:21.058 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:21.058 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:08:21.058 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:08:21.058 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:21.073 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:08:21.119 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:08:21.120 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:08:21.136 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:08:21.136 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:08:21.142 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:21.146+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:21.153 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:08:21.156 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:08:21.170 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:08:21.170 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:08:21.170 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:08:21.191 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:08:21.192 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:08:21.212 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:08:21.212 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:08:21.231 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:21.232 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:21.234 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:08:21.234 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:08:21.237 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:08:21.237 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T09:08:21.247 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:08:21.247 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:21.247+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:21.344 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:21.343+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:21.360 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T09:08:21.421 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:21.421 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:21.438 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:08:21.482 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:21.483+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:21.546 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:21.550+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:21.582 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:21.582 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:21.597 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:08:21.682 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:21.683+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:21.748 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:21.747+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:21.759 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:21.759 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:21.773 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:08:21.849 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:21.850+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:21.943 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:21.943 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:21.950 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:21.951+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:21.959 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:08:22.024 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-14T09:08:22.024 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:08:22.024 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T09:08:22.024 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.000000 2024-09-14T09:08:22.024 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0 2024-09-14T09:08:22.090 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:22.091+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:22.127 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:22.127 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:22.143 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:08:22.190 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:22.191+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:22.252 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:22.251+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:22.304 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:22.304 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:22.318 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:08:22.357 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is unset 2024-09-14T09:08:22.357 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:22.355+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:22.453 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:22.455+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:22.489 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:22.490 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:22.504 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:08:22.590 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:22.591+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:22.655 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:22.658+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:22.663 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:22.663 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:22.677 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:08:22.766 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:22.767+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:22.841 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:22.841 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:22.855 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:08:22.898 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:22.899+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:22.958 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:22.959+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:22.987 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:22.987 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:23.002 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:08:23.059 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:23.062+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:23.119 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:23.120 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:23.134 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:08:23.177 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:23.182+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:23.239 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:08:23.286 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:23.287+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:23.302 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:23.303 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:23.317 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:08:23.361 INFO:teuthology.orchestra.run.smithi176.stderr:set osd.1 primary-affinity to 0 (802) 2024-09-14T09:08:23.361 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:23.359+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:23.418 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:08:23.419 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:08:23.437 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:08:23.437 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:08:23.448 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:23.448 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:23.453 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:08:23.453 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:08:23.461 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:23.466+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:23.463 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:08:23.470 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:08:23.470 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:08:23.470 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:08:23.489 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:08:23.542 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:08:23.543 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:08:23.543 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:08:23.543 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:08:23.544 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:08:23.544 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:08:23.544 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T09:08:23.596 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:23.591+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:23.664 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:23.666+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:23.672 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:23.672 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:23.672 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:08:23.673 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:08:23.673 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:23.686 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:08:23.764 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:23.763+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:23.858 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:23.858 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:08:23.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:08:23.858 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:08:23.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.859 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65139 3.a 3:54c90b7d:::rbd_data.115e21c31fcf.0000000000000008:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:08:23.859 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:08:23.859 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 122.48219773, 2024-09-14T09:08:23.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14429895700000001, 2024-09-14T09:08:23.859 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.859 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.859 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:23.859 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.859 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.859 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.859 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65139 2024-09-14T09:08:23.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.860 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:08:23.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:08:23.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379715+0000", 2024-09-14T09:08:23.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.576e-06 2024-09-14T09:08:23.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379724+0000", 2024-09-14T09:08:23.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1779999999999992e-06 2024-09-14T09:08:23.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379746+0000", 2024-09-14T09:08:23.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1661000000000001e-05 2024-09-14T09:08:23.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379759+0000", 2024-09-14T09:08:23.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.324e-05 2024-09-14T09:08:23.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379882+0000", 2024-09-14T09:08:23.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012239400000000001 2024-09-14T09:08:23.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.380023+0000", 2024-09-14T09:08:23.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000141402 2024-09-14T09:08:23.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:08:23.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.391553+0000", 2024-09-14T09:08:23.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011530197000000001 2024-09-14T09:08:23.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:23.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523927+0000", 2024-09-14T09:08:23.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.132373935 2024-09-14T09:08:23.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:08:23.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523941+0000", 2024-09-14T09:08:23.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3326000000000001e-05 2024-09-14T09:08:23.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:08:23.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523977+0000", 2024-09-14T09:08:23.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6185e-05 2024-09-14T09:08:23.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.524012+0000", 2024-09-14T09:08:23.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4863000000000001e-05 2024-09-14T09:08:23.865 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.865 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.865 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.865 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107606 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 14336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:23.865 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:23.865 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.455740829, 2024-09-14T09:08:23.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14293513799999999, 2024-09-14T09:08:23.865 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.866 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.866 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:23.866 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.866 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.866 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.866 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107606 2024-09-14T09:08:23.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.866 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:23.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:23.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:23.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1600000000000001e-07 2024-09-14T09:08:23.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406174+0000", 2024-09-14T09:08:23.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4199999999999999e-06 2024-09-14T09:08:23.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406188+0000", 2024-09-14T09:08:23.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4114000000000001e-05 2024-09-14T09:08:23.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406198+0000", 2024-09-14T09:08:23.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.041e-05 2024-09-14T09:08:23.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.548700+0000", 2024-09-14T09:08:23.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14250141299999999 2024-09-14T09:08:23.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.548783+0000", 2024-09-14T09:08:23.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3578999999999994e-05 2024-09-14T09:08:23.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549105+0000", 2024-09-14T09:08:23.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032158600000000001 2024-09-14T09:08:23.869 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.869 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.869 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.870 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107608 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:23.870 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:23.870 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.455674793, 2024-09-14T09:08:23.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14603042799999999, 2024-09-14T09:08:23.870 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.870 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.870 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:23.870 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.870 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.870 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.870 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107608 2024-09-14T09:08:23.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.871 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:23.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:23.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:23.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1200000000000002e-07 2024-09-14T09:08:23.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406241+0000", 2024-09-14T09:08:23.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7539999999999997e-06 2024-09-14T09:08:23.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406275+0000", 2024-09-14T09:08:23.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3512e-05 2024-09-14T09:08:23.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406322+0000", 2024-09-14T09:08:23.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6764999999999997e-05 2024-09-14T09:08:23.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549149+0000", 2024-09-14T09:08:23.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14282725499999999 2024-09-14T09:08:23.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549198+0000", 2024-09-14T09:08:23.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9599999999999999e-05 2024-09-14T09:08:23.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552266+0000", 2024-09-14T09:08:23.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030677299999999998 2024-09-14T09:08:23.874 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.874 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.874 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.874 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107610 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 22528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:23.874 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:23.874 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.455552918000002, 2024-09-14T09:08:23.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146437919, 2024-09-14T09:08:23.874 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.875 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.875 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:23.875 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.875 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.875 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.875 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107610 2024-09-14T09:08:23.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.875 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:23.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:23.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:23.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.02e-07 2024-09-14T09:08:23.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406363+0000", 2024-09-14T09:08:23.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8099999999999997e-06 2024-09-14T09:08:23.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406390+0000", 2024-09-14T09:08:23.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7004000000000001e-05 2024-09-14T09:08:23.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406404+0000", 2024-09-14T09:08:23.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3682e-05 2024-09-14T09:08:23.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552335+0000", 2024-09-14T09:08:23.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14593077800000001 2024-09-14T09:08:23.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552431+0000", 2024-09-14T09:08:23.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6086000000000001e-05 2024-09-14T09:08:23.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552795+0000", 2024-09-14T09:08:23.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000364657 2024-09-14T09:08:23.879 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.879 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.879 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.879 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107612 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 26624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:23.879 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:08:23.879 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.455487654999999, 2024-09-14T09:08:23.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146762061, 2024-09-14T09:08:23.879 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.880 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.880 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:23.880 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.880 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.880 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.880 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107612 2024-09-14T09:08:23.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.880 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:08:23.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:08:23.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406424+0000", 2024-09-14T09:08:23.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2600000000000002e-07 2024-09-14T09:08:23.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406451+0000", 2024-09-14T09:08:23.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7250000000000002e-05 2024-09-14T09:08:23.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406487+0000", 2024-09-14T09:08:23.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6086e-05 2024-09-14T09:08:23.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406498+0000", 2024-09-14T09:08:23.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0858e-05 2024-09-14T09:08:23.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552821+0000", 2024-09-14T09:08:23.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14632319099999999 2024-09-14T09:08:23.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552875+0000", 2024-09-14T09:08:23.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4262999999999999e-05 2024-09-14T09:08:23.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553185+0000", 2024-09-14T09:08:23.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000309687 2024-09-14T09:08:23.884 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.884 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.884 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.884 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107614 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:23.884 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:08:23.884 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.455400120999997, 2024-09-14T09:08:23.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14737121, 2024-09-14T09:08:23.884 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.884 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.884 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:23.884 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.885 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.885 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.885 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107614 2024-09-14T09:08:23.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.885 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:08:23.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:08:23.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406511+0000", 2024-09-14T09:08:23.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1799999999999995e-07 2024-09-14T09:08:23.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406514+0000", 2024-09-14T09:08:23.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8660000000000002e-06 2024-09-14T09:08:23.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406523+0000", 2024-09-14T09:08:23.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1940000000000004e-06 2024-09-14T09:08:23.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406531+0000", 2024-09-14T09:08:23.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3159999999999997e-06 2024-09-14T09:08:23.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553233+0000", 2024-09-14T09:08:23.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14670117899999999 2024-09-14T09:08:23.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553334+0000", 2024-09-14T09:08:23.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100982 2024-09-14T09:08:23.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553882+0000", 2024-09-14T09:08:23.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00054805499999999998 2024-09-14T09:08:23.888 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.888 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.888 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.888 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107616 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 34816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:23.888 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:08:23.889 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.455368485999998, 2024-09-14T09:08:23.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14780390500000001, 2024-09-14T09:08:23.889 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.889 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.889 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:23.889 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.889 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.889 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.889 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107616 2024-09-14T09:08:23.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.889 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:08:23.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:08:23.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406543+0000", 2024-09-14T09:08:23.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0200000000000001e-07 2024-09-14T09:08:23.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406545+0000", 2024-09-14T09:08:23.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3360000000000002e-06 2024-09-14T09:08:23.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406552+0000", 2024-09-14T09:08:23.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9779999999999999e-06 2024-09-14T09:08:23.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406559+0000", 2024-09-14T09:08:23.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6919999999999996e-06 2024-09-14T09:08:23.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553917+0000", 2024-09-14T09:08:23.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14735791600000001 2024-09-14T09:08:23.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554033+0000", 2024-09-14T09:08:23.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116047 2024-09-14T09:08:23.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554346+0000", 2024-09-14T09:08:23.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031323400000000002 2024-09-14T09:08:23.893 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.893 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.893 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.893 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107618 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 38912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:23.893 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:23.894 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.455339686000002, 2024-09-14T09:08:23.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14816009599999999, 2024-09-14T09:08:23.894 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.894 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.894 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:23.894 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.894 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.894 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.894 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107618 2024-09-14T09:08:23.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.894 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:23.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:23.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:23.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9999999999999998e-07 2024-09-14T09:08:23.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406574+0000", 2024-09-14T09:08:23.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2560000000000001e-06 2024-09-14T09:08:23.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406581+0000", 2024-09-14T09:08:23.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1539999999999996e-06 2024-09-14T09:08:23.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406586+0000", 2024-09-14T09:08:23.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.904e-06 2024-09-14T09:08:23.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554385+0000", 2024-09-14T09:08:23.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.147799443 2024-09-14T09:08:23.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554441+0000", 2024-09-14T09:08:23.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5921000000000002e-05 2024-09-14T09:08:23.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554731+0000", 2024-09-14T09:08:23.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028991800000000001 2024-09-14T09:08:23.898 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.898 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.898 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.898 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107620 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:23.898 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:23.898 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.455317903999997, 2024-09-14T09:08:23.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.148512643, 2024-09-14T09:08:23.898 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.898 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.898 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:23.899 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.899 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.899 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.899 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107620 2024-09-14T09:08:23.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.899 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:23.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:23.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:23.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9999999999999998e-07 2024-09-14T09:08:23.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406595+0000", 2024-09-14T09:08:23.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9360000000000002e-06 2024-09-14T09:08:23.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406605+0000", 2024-09-14T09:08:23.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0268000000000001e-05 2024-09-14T09:08:23.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406614+0000", 2024-09-14T09:08:23.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7180000000000002e-06 2024-09-14T09:08:23.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554750+0000", 2024-09-14T09:08:23.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14813599699999999 2024-09-14T09:08:23.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554794+0000", 2024-09-14T09:08:23.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.401e-05 2024-09-14T09:08:23.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555105+0000", 2024-09-14T09:08:23.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031131399999999998 2024-09-14T09:08:23.902 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.902 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.902 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.902 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107622 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:23.903 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:23.903 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.455286647999998, 2024-09-14T09:08:23.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14902820899999999, 2024-09-14T09:08:23.903 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.903 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.903 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:23.903 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.903 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.903 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.903 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107622 2024-09-14T09:08:23.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.903 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:23.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:23.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:23.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9200000000000001e-07 2024-09-14T09:08:23.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406636+0000", 2024-09-14T09:08:23.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.189e-05 2024-09-14T09:08:23.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406649+0000", 2024-09-14T09:08:23.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2442e-05 2024-09-14T09:08:23.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406658+0000", 2024-09-14T09:08:23.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3239999999999992e-06 2024-09-14T09:08:23.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555287+0000", 2024-09-14T09:08:23.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14862868400000001 2024-09-14T09:08:23.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555348+0000", 2024-09-14T09:08:23.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1686999999999994e-05 2024-09-14T09:08:23.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555652+0000", 2024-09-14T09:08:23.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00030369000000000001 2024-09-14T09:08:23.907 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.907 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.907 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.908 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107624 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:23.908 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:23.908 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.455242955000003, 2024-09-14T09:08:23.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14934019900000001, 2024-09-14T09:08:23.908 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.908 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.908 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:23.908 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.908 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.908 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.908 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107624 2024-09-14T09:08:23.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.909 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:23.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:23.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:23.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1399999999999997e-07 2024-09-14T09:08:23.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406670+0000", 2024-09-14T09:08:23.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4099999999999998e-06 2024-09-14T09:08:23.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406680+0000", 2024-09-14T09:08:23.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6460000000000005e-06 2024-09-14T09:08:23.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406688+0000", 2024-09-14T09:08:23.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5859999999999999e-06 2024-09-14T09:08:23.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555693+0000", 2024-09-14T09:08:23.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14900544499999999 2024-09-14T09:08:23.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555743+0000", 2024-09-14T09:08:23.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9966999999999998e-05 2024-09-14T09:08:23.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556008+0000", 2024-09-14T09:08:23.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026463099999999998 2024-09-14T09:08:23.912 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.912 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.912 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.912 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107626 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:23.912 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:08:23.912 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.455210981, 2024-09-14T09:08:23.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14983149500000001, 2024-09-14T09:08:23.913 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.913 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.913 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:23.913 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.913 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.913 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.913 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107626 2024-09-14T09:08:23.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.913 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:08:23.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:08:23.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408087+0000", 2024-09-14T09:08:23.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001387735 2024-09-14T09:08:23.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408094+0000", 2024-09-14T09:08:23.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9399999999999996e-06 2024-09-14T09:08:23.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408122+0000", 2024-09-14T09:08:23.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8076e-05 2024-09-14T09:08:23.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408147+0000", 2024-09-14T09:08:23.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4726e-05 2024-09-14T09:08:23.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556048+0000", 2024-09-14T09:08:23.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14790144299999999 2024-09-14T09:08:23.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556175+0000", 2024-09-14T09:08:23.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012663 2024-09-14T09:08:23.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556531+0000", 2024-09-14T09:08:23.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035594499999999998 2024-09-14T09:08:23.916 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.917 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.917 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.917 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107628 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:23.917 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:08:23.917 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.453741370000003, 2024-09-14T09:08:23.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14882648900000001, 2024-09-14T09:08:23.918 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.918 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.918 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:23.918 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.918 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.918 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.918 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107628 2024-09-14T09:08:23.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.918 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:08:23.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:08:23.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408878+0000", 2024-09-14T09:08:23.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070858800000000001 2024-09-14T09:08:23.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408885+0000", 2024-09-14T09:08:23.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4839999999999998e-06 2024-09-14T09:08:23.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408900+0000", 2024-09-14T09:08:23.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5174000000000001e-05 2024-09-14T09:08:23.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408918+0000", 2024-09-14T09:08:23.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.808e-05 2024-09-14T09:08:23.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556572+0000", 2024-09-14T09:08:23.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14765399500000001 2024-09-14T09:08:23.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556651+0000", 2024-09-14T09:08:23.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8919999999999997e-05 2024-09-14T09:08:23.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556996+0000", 2024-09-14T09:08:23.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034424800000000001 2024-09-14T09:08:23.922 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.922 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.922 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.922 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107630 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 63488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:23.923 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:08:23.923 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.452972535999997, 2024-09-14T09:08:23.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14855159200000001, 2024-09-14T09:08:23.923 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.923 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.923 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:23.923 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.923 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.923 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.923 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107630 2024-09-14T09:08:23.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.924 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:08:23.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:08:23.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409510+0000", 2024-09-14T09:08:23.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00057189700000000003 2024-09-14T09:08:23.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409518+0000", 2024-09-14T09:08:23.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7619999999999995e-06 2024-09-14T09:08:23.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409534+0000", 2024-09-14T09:08:23.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.594e-05 2024-09-14T09:08:23.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409546+0000", 2024-09-14T09:08:23.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2604e-05 2024-09-14T09:08:23.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557048+0000", 2024-09-14T09:08:23.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14750137899999999 2024-09-14T09:08:23.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557117+0000", 2024-09-14T09:08:23.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8998999999999994e-05 2024-09-14T09:08:23.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557490+0000", 2024-09-14T09:08:23.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037301100000000002 2024-09-14T09:08:23.927 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107632 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:08:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.452341591, 2024-09-14T09:08:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14833944199999999, 2024-09-14T09:08:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107632 2024-09-14T09:08:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:08:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:08:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412609+0000", 2024-09-14T09:08:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003040105 2024-09-14T09:08:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412618+0000", 2024-09-14T09:08:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2960000000000004e-06 2024-09-14T09:08:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412640+0000", 2024-09-14T09:08:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1773999999999999e-05 2024-09-14T09:08:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412655+0000", 2024-09-14T09:08:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4538e-05 2024-09-14T09:08:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557538+0000", 2024-09-14T09:08:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14488311800000001 2024-09-14T09:08:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557594+0000", 2024-09-14T09:08:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6122999999999998e-05 2024-09-14T09:08:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557908+0000", 2024-09-14T09:08:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031448799999999997 2024-09-14T09:08:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107634 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:08:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.449223944000003, 2024-09-14T09:08:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.145666186, 2024-09-14T09:08:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107634 2024-09-14T09:08:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:08:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:08:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412688+0000", 2024-09-14T09:08:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9000000000000005e-07 2024-09-14T09:08:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412707+0000", 2024-09-14T09:08:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9232000000000002e-05 2024-09-14T09:08:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412722+0000", 2024-09-14T09:08:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5722e-05 2024-09-14T09:08:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412733+0000", 2024-09-14T09:08:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0909e-05 2024-09-14T09:08:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557946+0000", 2024-09-14T09:08:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14521263100000001 2024-09-14T09:08:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558020+0000", 2024-09-14T09:08:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4294000000000004e-05 2024-09-14T09:08:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558353+0000", 2024-09-14T09:08:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033240800000000002 2024-09-14T09:08:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107636 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:08:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.449159641000001, 2024-09-14T09:08:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14599377399999999, 2024-09-14T09:08:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107636 2024-09-14T09:08:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:08:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:08:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412752+0000", 2024-09-14T09:08:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.156e-06 2024-09-14T09:08:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412770+0000", 2024-09-14T09:08:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7521999999999999e-05 2024-09-14T09:08:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412790+0000", 2024-09-14T09:08:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0942e-05 2024-09-14T09:08:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412809+0000", 2024-09-14T09:08:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8887999999999999e-05 2024-09-14T09:08:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558387+0000", 2024-09-14T09:08:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14557798799999999 2024-09-14T09:08:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558448+0000", 2024-09-14T09:08:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0862000000000001e-05 2024-09-14T09:08:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558745+0000", 2024-09-14T09:08:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029641599999999998 2024-09-14T09:08:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107638 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:08:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.449072385000001, 2024-09-14T09:08:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146344799, 2024-09-14T09:08:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107638 2024-09-14T09:08:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:08:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:08:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412839+0000", 2024-09-14T09:08:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0720000000000001e-06 2024-09-14T09:08:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412844+0000", 2024-09-14T09:08:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7759999999999997e-06 2024-09-14T09:08:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412857+0000", 2024-09-14T09:08:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3138e-05 2024-09-14T09:08:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412867+0000", 2024-09-14T09:08:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7229999999999999e-06 2024-09-14T09:08:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558785+0000", 2024-09-14T09:08:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14591828700000001 2024-09-14T09:08:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558833+0000", 2024-09-14T09:08:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7790000000000002e-05 2024-09-14T09:08:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559183+0000", 2024-09-14T09:08:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035001299999999998 2024-09-14T09:08:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107640 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:08:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.449027233999999, 2024-09-14T09:08:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146686804, 2024-09-14T09:08:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107640 2024-09-14T09:08:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:08:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:08:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420446+0000", 2024-09-14T09:08:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075632529999999998 2024-09-14T09:08:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420456+0000", 2024-09-14T09:08:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0850000000000003e-06 2024-09-14T09:08:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420475+0000", 2024-09-14T09:08:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9406e-05 2024-09-14T09:08:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420488+0000", 2024-09-14T09:08:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3278000000000001e-05 2024-09-14T09:08:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559205+0000", 2024-09-14T09:08:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13871644999999999 2024-09-14T09:08:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559252+0000", 2024-09-14T09:08:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7666000000000001e-05 2024-09-14T09:08:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559570+0000", 2024-09-14T09:08:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031766599999999998 2024-09-14T09:08:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107660 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 124928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:08:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.440956991999997, 2024-09-14T09:08:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.142954628, 2024-09-14T09:08:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107660 2024-09-14T09:08:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:08:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:08:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428190+0000", 2024-09-14T09:08:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072360339999999997 2024-09-14T09:08:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428398+0000", 2024-09-14T09:08:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020898499999999999 2024-09-14T09:08:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428458+0000", 2024-09-14T09:08:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9985999999999998e-05 2024-09-14T09:08:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428480+0000", 2024-09-14T09:08:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1284000000000002e-05 2024-09-14T09:08:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563583+0000", 2024-09-14T09:08:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13510361800000001 2024-09-14T09:08:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563644+0000", 2024-09-14T09:08:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1092000000000004e-05 2024-09-14T09:08:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563908+0000", 2024-09-14T09:08:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026362899999999998 2024-09-14T09:08:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:23.957 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:23.958 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:08:23.959 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:23.962+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:24.059 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:24.062+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:24.131 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:24.131 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:24.131 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:08:24.132 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:24.146 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:08:24.189 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:24.194+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:24.261 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:24.263+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:24.314 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:24.314 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:24.314 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:08:24.314 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:08:24.314 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:24.329 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:08:24.362 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:24.363+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:24.383 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T09:08:24.458 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:24.458 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:08:24.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:08:24.458 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:08:24.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.459 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38593 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [watch reconnect cookie 139718844451680 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:08:24.459 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:08:24.459 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 142.64461274799999, 2024-09-14T09:08:24.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0218296630000001, 2024-09-14T09:08:24.459 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.459 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.459 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:24.459 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.459 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.459 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.459 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38593 2024-09-14T09:08:24.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.459 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:08:24.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:08:24.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818420+0000", 2024-09-14T09:08:24.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8399999999999999e-06 2024-09-14T09:08:24.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818431+0000", 2024-09-14T09:08:24.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0859999999999999e-05 2024-09-14T09:08:24.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818466+0000", 2024-09-14T09:08:24.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4486999999999997e-05 2024-09-14T09:08:24.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818481+0000", 2024-09-14T09:08:24.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5362999999999998e-05 2024-09-14T09:08:24.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818659+0000", 2024-09-14T09:08:24.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017807500000000001 2024-09-14T09:08:24.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op must wait for map", 2024-09-14T09:08:24.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818681+0000", 2024-09-14T09:08:24.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2096999999999999e-05 2024-09-14T09:08:24.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825472+0000", 2024-09-14T09:08:24.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067905170000000003 2024-09-14T09:08:24.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:24.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825491+0000", 2024-09-14T09:08:24.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9372999999999999e-05 2024-09-14T09:08:24.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835402+0000", 2024-09-14T09:08:24.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0099111620000001 2024-09-14T09:08:24.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:24.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835548+0000", 2024-09-14T09:08:24.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014607300000000001 2024-09-14T09:08:24.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.840247+0000", 2024-09-14T09:08:24.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046988159999999998 2024-09-14T09:08:24.465 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.465 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.465 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.465 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38615 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:08:24.465 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:08:24.466 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 142.59624754000001, 2024-09-14T09:08:24.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.97014128899999996, 2024-09-14T09:08:24.466 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.466 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.466 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:24.466 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.466 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.466 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.466 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38615 2024-09-14T09:08:24.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.466 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:08:24.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:08:24.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866787+0000", 2024-09-14T09:08:24.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.476e-06 2024-09-14T09:08:24.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866806+0000", 2024-09-14T09:08:24.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8606000000000001e-05 2024-09-14T09:08:24.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866839+0000", 2024-09-14T09:08:24.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3510000000000003e-05 2024-09-14T09:08:24.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866867+0000", 2024-09-14T09:08:24.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7610000000000002e-05 2024-09-14T09:08:24.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867031+0000", 2024-09-14T09:08:24.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016414199999999999 2024-09-14T09:08:24.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:24.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867054+0000", 2024-09-14T09:08:24.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2693000000000001e-05 2024-09-14T09:08:24.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836462+0000", 2024-09-14T09:08:24.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.96940797700000003 2024-09-14T09:08:24.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:24.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836547+0000", 2024-09-14T09:08:24.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5294e-05 2024-09-14T09:08:24.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836924+0000", 2024-09-14T09:08:24.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037698099999999997 2024-09-14T09:08:24.470 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.471 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.471 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.471 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105723 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:24.471 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:24.471 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.367813495999997, 2024-09-14T09:08:24.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77512799700000001, 2024-09-14T09:08:24.471 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.471 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.471 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:24.471 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.471 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.472 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.472 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105723 2024-09-14T09:08:24.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.472 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:24.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:24.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:24.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1399999999999997e-07 2024-09-14T09:08:24.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095220+0000", 2024-09-14T09:08:24.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0510000000000001e-06 2024-09-14T09:08:24.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095228+0000", 2024-09-14T09:08:24.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4159999999999998e-06 2024-09-14T09:08:24.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095234+0000", 2024-09-14T09:08:24.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3010000000000003e-06 2024-09-14T09:08:24.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103902+0000", 2024-09-14T09:08:24.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086678299999999996 2024-09-14T09:08:24.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:24.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103905+0000", 2024-09-14T09:08:24.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5020000000000001e-06 2024-09-14T09:08:24.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870332+0000", 2024-09-14T09:08:24.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76642700900000005 2024-09-14T09:08:24.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870345+0000", 2024-09-14T09:08:24.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2274e-05 2024-09-14T09:08:24.475 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.476 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.476 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.476 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105724 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:24.476 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:08:24.476 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.367788982999997, 2024-09-14T09:08:24.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77515698099999997, 2024-09-14T09:08:24.476 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.476 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.476 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:24.476 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.476 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.476 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.477 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105724 2024-09-14T09:08:24.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.477 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:08:24.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:08:24.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095242+0000", 2024-09-14T09:08:24.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9400000000000001e-07 2024-09-14T09:08:24.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095243+0000", 2024-09-14T09:08:24.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7540000000000001e-06 2024-09-14T09:08:24.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095248+0000", 2024-09-14T09:08:24.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1320000000000004e-06 2024-09-14T09:08:24.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095251+0000", 2024-09-14T09:08:24.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2770000000000001e-06 2024-09-14T09:08:24.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103947+0000", 2024-09-14T09:08:24.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086960089999999993 2024-09-14T09:08:24.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:24.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103955+0000", 2024-09-14T09:08:24.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9519999999999994e-06 2024-09-14T09:08:24.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870386+0000", 2024-09-14T09:08:24.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76643141100000001 2024-09-14T09:08:24.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870398+0000", 2024-09-14T09:08:24.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2052000000000001e-05 2024-09-14T09:08:24.480 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.480 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.480 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.481 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105727 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:24.481 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:24.481 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.367749277000001, 2024-09-14T09:08:24.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517292900000001, 2024-09-14T09:08:24.481 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.481 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.481 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:24.481 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.481 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.481 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.481 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105727 2024-09-14T09:08:24.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.482 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:24.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:24.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:24.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3300000000000002e-07 2024-09-14T09:08:24.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095284+0000", 2024-09-14T09:08:24.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9790000000000002e-06 2024-09-14T09:08:24.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095292+0000", 2024-09-14T09:08:24.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3810000000000001e-06 2024-09-14T09:08:24.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095298+0000", 2024-09-14T09:08:24.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9710000000000001e-06 2024-09-14T09:08:24.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103987+0000", 2024-09-14T09:08:24.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086893820000000007 2024-09-14T09:08:24.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:24.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103991+0000", 2024-09-14T09:08:24.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5480000000000002e-06 2024-09-14T09:08:24.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870442+0000", 2024-09-14T09:08:24.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.766451145 2024-09-14T09:08:24.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870454+0000", 2024-09-14T09:08:24.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.199e-05 2024-09-14T09:08:24.485 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.485 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.486 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.486 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105729 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:24.486 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:08:24.486 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.367700919999997, 2024-09-14T09:08:24.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517686500000005, 2024-09-14T09:08:24.486 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.486 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.486 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:24.486 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.486 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.486 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.486 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105729 2024-09-14T09:08:24.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.487 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:08:24.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:08:24.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095330+0000", 2024-09-14T09:08:24.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-09-14T09:08:24.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095332+0000", 2024-09-14T09:08:24.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1560000000000002e-06 2024-09-14T09:08:24.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095338+0000", 2024-09-14T09:08:24.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.592e-06 2024-09-14T09:08:24.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095341+0000", 2024-09-14T09:08:24.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8139999999999998e-06 2024-09-14T09:08:24.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104031+0000", 2024-09-14T09:08:24.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086896330000000004 2024-09-14T09:08:24.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:24.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104039+0000", 2024-09-14T09:08:24.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0900000000000005e-06 2024-09-14T09:08:24.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870495+0000", 2024-09-14T09:08:24.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76645611199999997 2024-09-14T09:08:24.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870506+0000", 2024-09-14T09:08:24.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0986e-05 2024-09-14T09:08:24.491 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.491 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.491 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.491 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105731 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:24.491 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:24.491 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.367682653000003, 2024-09-14T09:08:24.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77520598600000001, 2024-09-14T09:08:24.491 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.491 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.491 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:24.491 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.491 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.492 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.492 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105731 2024-09-14T09:08:24.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.492 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:24.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:24.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:24.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8599999999999999e-07 2024-09-14T09:08:24.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095350+0000", 2024-09-14T09:08:24.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1110000000000002e-06 2024-09-14T09:08:24.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095357+0000", 2024-09-14T09:08:24.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9020000000000002e-06 2024-09-14T09:08:24.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095360+0000", 2024-09-14T09:08:24.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1939999999999998e-06 2024-09-14T09:08:24.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104073+0000", 2024-09-14T09:08:24.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087131210000000008 2024-09-14T09:08:24.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:24.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104077+0000", 2024-09-14T09:08:24.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.501e-06 2024-09-14T09:08:24.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870541+0000", 2024-09-14T09:08:24.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76646461700000001 2024-09-14T09:08:24.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870554+0000", 2024-09-14T09:08:24.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2153999999999999e-05 2024-09-14T09:08:24.495 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.495 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.495 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.496 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105733 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:24.496 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:24.496 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.367662621999997, 2024-09-14T09:08:24.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.775237277, 2024-09-14T09:08:24.496 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.496 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.496 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:24.496 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.496 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.496 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.497 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105733 2024-09-14T09:08:24.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.497 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:24.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:24.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:24.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8699999999999995e-07 2024-09-14T09:08:24.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095371+0000", 2024-09-14T09:08:24.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5450000000000002e-06 2024-09-14T09:08:24.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095377+0000", 2024-09-14T09:08:24.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.776e-06 2024-09-14T09:08:24.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095383+0000", 2024-09-14T09:08:24.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3129999999999998e-06 2024-09-14T09:08:24.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104124+0000", 2024-09-14T09:08:24.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087409840000000003 2024-09-14T09:08:24.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:24.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104132+0000", 2024-09-14T09:08:24.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3159999999999997e-06 2024-09-14T09:08:24.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870593+0000", 2024-09-14T09:08:24.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76646064800000002 2024-09-14T09:08:24.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870605+0000", 2024-09-14T09:08:24.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2208000000000001e-05 2024-09-14T09:08:24.500 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.500 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.501 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.501 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105734 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:24.501 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:24.501 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.367639539999999, 2024-09-14T09:08:24.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77526569599999995, 2024-09-14T09:08:24.501 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.501 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.501 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:24.501 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.501 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.501 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.502 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105734 2024-09-14T09:08:24.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.502 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:24.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:24.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:24.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7399999999999998e-07 2024-09-14T09:08:24.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095394+0000", 2024-09-14T09:08:24.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8030000000000001e-06 2024-09-14T09:08:24.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095401+0000", 2024-09-14T09:08:24.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.046e-06 2024-09-14T09:08:24.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095407+0000", 2024-09-14T09:08:24.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6060000000000002e-06 2024-09-14T09:08:24.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104167+0000", 2024-09-14T09:08:24.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087602530000000008 2024-09-14T09:08:24.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:24.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104171+0000", 2024-09-14T09:08:24.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7620000000000001e-06 2024-09-14T09:08:24.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870645+0000", 2024-09-14T09:08:24.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76647400600000004 2024-09-14T09:08:24.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870656+0000", 2024-09-14T09:08:24.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1746000000000001e-05 2024-09-14T09:08:24.506 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.506 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.506 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.506 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105738 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:24.506 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:08:24.506 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.367599812999998, 2024-09-14T09:08:24.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77527530700000002, 2024-09-14T09:08:24.506 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.506 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.506 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:24.507 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.507 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.507 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.507 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105738 2024-09-14T09:08:24.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.507 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:08:24.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:08:24.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095431+0000", 2024-09-14T09:08:24.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0999999999999999e-07 2024-09-14T09:08:24.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095434+0000", 2024-09-14T09:08:24.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.757e-06 2024-09-14T09:08:24.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095442+0000", 2024-09-14T09:08:24.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9929999999999995e-06 2024-09-14T09:08:24.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095448+0000", 2024-09-14T09:08:24.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1340000000000001e-06 2024-09-14T09:08:24.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104213+0000", 2024-09-14T09:08:24.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087650000000000002 2024-09-14T09:08:24.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:24.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104221+0000", 2024-09-14T09:08:24.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4589999999999998e-06 2024-09-14T09:08:24.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870694+0000", 2024-09-14T09:08:24.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76647303200000005 2024-09-14T09:08:24.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870706+0000", 2024-09-14T09:08:24.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1522000000000001e-05 2024-09-14T09:08:24.511 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.512 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.512 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.512 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105739 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:24.512 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:24.512 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.367575283999997, 2024-09-14T09:08:24.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77530221700000002, 2024-09-14T09:08:24.512 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.512 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.512 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:24.512 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.513 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.513 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.513 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105739 2024-09-14T09:08:24.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.513 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:24.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:24.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:24.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.15e-07 2024-09-14T09:08:24.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095457+0000", 2024-09-14T09:08:24.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6470000000000001e-06 2024-09-14T09:08:24.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095461+0000", 2024-09-14T09:08:24.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0910000000000003e-06 2024-09-14T09:08:24.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095464+0000", 2024-09-14T09:08:24.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2059999999999999e-06 2024-09-14T09:08:24.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104253+0000", 2024-09-14T09:08:24.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087884120000000007 2024-09-14T09:08:24.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:24.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104256+0000", 2024-09-14T09:08:24.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5439999999999999e-06 2024-09-14T09:08:24.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870745+0000", 2024-09-14T09:08:24.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76648908000000004 2024-09-14T09:08:24.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870757+0000", 2024-09-14T09:08:24.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1822e-05 2024-09-14T09:08:24.516 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.516 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.517 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.517 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105741 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:24.517 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:08:24.517 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.367506157999998, 2024-09-14T09:08:24.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77528469899999997, 2024-09-14T09:08:24.517 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.517 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.517 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:24.517 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.517 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.517 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.517 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105741 2024-09-14T09:08:24.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.518 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:08:24.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:08:24.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095525+0000", 2024-09-14T09:08:24.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3740000000000001e-06 2024-09-14T09:08:24.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095533+0000", 2024-09-14T09:08:24.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6380000000000004e-06 2024-09-14T09:08:24.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095549+0000", 2024-09-14T09:08:24.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5554999999999999e-05 2024-09-14T09:08:24.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095566+0000", 2024-09-14T09:08:24.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7146000000000002e-05 2024-09-14T09:08:24.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104297+0000", 2024-09-14T09:08:24.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087315750000000001 2024-09-14T09:08:24.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:24.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104305+0000", 2024-09-14T09:08:24.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1389999999999995e-06 2024-09-14T09:08:24.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870797+0000", 2024-09-14T09:08:24.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76649129599999999 2024-09-14T09:08:24.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870809+0000", 2024-09-14T09:08:24.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1976e-05 2024-09-14T09:08:24.522 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.522 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.522 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.522 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105742 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:24.522 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:08:24.522 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.367441374000002, 2024-09-14T09:08:24.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.775271873, 2024-09-14T09:08:24.522 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.522 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.522 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:24.523 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.523 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.523 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.523 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105742 2024-09-14T09:08:24.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.523 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:08:24.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:08:24.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095590+0000", 2024-09-14T09:08:24.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0400000000000005e-07 2024-09-14T09:08:24.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095594+0000", 2024-09-14T09:08:24.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5639999999999998e-06 2024-09-14T09:08:24.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095620+0000", 2024-09-14T09:08:24.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5494999999999999e-05 2024-09-14T09:08:24.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095639+0000", 2024-09-14T09:08:24.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9639999999999998e-05 2024-09-14T09:08:24.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104340+0000", 2024-09-14T09:08:24.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087004950000000008 2024-09-14T09:08:24.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:24.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104343+0000", 2024-09-14T09:08:24.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4020000000000002e-06 2024-09-14T09:08:24.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870850+0000", 2024-09-14T09:08:24.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76650648499999996 2024-09-14T09:08:24.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870861+0000", 2024-09-14T09:08:24.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0888e-05 2024-09-14T09:08:24.527 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.527 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.527 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.527 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105745 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:24.527 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:08:24.527 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.367342759000003, 2024-09-14T09:08:24.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77521998700000005, 2024-09-14T09:08:24.527 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.527 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.528 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:24.528 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.528 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.528 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.528 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105745 2024-09-14T09:08:24.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.528 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:08:24.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:08:24.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095688+0000", 2024-09-14T09:08:24.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2699999999999998e-07 2024-09-14T09:08:24.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095693+0000", 2024-09-14T09:08:24.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0479999999999998e-06 2024-09-14T09:08:24.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095707+0000", 2024-09-14T09:08:24.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.342e-05 2024-09-14T09:08:24.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095716+0000", 2024-09-14T09:08:24.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9800000000000004e-06 2024-09-14T09:08:24.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104388+0000", 2024-09-14T09:08:24.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086717349999999999 2024-09-14T09:08:24.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:24.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104396+0000", 2024-09-14T09:08:24.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1769999999999998e-06 2024-09-14T09:08:24.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870897+0000", 2024-09-14T09:08:24.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76650102600000003 2024-09-14T09:08:24.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870907+0000", 2024-09-14T09:08:24.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0674e-05 2024-09-14T09:08:24.532 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.532 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.532 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.532 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105747 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:24.532 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:24.532 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.367296539000002, 2024-09-14T09:08:24.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77522190300000005, 2024-09-14T09:08:24.532 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.532 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.533 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:24.533 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.533 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.533 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.533 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105747 2024-09-14T09:08:24.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.533 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:24.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:24.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:24.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8999999999999995e-07 2024-09-14T09:08:24.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095738+0000", 2024-09-14T09:08:24.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8430000000000003e-06 2024-09-14T09:08:24.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095750+0000", 2024-09-14T09:08:24.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1229999999999999e-05 2024-09-14T09:08:24.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095758+0000", 2024-09-14T09:08:24.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.337e-06 2024-09-14T09:08:24.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104431+0000", 2024-09-14T09:08:24.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086728780000000002 2024-09-14T09:08:24.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:24.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104434+0000", 2024-09-14T09:08:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5049999999999999e-06 2024-09-14T09:08:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870945+0000", 2024-09-14T09:08:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651031800000002 2024-09-14T09:08:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870956+0000", 2024-09-14T09:08:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1002000000000001e-05 2024-09-14T09:08:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105750 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.367226592999998, 2024-09-14T09:08:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77519961800000003, 2024-09-14T09:08:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.539 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.539 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.539 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105750 2024-09-14T09:08:24.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.539 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:24.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:24.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:24.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3600000000000002e-07 2024-09-14T09:08:24.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095809+0000", 2024-09-14T09:08:24.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4000000000000002e-06 2024-09-14T09:08:24.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095822+0000", 2024-09-14T09:08:24.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2704000000000001e-05 2024-09-14T09:08:24.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095831+0000", 2024-09-14T09:08:24.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9530000000000005e-06 2024-09-14T09:08:24.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104477+0000", 2024-09-14T09:08:24.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086459989999999997 2024-09-14T09:08:24.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:24.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104488+0000", 2024-09-14T09:08:24.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:08:24.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870992+0000", 2024-09-14T09:08:24.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.766504097 2024-09-14T09:08:24.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871003+0000", 2024-09-14T09:08:24.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1358999999999999e-05 2024-09-14T09:08:24.543 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.543 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.543 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.544 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105751 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:24.544 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:24.544 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.367168708000001, 2024-09-14T09:08:24.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77518966700000003, 2024-09-14T09:08:24.544 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.544 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.544 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:24.544 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.544 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.544 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.544 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105751 2024-09-14T09:08:24.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.544 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:24.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:24.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:24.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7300000000000005e-07 2024-09-14T09:08:24.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095867+0000", 2024-09-14T09:08:24.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2729999999999996e-06 2024-09-14T09:08:24.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095879+0000", 2024-09-14T09:08:24.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2917e-05 2024-09-14T09:08:24.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095888+0000", 2024-09-14T09:08:24.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2609999999999993e-06 2024-09-14T09:08:24.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104520+0000", 2024-09-14T09:08:24.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086322970000000006 2024-09-14T09:08:24.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:24.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104524+0000", 2024-09-14T09:08:24.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.54e-06 2024-09-14T09:08:24.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871040+0000", 2024-09-14T09:08:24.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651615399999995 2024-09-14T09:08:24.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871051+0000", 2024-09-14T09:08:24.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1452e-05 2024-09-14T09:08:24.548 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.548 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.548 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.549 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105752 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:24.549 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:08:24.549 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.367107943999997, 2024-09-14T09:08:24.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517684899999995, 2024-09-14T09:08:24.549 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.549 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.549 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:24.549 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.549 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.549 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.549 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105752 2024-09-14T09:08:24.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.549 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:08:24.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:08:24.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095923+0000", 2024-09-14T09:08:24.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3300000000000001e-07 2024-09-14T09:08:24.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095927+0000", 2024-09-14T09:08:24.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:08:24.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095940+0000", 2024-09-14T09:08:24.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2309e-05 2024-09-14T09:08:24.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095948+0000", 2024-09-14T09:08:24.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9199999999999993e-06 2024-09-14T09:08:24.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104566+0000", 2024-09-14T09:08:24.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086171449999999997 2024-09-14T09:08:24.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:24.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104574+0000", 2024-09-14T09:08:24.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0690000000000002e-06 2024-09-14T09:08:24.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871087+0000", 2024-09-14T09:08:24.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651370699999999 2024-09-14T09:08:24.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871099+0000", 2024-09-14T09:08:24.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1718e-05 2024-09-14T09:08:24.553 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.553 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.553 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.554 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105755 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:24.554 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:08:24.554 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.367050001000003, 2024-09-14T09:08:24.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77516787899999995, 2024-09-14T09:08:24.554 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.554 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.554 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:24.554 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.554 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.554 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.554 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105755 2024-09-14T09:08:24.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.555 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:08:24.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:08:24.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095981+0000", 2024-09-14T09:08:24.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2699999999999998e-07 2024-09-14T09:08:24.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095985+0000", 2024-09-14T09:08:24.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4349999999999999e-06 2024-09-14T09:08:24.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095999+0000", 2024-09-14T09:08:24.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3139000000000001e-05 2024-09-14T09:08:24.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096007+0000", 2024-09-14T09:08:24.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6610000000000006e-06 2024-09-14T09:08:24.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104591+0000", 2024-09-14T09:08:24.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0085841709999999998 2024-09-14T09:08:24.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:24.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104595+0000", 2024-09-14T09:08:24.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5590000000000001e-06 2024-09-14T09:08:24.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871137+0000", 2024-09-14T09:08:24.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76654202100000002 2024-09-14T09:08:24.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871148+0000", 2024-09-14T09:08:24.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1066e-05 2024-09-14T09:08:24.558 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.558 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.558 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.558 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105757 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:24.559 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:08:24.559 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.366991421000002, 2024-09-14T09:08:24.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77516401300000004, 2024-09-14T09:08:24.559 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:24.559 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:24.559 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:24.559 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:24.559 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:24.559 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:24.559 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105757 2024-09-14T09:08:24.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.559 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:24.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:24.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:08:24.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:24.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:08:24.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:24.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:24.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856228+0000", 2024-09-14T09:08:24.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76018875100000005 2024-09-14T09:08:24.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:24.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856240+0000", 2024-09-14T09:08:24.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-05 2024-09-14T09:08:24.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:24.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856268+0000", 2024-09-14T09:08:24.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7628999999999999e-05 2024-09-14T09:08:24.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:24.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856307+0000", 2024-09-14T09:08:24.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8967000000000001e-05 2024-09-14T09:08:24.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:24.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871192+0000", 2024-09-14T09:08:24.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.01488449 2024-09-14T09:08:24.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:24.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:24.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:24.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871203+0000", 2024-09-14T09:08:24.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1266e-05 2024-09-14T09:08:24.563 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.563 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.563 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.563 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:24.563 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:24.563 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:24.565 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:08:24.565 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:24.563+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:24.666 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:24.667+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:24.684 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:24.685 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:24.685 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:24.685 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:24.699 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:08:24.786 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:24.787+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:24.859 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:24.859 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:24.859 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:08:24.859 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:08:24.860 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:24.867 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:24.870+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:24.873 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:08:24.968 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:24.970+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:25.045 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:25.045 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:08:25.045 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:08:25.045 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:08:25.046 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.046 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:71052 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:08:25.048 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:08:25.048 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.30622336499999, 2024-09-14T09:08:25.048 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14854635599999999, 2024-09-14T09:08:25.048 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.048 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.048 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:25.049 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.049 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.049 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.049 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 71052 2024-09-14T09:08:25.049 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.049 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.049 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.049 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.049 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:08:25.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.050 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:08:25.050 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.050 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.050 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.050 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737570+0000", 2024-09-14T09:08:25.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4770000000000002e-06 2024-09-14T09:08:25.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.051 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.051 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.051 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737584+0000", 2024-09-14T09:08:25.051 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3457e-05 2024-09-14T09:08:25.051 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737608+0000", 2024-09-14T09:08:25.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3893999999999999e-05 2024-09-14T09:08:25.052 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.052 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.052 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.052 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737620+0000", 2024-09-14T09:08:25.052 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2157000000000001e-05 2024-09-14T09:08:25.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.053 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737738+0000", 2024-09-14T09:08:25.053 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118409 2024-09-14T09:08:25.053 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.053 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.053 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737892+0000", 2024-09-14T09:08:25.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154222 2024-09-14T09:08:25.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.054 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.054 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:08:25.054 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.749622+0000", 2024-09-14T09:08:25.054 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011729537 2024-09-14T09:08:25.054 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:25.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885934+0000", 2024-09-14T09:08:25.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13631222200000001 2024-09-14T09:08:25.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:25.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885996+0000", 2024-09-14T09:08:25.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1555000000000003e-05 2024-09-14T09:08:25.055 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.055 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.055 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:25.055 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886060+0000", 2024-09-14T09:08:25.055 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3980999999999996e-05 2024-09-14T09:08:25.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.056 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.056 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886114+0000", 2024-09-14T09:08:25.056 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4444999999999998e-05 2024-09-14T09:08:25.056 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.056 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.056 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.056 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.056 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.056 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:71052 3.1 e71/34 ondisk, result = 0)", 2024-09-14T09:08:25.056 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:08:25.057 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.263319987, 2024-09-14T09:08:25.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10565334799999999, 2024-09-14T09:08:25.057 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.057 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.057 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.057 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:08:25.057 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.057 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.057 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.057 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.057 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:08:25.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780473+0000", 2024-09-14T09:08:25.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8980000000000001e-06 2024-09-14T09:08:25.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780482+0000", 2024-09-14T09:08:25.058 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9080000000000001e-06 2024-09-14T09:08:25.058 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.058 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.058 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.058 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780511+0000", 2024-09-14T09:08:25.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9490999999999999e-05 2024-09-14T09:08:25.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780527+0000", 2024-09-14T09:08:25.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.508e-05 2024-09-14T09:08:25.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.059 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.059 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885978+0000", 2024-09-14T09:08:25.059 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.105451328 2024-09-14T09:08:25.059 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.059 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885989+0000", 2024-09-14T09:08:25.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0875e-05 2024-09-14T09:08:25.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.060 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.060 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886125+0000", 2024-09-14T09:08:25.060 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013576800000000001 2024-09-14T09:08:25.060 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.060 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.060 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.060 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.060 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.060 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:98007 3.12 3:4faf1f2a:::rbd_data.115e113dc8ca.000000000000004f:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:08:25.060 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:08:25.060 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 72.186347763000001, 2024-09-14T09:08:25.061 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.098233707000000003, 2024-09-14T09:08:25.061 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.061 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.061 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:25.061 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.061 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.061 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.061 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 98007 2024-09-14T09:08:25.061 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.061 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.061 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.061 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.061 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:08:25.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:08:25.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857446+0000", 2024-09-14T09:08:25.062 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7599999999999998e-06 2024-09-14T09:08:25.062 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.062 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.062 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.062 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857459+0000", 2024-09-14T09:08:25.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3123e-05 2024-09-14T09:08:25.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857491+0000", 2024-09-14T09:08:25.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1603000000000002e-05 2024-09-14T09:08:25.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857506+0000", 2024-09-14T09:08:25.063 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.537e-05 2024-09-14T09:08:25.063 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.063 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.063 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.063 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857694+0000", 2024-09-14T09:08:25.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018766599999999999 2024-09-14T09:08:25.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857955+0000", 2024-09-14T09:08:25.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000260979 2024-09-14T09:08:25.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:08:25.064 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.863125+0000", 2024-09-14T09:08:25.064 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.005170582 2024-09-14T09:08:25.064 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.064 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.064 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:25.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955268+0000", 2024-09-14T09:08:25.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092142659000000002 2024-09-14T09:08:25.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:25.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955385+0000", 2024-09-14T09:08:25.065 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116723 2024-09-14T09:08:25.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.065 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.065 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:25.065 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955553+0000", 2024-09-14T09:08:25.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016809299999999999 2024-09-14T09:08:25.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.066 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.066 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955677+0000", 2024-09-14T09:08:25.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012414899999999999 2024-09-14T09:08:25.066 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.066 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.066 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.066 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.066 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118153 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 445440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:25.066 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:08:25.067 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.205790387999997, 2024-09-14T09:08:25.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095358268999999996, 2024-09-14T09:08:25.067 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.067 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.067 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.067 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.067 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.067 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.067 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118153 2024-09-14T09:08:25.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.067 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.067 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:08:25.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:08:25.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:08:25.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0999999999999999e-07 2024-09-14T09:08:25.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838003+0000", 2024-09-14T09:08:25.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2000000000000001e-06 2024-09-14T09:08:25.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838009+0000", 2024-09-14T09:08:25.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.105e-06 2024-09-14T09:08:25.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838015+0000", 2024-09-14T09:08:25.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9030000000000001e-06 2024-09-14T09:08:25.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933049+0000", 2024-09-14T09:08:25.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095033513999999999 2024-09-14T09:08:25.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933113+0000", 2024-09-14T09:08:25.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3616999999999998e-05 2024-09-14T09:08:25.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933359+0000", 2024-09-14T09:08:25.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024652000000000002 2024-09-14T09:08:25.071 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.071 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.071 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.071 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118155 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 449536~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:25.071 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:08:25.071 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.205767862999998, 2024-09-14T09:08:25.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095668000000000003, 2024-09-14T09:08:25.071 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.072 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.072 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.072 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.072 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.072 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.072 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118155 2024-09-14T09:08:25.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.072 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.072 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:08:25.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.072 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:08:25.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838024+0000", 2024-09-14T09:08:25.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0400000000000002e-07 2024-09-14T09:08:25.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838026+0000", 2024-09-14T09:08:25.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.508e-06 2024-09-14T09:08:25.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838033+0000", 2024-09-14T09:08:25.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.392e-06 2024-09-14T09:08:25.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838038+0000", 2024-09-14T09:08:25.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1050000000000003e-06 2024-09-14T09:08:25.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933400+0000", 2024-09-14T09:08:25.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095362279999999994 2024-09-14T09:08:25.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933451+0000", 2024-09-14T09:08:25.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0927000000000002e-05 2024-09-14T09:08:25.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933691+0000", 2024-09-14T09:08:25.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024038400000000001 2024-09-14T09:08:25.075 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.075 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.075 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.076 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118158 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 453632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:25.076 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:08:25.076 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.205746841, 2024-09-14T09:08:25.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096024231000000002, 2024-09-14T09:08:25.076 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.076 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.076 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.076 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.076 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.076 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.077 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118158 2024-09-14T09:08:25.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.077 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:08:25.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:08:25.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838045+0000", 2024-09-14T09:08:25.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2199999999999999e-07 2024-09-14T09:08:25.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838047+0000", 2024-09-14T09:08:25.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1059999999999998e-06 2024-09-14T09:08:25.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838054+0000", 2024-09-14T09:08:25.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4089999999999997e-06 2024-09-14T09:08:25.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838059+0000", 2024-09-14T09:08:25.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8590000000000004e-06 2024-09-14T09:08:25.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933729+0000", 2024-09-14T09:08:25.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095670274999999999 2024-09-14T09:08:25.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933794+0000", 2024-09-14T09:08:25.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4846999999999998e-05 2024-09-14T09:08:25.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934069+0000", 2024-09-14T09:08:25.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027431299999999999 2024-09-14T09:08:25.080 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.080 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.080 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.080 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118160 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 457728~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:25.080 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:08:25.081 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.205724721000003, 2024-09-14T09:08:25.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096394812999999996, 2024-09-14T09:08:25.081 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.081 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.081 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.081 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.081 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.081 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.081 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118160 2024-09-14T09:08:25.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.081 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:08:25.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:08:25.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838067+0000", 2024-09-14T09:08:25.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9399999999999995e-07 2024-09-14T09:08:25.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838069+0000", 2024-09-14T09:08:25.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1579999999999999e-06 2024-09-14T09:08:25.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838077+0000", 2024-09-14T09:08:25.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6939999999999995e-06 2024-09-14T09:08:25.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.083 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.083 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838082+0000", 2024-09-14T09:08:25.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2290000000000002e-06 2024-09-14T09:08:25.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934106+0000", 2024-09-14T09:08:25.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096023874999999995 2024-09-14T09:08:25.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934158+0000", 2024-09-14T09:08:25.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2052000000000002e-05 2024-09-14T09:08:25.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934461+0000", 2024-09-14T09:08:25.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030331100000000001 2024-09-14T09:08:25.085 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.085 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.085 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.085 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118162 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 461824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:25.085 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:08:25.085 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.205702141000003, 2024-09-14T09:08:25.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096615294000000004, 2024-09-14T09:08:25.085 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.085 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.085 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.085 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.085 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.086 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.086 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118162 2024-09-14T09:08:25.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.086 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:08:25.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:08:25.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838090+0000", 2024-09-14T09:08:25.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9699999999999996e-07 2024-09-14T09:08:25.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838092+0000", 2024-09-14T09:08:25.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1119999999999999e-06 2024-09-14T09:08:25.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838098+0000", 2024-09-14T09:08:25.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4359999999999999e-06 2024-09-14T09:08:25.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838103+0000", 2024-09-14T09:08:25.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1200000000000001e-06 2024-09-14T09:08:25.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934504+0000", 2024-09-14T09:08:25.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096400885000000006 2024-09-14T09:08:25.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934596+0000", 2024-09-14T09:08:25.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1775999999999997e-05 2024-09-14T09:08:25.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.089 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.089 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934704+0000", 2024-09-14T09:08:25.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010846800000000001 2024-09-14T09:08:25.089 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.089 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.089 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.089 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118164 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 465920~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:25.089 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:08:25.089 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.205680880999999, 2024-09-14T09:08:25.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096951372999999993, 2024-09-14T09:08:25.089 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.090 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.090 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.090 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.090 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.090 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.090 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118164 2024-09-14T09:08:25.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.090 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.090 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:08:25.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:08:25.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838111+0000", 2024-09-14T09:08:25.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0100000000000005e-07 2024-09-14T09:08:25.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838113+0000", 2024-09-14T09:08:25.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0339999999999999e-06 2024-09-14T09:08:25.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838119+0000", 2024-09-14T09:08:25.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.409e-06 2024-09-14T09:08:25.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838124+0000", 2024-09-14T09:08:25.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9030000000000003e-06 2024-09-14T09:08:25.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934739+0000", 2024-09-14T09:08:25.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096615124999999996 2024-09-14T09:08:25.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934789+0000", 2024-09-14T09:08:25.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9554e-05 2024-09-14T09:08:25.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935062+0000", 2024-09-14T09:08:25.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000272847 2024-09-14T09:08:25.093 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.093 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.093 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.094 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118166 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 470016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:25.094 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:08:25.094 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.205660127000002, 2024-09-14T09:08:25.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097306410999999995, 2024-09-14T09:08:25.094 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.094 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.094 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.094 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.094 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.094 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.094 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118166 2024-09-14T09:08:25.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.094 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:08:25.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:08:25.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:08:25.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.03e-07 2024-09-14T09:08:25.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838133+0000", 2024-09-14T09:08:25.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9479999999999998e-06 2024-09-14T09:08:25.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838141+0000", 2024-09-14T09:08:25.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5739999999999998e-06 2024-09-14T09:08:25.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838146+0000", 2024-09-14T09:08:25.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3750000000000002e-06 2024-09-14T09:08:25.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935095+0000", 2024-09-14T09:08:25.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096948435999999999 2024-09-14T09:08:25.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935152+0000", 2024-09-14T09:08:25.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.694e-05 2024-09-14T09:08:25.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935437+0000", 2024-09-14T09:08:25.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028573500000000001 2024-09-14T09:08:25.098 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.098 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.098 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.098 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118168 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 474112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:25.098 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:08:25.098 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.205637906, 2024-09-14T09:08:25.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097658024999999996, 2024-09-14T09:08:25.099 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.099 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.099 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.099 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118168 2024-09-14T09:08:25.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.099 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.099 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:08:25.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:08:25.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838154+0000", 2024-09-14T09:08:25.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4200000000000001e-07 2024-09-14T09:08:25.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838160+0000", 2024-09-14T09:08:25.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8510000000000004e-06 2024-09-14T09:08:25.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838168+0000", 2024-09-14T09:08:25.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9899999999999997e-06 2024-09-14T09:08:25.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838172+0000", 2024-09-14T09:08:25.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.887e-06 2024-09-14T09:08:25.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935473+0000", 2024-09-14T09:08:25.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097300132999999997 2024-09-14T09:08:25.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935523+0000", 2024-09-14T09:08:25.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0862000000000002e-05 2024-09-14T09:08:25.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935811+0000", 2024-09-14T09:08:25.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028786000000000003 2024-09-14T09:08:25.102 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.102 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.102 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.102 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118170 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 478208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:25.102 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:08:25.103 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.205611095000002, 2024-09-14T09:08:25.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.098077529999999996, 2024-09-14T09:08:25.103 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.103 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.103 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.103 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.103 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.103 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.103 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118170 2024-09-14T09:08:25.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.103 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:08:25.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:08:25.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.104 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842901+0000", 2024-09-14T09:08:25.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004721416 2024-09-14T09:08:25.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.104 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842906+0000", 2024-09-14T09:08:25.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7199999999999997e-06 2024-09-14T09:08:25.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842917+0000", 2024-09-14T09:08:25.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0671e-05 2024-09-14T09:08:25.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842923+0000", 2024-09-14T09:08:25.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.088e-06 2024-09-14T09:08:25.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935857+0000", 2024-09-14T09:08:25.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092933624000000006 2024-09-14T09:08:25.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935958+0000", 2024-09-14T09:08:25.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100933 2024-09-14T09:08:25.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.936258+0000", 2024-09-14T09:08:25.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030007800000000001 2024-09-14T09:08:25.106 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.106 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.107 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.107 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118181 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 498688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:25.107 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:08:25.107 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.200804826000002, 2024-09-14T09:08:25.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095212643999999999, 2024-09-14T09:08:25.107 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.107 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.107 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.107 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.107 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.107 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.107 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118181 2024-09-14T09:08:25.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.108 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:08:25.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:08:25.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842987+0000", 2024-09-14T09:08:25.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.29e-07 2024-09-14T09:08:25.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842988+0000", 2024-09-14T09:08:25.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.229e-06 2024-09-14T09:08:25.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.109 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.109 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842993+0000", 2024-09-14T09:08:25.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.792e-06 2024-09-14T09:08:25.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842996+0000", 2024-09-14T09:08:25.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3349999999999998e-06 2024-09-14T09:08:25.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.937807+0000", 2024-09-14T09:08:25.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.094811585000000004 2024-09-14T09:08:25.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.937899+0000", 2024-09-14T09:08:25.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1719999999999996e-05 2024-09-14T09:08:25.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938199+0000", 2024-09-14T09:08:25.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000299754 2024-09-14T09:08:25.111 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.111 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.111 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.111 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118182 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 502784~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:25.111 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:08:25.112 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.200791649000003, 2024-09-14T09:08:25.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095557953000000001, 2024-09-14T09:08:25.112 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.112 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.112 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.112 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.112 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.112 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.112 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118182 2024-09-14T09:08:25.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.112 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:08:25.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:08:25.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:08:25.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2999999999999999e-07 2024-09-14T09:08:25.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843001+0000", 2024-09-14T09:08:25.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.652e-06 2024-09-14T09:08:25.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843005+0000", 2024-09-14T09:08:25.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2780000000000002e-06 2024-09-14T09:08:25.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.114 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.114 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843007+0000", 2024-09-14T09:08:25.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.553e-06 2024-09-14T09:08:25.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938240+0000", 2024-09-14T09:08:25.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095232864 2024-09-14T09:08:25.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938295+0000", 2024-09-14T09:08:25.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5095e-05 2024-09-14T09:08:25.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938557+0000", 2024-09-14T09:08:25.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026228100000000001 2024-09-14T09:08:25.115 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.116 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.116 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.116 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118184 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 506880~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:25.116 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:08:25.116 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.200780449, 2024-09-14T09:08:25.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095950246000000003, 2024-09-14T09:08:25.116 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.116 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.116 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.116 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.116 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.116 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.117 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118184 2024-09-14T09:08:25.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.117 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:08:25.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:08:25.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:08:25.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2399999999999999e-07 2024-09-14T09:08:25.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843012+0000", 2024-09-14T09:08:25.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.269e-06 2024-09-14T09:08:25.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843015+0000", 2024-09-14T09:08:25.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0469999999999998e-06 2024-09-14T09:08:25.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843018+0000", 2024-09-14T09:08:25.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.044e-06 2024-09-14T09:08:25.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.119 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.119 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938597+0000", 2024-09-14T09:08:25.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095578688999999994 2024-09-14T09:08:25.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938687+0000", 2024-09-14T09:08:25.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0284000000000005e-05 2024-09-14T09:08:25.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938961+0000", 2024-09-14T09:08:25.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027368899999999997 2024-09-14T09:08:25.120 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.120 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.120 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.121 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118186 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 510976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:25.121 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:08:25.121 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.200769489000002, 2024-09-14T09:08:25.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096286497999999998, 2024-09-14T09:08:25.121 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.121 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.121 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.121 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.121 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.121 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.121 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118186 2024-09-14T09:08:25.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.121 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:08:25.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:08:25.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:08:25.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.35e-07 2024-09-14T09:08:25.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843023+0000", 2024-09-14T09:08:25.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.195e-06 2024-09-14T09:08:25.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843026+0000", 2024-09-14T09:08:25.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.055e-06 2024-09-14T09:08:25.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843029+0000", 2024-09-14T09:08:25.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6539999999999999e-06 2024-09-14T09:08:25.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939000+0000", 2024-09-14T09:08:25.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095971476 2024-09-14T09:08:25.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939049+0000", 2024-09-14T09:08:25.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8986999999999997e-05 2024-09-14T09:08:25.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939308+0000", 2024-09-14T09:08:25.125 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000258896 2024-09-14T09:08:25.125 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.125 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.125 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.125 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118188 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 515072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:25.126 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:08:25.126 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.200759099999999, 2024-09-14T09:08:25.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096722717999999999, 2024-09-14T09:08:25.126 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.126 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.126 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.126 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.126 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.126 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.126 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118188 2024-09-14T09:08:25.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.126 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:08:25.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:08:25.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:08:25.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2999999999999999e-07 2024-09-14T09:08:25.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843034+0000", 2024-09-14T09:08:25.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T09:08:25.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843037+0000", 2024-09-14T09:08:25.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0989999999999999e-06 2024-09-14T09:08:25.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843039+0000", 2024-09-14T09:08:25.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.666e-06 2024-09-14T09:08:25.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939351+0000", 2024-09-14T09:08:25.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096311789999999994 2024-09-14T09:08:25.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939440+0000", 2024-09-14T09:08:25.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8843000000000003e-05 2024-09-14T09:08:25.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939755+0000", 2024-09-14T09:08:25.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031486399999999999 2024-09-14T09:08:25.130 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.130 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.130 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.130 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118191 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 519168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:25.130 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:08:25.130 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.200748406999999, 2024-09-14T09:08:25.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097080607999999999, 2024-09-14T09:08:25.130 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.130 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.130 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.130 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.130 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.131 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.131 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118191 2024-09-14T09:08:25.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.131 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:08:25.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:08:25.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:08:25.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2100000000000001e-07 2024-09-14T09:08:25.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843044+0000", 2024-09-14T09:08:25.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2249999999999999e-06 2024-09-14T09:08:25.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843047+0000", 2024-09-14T09:08:25.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0240000000000002e-06 2024-09-14T09:08:25.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843050+0000", 2024-09-14T09:08:25.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.508e-06 2024-09-14T09:08:25.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939791+0000", 2024-09-14T09:08:25.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096741399000000006 2024-09-14T09:08:25.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939849+0000", 2024-09-14T09:08:25.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7912999999999999e-05 2024-09-14T09:08:25.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940123+0000", 2024-09-14T09:08:25.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027431800000000001 2024-09-14T09:08:25.134 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.134 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.134 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.134 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118192 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 523264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:25.134 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:08:25.134 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.200737347, 2024-09-14T09:08:25.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097587224, 2024-09-14T09:08:25.135 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.135 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.135 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.135 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118192 2024-09-14T09:08:25.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.135 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.135 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.135 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:08:25.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:08:25.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844132+0000", 2024-09-14T09:08:25.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010777810000000001 2024-09-14T09:08:25.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844140+0000", 2024-09-14T09:08:25.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4470000000000006e-06 2024-09-14T09:08:25.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844155+0000", 2024-09-14T09:08:25.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4924e-05 2024-09-14T09:08:25.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844163+0000", 2024-09-14T09:08:25.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9980000000000003e-06 2024-09-14T09:08:25.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940199+0000", 2024-09-14T09:08:25.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096036047999999999 2024-09-14T09:08:25.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940282+0000", 2024-09-14T09:08:25.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3073999999999998e-05 2024-09-14T09:08:25.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940641+0000", 2024-09-14T09:08:25.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000358952 2024-09-14T09:08:25.139 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.139 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.139 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.139 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118197 3.17 3:ec5ec63a:::rbd_data.115e96ceebb2.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 3718221] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:25.139 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:08:25.139 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 53.195957931999999, 2024-09-14T09:08:25.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.141131069, 2024-09-14T09:08:25.139 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.140 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.140 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:25.140 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.140 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.140 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.140 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118197 2024-09-14T09:08:25.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.140 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:08:25.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:08:25.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847834+0000", 2024-09-14T09:08:25.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2500000000000004e-07 2024-09-14T09:08:25.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849227+0000", 2024-09-14T09:08:25.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0013932149999999999 2024-09-14T09:08:25.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849246+0000", 2024-09-14T09:08:25.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8958e-05 2024-09-14T09:08:25.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849254+0000", 2024-09-14T09:08:25.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2360000000000004e-06 2024-09-14T09:08:25.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940154+0000", 2024-09-14T09:08:25.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.090899468999999997 2024-09-14T09:08:25.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940312+0000", 2024-09-14T09:08:25.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000158326 2024-09-14T09:08:25.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:08:25.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940728+0000", 2024-09-14T09:08:25.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00041583100000000002 2024-09-14T09:08:25.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:25.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.962731+0000", 2024-09-14T09:08:25.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022002608999999999 2024-09-14T09:08:25.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:25.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988881+0000", 2024-09-14T09:08:25.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026150620999999999 2024-09-14T09:08:25.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:25.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988919+0000", 2024-09-14T09:08:25.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8108999999999997e-05 2024-09-14T09:08:25.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988964+0000", 2024-09-14T09:08:25.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4870000000000002e-05 2024-09-14T09:08:25.145 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.146 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.146 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.146 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.146 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.146 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:25.147 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:08:25.148 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:25.150+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:25.248 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:25.254+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:25.275 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:25.275 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:25.275 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:25.276 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:25.290 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:08:25.350 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:25.351+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:25.389 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is set 2024-09-14T09:08:25.416 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:25.416 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:25.416 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:08:25.416 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:08:25.416 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:25.431 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:08:25.451 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:25.454+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:25.545 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:08:25.551 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:25.554+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:25.598 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:25.599 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:08:25.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:08:25.599 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:08:25.599 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.599 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73542 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15319040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:25.599 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:08:25.599 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 111.328596952, 2024-09-14T09:08:25.599 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0237091359999999, 2024-09-14T09:08:25.599 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.599 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.599 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.599 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.599 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.599 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.600 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73542 2024-09-14T09:08:25.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.600 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:08:25.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.600 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.600 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:08:25.600 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.600 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.600 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268762+0000", 2024-09-14T09:08:25.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:08:25.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268766+0000", 2024-09-14T09:08:25.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5880000000000002e-06 2024-09-14T09:08:25.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.601 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.601 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.601 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268776+0000", 2024-09-14T09:08:25.601 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0168e-05 2024-09-14T09:08:25.601 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268789+0000", 2024-09-14T09:08:25.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:08:25.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282053+0000", 2024-09-14T09:08:25.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013264496000000001 2024-09-14T09:08:25.602 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.602 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.602 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.602 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282057+0000", 2024-09-14T09:08:25.602 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6090000000000001e-06 2024-09-14T09:08:25.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292052+0000", 2024-09-14T09:08:25.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0099950070000001 2024-09-14T09:08:25.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.603 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.603 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292262+0000", 2024-09-14T09:08:25.603 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021009800000000001 2024-09-14T09:08:25.603 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.603 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.604 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.604 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292470+0000", 2024-09-14T09:08:25.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000208314 2024-09-14T09:08:25.604 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.604 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.604 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.604 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.604 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.604 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73543 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:25.604 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:08:25.604 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 111.32855816199999, 2024-09-14T09:08:25.604 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023985473, 2024-09-14T09:08:25.604 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.604 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.604 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.605 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.605 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73543 2024-09-14T09:08:25.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.605 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:08:25.605 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.605 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.605 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.605 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.605 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:08:25.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268801+0000", 2024-09-14T09:08:25.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:08:25.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.606 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268803+0000", 2024-09-14T09:08:25.606 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7439999999999999e-06 2024-09-14T09:08:25.606 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.606 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.606 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268812+0000", 2024-09-14T09:08:25.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7080000000000003e-06 2024-09-14T09:08:25.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268819+0000", 2024-09-14T09:08:25.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T09:08:25.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282068+0000", 2024-09-14T09:08:25.607 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013249540000000001 2024-09-14T09:08:25.607 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.607 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.607 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.607 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282072+0000", 2024-09-14T09:08:25.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0960000000000001e-06 2024-09-14T09:08:25.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292573+0000", 2024-09-14T09:08:25.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0105014560000001 2024-09-14T09:08:25.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.608 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292679+0000", 2024-09-14T09:08:25.608 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010545899999999999 2024-09-14T09:08:25.608 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.608 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.608 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.609 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292786+0000", 2024-09-14T09:08:25.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107052 2024-09-14T09:08:25.609 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.609 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.609 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.609 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.609 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73544 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:25.609 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:25.609 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 111.32853046, 2024-09-14T09:08:25.609 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0244830549999999, 2024-09-14T09:08:25.609 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.609 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.609 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.610 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.610 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.610 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.610 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73544 2024-09-14T09:08:25.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.610 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:25.610 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.610 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.610 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.610 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:25.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:25.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5599999999999995e-07 2024-09-14T09:08:25.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268831+0000", 2024-09-14T09:08:25.611 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.57e-06 2024-09-14T09:08:25.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.611 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.611 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.611 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268844+0000", 2024-09-14T09:08:25.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2811e-05 2024-09-14T09:08:25.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268851+0000", 2024-09-14T09:08:25.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9140000000000002e-06 2024-09-14T09:08:25.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282083+0000", 2024-09-14T09:08:25.612 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013232111 2024-09-14T09:08:25.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.612 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.612 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.612 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282086+0000", 2024-09-14T09:08:25.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2909999999999999e-06 2024-09-14T09:08:25.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292884+0000", 2024-09-14T09:08:25.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0107975069999999 2024-09-14T09:08:25.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.613 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.613 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293100+0000", 2024-09-14T09:08:25.613 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021639399999999999 2024-09-14T09:08:25.613 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.613 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.614 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.614 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293311+0000", 2024-09-14T09:08:25.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021090099999999999 2024-09-14T09:08:25.614 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.614 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.614 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.614 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.614 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73545 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15564800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:25.614 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:08:25.614 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 111.32847927500001, 2024-09-14T09:08:25.614 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0249746740000001, 2024-09-14T09:08:25.615 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.615 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.615 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.615 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.615 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.615 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.615 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73545 2024-09-14T09:08:25.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.615 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.615 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.615 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:08:25.615 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.615 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:08:25.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268880+0000", 2024-09-14T09:08:25.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5600000000000005e-07 2024-09-14T09:08:25.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.616 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.616 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268889+0000", 2024-09-14T09:08:25.616 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8559999999999996e-06 2024-09-14T09:08:25.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.616 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268913+0000", 2024-09-14T09:08:25.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3958e-05 2024-09-14T09:08:25.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268921+0000", 2024-09-14T09:08:25.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7479999999999999e-06 2024-09-14T09:08:25.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.617 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.617 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.617 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282098+0000", 2024-09-14T09:08:25.617 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013176297999999999 2024-09-14T09:08:25.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282101+0000", 2024-09-14T09:08:25.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2329999999999998e-06 2024-09-14T09:08:25.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.618 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293435+0000", 2024-09-14T09:08:25.618 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0113339459999999 2024-09-14T09:08:25.618 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.618 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.618 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293648+0000", 2024-09-14T09:08:25.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021307100000000001 2024-09-14T09:08:25.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.619 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.619 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293854+0000", 2024-09-14T09:08:25.619 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020580800000000001 2024-09-14T09:08:25.619 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.619 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.619 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.619 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.619 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.619 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73546 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15646720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:25.619 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:08:25.620 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 111.328425404, 2024-09-14T09:08:25.620 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0254846360000001, 2024-09-14T09:08:25.620 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.620 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.620 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.620 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.620 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.620 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.620 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73546 2024-09-14T09:08:25.620 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.620 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.620 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:08:25.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:08:25.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.621 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.621 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.621 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268934+0000", 2024-09-14T09:08:25.621 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7999999999999995e-07 2024-09-14T09:08:25.621 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268937+0000", 2024-09-14T09:08:25.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9840000000000001e-06 2024-09-14T09:08:25.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268947+0000", 2024-09-14T09:08:25.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0522e-05 2024-09-14T09:08:25.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.622 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.622 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.622 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268955+0000", 2024-09-14T09:08:25.622 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8059999999999995e-06 2024-09-14T09:08:25.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282112+0000", 2024-09-14T09:08:25.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013157189 2024-09-14T09:08:25.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.623 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282115+0000", 2024-09-14T09:08:25.623 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2949999999999998e-06 2024-09-14T09:08:25.623 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.623 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.623 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293968+0000", 2024-09-14T09:08:25.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0118530990000001 2024-09-14T09:08:25.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294197+0000", 2024-09-14T09:08:25.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228507 2024-09-14T09:08:25.624 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.624 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.624 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.624 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294417+0000", 2024-09-14T09:08:25.624 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000220554 2024-09-14T09:08:25.624 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.624 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.625 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.625 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.625 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73547 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:25.625 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:08:25.625 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 111.32839295799999, 2024-09-14T09:08:25.625 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.025970525, 2024-09-14T09:08:25.625 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.625 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.625 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.625 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.625 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.625 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.626 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73547 2024-09-14T09:08:25.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.626 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:08:25.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.626 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.626 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:08:25.626 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.626 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.626 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268966+0000", 2024-09-14T09:08:25.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3799999999999997e-07 2024-09-14T09:08:25.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268969+0000", 2024-09-14T09:08:25.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8739999999999999e-06 2024-09-14T09:08:25.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.627 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.627 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.627 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268979+0000", 2024-09-14T09:08:25.627 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9939999999999995e-06 2024-09-14T09:08:25.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268986+0000", 2024-09-14T09:08:25.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3459999999999996e-06 2024-09-14T09:08:25.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282127+0000", 2024-09-14T09:08:25.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01314039 2024-09-14T09:08:25.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.628 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.628 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.628 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282130+0000", 2024-09-14T09:08:25.628 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2219999999999998e-06 2024-09-14T09:08:25.628 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294540+0000", 2024-09-14T09:08:25.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124105880000001 2024-09-14T09:08:25.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294759+0000", 2024-09-14T09:08:25.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000218798 2024-09-14T09:08:25.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.629 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.629 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.629 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294936+0000", 2024-09-14T09:08:25.629 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000176675 2024-09-14T09:08:25.630 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.630 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.630 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.630 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.630 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73548 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15810560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:25.630 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:08:25.630 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 111.32836184600001, 2024-09-14T09:08:25.630 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0265045129999999, 2024-09-14T09:08:25.630 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.630 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.630 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.630 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.630 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.630 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.630 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73548 2024-09-14T09:08:25.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.631 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:08:25.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:08:25.631 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.631 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.631 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.631 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268997+0000", 2024-09-14T09:08:25.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7599999999999996e-07 2024-09-14T09:08:25.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269000+0000", 2024-09-14T09:08:25.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0580000000000002e-06 2024-09-14T09:08:25.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.632 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269010+0000", 2024-09-14T09:08:25.632 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124e-05 2024-09-14T09:08:25.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.632 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.632 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269018+0000", 2024-09-14T09:08:25.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5399999999999998e-06 2024-09-14T09:08:25.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282141+0000", 2024-09-14T09:08:25.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013123012 2024-09-14T09:08:25.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.633 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.633 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.633 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282144+0000", 2024-09-14T09:08:25.633 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.196e-06 2024-09-14T09:08:25.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295042+0000", 2024-09-14T09:08:25.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0128975689999999 2024-09-14T09:08:25.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295272+0000", 2024-09-14T09:08:25.634 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023048099999999999 2024-09-14T09:08:25.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.634 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.634 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.634 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295501+0000", 2024-09-14T09:08:25.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228657 2024-09-14T09:08:25.635 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.635 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.635 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.635 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.635 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73549 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15892480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:25.635 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:08:25.635 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 111.32831596699999, 2024-09-14T09:08:25.635 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0268354099999999, 2024-09-14T09:08:25.635 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.635 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.635 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.636 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.636 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.636 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.636 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73549 2024-09-14T09:08:25.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.636 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.636 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:08:25.636 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.636 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.636 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:08:25.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269043+0000", 2024-09-14T09:08:25.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6199999999999997e-07 2024-09-14T09:08:25.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.637 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.637 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269046+0000", 2024-09-14T09:08:25.637 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2480000000000001e-06 2024-09-14T09:08:25.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.637 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269057+0000", 2024-09-14T09:08:25.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0436e-05 2024-09-14T09:08:25.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269064+0000", 2024-09-14T09:08:25.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5179999999999998e-06 2024-09-14T09:08:25.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.638 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.638 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.638 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282155+0000", 2024-09-14T09:08:25.638 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013091190000000001 2024-09-14T09:08:25.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282159+0000", 2024-09-14T09:08:25.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.366e-06 2024-09-14T09:08:25.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.639 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295599+0000", 2024-09-14T09:08:25.639 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0134405559999999 2024-09-14T09:08:25.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.639 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.639 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295765+0000", 2024-09-14T09:08:25.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016575499999999999 2024-09-14T09:08:25.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.640 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.640 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295878+0000", 2024-09-14T09:08:25.640 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112679 2024-09-14T09:08:25.640 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.640 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.640 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.640 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.641 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73550 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15974400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:25.641 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:08:25.641 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 111.328283367, 2024-09-14T09:08:25.641 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027357657, 2024-09-14T09:08:25.641 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.641 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.641 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.641 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.641 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.641 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.641 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73550 2024-09-14T09:08:25.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.641 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.641 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:08:25.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:08:25.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.642 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.642 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.642 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269076+0000", 2024-09-14T09:08:25.642 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:08:25.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269079+0000", 2024-09-14T09:08:25.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:08:25.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269098+0000", 2024-09-14T09:08:25.643 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9698e-05 2024-09-14T09:08:25.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.643 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.643 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.643 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269106+0000", 2024-09-14T09:08:25.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6179999999999997e-06 2024-09-14T09:08:25.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282170+0000", 2024-09-14T09:08:25.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013063874 2024-09-14T09:08:25.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.644 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.644 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282173+0000", 2024-09-14T09:08:25.644 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1999999999999999e-06 2024-09-14T09:08:25.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.644 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295996+0000", 2024-09-14T09:08:25.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.013823151 2024-09-14T09:08:25.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.645 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.645 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296214+0000", 2024-09-14T09:08:25.645 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021817700000000001 2024-09-14T09:08:25.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.645 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.646 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.646 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296433+0000", 2024-09-14T09:08:25.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021829099999999999 2024-09-14T09:08:25.646 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.646 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.646 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.646 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.646 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73551 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:25.646 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:08:25.646 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 111.32824195000001, 2024-09-14T09:08:25.646 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027730276, 2024-09-14T09:08:25.646 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.646 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.647 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.647 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.647 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.647 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.647 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73551 2024-09-14T09:08:25.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.647 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.647 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.647 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:08:25.647 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.647 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.647 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:08:25.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269117+0000", 2024-09-14T09:08:25.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:08:25.648 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.648 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.648 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.648 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269120+0000", 2024-09-14T09:08:25.648 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.056e-06 2024-09-14T09:08:25.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269130+0000", 2024-09-14T09:08:25.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0084e-05 2024-09-14T09:08:25.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269137+0000", 2024-09-14T09:08:25.649 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1359999999999999e-06 2024-09-14T09:08:25.649 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.649 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.649 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.649 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282184+0000", 2024-09-14T09:08:25.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013046537 2024-09-14T09:08:25.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282188+0000", 2024-09-14T09:08:25.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3220000000000003e-06 2024-09-14T09:08:25.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296529+0000", 2024-09-14T09:08:25.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0143406109999999 2024-09-14T09:08:25.650 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.650 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.650 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.650 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296695+0000", 2024-09-14T09:08:25.650 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016604799999999999 2024-09-14T09:08:25.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.651 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.651 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296847+0000", 2024-09-14T09:08:25.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000151822 2024-09-14T09:08:25.651 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.651 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.651 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.651 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.651 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.651 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73552 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:25.651 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:25.651 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 111.32821137000001, 2024-09-14T09:08:25.651 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0281767500000001, 2024-09-14T09:08:25.651 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.652 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.652 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.652 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.652 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.652 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.652 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73552 2024-09-14T09:08:25.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.652 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.652 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.652 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:25.652 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.652 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.652 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:25.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:25.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:08:25.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.653 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.653 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.653 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269150+0000", 2024-09-14T09:08:25.653 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9119999999999998e-06 2024-09-14T09:08:25.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269160+0000", 2024-09-14T09:08:25.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6819999999999998e-06 2024-09-14T09:08:25.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269168+0000", 2024-09-14T09:08:25.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6079999999999998e-06 2024-09-14T09:08:25.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.654 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.654 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.654 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282205+0000", 2024-09-14T09:08:25.654 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013036982000000001 2024-09-14T09:08:25.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282208+0000", 2024-09-14T09:08:25.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4620000000000001e-06 2024-09-14T09:08:25.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296968+0000", 2024-09-14T09:08:25.655 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0147601209999999 2024-09-14T09:08:25.655 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.655 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.655 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.655 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297115+0000", 2024-09-14T09:08:25.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014673900000000001 2024-09-14T09:08:25.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.656 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.656 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297324+0000", 2024-09-14T09:08:25.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020866400000000001 2024-09-14T09:08:25.656 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.656 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.656 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.656 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.656 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73553 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:25.656 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:08:25.656 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 111.328181306, 2024-09-14T09:08:25.656 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0286556739999999, 2024-09-14T09:08:25.657 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.657 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.657 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.657 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.657 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.657 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.657 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73553 2024-09-14T09:08:25.657 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.657 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.657 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.657 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.657 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:08:25.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.658 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:08:25.658 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.658 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.658 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.658 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269178+0000", 2024-09-14T09:08:25.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:08:25.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269180+0000", 2024-09-14T09:08:25.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T09:08:25.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.660 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.660 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.660 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269189+0000", 2024-09-14T09:08:25.660 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2e-06 2024-09-14T09:08:25.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269197+0000", 2024-09-14T09:08:25.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2980000000000003e-06 2024-09-14T09:08:25.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282219+0000", 2024-09-14T09:08:25.661 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013022687 2024-09-14T09:08:25.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.661 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.661 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.661 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282223+0000", 2024-09-14T09:08:25.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.241e-06 2024-09-14T09:08:25.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297470+0000", 2024-09-14T09:08:25.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.015247778 2024-09-14T09:08:25.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.662 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297704+0000", 2024-09-14T09:08:25.662 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000233367 2024-09-14T09:08:25.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.662 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.662 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.663 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297833+0000", 2024-09-14T09:08:25.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012888900000000001 2024-09-14T09:08:25.663 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.663 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.663 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.663 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.663 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73554 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:25.663 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:08:25.663 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 111.32815198900001, 2024-09-14T09:08:25.663 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.029005113, 2024-09-14T09:08:25.663 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.663 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.663 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.664 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.664 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.664 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.664 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73554 2024-09-14T09:08:25.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.664 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.664 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:08:25.664 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.664 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.664 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.664 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:08:25.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269207+0000", 2024-09-14T09:08:25.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.68e-07 2024-09-14T09:08:25.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.665 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.665 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.665 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269210+0000", 2024-09-14T09:08:25.665 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8100000000000002e-06 2024-09-14T09:08:25.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269219+0000", 2024-09-14T09:08:25.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:08:25.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269227+0000", 2024-09-14T09:08:25.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3839999999999999e-06 2024-09-14T09:08:25.666 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.666 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.666 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.666 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282234+0000", 2024-09-14T09:08:25.666 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013007187 2024-09-14T09:08:25.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282237+0000", 2024-09-14T09:08:25.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1200000000000002e-06 2024-09-14T09:08:25.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.667 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297942+0000", 2024-09-14T09:08:25.667 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0157048040000001 2024-09-14T09:08:25.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.667 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.667 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298087+0000", 2024-09-14T09:08:25.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014499099999999999 2024-09-14T09:08:25.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.668 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.668 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298211+0000", 2024-09-14T09:08:25.668 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012461899999999999 2024-09-14T09:08:25.668 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.668 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.668 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.668 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.668 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.669 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73555 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:25.669 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:08:25.669 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 111.32812214099999, 2024-09-14T09:08:25.669 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.02941558, 2024-09-14T09:08:25.669 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.669 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.669 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.669 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.669 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.669 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.669 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73555 2024-09-14T09:08:25.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.669 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.669 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:08:25.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:08:25.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.670 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.670 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.670 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269237+0000", 2024-09-14T09:08:25.670 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3399999999999999e-07 2024-09-14T09:08:25.670 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269240+0000", 2024-09-14T09:08:25.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8480000000000001e-06 2024-09-14T09:08:25.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.671 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269249+0000", 2024-09-14T09:08:25.671 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1880000000000008e-06 2024-09-14T09:08:25.671 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.671 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.671 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269256+0000", 2024-09-14T09:08:25.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:08:25.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282248+0000", 2024-09-14T09:08:25.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012992327 2024-09-14T09:08:25.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.672 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.672 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.672 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282251+0000", 2024-09-14T09:08:25.672 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2160000000000002e-06 2024-09-14T09:08:25.672 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298299+0000", 2024-09-14T09:08:25.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0160479330000001 2024-09-14T09:08:25.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.673 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298446+0000", 2024-09-14T09:08:25.673 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000146525 2024-09-14T09:08:25.673 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.673 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.673 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.674 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298652+0000", 2024-09-14T09:08:25.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020584100000000001 2024-09-14T09:08:25.674 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.674 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.674 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.674 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.674 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.674 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73556 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:25.674 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:25.674 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 111.328092395, 2024-09-14T09:08:25.674 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0299854239999999, 2024-09-14T09:08:25.674 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.674 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.675 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.675 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.675 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.675 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.675 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73556 2024-09-14T09:08:25.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.675 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.675 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.675 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:25.675 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.675 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.675 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:25.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.676 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:25.676 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:08:25.676 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.676 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.676 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269270+0000", 2024-09-14T09:08:25.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0920000000000002e-06 2024-09-14T09:08:25.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269280+0000", 2024-09-14T09:08:25.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0013999999999999e-05 2024-09-14T09:08:25.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.677 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.677 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.677 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269288+0000", 2024-09-14T09:08:25.677 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:08:25.677 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282263+0000", 2024-09-14T09:08:25.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012974703000000001 2024-09-14T09:08:25.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282266+0000", 2024-09-14T09:08:25.678 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1520000000000001e-06 2024-09-14T09:08:25.678 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.678 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.678 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.678 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298729+0000", 2024-09-14T09:08:25.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.016463506 2024-09-14T09:08:25.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298884+0000", 2024-09-14T09:08:25.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015450800000000001 2024-09-14T09:08:25.679 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.679 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.679 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.679 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299251+0000", 2024-09-14T09:08:25.679 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036759899999999997 2024-09-14T09:08:25.679 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.679 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.679 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.680 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.680 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.680 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73557 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16547840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:25.680 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:08:25.680 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 111.328045745, 2024-09-14T09:08:25.680 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0307969480000001, 2024-09-14T09:08:25.680 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.680 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.680 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.680 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.680 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.680 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.680 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73557 2024-09-14T09:08:25.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.681 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:08:25.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.681 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.681 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:08:25.681 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.681 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.681 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269313+0000", 2024-09-14T09:08:25.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.92e-07 2024-09-14T09:08:25.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269336+0000", 2024-09-14T09:08:25.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3118000000000001e-05 2024-09-14T09:08:25.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.682 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.682 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269346+0000", 2024-09-14T09:08:25.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4069999999999999e-06 2024-09-14T09:08:25.682 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269352+0000", 2024-09-14T09:08:25.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5810000000000002e-06 2024-09-14T09:08:25.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282277+0000", 2024-09-14T09:08:25.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012924658 2024-09-14T09:08:25.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282280+0000", 2024-09-14T09:08:25.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2880000000000001e-06 2024-09-14T09:08:25.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299343+0000", 2024-09-14T09:08:25.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0170630190000001 2024-09-14T09:08:25.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299554+0000", 2024-09-14T09:08:25.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021085099999999999 2024-09-14T09:08:25.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300109+0000", 2024-09-14T09:08:25.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00055533399999999999 2024-09-14T09:08:25.684 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.685 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.685 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.685 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73558 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:25.685 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:08:25.685 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 111.32799412200001, 2024-09-14T09:08:25.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0314849340000001, 2024-09-14T09:08:25.685 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.685 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.685 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.685 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.685 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.685 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.685 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73558 2024-09-14T09:08:25.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.686 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:08:25.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:08:25.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269365+0000", 2024-09-14T09:08:25.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7400000000000003e-07 2024-09-14T09:08:25.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269367+0000", 2024-09-14T09:08:25.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4930000000000001e-06 2024-09-14T09:08:25.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269374+0000", 2024-09-14T09:08:25.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9519999999999997e-06 2024-09-14T09:08:25.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269379+0000", 2024-09-14T09:08:25.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9710000000000003e-06 2024-09-14T09:08:25.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282292+0000", 2024-09-14T09:08:25.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012912704000000001 2024-09-14T09:08:25.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.688 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.688 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282295+0000", 2024-09-14T09:08:25.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.179e-06 2024-09-14T09:08:25.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300195+0000", 2024-09-14T09:08:25.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.017899975 2024-09-14T09:08:25.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300345+0000", 2024-09-14T09:08:25.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014957499999999999 2024-09-14T09:08:25.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300849+0000", 2024-09-14T09:08:25.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050451100000000002 2024-09-14T09:08:25.689 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.690 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.690 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.690 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73559 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:25.690 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:25.690 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 111.327971713, 2024-09-14T09:08:25.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.031916343, 2024-09-14T09:08:25.690 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.690 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.690 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:25.690 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.690 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.690 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.690 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73559 2024-09-14T09:08:25.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.691 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:25.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:25.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:25.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3500000000000002e-07 2024-09-14T09:08:25.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269390+0000", 2024-09-14T09:08:25.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5890000000000001e-06 2024-09-14T09:08:25.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269396+0000", 2024-09-14T09:08:25.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3300000000000004e-06 2024-09-14T09:08:25.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269401+0000", 2024-09-14T09:08:25.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0340000000000004e-06 2024-09-14T09:08:25.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282311+0000", 2024-09-14T09:08:25.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012910204 2024-09-14T09:08:25.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282317+0000", 2024-09-14T09:08:25.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2639999999999997e-06 2024-09-14T09:08:25.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300933+0000", 2024-09-14T09:08:25.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0186160710000001 2024-09-14T09:08:25.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301158+0000", 2024-09-14T09:08:25.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225021 2024-09-14T09:08:25.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.694 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.694 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301303+0000", 2024-09-14T09:08:25.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014439499999999999 2024-09-14T09:08:25.694 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.694 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.695 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.695 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106964 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:25.695 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:08:25.695 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.482043879000003, 2024-09-14T09:08:25.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8802924409999999, 2024-09-14T09:08:25.695 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.695 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.695 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:25.695 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.695 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.695 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.695 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106964 2024-09-14T09:08:25.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.695 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:08:25.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:08:25.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115317+0000", 2024-09-14T09:08:25.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6429999999999999e-06 2024-09-14T09:08:25.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115326+0000", 2024-09-14T09:08:25.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2140000000000002e-06 2024-09-14T09:08:25.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115346+0000", 2024-09-14T09:08:25.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9419000000000002e-05 2024-09-14T09:08:25.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115357+0000", 2024-09-14T09:08:25.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1014e-05 2024-09-14T09:08:25.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115431+0000", 2024-09-14T09:08:25.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4040000000000003e-05 2024-09-14T09:08:25.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115438+0000", 2024-09-14T09:08:25.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9920000000000001e-06 2024-09-14T09:08:25.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844944+0000", 2024-09-14T09:08:25.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72950628399999995 2024-09-14T09:08:25.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844961+0000", 2024-09-14T09:08:25.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7156e-05 2024-09-14T09:08:25.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988004+0000", 2024-09-14T09:08:25.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.14304297 2024-09-14T09:08:25.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.699 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988511+0000", 2024-09-14T09:08:25.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050694100000000001 2024-09-14T09:08:25.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:08:25.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.989802+0000", 2024-09-14T09:08:25.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001291195 2024-09-14T09:08:25.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:25.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994866+0000", 2024-09-14T09:08:25.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050632760000000002 2024-09-14T09:08:25.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:25.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994988+0000", 2024-09-14T09:08:25.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012252899999999999 2024-09-14T09:08:25.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:25.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995198+0000", 2024-09-14T09:08:25.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021010899999999999 2024-09-14T09:08:25.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995607+0000", 2024-09-14T09:08:25.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040865900000000002 2024-09-14T09:08:25.701 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.701 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.701 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.702 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106965 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:25.702 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:25.702 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 64.480841502000004, 2024-09-14T09:08:25.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.879730162, 2024-09-14T09:08:25.702 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:25.702 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:25.702 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:25.702 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:25.702 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:25.702 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:25.702 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106965 2024-09-14T09:08:25.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.702 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:25.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:25.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:25.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:25.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:25.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:25.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:25.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:25.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:08:25.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:25.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116522+0000", 2024-09-14T09:08:25.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3989999999999997e-06 2024-09-14T09:08:25.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:25.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116533+0000", 2024-09-14T09:08:25.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0669000000000001e-05 2024-09-14T09:08:25.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:25.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116541+0000", 2024-09-14T09:08:25.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2730000000000002e-06 2024-09-14T09:08:25.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116604+0000", 2024-09-14T09:08:25.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2706999999999994e-05 2024-09-14T09:08:25.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116610+0000", 2024-09-14T09:08:25.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8630000000000002e-06 2024-09-14T09:08:25.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.705 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.705 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845060+0000", 2024-09-14T09:08:25.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72845003699999999 2024-09-14T09:08:25.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:25.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845079+0000", 2024-09-14T09:08:25.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8287999999999998e-05 2024-09-14T09:08:25.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:25.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.991866+0000", 2024-09-14T09:08:25.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1467871519999999 2024-09-14T09:08:25.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:25.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.992274+0000", 2024-09-14T09:08:25.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040813500000000001 2024-09-14T09:08:25.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:08:25.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.993538+0000", 2024-09-14T09:08:25.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001263655 2024-09-14T09:08:25.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:25.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995770+0000", 2024-09-14T09:08:25.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002231949 2024-09-14T09:08:25.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:25.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995810+0000", 2024-09-14T09:08:25.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0852999999999997e-05 2024-09-14T09:08:25.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:25.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995979+0000", 2024-09-14T09:08:25.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016866 2024-09-14T09:08:25.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:25.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:25.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:25.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996247+0000", 2024-09-14T09:08:25.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000267769 2024-09-14T09:08:25.708 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.708 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.708 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.709 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:25.709 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:25.709 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:25.710 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:08:25.711 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:08:25.711 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:08:25.711 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:25.711+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:25.718 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:08:25.719 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:08:25.736 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:08:25.736 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:08:25.752 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:08:25.753 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:08:25.753 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:08:25.773 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:08:25.774 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:08:25.794 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:08:25.794 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:08:25.811 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:25.811+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:25.815 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:08:25.815 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:08:25.819 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:08:25.819 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T09:08:25.835 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:25.835 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:25.849 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:08:25.942 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:25.943+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:26.013 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:26.018+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:26.018 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:26.019 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:26.033 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:08:26.114 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:26.118+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:26.195 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:26.195 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:26.209 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:08:26.215 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:26.218+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:26.315 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:26.318+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:26.332 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:26.332 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:26.346 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:08:26.416 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:26.415+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:26.481 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:26.481 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:26.496 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:08:26.538 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:26.539+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:26.618 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:26.619+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:26.661 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:26.661 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:26.676 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:08:26.719 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:26.719+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:26.806 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:26.807 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:26.820 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:26.819+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:26.822 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:08:26.922 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:26.926+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:26.944 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:26.945 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:26.959 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:08:27.023 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:27.023+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:27.123 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:27.123 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:27.124 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:27.126+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:27.137 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:08:27.225 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:27.223+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:27.301 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:27.301 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:27.315 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:08:27.326 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:27.327+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:27.420 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T09:08:27.434 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:27.435 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:27.451 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:08:27.470 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:27.471+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:27.527 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:27.530+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:27.581 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:27.581 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:27.600 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:08:27.628 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:27.627+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:27.729 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:27.734+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:27.762 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:27.762 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:27.777 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:08:27.820 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:08:27.866 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:27.867+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:27.931 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:27.934+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:27.951 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:27.951 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:27.966 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:08:27.969 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:08:27.969 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:08:27.986 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:08:27.986 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:08:28.004 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:08:28.005 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:08:28.021 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:08:28.022 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:08:28.022 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:08:28.042 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:08:28.042 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:08:28.054 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:28.058+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:28.060 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:08:28.060 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:08:28.081 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:08:28.081 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:08:28.085 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:08:28.085 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T09:08:28.133 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:28.133 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:08:28.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:08:28.133 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:08:28.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.133 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38593 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [watch reconnect cookie 139718844451680 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:08:28.133 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:08:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 146.31800818799999, 2024-09-14T09:08:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0218296630000001, 2024-09-14T09:08:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38593 2024-09-14T09:08:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:08:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:08:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818420+0000", 2024-09-14T09:08:28.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8399999999999999e-06 2024-09-14T09:08:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818431+0000", 2024-09-14T09:08:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0859999999999999e-05 2024-09-14T09:08:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818466+0000", 2024-09-14T09:08:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4486999999999997e-05 2024-09-14T09:08:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818481+0000", 2024-09-14T09:08:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5362999999999998e-05 2024-09-14T09:08:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818659+0000", 2024-09-14T09:08:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017807500000000001 2024-09-14T09:08:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op must wait for map", 2024-09-14T09:08:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818681+0000", 2024-09-14T09:08:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2096999999999999e-05 2024-09-14T09:08:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825472+0000", 2024-09-14T09:08:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067905170000000003 2024-09-14T09:08:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825491+0000", 2024-09-14T09:08:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9372999999999999e-05 2024-09-14T09:08:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835402+0000", 2024-09-14T09:08:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0099111620000001 2024-09-14T09:08:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835548+0000", 2024-09-14T09:08:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014607300000000001 2024-09-14T09:08:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.840247+0000", 2024-09-14T09:08:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046988159999999998 2024-09-14T09:08:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.139 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38615 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:08:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:08:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 146.26964297999999, 2024-09-14T09:08:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.97014128899999996, 2024-09-14T09:08:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.140 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38615 2024-09-14T09:08:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:08:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:08:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866787+0000", 2024-09-14T09:08:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.476e-06 2024-09-14T09:08:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866806+0000", 2024-09-14T09:08:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8606000000000001e-05 2024-09-14T09:08:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866839+0000", 2024-09-14T09:08:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3510000000000003e-05 2024-09-14T09:08:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866867+0000", 2024-09-14T09:08:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7610000000000002e-05 2024-09-14T09:08:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867031+0000", 2024-09-14T09:08:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016414199999999999 2024-09-14T09:08:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867054+0000", 2024-09-14T09:08:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2693000000000001e-05 2024-09-14T09:08:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836462+0000", 2024-09-14T09:08:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.96940797700000003 2024-09-14T09:08:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836547+0000", 2024-09-14T09:08:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5294e-05 2024-09-14T09:08:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836924+0000", 2024-09-14T09:08:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037698099999999997 2024-09-14T09:08:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105723 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.041208936000004, 2024-09-14T09:08:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77512799700000001, 2024-09-14T09:08:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.145 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105723 2024-09-14T09:08:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:28.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1399999999999997e-07 2024-09-14T09:08:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095220+0000", 2024-09-14T09:08:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0510000000000001e-06 2024-09-14T09:08:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095228+0000", 2024-09-14T09:08:28.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4159999999999998e-06 2024-09-14T09:08:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095234+0000", 2024-09-14T09:08:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3010000000000003e-06 2024-09-14T09:08:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103902+0000", 2024-09-14T09:08:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086678299999999996 2024-09-14T09:08:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:28.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103905+0000", 2024-09-14T09:08:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5020000000000001e-06 2024-09-14T09:08:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870332+0000", 2024-09-14T09:08:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76642700900000005 2024-09-14T09:08:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870345+0000", 2024-09-14T09:08:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2274e-05 2024-09-14T09:08:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105724 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:08:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.041184423000004, 2024-09-14T09:08:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77515698099999997, 2024-09-14T09:08:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:28.150 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105724 2024-09-14T09:08:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:08:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:08:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095242+0000", 2024-09-14T09:08:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9400000000000001e-07 2024-09-14T09:08:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095243+0000", 2024-09-14T09:08:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7540000000000001e-06 2024-09-14T09:08:28.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095248+0000", 2024-09-14T09:08:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1320000000000004e-06 2024-09-14T09:08:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095251+0000", 2024-09-14T09:08:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2770000000000001e-06 2024-09-14T09:08:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103947+0000", 2024-09-14T09:08:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086960089999999993 2024-09-14T09:08:28.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103955+0000", 2024-09-14T09:08:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9519999999999994e-06 2024-09-14T09:08:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870386+0000", 2024-09-14T09:08:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76643141100000001 2024-09-14T09:08:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870398+0000", 2024-09-14T09:08:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2052000000000001e-05 2024-09-14T09:08:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105727 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:28.155 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.041144716999995, 2024-09-14T09:08:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517292900000001, 2024-09-14T09:08:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105727 2024-09-14T09:08:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3300000000000002e-07 2024-09-14T09:08:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095284+0000", 2024-09-14T09:08:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9790000000000002e-06 2024-09-14T09:08:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095292+0000", 2024-09-14T09:08:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3810000000000001e-06 2024-09-14T09:08:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095298+0000", 2024-09-14T09:08:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9710000000000001e-06 2024-09-14T09:08:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103987+0000", 2024-09-14T09:08:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086893820000000007 2024-09-14T09:08:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103991+0000", 2024-09-14T09:08:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5480000000000002e-06 2024-09-14T09:08:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870442+0000", 2024-09-14T09:08:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.766451145 2024-09-14T09:08:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870454+0000", 2024-09-14T09:08:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.199e-05 2024-09-14T09:08:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105729 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:08:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.041096359999997, 2024-09-14T09:08:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517686500000005, 2024-09-14T09:08:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:28.160 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105729 2024-09-14T09:08:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:08:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:08:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095330+0000", 2024-09-14T09:08:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-09-14T09:08:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095332+0000", 2024-09-14T09:08:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1560000000000002e-06 2024-09-14T09:08:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095338+0000", 2024-09-14T09:08:28.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.592e-06 2024-09-14T09:08:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095341+0000", 2024-09-14T09:08:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8139999999999998e-06 2024-09-14T09:08:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104031+0000", 2024-09-14T09:08:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086896330000000004 2024-09-14T09:08:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104039+0000", 2024-09-14T09:08:28.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0900000000000005e-06 2024-09-14T09:08:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870495+0000", 2024-09-14T09:08:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76645611199999997 2024-09-14T09:08:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870506+0000", 2024-09-14T09:08:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0986e-05 2024-09-14T09:08:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.164 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105731 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.041078092999996, 2024-09-14T09:08:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77520598600000001, 2024-09-14T09:08:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105731 2024-09-14T09:08:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.165 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8599999999999999e-07 2024-09-14T09:08:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095350+0000", 2024-09-14T09:08:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1110000000000002e-06 2024-09-14T09:08:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095357+0000", 2024-09-14T09:08:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9020000000000002e-06 2024-09-14T09:08:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095360+0000", 2024-09-14T09:08:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1939999999999998e-06 2024-09-14T09:08:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104073+0000", 2024-09-14T09:08:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087131210000000008 2024-09-14T09:08:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:28.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104077+0000", 2024-09-14T09:08:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.501e-06 2024-09-14T09:08:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870541+0000", 2024-09-14T09:08:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76646461700000001 2024-09-14T09:08:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870554+0000", 2024-09-14T09:08:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2153999999999999e-05 2024-09-14T09:08:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.169 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105733 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.041058062000005, 2024-09-14T09:08:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.775237277, 2024-09-14T09:08:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.170 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105733 2024-09-14T09:08:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8699999999999995e-07 2024-09-14T09:08:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095371+0000", 2024-09-14T09:08:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5450000000000002e-06 2024-09-14T09:08:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095377+0000", 2024-09-14T09:08:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.776e-06 2024-09-14T09:08:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095383+0000", 2024-09-14T09:08:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3129999999999998e-06 2024-09-14T09:08:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104124+0000", 2024-09-14T09:08:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087409840000000003 2024-09-14T09:08:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:28.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104132+0000", 2024-09-14T09:08:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3159999999999997e-06 2024-09-14T09:08:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870593+0000", 2024-09-14T09:08:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76646064800000002 2024-09-14T09:08:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870605+0000", 2024-09-14T09:08:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2208000000000001e-05 2024-09-14T09:08:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105734 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.041034980000006, 2024-09-14T09:08:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77526569599999995, 2024-09-14T09:08:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.175 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105734 2024-09-14T09:08:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7399999999999998e-07 2024-09-14T09:08:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095394+0000", 2024-09-14T09:08:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8030000000000001e-06 2024-09-14T09:08:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095401+0000", 2024-09-14T09:08:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.046e-06 2024-09-14T09:08:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095407+0000", 2024-09-14T09:08:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6060000000000002e-06 2024-09-14T09:08:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104167+0000", 2024-09-14T09:08:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087602530000000008 2024-09-14T09:08:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104171+0000", 2024-09-14T09:08:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7620000000000001e-06 2024-09-14T09:08:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870645+0000", 2024-09-14T09:08:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76647400600000004 2024-09-14T09:08:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870656+0000", 2024-09-14T09:08:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1746000000000001e-05 2024-09-14T09:08:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105738 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:08:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.040995253000006, 2024-09-14T09:08:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77527530700000002, 2024-09-14T09:08:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105738 2024-09-14T09:08:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.180 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:08:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:08:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095431+0000", 2024-09-14T09:08:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0999999999999999e-07 2024-09-14T09:08:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095434+0000", 2024-09-14T09:08:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.757e-06 2024-09-14T09:08:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095442+0000", 2024-09-14T09:08:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9929999999999995e-06 2024-09-14T09:08:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095448+0000", 2024-09-14T09:08:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1340000000000001e-06 2024-09-14T09:08:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104213+0000", 2024-09-14T09:08:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087650000000000002 2024-09-14T09:08:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:28.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104221+0000", 2024-09-14T09:08:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4589999999999998e-06 2024-09-14T09:08:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870694+0000", 2024-09-14T09:08:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76647303200000005 2024-09-14T09:08:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870706+0000", 2024-09-14T09:08:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1522000000000001e-05 2024-09-14T09:08:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105739 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.040970724000005, 2024-09-14T09:08:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77530221700000002, 2024-09-14T09:08:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105739 2024-09-14T09:08:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:28.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.15e-07 2024-09-14T09:08:28.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095457+0000", 2024-09-14T09:08:28.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6470000000000001e-06 2024-09-14T09:08:28.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095461+0000", 2024-09-14T09:08:28.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0910000000000003e-06 2024-09-14T09:08:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095464+0000", 2024-09-14T09:08:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2059999999999999e-06 2024-09-14T09:08:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104253+0000", 2024-09-14T09:08:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087884120000000007 2024-09-14T09:08:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104256+0000", 2024-09-14T09:08:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5439999999999999e-06 2024-09-14T09:08:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870745+0000", 2024-09-14T09:08:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76648908000000004 2024-09-14T09:08:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870757+0000", 2024-09-14T09:08:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1822e-05 2024-09-14T09:08:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105741 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:08:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.040901598000005, 2024-09-14T09:08:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77528469899999997, 2024-09-14T09:08:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.190 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105741 2024-09-14T09:08:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:08:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:08:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095525+0000", 2024-09-14T09:08:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3740000000000001e-06 2024-09-14T09:08:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095533+0000", 2024-09-14T09:08:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6380000000000004e-06 2024-09-14T09:08:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095549+0000", 2024-09-14T09:08:28.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5554999999999999e-05 2024-09-14T09:08:28.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095566+0000", 2024-09-14T09:08:28.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7146000000000002e-05 2024-09-14T09:08:28.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104297+0000", 2024-09-14T09:08:28.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087315750000000001 2024-09-14T09:08:28.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:28.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104305+0000", 2024-09-14T09:08:28.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1389999999999995e-06 2024-09-14T09:08:28.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870797+0000", 2024-09-14T09:08:28.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76649129599999999 2024-09-14T09:08:28.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870809+0000", 2024-09-14T09:08:28.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1976e-05 2024-09-14T09:08:28.195 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.195 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.195 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.195 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105742 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:28.195 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:08:28.195 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.040836814000002, 2024-09-14T09:08:28.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.775271873, 2024-09-14T09:08:28.195 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.195 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.195 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:28.195 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.195 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.196 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.196 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105742 2024-09-14T09:08:28.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.196 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:08:28.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:08:28.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095590+0000", 2024-09-14T09:08:28.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0400000000000005e-07 2024-09-14T09:08:28.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095594+0000", 2024-09-14T09:08:28.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5639999999999998e-06 2024-09-14T09:08:28.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095620+0000", 2024-09-14T09:08:28.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5494999999999999e-05 2024-09-14T09:08:28.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095639+0000", 2024-09-14T09:08:28.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9639999999999998e-05 2024-09-14T09:08:28.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104340+0000", 2024-09-14T09:08:28.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087004950000000008 2024-09-14T09:08:28.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:28.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104343+0000", 2024-09-14T09:08:28.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4020000000000002e-06 2024-09-14T09:08:28.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870850+0000", 2024-09-14T09:08:28.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76650648499999996 2024-09-14T09:08:28.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870861+0000", 2024-09-14T09:08:28.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0888e-05 2024-09-14T09:08:28.199 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.199 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.199 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.200 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105745 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:28.200 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:08:28.200 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.040738199000003, 2024-09-14T09:08:28.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77521998700000005, 2024-09-14T09:08:28.200 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.200 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.200 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:28.200 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.200 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.200 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.200 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105745 2024-09-14T09:08:28.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.200 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:08:28.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:08:28.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095688+0000", 2024-09-14T09:08:28.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2699999999999998e-07 2024-09-14T09:08:28.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095693+0000", 2024-09-14T09:08:28.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0479999999999998e-06 2024-09-14T09:08:28.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095707+0000", 2024-09-14T09:08:28.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.342e-05 2024-09-14T09:08:28.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095716+0000", 2024-09-14T09:08:28.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9800000000000004e-06 2024-09-14T09:08:28.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104388+0000", 2024-09-14T09:08:28.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086717349999999999 2024-09-14T09:08:28.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:28.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104396+0000", 2024-09-14T09:08:28.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1769999999999998e-06 2024-09-14T09:08:28.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870897+0000", 2024-09-14T09:08:28.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76650102600000003 2024-09-14T09:08:28.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870907+0000", 2024-09-14T09:08:28.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0674e-05 2024-09-14T09:08:28.204 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.204 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.204 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.204 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105747 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:28.204 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:28.204 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.040691979000002, 2024-09-14T09:08:28.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77522190300000005, 2024-09-14T09:08:28.204 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.204 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.204 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:28.205 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.205 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.205 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.205 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105747 2024-09-14T09:08:28.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.205 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:28.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:28.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:28.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8999999999999995e-07 2024-09-14T09:08:28.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095738+0000", 2024-09-14T09:08:28.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8430000000000003e-06 2024-09-14T09:08:28.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095750+0000", 2024-09-14T09:08:28.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1229999999999999e-05 2024-09-14T09:08:28.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095758+0000", 2024-09-14T09:08:28.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.337e-06 2024-09-14T09:08:28.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104431+0000", 2024-09-14T09:08:28.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086728780000000002 2024-09-14T09:08:28.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:28.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104434+0000", 2024-09-14T09:08:28.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5049999999999999e-06 2024-09-14T09:08:28.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870945+0000", 2024-09-14T09:08:28.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651031800000002 2024-09-14T09:08:28.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870956+0000", 2024-09-14T09:08:28.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1002000000000001e-05 2024-09-14T09:08:28.209 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.209 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.209 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.209 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105750 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:28.209 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:28.209 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.040622033000005, 2024-09-14T09:08:28.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77519961800000003, 2024-09-14T09:08:28.210 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.210 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.210 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:28.210 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.210 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.210 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.210 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105750 2024-09-14T09:08:28.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.210 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:28.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:28.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:28.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3600000000000002e-07 2024-09-14T09:08:28.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095809+0000", 2024-09-14T09:08:28.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4000000000000002e-06 2024-09-14T09:08:28.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095822+0000", 2024-09-14T09:08:28.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2704000000000001e-05 2024-09-14T09:08:28.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095831+0000", 2024-09-14T09:08:28.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9530000000000005e-06 2024-09-14T09:08:28.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104477+0000", 2024-09-14T09:08:28.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086459989999999997 2024-09-14T09:08:28.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:28.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104488+0000", 2024-09-14T09:08:28.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:08:28.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870992+0000", 2024-09-14T09:08:28.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.766504097 2024-09-14T09:08:28.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871003+0000", 2024-09-14T09:08:28.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1358999999999999e-05 2024-09-14T09:08:28.214 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.214 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.214 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.214 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105751 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:28.214 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:28.214 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.040564148000001, 2024-09-14T09:08:28.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77518966700000003, 2024-09-14T09:08:28.214 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.215 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.215 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:28.215 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.215 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.215 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.215 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105751 2024-09-14T09:08:28.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.215 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:28.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:28.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:28.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7300000000000005e-07 2024-09-14T09:08:28.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095867+0000", 2024-09-14T09:08:28.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2729999999999996e-06 2024-09-14T09:08:28.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095879+0000", 2024-09-14T09:08:28.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2917e-05 2024-09-14T09:08:28.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095888+0000", 2024-09-14T09:08:28.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2609999999999993e-06 2024-09-14T09:08:28.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104520+0000", 2024-09-14T09:08:28.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086322970000000006 2024-09-14T09:08:28.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:28.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104524+0000", 2024-09-14T09:08:28.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.54e-06 2024-09-14T09:08:28.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871040+0000", 2024-09-14T09:08:28.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651615399999995 2024-09-14T09:08:28.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871051+0000", 2024-09-14T09:08:28.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1452e-05 2024-09-14T09:08:28.219 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.219 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.219 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.219 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105752 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:28.219 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:08:28.219 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.040503384000004, 2024-09-14T09:08:28.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517684899999995, 2024-09-14T09:08:28.220 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.220 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.220 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:28.220 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.220 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.220 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.220 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105752 2024-09-14T09:08:28.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.220 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:08:28.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:08:28.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095923+0000", 2024-09-14T09:08:28.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3300000000000001e-07 2024-09-14T09:08:28.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095927+0000", 2024-09-14T09:08:28.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:08:28.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095940+0000", 2024-09-14T09:08:28.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2309e-05 2024-09-14T09:08:28.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095948+0000", 2024-09-14T09:08:28.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9199999999999993e-06 2024-09-14T09:08:28.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104566+0000", 2024-09-14T09:08:28.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086171449999999997 2024-09-14T09:08:28.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:28.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104574+0000", 2024-09-14T09:08:28.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0690000000000002e-06 2024-09-14T09:08:28.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871087+0000", 2024-09-14T09:08:28.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651370699999999 2024-09-14T09:08:28.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.224 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.224 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871099+0000", 2024-09-14T09:08:28.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1718e-05 2024-09-14T09:08:28.224 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.224 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.224 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.224 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105755 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:28.225 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:08:28.225 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.040445441000003, 2024-09-14T09:08:28.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77516787899999995, 2024-09-14T09:08:28.225 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.225 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.225 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:28.225 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.225 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.225 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.225 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105755 2024-09-14T09:08:28.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.225 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:08:28.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:08:28.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095981+0000", 2024-09-14T09:08:28.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2699999999999998e-07 2024-09-14T09:08:28.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095985+0000", 2024-09-14T09:08:28.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4349999999999999e-06 2024-09-14T09:08:28.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095999+0000", 2024-09-14T09:08:28.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3139000000000001e-05 2024-09-14T09:08:28.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096007+0000", 2024-09-14T09:08:28.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6610000000000006e-06 2024-09-14T09:08:28.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104591+0000", 2024-09-14T09:08:28.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0085841709999999998 2024-09-14T09:08:28.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:28.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104595+0000", 2024-09-14T09:08:28.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5590000000000001e-06 2024-09-14T09:08:28.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871137+0000", 2024-09-14T09:08:28.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76654202100000002 2024-09-14T09:08:28.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871148+0000", 2024-09-14T09:08:28.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1066e-05 2024-09-14T09:08:28.229 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.230 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.230 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.230 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105757 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:28.230 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:08:28.230 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.040386861000002, 2024-09-14T09:08:28.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77516401300000004, 2024-09-14T09:08:28.230 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:28.230 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:28.230 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:28.230 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:28.230 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:28.231 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.231 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105757 2024-09-14T09:08:28.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.231 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:28.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:28.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:08:28.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:28.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:08:28.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:28.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:28.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856228+0000", 2024-09-14T09:08:28.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76018875100000005 2024-09-14T09:08:28.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:28.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856240+0000", 2024-09-14T09:08:28.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-05 2024-09-14T09:08:28.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:28.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856268+0000", 2024-09-14T09:08:28.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7628999999999999e-05 2024-09-14T09:08:28.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856307+0000", 2024-09-14T09:08:28.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8967000000000001e-05 2024-09-14T09:08:28.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:28.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871192+0000", 2024-09-14T09:08:28.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.01488449 2024-09-14T09:08:28.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:28.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:28.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:28.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871203+0000", 2024-09-14T09:08:28.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1266e-05 2024-09-14T09:08:28.234 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.234 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.234 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.234 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:28.234 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:28.234 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:28.235 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:28.131+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:28.236 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:28.238+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:28.236 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:08:28.337 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:28.338+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:28.368 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:28.369 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:28.369 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:28.369 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:28.383 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:08:28.402 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-14T09:08:28.402 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T09:08:28.403 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T09:08:28.403 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T09:08:28.443 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is set 2024-09-14T09:08:28.443 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:28.443+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:28.524 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:28.525 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:28.525 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:08:28.525 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:08:28.525 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:28.538 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:28.539+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:28.540 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:08:28.639 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:28.639+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:28.707 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:28.707 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:08:28.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:08:28.707 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:08:28.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.708 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:71052 3.1 3:80855826:::rbd_data.115ebd4a9f36.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-14T09:08:28.708 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:08:28.708 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.968964301, 2024-09-14T09:08:28.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14854635599999999, 2024-09-14T09:08:28.708 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.708 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.708 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:28.708 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:28.708 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:28.708 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.708 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 71052 2024-09-14T09:08:28.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.709 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:08:28.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737568+0000", 2024-09-14T09:08:28.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737570+0000", 2024-09-14T09:08:28.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4770000000000002e-06 2024-09-14T09:08:28.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737584+0000", 2024-09-14T09:08:28.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3457e-05 2024-09-14T09:08:28.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.710 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737608+0000", 2024-09-14T09:08:28.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3893999999999999e-05 2024-09-14T09:08:28.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737620+0000", 2024-09-14T09:08:28.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2157000000000001e-05 2024-09-14T09:08:28.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737738+0000", 2024-09-14T09:08:28.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000118409 2024-09-14T09:08:28.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.737892+0000", 2024-09-14T09:08:28.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000154222 2024-09-14T09:08:28.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:08:28.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.749622+0000", 2024-09-14T09:08:28.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011729537 2024-09-14T09:08:28.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:28.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885934+0000", 2024-09-14T09:08:28.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13631222200000001 2024-09-14T09:08:28.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:28.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885996+0000", 2024-09-14T09:08:28.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1555000000000003e-05 2024-09-14T09:08:28.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:28.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886060+0000", 2024-09-14T09:08:28.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3980999999999996e-05 2024-09-14T09:08:28.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886114+0000", 2024-09-14T09:08:28.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.4444999999999998e-05 2024-09-14T09:08:28.713 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.713 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.714 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.714 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:71052 3.1 e71/34 ondisk, result = 0)", 2024-09-14T09:08:28.714 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:08:28.714 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.92606092299999, 2024-09-14T09:08:28.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.10565334799999999, 2024-09-14T09:08:28.714 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.714 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.714 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:28.714 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:08:28.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780471+0000", 2024-09-14T09:08:28.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780473+0000", 2024-09-14T09:08:28.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8980000000000001e-06 2024-09-14T09:08:28.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.715 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.715 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780482+0000", 2024-09-14T09:08:28.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9080000000000001e-06 2024-09-14T09:08:28.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780511+0000", 2024-09-14T09:08:28.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9490999999999999e-05 2024-09-14T09:08:28.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.780527+0000", 2024-09-14T09:08:28.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.508e-05 2024-09-14T09:08:28.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885978+0000", 2024-09-14T09:08:28.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.105451328 2024-09-14T09:08:28.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.885989+0000", 2024-09-14T09:08:28.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0875e-05 2024-09-14T09:08:28.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:29.886125+0000", 2024-09-14T09:08:28.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013576800000000001 2024-09-14T09:08:28.717 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.717 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.717 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.718 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:98007 3.12 3:4faf1f2a:::rbd_data.115e113dc8ca.000000000000004f:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:08:28.718 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:08:28.718 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 75.849088699000006, 2024-09-14T09:08:28.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.098233707000000003, 2024-09-14T09:08:28.718 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.718 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.718 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:28.718 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:28.718 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:28.718 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.718 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 98007 2024-09-14T09:08:28.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.718 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:08:28.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857443+0000", 2024-09-14T09:08:28.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857446+0000", 2024-09-14T09:08:28.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7599999999999998e-06 2024-09-14T09:08:28.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857459+0000", 2024-09-14T09:08:28.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3123e-05 2024-09-14T09:08:28.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857491+0000", 2024-09-14T09:08:28.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1603000000000002e-05 2024-09-14T09:08:28.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857506+0000", 2024-09-14T09:08:28.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.537e-05 2024-09-14T09:08:28.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857694+0000", 2024-09-14T09:08:28.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018766599999999999 2024-09-14T09:08:28.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.857955+0000", 2024-09-14T09:08:28.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000260979 2024-09-14T09:08:28.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 3", 2024-09-14T09:08:28.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.863125+0000", 2024-09-14T09:08:28.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.005170582 2024-09-14T09:08:28.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:28.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955268+0000", 2024-09-14T09:08:28.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092142659000000002 2024-09-14T09:08:28.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:28.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955385+0000", 2024-09-14T09:08:28.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116723 2024-09-14T09:08:28.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:28.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955553+0000", 2024-09-14T09:08:28.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016809299999999999 2024-09-14T09:08:28.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.955677+0000", 2024-09-14T09:08:28.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012414899999999999 2024-09-14T09:08:28.723 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.723 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.723 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.723 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118153 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 445440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:28.723 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:08:28.724 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.868531324000003, 2024-09-14T09:08:28.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095358268999999996, 2024-09-14T09:08:28.724 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.724 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.724 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:28.724 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:28.724 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:28.724 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.724 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118153 2024-09-14T09:08:28.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.724 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:08:28.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:08:28.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838001+0000", 2024-09-14T09:08:28.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0999999999999999e-07 2024-09-14T09:08:28.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838003+0000", 2024-09-14T09:08:28.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2000000000000001e-06 2024-09-14T09:08:28.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838009+0000", 2024-09-14T09:08:28.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.105e-06 2024-09-14T09:08:28.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.726 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838015+0000", 2024-09-14T09:08:28.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.9030000000000001e-06 2024-09-14T09:08:28.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.726 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.726 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933049+0000", 2024-09-14T09:08:28.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095033513999999999 2024-09-14T09:08:28.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933113+0000", 2024-09-14T09:08:28.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3616999999999998e-05 2024-09-14T09:08:28.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933359+0000", 2024-09-14T09:08:28.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024652000000000002 2024-09-14T09:08:28.727 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.727 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.728 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.728 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118155 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 449536~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:28.728 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:08:28.728 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.868508798999997, 2024-09-14T09:08:28.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095668000000000003, 2024-09-14T09:08:28.728 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.728 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.728 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:28.728 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:28.728 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:28.729 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.729 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118155 2024-09-14T09:08:28.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.729 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:08:28.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838023+0000", 2024-09-14T09:08:28.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838024+0000", 2024-09-14T09:08:28.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0400000000000002e-07 2024-09-14T09:08:28.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838026+0000", 2024-09-14T09:08:28.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.508e-06 2024-09-14T09:08:28.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838033+0000", 2024-09-14T09:08:28.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.392e-06 2024-09-14T09:08:28.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838038+0000", 2024-09-14T09:08:28.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1050000000000003e-06 2024-09-14T09:08:28.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933400+0000", 2024-09-14T09:08:28.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095362279999999994 2024-09-14T09:08:28.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933451+0000", 2024-09-14T09:08:28.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0927000000000002e-05 2024-09-14T09:08:28.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.732 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933691+0000", 2024-09-14T09:08:28.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024038400000000001 2024-09-14T09:08:28.732 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.732 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.732 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.732 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118158 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 453632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:28.732 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:08:28.732 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.868487776999999, 2024-09-14T09:08:28.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096024231000000002, 2024-09-14T09:08:28.732 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.732 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.732 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:28.733 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:28.733 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:28.733 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.733 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118158 2024-09-14T09:08:28.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.733 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:08:28.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838044+0000", 2024-09-14T09:08:28.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838045+0000", 2024-09-14T09:08:28.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2199999999999999e-07 2024-09-14T09:08:28.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838047+0000", 2024-09-14T09:08:28.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1059999999999998e-06 2024-09-14T09:08:28.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.737 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838054+0000", 2024-09-14T09:08:28.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4089999999999997e-06 2024-09-14T09:08:28.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.737 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838059+0000", 2024-09-14T09:08:28.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8590000000000004e-06 2024-09-14T09:08:28.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933729+0000", 2024-09-14T09:08:28.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095670274999999999 2024-09-14T09:08:28.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.933794+0000", 2024-09-14T09:08:28.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4846999999999998e-05 2024-09-14T09:08:28.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934069+0000", 2024-09-14T09:08:28.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027431299999999999 2024-09-14T09:08:28.739 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.739 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.739 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.739 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118160 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 457728~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:28.739 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:08:28.739 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.868465657000002, 2024-09-14T09:08:28.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096394812999999996, 2024-09-14T09:08:28.739 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.739 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.739 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:28.739 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:28.740 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:28.740 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.740 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118160 2024-09-14T09:08:28.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.740 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:08:28.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838066+0000", 2024-09-14T09:08:28.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838067+0000", 2024-09-14T09:08:28.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9399999999999995e-07 2024-09-14T09:08:28.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838069+0000", 2024-09-14T09:08:28.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1579999999999999e-06 2024-09-14T09:08:28.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838077+0000", 2024-09-14T09:08:28.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6939999999999995e-06 2024-09-14T09:08:28.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838082+0000", 2024-09-14T09:08:28.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2290000000000002e-06 2024-09-14T09:08:28.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934106+0000", 2024-09-14T09:08:28.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096023874999999995 2024-09-14T09:08:28.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934158+0000", 2024-09-14T09:08:28.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2052000000000002e-05 2024-09-14T09:08:28.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.743 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.743 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934461+0000", 2024-09-14T09:08:28.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030331100000000001 2024-09-14T09:08:28.743 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.743 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.743 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.743 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118162 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 461824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:28.743 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:08:28.743 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.868443077000002, 2024-09-14T09:08:28.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096615294000000004, 2024-09-14T09:08:28.744 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.744 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.744 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:28.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:28.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:28.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.744 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118162 2024-09-14T09:08:28.744 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.744 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.744 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.744 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.744 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:08:28.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838089+0000", 2024-09-14T09:08:28.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838090+0000", 2024-09-14T09:08:28.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9699999999999996e-07 2024-09-14T09:08:28.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838092+0000", 2024-09-14T09:08:28.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.1119999999999999e-06 2024-09-14T09:08:28.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838098+0000", 2024-09-14T09:08:28.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.4359999999999999e-06 2024-09-14T09:08:28.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838103+0000", 2024-09-14T09:08:28.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.1200000000000001e-06 2024-09-14T09:08:28.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934504+0000", 2024-09-14T09:08:28.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096400885000000006 2024-09-14T09:08:28.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934596+0000", 2024-09-14T09:08:28.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1775999999999997e-05 2024-09-14T09:08:28.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934704+0000", 2024-09-14T09:08:28.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010846800000000001 2024-09-14T09:08:28.748 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.748 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.748 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.748 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118164 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 465920~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:28.749 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:08:28.749 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.868421816999998, 2024-09-14T09:08:28.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096951372999999993, 2024-09-14T09:08:28.749 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.749 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.749 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:28.749 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:28.749 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:28.749 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.749 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118164 2024-09-14T09:08:28.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.749 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:08:28.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838110+0000", 2024-09-14T09:08:28.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838111+0000", 2024-09-14T09:08:28.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0100000000000005e-07 2024-09-14T09:08:28.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838113+0000", 2024-09-14T09:08:28.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.0339999999999999e-06 2024-09-14T09:08:28.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838119+0000", 2024-09-14T09:08:28.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.409e-06 2024-09-14T09:08:28.751 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.751 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.751 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.751 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838124+0000", 2024-09-14T09:08:28.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9030000000000003e-06 2024-09-14T09:08:28.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934739+0000", 2024-09-14T09:08:28.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096615124999999996 2024-09-14T09:08:28.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.934789+0000", 2024-09-14T09:08:28.752 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9554e-05 2024-09-14T09:08:28.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.752 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.752 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935062+0000", 2024-09-14T09:08:28.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000272847 2024-09-14T09:08:28.753 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.753 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.753 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.753 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118166 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 470016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:28.753 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:08:28.753 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.868401063, 2024-09-14T09:08:28.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097306410999999995, 2024-09-14T09:08:28.753 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.753 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.753 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:28.753 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:28.754 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:28.754 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.754 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118166 2024-09-14T09:08:28.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.754 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:08:28.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:08:28.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838131+0000", 2024-09-14T09:08:28.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.03e-07 2024-09-14T09:08:28.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838133+0000", 2024-09-14T09:08:28.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9479999999999998e-06 2024-09-14T09:08:28.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838141+0000", 2024-09-14T09:08:28.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5739999999999998e-06 2024-09-14T09:08:28.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838146+0000", 2024-09-14T09:08:28.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3750000000000002e-06 2024-09-14T09:08:28.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935095+0000", 2024-09-14T09:08:28.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096948435999999999 2024-09-14T09:08:28.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935152+0000", 2024-09-14T09:08:28.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.694e-05 2024-09-14T09:08:28.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.758 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935437+0000", 2024-09-14T09:08:28.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028573500000000001 2024-09-14T09:08:28.758 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.758 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.758 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.758 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118168 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 474112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:28.758 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:08:28.758 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.868378841999998, 2024-09-14T09:08:28.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097658024999999996, 2024-09-14T09:08:28.758 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.758 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.758 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:28.759 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:28.759 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:28.759 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.759 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118168 2024-09-14T09:08:28.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.759 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:08:28.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838153+0000", 2024-09-14T09:08:28.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838154+0000", 2024-09-14T09:08:28.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4200000000000001e-07 2024-09-14T09:08:28.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838160+0000", 2024-09-14T09:08:28.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8510000000000004e-06 2024-09-14T09:08:28.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838168+0000", 2024-09-14T09:08:28.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9899999999999997e-06 2024-09-14T09:08:28.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838172+0000", 2024-09-14T09:08:28.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.887e-06 2024-09-14T09:08:28.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935473+0000", 2024-09-14T09:08:28.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097300132999999997 2024-09-14T09:08:28.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935523+0000", 2024-09-14T09:08:28.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0862000000000002e-05 2024-09-14T09:08:28.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935811+0000", 2024-09-14T09:08:28.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00028786000000000003 2024-09-14T09:08:28.762 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.762 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.762 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.763 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118170 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 478208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:28.763 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:08:28.763 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.868352031000001, 2024-09-14T09:08:28.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.098077529999999996, 2024-09-14T09:08:28.763 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.763 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.763 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:28.763 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:28.763 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:28.763 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.763 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118170 2024-09-14T09:08:28.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.763 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:08:28.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838180+0000", 2024-09-14T09:08:28.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.764 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.764 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842901+0000", 2024-09-14T09:08:28.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.004721416 2024-09-14T09:08:28.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842906+0000", 2024-09-14T09:08:28.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7199999999999997e-06 2024-09-14T09:08:28.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842917+0000", 2024-09-14T09:08:28.765 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0671e-05 2024-09-14T09:08:28.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.765 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.765 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.765 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842923+0000", 2024-09-14T09:08:28.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.088e-06 2024-09-14T09:08:28.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935857+0000", 2024-09-14T09:08:28.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.092933624000000006 2024-09-14T09:08:28.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.935958+0000", 2024-09-14T09:08:28.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000100933 2024-09-14T09:08:28.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.936258+0000", 2024-09-14T09:08:28.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00030007800000000001 2024-09-14T09:08:28.767 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.767 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.767 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.767 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118181 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 498688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:28.767 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:08:28.767 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.863545762000001, 2024-09-14T09:08:28.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095212643999999999, 2024-09-14T09:08:28.767 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.767 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.767 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:28.767 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:28.767 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:28.767 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.768 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118181 2024-09-14T09:08:28.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.768 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:08:28.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842986+0000", 2024-09-14T09:08:28.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842987+0000", 2024-09-14T09:08:28.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.29e-07 2024-09-14T09:08:28.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842988+0000", 2024-09-14T09:08:28.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.229e-06 2024-09-14T09:08:28.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842993+0000", 2024-09-14T09:08:28.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.792e-06 2024-09-14T09:08:28.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.842996+0000", 2024-09-14T09:08:28.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3349999999999998e-06 2024-09-14T09:08:28.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.937807+0000", 2024-09-14T09:08:28.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.094811585000000004 2024-09-14T09:08:28.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.937899+0000", 2024-09-14T09:08:28.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1719999999999996e-05 2024-09-14T09:08:28.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938199+0000", 2024-09-14T09:08:28.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000299754 2024-09-14T09:08:28.771 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.771 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.771 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.771 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118182 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 502784~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:28.771 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:08:28.772 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.863532585000002, 2024-09-14T09:08:28.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095557953000000001, 2024-09-14T09:08:28.772 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.772 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.772 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:28.772 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:28.772 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:28.772 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.772 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118182 2024-09-14T09:08:28.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.772 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:08:28.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:08:28.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843000+0000", 2024-09-14T09:08:28.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2999999999999999e-07 2024-09-14T09:08:28.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843001+0000", 2024-09-14T09:08:28.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.652e-06 2024-09-14T09:08:28.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843005+0000", 2024-09-14T09:08:28.774 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2780000000000002e-06 2024-09-14T09:08:28.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.774 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.774 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843007+0000", 2024-09-14T09:08:28.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.553e-06 2024-09-14T09:08:28.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938240+0000", 2024-09-14T09:08:28.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095232864 2024-09-14T09:08:28.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.775 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938295+0000", 2024-09-14T09:08:28.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5095e-05 2024-09-14T09:08:28.775 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.775 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.775 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938557+0000", 2024-09-14T09:08:28.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026228100000000001 2024-09-14T09:08:28.776 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.776 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.776 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.776 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118184 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 506880~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:28.776 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:08:28.776 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.863521384999999, 2024-09-14T09:08:28.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095950246000000003, 2024-09-14T09:08:28.776 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.776 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.776 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:28.777 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:28.777 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:28.777 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.777 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118184 2024-09-14T09:08:28.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.777 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:08:28.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:08:28.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843011+0000", 2024-09-14T09:08:28.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2399999999999999e-07 2024-09-14T09:08:28.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843012+0000", 2024-09-14T09:08:28.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.269e-06 2024-09-14T09:08:28.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843015+0000", 2024-09-14T09:08:28.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0469999999999998e-06 2024-09-14T09:08:28.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843018+0000", 2024-09-14T09:08:28.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.044e-06 2024-09-14T09:08:28.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938597+0000", 2024-09-14T09:08:28.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095578688999999994 2024-09-14T09:08:28.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938687+0000", 2024-09-14T09:08:28.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0284000000000005e-05 2024-09-14T09:08:28.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.938961+0000", 2024-09-14T09:08:28.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027368899999999997 2024-09-14T09:08:28.780 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.780 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.780 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.780 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118186 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 510976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:28.781 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:08:28.781 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.863510425000001, 2024-09-14T09:08:28.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096286497999999998, 2024-09-14T09:08:28.781 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.781 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.781 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:28.781 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:28.781 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:28.781 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.781 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118186 2024-09-14T09:08:28.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.781 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:08:28.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:08:28.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843022+0000", 2024-09-14T09:08:28.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.35e-07 2024-09-14T09:08:28.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843023+0000", 2024-09-14T09:08:28.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.195e-06 2024-09-14T09:08:28.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843026+0000", 2024-09-14T09:08:28.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.055e-06 2024-09-14T09:08:28.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843029+0000", 2024-09-14T09:08:28.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6539999999999999e-06 2024-09-14T09:08:28.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939000+0000", 2024-09-14T09:08:28.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.095971476 2024-09-14T09:08:28.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939049+0000", 2024-09-14T09:08:28.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8986999999999997e-05 2024-09-14T09:08:28.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.784 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.784 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.784 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939308+0000", 2024-09-14T09:08:28.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000258896 2024-09-14T09:08:28.785 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.785 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.785 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.785 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118188 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 515072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:28.785 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:08:28.785 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.863500035999998, 2024-09-14T09:08:28.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096722717999999999, 2024-09-14T09:08:28.785 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.785 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.785 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:28.785 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:28.785 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:28.786 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.786 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118188 2024-09-14T09:08:28.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.786 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:08:28.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:08:28.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843032+0000", 2024-09-14T09:08:28.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2999999999999999e-07 2024-09-14T09:08:28.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843034+0000", 2024-09-14T09:08:28.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.226e-06 2024-09-14T09:08:28.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843037+0000", 2024-09-14T09:08:28.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0989999999999999e-06 2024-09-14T09:08:28.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843039+0000", 2024-09-14T09:08:28.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.666e-06 2024-09-14T09:08:28.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939351+0000", 2024-09-14T09:08:28.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096311789999999994 2024-09-14T09:08:28.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939440+0000", 2024-09-14T09:08:28.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8843000000000003e-05 2024-09-14T09:08:28.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939755+0000", 2024-09-14T09:08:28.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031486399999999999 2024-09-14T09:08:28.789 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.789 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.789 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.789 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118191 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 519168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:28.789 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:08:28.789 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.863489342999998, 2024-09-14T09:08:28.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097080607999999999, 2024-09-14T09:08:28.789 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.789 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.790 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:28.790 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:28.790 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:28.790 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.790 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118191 2024-09-14T09:08:28.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.790 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:08:28.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:08:28.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843043+0000", 2024-09-14T09:08:28.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2100000000000001e-07 2024-09-14T09:08:28.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843044+0000", 2024-09-14T09:08:28.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2249999999999999e-06 2024-09-14T09:08:28.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843047+0000", 2024-09-14T09:08:28.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0240000000000002e-06 2024-09-14T09:08:28.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843050+0000", 2024-09-14T09:08:28.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.508e-06 2024-09-14T09:08:28.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939791+0000", 2024-09-14T09:08:28.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096741399000000006 2024-09-14T09:08:28.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.939849+0000", 2024-09-14T09:08:28.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7912999999999999e-05 2024-09-14T09:08:28.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940123+0000", 2024-09-14T09:08:28.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027431800000000001 2024-09-14T09:08:28.793 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.793 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.793 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.793 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118192 3.1f 3:f9bdbc42:::rbd_data.115e500fbcd9.0000000000000005:27 [read 523264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:28.793 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:08:28.793 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.863478282999999, 2024-09-14T09:08:28.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.097587224, 2024-09-14T09:08:28.794 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.794 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.794 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:28.794 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:28.794 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:28.794 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.794 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118192 2024-09-14T09:08:28.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.794 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.794 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.794 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:08:28.794 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.843054+0000", 2024-09-14T09:08:28.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844132+0000", 2024-09-14T09:08:28.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0010777810000000001 2024-09-14T09:08:28.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.795 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.795 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844140+0000", 2024-09-14T09:08:28.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4470000000000006e-06 2024-09-14T09:08:28.795 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.795 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844155+0000", 2024-09-14T09:08:28.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4924e-05 2024-09-14T09:08:28.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.844163+0000", 2024-09-14T09:08:28.796 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9980000000000003e-06 2024-09-14T09:08:28.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.796 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940199+0000", 2024-09-14T09:08:28.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.096036047999999999 2024-09-14T09:08:28.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940282+0000", 2024-09-14T09:08:28.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3073999999999998e-05 2024-09-14T09:08:28.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940641+0000", 2024-09-14T09:08:28.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000358952 2024-09-14T09:08:28.797 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.798 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.798 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.798 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:118197 3.17 3:ec5ec63a:::rbd_data.115e96ceebb2.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,truncate 3718221] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:28.798 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:08:28.798 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 56.858698867999998, 2024-09-14T09:08:28.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.141131069, 2024-09-14T09:08:28.798 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:28.798 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:28.798 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:28.798 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:28.798 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:28.798 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:28.798 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 118197 2024-09-14T09:08:28.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.799 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:28.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:28.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:08:28.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:28.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847833+0000", 2024-09-14T09:08:28.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:28.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:28.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.847834+0000", 2024-09-14T09:08:28.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2500000000000004e-07 2024-09-14T09:08:28.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:28.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849227+0000", 2024-09-14T09:08:28.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0013932149999999999 2024-09-14T09:08:28.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:28.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849246+0000", 2024-09-14T09:08:28.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8958e-05 2024-09-14T09:08:28.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:28.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.849254+0000", 2024-09-14T09:08:28.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2360000000000004e-06 2024-09-14T09:08:28.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:28.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940154+0000", 2024-09-14T09:08:28.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.090899468999999997 2024-09-14T09:08:28.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:28.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940312+0000", 2024-09-14T09:08:28.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000158326 2024-09-14T09:08:28.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 5", 2024-09-14T09:08:28.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.940728+0000", 2024-09-14T09:08:28.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00041583100000000002 2024-09-14T09:08:28.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:28.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.962731+0000", 2024-09-14T09:08:28.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.022002608999999999 2024-09-14T09:08:28.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:28.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988881+0000", 2024-09-14T09:08:28.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.026150620999999999 2024-09-14T09:08:28.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:28.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988919+0000", 2024-09-14T09:08:28.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8108999999999997e-05 2024-09-14T09:08:28.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:28.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.988964+0000", 2024-09-14T09:08:28.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4870000000000002e-05 2024-09-14T09:08:28.803 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.803 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.803 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.803 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.803 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.803 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:28.805 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:08:28.806 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:08:28.806 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:28.806 INFO:teuthology.orchestra.run.smithi176.stdout: "epoch": 176, 2024-09-14T09:08:28.806 INFO:teuthology.orchestra.run.smithi176.stdout: "fsid": "c635667d-994e-454f-bcea-51b45637b2a2", 2024-09-14T09:08:28.806 INFO:teuthology.orchestra.run.smithi176.stdout: "created": "2024-09-14T09:04:57.077293+0000", 2024-09-14T09:08:28.806 INFO:teuthology.orchestra.run.smithi176.stdout: "modified": "2024-09-14T09:08:28.421038+0000", 2024-09-14T09:08:28.806 INFO:teuthology.orchestra.run.smithi176.stdout: "last_up_change": "2024-09-14T09:07:46.980109+0000", 2024-09-14T09:08:28.806 INFO:teuthology.orchestra.run.smithi176.stdout: "last_in_change": "2024-09-14T09:07:21.060249+0000", 2024-09-14T09:08:28.806 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T09:08:28.806 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_num": 5806080, 2024-09-14T09:08:28.806 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_set": [ 2024-09-14T09:08:28.807 INFO:teuthology.orchestra.run.smithi176.stdout: "nodeep-scrub", 2024-09-14T09:08:28.807 INFO:teuthology.orchestra.run.smithi176.stdout: "noscrub", 2024-09-14T09:08:28.807 INFO:teuthology.orchestra.run.smithi176.stdout: "pglog_hardlimit", 2024-09-14T09:08:28.807 INFO:teuthology.orchestra.run.smithi176.stdout: "purged_snapdirs", 2024-09-14T09:08:28.807 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_deletes", 2024-09-14T09:08:28.807 INFO:teuthology.orchestra.run.smithi176.stdout: "sortbitwise" 2024-09-14T09:08:28.807 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:08:28.807 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_version": 5, 2024-09-14T09:08:28.807 INFO:teuthology.orchestra.run.smithi176.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T09:08:28.807 INFO:teuthology.orchestra.run.smithi176.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T09:08:28.807 INFO:teuthology.orchestra.run.smithi176.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T09:08:28.807 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_snapshot": "", 2024-09-14T09:08:28.807 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_max": 3, 2024-09-14T09:08:28.807 INFO:teuthology.orchestra.run.smithi176.stdout: "max_osd": 8, 2024-09-14T09:08:28.807 INFO:teuthology.orchestra.run.smithi176.stdout: "require_min_compat_client": "luminous", 2024-09-14T09:08:28.808 INFO:teuthology.orchestra.run.smithi176.stdout: "min_compat_client": "luminous", 2024-09-14T09:08:28.808 INFO:teuthology.orchestra.run.smithi176.stdout: "require_osd_release": "squid", 2024-09-14T09:08:28.808 INFO:teuthology.orchestra.run.smithi176.stdout: "allow_crimson": false, 2024-09-14T09:08:28.808 INFO:teuthology.orchestra.run.smithi176.stdout: "pools": [ 2024-09-14T09:08:28.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.808 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 1, 2024-09-14T09:08:28.808 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_name": ".mgr", 2024-09-14T09:08:28.808 INFO:teuthology.orchestra.run.smithi176.stdout: "create_time": "2024-09-14T09:05:11.997673+0000", 2024-09-14T09:08:28.808 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": 1, 2024-09-14T09:08:28.808 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_names": "hashpspool", 2024-09-14T09:08:28.808 INFO:teuthology.orchestra.run.smithi176.stdout: "type": 1, 2024-09-14T09:08:28.808 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 2, 2024-09-14T09:08:28.808 INFO:teuthology.orchestra.run.smithi176.stdout: "min_size": 1, 2024-09-14T09:08:28.808 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_rule": 0, 2024-09-14T09:08:28.808 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_count": 0, 2024-09-14T09:08:28.809 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_target": 0, 2024-09-14T09:08:28.809 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T09:08:28.809 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T09:08:28.809 INFO:teuthology.orchestra.run.smithi176.stdout: "is_stretch_pool": false, 2024-09-14T09:08:28.809 INFO:teuthology.orchestra.run.smithi176.stdout: "object_hash": 2, 2024-09-14T09:08:28.809 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_autoscale_mode": "off", 2024-09-14T09:08:28.809 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num": 1, 2024-09-14T09:08:28.809 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num": 1, 2024-09-14T09:08:28.809 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num_target": 1, 2024-09-14T09:08:28.809 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_target": 1, 2024-09-14T09:08:28.809 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_pending": 1, 2024-09-14T09:08:28.809 INFO:teuthology.orchestra.run.smithi176.stdout: "last_pg_merge_meta": { 2024-09-14T09:08:28.809 INFO:teuthology.orchestra.run.smithi176.stdout: "source_pgid": "0.0", 2024-09-14T09:08:28.809 INFO:teuthology.orchestra.run.smithi176.stdout: "ready_epoch": 0, 2024-09-14T09:08:28.809 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_started": 0, 2024-09-14T09:08:28.810 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_clean": 0, 2024-09-14T09:08:28.810 INFO:teuthology.orchestra.run.smithi176.stdout: "source_version": "0'0", 2024-09-14T09:08:28.810 INFO:teuthology.orchestra.run.smithi176.stdout: "target_version": "0'0" 2024-09-14T09:08:28.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.810 INFO:teuthology.orchestra.run.smithi176.stdout: "last_change": "18", 2024-09-14T09:08:28.810 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend": "0", 2024-09-14T09:08:28.810 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T09:08:28.810 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T09:08:28.810 INFO:teuthology.orchestra.run.smithi176.stdout: "auid": 0, 2024-09-14T09:08:28.810 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_mode": "selfmanaged", 2024-09-14T09:08:28.810 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_seq": 0, 2024-09-14T09:08:28.810 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_epoch": 0, 2024-09-14T09:08:28.810 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_snaps": [], 2024-09-14T09:08:28.810 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps": "[]", 2024-09-14T09:08:28.810 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_bytes": 0, 2024-09-14T09:08:28.811 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_objects": 0, 2024-09-14T09:08:28.811 INFO:teuthology.orchestra.run.smithi176.stdout: "tiers": [], 2024-09-14T09:08:28.811 INFO:teuthology.orchestra.run.smithi176.stdout: "tier_of": -1, 2024-09-14T09:08:28.811 INFO:teuthology.orchestra.run.smithi176.stdout: "read_tier": -1, 2024-09-14T09:08:28.811 INFO:teuthology.orchestra.run.smithi176.stdout: "write_tier": -1, 2024-09-14T09:08:28.811 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_mode": "none", 2024-09-14T09:08:28.811 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_bytes": 0, 2024-09-14T09:08:28.811 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_objects": 0, 2024-09-14T09:08:28.811 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T09:08:28.811 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T09:08:28.811 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T09:08:28.811 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_flush_age": 0, 2024-09-14T09:08:28.811 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_evict_age": 0, 2024-09-14T09:08:28.811 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profile": "", 2024-09-14T09:08:28.811 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_params": { 2024-09-14T09:08:28.811 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "none" 2024-09-14T09:08:28.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.812 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_period": 0, 2024-09-14T09:08:28.812 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_count": 0, 2024-09-14T09:08:28.812 INFO:teuthology.orchestra.run.smithi176.stdout: "use_gmt_hitset": true, 2024-09-14T09:08:28.812 INFO:teuthology.orchestra.run.smithi176.stdout: "min_read_recency_for_promote": 0, 2024-09-14T09:08:28.812 INFO:teuthology.orchestra.run.smithi176.stdout: "min_write_recency_for_promote": 0, 2024-09-14T09:08:28.812 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T09:08:28.812 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_search_last_n": 0, 2024-09-14T09:08:28.812 INFO:teuthology.orchestra.run.smithi176.stdout: "grade_table": [], 2024-09-14T09:08:28.812 INFO:teuthology.orchestra.run.smithi176.stdout: "stripe_width": 0, 2024-09-14T09:08:28.812 INFO:teuthology.orchestra.run.smithi176.stdout: "expected_num_objects": 0, 2024-09-14T09:08:28.812 INFO:teuthology.orchestra.run.smithi176.stdout: "fast_read": false, 2024-09-14T09:08:28.812 INFO:teuthology.orchestra.run.smithi176.stdout: "options": { 2024-09-14T09:08:28.812 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_max": 32, 2024-09-14T09:08:28.813 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_min": 1 2024-09-14T09:08:28.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.813 INFO:teuthology.orchestra.run.smithi176.stdout: "application_metadata": { 2024-09-14T09:08:28.813 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": {} 2024-09-14T09:08:28.813 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.813 INFO:teuthology.orchestra.run.smithi176.stdout: "read_balance": { 2024-09-14T09:08:28.813 INFO:teuthology.orchestra.run.smithi176.stdout: "score_type": "Fair distribution", 2024-09-14T09:08:28.813 INFO:teuthology.orchestra.run.smithi176.stdout: "score_acting": 6.9000000953674316, 2024-09-14T09:08:28.813 INFO:teuthology.orchestra.run.smithi176.stdout: "score_stable": 6.9000000953674316, 2024-09-14T09:08:28.813 INFO:teuthology.orchestra.run.smithi176.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T09:08:28.813 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_acting": 2, 2024-09-14T09:08:28.813 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_stable": 2, 2024-09-14T09:08:28.813 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T09:08:28.814 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity": 1, 2024-09-14T09:08:28.814 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T09:08:28.814 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.814 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 2, 2024-09-14T09:08:28.814 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_name": "rbd", 2024-09-14T09:08:28.814 INFO:teuthology.orchestra.run.smithi176.stdout: "create_time": "2024-09-14T09:05:15.352551+0000", 2024-09-14T09:08:28.814 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": 8193, 2024-09-14T09:08:28.814 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T09:08:28.814 INFO:teuthology.orchestra.run.smithi176.stdout: "type": 1, 2024-09-14T09:08:28.814 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 2, 2024-09-14T09:08:28.814 INFO:teuthology.orchestra.run.smithi176.stdout: "min_size": 1, 2024-09-14T09:08:28.814 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_rule": 0, 2024-09-14T09:08:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_count": 0, 2024-09-14T09:08:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_target": 0, 2024-09-14T09:08:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T09:08:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T09:08:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "is_stretch_pool": false, 2024-09-14T09:08:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "object_hash": 2, 2024-09-14T09:08:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_autoscale_mode": "off", 2024-09-14T09:08:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num": 8, 2024-09-14T09:08:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num": 8, 2024-09-14T09:08:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num_target": 8, 2024-09-14T09:08:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_target": 8, 2024-09-14T09:08:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_pending": 8, 2024-09-14T09:08:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "last_pg_merge_meta": { 2024-09-14T09:08:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "source_pgid": "0.0", 2024-09-14T09:08:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "ready_epoch": 0, 2024-09-14T09:08:28.815 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_started": 0, 2024-09-14T09:08:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_clean": 0, 2024-09-14T09:08:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "source_version": "0'0", 2024-09-14T09:08:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "target_version": "0'0" 2024-09-14T09:08:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "last_change": "22", 2024-09-14T09:08:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend": "0", 2024-09-14T09:08:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T09:08:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T09:08:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "auid": 0, 2024-09-14T09:08:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_mode": "selfmanaged", 2024-09-14T09:08:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_seq": 2, 2024-09-14T09:08:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_epoch": 22, 2024-09-14T09:08:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_snaps": [], 2024-09-14T09:08:28.816 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps": "[]", 2024-09-14T09:08:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_bytes": 0, 2024-09-14T09:08:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_objects": 0, 2024-09-14T09:08:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "tiers": [], 2024-09-14T09:08:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "tier_of": -1, 2024-09-14T09:08:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "read_tier": -1, 2024-09-14T09:08:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "write_tier": -1, 2024-09-14T09:08:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_mode": "none", 2024-09-14T09:08:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_bytes": 0, 2024-09-14T09:08:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_objects": 0, 2024-09-14T09:08:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T09:08:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T09:08:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T09:08:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_flush_age": 0, 2024-09-14T09:08:28.817 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_evict_age": 0, 2024-09-14T09:08:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profile": "", 2024-09-14T09:08:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_params": { 2024-09-14T09:08:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "none" 2024-09-14T09:08:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_period": 0, 2024-09-14T09:08:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_count": 0, 2024-09-14T09:08:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "use_gmt_hitset": true, 2024-09-14T09:08:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "min_read_recency_for_promote": 0, 2024-09-14T09:08:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "min_write_recency_for_promote": 0, 2024-09-14T09:08:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T09:08:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_search_last_n": 0, 2024-09-14T09:08:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "grade_table": [], 2024-09-14T09:08:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "stripe_width": 0, 2024-09-14T09:08:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "expected_num_objects": 0, 2024-09-14T09:08:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "fast_read": false, 2024-09-14T09:08:28.818 INFO:teuthology.orchestra.run.smithi176.stdout: "options": {}, 2024-09-14T09:08:28.819 INFO:teuthology.orchestra.run.smithi176.stdout: "application_metadata": { 2024-09-14T09:08:28.819 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd": {} 2024-09-14T09:08:28.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.819 INFO:teuthology.orchestra.run.smithi176.stdout: "read_balance": { 2024-09-14T09:08:28.819 INFO:teuthology.orchestra.run.smithi176.stdout: "score_type": "Fair distribution", 2024-09-14T09:08:28.819 INFO:teuthology.orchestra.run.smithi176.stdout: "score_acting": 1.75, 2024-09-14T09:08:28.819 INFO:teuthology.orchestra.run.smithi176.stdout: "score_stable": 1.75, 2024-09-14T09:08:28.819 INFO:teuthology.orchestra.run.smithi176.stdout: "optimal_score": 1, 2024-09-14T09:08:28.819 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_acting": 1.75, 2024-09-14T09:08:28.819 INFO:teuthology.orchestra.run.smithi176.stdout: "raw_score_stable": 1.75, 2024-09-14T09:08:28.819 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T09:08:28.819 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity": 1, 2024-09-14T09:08:28.819 INFO:teuthology.orchestra.run.smithi176.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T09:08:28.819 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.819 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.819 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.820 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 3, 2024-09-14T09:08:28.820 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_name": "pool_client.0", 2024-09-14T09:08:28.820 INFO:teuthology.orchestra.run.smithi176.stdout: "create_time": "2024-09-14T09:05:25.840963+0000", 2024-09-14T09:08:28.820 INFO:teuthology.orchestra.run.smithi176.stdout: "flags": 8193, 2024-09-14T09:08:28.820 INFO:teuthology.orchestra.run.smithi176.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T09:08:28.820 INFO:teuthology.orchestra.run.smithi176.stdout: "type": 1, 2024-09-14T09:08:28.820 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 2, 2024-09-14T09:08:28.820 INFO:teuthology.orchestra.run.smithi176.stdout: "min_size": 1, 2024-09-14T09:08:28.820 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_rule": 0, 2024-09-14T09:08:28.820 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_count": 0, 2024-09-14T09:08:28.820 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_target": 0, 2024-09-14T09:08:28.820 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T09:08:28.820 INFO:teuthology.orchestra.run.smithi176.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T09:08:28.820 INFO:teuthology.orchestra.run.smithi176.stdout: "is_stretch_pool": false, 2024-09-14T09:08:28.820 INFO:teuthology.orchestra.run.smithi176.stdout: "object_hash": 2, 2024-09-14T09:08:28.821 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_autoscale_mode": "off", 2024-09-14T09:08:28.821 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num": 32, 2024-09-14T09:08:28.821 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num": 32, 2024-09-14T09:08:28.821 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_placement_num_target": 32, 2024-09-14T09:08:28.821 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_target": 32, 2024-09-14T09:08:28.821 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_num_pending": 32, 2024-09-14T09:08:28.821 INFO:teuthology.orchestra.run.smithi176.stdout: "last_pg_merge_meta": { 2024-09-14T09:08:28.821 INFO:teuthology.orchestra.run.smithi176.stdout: "source_pgid": "0.0", 2024-09-14T09:08:28.821 INFO:teuthology.orchestra.run.smithi176.stdout: "ready_epoch": 0, 2024-09-14T09:08:28.821 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_started": 0, 2024-09-14T09:08:28.821 INFO:teuthology.orchestra.run.smithi176.stdout: "last_epoch_clean": 0, 2024-09-14T09:08:28.821 INFO:teuthology.orchestra.run.smithi176.stdout: "source_version": "0'0", 2024-09-14T09:08:28.821 INFO:teuthology.orchestra.run.smithi176.stdout: "target_version": "0'0" 2024-09-14T09:08:28.821 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.822 INFO:teuthology.orchestra.run.smithi176.stdout: "last_change": "176", 2024-09-14T09:08:28.822 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend": "0", 2024-09-14T09:08:28.822 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T09:08:28.822 INFO:teuthology.orchestra.run.smithi176.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T09:08:28.822 INFO:teuthology.orchestra.run.smithi176.stdout: "auid": 0, 2024-09-14T09:08:28.822 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_mode": "selfmanaged", 2024-09-14T09:08:28.822 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_seq": 46, 2024-09-14T09:08:28.822 INFO:teuthology.orchestra.run.smithi176.stdout: "snap_epoch": 176, 2024-09-14T09:08:28.822 INFO:teuthology.orchestra.run.smithi176.stdout: "pool_snaps": [], 2024-09-14T09:08:28.822 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps": "[]", 2024-09-14T09:08:28.822 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_bytes": 0, 2024-09-14T09:08:28.822 INFO:teuthology.orchestra.run.smithi176.stdout: "quota_max_objects": 0, 2024-09-14T09:08:28.822 INFO:teuthology.orchestra.run.smithi176.stdout: "tiers": [], 2024-09-14T09:08:28.822 INFO:teuthology.orchestra.run.smithi176.stdout: "tier_of": -1, 2024-09-14T09:08:28.822 INFO:teuthology.orchestra.run.smithi176.stdout: "read_tier": -1, 2024-09-14T09:08:28.822 INFO:teuthology.orchestra.run.smithi176.stdout: "write_tier": -1, 2024-09-14T09:08:28.823 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_mode": "none", 2024-09-14T09:08:28.823 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_bytes": 0, 2024-09-14T09:08:28.823 INFO:teuthology.orchestra.run.smithi176.stdout: "target_max_objects": 0, 2024-09-14T09:08:28.823 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T09:08:28.823 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T09:08:28.823 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T09:08:28.823 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_flush_age": 0, 2024-09-14T09:08:28.823 INFO:teuthology.orchestra.run.smithi176.stdout: "cache_min_evict_age": 0, 2024-09-14T09:08:28.823 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profile": "", 2024-09-14T09:08:28.823 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_params": { 2024-09-14T09:08:28.823 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "none" 2024-09-14T09:08:28.823 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.823 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_period": 0, 2024-09-14T09:08:28.823 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_count": 0, 2024-09-14T09:08:28.823 INFO:teuthology.orchestra.run.smithi176.stdout: "use_gmt_hitset": true, 2024-09-14T09:08:28.824 INFO:teuthology.orchestra.run.smithi176.stdout: "min_read_recency_for_promote": 0, 2024-09-14T09:08:28.824 INFO:teuthology.orchestra.run.smithi176.stdout: "min_write_recency_for_promote": 0, 2024-09-14T09:08:28.824 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T09:08:28.824 INFO:teuthology.orchestra.run.smithi176.stdout: "hit_set_search_last_n": 0, 2024-09-14T09:08:28.824 INFO:teuthology.orchestra.run.smithi176.stdout: "grade_table": [], 2024-09-14T09:08:28.824 INFO:teuthology.orchestra.run.smithi176.stdout: "stripe_width": 0, 2024-09-14T09:08:28.824 INFO:teuthology.orchestra.run.smithi176.stdout: "expected_num_objects": 0, 2024-09-14T09:08:28.824 INFO:teuthology.orchestra.run.smithi176.stdout: "fast_read": false, 2024-09-14T09:08:28.824 INFO:teuthology.orchestra.run.smithi176.stdout: "options": {}, 2024-09-14T09:08:28.824 INFO:teuthology.orchestra.run.smithi176.stdout: "application_metadata": { 2024-09-14T09:08:28.824 INFO:teuthology.orchestra.run.smithi176.stdout: "rbd": {} 2024-09-14T09:08:28.824 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.824 INFO:teuthology.orchestra.run.smithi176.stdout: "read_balance": { 2024-09-14T09:08:28.824 INFO:teuthology.orchestra.run.smithi176.stdout: "error_message": "pool 3 has some PGs where all OSDs are with primary_affinity 0 (at least pg 3.1a), read balance score may not be reliable", 2024-09-14T09:08:28.825 INFO:teuthology.orchestra.run.smithi176.stdout: "score_acting": 0, 2024-09-14T09:08:28.825 INFO:teuthology.orchestra.run.smithi176.stdout: "score_stable": 0 2024-09-14T09:08:28.825 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.825 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.825 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:08:28.825 INFO:teuthology.orchestra.run.smithi176.stdout: "osds": [ 2024-09-14T09:08:28.825 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.825 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 0, 2024-09-14T09:08:28.825 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "4ceedb4f-6398-467f-9562-ccf27200860b", 2024-09-14T09:08:28.825 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:08:28.825 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:08:28.825 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:08:28.825 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:08:28.825 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:08:28.825 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:08:28.825 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:08:28.826 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 171, 2024-09-14T09:08:28.826 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:08:28.826 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:08:28.826 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:08:28.826 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.826 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.826 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6824", 2024-09-14T09:08:28.826 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:08:28.826 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.826 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.826 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.826 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6825", 2024-09-14T09:08:28.826 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:08:28.826 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.826 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.827 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:08:28.827 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.827 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.827 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6826", 2024-09-14T09:08:28.827 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:08:28.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.827 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.827 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.827 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6827", 2024-09-14T09:08:28.827 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:08:28.827 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.827 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.827 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.827 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:08:28.828 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.828 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.828 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6830", 2024-09-14T09:08:28.828 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:08:28.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.828 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.828 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.828 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6831", 2024-09-14T09:08:28.828 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:08:28.828 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.828 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.828 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.828 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:08:28.828 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.829 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.829 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6828", 2024-09-14T09:08:28.829 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:08:28.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.829 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.829 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.829 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6829", 2024-09-14T09:08:28.829 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3384710819 2024-09-14T09:08:28.829 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.829 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.829 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.829 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6825/3384710819", 2024-09-14T09:08:28.829 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6827/3384710819", 2024-09-14T09:08:28.830 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6831/3384710819", 2024-09-14T09:08:28.830 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6829/3384710819", 2024-09-14T09:08:28.830 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:08:28.830 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:08:28.830 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:08:28.830 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.830 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.830 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.830 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 1, 2024-09-14T09:08:28.830 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "604e2f62-7c13-46be-a97f-029b28c44a89", 2024-09-14T09:08:28.830 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:08:28.830 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:08:28.830 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:08:28.830 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 0, 2024-09-14T09:08:28.830 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:08:28.831 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:08:28.831 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:08:28.831 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 135, 2024-09-14T09:08:28.831 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:08:28.831 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:08:28.831 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:08:28.831 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.831 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.831 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6800", 2024-09-14T09:08:28.831 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:08:28.831 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.831 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.831 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.831 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6801", 2024-09-14T09:08:28.831 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:08:28.832 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.832 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.832 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:08:28.832 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.832 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.832 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6802", 2024-09-14T09:08:28.832 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:08:28.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.832 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.832 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.832 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6803", 2024-09-14T09:08:28.832 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:08:28.832 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.833 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.833 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:08:28.833 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.833 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.833 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6806", 2024-09-14T09:08:28.833 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:08:28.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.833 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.833 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.833 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6807", 2024-09-14T09:08:28.833 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:08:28.834 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.834 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.834 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:08:28.834 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.834 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.834 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6804", 2024-09-14T09:08:28.834 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:08:28.834 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.834 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.834 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.834 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6805", 2024-09-14T09:08:28.835 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 612012679 2024-09-14T09:08:28.835 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.835 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.835 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6801/612012679", 2024-09-14T09:08:28.835 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6803/612012679", 2024-09-14T09:08:28.835 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6807/612012679", 2024-09-14T09:08:28.835 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6805/612012679", 2024-09-14T09:08:28.835 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:08:28.835 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:08:28.835 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:08:28.835 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.835 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.835 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.835 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 2, 2024-09-14T09:08:28.836 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "1f65589e-8b0a-4033-94c0-4ab66531db2b", 2024-09-14T09:08:28.836 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:08:28.836 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:08:28.836 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:08:28.836 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:08:28.836 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:08:28.836 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:08:28.836 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:08:28.836 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 171, 2024-09-14T09:08:28.836 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:08:28.836 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:08:28.836 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:08:28.837 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.837 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.837 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6808", 2024-09-14T09:08:28.837 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:08:28.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.837 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.837 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.837 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6809", 2024-09-14T09:08:28.837 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:08:28.837 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.837 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.837 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.838 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:08:28.838 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.838 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.838 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6810", 2024-09-14T09:08:28.838 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:08:28.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.838 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.838 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.838 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6811", 2024-09-14T09:08:28.838 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:08:28.838 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.838 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.838 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.838 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:08:28.839 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.839 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.839 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6814", 2024-09-14T09:08:28.839 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:08:28.839 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.839 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.839 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.839 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6815", 2024-09-14T09:08:28.839 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:08:28.839 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.839 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.840 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:08:28.840 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.840 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.840 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6812", 2024-09-14T09:08:28.840 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:08:28.840 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.840 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.840 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.840 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6813", 2024-09-14T09:08:28.840 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 4167379279 2024-09-14T09:08:28.840 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.840 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.841 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6809/4167379279", 2024-09-14T09:08:28.841 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6811/4167379279", 2024-09-14T09:08:28.841 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6815/4167379279", 2024-09-14T09:08:28.841 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6813/4167379279", 2024-09-14T09:08:28.841 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:08:28.841 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:08:28.841 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:08:28.841 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.841 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.841 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.841 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 3, 2024-09-14T09:08:28.841 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "209d4f65-79f9-4a7d-9d23-832a4209507f", 2024-09-14T09:08:28.841 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:08:28.842 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:08:28.842 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:08:28.842 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:08:28.842 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:08:28.842 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:08:28.842 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:08:28.842 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 117, 2024-09-14T09:08:28.842 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:08:28.842 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:08:28.842 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:08:28.842 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.842 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.842 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.842 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6816", 2024-09-14T09:08:28.842 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:08:28.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.843 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.843 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6817", 2024-09-14T09:08:28.843 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:08:28.843 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.843 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.843 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:08:28.843 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.843 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.843 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.843 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6818", 2024-09-14T09:08:28.843 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:08:28.843 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.844 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.844 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6819", 2024-09-14T09:08:28.844 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:08:28.844 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.844 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.844 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:08:28.844 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.844 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.844 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.844 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6822", 2024-09-14T09:08:28.844 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:08:28.844 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.845 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.845 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6823", 2024-09-14T09:08:28.845 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:08:28.845 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.845 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.845 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.845 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:08:28.845 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.845 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.845 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.845 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6820", 2024-09-14T09:08:28.845 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:08:28.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.846 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.846 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.846 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.176:6821", 2024-09-14T09:08:28.846 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1658209015 2024-09-14T09:08:28.846 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.846 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.846 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.846 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.176:6817/1658209015", 2024-09-14T09:08:28.846 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.176:6819/1658209015", 2024-09-14T09:08:28.846 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.176:6823/1658209015", 2024-09-14T09:08:28.846 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.176:6821/1658209015", 2024-09-14T09:08:28.846 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:08:28.846 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:08:28.846 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:08:28.847 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.847 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.847 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.847 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 4, 2024-09-14T09:08:28.847 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "8f9a8c9b-282e-4523-9e62-5824f4aba401", 2024-09-14T09:08:28.847 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 0, 2024-09-14T09:08:28.847 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:08:28.847 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:08:28.847 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:08:28.847 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:08:28.847 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:08:28.847 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:08:28.847 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 116, 2024-09-14T09:08:28.848 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 135, 2024-09-14T09:08:28.848 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:08:28.848 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:08:28.848 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.848 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.848 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6808", 2024-09-14T09:08:28.848 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:08:28.848 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.848 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.848 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.848 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6809", 2024-09-14T09:08:28.848 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:08:28.848 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.849 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.849 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:08:28.849 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.849 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.849 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6810", 2024-09-14T09:08:28.849 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:08:28.849 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.849 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.849 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.849 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6811", 2024-09-14T09:08:28.849 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:08:28.850 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.850 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.850 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:08:28.850 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.850 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.850 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6814", 2024-09-14T09:08:28.850 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:08:28.850 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.850 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.850 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.850 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6815", 2024-09-14T09:08:28.850 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:08:28.851 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.851 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.851 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:08:28.851 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.851 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.851 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6812", 2024-09-14T09:08:28.851 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:08:28.851 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.851 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.851 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.851 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6813", 2024-09-14T09:08:28.851 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1576054389 2024-09-14T09:08:28.852 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.852 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.852 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6809/1576054389", 2024-09-14T09:08:28.852 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6811/1576054389", 2024-09-14T09:08:28.852 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6815/1576054389", 2024-09-14T09:08:28.852 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6813/1576054389", 2024-09-14T09:08:28.852 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:08:28.852 INFO:teuthology.orchestra.run.smithi176.stdout: "exists" 2024-09-14T09:08:28.852 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.852 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.852 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.852 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 5, 2024-09-14T09:08:28.853 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80", 2024-09-14T09:08:28.853 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:08:28.853 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:08:28.853 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:08:28.853 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:08:28.853 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:08:28.853 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:08:28.853 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:08:28.853 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 153, 2024-09-14T09:08:28.853 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:08:28.853 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:08:28.853 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:08:28.853 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.854 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.854 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6824", 2024-09-14T09:08:28.854 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:08:28.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.854 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.854 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.854 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6825", 2024-09-14T09:08:28.854 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:08:28.854 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.854 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.854 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.854 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:08:28.855 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.855 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.855 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6826", 2024-09-14T09:08:28.855 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:08:28.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.855 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.855 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.855 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6827", 2024-09-14T09:08:28.855 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:08:28.855 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.855 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.855 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.855 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:08:28.856 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.856 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.856 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6830", 2024-09-14T09:08:28.856 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:08:28.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.856 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.856 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.856 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6831", 2024-09-14T09:08:28.856 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:08:28.856 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.856 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.856 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.856 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:08:28.857 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.857 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.857 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6828", 2024-09-14T09:08:28.857 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:08:28.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.857 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.857 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.857 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6829", 2024-09-14T09:08:28.857 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 1711292386 2024-09-14T09:08:28.857 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.857 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.857 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.858 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6825/1711292386", 2024-09-14T09:08:28.858 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6827/1711292386", 2024-09-14T09:08:28.858 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6831/1711292386", 2024-09-14T09:08:28.858 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6829/1711292386", 2024-09-14T09:08:28.858 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:08:28.858 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:08:28.858 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:08:28.858 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.858 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.858 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.858 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 6, 2024-09-14T09:08:28.858 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "59d9259d-b9dd-4326-8ab9-9d599a2bf57b", 2024-09-14T09:08:28.858 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:08:28.859 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:08:28.859 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:08:28.859 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:08:28.859 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:08:28.859 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:08:28.859 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 15, 2024-09-14T09:08:28.859 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 171, 2024-09-14T09:08:28.859 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:08:28.859 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:08:28.859 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:08:28.859 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.859 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.859 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.859 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6816", 2024-09-14T09:08:28.859 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:08:28.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.860 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.860 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6817", 2024-09-14T09:08:28.860 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:08:28.860 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.860 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.860 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.860 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:08:28.860 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.860 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.860 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.861 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6818", 2024-09-14T09:08:28.861 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:08:28.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.861 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.861 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6819", 2024-09-14T09:08:28.861 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:08:28.861 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.861 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.861 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.861 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:08:28.861 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.861 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.861 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.862 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6822", 2024-09-14T09:08:28.862 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:08:28.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.862 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.862 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6823", 2024-09-14T09:08:28.862 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:08:28.862 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.862 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.862 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.862 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:08:28.862 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.862 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.862 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.862 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6820", 2024-09-14T09:08:28.862 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:08:28.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.863 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.863 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.863 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6821", 2024-09-14T09:08:28.863 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2721245607 2024-09-14T09:08:28.863 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.863 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.863 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.863 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6817/2721245607", 2024-09-14T09:08:28.863 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6819/2721245607", 2024-09-14T09:08:28.863 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6823/2721245607", 2024-09-14T09:08:28.863 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6821/2721245607", 2024-09-14T09:08:28.863 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:08:28.863 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:08:28.863 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:08:28.864 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.864 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.864 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.864 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 7, 2024-09-14T09:08:28.864 INFO:teuthology.orchestra.run.smithi176.stdout: "uuid": "e2f27d42-786b-449b-a2fb-9dd28f4e9788", 2024-09-14T09:08:28.864 INFO:teuthology.orchestra.run.smithi176.stdout: "up": 1, 2024-09-14T09:08:28.864 INFO:teuthology.orchestra.run.smithi176.stdout: "in": 1, 2024-09-14T09:08:28.864 INFO:teuthology.orchestra.run.smithi176.stdout: "weight": 1, 2024-09-14T09:08:28.864 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_affinity": 1, 2024-09-14T09:08:28.864 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_begin": 0, 2024-09-14T09:08:28.864 INFO:teuthology.orchestra.run.smithi176.stdout: "last_clean_end": 0, 2024-09-14T09:08:28.864 INFO:teuthology.orchestra.run.smithi176.stdout: "up_from": 14, 2024-09-14T09:08:28.864 INFO:teuthology.orchestra.run.smithi176.stdout: "up_thru": 171, 2024-09-14T09:08:28.864 INFO:teuthology.orchestra.run.smithi176.stdout: "down_at": 0, 2024-09-14T09:08:28.865 INFO:teuthology.orchestra.run.smithi176.stdout: "lost_at": 0, 2024-09-14T09:08:28.865 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addrs": { 2024-09-14T09:08:28.865 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.865 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.865 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6800", 2024-09-14T09:08:28.865 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:08:28.865 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.865 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.865 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.865 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6801", 2024-09-14T09:08:28.865 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:08:28.865 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.865 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.866 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addrs": { 2024-09-14T09:08:28.866 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.866 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.866 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6802", 2024-09-14T09:08:28.866 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:08:28.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.866 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.866 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.866 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6803", 2024-09-14T09:08:28.866 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:08:28.866 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.866 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.866 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.866 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addrs": { 2024-09-14T09:08:28.867 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.867 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.867 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6806", 2024-09-14T09:08:28.867 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:08:28.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.867 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.867 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.867 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6807", 2024-09-14T09:08:28.867 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:08:28.867 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.867 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.867 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.868 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addrs": { 2024-09-14T09:08:28.868 INFO:teuthology.orchestra.run.smithi176.stdout: "addrvec": [ 2024-09-14T09:08:28.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.868 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v2", 2024-09-14T09:08:28.868 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6804", 2024-09-14T09:08:28.868 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:08:28.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.868 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.868 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-09-14T09:08:28.868 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "172.21.15.183:6805", 2024-09-14T09:08:28.868 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 2122823465 2024-09-14T09:08:28.868 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.868 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.868 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.869 INFO:teuthology.orchestra.run.smithi176.stdout: "public_addr": "172.21.15.183:6801/2122823465", 2024-09-14T09:08:28.869 INFO:teuthology.orchestra.run.smithi176.stdout: "cluster_addr": "172.21.15.183:6803/2122823465", 2024-09-14T09:08:28.869 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_back_addr": "172.21.15.183:6807/2122823465", 2024-09-14T09:08:28.869 INFO:teuthology.orchestra.run.smithi176.stdout: "heartbeat_front_addr": "172.21.15.183:6805/2122823465", 2024-09-14T09:08:28.869 INFO:teuthology.orchestra.run.smithi176.stdout: "state": [ 2024-09-14T09:08:28.869 INFO:teuthology.orchestra.run.smithi176.stdout: "exists", 2024-09-14T09:08:28.869 INFO:teuthology.orchestra.run.smithi176.stdout: "up" 2024-09-14T09:08:28.869 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.869 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.869 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:08:28.869 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_xinfo": [ 2024-09-14T09:08:28.869 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.869 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 0, 2024-09-14T09:08:28.870 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:08:28.870 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:08:28.870 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:08:28.870 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:08:28.870 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:08:28.870 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.411789+0000", 2024-09-14T09:08:28.870 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:08:28.870 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.870 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.870 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 1, 2024-09-14T09:08:28.870 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:08:28.870 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:08:28.870 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:08:28.870 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:08:28.871 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:08:28.871 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:05.801873+0000", 2024-09-14T09:08:28.871 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:08:28.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.871 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.871 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 2, 2024-09-14T09:08:28.871 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:08:28.871 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:08:28.871 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:08:28.871 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:08:28.871 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:08:28.871 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.206725+0000", 2024-09-14T09:08:28.871 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:08:28.871 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.872 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 3, 2024-09-14T09:08:28.872 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:08:28.872 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:08:28.872 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:08:28.872 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:08:28.872 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:08:28.872 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:05.882180+0000", 2024-09-14T09:08:28.872 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:08:28.872 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.872 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.872 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 4, 2024-09-14T09:08:28.873 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:08:28.873 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:08:28.873 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:08:28.873 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:08:28.873 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:08:28.873 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.013864+0000", 2024-09-14T09:08:28.873 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 135 2024-09-14T09:08:28.873 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.873 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.873 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 5, 2024-09-14T09:08:28.873 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:08:28.873 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:08:28.873 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:08:28.874 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:08:28.874 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:08:28.874 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.652975+0000", 2024-09-14T09:08:28.874 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:08:28.874 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.874 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.874 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 6, 2024-09-14T09:08:28.874 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:08:28.874 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:08:28.874 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:08:28.874 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:08:28.874 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:08:28.874 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:06.741359+0000", 2024-09-14T09:08:28.874 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:08:28.875 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.875 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.875 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": 7, 2024-09-14T09:08:28.875 INFO:teuthology.orchestra.run.smithi176.stdout: "down_stamp": "0.000000", 2024-09-14T09:08:28.875 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_probability": 0, 2024-09-14T09:08:28.875 INFO:teuthology.orchestra.run.smithi176.stdout: "laggy_interval": 0, 2024-09-14T09:08:28.875 INFO:teuthology.orchestra.run.smithi176.stdout: "features": 4540719139924082687, 2024-09-14T09:08:28.875 INFO:teuthology.orchestra.run.smithi176.stdout: "old_weight": 0, 2024-09-14T09:08:28.875 INFO:teuthology.orchestra.run.smithi176.stdout: "last_purged_snaps_scrub": "2024-09-14T09:05:05.873675+0000", 2024-09-14T09:08:28.875 INFO:teuthology.orchestra.run.smithi176.stdout: "dead_epoch": 0 2024-09-14T09:08:28.875 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.875 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:08:28.875 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_upmap": [ 2024-09-14T09:08:28.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.876 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "3.4", 2024-09-14T09:08:28.876 INFO:teuthology.orchestra.run.smithi176.stdout: "osds": [ 2024-09-14T09:08:28.876 INFO:teuthology.orchestra.run.smithi176.stdout: 1, 2024-09-14T09:08:28.876 INFO:teuthology.orchestra.run.smithi176.stdout: 7 2024-09-14T09:08:28.876 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.876 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.876 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:08:28.876 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_upmap_items": [ 2024-09-14T09:08:28.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.876 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "3.9", 2024-09-14T09:08:28.876 INFO:teuthology.orchestra.run.smithi176.stdout: "mappings": [ 2024-09-14T09:08:28.876 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.876 INFO:teuthology.orchestra.run.smithi176.stdout: "from": 4, 2024-09-14T09:08:28.876 INFO:teuthology.orchestra.run.smithi176.stdout: "to": 6 2024-09-14T09:08:28.877 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.877 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.877 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "3.19", 2024-09-14T09:08:28.877 INFO:teuthology.orchestra.run.smithi176.stdout: "mappings": [ 2024-09-14T09:08:28.877 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.877 INFO:teuthology.orchestra.run.smithi176.stdout: "from": 3, 2024-09-14T09:08:28.877 INFO:teuthology.orchestra.run.smithi176.stdout: "to": 0 2024-09-14T09:08:28.877 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.877 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.877 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.878 INFO:teuthology.orchestra.run.smithi176.stdout: "pgid": "3.1b", 2024-09-14T09:08:28.878 INFO:teuthology.orchestra.run.smithi176.stdout: "mappings": [ 2024-09-14T09:08:28.878 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.878 INFO:teuthology.orchestra.run.smithi176.stdout: "from": 4, 2024-09-14T09:08:28.878 INFO:teuthology.orchestra.run.smithi176.stdout: "to": 6 2024-09-14T09:08:28.878 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.878 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.878 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.878 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:08:28.878 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_upmap_primaries": [], 2024-09-14T09:08:28.878 INFO:teuthology.orchestra.run.smithi176.stdout: "pg_temp": [], 2024-09-14T09:08:28.878 INFO:teuthology.orchestra.run.smithi176.stdout: "primary_temp": [], 2024-09-14T09:08:28.878 INFO:teuthology.orchestra.run.smithi176.stdout: "blocklist": {}, 2024-09-14T09:08:28.879 INFO:teuthology.orchestra.run.smithi176.stdout: "range_blocklist": {}, 2024-09-14T09:08:28.879 INFO:teuthology.orchestra.run.smithi176.stdout: "erasure_code_profiles": { 2024-09-14T09:08:28.879 INFO:teuthology.orchestra.run.smithi176.stdout: "default": { 2024-09-14T09:08:28.879 INFO:teuthology.orchestra.run.smithi176.stdout: "crush-failure-domain": "osd", 2024-09-14T09:08:28.879 INFO:teuthology.orchestra.run.smithi176.stdout: "k": "2", 2024-09-14T09:08:28.879 INFO:teuthology.orchestra.run.smithi176.stdout: "m": "1", 2024-09-14T09:08:28.879 INFO:teuthology.orchestra.run.smithi176.stdout: "plugin": "jerasure", 2024-09-14T09:08:28.879 INFO:teuthology.orchestra.run.smithi176.stdout: "technique": "reed_sol_van" 2024-09-14T09:08:28.879 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.879 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:28.879 INFO:teuthology.orchestra.run.smithi176.stdout: "removed_snaps_queue": [ 2024-09-14T09:08:28.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.879 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 3, 2024-09-14T09:08:28.879 INFO:teuthology.orchestra.run.smithi176.stdout: "snaps": [ 2024-09-14T09:08:28.879 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.880 INFO:teuthology.orchestra.run.smithi176.stdout: "begin": 9, 2024-09-14T09:08:28.880 INFO:teuthology.orchestra.run.smithi176.stdout: "length": 38 2024-09-14T09:08:28.880 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.880 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.880 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.880 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:08:28.880 INFO:teuthology.orchestra.run.smithi176.stdout: "new_removed_snaps": [ 2024-09-14T09:08:28.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.880 INFO:teuthology.orchestra.run.smithi176.stdout: "pool": 3, 2024-09-14T09:08:28.880 INFO:teuthology.orchestra.run.smithi176.stdout: "snaps": [ 2024-09-14T09:08:28.880 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:28.880 INFO:teuthology.orchestra.run.smithi176.stdout: "begin": 9, 2024-09-14T09:08:28.880 INFO:teuthology.orchestra.run.smithi176.stdout: "length": 1 2024-09-14T09:08:28.880 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.880 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:28.881 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.881 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-09-14T09:08:28.881 INFO:teuthology.orchestra.run.smithi176.stdout: "new_purged_snaps": [], 2024-09-14T09:08:28.881 INFO:teuthology.orchestra.run.smithi176.stdout: "crush_node_flags": {}, 2024-09-14T09:08:28.881 INFO:teuthology.orchestra.run.smithi176.stdout: "device_class_flags": {}, 2024-09-14T09:08:28.881 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_mode": { 2024-09-14T09:08:28.881 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_mode_enabled": false, 2024-09-14T09:08:28.881 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_bucket_count": 0, 2024-09-14T09:08:28.881 INFO:teuthology.orchestra.run.smithi176.stdout: "degraded_stretch_mode": 0, 2024-09-14T09:08:28.881 INFO:teuthology.orchestra.run.smithi176.stdout: "recovering_stretch_mode": 0, 2024-09-14T09:08:28.881 INFO:teuthology.orchestra.run.smithi176.stdout: "stretch_mode_bucket": 0 2024-09-14T09:08:28.881 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:28.881 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:28.882 INFO:tasks.thrashosds.thrasher:j is {'epoch': 176, 'fsid': 'c635667d-994e-454f-bcea-51b45637b2a2', 'created': '2024-09-14T09:04:57.077293+0000', 'modified': '2024-09-14T09:08:28.421038+0000', 'last_up_change': '2024-09-14T09:07:46.980109+0000', 'last_in_change': '2024-09-14T09:07:21.060249+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T09:05:11.997673+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T09:05:15.352551+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T09:05:25.840963+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '176', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 46, 'snap_epoch': 176, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'error_message': 'pool 3 has some PGs where all OSDs are with primary_affinity 0 (at least pg 3.1a), read balance score may not be reliable', 'score_acting': 0, 'score_stable': 0}}], 'osds': [{'osd': 0, 'uuid': '4ceedb4f-6398-467f-9562-ccf27200860b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 171, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6824', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6825', 'nonce': 3384710819}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6826', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6827', 'nonce': 3384710819}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6830', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6831', 'nonce': 3384710819}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6828', 'nonce': 3384710819}, {'type': 'v1', 'addr': '172.21.15.176:6829', 'nonce': 3384710819}]}, 'public_addr': '172.21.15.176:6825/3384710819', 'cluster_addr': '172.21.15.176:6827/3384710819', 'heartbeat_back_addr': '172.21.15.176:6831/3384710819', 'heartbeat_front_addr': '172.21.15.176:6829/3384710819', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '604e2f62-7c13-46be-a97f-029b28c44a89', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 135, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6800', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6801', 'nonce': 612012679}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6802', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6803', 'nonce': 612012679}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6806', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6807', 'nonce': 612012679}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6804', 'nonce': 612012679}, {'type': 'v1', 'addr': '172.21.15.176:6805', 'nonce': 612012679}]}, 'public_addr': '172.21.15.176:6801/612012679', 'cluster_addr': '172.21.15.176:6803/612012679', 'heartbeat_back_addr': '172.21.15.176:6807/612012679', 'heartbeat_front_addr': '172.21.15.176:6805/612012679', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1f65589e-8b0a-4033-94c0-4ab66531db2b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 171, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6808', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6809', 'nonce': 4167379279}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6810', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6811', 'nonce': 4167379279}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6814', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6815', 'nonce': 4167379279}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6812', 'nonce': 4167379279}, {'type': 'v1', 'addr': '172.21.15.176:6813', 'nonce': 4167379279}]}, 'public_addr': '172.21.15.176:6809/4167379279', 'cluster_addr': '172.21.15.176:6811/4167379279', 'heartbeat_back_addr': '172.21.15.176:6815/4167379279', 'heartbeat_front_addr': '172.21.15.176:6813/4167379279', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '209d4f65-79f9-4a7d-9d23-832a4209507f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6816', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6817', 'nonce': 1658209015}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6818', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6819', 'nonce': 1658209015}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6822', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6823', 'nonce': 1658209015}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6820', 'nonce': 1658209015}, {'type': 'v1', 'addr': '172.21.15.176:6821', 'nonce': 1658209015}]}, 'public_addr': '172.21.15.176:6817/1658209015', 'cluster_addr': '172.21.15.176:6819/1658209015', 'heartbeat_back_addr': '172.21.15.176:6823/1658209015', 'heartbeat_front_addr': '172.21.15.176:6821/1658209015', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8f9a8c9b-282e-4523-9e62-5824f4aba401', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 116, 'down_at': 135, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6808', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6809', 'nonce': 1576054389}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6810', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6811', 'nonce': 1576054389}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6814', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6815', 'nonce': 1576054389}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6812', 'nonce': 1576054389}, {'type': 'v1', 'addr': '172.21.15.183:6813', 'nonce': 1576054389}]}, 'public_addr': '172.21.15.183:6809/1576054389', 'cluster_addr': '172.21.15.183:6811/1576054389', 'heartbeat_back_addr': '172.21.15.183:6815/1576054389', 'heartbeat_front_addr': '172.21.15.183:6813/1576054389', 'state': ['exists']}, {'osd': 5, 'uuid': '1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 153, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6824', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6825', 'nonce': 1711292386}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6826', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6827', 'nonce': 1711292386}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6830', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6831', 'nonce': 1711292386}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6828', 'nonce': 1711292386}, {'type': 'v1', 'addr': '172.21.15.183:6829', 'nonce': 1711292386}]}, 'public_addr': '172.21.15.183:6825/1711292386', 'cluster_addr': '172.21.15.183:6827/1711292386', 'heartbeat_back_addr': '172.21.15.183:6831/1711292386', 'heartbeat_front_addr': '172.21.15.183:6829/1711292386', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '59d9259d-b9dd-4326-8ab9-9d599a2bf57b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 171, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6816', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6817', 'nonce': 2721245607}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6818', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6819', 'nonce': 2721245607}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6822', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6823', 'nonce': 2721245607}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6820', 'nonce': 2721245607}, {'type': 'v1', 'addr': '172.21.15.183:6821', 'nonce': 2721245607}]}, 'public_addr': '172.21.15.183:6817/2721245607', 'cluster_addr': '172.21.15.183:6819/2721245607', 'heartbeat_back_addr': '172.21.15.183:6823/2721245607', 'heartbeat_front_addr': '172.21.15.183:6821/2721245607', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'e2f27d42-786b-449b-a2fb-9dd28f4e9788', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 171, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6800', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6801', 'nonce': 2122823465}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6802', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6803', 'nonce': 2122823465}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6806', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6807', 'nonce': 2122823465}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6804', 'nonce': 2122823465}, {'type': 'v1', 'addr': '172.21.15.183:6805', 'nonce': 2122823465}]}, 'public_addr': '172.21.15.183:6801/2122823465', 'cluster_addr': '172.21.15.183:6803/2122823465', 'heartbeat_back_addr': '172.21.15.183:6807/2122823465', 'heartbeat_front_addr': '172.21.15.183:6805/2122823465', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.411789+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:05.801873+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.206725+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:05.882180+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.013864+0000', 'dead_epoch': 135}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.652975+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:06.741359+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T09:05:05.873675+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.4', 'osds': [1, 7]}], 'pg_upmap_items': [{'pgid': '3.9', 'mappings': [{'from': 4, 'to': 6}]}, {'pgid': '3.19', 'mappings': [{'from': 3, 'to': 0}]}, {'pgid': '3.1b', 'mappings': [{'from': 4, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 9, 'length': 38}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 9, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T09:08:28.882 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:08:28.883 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:28.810+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:28.905 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:28.910+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:29.006 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:29.010+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:29.009 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:29.010 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:29.010 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:29.010 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:29.028 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:08:29.107 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:29.107+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:29.144 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:29.144 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:29.144 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:08:29.144 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:08:29.144 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:29.156 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:08:29.198 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:08:29.199 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:08:29.218 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-14T09:08:28.435739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":421832832,"num_objects":1019,"num_object_clones":332,"num_object_copies":2038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":727,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1019,"num_whiteouts":590,"num_read":195961,"num_read_kb":858290,"num_write":24808,"num_write_kb":387452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3805,"num_bytes_recovered":203007312,"num_keys_recovered":652,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21327,"ondisk_log_size":21327,"up":71,"acting":71,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":450224,"kb_used_data":35484,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754524496,"statfs":{"total":773094113280,"available":772633083904,"internally_reserved":0,"allocated":36335616,"data_stored":164403328,"data_compressed":5694831,"data_compressed_allocated":14487552,"data_compressed_original":153197221,"omap_allocated":12751,"internal_metadata":424463921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-154047961,"num_objects":-169,"num_object_clones":-54,"num_object_copies":-338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-25,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-169,"num_whiteouts":-52,"num_read":1410,"num_read_kb":1202,"num_write":200,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.065742"},"pg_stats":[{"pgid":"3.1f","version":"171'1746","reported_seq":7724,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T09:08:26.416899+0000","last_change":"2024-09-14T09:08:26.417608+0000","last_active":"2024-09-14T09:08:26.416899+0000","last_peered":"2024-09-14T09:08:26.416899+0000","last_clean":"2024-09-14T09:08:26.416899+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:08:26.416899+0000","last_undegraded":"2024-09-14T09:08:26.416899+0000","last_fullsized":"2024-09-14T09:08:26.416899+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1746,"log_dups_size":0,"ondisk_log_size":1746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:48:11.228257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002531185,"stat_sum":{"num_bytes":1572919,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":5,"num_read":13799,"num_read_kb":24692,"num_write":2668,"num_write_kb":8604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"24"}]},{"pgid":"3.1e","version":"173'459","reported_seq":5862,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-14T09:08:25.402841+0000","last_change":"2024-09-14T09:08:25.402841+0000","last_active":"2024-09-14T09:08:25.402841+0000","last_peered":"2024-09-14T09:08:25.402841+0000","last_clean":"2024-09-14T09:08:25.402841+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:08:25.402841+0000","last_undegraded":"2024-09-14T09:08:25.402841+0000","last_fullsized":"2024-09-14T09:08:25.402841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:38:03.522549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043354860000000004,"stat_sum":{"num_bytes":7143225,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":9,"num_read":5032,"num_read_kb":31996,"num_write":415,"num_write_kb":13378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"23"}]},{"pgid":"3.1d","version":"174'599","reported_seq":6225,"reported_epoch":174,"state":"active+undersized+degraded","last_fresh":"2024-09-14T09:08:26.745421+0000","last_change":"2024-09-14T09:07:48.009443+0000","last_active":"2024-09-14T09:08:26.745421+0000","last_peered":"2024-09-14T09:08:26.745421+0000","last_clean":"2024-09-14T09:07:46.406260+0000","last_became_active":"2024-09-14T09:07:48.009443+0000","last_became_peered":"2024-09-14T09:07:48.009443+0000","last_unstale":"2024-09-14T09:08:26.745421+0000","last_undegraded":"2024-09-14T09:07:48.007028+0000","last_fullsized":"2024-09-14T09:07:48.006949+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":36,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:54:30.822221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014149329999999999,"stat_sum":{"num_bytes":41336940,"num_objects":73,"num_object_clones":31,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":73,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":36,"num_read":6644,"num_read_kb":58029,"num_write":710,"num_write_kb":12432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":73}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"174'775","reported_seq":5117,"reported_epoch":174,"state":"active+undersized+degraded","last_fresh":"2024-09-14T09:08:26.714625+0000","last_change":"2024-09-14T09:07:48.009089+0000","last_active":"2024-09-14T09:08:26.714625+0000","last_peered":"2024-09-14T09:08:26.714625+0000","last_clean":"2024-09-14T09:07:46.405024+0000","last_became_active":"2024-09-14T09:07:48.009089+0000","last_became_peered":"2024-09-14T09:07:48.009089+0000","last_unstale":"2024-09-14T09:08:26.714625+0000","last_undegraded":"2024-09-14T09:07:48.006836+0000","last_fullsized":"2024-09-14T09:07:48.006754+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":36,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:32:10.119693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014661419999999999,"stat_sum":{"num_bytes":16427879,"num_objects":69,"num_object_clones":30,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":69,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":36,"num_read":7346,"num_read_kb":27175,"num_write":1009,"num_write_kb":9246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":69}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"174'733","reported_seq":5598,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T09:08:26.421260+0000","last_change":"2024-09-14T09:08:26.421733+0000","last_active":"2024-09-14T09:08:26.421260+0000","last_peered":"2024-09-14T09:08:26.421260+0000","last_clean":"2024-09-14T09:08:26.421260+0000","last_became_active":"2024-09-14T09:07:06.842337+0000","last_became_peered":"2024-09-14T09:07:06.842337+0000","last_unstale":"2024-09-14T09:08:26.421260+0000","last_undegraded":"2024-09-14T09:08:26.421260+0000","last_fullsized":"2024-09-14T09:08:26.421260+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:49:15.054032+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0069198890000000002,"stat_sum":{"num_bytes":6961500,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":9,"num_read":6703,"num_read_kb":24544,"num_write":873,"num_write_kb":16463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":14145319,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"24"}]},{"pgid":"3.1a","version":"170'676","reported_seq":5665,"reported_epoch":173,"state":"active+undersized+degraded","last_fresh":"2024-09-14T09:08:25.716465+0000","last_change":"2024-09-14T09:07:48.003614+0000","last_active":"2024-09-14T09:08:25.716465+0000","last_peered":"2024-09-14T09:08:25.716465+0000","last_clean":"2024-09-14T09:07:46.238929+0000","last_became_active":"2024-09-14T09:07:48.003614+0000","last_became_peered":"2024-09-14T09:07:48.003614+0000","last_unstale":"2024-09-14T09:08:25.716465+0000","last_undegraded":"2024-09-14T09:07:48.002626+0000","last_fullsized":"2024-09-14T09:07:48.002391+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":35,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:06:07.609511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020845550000000001,"stat_sum":{"num_bytes":19869286,"num_objects":55,"num_object_clones":23,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":55,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":29,"num_read":6329,"num_read_kb":30617,"num_write":961,"num_write_kb":14233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":55}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"174'375","reported_seq":4358,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T09:08:26.579075+0000","last_change":"2024-09-14T09:08:26.441872+0000","last_active":"2024-09-14T09:08:26.579075+0000","last_peered":"2024-09-14T09:08:26.579075+0000","last_clean":"2024-09-14T09:08:26.579075+0000","last_became_active":"2024-09-14T09:08:24.378557+0000","last_became_peered":"2024-09-14T09:08:24.378557+0000","last_unstale":"2024-09-14T09:08:26.579075+0000","last_undegraded":"2024-09-14T09:08:26.579075+0000","last_fullsized":"2024-09-14T09:08:26.579075+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:09:52.207357+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.025218744000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3327,"num_read_kb":10838,"num_write":357,"num_write_kb":6673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":3727781,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"24"}]},{"pgid":"3.18","version":"171'604","reported_seq":6242,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-14T09:08:25.398961+0000","last_change":"2024-09-14T09:08:25.400135+0000","last_active":"2024-09-14T09:08:25.398961+0000","last_peered":"2024-09-14T09:08:25.398961+0000","last_clean":"2024-09-14T09:08:25.398961+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:08:25.398961+0000","last_undegraded":"2024-09-14T09:08:25.398961+0000","last_fullsized":"2024-09-14T09:08:25.398961+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:04:10.051109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044252839999999998,"stat_sum":{"num_bytes":30735708,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":10,"num_read":7486,"num_read_kb":17479,"num_write":529,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"23"}]},{"pgid":"3.17","version":"174'884","reported_seq":7615,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T09:08:26.648651+0000","last_change":"2024-09-14T09:08:26.430296+0000","last_active":"2024-09-14T09:08:26.648651+0000","last_peered":"2024-09-14T09:08:26.648651+0000","last_clean":"2024-09-14T09:08:26.648651+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:08:26.648651+0000","last_undegraded":"2024-09-14T09:08:26.648651+0000","last_fullsized":"2024-09-14T09:08:26.648651+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T21:04:49.860026+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015026681,"stat_sum":{"num_bytes":417294,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":9896,"num_read_kb":35330,"num_write":1109,"num_write_kb":10789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"24"}]},{"pgid":"3.16","version":"171'592","reported_seq":3716,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T09:08:26.678978+0000","last_change":"2024-09-14T09:08:26.427798+0000","last_active":"2024-09-14T09:08:26.678978+0000","last_peered":"2024-09-14T09:08:26.678978+0000","last_clean":"2024-09-14T09:08:26.678978+0000","last_became_active":"2024-09-14T09:07:25.105674+0000","last_became_peered":"2024-09-14T09:07:25.105674+0000","last_unstale":"2024-09-14T09:08:26.678978+0000","last_undegraded":"2024-09-14T09:08:26.678978+0000","last_fullsized":"2024-09-14T09:08:26.678978+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:24:11.990165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001647244,"stat_sum":{"num_bytes":131107,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":5,"num_read":4493,"num_read_kb":15400,"num_write":669,"num_write_kb":7548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":8467448,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"b","length":"24"}]},{"pgid":"3.15","version":"173'1087","reported_seq":5041,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-14T09:08:25.710827+0000","last_change":"2024-09-14T09:08:25.416683+0000","last_active":"2024-09-14T09:08:25.710827+0000","last_peered":"2024-09-14T09:08:25.710827+0000","last_clean":"2024-09-14T09:08:25.710827+0000","last_became_active":"2024-09-14T09:07:27.854998+0000","last_became_peered":"2024-09-14T09:07:27.854998+0000","last_unstale":"2024-09-14T09:08:25.710827+0000","last_undegraded":"2024-09-14T09:08:25.710827+0000","last_fullsized":"2024-09-14T09:08:25.710827+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:32:11.452704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021310959999999999,"stat_sum":{"num_bytes":131109,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":12,"num_read":8119,"num_read_kb":26180,"num_write":1567,"num_write_kb":14705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":4170600,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"23"}]},{"pgid":"3.14","version":"173'605","reported_seq":3402,"reported_epoch":173,"state":"active+undersized+degraded","last_fresh":"2024-09-14T09:08:25.589408+0000","last_change":"2024-09-14T09:07:48.003957+0000","last_active":"2024-09-14T09:08:25.589408+0000","last_peered":"2024-09-14T09:08:25.589408+0000","last_clean":"2024-09-14T09:07:43.255519+0000","last_became_active":"2024-09-14T09:07:48.003957+0000","last_became_peered":"2024-09-14T09:07:48.003957+0000","last_unstale":"2024-09-14T09:08:25.589408+0000","last_undegraded":"2024-09-14T09:07:48.002075+0000","last_fullsized":"2024-09-14T09:07:48.001966+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":35,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:21:15.195112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014079679999999999,"stat_sum":{"num_bytes":9392676,"num_objects":64,"num_object_clones":25,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":64,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":35,"num_read":3543,"num_read_kb":9171,"num_write":674,"num_write_kb":3707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":2686146,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":64}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":337,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T09:08:26.420507+0000","last_change":"2024-09-14T09:07:21.857426+0000","last_active":"2024-09-14T09:08:26.420507+0000","last_peered":"2024-09-14T09:08:26.420507+0000","last_clean":"2024-09-14T09:08:26.420507+0000","last_became_active":"2024-09-14T09:07:21.856579+0000","last_became_peered":"2024-09-14T09:07:21.856579+0000","last_unstale":"2024-09-14T09:08:26.420507+0000","last_undegraded":"2024-09-14T09:08:26.420507+0000","last_fullsized":"2024-09-14T09:08:26.420507+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:56.694930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"174'755","reported_seq":6453,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T09:08:26.684982+0000","last_change":"2024-09-14T09:08:26.415865+0000","last_active":"2024-09-14T09:08:26.684982+0000","last_peered":"2024-09-14T09:08:26.684982+0000","last_clean":"2024-09-14T09:08:26.684982+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:08:26.684982+0000","last_undegraded":"2024-09-14T09:08:26.684982+0000","last_fullsized":"2024-09-14T09:08:26.684982+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:05:53.230220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097996299999999993,"stat_sum":{"num_bytes":16071045,"num_objects":21,"num_object_clones":1,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":15,"num_read":8430,"num_read_kb":45180,"num_write":686,"num_write_kb":8858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"24"}]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T09:08:26.420858+0000","last_change":"2024-09-14T09:08:24.383710+0000","last_active":"2024-09-14T09:08:26.420858+0000","last_peered":"2024-09-14T09:08:26.420858+0000","last_clean":"2024-09-14T09:08:26.420858+0000","last_became_active":"2024-09-14T09:08:24.383380+0000","last_became_peered":"2024-09-14T09:08:24.383380+0000","last_unstale":"2024-09-14T09:08:26.420858+0000","last_undegraded":"2024-09-14T09:08:26.420858+0000","last_fullsized":"2024-09-14T09:08:26.420858+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:06:25.390635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"171'1180","reported_seq":7135,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-14T09:08:25.401742+0000","last_change":"2024-09-14T09:08:25.402167+0000","last_active":"2024-09-14T09:08:25.401742+0000","last_peered":"2024-09-14T09:08:25.401742+0000","last_clean":"2024-09-14T09:08:25.401742+0000","last_became_active":"2024-09-14T09:07:26.102666+0000","last_became_peered":"2024-09-14T09:07:26.102666+0000","last_unstale":"2024-09-14T09:08:25.401742+0000","last_undegraded":"2024-09-14T09:08:25.401742+0000","last_fullsized":"2024-09-14T09:08:25.401742+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:46:01.505947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035967260000000002,"stat_sum":{"num_bytes":9717754,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":8,"num_read":10102,"num_read_kb":27247,"num_write":1620,"num_write_kb":13456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":8334712,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"23"}]},{"pgid":"2.5","version":"0'0","reported_seq":131,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-14T09:08:25.413977+0000","last_change":"2024-09-14T09:07:21.863290+0000","last_active":"2024-09-14T09:08:25.413977+0000","last_peered":"2024-09-14T09:08:25.413977+0000","last_clean":"2024-09-14T09:08:25.413977+0000","last_became_active":"2024-09-14T09:07:21.862352+0000","last_became_peered":"2024-09-14T09:07:21.862352+0000","last_unstale":"2024-09-14T09:08:25.413977+0000","last_undegraded":"2024-09-14T09:08:25.413977+0000","last_fullsized":"2024-09-14T09:08:25.413977+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:48:30.332891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"172'915","reported_seq":5797,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-14T09:08:25.726025+0000","last_change":"2024-09-14T09:08:25.417679+0000","last_active":"2024-09-14T09:08:25.726025+0000","last_peered":"2024-09-14T09:08:25.726025+0000","last_clean":"2024-09-14T09:08:25.726025+0000","last_became_active":"2024-09-14T09:08:24.383152+0000","last_became_peered":"2024-09-14T09:08:24.383152+0000","last_unstale":"2024-09-14T09:08:25.726025+0000","last_undegraded":"2024-09-14T09:08:25.726025+0000","last_fullsized":"2024-09-14T09:08:25.726025+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":915,"log_dups_size":0,"ondisk_log_size":915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:29:57.083596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025870709999999998,"stat_sum":{"num_bytes":559721,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":13,"num_read":7777,"num_read_kb":22823,"num_write":1133,"num_write_kb":9259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":8151041,"num_keys_recovered":91,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"23"}]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T09:08:26.415070+0000","last_change":"2024-09-14T09:08:24.380384+0000","last_active":"2024-09-14T09:08:26.415070+0000","last_peered":"2024-09-14T09:08:26.415070+0000","last_clean":"2024-09-14T09:08:26.415070+0000","last_became_active":"2024-09-14T09:08:24.379135+0000","last_became_peered":"2024-09-14T09:08:24.379135+0000","last_unstale":"2024-09-14T09:08:26.415070+0000","last_undegraded":"2024-09-14T09:08:26.415070+0000","last_fullsized":"2024-09-14T09:08:26.415070+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:01:26.202865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"174'677","reported_seq":2797,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T09:08:26.439933+0000","last_change":"2024-09-14T09:08:26.440964+0000","last_active":"2024-09-14T09:08:26.439933+0000","last_peered":"2024-09-14T09:08:26.439933+0000","last_clean":"2024-09-14T09:08:26.439933+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:08:26.439933+0000","last_undegraded":"2024-09-14T09:08:26.439933+0000","last_fullsized":"2024-09-14T09:08:26.439933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:11:45.008604+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.014316534000000001,"stat_sum":{"num_bytes":28853693,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":13,"num_read":3588,"num_read_kb":16892,"num_write":788,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"b","length":"24"}]},{"pgid":"3.3","version":"174'791","reported_seq":6333,"reported_epoch":174,"state":"active+undersized+degraded","last_fresh":"2024-09-14T09:08:26.604680+0000","last_change":"2024-09-14T09:07:48.007391+0000","last_active":"2024-09-14T09:08:26.604680+0000","last_peered":"2024-09-14T09:08:26.604680+0000","last_clean":"2024-09-14T09:07:46.200321+0000","last_became_active":"2024-09-14T09:07:48.007391+0000","last_became_peered":"2024-09-14T09:07:48.007391+0000","last_unstale":"2024-09-14T09:08:26.604680+0000","last_undegraded":"2024-09-14T09:07:48.005984+0000","last_fullsized":"2024-09-14T09:07:48.005905+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":36,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:49:29.128106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001014128,"stat_sum":{"num_bytes":24220798,"num_objects":58,"num_object_clones":25,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":58,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":30,"num_read":8181,"num_read_kb":26821,"num_write":848,"num_write_kb":12141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":58}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":327,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T09:08:26.425717+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:08:26.425717+0000","last_peered":"2024-09-14T09:08:26.425717+0000","last_clean":"2024-09-14T09:08:26.425717+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:08:26.425717+0000","last_undegraded":"2024-09-14T09:08:26.425717+0000","last_fullsized":"2024-09-14T09:08:26.425717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"174'398","reported_seq":2451,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T09:08:26.422653+0000","last_change":"2024-09-14T09:08:26.422653+0000","last_active":"2024-09-14T09:08:26.422653+0000","last_peered":"2024-09-14T09:08:26.422653+0000","last_clean":"2024-09-14T09:08:26.422653+0000","last_became_active":"2024-09-14T09:08:24.379178+0000","last_became_peered":"2024-09-14T09:08:24.379178+0000","last_unstale":"2024-09-14T09:08:26.422653+0000","last_undegraded":"2024-09-14T09:08:26.422653+0000","last_fullsized":"2024-09-14T09:08:26.422653+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:44:25.717039+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0082873410000000002,"stat_sum":{"num_bytes":16,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":1870,"num_read_kb":11981,"num_write":351,"num_write_kb":5539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"b","length":"24"}]},{"pgid":"2.1","version":"0'0","reported_seq":325,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T09:08:26.414275+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:08:26.414275+0000","last_peered":"2024-09-14T09:08:26.414275+0000","last_clean":"2024-09-14T09:08:26.414275+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:08:26.414275+0000","last_undegraded":"2024-09-14T09:08:26.414275+0000","last_fullsized":"2024-09-14T09:08:26.414275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"173'398","reported_seq":4741,"reported_epoch":174,"state":"active+undersized+degraded","last_fresh":"2024-09-14T09:08:26.415772+0000","last_change":"2024-09-14T09:07:48.007000+0000","last_active":"2024-09-14T09:08:26.415772+0000","last_peered":"2024-09-14T09:08:26.415772+0000","last_clean":"2024-09-14T09:07:46.276561+0000","last_became_active":"2024-09-14T09:07:48.007000+0000","last_became_peered":"2024-09-14T09:07:48.007000+0000","last_unstale":"2024-09-14T09:08:26.415772+0000","last_undegraded":"2024-09-14T09:07:48.005399+0000","last_fullsized":"2024-09-14T09:07:48.005305+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":36,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:00:29.680944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074815999999999999,"stat_sum":{"num_bytes":20032975,"num_objects":75,"num_object_clones":31,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":75,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":44,"num_read":3905,"num_read_kb":25149,"num_write":375,"num_write_kb":14483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":75}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":131,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-14T09:08:25.413451+0000","last_change":"2024-09-14T09:07:21.856721+0000","last_active":"2024-09-14T09:08:25.413451+0000","last_peered":"2024-09-14T09:08:25.413451+0000","last_clean":"2024-09-14T09:08:25.413451+0000","last_became_active":"2024-09-14T09:07:21.856091+0000","last_became_peered":"2024-09-14T09:07:21.856091+0000","last_unstale":"2024-09-14T09:08:25.413451+0000","last_undegraded":"2024-09-14T09:08:25.413451+0000","last_fullsized":"2024-09-14T09:08:25.413451+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:16:17.683341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"171'417","reported_seq":2842,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-14T09:08:25.399209+0000","last_change":"2024-09-14T09:08:25.400667+0000","last_active":"2024-09-14T09:08:25.399209+0000","last_peered":"2024-09-14T09:08:25.399209+0000","last_clean":"2024-09-14T09:08:25.399209+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:08:25.399209+0000","last_undegraded":"2024-09-14T09:08:25.399209+0000","last_fullsized":"2024-09-14T09:08:25.399209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:29:06.108716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045463079999999998,"stat_sum":{"num_bytes":6869723,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":12,"num_read":2269,"num_read_kb":11194,"num_write":421,"num_write_kb":6866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"23"}]},{"pgid":"2.3","version":"20'1","reported_seq":55,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T09:08:26.425749+0000","last_change":"2024-09-14T09:08:06.210516+0000","last_active":"2024-09-14T09:08:26.425749+0000","last_peered":"2024-09-14T09:08:26.425749+0000","last_clean":"2024-09-14T09:08:26.425749+0000","last_became_active":"2024-09-14T09:08:06.205108+0000","last_became_peered":"2024-09-14T09:08:06.205108+0000","last_unstale":"2024-09-14T09:08:26.425749+0000","last_undegraded":"2024-09-14T09:08:26.425749+0000","last_fullsized":"2024-09-14T09:08:26.425749+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:22:05.543865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":137,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-14T09:08:25.413342+0000","last_change":"2024-09-14T09:07:21.876180+0000","last_active":"2024-09-14T09:08:25.413342+0000","last_peered":"2024-09-14T09:08:25.413342+0000","last_clean":"2024-09-14T09:08:25.413342+0000","last_became_active":"2024-09-14T09:07:21.860117+0000","last_became_peered":"2024-09-14T09:07:21.860117+0000","last_unstale":"2024-09-14T09:08:25.413342+0000","last_undegraded":"2024-09-14T09:08:25.413342+0000","last_fullsized":"2024-09-14T09:08:25.413342+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:58:36.142971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"173'677","reported_seq":6917,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-14T09:08:25.400862+0000","last_change":"2024-09-14T09:08:25.400862+0000","last_active":"2024-09-14T09:08:25.400862+0000","last_peered":"2024-09-14T09:08:25.400862+0000","last_clean":"2024-09-14T09:08:25.400862+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:08:25.400862+0000","last_undegraded":"2024-09-14T09:08:25.400862+0000","last_fullsized":"2024-09-14T09:08:25.400862+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:40:15.849503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043321189999999997,"stat_sum":{"num_bytes":387414,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":9,"num_read":7685,"num_read_kb":41729,"num_write":747,"num_write_kb":19769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"23"}]},{"pgid":"3.9","version":"173'464","reported_seq":3099,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T09:08:26.421360+0000","last_change":"2024-09-14T09:08:26.423954+0000","last_active":"2024-09-14T09:08:26.421360+0000","last_peered":"2024-09-14T09:08:26.421360+0000","last_clean":"2024-09-14T09:08:26.421360+0000","last_became_active":"2024-09-14T09:07:06.851020+0000","last_became_peered":"2024-09-14T09:07:06.851020+0000","last_unstale":"2024-09-14T09:08:26.421360+0000","last_undegraded":"2024-09-14T09:08:26.421360+0000","last_fullsized":"2024-09-14T09:08:26.421360+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:06:00.769043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003233302,"stat_sum":{"num_bytes":6274984,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":11,"num_read":2080,"num_read_kb":17783,"num_write":388,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":12315549,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"b","length":"24"}]},{"pgid":"3.a","version":"174'666","reported_seq":4634,"reported_epoch":174,"state":"active+undersized+degraded","last_fresh":"2024-09-14T09:08:26.644063+0000","last_change":"2024-09-14T09:07:48.008657+0000","last_active":"2024-09-14T09:08:26.644063+0000","last_peered":"2024-09-14T09:08:26.644063+0000","last_clean":"2024-09-14T09:07:46.117822+0000","last_became_active":"2024-09-14T09:07:48.008657+0000","last_became_peered":"2024-09-14T09:07:48.008657+0000","last_unstale":"2024-09-14T09:08:26.644063+0000","last_undegraded":"2024-09-14T09:07:48.007810+0000","last_fullsized":"2024-09-14T09:07:48.007673+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":36,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:55:47.487492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185731,"stat_sum":{"num_bytes":40652062,"num_objects":65,"num_object_clones":26,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":65,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":35,"num_read":5316,"num_read_kb":38945,"num_write":857,"num_write_kb":19956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":65}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"173'527","reported_seq":9688,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-14T09:08:25.401653+0000","last_change":"2024-09-14T09:08:25.401653+0000","last_active":"2024-09-14T09:08:25.401653+0000","last_peered":"2024-09-14T09:08:25.401653+0000","last_clean":"2024-09-14T09:08:25.401653+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:08:25.401653+0000","last_undegraded":"2024-09-14T09:08:25.401653+0000","last_fullsized":"2024-09-14T09:08:25.401653+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:08:16.581184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047105189999999998,"stat_sum":{"num_bytes":13737761,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":11,"num_read":10362,"num_read_kb":61098,"num_write":473,"num_write_kb":11296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"23"}]},{"pgid":"3.c","version":"173'489","reported_seq":3279,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T09:08:26.426774+0000","last_change":"2024-09-14T09:08:26.428145+0000","last_active":"2024-09-14T09:08:26.426774+0000","last_peered":"2024-09-14T09:08:26.426774+0000","last_clean":"2024-09-14T09:08:26.426774+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:08:26.426774+0000","last_undegraded":"2024-09-14T09:08:26.426774+0000","last_fullsized":"2024-09-14T09:08:26.426774+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:06:03.770068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001988011,"stat_sum":{"num_bytes":983040,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":9,"num_read":3266,"num_read_kb":13215,"num_write":424,"num_write_kb":10574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"b","length":"24"}]},{"pgid":"3.d","version":"173'497","reported_seq":5719,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-14T09:08:25.416401+0000","last_change":"2024-09-14T09:08:25.416401+0000","last_active":"2024-09-14T09:08:25.416401+0000","last_peered":"2024-09-14T09:08:25.416401+0000","last_clean":"2024-09-14T09:08:25.416401+0000","last_became_active":"2024-09-14T09:07:26.853961+0000","last_became_peered":"2024-09-14T09:07:26.853961+0000","last_unstale":"2024-09-14T09:08:25.416401+0000","last_undegraded":"2024-09-14T09:08:25.416401+0000","last_fullsized":"2024-09-14T09:08:25.416401+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:27:41.551929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023708760000000001,"stat_sum":{"num_bytes":12123336,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":11,"num_read":5399,"num_read_kb":32061,"num_write":547,"num_write_kb":21536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":25396220,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"23"}]},{"pgid":"3.e","version":"173'444","reported_seq":3680,"reported_epoch":173,"state":"active+undersized+degraded","last_fresh":"2024-09-14T09:08:25.414411+0000","last_change":"2024-09-14T09:07:48.003770+0000","last_active":"2024-09-14T09:08:25.414411+0000","last_peered":"2024-09-14T09:08:25.414411+0000","last_clean":"2024-09-14T09:07:46.413543+0000","last_became_active":"2024-09-14T09:07:48.003770+0000","last_became_peered":"2024-09-14T09:07:48.003770+0000","last_unstale":"2024-09-14T09:08:25.414411+0000","last_undegraded":"2024-09-14T09:07:48.002530+0000","last_fullsized":"2024-09-14T09:07:48.002301+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":35,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:47:46.983650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057596100000000005,"stat_sum":{"num_bytes":25984033,"num_objects":55,"num_object_clones":19,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":55,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":30,"num_read":4282,"num_read_kb":17652,"num_write":403,"num_write_kb":7874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":14123394,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":55}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"170'529","reported_seq":3910,"reported_epoch":173,"state":"active+undersized+degraded","last_fresh":"2024-09-14T09:08:25.617664+0000","last_change":"2024-09-14T09:07:48.006091+0000","last_active":"2024-09-14T09:08:25.617664+0000","last_peered":"2024-09-14T09:08:25.617664+0000","last_clean":"2024-09-14T09:07:46.404254+0000","last_became_active":"2024-09-14T09:07:48.006091+0000","last_became_peered":"2024-09-14T09:07:48.006091+0000","last_unstale":"2024-09-14T09:08:25.617664+0000","last_undegraded":"2024-09-14T09:07:48.002841+0000","last_fullsized":"2024-09-14T09:07:48.002611+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":35,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:17:47.324339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000909463,"stat_sum":{"num_bytes":18645010,"num_objects":63,"num_object_clones":26,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":63,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":32,"num_read":3637,"num_read_kb":14847,"num_write":549,"num_write_kb":12244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":333,"num_bytes_recovered":11864981,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":63}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"173'504","reported_seq":6134,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T09:08:26.421470+0000","last_change":"2024-09-14T09:08:26.424405+0000","last_active":"2024-09-14T09:08:26.421470+0000","last_peered":"2024-09-14T09:08:26.421470+0000","last_clean":"2024-09-14T09:08:26.421470+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:08:26.421470+0000","last_undegraded":"2024-09-14T09:08:26.421470+0000","last_fullsized":"2024-09-14T09:08:26.421470+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:39:50.826805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003681325,"stat_sum":{"num_bytes":24679257,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5422,"num_read_kb":60065,"num_write":521,"num_write_kb":20523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"b","length":"24"}]},{"pgid":"3.11","version":"173'544","reported_seq":5695,"reported_epoch":173,"state":"active+undersized+degraded","last_fresh":"2024-09-14T09:08:25.761928+0000","last_change":"2024-09-14T09:07:48.003932+0000","last_active":"2024-09-14T09:08:25.761928+0000","last_peered":"2024-09-14T09:08:25.761928+0000","last_clean":"2024-09-14T09:07:46.117067+0000","last_became_active":"2024-09-14T09:07:48.003932+0000","last_became_peered":"2024-09-14T09:07:48.003932+0000","last_unstale":"2024-09-14T09:08:25.761928+0000","last_undegraded":"2024-09-14T09:07:48.002288+0000","last_fullsized":"2024-09-14T09:07:48.002058+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":35,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:49:11.816845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092949299999999999,"stat_sum":{"num_bytes":10716281,"num_objects":87,"num_object_clones":32,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":87,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":50,"num_read":5926,"num_read_kb":13462,"num_write":614,"num_write_kb":7014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":7405746,"num_keys_recovered":72,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":87}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"174'765","reported_seq":5912,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T09:08:26.660315+0000","last_change":"2024-09-14T09:08:26.432616+0000","last_active":"2024-09-14T09:08:26.660315+0000","last_peered":"2024-09-14T09:08:26.660315+0000","last_clean":"2024-09-14T09:08:26.660315+0000","last_became_active":"2024-09-14T09:07:26.101586+0000","last_became_peered":"2024-09-14T09:07:26.101586+0000","last_unstale":"2024-09-14T09:08:26.660315+0000","last_undegraded":"2024-09-14T09:08:26.660315+0000","last_fullsized":"2024-09-14T09:08:26.660315+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":765,"log_dups_size":0,"ondisk_log_size":765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:03:47.472921+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.017400677,"stat_sum":{"num_bytes":9386848,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":9,"num_read":6842,"num_read_kb":19344,"num_write":974,"num_write_kb":12517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":10260744,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"b","length":"24"}]},{"pgid":"3.13","version":"173'480","reported_seq":6222,"reported_epoch":173,"state":"active+undersized+degraded","last_fresh":"2024-09-14T09:08:25.603572+0000","last_change":"2024-09-14T09:07:48.003956+0000","last_active":"2024-09-14T09:08:25.603572+0000","last_peered":"2024-09-14T09:08:25.603572+0000","last_clean":"2024-09-14T09:07:46.404247+0000","last_became_active":"2024-09-14T09:07:48.003956+0000","last_became_peered":"2024-09-14T09:07:48.003956+0000","last_unstale":"2024-09-14T09:08:25.603572+0000","last_undegraded":"2024-09-14T09:07:48.002482+0000","last_fullsized":"2024-09-14T09:07:48.002272+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":35,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:05:06.430454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061978900000000004,"stat_sum":{"num_bytes":17227051,"num_objects":63,"num_object_clones":26,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":63,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":33,"num_read":6839,"num_read_kb":29296,"num_write":445,"num_write_kb":10682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":24080444,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":63}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":421242445,"num_objects":1015,"num_object_clones":332,"num_object_copies":2030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":727,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1015,"num_whiteouts":590,"num_read":195895,"num_read_kb":858235,"num_write":24705,"num_write_kb":385822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3800,"num_bytes_recovered":201826576,"num_keys_recovered":652,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20389888,"data_stored":158190861,"data_compressed":5754519,"data_compressed_allocated":14659584,"data_compressed_original":154714388,"omap_allocated":0,"internal_metadata":0},"log_size":21252,"ondisk_log_size":21252,"up":53,"acting":53,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20292,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509483,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30840,"kb_used_data":3808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341000,"statfs":{"total":96636764160,"available":96605184000,"internally_reserved":0,"allocated":3899392,"data_stored":17867551,"data_compressed":510294,"data_compressed_allocated":1515520,"data_compressed_original":16707584,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509483,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79096,"kb_used_data":3492,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94292744,"statfs":{"total":96636764160,"available":96555769856,"internally_reserved":0,"allocated":3575808,"data_stored":12737938,"data_compressed":423550,"data_compressed_allocated":1077248,"data_compressed_original":11498647,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509484,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47548,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324292,"statfs":{"total":96636764160,"available":96588075008,"internally_reserved":0,"allocated":2531328,"data_stored":5351460,"data_compressed":179622,"data_compressed_allocated":434176,"data_compressed_original":4337664,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509486,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61420,"kb_used_data":3864,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310420,"statfs":{"total":96636764160,"available":96573870080,"internally_reserved":0,"allocated":3956736,"data_stored":16130923,"data_compressed":407071,"data_compressed_allocated":1306624,"data_compressed_original":14807072,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542186,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53888,"kb_used_data":4632,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317952,"statfs":{"total":96636764160,"available":96581582848,"internally_reserved":0,"allocated":4743168,"data_stored":18999670,"data_compressed":749983,"data_compressed_allocated":1814528,"data_compressed_original":17496456,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542179,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61120,"kb_used_data":11884,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94310720,"statfs":{"total":96636764160,"available":96574177280,"internally_reserved":0,"allocated":12169216,"data_stored":87056972,"data_compressed":2906952,"data_compressed_allocated":7544832,"data_compressed_original":84363515,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542187,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60216,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311624,"statfs":{"total":96636764160,"available":96575102976,"internally_reserved":0,"allocated":2691072,"data_stored":3667344,"data_compressed":235000,"data_compressed_allocated":409600,"data_compressed_original":2568488,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56096,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315744,"statfs":{"total":96636764160,"available":96579321856,"internally_reserved":0,"allocated":2768896,"data_stored":2591470,"data_compressed":282359,"data_compressed_allocated":385024,"data_compressed_original":1417795,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1810432,"data_stored":14535722,"data_compressed":400859,"data_compressed_allocated":1269760,"data_compressed_original":14233632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":688128,"data_stored":1592961,"data_compressed":282359,"data_compressed_allocated":385024,"data_compressed_original":1417795,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":4221898,"data_compressed":138805,"data_compressed_allocated":389120,"data_compressed_original":4206592,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":614400,"data_stored":2668854,"data_compressed":235000,"data_compressed_allocated":409600,"data_compressed_original":2568488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10948608,"data_stored":89195710,"data_compressed":3020772,"data_compressed_allocated":7839744,"data_compressed_original":87207786,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1478656,"data_stored":11728775,"data_compressed":423550,"data_compressed_allocated":1077248,"data_compressed_original":11498647,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1814528,"data_stored":16836129,"data_compressed":510231,"data_compressed_allocated":1511424,"data_compressed_original":16674816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2625536,"data_stored":17410812,"data_compressed":742943,"data_compressed_allocated":1777664,"data_compressed_original":16906632,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:08:29.220 INFO:tasks.thrashosds.thrasher:Setting 2.0 to [4, 2] 2024-09-14T09:08:29.220 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.0', '4', '2'] 2024-09-14T09:08:29.220 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.0 4 2 2024-09-14T09:08:29.221 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:29.219+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:29.309 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:29.310+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:29.333 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:08:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:08:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:08:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73542 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15319040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:08:29.333 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.063054427, 2024-09-14T09:08:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0237091359999999, 2024-09-14T09:08:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73542 2024-09-14T09:08:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.334 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:08:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:08:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268762+0000", 2024-09-14T09:08:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:08:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268766+0000", 2024-09-14T09:08:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5880000000000002e-06 2024-09-14T09:08:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268776+0000", 2024-09-14T09:08:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0168e-05 2024-09-14T09:08:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268789+0000", 2024-09-14T09:08:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:08:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282053+0000", 2024-09-14T09:08:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013264496000000001 2024-09-14T09:08:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282057+0000", 2024-09-14T09:08:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6090000000000001e-06 2024-09-14T09:08:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292052+0000", 2024-09-14T09:08:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0099950070000001 2024-09-14T09:08:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292262+0000", 2024-09-14T09:08:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021009800000000001 2024-09-14T09:08:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292470+0000", 2024-09-14T09:08:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000208314 2024-09-14T09:08:29.338 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73543 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:08:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.06301563700001, 2024-09-14T09:08:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023985473, 2024-09-14T09:08:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.339 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73543 2024-09-14T09:08:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:08:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:08:29.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268801+0000", 2024-09-14T09:08:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:08:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268803+0000", 2024-09-14T09:08:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7439999999999999e-06 2024-09-14T09:08:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268812+0000", 2024-09-14T09:08:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7080000000000003e-06 2024-09-14T09:08:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268819+0000", 2024-09-14T09:08:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T09:08:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282068+0000", 2024-09-14T09:08:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013249540000000001 2024-09-14T09:08:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282072+0000", 2024-09-14T09:08:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0960000000000001e-06 2024-09-14T09:08:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292573+0000", 2024-09-14T09:08:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0105014560000001 2024-09-14T09:08:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292679+0000", 2024-09-14T09:08:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010545899999999999 2024-09-14T09:08:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292786+0000", 2024-09-14T09:08:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107052 2024-09-14T09:08:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73544 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.062987935, 2024-09-14T09:08:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0244830549999999, 2024-09-14T09:08:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.344 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73544 2024-09-14T09:08:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.345 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5599999999999995e-07 2024-09-14T09:08:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268831+0000", 2024-09-14T09:08:29.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.57e-06 2024-09-14T09:08:29.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268844+0000", 2024-09-14T09:08:29.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2811e-05 2024-09-14T09:08:29.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268851+0000", 2024-09-14T09:08:29.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9140000000000002e-06 2024-09-14T09:08:29.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282083+0000", 2024-09-14T09:08:29.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013232111 2024-09-14T09:08:29.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282086+0000", 2024-09-14T09:08:29.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2909999999999999e-06 2024-09-14T09:08:29.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292884+0000", 2024-09-14T09:08:29.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0107975069999999 2024-09-14T09:08:29.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293100+0000", 2024-09-14T09:08:29.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021639399999999999 2024-09-14T09:08:29.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.350 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.350 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293311+0000", 2024-09-14T09:08:29.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021090099999999999 2024-09-14T09:08:29.351 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.351 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.351 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.351 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73545 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15564800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:29.351 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:08:29.351 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.06293675000001, 2024-09-14T09:08:29.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0249746740000001, 2024-09-14T09:08:29.351 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.351 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.351 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.352 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.352 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.352 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.352 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73545 2024-09-14T09:08:29.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.352 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:08:29.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:08:29.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268880+0000", 2024-09-14T09:08:29.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5600000000000005e-07 2024-09-14T09:08:29.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268889+0000", 2024-09-14T09:08:29.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8559999999999996e-06 2024-09-14T09:08:29.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268913+0000", 2024-09-14T09:08:29.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3958e-05 2024-09-14T09:08:29.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268921+0000", 2024-09-14T09:08:29.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7479999999999999e-06 2024-09-14T09:08:29.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.354 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282098+0000", 2024-09-14T09:08:29.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013176297999999999 2024-09-14T09:08:29.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282101+0000", 2024-09-14T09:08:29.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2329999999999998e-06 2024-09-14T09:08:29.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293435+0000", 2024-09-14T09:08:29.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0113339459999999 2024-09-14T09:08:29.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293648+0000", 2024-09-14T09:08:29.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021307100000000001 2024-09-14T09:08:29.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293854+0000", 2024-09-14T09:08:29.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020580800000000001 2024-09-14T09:08:29.356 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.356 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.356 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.356 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73546 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15646720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:29.356 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:08:29.356 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.062882879, 2024-09-14T09:08:29.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0254846360000001, 2024-09-14T09:08:29.356 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.357 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.357 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.357 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.357 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73546 2024-09-14T09:08:29.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.357 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:08:29.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:08:29.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268934+0000", 2024-09-14T09:08:29.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7999999999999995e-07 2024-09-14T09:08:29.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268937+0000", 2024-09-14T09:08:29.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9840000000000001e-06 2024-09-14T09:08:29.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268947+0000", 2024-09-14T09:08:29.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0522e-05 2024-09-14T09:08:29.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268955+0000", 2024-09-14T09:08:29.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8059999999999995e-06 2024-09-14T09:08:29.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282112+0000", 2024-09-14T09:08:29.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013157189 2024-09-14T09:08:29.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282115+0000", 2024-09-14T09:08:29.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2949999999999998e-06 2024-09-14T09:08:29.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293968+0000", 2024-09-14T09:08:29.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0118530990000001 2024-09-14T09:08:29.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294197+0000", 2024-09-14T09:08:29.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228507 2024-09-14T09:08:29.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294417+0000", 2024-09-14T09:08:29.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000220554 2024-09-14T09:08:29.361 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.361 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.361 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.362 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73547 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:29.362 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:08:29.362 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.06285043299999, 2024-09-14T09:08:29.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.025970525, 2024-09-14T09:08:29.362 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.362 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.362 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.362 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.362 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.362 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.362 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73547 2024-09-14T09:08:29.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.362 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:08:29.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:08:29.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268966+0000", 2024-09-14T09:08:29.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3799999999999997e-07 2024-09-14T09:08:29.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268969+0000", 2024-09-14T09:08:29.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8739999999999999e-06 2024-09-14T09:08:29.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268979+0000", 2024-09-14T09:08:29.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9939999999999995e-06 2024-09-14T09:08:29.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268986+0000", 2024-09-14T09:08:29.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3459999999999996e-06 2024-09-14T09:08:29.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282127+0000", 2024-09-14T09:08:29.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01314039 2024-09-14T09:08:29.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282130+0000", 2024-09-14T09:08:29.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2219999999999998e-06 2024-09-14T09:08:29.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294540+0000", 2024-09-14T09:08:29.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124105880000001 2024-09-14T09:08:29.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294759+0000", 2024-09-14T09:08:29.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000218798 2024-09-14T09:08:29.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294936+0000", 2024-09-14T09:08:29.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000176675 2024-09-14T09:08:29.367 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.367 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.367 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.367 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73548 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15810560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:29.367 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:08:29.367 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.06281932100001, 2024-09-14T09:08:29.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0265045129999999, 2024-09-14T09:08:29.367 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.367 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.367 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.367 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.368 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.368 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.368 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73548 2024-09-14T09:08:29.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.368 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:08:29.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:08:29.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268997+0000", 2024-09-14T09:08:29.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7599999999999996e-07 2024-09-14T09:08:29.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269000+0000", 2024-09-14T09:08:29.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0580000000000002e-06 2024-09-14T09:08:29.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269010+0000", 2024-09-14T09:08:29.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124e-05 2024-09-14T09:08:29.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269018+0000", 2024-09-14T09:08:29.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5399999999999998e-06 2024-09-14T09:08:29.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282141+0000", 2024-09-14T09:08:29.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013123012 2024-09-14T09:08:29.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282144+0000", 2024-09-14T09:08:29.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.196e-06 2024-09-14T09:08:29.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295042+0000", 2024-09-14T09:08:29.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0128975689999999 2024-09-14T09:08:29.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295272+0000", 2024-09-14T09:08:29.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023048099999999999 2024-09-14T09:08:29.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.372 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.372 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295501+0000", 2024-09-14T09:08:29.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228657 2024-09-14T09:08:29.372 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.372 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.372 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.372 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73549 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15892480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:29.372 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:08:29.372 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.06277344199999, 2024-09-14T09:08:29.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0268354099999999, 2024-09-14T09:08:29.373 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.373 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.373 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.373 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.373 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.373 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.373 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73549 2024-09-14T09:08:29.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.373 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:08:29.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:08:29.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269043+0000", 2024-09-14T09:08:29.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6199999999999997e-07 2024-09-14T09:08:29.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269046+0000", 2024-09-14T09:08:29.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2480000000000001e-06 2024-09-14T09:08:29.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269057+0000", 2024-09-14T09:08:29.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0436e-05 2024-09-14T09:08:29.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269064+0000", 2024-09-14T09:08:29.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5179999999999998e-06 2024-09-14T09:08:29.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282155+0000", 2024-09-14T09:08:29.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013091190000000001 2024-09-14T09:08:29.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282159+0000", 2024-09-14T09:08:29.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.366e-06 2024-09-14T09:08:29.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295599+0000", 2024-09-14T09:08:29.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0134405559999999 2024-09-14T09:08:29.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295765+0000", 2024-09-14T09:08:29.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016575499999999999 2024-09-14T09:08:29.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.377 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.377 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295878+0000", 2024-09-14T09:08:29.377 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112679 2024-09-14T09:08:29.377 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.377 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.377 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.377 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.377 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.377 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73550 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15974400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:29.377 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:08:29.377 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.062740842, 2024-09-14T09:08:29.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027357657, 2024-09-14T09:08:29.378 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.378 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.378 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.378 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.378 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.378 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.378 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73550 2024-09-14T09:08:29.378 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.378 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.378 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.378 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.378 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:08:29.378 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:08:29.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269076+0000", 2024-09-14T09:08:29.379 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:08:29.379 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.379 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.379 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.379 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269079+0000", 2024-09-14T09:08:29.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:08:29.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269098+0000", 2024-09-14T09:08:29.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9698e-05 2024-09-14T09:08:29.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.380 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.380 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.380 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269106+0000", 2024-09-14T09:08:29.380 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6179999999999997e-06 2024-09-14T09:08:29.380 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282170+0000", 2024-09-14T09:08:29.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013063874 2024-09-14T09:08:29.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282173+0000", 2024-09-14T09:08:29.381 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1999999999999999e-06 2024-09-14T09:08:29.381 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.381 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.381 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.381 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295996+0000", 2024-09-14T09:08:29.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.013823151 2024-09-14T09:08:29.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296214+0000", 2024-09-14T09:08:29.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021817700000000001 2024-09-14T09:08:29.382 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.382 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.382 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.382 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296433+0000", 2024-09-14T09:08:29.382 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021829099999999999 2024-09-14T09:08:29.382 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.382 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.382 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.383 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.383 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73551 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:29.383 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:08:29.383 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.06269942500001, 2024-09-14T09:08:29.383 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027730276, 2024-09-14T09:08:29.383 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.383 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.383 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.383 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.383 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.383 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.383 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73551 2024-09-14T09:08:29.383 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.384 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:08:29.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:08:29.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.384 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.384 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.384 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.384 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269117+0000", 2024-09-14T09:08:29.384 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:08:29.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269120+0000", 2024-09-14T09:08:29.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.056e-06 2024-09-14T09:08:29.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.385 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269130+0000", 2024-09-14T09:08:29.385 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0084e-05 2024-09-14T09:08:29.385 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.385 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.385 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269137+0000", 2024-09-14T09:08:29.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1359999999999999e-06 2024-09-14T09:08:29.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282184+0000", 2024-09-14T09:08:29.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013046537 2024-09-14T09:08:29.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.386 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.386 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.386 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282188+0000", 2024-09-14T09:08:29.386 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3220000000000003e-06 2024-09-14T09:08:29.386 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296529+0000", 2024-09-14T09:08:29.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0143406109999999 2024-09-14T09:08:29.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296695+0000", 2024-09-14T09:08:29.387 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016604799999999999 2024-09-14T09:08:29.387 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.387 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.387 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.387 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296847+0000", 2024-09-14T09:08:29.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000151822 2024-09-14T09:08:29.388 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.388 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.388 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.388 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.388 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.388 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73552 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:29.388 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:29.388 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.062668845, 2024-09-14T09:08:29.388 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0281767500000001, 2024-09-14T09:08:29.388 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.388 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.388 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.389 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.389 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73552 2024-09-14T09:08:29.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.389 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:29.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.389 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.389 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.389 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.389 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:29.389 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:29.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:08:29.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269150+0000", 2024-09-14T09:08:29.390 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9119999999999998e-06 2024-09-14T09:08:29.390 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.390 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.390 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.390 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269160+0000", 2024-09-14T09:08:29.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6819999999999998e-06 2024-09-14T09:08:29.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269168+0000", 2024-09-14T09:08:29.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6079999999999998e-06 2024-09-14T09:08:29.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.391 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282205+0000", 2024-09-14T09:08:29.391 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013036982000000001 2024-09-14T09:08:29.391 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.391 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.391 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282208+0000", 2024-09-14T09:08:29.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4620000000000001e-06 2024-09-14T09:08:29.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296968+0000", 2024-09-14T09:08:29.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0147601209999999 2024-09-14T09:08:29.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.392 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297115+0000", 2024-09-14T09:08:29.392 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014673900000000001 2024-09-14T09:08:29.392 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.392 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.392 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.393 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297324+0000", 2024-09-14T09:08:29.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020866400000000001 2024-09-14T09:08:29.393 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.393 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.393 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.393 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.393 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.393 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73553 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:29.393 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:08:29.393 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.062638781, 2024-09-14T09:08:29.393 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0286556739999999, 2024-09-14T09:08:29.393 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.393 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.393 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.394 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.394 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73553 2024-09-14T09:08:29.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.394 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:08:29.394 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.394 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.394 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.394 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.394 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:08:29.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269178+0000", 2024-09-14T09:08:29.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:08:29.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.395 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269180+0000", 2024-09-14T09:08:29.395 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T09:08:29.395 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.395 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.395 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269189+0000", 2024-09-14T09:08:29.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2e-06 2024-09-14T09:08:29.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269197+0000", 2024-09-14T09:08:29.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2980000000000003e-06 2024-09-14T09:08:29.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.396 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282219+0000", 2024-09-14T09:08:29.396 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013022687 2024-09-14T09:08:29.396 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.396 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.396 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282223+0000", 2024-09-14T09:08:29.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.241e-06 2024-09-14T09:08:29.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297470+0000", 2024-09-14T09:08:29.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.015247778 2024-09-14T09:08:29.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.397 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297704+0000", 2024-09-14T09:08:29.397 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000233367 2024-09-14T09:08:29.397 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.397 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.397 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.398 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297833+0000", 2024-09-14T09:08:29.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012888900000000001 2024-09-14T09:08:29.398 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.398 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.398 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.398 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.398 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.398 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73554 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:29.398 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:08:29.398 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.062609464, 2024-09-14T09:08:29.398 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.029005113, 2024-09-14T09:08:29.398 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.398 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.398 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.399 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.399 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.399 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.399 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73554 2024-09-14T09:08:29.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.399 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.399 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:08:29.399 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.399 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.399 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.399 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:08:29.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269207+0000", 2024-09-14T09:08:29.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.68e-07 2024-09-14T09:08:29.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.400 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269210+0000", 2024-09-14T09:08:29.400 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8100000000000002e-06 2024-09-14T09:08:29.400 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.400 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.400 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269219+0000", 2024-09-14T09:08:29.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:08:29.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269227+0000", 2024-09-14T09:08:29.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3839999999999999e-06 2024-09-14T09:08:29.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.401 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282234+0000", 2024-09-14T09:08:29.401 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013007187 2024-09-14T09:08:29.401 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.401 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.401 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282237+0000", 2024-09-14T09:08:29.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1200000000000002e-06 2024-09-14T09:08:29.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297942+0000", 2024-09-14T09:08:29.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0157048040000001 2024-09-14T09:08:29.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.402 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298087+0000", 2024-09-14T09:08:29.402 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014499099999999999 2024-09-14T09:08:29.402 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.402 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.402 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.403 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298211+0000", 2024-09-14T09:08:29.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012461899999999999 2024-09-14T09:08:29.403 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.403 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.403 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.403 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.403 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.403 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73555 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:29.403 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:08:29.403 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.06257961599999, 2024-09-14T09:08:29.403 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.02941558, 2024-09-14T09:08:29.403 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.403 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.403 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.403 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.404 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.404 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.404 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73555 2024-09-14T09:08:29.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.404 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:08:29.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.404 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.404 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.404 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.404 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:08:29.404 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269237+0000", 2024-09-14T09:08:29.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3399999999999999e-07 2024-09-14T09:08:29.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269240+0000", 2024-09-14T09:08:29.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8480000000000001e-06 2024-09-14T09:08:29.405 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.405 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.405 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.405 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269249+0000", 2024-09-14T09:08:29.405 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1880000000000008e-06 2024-09-14T09:08:29.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269256+0000", 2024-09-14T09:08:29.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:08:29.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282248+0000", 2024-09-14T09:08:29.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012992327 2024-09-14T09:08:29.406 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.406 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.406 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.406 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282251+0000", 2024-09-14T09:08:29.406 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2160000000000002e-06 2024-09-14T09:08:29.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298299+0000", 2024-09-14T09:08:29.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0160479330000001 2024-09-14T09:08:29.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298446+0000", 2024-09-14T09:08:29.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000146525 2024-09-14T09:08:29.407 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.407 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.407 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.407 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298652+0000", 2024-09-14T09:08:29.407 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020584100000000001 2024-09-14T09:08:29.408 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.408 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.408 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.408 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.408 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.408 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73556 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:29.408 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:29.408 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.06254987, 2024-09-14T09:08:29.408 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0299854239999999, 2024-09-14T09:08:29.408 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.408 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.408 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.408 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.408 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.408 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.409 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73556 2024-09-14T09:08:29.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.409 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:29.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.409 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:29.409 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.409 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.409 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.409 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:29.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:08:29.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269270+0000", 2024-09-14T09:08:29.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0920000000000002e-06 2024-09-14T09:08:29.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.410 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269280+0000", 2024-09-14T09:08:29.410 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0013999999999999e-05 2024-09-14T09:08:29.410 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.410 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.410 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269288+0000", 2024-09-14T09:08:29.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:08:29.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282263+0000", 2024-09-14T09:08:29.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012974703000000001 2024-09-14T09:08:29.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.411 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.411 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282266+0000", 2024-09-14T09:08:29.411 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1520000000000001e-06 2024-09-14T09:08:29.411 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.411 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298729+0000", 2024-09-14T09:08:29.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.016463506 2024-09-14T09:08:29.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298884+0000", 2024-09-14T09:08:29.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015450800000000001 2024-09-14T09:08:29.412 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.412 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.412 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.412 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299251+0000", 2024-09-14T09:08:29.412 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036759899999999997 2024-09-14T09:08:29.412 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.413 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.413 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.413 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.413 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.413 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73557 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16547840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:29.413 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:08:29.413 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.06250322, 2024-09-14T09:08:29.413 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0307969480000001, 2024-09-14T09:08:29.413 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.413 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.413 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.413 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.413 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.413 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.414 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73557 2024-09-14T09:08:29.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.414 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:08:29.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.414 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:08:29.414 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.414 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.414 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.414 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269313+0000", 2024-09-14T09:08:29.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.92e-07 2024-09-14T09:08:29.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269336+0000", 2024-09-14T09:08:29.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3118000000000001e-05 2024-09-14T09:08:29.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.415 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269346+0000", 2024-09-14T09:08:29.415 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4069999999999999e-06 2024-09-14T09:08:29.415 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.415 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.415 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269352+0000", 2024-09-14T09:08:29.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5810000000000002e-06 2024-09-14T09:08:29.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282277+0000", 2024-09-14T09:08:29.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012924658 2024-09-14T09:08:29.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.416 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.416 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282280+0000", 2024-09-14T09:08:29.416 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2880000000000001e-06 2024-09-14T09:08:29.416 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.416 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299343+0000", 2024-09-14T09:08:29.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0170630190000001 2024-09-14T09:08:29.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299554+0000", 2024-09-14T09:08:29.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021085099999999999 2024-09-14T09:08:29.417 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.417 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.417 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.417 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300109+0000", 2024-09-14T09:08:29.417 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00055533399999999999 2024-09-14T09:08:29.417 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.417 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.418 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.418 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.418 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.418 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73558 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:29.418 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:08:29.418 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.06245159700001, 2024-09-14T09:08:29.418 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0314849340000001, 2024-09-14T09:08:29.418 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.418 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.418 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.418 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.418 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.418 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.418 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73558 2024-09-14T09:08:29.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.419 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:08:29.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.419 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:08:29.419 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.419 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.419 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.419 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269365+0000", 2024-09-14T09:08:29.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7400000000000003e-07 2024-09-14T09:08:29.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269367+0000", 2024-09-14T09:08:29.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4930000000000001e-06 2024-09-14T09:08:29.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.420 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.420 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.420 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269374+0000", 2024-09-14T09:08:29.420 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9519999999999997e-06 2024-09-14T09:08:29.420 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269379+0000", 2024-09-14T09:08:29.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9710000000000003e-06 2024-09-14T09:08:29.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.421 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282292+0000", 2024-09-14T09:08:29.421 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012912704000000001 2024-09-14T09:08:29.421 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.421 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.421 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282295+0000", 2024-09-14T09:08:29.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.179e-06 2024-09-14T09:08:29.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300195+0000", 2024-09-14T09:08:29.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.017899975 2024-09-14T09:08:29.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.422 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.422 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300345+0000", 2024-09-14T09:08:29.422 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014957499999999999 2024-09-14T09:08:29.422 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.422 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.423 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.423 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300849+0000", 2024-09-14T09:08:29.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050451100000000002 2024-09-14T09:08:29.423 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.423 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.423 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.423 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.423 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.423 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73559 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:29.423 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:29.423 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 115.062429188, 2024-09-14T09:08:29.423 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.031916343, 2024-09-14T09:08:29.423 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.423 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.423 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.423 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.424 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.424 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.424 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73559 2024-09-14T09:08:29.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.424 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:29.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.424 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.424 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.424 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:29.424 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.424 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:29.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3500000000000002e-07 2024-09-14T09:08:29.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269390+0000", 2024-09-14T09:08:29.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5890000000000001e-06 2024-09-14T09:08:29.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.425 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.425 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269396+0000", 2024-09-14T09:08:29.425 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3300000000000004e-06 2024-09-14T09:08:29.425 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.425 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269401+0000", 2024-09-14T09:08:29.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0340000000000004e-06 2024-09-14T09:08:29.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282311+0000", 2024-09-14T09:08:29.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012910204 2024-09-14T09:08:29.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.426 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.426 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.426 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282317+0000", 2024-09-14T09:08:29.426 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2639999999999997e-06 2024-09-14T09:08:29.426 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300933+0000", 2024-09-14T09:08:29.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0186160710000001 2024-09-14T09:08:29.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301158+0000", 2024-09-14T09:08:29.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225021 2024-09-14T09:08:29.427 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.427 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.427 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.427 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301303+0000", 2024-09-14T09:08:29.427 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014439499999999999 2024-09-14T09:08:29.428 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.428 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.428 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.428 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.428 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.428 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106964 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:29.428 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:08:29.428 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.216501354000002, 2024-09-14T09:08:29.428 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8802924409999999, 2024-09-14T09:08:29.428 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.428 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.428 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:29.428 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.429 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.429 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.429 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106964 2024-09-14T09:08:29.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.429 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:08:29.429 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.429 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.429 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.429 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.429 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:08:29.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115317+0000", 2024-09-14T09:08:29.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6429999999999999e-06 2024-09-14T09:08:29.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.430 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.430 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115326+0000", 2024-09-14T09:08:29.430 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2140000000000002e-06 2024-09-14T09:08:29.430 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.430 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115346+0000", 2024-09-14T09:08:29.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9419000000000002e-05 2024-09-14T09:08:29.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115357+0000", 2024-09-14T09:08:29.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1014e-05 2024-09-14T09:08:29.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.431 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.431 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115431+0000", 2024-09-14T09:08:29.431 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4040000000000003e-05 2024-09-14T09:08:29.431 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.431 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115438+0000", 2024-09-14T09:08:29.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9920000000000001e-06 2024-09-14T09:08:29.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844944+0000", 2024-09-14T09:08:29.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72950628399999995 2024-09-14T09:08:29.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.432 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.432 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.432 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844961+0000", 2024-09-14T09:08:29.432 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7156e-05 2024-09-14T09:08:29.432 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988004+0000", 2024-09-14T09:08:29.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.14304297 2024-09-14T09:08:29.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988511+0000", 2024-09-14T09:08:29.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050694100000000001 2024-09-14T09:08:29.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.433 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.433 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:08:29.433 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.989802+0000", 2024-09-14T09:08:29.433 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001291195 2024-09-14T09:08:29.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:29.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994866+0000", 2024-09-14T09:08:29.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050632760000000002 2024-09-14T09:08:29.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:29.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994988+0000", 2024-09-14T09:08:29.434 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012252899999999999 2024-09-14T09:08:29.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.434 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.434 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:29.434 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995198+0000", 2024-09-14T09:08:29.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021010899999999999 2024-09-14T09:08:29.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.435 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.435 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995607+0000", 2024-09-14T09:08:29.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040865900000000002 2024-09-14T09:08:29.435 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.435 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.435 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.435 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.435 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.435 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106965 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:29.435 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:29.435 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.215298977000003, 2024-09-14T09:08:29.435 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.879730162, 2024-09-14T09:08:29.435 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.436 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.436 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:29.436 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.436 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.436 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.436 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106965 2024-09-14T09:08:29.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.436 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.436 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.436 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:29.436 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.436 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.436 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:29.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:29.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:08:29.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.437 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.437 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.437 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116522+0000", 2024-09-14T09:08:29.437 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3989999999999997e-06 2024-09-14T09:08:29.437 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116533+0000", 2024-09-14T09:08:29.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0669000000000001e-05 2024-09-14T09:08:29.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116541+0000", 2024-09-14T09:08:29.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2730000000000002e-06 2024-09-14T09:08:29.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.438 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.438 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.438 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116604+0000", 2024-09-14T09:08:29.438 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2706999999999994e-05 2024-09-14T09:08:29.438 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116610+0000", 2024-09-14T09:08:29.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8630000000000002e-06 2024-09-14T09:08:29.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845060+0000", 2024-09-14T09:08:29.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72845003699999999 2024-09-14T09:08:29.439 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.439 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.439 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.439 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845079+0000", 2024-09-14T09:08:29.439 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8287999999999998e-05 2024-09-14T09:08:29.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.991866+0000", 2024-09-14T09:08:29.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1467871519999999 2024-09-14T09:08:29.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.992274+0000", 2024-09-14T09:08:29.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040813500000000001 2024-09-14T09:08:29.440 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.440 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.440 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:08:29.440 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.993538+0000", 2024-09-14T09:08:29.440 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001263655 2024-09-14T09:08:29.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:29.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995770+0000", 2024-09-14T09:08:29.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002231949 2024-09-14T09:08:29.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:29.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995810+0000", 2024-09-14T09:08:29.441 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0852999999999997e-05 2024-09-14T09:08:29.441 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.441 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.441 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:29.441 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995979+0000", 2024-09-14T09:08:29.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016866 2024-09-14T09:08:29.442 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.442 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.442 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.442 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996247+0000", 2024-09-14T09:08:29.442 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000267769 2024-09-14T09:08:29.442 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.442 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.442 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.442 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.442 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.442 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:29.444 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:08:29.444 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:29.443+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:29.545 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:29.546+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:29.618 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:29.618 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:29.618 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:29.618 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:29.634 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:08:29.678 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:29.679+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:29.747 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:29.750+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:29.749 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:29.749 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:29.749 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:08:29.749 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:08:29.750 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:29.763 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:08:29.854 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:29.855+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:29.934 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:29.934 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:08:29.934 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:08:29.935 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:08:29.935 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.935 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106705 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 518144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:29.935 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:29.935 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085712841000003, 2024-09-14T09:08:29.935 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3001326799999999, 2024-09-14T09:08:29.935 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.935 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.935 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.935 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.935 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.935 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.935 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106705 2024-09-14T09:08:29.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.936 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:29.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.936 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.936 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:29.936 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.936 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.936 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:29.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4699999999999998e-07 2024-09-14T09:08:29.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847500+0000", 2024-09-14T09:08:29.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0860000000000001e-06 2024-09-14T09:08:29.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.937 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.937 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.937 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847503+0000", 2024-09-14T09:08:29.937 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1269999999999999e-06 2024-09-14T09:08:29.937 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847506+0000", 2024-09-14T09:08:29.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5780000000000001e-06 2024-09-14T09:08:29.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858911+0000", 2024-09-14T09:08:29.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011404925 2024-09-14T09:08:29.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.938 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.938 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.938 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858919+0000", 2024-09-14T09:08:29.938 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8620000000000003e-06 2024-09-14T09:08:29.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147254+0000", 2024-09-14T09:08:29.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2883353719999999 2024-09-14T09:08:29.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147408+0000", 2024-09-14T09:08:29.939 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015389200000000001 2024-09-14T09:08:29.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.939 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.939 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.939 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147631+0000", 2024-09-14T09:08:29.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022359100000000001 2024-09-14T09:08:29.940 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.940 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.940 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.940 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.940 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.940 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106707 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 522240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:29.940 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:29.940 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085702370999996, 2024-09-14T09:08:29.940 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.300471801, 2024-09-14T09:08:29.940 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.940 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.940 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.940 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.941 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.941 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.941 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106707 2024-09-14T09:08:29.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.941 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:29.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.941 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.941 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.941 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:29.941 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:29.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.23e-07 2024-09-14T09:08:29.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.942 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847514+0000", 2024-09-14T09:08:29.942 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9150000000000004e-06 2024-09-14T09:08:29.942 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.942 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.942 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847520+0000", 2024-09-14T09:08:29.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.203e-06 2024-09-14T09:08:29.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847532+0000", 2024-09-14T09:08:29.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2441e-05 2024-09-14T09:08:29.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.943 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.943 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858953+0000", 2024-09-14T09:08:29.943 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01142108 2024-09-14T09:08:29.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.943 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858957+0000", 2024-09-14T09:08:29.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2830000000000003e-06 2024-09-14T09:08:29.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147667+0000", 2024-09-14T09:08:29.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2887094619999999 2024-09-14T09:08:29.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.944 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.944 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147784+0000", 2024-09-14T09:08:29.944 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117465 2024-09-14T09:08:29.944 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.944 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.945 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.945 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147981+0000", 2024-09-14T09:08:29.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019672900000000001 2024-09-14T09:08:29.945 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.945 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.945 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.945 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.945 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106709 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 3072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:29.945 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:08:29.945 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085674080000004, 2024-09-14T09:08:29.945 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.301335192, 2024-09-14T09:08:29.945 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.946 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.946 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.946 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.946 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106709 2024-09-14T09:08:29.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.946 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.946 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.946 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:08:29.946 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.946 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.946 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:08:29.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847538+0000", 2024-09-14T09:08:29.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2800000000000003e-07 2024-09-14T09:08:29.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.947 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.947 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.947 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847539+0000", 2024-09-14T09:08:29.947 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2720000000000001e-06 2024-09-14T09:08:29.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847544+0000", 2024-09-14T09:08:29.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7219999999999999e-06 2024-09-14T09:08:29.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847547+0000", 2024-09-14T09:08:29.948 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9960000000000002e-06 2024-09-14T09:08:29.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.948 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.948 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.948 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858991+0000", 2024-09-14T09:08:29.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011444218000000001 2024-09-14T09:08:29.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858999+0000", 2024-09-14T09:08:29.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9419999999999995e-06 2024-09-14T09:08:29.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.949 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148053+0000", 2024-09-14T09:08:29.949 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2890544099999999 2024-09-14T09:08:29.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.949 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.949 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148203+0000", 2024-09-14T09:08:29.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149419 2024-09-14T09:08:29.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.950 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.950 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148873+0000", 2024-09-14T09:08:29.950 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00066988499999999995 2024-09-14T09:08:29.950 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.950 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.950 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.950 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.950 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106711 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 7168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:29.950 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:29.950 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085660821999994, 2024-09-14T09:08:29.951 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3022241219999999, 2024-09-14T09:08:29.951 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.951 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.951 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.951 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.951 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.951 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.951 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106711 2024-09-14T09:08:29.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.951 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.951 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.951 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.951 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:29.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:29.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.952 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.952 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:29.952 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8000000000000002e-07 2024-09-14T09:08:29.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.952 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847552+0000", 2024-09-14T09:08:29.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1340000000000001e-06 2024-09-14T09:08:29.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.953 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847557+0000", 2024-09-14T09:08:29.953 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6179999999999997e-06 2024-09-14T09:08:29.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.953 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.953 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847560+0000", 2024-09-14T09:08:29.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5899999999999999e-06 2024-09-14T09:08:29.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859050+0000", 2024-09-14T09:08:29.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01148914 2024-09-14T09:08:29.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.954 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.954 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.954 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859056+0000", 2024-09-14T09:08:29.954 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6630000000000004e-06 2024-09-14T09:08:29.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148948+0000", 2024-09-14T09:08:29.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2898914370000001 2024-09-14T09:08:29.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149079+0000", 2024-09-14T09:08:29.955 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013145400000000001 2024-09-14T09:08:29.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.955 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.955 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.955 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149775+0000", 2024-09-14T09:08:29.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00069580599999999999 2024-09-14T09:08:29.956 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.956 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.956 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.956 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.956 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106713 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 11264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:29.956 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:29.956 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085647356999999, 2024-09-14T09:08:29.956 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3035188959999999, 2024-09-14T09:08:29.956 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.956 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.956 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.956 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.957 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.957 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.957 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106713 2024-09-14T09:08:29.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.957 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:29.957 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.957 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.957 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.957 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:29.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:29.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6e-07 2024-09-14T09:08:29.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847566+0000", 2024-09-14T09:08:29.958 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1039999999999999e-06 2024-09-14T09:08:29.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.958 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.958 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.958 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847569+0000", 2024-09-14T09:08:29.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.191e-06 2024-09-14T09:08:29.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847571+0000", 2024-09-14T09:08:29.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6390000000000001e-06 2024-09-14T09:08:29.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859099+0000", 2024-09-14T09:08:29.959 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011527379000000001 2024-09-14T09:08:29.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.959 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.959 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.959 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859106+0000", 2024-09-14T09:08:29.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3679999999999996e-06 2024-09-14T09:08:29.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149878+0000", 2024-09-14T09:08:29.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.29077208 2024-09-14T09:08:29.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.150175+0000", 2024-09-14T09:08:29.960 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029685699999999998 2024-09-14T09:08:29.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.960 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.960 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.960 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151083+0000", 2024-09-14T09:08:29.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00090801800000000002 2024-09-14T09:08:29.961 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.961 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.961 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.961 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.961 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106715 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 15360~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:29.961 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:29.961 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085636347999994, 2024-09-14T09:08:29.961 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3047478800000001, 2024-09-14T09:08:29.961 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.961 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.961 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.961 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.961 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.961 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.962 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106715 2024-09-14T09:08:29.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.962 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:29.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.962 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.962 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:29.962 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.962 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.963 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.963 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:29.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6e-07 2024-09-14T09:08:29.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.963 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.963 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847580+0000", 2024-09-14T09:08:29.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9250000000000003e-06 2024-09-14T09:08:29.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.963 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.963 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.963 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847585+0000", 2024-09-14T09:08:29.963 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8409999999999999e-06 2024-09-14T09:08:29.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847588+0000", 2024-09-14T09:08:29.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6989999999999999e-06 2024-09-14T09:08:29.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859151+0000", 2024-09-14T09:08:29.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011562604000000001 2024-09-14T09:08:29.964 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.964 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.964 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.964 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859159+0000", 2024-09-14T09:08:29.964 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4179999999999997e-06 2024-09-14T09:08:29.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151155+0000", 2024-09-14T09:08:29.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.291996068 2024-09-14T09:08:29.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.965 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151383+0000", 2024-09-14T09:08:29.965 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022746100000000001 2024-09-14T09:08:29.965 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.965 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.965 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.966 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152323+0000", 2024-09-14T09:08:29.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00094060400000000005 2024-09-14T09:08:29.966 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.966 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.966 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.966 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.966 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.966 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106717 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 19456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:29.966 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:29.966 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085619660000006, 2024-09-14T09:08:29.966 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.305574579, 2024-09-14T09:08:29.966 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.966 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.966 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.967 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.967 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.967 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.967 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106717 2024-09-14T09:08:29.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.967 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:29.967 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.967 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.967 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.967 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.967 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:29.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:29.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1199999999999999e-07 2024-09-14T09:08:29.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.968 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847593+0000", 2024-09-14T09:08:29.968 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.257e-06 2024-09-14T09:08:29.968 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.968 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.968 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847598+0000", 2024-09-14T09:08:29.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4139999999999996e-06 2024-09-14T09:08:29.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847602+0000", 2024-09-14T09:08:29.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3440000000000003e-06 2024-09-14T09:08:29.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.969 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859203+0000", 2024-09-14T09:08:29.969 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011600572999999999 2024-09-14T09:08:29.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.969 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.969 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859211+0000", 2024-09-14T09:08:29.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-06 2024-09-14T09:08:29.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152398+0000", 2024-09-14T09:08:29.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.293187812 2024-09-14T09:08:29.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.970 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152543+0000", 2024-09-14T09:08:29.970 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014478000000000001 2024-09-14T09:08:29.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.970 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.970 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.971 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153166+0000", 2024-09-14T09:08:29.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000623257 2024-09-14T09:08:29.971 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.971 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.971 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.971 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.971 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106719 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 23552~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:29.971 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:29.971 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085605344000001, 2024-09-14T09:08:29.971 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.306394982, 2024-09-14T09:08:29.971 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.971 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.971 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.971 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.971 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.972 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.972 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106719 2024-09-14T09:08:29.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.972 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:29.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.972 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.972 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:29.972 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.972 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.972 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:29.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5100000000000001e-07 2024-09-14T09:08:29.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847608+0000", 2024-09-14T09:08:29.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.189e-06 2024-09-14T09:08:29.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.973 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.973 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.973 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847612+0000", 2024-09-14T09:08:29.973 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9519999999999996e-06 2024-09-14T09:08:29.973 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847614+0000", 2024-09-14T09:08:29.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6280000000000001e-06 2024-09-14T09:08:29.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859254+0000", 2024-09-14T09:08:29.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011640157 2024-09-14T09:08:29.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.974 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.974 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.974 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859261+0000", 2024-09-14T09:08:29.974 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0570000000000004e-06 2024-09-14T09:08:29.974 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153236+0000", 2024-09-14T09:08:29.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2939744040000001 2024-09-14T09:08:29.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153380+0000", 2024-09-14T09:08:29.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014410399999999999 2024-09-14T09:08:29.975 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.975 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.975 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.975 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154001+0000", 2024-09-14T09:08:29.975 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062124000000000001 2024-09-14T09:08:29.976 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.976 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.976 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.976 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.976 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106721 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 27648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:29.976 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:29.976 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085593693999996, 2024-09-14T09:08:29.976 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3074446659999999, 2024-09-14T09:08:29.976 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.976 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.976 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.976 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.976 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.977 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.977 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106721 2024-09-14T09:08:29.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.977 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:29.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.977 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.977 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.977 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:29.977 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:29.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4299999999999999e-07 2024-09-14T09:08:29.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847619+0000", 2024-09-14T09:08:29.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T09:08:29.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.978 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.978 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.978 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847624+0000", 2024-09-14T09:08:29.978 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4229999999999998e-06 2024-09-14T09:08:29.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847627+0000", 2024-09-14T09:08:29.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0120000000000001e-06 2024-09-14T09:08:29.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.979 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859305+0000", 2024-09-14T09:08:29.979 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01167812 2024-09-14T09:08:29.979 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.979 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.979 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859313+0000", 2024-09-14T09:08:29.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2409999999999995e-06 2024-09-14T09:08:29.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154081+0000", 2024-09-14T09:08:29.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.294768296 2024-09-14T09:08:29.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.980 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.980 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.980 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154280+0000", 2024-09-14T09:08:29.980 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019824 2024-09-14T09:08:29.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.981 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.981 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155063+0000", 2024-09-14T09:08:29.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00078303400000000005 2024-09-14T09:08:29.981 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.981 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.981 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.981 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.981 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106723 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 31744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:29.981 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:08:29.981 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085581263999998, 2024-09-14T09:08:29.981 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308293648, 2024-09-14T09:08:29.982 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.982 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.982 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.982 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.982 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.982 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.982 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106723 2024-09-14T09:08:29.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.982 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.982 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.982 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.982 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:08:29.982 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.982 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:08:29.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.983 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847631+0000", 2024-09-14T09:08:29.983 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6100000000000002e-07 2024-09-14T09:08:29.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.983 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.983 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847632+0000", 2024-09-14T09:08:29.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0589999999999999e-06 2024-09-14T09:08:29.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847635+0000", 2024-09-14T09:08:29.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.551e-06 2024-09-14T09:08:29.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.984 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.984 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.984 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847638+0000", 2024-09-14T09:08:29.984 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.644e-06 2024-09-14T09:08:29.984 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859344+0000", 2024-09-14T09:08:29.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011706397 2024-09-14T09:08:29.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859348+0000", 2024-09-14T09:08:29.985 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5779999999999999e-06 2024-09-14T09:08:29.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.985 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.985 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.985 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155135+0000", 2024-09-14T09:08:29.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.295787448 2024-09-14T09:08:29.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155279+0000", 2024-09-14T09:08:29.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014400700000000001 2024-09-14T09:08:29.986 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.986 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.986 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.986 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155924+0000", 2024-09-14T09:08:29.986 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00064470300000000005 2024-09-14T09:08:29.986 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.986 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.986 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.987 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.987 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.987 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106725 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 35840~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:29.987 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:08:29.987 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085570153000006, 2024-09-14T09:08:29.987 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.309103218, 2024-09-14T09:08:29.987 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.987 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.987 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.987 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.987 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.987 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.988 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106725 2024-09-14T09:08:29.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.988 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:08:29.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.988 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:08:29.988 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.988 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.988 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.988 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847642+0000", 2024-09-14T09:08:29.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4499999999999998e-07 2024-09-14T09:08:29.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847643+0000", 2024-09-14T09:08:29.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4e-07 2024-09-14T09:08:29.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.989 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847647+0000", 2024-09-14T09:08:29.989 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6190000000000002e-06 2024-09-14T09:08:29.989 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.989 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.989 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847655+0000", 2024-09-14T09:08:29.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:08:29.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859381+0000", 2024-09-14T09:08:29.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011725793 2024-09-14T09:08:29.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859391+0000", 2024-09-14T09:08:29.990 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4390000000000006e-06 2024-09-14T09:08:29.990 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.990 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.990 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.990 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155997+0000", 2024-09-14T09:08:29.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2966067830000001 2024-09-14T09:08:29.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156133+0000", 2024-09-14T09:08:29.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000135609 2024-09-14T09:08:29.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.991 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.991 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.991 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156745+0000", 2024-09-14T09:08:29.991 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000611592 2024-09-14T09:08:29.991 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.991 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.991 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.991 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.992 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.992 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106727 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 39936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:29.992 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:29.992 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085551852999998, 2024-09-14T09:08:29.992 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3099046560000001, 2024-09-14T09:08:29.992 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.992 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.992 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.992 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.992 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.992 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.992 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106727 2024-09-14T09:08:29.992 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.992 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:29.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:29.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.993 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.993 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.993 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:29.993 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1300000000000001e-07 2024-09-14T09:08:29.993 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847661+0000", 2024-09-14T09:08:29.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.192e-06 2024-09-14T09:08:29.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847665+0000", 2024-09-14T09:08:29.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0999999999999997e-06 2024-09-14T09:08:29.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.994 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.994 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.994 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847668+0000", 2024-09-14T09:08:29.994 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.644e-06 2024-09-14T09:08:29.994 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859422+0000", 2024-09-14T09:08:29.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011753559 2024-09-14T09:08:29.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:29.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859425+0000", 2024-09-14T09:08:29.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.343e-06 2024-09-14T09:08:29.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.995 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.995 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:29.995 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156817+0000", 2024-09-14T09:08:29.995 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2973924800000001 2024-09-14T09:08:29.995 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:29.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156956+0000", 2024-09-14T09:08:29.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013835 2024-09-14T09:08:29.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.996 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:29.996 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157564+0000", 2024-09-14T09:08:29.996 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060867500000000004 2024-09-14T09:08:29.996 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.996 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:29.996 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:29.996 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.996 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.996 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106729 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 44032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:29.997 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:29.997 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085539879999999, 2024-09-14T09:08:29.997 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3107195970000001, 2024-09-14T09:08:29.997 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:29.997 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:29.997 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:29.997 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:29.997 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:29.997 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:29.997 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106729 2024-09-14T09:08:29.997 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.997 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:29.997 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:29.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:29.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:29.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:29.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:29.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.998 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.998 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:29.998 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:29.998 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.29e-07 2024-09-14T09:08:29.998 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:29.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847673+0000", 2024-09-14T09:08:29.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.096e-06 2024-09-14T09:08:29.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:29.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847678+0000", 2024-09-14T09:08:29.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.527e-06 2024-09-14T09:08:29.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:29.999 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:29.999 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:29.999 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847681+0000", 2024-09-14T09:08:29.999 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.788e-06 2024-09-14T09:08:29.999 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:30.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859462+0000", 2024-09-14T09:08:30.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01178032 2024-09-14T09:08:30.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:30.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859471+0000", 2024-09-14T09:08:30.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3430000000000002e-06 2024-09-14T09:08:30.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.000 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.000 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:30.000 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157635+0000", 2024-09-14T09:08:30.000 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2981640320000001 2024-09-14T09:08:30.000 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:30.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157773+0000", 2024-09-14T09:08:30.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763 2024-09-14T09:08:30.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.001 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:30.001 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158391+0000", 2024-09-14T09:08:30.001 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061863200000000004 2024-09-14T09:08:30.001 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:30.001 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:30.001 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:30.001 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.001 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.002 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106731 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 48128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:30.002 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:08:30.002 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085524234999994, 2024-09-14T09:08:30.002 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3115142020000001, 2024-09-14T09:08:30.002 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:30.002 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:30.002 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:30.002 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:30.002 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:30.002 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:30.002 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106731 2024-09-14T09:08:30.002 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.002 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:30.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:30.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:08:30.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:30.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:30.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:08:30.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:30.003 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.003 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.003 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:30.003 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847688+0000", 2024-09-14T09:08:30.003 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0199999999999998e-07 2024-09-14T09:08:30.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:30.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847689+0000", 2024-09-14T09:08:30.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4529999999999999e-06 2024-09-14T09:08:30.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:30.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847694+0000", 2024-09-14T09:08:30.004 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9080000000000003e-06 2024-09-14T09:08:30.004 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.004 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.004 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:30.004 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847697+0000", 2024-09-14T09:08:30.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4920000000000002e-06 2024-09-14T09:08:30.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:30.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859499+0000", 2024-09-14T09:08:30.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011801676000000001 2024-09-14T09:08:30.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:30.005 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859502+0000", 2024-09-14T09:08:30.005 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2899999999999998e-06 2024-09-14T09:08:30.005 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.005 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.005 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:30.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158461+0000", 2024-09-14T09:08:30.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.298958796 2024-09-14T09:08:30.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:30.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158594+0000", 2024-09-14T09:08:30.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013321599999999999 2024-09-14T09:08:30.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.006 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.006 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:30.006 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159202+0000", 2024-09-14T09:08:30.006 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060706900000000001 2024-09-14T09:08:30.006 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:30.006 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:30.006 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:30.006 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.007 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106733 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 52224~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:30.007 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:30.007 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085509978999994, 2024-09-14T09:08:30.007 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3123343809999999, 2024-09-14T09:08:30.007 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:30.007 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:30.007 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:30.007 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:30.007 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:30.007 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:30.007 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106733 2024-09-14T09:08:30.007 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.007 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:30.007 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.007 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:30.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:30.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:30.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:30.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:30.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:30.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:30.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:30.008 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5800000000000001e-07 2024-09-14T09:08:30.008 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.008 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.008 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:30.008 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847703+0000", 2024-09-14T09:08:30.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.127e-06 2024-09-14T09:08:30.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:30.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847706+0000", 2024-09-14T09:08:30.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3210000000000001e-06 2024-09-14T09:08:30.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:30.009 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847709+0000", 2024-09-14T09:08:30.009 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7539999999999998e-06 2024-09-14T09:08:30.009 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.009 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.009 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:30.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859538+0000", 2024-09-14T09:08:30.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01182942 2024-09-14T09:08:30.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:30.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859547+0000", 2024-09-14T09:08:30.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T09:08:30.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:30.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159273+0000", 2024-09-14T09:08:30.010 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2997263700000001 2024-09-14T09:08:30.010 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.010 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.010 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:30.010 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159415+0000", 2024-09-14T09:08:30.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000141765 2024-09-14T09:08:30.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.011 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:30.011 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160036+0000", 2024-09-14T09:08:30.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062122100000000003 2024-09-14T09:08:30.011 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:30.011 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:30.011 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:30.011 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.011 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.011 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106735 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 56320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:30.011 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:30.011 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085498534999999, 2024-09-14T09:08:30.011 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3131339309999999, 2024-09-14T09:08:30.012 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:30.012 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:30.012 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:30.012 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:30.012 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:30.012 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:30.012 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106735 2024-09-14T09:08:30.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.012 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:30.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.012 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:30.012 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:30.012 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:30.012 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.012 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:30.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:30.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:30.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:30.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:30.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2100000000000001e-07 2024-09-14T09:08:30.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.013 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:30.013 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847714+0000", 2024-09-14T09:08:30.013 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.051e-06 2024-09-14T09:08:30.013 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.013 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:30.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847719+0000", 2024-09-14T09:08:30.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3640000000000001e-06 2024-09-14T09:08:30.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:30.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847723+0000", 2024-09-14T09:08:30.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8789999999999996e-06 2024-09-14T09:08:30.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.014 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.014 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:30.014 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859577+0000", 2024-09-14T09:08:30.014 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011854416 2024-09-14T09:08:30.014 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:30.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859580+0000", 2024-09-14T09:08:30.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.292e-06 2024-09-14T09:08:30.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:30.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160108+0000", 2024-09-14T09:08:30.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.30052758 2024-09-14T09:08:30.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.015 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:30.015 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160238+0000", 2024-09-14T09:08:30.015 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013043500000000001 2024-09-14T09:08:30.015 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.015 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.016 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:30.016 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160847+0000", 2024-09-14T09:08:30.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060869299999999999 2024-09-14T09:08:30.016 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:30.016 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:30.016 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:30.016 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.016 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.016 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106738 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 60416~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:30.016 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:30.016 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085485331000001, 2024-09-14T09:08:30.016 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3139482179999999, 2024-09-14T09:08:30.016 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:30.017 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:30.017 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:30.017 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:30.017 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:30.017 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:30.017 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106738 2024-09-14T09:08:30.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.017 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:30.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:30.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:30.017 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:30.017 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.017 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.017 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:30.017 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:30.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:30.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:30.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:30.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3999999999999998e-07 2024-09-14T09:08:30.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:30.018 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847733+0000", 2024-09-14T09:08:30.018 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8070000000000002e-06 2024-09-14T09:08:30.018 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.018 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.018 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:30.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847738+0000", 2024-09-14T09:08:30.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9740000000000001e-06 2024-09-14T09:08:30.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:30.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847742+0000", 2024-09-14T09:08:30.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6009999999999999e-06 2024-09-14T09:08:30.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.019 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.019 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:30.019 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859612+0000", 2024-09-14T09:08:30.019 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011870537 2024-09-14T09:08:30.019 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:30.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859621+0000", 2024-09-14T09:08:30.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3019999999999995e-06 2024-09-14T09:08:30.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:30.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160917+0000", 2024-09-14T09:08:30.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3012961080000001 2024-09-14T09:08:30.020 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.020 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.020 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:30.020 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161054+0000", 2024-09-14T09:08:30.020 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000137422 2024-09-14T09:08:30.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.021 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:30.021 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161674+0000", 2024-09-14T09:08:30.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000620227 2024-09-14T09:08:30.021 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:30.021 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:30.021 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:30.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.021 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.021 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106739 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 64512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:30.021 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:30.021 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085465490999994, 2024-09-14T09:08:30.021 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.314753777, 2024-09-14T09:08:30.021 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:30.022 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:30.022 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:30.022 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:30.022 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:30.022 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:30.022 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106739 2024-09-14T09:08:30.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.022 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:30.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:30.022 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:30.022 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:30.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.022 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.022 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:30.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:30.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:30.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:30.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:30.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9700000000000003e-07 2024-09-14T09:08:30.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:30.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847748+0000", 2024-09-14T09:08:30.023 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2589999999999999e-06 2024-09-14T09:08:30.023 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.023 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.023 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:30.023 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847752+0000", 2024-09-14T09:08:30.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6099999999999999e-06 2024-09-14T09:08:30.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:30.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847755+0000", 2024-09-14T09:08:30.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0520000000000002e-06 2024-09-14T09:08:30.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:30.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859653+0000", 2024-09-14T09:08:30.024 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011897575000000001 2024-09-14T09:08:30.024 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.024 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.024 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:30.024 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859658+0000", 2024-09-14T09:08:30.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8810000000000004e-06 2024-09-14T09:08:30.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:30.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161745+0000", 2024-09-14T09:08:30.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3020870819999999 2024-09-14T09:08:30.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:30.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161877+0000", 2024-09-14T09:08:30.025 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000132657 2024-09-14T09:08:30.025 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.025 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.025 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:30.025 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162500+0000", 2024-09-14T09:08:30.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062236400000000003 2024-09-14T09:08:30.026 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:30.026 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:30.026 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:30.026 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.026 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.026 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106741 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 68608~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:30.026 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:30.026 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085452699000001, 2024-09-14T09:08:30.026 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3155568449999999, 2024-09-14T09:08:30.026 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:30.026 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:30.026 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:30.026 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:30.026 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:30.027 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:30.027 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106741 2024-09-14T09:08:30.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.027 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:30.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:30.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:30.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:30.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.027 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:30.027 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:30.027 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:30.027 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.027 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:30.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:30.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5800000000000001e-07 2024-09-14T09:08:30.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:30.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847760+0000", 2024-09-14T09:08:30.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:08:30.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:30.028 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847764+0000", 2024-09-14T09:08:30.028 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3189999999999999e-06 2024-09-14T09:08:30.028 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.028 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.028 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:30.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847766+0000", 2024-09-14T09:08:30.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6429999999999999e-06 2024-09-14T09:08:30.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:30.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859693+0000", 2024-09-14T09:08:30.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011926724 2024-09-14T09:08:30.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.029 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.029 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:30.029 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859703+0000", 2024-09-14T09:08:30.029 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6160000000000008e-06 2024-09-14T09:08:30.029 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:30.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162575+0000", 2024-09-14T09:08:30.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.302872128 2024-09-14T09:08:30.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:30.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162709+0000", 2024-09-14T09:08:30.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013446000000000001 2024-09-14T09:08:30.030 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.030 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.030 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:30.030 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163316+0000", 2024-09-14T09:08:30.030 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060650700000000005 2024-09-14T09:08:30.030 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:30.031 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:30.031 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:30.031 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.031 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.031 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106744 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 76800~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:30.031 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:08:30.031 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 68.085441587999995, 2024-09-14T09:08:30.031 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3163901789999999, 2024-09-14T09:08:30.031 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:30.031 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:30.031 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:30.031 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:30.031 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:30.031 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:30.032 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106744 2024-09-14T09:08:30.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.032 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:30.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:30.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:08:30.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:30.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.032 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:30.032 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:08:30.032 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:30.032 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.032 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:30.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996947+0000", 2024-09-14T09:08:30.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.149177154 2024-09-14T09:08:30.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:30.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996964+0000", 2024-09-14T09:08:30.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.679e-05 2024-09-14T09:08:30.033 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.033 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.033 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:30.033 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996999+0000", 2024-09-14T09:08:30.033 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5463000000000002e-05 2024-09-14T09:08:30.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.034 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:30.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.997029+0000", 2024-09-14T09:08:30.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9567e-05 2024-09-14T09:08:30.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.034 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:30.034 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163390+0000", 2024-09-14T09:08:30.034 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.16636056799999999 2024-09-14T09:08:30.034 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.034 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.034 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:30.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163526+0000", 2024-09-14T09:08:30.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013655899999999999 2024-09-14T09:08:30.035 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:30.035 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:30.035 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:30.035 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.164160+0000", 2024-09-14T09:08:30.035 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00063407800000000005 2024-09-14T09:08:30.035 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:30.035 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:30.035 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:30.035 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:30.035 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:30.035 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:30.037 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:08:30.037 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:30.035+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:30.087 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T09:08:30.137 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:30.142+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:30.205 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:30.205 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:30.205 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:30.205 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:30.220 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:08:30.238 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:30.242+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:30.266 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:08:30.266 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:08:30.299 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:08:30.299 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:08:30.331 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:08:30.331 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:08:30.339 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:30.339+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:30.352 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:08:30.353 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:08:30.353 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:08:30.364 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:30.364 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:30.374 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:08:30.374 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:08:30.378 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:08:30.395 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:08:30.395 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:08:30.416 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:08:30.416 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:08:30.419 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:08:30.420 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T09:08:30.440 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:30.442+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:30.443 INFO:teuthology.orchestra.run.smithi176.stderr:set 2.0 pg_upmap mapping to [4,2] 2024-09-14T09:08:30.479 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T09:08:30.539 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:30.539 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:30.540 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:30.539+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:30.558 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:08:30.641 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:30.643+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:30.699 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:30.700 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:30.714 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:08:30.761 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:30.766+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:30.837 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:30.838 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:30.844 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:30.843+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:30.852 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:08:30.945 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:30.950+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:31.016 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:31.016 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:31.030 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:08:31.073 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:31.078+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:31.147 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:31.147+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:31.200 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:31.201 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:31.213 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:08:31.257 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:31.262+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:31.336 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:31.336 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:31.348 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:31.347+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:31.350 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:08:31.449 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:31.450+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:31.452 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is unset 2024-09-14T09:08:31.531 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:31.531 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:31.547 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:08:31.590 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:31.594+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:31.650 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:31.654+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:31.714 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:31.714 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:31.728 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:08:31.751 INFO:tasks.ceph.osd.1.smithi176.stderr:2024-09-14T09:08:31.751+0000 7f73cf635640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26413) UID: 0 2024-09-14T09:08:31.851 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:31.854+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:31.890 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:31.890 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:31.904 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:08:31.952 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:31.951+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:32.053 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:32.051+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:32.068 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:32.068 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:32.083 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T09:08:32.154 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:32.155+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:32.217 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:32.217 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:32.232 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T09:08:32.274 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:32.275+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:32.356 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:32.358+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:32.391 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:32.391 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:32.405 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T09:08:32.420 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T09:08:32.494 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:32.494+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:32.543 INFO:teuthology.orchestra.run.smithi176.stdout:ERROR: (22) Invalid argument 2024-09-14T09:08:32.543 INFO:teuthology.orchestra.run.smithi176.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T09:08:32.556 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:32.554+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:32.561 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T09:08:32.611 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0: {} 2024-09-14T09:08:32.611 INFO:teuthology.orchestra.run.smithi176.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T09:08:32.629 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1: {} 2024-09-14T09:08:32.629 INFO:teuthology.orchestra.run.smithi176.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T09:08:32.646 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2: {} 2024-09-14T09:08:32.647 INFO:teuthology.orchestra.run.smithi176.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T09:08:32.658 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:32.658+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:32.663 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3: {} 2024-09-14T09:08:32.663 INFO:teuthology.orchestra.run.smithi176.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T09:08:32.663 INFO:teuthology.orchestra.run.smithi176.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T09:08:32.699 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5: {} 2024-09-14T09:08:32.700 INFO:teuthology.orchestra.run.smithi176.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T09:08:32.734 INFO:teuthology.orchestra.run.smithi176.stdout:osd.6: {} 2024-09-14T09:08:32.734 INFO:teuthology.orchestra.run.smithi176.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T09:08:32.749 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:32.749 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:32.749 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:32.749 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:32.758 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:32.758+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:32.768 INFO:teuthology.orchestra.run.smithi176.stdout:osd.7: {} 2024-09-14T09:08:32.768 INFO:teuthology.orchestra.run.smithi176.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T09:08:32.769 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T09:08:32.772 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T09:08:32.772 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T09:08:32.859 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:32.862+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:32.934 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:32.934 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:32.934 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:08:32.934 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:08:32.934 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:32.948 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T09:08:32.994 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:32.994+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:33.060 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:33.062+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:33.066 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:33.066 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:08:33.066 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:08:33.066 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:08:33.066 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.066 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73542 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15319040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:33.067 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:08:33.067 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.797792887, 2024-09-14T09:08:33.067 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0237091359999999, 2024-09-14T09:08:33.067 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.067 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.067 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.067 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.067 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.067 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.067 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73542 2024-09-14T09:08:33.067 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.067 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.067 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.067 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:08:33.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268761+0000", 2024-09-14T09:08:33.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.068 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.068 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.068 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.068 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268762+0000", 2024-09-14T09:08:33.068 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:08:33.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268766+0000", 2024-09-14T09:08:33.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5880000000000002e-06 2024-09-14T09:08:33.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.069 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.069 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268776+0000", 2024-09-14T09:08:33.069 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0168e-05 2024-09-14T09:08:33.069 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.069 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268789+0000", 2024-09-14T09:08:33.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3203999999999999e-05 2024-09-14T09:08:33.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282053+0000", 2024-09-14T09:08:33.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013264496000000001 2024-09-14T09:08:33.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.070 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.070 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.070 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282057+0000", 2024-09-14T09:08:33.070 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6090000000000001e-06 2024-09-14T09:08:33.070 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292052+0000", 2024-09-14T09:08:33.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0099950070000001 2024-09-14T09:08:33.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292262+0000", 2024-09-14T09:08:33.071 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021009800000000001 2024-09-14T09:08:33.071 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.071 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.071 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.071 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292470+0000", 2024-09-14T09:08:33.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000208314 2024-09-14T09:08:33.072 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.072 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.072 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.072 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.072 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.072 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73543 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:33.072 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:08:33.072 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.79775409699999, 2024-09-14T09:08:33.072 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.023985473, 2024-09-14T09:08:33.072 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.072 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.073 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.073 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.073 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.073 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.073 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73543 2024-09-14T09:08:33.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.073 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.073 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:08:33.073 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.073 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.073 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.073 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268800+0000", 2024-09-14T09:08:33.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268801+0000", 2024-09-14T09:08:33.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:08:33.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.074 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.074 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.074 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268803+0000", 2024-09-14T09:08:33.074 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7439999999999999e-06 2024-09-14T09:08:33.074 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268812+0000", 2024-09-14T09:08:33.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7080000000000003e-06 2024-09-14T09:08:33.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268819+0000", 2024-09-14T09:08:33.075 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7660000000000001e-06 2024-09-14T09:08:33.075 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.075 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.075 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.075 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282068+0000", 2024-09-14T09:08:33.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013249540000000001 2024-09-14T09:08:33.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282072+0000", 2024-09-14T09:08:33.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0960000000000001e-06 2024-09-14T09:08:33.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292573+0000", 2024-09-14T09:08:33.076 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0105014560000001 2024-09-14T09:08:33.076 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.076 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.076 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.076 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292679+0000", 2024-09-14T09:08:33.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00010545899999999999 2024-09-14T09:08:33.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.077 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.077 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292786+0000", 2024-09-14T09:08:33.077 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000107052 2024-09-14T09:08:33.077 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.077 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.077 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.077 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.077 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.077 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73544 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:33.077 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:33.078 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.797726395, 2024-09-14T09:08:33.078 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0244830549999999, 2024-09-14T09:08:33.078 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.078 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.078 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.078 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.078 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.078 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.078 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73544 2024-09-14T09:08:33.078 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.078 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.078 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.078 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.078 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:33.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:33.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.079 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.079 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268828+0000", 2024-09-14T09:08:33.079 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5599999999999995e-07 2024-09-14T09:08:33.079 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.079 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268831+0000", 2024-09-14T09:08:33.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.57e-06 2024-09-14T09:08:33.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268844+0000", 2024-09-14T09:08:33.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2811e-05 2024-09-14T09:08:33.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.080 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.080 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.080 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268851+0000", 2024-09-14T09:08:33.080 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9140000000000002e-06 2024-09-14T09:08:33.080 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282083+0000", 2024-09-14T09:08:33.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013232111 2024-09-14T09:08:33.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282086+0000", 2024-09-14T09:08:33.081 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2909999999999999e-06 2024-09-14T09:08:33.081 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.081 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.081 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.081 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.292884+0000", 2024-09-14T09:08:33.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0107975069999999 2024-09-14T09:08:33.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293100+0000", 2024-09-14T09:08:33.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021639399999999999 2024-09-14T09:08:33.082 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.082 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.082 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.082 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293311+0000", 2024-09-14T09:08:33.082 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021090099999999999 2024-09-14T09:08:33.082 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.082 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.083 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.083 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.083 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.083 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73545 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15564800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:33.083 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:08:33.083 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.79767520999999, 2024-09-14T09:08:33.083 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0249746740000001, 2024-09-14T09:08:33.083 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.083 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.083 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.083 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.083 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.083 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.083 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73545 2024-09-14T09:08:33.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.084 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:08:33.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.084 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.084 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268879+0000", 2024-09-14T09:08:33.084 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.084 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.084 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268880+0000", 2024-09-14T09:08:33.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5600000000000005e-07 2024-09-14T09:08:33.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268889+0000", 2024-09-14T09:08:33.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8559999999999996e-06 2024-09-14T09:08:33.085 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.085 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.085 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.085 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268913+0000", 2024-09-14T09:08:33.085 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3958e-05 2024-09-14T09:08:33.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268921+0000", 2024-09-14T09:08:33.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7479999999999999e-06 2024-09-14T09:08:33.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.086 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.086 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282098+0000", 2024-09-14T09:08:33.086 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013176297999999999 2024-09-14T09:08:33.086 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.086 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282101+0000", 2024-09-14T09:08:33.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2329999999999998e-06 2024-09-14T09:08:33.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293435+0000", 2024-09-14T09:08:33.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0113339459999999 2024-09-14T09:08:33.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.087 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.087 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.087 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293648+0000", 2024-09-14T09:08:33.087 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021307100000000001 2024-09-14T09:08:33.087 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.088 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.088 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293854+0000", 2024-09-14T09:08:33.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020580800000000001 2024-09-14T09:08:33.088 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.088 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.088 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.088 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.088 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.088 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73546 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15646720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:33.088 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:08:33.088 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.797621339, 2024-09-14T09:08:33.088 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0254846360000001, 2024-09-14T09:08:33.088 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.088 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.089 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.089 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.089 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.089 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.089 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73546 2024-09-14T09:08:33.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.089 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.089 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.089 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:08:33.089 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.089 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.089 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.089 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.090 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268933+0000", 2024-09-14T09:08:33.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.090 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268934+0000", 2024-09-14T09:08:33.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7999999999999995e-07 2024-09-14T09:08:33.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.090 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.090 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268937+0000", 2024-09-14T09:08:33.090 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9840000000000001e-06 2024-09-14T09:08:33.090 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.090 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268947+0000", 2024-09-14T09:08:33.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0522e-05 2024-09-14T09:08:33.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268955+0000", 2024-09-14T09:08:33.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8059999999999995e-06 2024-09-14T09:08:33.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.091 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.091 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.091 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282112+0000", 2024-09-14T09:08:33.091 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013157189 2024-09-14T09:08:33.091 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282115+0000", 2024-09-14T09:08:33.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2949999999999998e-06 2024-09-14T09:08:33.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.293968+0000", 2024-09-14T09:08:33.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0118530990000001 2024-09-14T09:08:33.092 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.092 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.092 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.092 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294197+0000", 2024-09-14T09:08:33.092 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228507 2024-09-14T09:08:33.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.093 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.093 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294417+0000", 2024-09-14T09:08:33.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000220554 2024-09-14T09:08:33.093 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.093 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.093 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.093 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.093 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.093 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73547 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:33.093 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:08:33.093 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.797588893, 2024-09-14T09:08:33.093 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.025970525, 2024-09-14T09:08:33.094 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.094 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.094 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.094 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.094 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.094 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.094 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73547 2024-09-14T09:08:33.094 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.094 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.094 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.094 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.094 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:08:33.094 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268965+0000", 2024-09-14T09:08:33.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.095 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268966+0000", 2024-09-14T09:08:33.095 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3799999999999997e-07 2024-09-14T09:08:33.095 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.095 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.095 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268969+0000", 2024-09-14T09:08:33.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8739999999999999e-06 2024-09-14T09:08:33.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268979+0000", 2024-09-14T09:08:33.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.9939999999999995e-06 2024-09-14T09:08:33.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.096 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.096 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268986+0000", 2024-09-14T09:08:33.096 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3459999999999996e-06 2024-09-14T09:08:33.096 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.096 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282127+0000", 2024-09-14T09:08:33.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01314039 2024-09-14T09:08:33.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282130+0000", 2024-09-14T09:08:33.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2219999999999998e-06 2024-09-14T09:08:33.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.097 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.097 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.097 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294540+0000", 2024-09-14T09:08:33.097 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124105880000001 2024-09-14T09:08:33.097 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294759+0000", 2024-09-14T09:08:33.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000218798 2024-09-14T09:08:33.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.098 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.098 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.294936+0000", 2024-09-14T09:08:33.098 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000176675 2024-09-14T09:08:33.098 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.098 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.098 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.098 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.098 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.098 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73548 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15810560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:33.099 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:08:33.099 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.79755778099999, 2024-09-14T09:08:33.099 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0265045129999999, 2024-09-14T09:08:33.099 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.099 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.099 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.099 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.099 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73548 2024-09-14T09:08:33.099 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.099 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.099 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.099 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:08:33.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268996+0000", 2024-09-14T09:08:33.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.100 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.100 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.100 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.268997+0000", 2024-09-14T09:08:33.100 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.7599999999999996e-07 2024-09-14T09:08:33.100 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269000+0000", 2024-09-14T09:08:33.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0580000000000002e-06 2024-09-14T09:08:33.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269010+0000", 2024-09-14T09:08:33.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0124e-05 2024-09-14T09:08:33.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.101 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.101 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.101 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269018+0000", 2024-09-14T09:08:33.101 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5399999999999998e-06 2024-09-14T09:08:33.101 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282141+0000", 2024-09-14T09:08:33.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013123012 2024-09-14T09:08:33.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282144+0000", 2024-09-14T09:08:33.102 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.196e-06 2024-09-14T09:08:33.102 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.102 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.102 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.102 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295042+0000", 2024-09-14T09:08:33.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0128975689999999 2024-09-14T09:08:33.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295272+0000", 2024-09-14T09:08:33.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00023048099999999999 2024-09-14T09:08:33.103 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.103 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.103 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.103 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295501+0000", 2024-09-14T09:08:33.103 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000228657 2024-09-14T09:08:33.103 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.103 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.103 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.104 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.104 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73549 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15892480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:33.104 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:08:33.104 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.797511902, 2024-09-14T09:08:33.104 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0268354099999999, 2024-09-14T09:08:33.104 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.104 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.104 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.104 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.104 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.104 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.104 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73549 2024-09-14T09:08:33.104 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.105 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:08:33.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269042+0000", 2024-09-14T09:08:33.105 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.105 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.105 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.105 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.105 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269043+0000", 2024-09-14T09:08:33.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6199999999999997e-07 2024-09-14T09:08:33.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269046+0000", 2024-09-14T09:08:33.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2480000000000001e-06 2024-09-14T09:08:33.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.106 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269057+0000", 2024-09-14T09:08:33.106 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0436e-05 2024-09-14T09:08:33.106 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.106 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.106 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269064+0000", 2024-09-14T09:08:33.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5179999999999998e-06 2024-09-14T09:08:33.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282155+0000", 2024-09-14T09:08:33.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013091190000000001 2024-09-14T09:08:33.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.107 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.107 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.107 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282159+0000", 2024-09-14T09:08:33.107 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.366e-06 2024-09-14T09:08:33.107 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295599+0000", 2024-09-14T09:08:33.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0134405559999999 2024-09-14T09:08:33.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295765+0000", 2024-09-14T09:08:33.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016575499999999999 2024-09-14T09:08:33.108 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.108 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.108 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.108 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295878+0000", 2024-09-14T09:08:33.108 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000112679 2024-09-14T09:08:33.108 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.109 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.109 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.109 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.109 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.109 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73550 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 15974400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:33.109 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:08:33.109 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.797479302, 2024-09-14T09:08:33.109 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027357657, 2024-09-14T09:08:33.109 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.109 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.109 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.109 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.109 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.109 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.110 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73550 2024-09-14T09:08:33.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.110 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:08:33.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269075+0000", 2024-09-14T09:08:33.110 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.110 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.110 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.110 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.110 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269076+0000", 2024-09-14T09:08:33.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9800000000000003e-07 2024-09-14T09:08:33.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269079+0000", 2024-09-14T09:08:33.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9500000000000001e-06 2024-09-14T09:08:33.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.111 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269098+0000", 2024-09-14T09:08:33.111 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9698e-05 2024-09-14T09:08:33.111 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.111 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.111 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269106+0000", 2024-09-14T09:08:33.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6179999999999997e-06 2024-09-14T09:08:33.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282170+0000", 2024-09-14T09:08:33.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013063874 2024-09-14T09:08:33.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.112 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.112 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282173+0000", 2024-09-14T09:08:33.112 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1999999999999999e-06 2024-09-14T09:08:33.112 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.112 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.295996+0000", 2024-09-14T09:08:33.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.013823151 2024-09-14T09:08:33.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296214+0000", 2024-09-14T09:08:33.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021817700000000001 2024-09-14T09:08:33.113 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.113 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.113 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.113 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296433+0000", 2024-09-14T09:08:33.113 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021829099999999999 2024-09-14T09:08:33.113 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.113 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.114 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.114 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.114 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.114 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73551 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:33.114 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:08:33.114 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.79743788499999, 2024-09-14T09:08:33.114 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.027730276, 2024-09-14T09:08:33.114 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.114 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.114 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.114 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.114 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.114 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.114 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73551 2024-09-14T09:08:33.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.115 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:08:33.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.115 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269116+0000", 2024-09-14T09:08:33.115 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.115 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.115 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.115 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269117+0000", 2024-09-14T09:08:33.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6000000000000003e-07 2024-09-14T09:08:33.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269120+0000", 2024-09-14T09:08:33.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.056e-06 2024-09-14T09:08:33.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.116 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.116 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269130+0000", 2024-09-14T09:08:33.116 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0084e-05 2024-09-14T09:08:33.116 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.116 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269137+0000", 2024-09-14T09:08:33.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1359999999999999e-06 2024-09-14T09:08:33.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282184+0000", 2024-09-14T09:08:33.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013046537 2024-09-14T09:08:33.117 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.117 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.117 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.117 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282188+0000", 2024-09-14T09:08:33.117 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3220000000000003e-06 2024-09-14T09:08:33.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296529+0000", 2024-09-14T09:08:33.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0143406109999999 2024-09-14T09:08:33.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296695+0000", 2024-09-14T09:08:33.118 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016604799999999999 2024-09-14T09:08:33.118 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.118 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.118 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.118 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296847+0000", 2024-09-14T09:08:33.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000151822 2024-09-14T09:08:33.119 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.119 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.119 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.119 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.119 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.119 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73552 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16138240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:33.119 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:33.119 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.79740730499999, 2024-09-14T09:08:33.119 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0281767500000001, 2024-09-14T09:08:33.119 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.119 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.119 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.119 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.120 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.120 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.120 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73552 2024-09-14T09:08:33.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.120 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:33.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.120 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.120 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.120 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:33.120 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.120 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269147+0000", 2024-09-14T09:08:33.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:08:33.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269150+0000", 2024-09-14T09:08:33.121 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9119999999999998e-06 2024-09-14T09:08:33.121 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.121 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.121 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.121 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269160+0000", 2024-09-14T09:08:33.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6819999999999998e-06 2024-09-14T09:08:33.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269168+0000", 2024-09-14T09:08:33.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6079999999999998e-06 2024-09-14T09:08:33.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.122 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282205+0000", 2024-09-14T09:08:33.122 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013036982000000001 2024-09-14T09:08:33.122 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.122 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.122 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282208+0000", 2024-09-14T09:08:33.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4620000000000001e-06 2024-09-14T09:08:33.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.296968+0000", 2024-09-14T09:08:33.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0147601209999999 2024-09-14T09:08:33.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.123 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.123 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.123 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297115+0000", 2024-09-14T09:08:33.123 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014673900000000001 2024-09-14T09:08:33.123 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.124 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.124 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297324+0000", 2024-09-14T09:08:33.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020866400000000001 2024-09-14T09:08:33.124 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.124 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.124 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.124 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.124 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73553 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:33.124 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:08:33.124 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.79737724100001, 2024-09-14T09:08:33.124 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0286556739999999, 2024-09-14T09:08:33.125 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.125 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.125 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.125 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.125 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.125 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.125 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73553 2024-09-14T09:08:33.125 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.125 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.125 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.125 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.125 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:08:33.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269177+0000", 2024-09-14T09:08:33.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.126 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269178+0000", 2024-09-14T09:08:33.126 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7999999999999995e-07 2024-09-14T09:08:33.126 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.126 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.126 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269180+0000", 2024-09-14T09:08:33.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6340000000000001e-06 2024-09-14T09:08:33.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269189+0000", 2024-09-14T09:08:33.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2e-06 2024-09-14T09:08:33.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.127 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.127 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.127 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269197+0000", 2024-09-14T09:08:33.127 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.2980000000000003e-06 2024-09-14T09:08:33.127 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282219+0000", 2024-09-14T09:08:33.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013022687 2024-09-14T09:08:33.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282223+0000", 2024-09-14T09:08:33.128 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.241e-06 2024-09-14T09:08:33.128 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.128 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.128 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.128 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297470+0000", 2024-09-14T09:08:33.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.015247778 2024-09-14T09:08:33.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297704+0000", 2024-09-14T09:08:33.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000233367 2024-09-14T09:08:33.129 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.129 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.129 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.129 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297833+0000", 2024-09-14T09:08:33.129 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012888900000000001 2024-09-14T09:08:33.129 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.129 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.129 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.130 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.130 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.130 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73554 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16302080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:33.130 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:08:33.130 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.79734792399999, 2024-09-14T09:08:33.130 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.029005113, 2024-09-14T09:08:33.130 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.130 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.130 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.130 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.130 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.130 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.130 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73554 2024-09-14T09:08:33.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.131 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:08:33.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.131 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269206+0000", 2024-09-14T09:08:33.131 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.131 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.131 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.131 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269207+0000", 2024-09-14T09:08:33.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.68e-07 2024-09-14T09:08:33.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269210+0000", 2024-09-14T09:08:33.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8100000000000002e-06 2024-09-14T09:08:33.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.132 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.132 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.132 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269219+0000", 2024-09-14T09:08:33.132 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5300000000000002e-06 2024-09-14T09:08:33.132 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269227+0000", 2024-09-14T09:08:33.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3839999999999999e-06 2024-09-14T09:08:33.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282234+0000", 2024-09-14T09:08:33.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.013007187 2024-09-14T09:08:33.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.133 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.133 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.133 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282237+0000", 2024-09-14T09:08:33.133 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1200000000000002e-06 2024-09-14T09:08:33.133 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.297942+0000", 2024-09-14T09:08:33.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0157048040000001 2024-09-14T09:08:33.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298087+0000", 2024-09-14T09:08:33.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014499099999999999 2024-09-14T09:08:33.134 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.134 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.134 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.134 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298211+0000", 2024-09-14T09:08:33.134 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012461899999999999 2024-09-14T09:08:33.135 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.135 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.135 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.135 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.135 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.135 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73555 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:33.135 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:08:33.135 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.797318076, 2024-09-14T09:08:33.135 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.02941558, 2024-09-14T09:08:33.135 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.135 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.135 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.135 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.136 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.136 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73555 2024-09-14T09:08:33.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.136 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:08:33.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.136 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.136 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269236+0000", 2024-09-14T09:08:33.136 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.136 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.136 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269237+0000", 2024-09-14T09:08:33.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3399999999999999e-07 2024-09-14T09:08:33.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269240+0000", 2024-09-14T09:08:33.137 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8480000000000001e-06 2024-09-14T09:08:33.137 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.137 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.137 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.137 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269249+0000", 2024-09-14T09:08:33.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1880000000000008e-06 2024-09-14T09:08:33.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269256+0000", 2024-09-14T09:08:33.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:08:33.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.138 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282248+0000", 2024-09-14T09:08:33.138 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012992327 2024-09-14T09:08:33.138 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.138 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.138 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282251+0000", 2024-09-14T09:08:33.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2160000000000002e-06 2024-09-14T09:08:33.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298299+0000", 2024-09-14T09:08:33.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0160479330000001 2024-09-14T09:08:33.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.139 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.139 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.139 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298446+0000", 2024-09-14T09:08:33.139 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000146525 2024-09-14T09:08:33.139 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.140 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.140 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298652+0000", 2024-09-14T09:08:33.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020584100000000001 2024-09-14T09:08:33.140 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.140 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.140 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.140 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.140 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.140 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73556 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16465920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:33.140 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:33.140 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.79728833, 2024-09-14T09:08:33.140 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0299854239999999, 2024-09-14T09:08:33.140 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.141 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.141 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.141 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.141 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.141 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.141 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73556 2024-09-14T09:08:33.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.141 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.141 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.141 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:33.141 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.141 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.141 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:33.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269266+0000", 2024-09-14T09:08:33.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5199999999999996e-07 2024-09-14T09:08:33.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.142 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.142 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.142 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269270+0000", 2024-09-14T09:08:33.142 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0920000000000002e-06 2024-09-14T09:08:33.142 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269280+0000", 2024-09-14T09:08:33.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0013999999999999e-05 2024-09-14T09:08:33.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269288+0000", 2024-09-14T09:08:33.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:08:33.143 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.143 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.143 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.143 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282263+0000", 2024-09-14T09:08:33.143 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012974703000000001 2024-09-14T09:08:33.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282266+0000", 2024-09-14T09:08:33.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1520000000000001e-06 2024-09-14T09:08:33.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.144 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298729+0000", 2024-09-14T09:08:33.144 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.016463506 2024-09-14T09:08:33.144 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.144 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.144 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.298884+0000", 2024-09-14T09:08:33.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015450800000000001 2024-09-14T09:08:33.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.145 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.145 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299251+0000", 2024-09-14T09:08:33.145 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036759899999999997 2024-09-14T09:08:33.145 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.145 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.145 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.145 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.145 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.145 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73557 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16547840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:33.145 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:08:33.146 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.79724168, 2024-09-14T09:08:33.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0307969480000001, 2024-09-14T09:08:33.146 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.146 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.146 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.146 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.146 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.146 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.146 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73557 2024-09-14T09:08:33.146 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.146 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.146 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.146 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.146 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:08:33.146 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269312+0000", 2024-09-14T09:08:33.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.147 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269313+0000", 2024-09-14T09:08:33.147 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.92e-07 2024-09-14T09:08:33.147 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.147 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.147 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269336+0000", 2024-09-14T09:08:33.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3118000000000001e-05 2024-09-14T09:08:33.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269346+0000", 2024-09-14T09:08:33.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4069999999999999e-06 2024-09-14T09:08:33.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.148 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.148 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269352+0000", 2024-09-14T09:08:33.148 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5810000000000002e-06 2024-09-14T09:08:33.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.148 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282277+0000", 2024-09-14T09:08:33.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012924658 2024-09-14T09:08:33.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282280+0000", 2024-09-14T09:08:33.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2880000000000001e-06 2024-09-14T09:08:33.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.149 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.149 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299343+0000", 2024-09-14T09:08:33.149 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0170630190000001 2024-09-14T09:08:33.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.149 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.299554+0000", 2024-09-14T09:08:33.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021085099999999999 2024-09-14T09:08:33.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.150 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.150 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300109+0000", 2024-09-14T09:08:33.150 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00055533399999999999 2024-09-14T09:08:33.150 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.150 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.150 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.150 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.150 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.150 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73558 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16629760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:33.150 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:08:33.151 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.79719005699999, 2024-09-14T09:08:33.151 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0314849340000001, 2024-09-14T09:08:33.151 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.151 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.151 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.151 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.151 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.151 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.151 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73558 2024-09-14T09:08:33.151 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.151 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.151 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.151 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:08:33.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269364+0000", 2024-09-14T09:08:33.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.152 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.152 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.152 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269365+0000", 2024-09-14T09:08:33.152 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7400000000000003e-07 2024-09-14T09:08:33.152 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269367+0000", 2024-09-14T09:08:33.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4930000000000001e-06 2024-09-14T09:08:33.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269374+0000", 2024-09-14T09:08:33.153 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9519999999999997e-06 2024-09-14T09:08:33.153 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.153 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.153 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.153 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269379+0000", 2024-09-14T09:08:33.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9710000000000003e-06 2024-09-14T09:08:33.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282292+0000", 2024-09-14T09:08:33.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012912704000000001 2024-09-14T09:08:33.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282295+0000", 2024-09-14T09:08:33.154 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.179e-06 2024-09-14T09:08:33.154 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.154 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.154 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.154 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300195+0000", 2024-09-14T09:08:33.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.017899975 2024-09-14T09:08:33.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300345+0000", 2024-09-14T09:08:33.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014957499999999999 2024-09-14T09:08:33.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.155 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.155 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.155 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300849+0000", 2024-09-14T09:08:33.155 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050451100000000002 2024-09-14T09:08:33.155 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.155 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.155 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.155 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.156 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.156 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:73559 3.4 3:26a29cac:::rbd_data.115ebd4a9f36.0000000000000001:1b [read 16711680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e74)", 2024-09-14T09:08:33.156 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:33.156 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 118.797167648, 2024-09-14T09:08:33.156 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.031916343, 2024-09-14T09:08:33.156 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.156 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.156 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.156 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.156 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.156 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.156 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 73559 2024-09-14T09:08:33.156 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.156 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:33.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:33.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.157 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.157 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.157 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269387+0000", 2024-09-14T09:08:33.157 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3500000000000002e-07 2024-09-14T09:08:33.157 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269390+0000", 2024-09-14T09:08:33.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5890000000000001e-06 2024-09-14T09:08:33.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269396+0000", 2024-09-14T09:08:33.158 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.3300000000000004e-06 2024-09-14T09:08:33.158 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.158 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.158 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.158 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.269401+0000", 2024-09-14T09:08:33.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.0340000000000004e-06 2024-09-14T09:08:33.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282311+0000", 2024-09-14T09:08:33.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.012910204 2024-09-14T09:08:33.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.159 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:34.282317+0000", 2024-09-14T09:08:33.159 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2639999999999997e-06 2024-09-14T09:08:33.159 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.159 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.159 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.300933+0000", 2024-09-14T09:08:33.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0186160710000001 2024-09-14T09:08:33.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301158+0000", 2024-09-14T09:08:33.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000225021 2024-09-14T09:08:33.160 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.160 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.160 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.160 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:35.301303+0000", 2024-09-14T09:08:33.160 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014439499999999999 2024-09-14T09:08:33.160 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.160 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.161 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.161 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.161 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.161 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106964 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:33.161 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:08:33.161 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.951239814000004, 2024-09-14T09:08:33.161 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8802924409999999, 2024-09-14T09:08:33.161 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.161 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.161 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:33.161 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.161 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.161 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.162 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106964 2024-09-14T09:08:33.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.162 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:08:33.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.162 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115314+0000", 2024-09-14T09:08:33.162 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.162 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.162 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.162 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115317+0000", 2024-09-14T09:08:33.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6429999999999999e-06 2024-09-14T09:08:33.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115326+0000", 2024-09-14T09:08:33.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.2140000000000002e-06 2024-09-14T09:08:33.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.163 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.163 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115346+0000", 2024-09-14T09:08:33.163 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.9419000000000002e-05 2024-09-14T09:08:33.163 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.163 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115357+0000", 2024-09-14T09:08:33.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1014e-05 2024-09-14T09:08:33.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115431+0000", 2024-09-14T09:08:33.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4040000000000003e-05 2024-09-14T09:08:33.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.164 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.164 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.164 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.115438+0000", 2024-09-14T09:08:33.164 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9920000000000001e-06 2024-09-14T09:08:33.164 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844944+0000", 2024-09-14T09:08:33.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72950628399999995 2024-09-14T09:08:33.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.844961+0000", 2024-09-14T09:08:33.165 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.7156e-05 2024-09-14T09:08:33.165 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.165 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.165 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.165 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988004+0000", 2024-09-14T09:08:33.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.14304297 2024-09-14T09:08:33.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.988511+0000", 2024-09-14T09:08:33.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00050694100000000001 2024-09-14T09:08:33.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:08:33.166 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.989802+0000", 2024-09-14T09:08:33.166 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001291195 2024-09-14T09:08:33.166 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.166 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.166 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:33.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994866+0000", 2024-09-14T09:08:33.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0050632760000000002 2024-09-14T09:08:33.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:33.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.994988+0000", 2024-09-14T09:08:33.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012252899999999999 2024-09-14T09:08:33.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:33.167 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995198+0000", 2024-09-14T09:08:33.167 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021010899999999999 2024-09-14T09:08:33.167 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.167 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.167 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.168 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995607+0000", 2024-09-14T09:08:33.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040865900000000002 2024-09-14T09:08:33.168 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.168 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.168 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.168 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.168 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.168 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106965 3.4 3:22b82cab:::rbd_object_map.115e2588c85c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:33.168 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:33.168 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.950037437000006, 2024-09-14T09:08:33.168 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.879730162, 2024-09-14T09:08:33.168 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.168 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.168 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:33.169 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.169 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.169 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.169 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106965 2024-09-14T09:08:33.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.169 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.169 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:33.169 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.169 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.169 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.169 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:33.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116517+0000", 2024-09-14T09:08:33.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.5300000000000003e-07 2024-09-14T09:08:33.170 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.170 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.170 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.170 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116522+0000", 2024-09-14T09:08:33.170 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3989999999999997e-06 2024-09-14T09:08:33.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116533+0000", 2024-09-14T09:08:33.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0669000000000001e-05 2024-09-14T09:08:33.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.171 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116541+0000", 2024-09-14T09:08:33.171 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2730000000000002e-06 2024-09-14T09:08:33.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.171 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.171 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116604+0000", 2024-09-14T09:08:33.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.2706999999999994e-05 2024-09-14T09:08:33.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.116610+0000", 2024-09-14T09:08:33.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8630000000000002e-06 2024-09-14T09:08:33.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.172 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.172 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.172 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845060+0000", 2024-09-14T09:08:33.172 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.72845003699999999 2024-09-14T09:08:33.172 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.845079+0000", 2024-09-14T09:08:33.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.8287999999999998e-05 2024-09-14T09:08:33.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.991866+0000", 2024-09-14T09:08:33.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1467871519999999 2024-09-14T09:08:33.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.173 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.173 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.173 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.992274+0000", 2024-09-14T09:08:33.173 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040813500000000001 2024-09-14T09:08:33.173 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 7", 2024-09-14T09:08:33.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.993538+0000", 2024-09-14T09:08:33.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.001263655 2024-09-14T09:08:33.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:33.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995770+0000", 2024-09-14T09:08:33.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.002231949 2024-09-14T09:08:33.174 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.174 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.174 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:33.174 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995810+0000", 2024-09-14T09:08:33.174 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0852999999999997e-05 2024-09-14T09:08:33.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:33.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.995979+0000", 2024-09-14T09:08:33.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00016866 2024-09-14T09:08:33.175 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.175 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.175 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.175 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996247+0000", 2024-09-14T09:08:33.175 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000267769 2024-09-14T09:08:33.175 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.175 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.175 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.175 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.176 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.176 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:33.177 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T09:08:33.178 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:33.182+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:33.278 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:33.282+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:33.350 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:33.350 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:33.350 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:33.350 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:33.364 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T09:08:33.414 INFO:tasks.ceph.osd.3.smithi176.stderr:2024-09-14T09:08:33.414+0000 7f7eae43c640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26417) UID: 0 2024-09-14T09:08:33.480 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:33.482+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:33.483 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T09:08:33.494 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:33.495 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:33.495 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:08:33.495 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:08:33.495 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:33.512 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T09:08:33.581 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:33.582+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:33.682 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:33.682 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:08:33.682 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:08:33.682 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:08:33.682 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.682 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106705 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 518144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.682 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:33.682 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.835154631999998, 2024-09-14T09:08:33.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3001326799999999, 2024-09-14T09:08:33.683 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.683 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.683 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.683 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.683 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.683 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.683 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106705 2024-09-14T09:08:33.683 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.683 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.683 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.683 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.683 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:33.683 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:33.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.684 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.684 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847499+0000", 2024-09-14T09:08:33.684 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4699999999999998e-07 2024-09-14T09:08:33.684 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.684 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847500+0000", 2024-09-14T09:08:33.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0860000000000001e-06 2024-09-14T09:08:33.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847503+0000", 2024-09-14T09:08:33.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1269999999999999e-06 2024-09-14T09:08:33.685 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.685 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.685 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.685 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847506+0000", 2024-09-14T09:08:33.685 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5780000000000001e-06 2024-09-14T09:08:33.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858911+0000", 2024-09-14T09:08:33.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011404925 2024-09-14T09:08:33.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.686 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.686 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858919+0000", 2024-09-14T09:08:33.686 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8620000000000003e-06 2024-09-14T09:08:33.686 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.686 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147254+0000", 2024-09-14T09:08:33.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2883353719999999 2024-09-14T09:08:33.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147408+0000", 2024-09-14T09:08:33.687 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015389200000000001 2024-09-14T09:08:33.687 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.687 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.687 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.687 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147631+0000", 2024-09-14T09:08:33.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022359100000000001 2024-09-14T09:08:33.688 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.688 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.688 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.688 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.688 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.688 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106707 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 522240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.688 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:33.688 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.835144162000006, 2024-09-14T09:08:33.688 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.300471801, 2024-09-14T09:08:33.688 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.688 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.689 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.689 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.689 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.689 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.689 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106707 2024-09-14T09:08:33.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.689 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.689 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.689 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.689 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:33.689 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.689 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:33.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847509+0000", 2024-09-14T09:08:33.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.23e-07 2024-09-14T09:08:33.690 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.690 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.690 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.690 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847514+0000", 2024-09-14T09:08:33.690 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9150000000000004e-06 2024-09-14T09:08:33.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847520+0000", 2024-09-14T09:08:33.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.203e-06 2024-09-14T09:08:33.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847532+0000", 2024-09-14T09:08:33.691 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2441e-05 2024-09-14T09:08:33.691 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.691 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.691 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.691 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858953+0000", 2024-09-14T09:08:33.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01142108 2024-09-14T09:08:33.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858957+0000", 2024-09-14T09:08:33.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.2830000000000003e-06 2024-09-14T09:08:33.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.692 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.692 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147667+0000", 2024-09-14T09:08:33.692 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2887094619999999 2024-09-14T09:08:33.692 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.692 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147784+0000", 2024-09-14T09:08:33.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000117465 2024-09-14T09:08:33.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.693 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.693 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.147981+0000", 2024-09-14T09:08:33.693 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019672900000000001 2024-09-14T09:08:33.693 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.693 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.693 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.693 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.693 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.694 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106709 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 3072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.694 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:08:33.694 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.835115870999999, 2024-09-14T09:08:33.694 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.301335192, 2024-09-14T09:08:33.694 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.694 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.694 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.694 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.694 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.694 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.694 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106709 2024-09-14T09:08:33.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.694 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.694 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:08:33.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847537+0000", 2024-09-14T09:08:33.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.695 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.695 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.695 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847538+0000", 2024-09-14T09:08:33.695 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2800000000000003e-07 2024-09-14T09:08:33.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847539+0000", 2024-09-14T09:08:33.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2720000000000001e-06 2024-09-14T09:08:33.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847544+0000", 2024-09-14T09:08:33.696 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.7219999999999999e-06 2024-09-14T09:08:33.696 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.696 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.696 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.696 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847547+0000", 2024-09-14T09:08:33.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9960000000000002e-06 2024-09-14T09:08:33.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858991+0000", 2024-09-14T09:08:33.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011444218000000001 2024-09-14T09:08:33.697 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.697 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.697 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.697 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.858999+0000", 2024-09-14T09:08:33.697 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9419999999999995e-06 2024-09-14T09:08:33.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148053+0000", 2024-09-14T09:08:33.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2890544099999999 2024-09-14T09:08:33.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.698 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148203+0000", 2024-09-14T09:08:33.698 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149419 2024-09-14T09:08:33.698 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.698 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.698 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.699 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148873+0000", 2024-09-14T09:08:33.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00066988499999999995 2024-09-14T09:08:33.699 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.699 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.699 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.699 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.699 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.699 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106711 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 7168~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.699 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:33.699 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.835102613000004, 2024-09-14T09:08:33.699 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3022241219999999, 2024-09-14T09:08:33.699 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.699 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.699 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.700 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.700 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.700 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.700 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106711 2024-09-14T09:08:33.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.700 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.700 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:33.700 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.700 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.700 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.700 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:33.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847551+0000", 2024-09-14T09:08:33.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.8000000000000002e-07 2024-09-14T09:08:33.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.701 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.701 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.701 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847552+0000", 2024-09-14T09:08:33.701 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1340000000000001e-06 2024-09-14T09:08:33.701 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847557+0000", 2024-09-14T09:08:33.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6179999999999997e-06 2024-09-14T09:08:33.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847560+0000", 2024-09-14T09:08:33.702 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5899999999999999e-06 2024-09-14T09:08:33.702 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.702 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.702 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.702 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859050+0000", 2024-09-14T09:08:33.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01148914 2024-09-14T09:08:33.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859056+0000", 2024-09-14T09:08:33.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6630000000000004e-06 2024-09-14T09:08:33.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.703 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.703 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.148948+0000", 2024-09-14T09:08:33.703 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2898914370000001 2024-09-14T09:08:33.703 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.703 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149079+0000", 2024-09-14T09:08:33.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013145400000000001 2024-09-14T09:08:33.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.704 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.704 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149775+0000", 2024-09-14T09:08:33.704 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00069580599999999999 2024-09-14T09:08:33.704 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.704 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.704 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.704 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.704 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.704 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106713 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 11264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.705 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:33.705 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.835089147999994, 2024-09-14T09:08:33.705 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3035188959999999, 2024-09-14T09:08:33.705 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.705 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.705 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.705 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.705 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.705 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.705 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106713 2024-09-14T09:08:33.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.705 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.705 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:33.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:33.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.706 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.706 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847564+0000", 2024-09-14T09:08:33.706 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6e-07 2024-09-14T09:08:33.706 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.706 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847566+0000", 2024-09-14T09:08:33.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1039999999999999e-06 2024-09-14T09:08:33.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847569+0000", 2024-09-14T09:08:33.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.191e-06 2024-09-14T09:08:33.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.707 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847571+0000", 2024-09-14T09:08:33.707 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6390000000000001e-06 2024-09-14T09:08:33.707 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.707 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.707 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859099+0000", 2024-09-14T09:08:33.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011527379000000001 2024-09-14T09:08:33.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859106+0000", 2024-09-14T09:08:33.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3679999999999996e-06 2024-09-14T09:08:33.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.708 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.708 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.149878+0000", 2024-09-14T09:08:33.708 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.29077208 2024-09-14T09:08:33.708 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.708 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.150175+0000", 2024-09-14T09:08:33.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029685699999999998 2024-09-14T09:08:33.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.709 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.709 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151083+0000", 2024-09-14T09:08:33.709 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00090801800000000002 2024-09-14T09:08:33.709 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.709 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.709 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.709 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.709 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.709 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106715 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 15360~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.710 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:33.710 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.835078139000004, 2024-09-14T09:08:33.710 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3047478800000001, 2024-09-14T09:08:33.710 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.710 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.710 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.710 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.710 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.710 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.710 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106715 2024-09-14T09:08:33.710 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.710 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.710 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.710 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:33.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:33.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.711 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.711 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847575+0000", 2024-09-14T09:08:33.711 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6e-07 2024-09-14T09:08:33.711 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.711 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847580+0000", 2024-09-14T09:08:33.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9250000000000003e-06 2024-09-14T09:08:33.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847585+0000", 2024-09-14T09:08:33.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8409999999999999e-06 2024-09-14T09:08:33.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.712 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.712 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.712 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847588+0000", 2024-09-14T09:08:33.712 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6989999999999999e-06 2024-09-14T09:08:33.712 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859151+0000", 2024-09-14T09:08:33.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011562604000000001 2024-09-14T09:08:33.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.713 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859159+0000", 2024-09-14T09:08:33.713 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4179999999999997e-06 2024-09-14T09:08:33.713 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.713 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.713 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151155+0000", 2024-09-14T09:08:33.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.291996068 2024-09-14T09:08:33.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.151383+0000", 2024-09-14T09:08:33.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00022746100000000001 2024-09-14T09:08:33.714 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.714 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.714 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.714 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152323+0000", 2024-09-14T09:08:33.714 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00094060400000000005 2024-09-14T09:08:33.714 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.714 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.715 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.715 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.715 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.715 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106717 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 19456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.715 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:33.715 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.835061451000001, 2024-09-14T09:08:33.715 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.305574579, 2024-09-14T09:08:33.715 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.715 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.715 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.715 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.715 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.715 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.715 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106717 2024-09-14T09:08:33.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.716 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:33.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.716 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:33.716 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.716 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.716 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.716 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847592+0000", 2024-09-14T09:08:33.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1199999999999999e-07 2024-09-14T09:08:33.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.717 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.717 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847593+0000", 2024-09-14T09:08:33.717 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.257e-06 2024-09-14T09:08:33.717 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.717 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847598+0000", 2024-09-14T09:08:33.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4139999999999996e-06 2024-09-14T09:08:33.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847602+0000", 2024-09-14T09:08:33.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3440000000000003e-06 2024-09-14T09:08:33.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.718 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.718 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.718 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859203+0000", 2024-09-14T09:08:33.718 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011600572999999999 2024-09-14T09:08:33.718 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859211+0000", 2024-09-14T09:08:33.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.8299999999999996e-06 2024-09-14T09:08:33.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152398+0000", 2024-09-14T09:08:33.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.293187812 2024-09-14T09:08:33.719 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.719 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.719 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.719 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.152543+0000", 2024-09-14T09:08:33.719 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014478000000000001 2024-09-14T09:08:33.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.720 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.720 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153166+0000", 2024-09-14T09:08:33.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000623257 2024-09-14T09:08:33.720 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.720 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.720 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.720 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.720 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.720 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106719 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 23552~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.720 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:33.720 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.835047134999996, 2024-09-14T09:08:33.720 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.306394982, 2024-09-14T09:08:33.721 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.721 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.721 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.721 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.721 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.721 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.721 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106719 2024-09-14T09:08:33.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.721 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.721 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.721 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.721 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:33.721 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.721 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:33.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847606+0000", 2024-09-14T09:08:33.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5100000000000001e-07 2024-09-14T09:08:33.722 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.722 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.722 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.722 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847608+0000", 2024-09-14T09:08:33.722 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.189e-06 2024-09-14T09:08:33.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847612+0000", 2024-09-14T09:08:33.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9519999999999996e-06 2024-09-14T09:08:33.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847614+0000", 2024-09-14T09:08:33.723 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6280000000000001e-06 2024-09-14T09:08:33.723 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.723 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.723 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.723 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859254+0000", 2024-09-14T09:08:33.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011640157 2024-09-14T09:08:33.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859261+0000", 2024-09-14T09:08:33.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0570000000000004e-06 2024-09-14T09:08:33.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.724 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.724 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153236+0000", 2024-09-14T09:08:33.724 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2939744040000001 2024-09-14T09:08:33.724 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.724 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.153380+0000", 2024-09-14T09:08:33.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014410399999999999 2024-09-14T09:08:33.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.725 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.725 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154001+0000", 2024-09-14T09:08:33.725 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062124000000000001 2024-09-14T09:08:33.725 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.725 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.725 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.725 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.725 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.726 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106721 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 27648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.726 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:33.726 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.835035485000006, 2024-09-14T09:08:33.726 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3074446659999999, 2024-09-14T09:08:33.726 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.726 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.726 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.726 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.726 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.726 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.726 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106721 2024-09-14T09:08:33.726 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.727 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:33.727 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.727 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.727 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.727 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.727 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:33.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847618+0000", 2024-09-14T09:08:33.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4299999999999999e-07 2024-09-14T09:08:33.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.728 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.728 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.728 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847619+0000", 2024-09-14T09:08:33.728 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.057e-06 2024-09-14T09:08:33.728 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847624+0000", 2024-09-14T09:08:33.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4229999999999998e-06 2024-09-14T09:08:33.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.729 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847627+0000", 2024-09-14T09:08:33.729 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0120000000000001e-06 2024-09-14T09:08:33.729 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.729 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.729 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859305+0000", 2024-09-14T09:08:33.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01167812 2024-09-14T09:08:33.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859313+0000", 2024-09-14T09:08:33.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2409999999999995e-06 2024-09-14T09:08:33.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.730 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.730 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.730 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154081+0000", 2024-09-14T09:08:33.730 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.294768296 2024-09-14T09:08:33.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.154280+0000", 2024-09-14T09:08:33.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00019824 2024-09-14T09:08:33.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.731 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.731 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155063+0000", 2024-09-14T09:08:33.731 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00078303400000000005 2024-09-14T09:08:33.731 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.731 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.731 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.731 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.731 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106723 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 31744~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.732 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:08:33.732 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.835023054999994, 2024-09-14T09:08:33.732 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.308293648, 2024-09-14T09:08:33.732 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.732 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.732 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.732 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.732 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.732 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.732 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106723 2024-09-14T09:08:33.732 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.732 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.732 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.732 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:08:33.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847630+0000", 2024-09-14T09:08:33.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.733 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.733 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847631+0000", 2024-09-14T09:08:33.733 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6100000000000002e-07 2024-09-14T09:08:33.733 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.733 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847632+0000", 2024-09-14T09:08:33.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0589999999999999e-06 2024-09-14T09:08:33.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847635+0000", 2024-09-14T09:08:33.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.551e-06 2024-09-14T09:08:33.734 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.734 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.734 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.734 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847638+0000", 2024-09-14T09:08:33.734 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.644e-06 2024-09-14T09:08:33.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859344+0000", 2024-09-14T09:08:33.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011706397 2024-09-14T09:08:33.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859348+0000", 2024-09-14T09:08:33.735 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5779999999999999e-06 2024-09-14T09:08:33.735 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.735 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.735 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.735 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155135+0000", 2024-09-14T09:08:33.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.295787448 2024-09-14T09:08:33.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155279+0000", 2024-09-14T09:08:33.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00014400700000000001 2024-09-14T09:08:33.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.736 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.736 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.736 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155924+0000", 2024-09-14T09:08:33.736 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00064470300000000005 2024-09-14T09:08:33.736 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.736 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.736 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.736 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.737 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.737 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106725 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 35840~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.737 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:08:33.737 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.835011944000001, 2024-09-14T09:08:33.737 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.309103218, 2024-09-14T09:08:33.737 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.737 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.737 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.737 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.737 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.737 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.737 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106725 2024-09-14T09:08:33.737 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.738 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:08:33.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.738 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847641+0000", 2024-09-14T09:08:33.738 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.738 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.738 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.738 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847642+0000", 2024-09-14T09:08:33.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.4499999999999998e-07 2024-09-14T09:08:33.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847643+0000", 2024-09-14T09:08:33.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4e-07 2024-09-14T09:08:33.739 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.739 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.739 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.739 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847647+0000", 2024-09-14T09:08:33.739 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6190000000000002e-06 2024-09-14T09:08:33.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847655+0000", 2024-09-14T09:08:33.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1980000000000001e-06 2024-09-14T09:08:33.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.740 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.740 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859381+0000", 2024-09-14T09:08:33.740 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011725793 2024-09-14T09:08:33.740 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.740 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859391+0000", 2024-09-14T09:08:33.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4390000000000006e-06 2024-09-14T09:08:33.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.741 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.741 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.155997+0000", 2024-09-14T09:08:33.741 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2966067830000001 2024-09-14T09:08:33.741 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.741 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156133+0000", 2024-09-14T09:08:33.742 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000135609 2024-09-14T09:08:33.742 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.742 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.742 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.742 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156745+0000", 2024-09-14T09:08:33.743 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000611592 2024-09-14T09:08:33.743 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.743 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.743 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.743 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.743 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.743 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106727 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 39936~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.743 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:33.743 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.834993643999994, 2024-09-14T09:08:33.744 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3099046560000001, 2024-09-14T09:08:33.744 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.744 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.744 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.744 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.744 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106727 2024-09-14T09:08:33.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.745 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:33.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.745 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.745 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.745 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.745 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:33.745 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847660+0000", 2024-09-14T09:08:33.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1300000000000001e-07 2024-09-14T09:08:33.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847661+0000", 2024-09-14T09:08:33.746 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.192e-06 2024-09-14T09:08:33.746 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.746 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.746 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.746 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847665+0000", 2024-09-14T09:08:33.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0999999999999997e-06 2024-09-14T09:08:33.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.747 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.747 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.747 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847668+0000", 2024-09-14T09:08:33.747 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.644e-06 2024-09-14T09:08:33.747 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859422+0000", 2024-09-14T09:08:33.748 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011753559 2024-09-14T09:08:33.748 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.748 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.748 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.748 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859425+0000", 2024-09-14T09:08:33.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.343e-06 2024-09-14T09:08:33.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.749 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156817+0000", 2024-09-14T09:08:33.749 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2973924800000001 2024-09-14T09:08:33.749 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.749 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.749 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.156956+0000", 2024-09-14T09:08:33.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013835 2024-09-14T09:08:33.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.750 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.750 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157564+0000", 2024-09-14T09:08:33.750 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060867500000000004 2024-09-14T09:08:33.750 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.750 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.750 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.750 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.750 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.751 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106729 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 44032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.751 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:33.751 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.834981670999994, 2024-09-14T09:08:33.751 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3107195970000001, 2024-09-14T09:08:33.751 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.751 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.751 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.751 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.752 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.752 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.752 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106729 2024-09-14T09:08:33.752 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.752 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.752 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:33.753 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.753 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.753 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.753 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.753 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:33.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847672+0000", 2024-09-14T09:08:33.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.29e-07 2024-09-14T09:08:33.754 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.754 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.754 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.754 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847673+0000", 2024-09-14T09:08:33.754 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.096e-06 2024-09-14T09:08:33.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847678+0000", 2024-09-14T09:08:33.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.527e-06 2024-09-14T09:08:33.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847681+0000", 2024-09-14T09:08:33.755 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.788e-06 2024-09-14T09:08:33.755 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.755 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.755 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.755 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859462+0000", 2024-09-14T09:08:33.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01178032 2024-09-14T09:08:33.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859471+0000", 2024-09-14T09:08:33.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.3430000000000002e-06 2024-09-14T09:08:33.756 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.756 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.756 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.756 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157635+0000", 2024-09-14T09:08:33.756 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2981640320000001 2024-09-14T09:08:33.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.157773+0000", 2024-09-14T09:08:33.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013763 2024-09-14T09:08:33.757 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.757 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.757 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.757 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158391+0000", 2024-09-14T09:08:33.757 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00061863200000000004 2024-09-14T09:08:33.757 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.757 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.757 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.758 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.758 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.758 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106731 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 48128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.758 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:08:33.758 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.834966026000004, 2024-09-14T09:08:33.758 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3115142020000001, 2024-09-14T09:08:33.758 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.758 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.758 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.758 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.759 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106731 2024-09-14T09:08:33.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.759 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:08:33.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.759 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.759 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.759 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847687+0000", 2024-09-14T09:08:33.759 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.759 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847688+0000", 2024-09-14T09:08:33.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0199999999999998e-07 2024-09-14T09:08:33.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.760 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.760 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847689+0000", 2024-09-14T09:08:33.760 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4529999999999999e-06 2024-09-14T09:08:33.760 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.760 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847694+0000", 2024-09-14T09:08:33.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9080000000000003e-06 2024-09-14T09:08:33.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847697+0000", 2024-09-14T09:08:33.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4920000000000002e-06 2024-09-14T09:08:33.761 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.761 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.761 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.761 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859499+0000", 2024-09-14T09:08:33.761 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011801676000000001 2024-09-14T09:08:33.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859502+0000", 2024-09-14T09:08:33.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.2899999999999998e-06 2024-09-14T09:08:33.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.762 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.762 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.762 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158461+0000", 2024-09-14T09:08:33.762 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.298958796 2024-09-14T09:08:33.762 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.158594+0000", 2024-09-14T09:08:33.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013321599999999999 2024-09-14T09:08:33.763 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.763 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.763 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.763 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159202+0000", 2024-09-14T09:08:33.763 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060706900000000001 2024-09-14T09:08:33.764 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.764 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.764 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.764 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.764 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.764 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106733 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 52224~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.764 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:33.764 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.834951770000004, 2024-09-14T09:08:33.764 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3123343809999999, 2024-09-14T09:08:33.765 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.765 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.765 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.765 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.765 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.765 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.765 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106733 2024-09-14T09:08:33.765 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.765 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.766 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:33.766 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.766 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.766 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.766 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:33.767 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.767 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.767 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.767 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.767 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847702+0000", 2024-09-14T09:08:33.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5800000000000001e-07 2024-09-14T09:08:33.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.768 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847703+0000", 2024-09-14T09:08:33.768 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.127e-06 2024-09-14T09:08:33.768 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.768 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.768 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847706+0000", 2024-09-14T09:08:33.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3210000000000001e-06 2024-09-14T09:08:33.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.769 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.769 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847709+0000", 2024-09-14T09:08:33.769 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.7539999999999998e-06 2024-09-14T09:08:33.769 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.769 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.770 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.770 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859538+0000", 2024-09-14T09:08:33.770 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.01182942 2024-09-14T09:08:33.770 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.770 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859547+0000", 2024-09-14T09:08:33.771 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1450000000000008e-06 2024-09-14T09:08:33.771 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.771 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.771 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.771 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159273+0000", 2024-09-14T09:08:33.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2997263700000001 2024-09-14T09:08:33.772 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.772 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.772 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.772 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.159415+0000", 2024-09-14T09:08:33.772 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000141765 2024-09-14T09:08:33.773 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.773 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.773 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.773 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160036+0000", 2024-09-14T09:08:33.773 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062122100000000003 2024-09-14T09:08:33.773 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.774 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.774 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.774 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.774 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.774 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106735 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 56320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.774 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:33.775 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.834940325999995, 2024-09-14T09:08:33.775 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3131339309999999, 2024-09-14T09:08:33.775 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.775 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.775 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.775 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.775 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.776 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.776 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106735 2024-09-14T09:08:33.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.776 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.776 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.776 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.776 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:33.776 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.776 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:33.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.777 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.777 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.777 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.777 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847713+0000", 2024-09-14T09:08:33.777 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.2100000000000001e-07 2024-09-14T09:08:33.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.778 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847714+0000", 2024-09-14T09:08:33.778 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.051e-06 2024-09-14T09:08:33.778 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.778 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.778 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847719+0000", 2024-09-14T09:08:33.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3640000000000001e-06 2024-09-14T09:08:33.779 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.779 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.779 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.779 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847723+0000", 2024-09-14T09:08:33.779 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8789999999999996e-06 2024-09-14T09:08:33.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.780 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.780 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.780 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859577+0000", 2024-09-14T09:08:33.780 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011854416 2024-09-14T09:08:33.780 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.781 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859580+0000", 2024-09-14T09:08:33.781 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.292e-06 2024-09-14T09:08:33.781 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.781 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.781 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160108+0000", 2024-09-14T09:08:33.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.30052758 2024-09-14T09:08:33.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.782 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.782 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160238+0000", 2024-09-14T09:08:33.782 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013043500000000001 2024-09-14T09:08:33.782 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.782 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.783 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.783 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160847+0000", 2024-09-14T09:08:33.783 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060869299999999999 2024-09-14T09:08:33.783 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.783 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.783 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.783 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.783 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.784 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106738 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 60416~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.784 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:33.784 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.834927121999996, 2024-09-14T09:08:33.784 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3139482179999999, 2024-09-14T09:08:33.784 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.784 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.784 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.784 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.784 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.784 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.784 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106738 2024-09-14T09:08:33.784 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.785 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.785 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.785 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:33.785 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.785 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.785 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:33.786 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.786 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.786 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.786 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.786 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847726+0000", 2024-09-14T09:08:33.787 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.3999999999999998e-07 2024-09-14T09:08:33.787 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.787 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.787 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.787 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847733+0000", 2024-09-14T09:08:33.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8070000000000002e-06 2024-09-14T09:08:33.788 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.788 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.788 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.788 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847738+0000", 2024-09-14T09:08:33.788 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9740000000000001e-06 2024-09-14T09:08:33.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.789 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.789 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.789 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847742+0000", 2024-09-14T09:08:33.789 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.6009999999999999e-06 2024-09-14T09:08:33.789 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.790 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.790 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859612+0000", 2024-09-14T09:08:33.790 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011870537 2024-09-14T09:08:33.790 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.790 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.791 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859621+0000", 2024-09-14T09:08:33.791 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3019999999999995e-06 2024-09-14T09:08:33.791 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.791 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.791 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.792 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.160917+0000", 2024-09-14T09:08:33.792 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3012961080000001 2024-09-14T09:08:33.792 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.792 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.792 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161054+0000", 2024-09-14T09:08:33.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000137422 2024-09-14T09:08:33.793 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.793 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.793 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.793 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161674+0000", 2024-09-14T09:08:33.793 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000620227 2024-09-14T09:08:33.794 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.794 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.794 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.794 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.794 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.795 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106739 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 64512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.795 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:33.795 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.834907282000003, 2024-09-14T09:08:33.795 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.314753777, 2024-09-14T09:08:33.795 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.795 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.795 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.796 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.796 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.796 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.796 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106739 2024-09-14T09:08:33.796 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.796 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.796 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.796 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:33.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.797 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.797 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:33.797 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.797 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.797 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847746+0000", 2024-09-14T09:08:33.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9700000000000003e-07 2024-09-14T09:08:33.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.798 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847748+0000", 2024-09-14T09:08:33.798 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2589999999999999e-06 2024-09-14T09:08:33.798 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.798 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.798 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847752+0000", 2024-09-14T09:08:33.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.6099999999999999e-06 2024-09-14T09:08:33.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.799 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.799 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.799 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847755+0000", 2024-09-14T09:08:33.799 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.0520000000000002e-06 2024-09-14T09:08:33.799 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.800 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859653+0000", 2024-09-14T09:08:33.800 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011897575000000001 2024-09-14T09:08:33.800 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.800 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.800 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859658+0000", 2024-09-14T09:08:33.801 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8810000000000004e-06 2024-09-14T09:08:33.801 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.801 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.801 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.801 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161745+0000", 2024-09-14T09:08:33.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3020870819999999 2024-09-14T09:08:33.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.802 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.802 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.802 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.161877+0000", 2024-09-14T09:08:33.802 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000132657 2024-09-14T09:08:33.802 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.803 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.803 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.803 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162500+0000", 2024-09-14T09:08:33.803 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00062236400000000003 2024-09-14T09:08:33.803 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.803 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.804 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.804 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.804 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.804 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106741 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 68608~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.804 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:33.804 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.834894489999996, 2024-09-14T09:08:33.804 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3155568449999999, 2024-09-14T09:08:33.804 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.805 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.805 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.805 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.805 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.805 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.805 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106741 2024-09-14T09:08:33.805 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.805 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.805 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.805 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.805 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:33.805 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:33.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.806 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.806 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847759+0000", 2024-09-14T09:08:33.806 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.5800000000000001e-07 2024-09-14T09:08:33.806 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.806 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847760+0000", 2024-09-14T09:08:33.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.19e-06 2024-09-14T09:08:33.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.807 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.807 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.807 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847764+0000", 2024-09-14T09:08:33.807 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3189999999999999e-06 2024-09-14T09:08:33.807 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.808 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847766+0000", 2024-09-14T09:08:33.808 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.6429999999999999e-06 2024-09-14T09:08:33.808 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.808 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.808 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859693+0000", 2024-09-14T09:08:33.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.011926724 2024-09-14T09:08:33.809 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.809 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.809 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:33.809 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.859703+0000", 2024-09-14T09:08:33.809 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6160000000000008e-06 2024-09-14T09:08:33.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.810 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.810 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162575+0000", 2024-09-14T09:08:33.810 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.302872128 2024-09-14T09:08:33.810 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.810 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.162709+0000", 2024-09-14T09:08:33.811 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013446000000000001 2024-09-14T09:08:33.811 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.811 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.811 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.811 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163316+0000", 2024-09-14T09:08:33.812 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00060650700000000005 2024-09-14T09:08:33.812 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.812 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.812 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.812 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.812 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.812 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:106744 3.13 3:cab6b86e:::rbd_data.115e500fbcd9.000000000000000f:27 [read 76800~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:33.813 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:08:33.813 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 71.834883379000004, 2024-09-14T09:08:33.813 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3163901789999999, 2024-09-14T09:08:33.813 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:33.813 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:33.813 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:33.813 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:33.813 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:33.813 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:33.814 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 106744 2024-09-14T09:08:33.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.814 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:33.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:33.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:08:33.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.814 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.814 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.814 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:33.814 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:21.847770+0000", 2024-09-14T09:08:33.814 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:33.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:33.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996947+0000", 2024-09-14T09:08:33.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.149177154 2024-09-14T09:08:33.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.815 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:33.815 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996964+0000", 2024-09-14T09:08:33.815 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.679e-05 2024-09-14T09:08:33.815 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.815 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:33.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.996999+0000", 2024-09-14T09:08:33.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5463000000000002e-05 2024-09-14T09:08:33.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.816 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:33.816 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:22.997029+0000", 2024-09-14T09:08:33.816 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 2.9567e-05 2024-09-14T09:08:33.816 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.816 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:33.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163390+0000", 2024-09-14T09:08:33.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.16636056799999999 2024-09-14T09:08:33.817 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.817 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.817 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:33.817 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.163526+0000", 2024-09-14T09:08:33.817 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00013655899999999999 2024-09-14T09:08:33.818 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:33.818 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:33.818 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:33.818 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:23.164160+0000", 2024-09-14T09:08:33.818 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00063407800000000005 2024-09-14T09:08:33.818 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.818 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.819 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.819 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:33.819 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:33.819 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:33.820 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-09-14T09:08:33.686+0000 7fb5cc9ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26519) UID: 0 2024-09-14T09:08:33.822 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T09:08:33.823 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:33.826+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:33.924 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:33.930+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:33.950 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:33.950 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:33.951 INFO:teuthology.orchestra.run.smithi176.stdout: "num_ops": 0 2024-09-14T09:08:33.951 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:33.964 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T09:08:34.023 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:34.022+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:34.087 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:34.087 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [], 2024-09-14T09:08:34.087 INFO:teuthology.orchestra.run.smithi176.stdout: "complaint_time": 30, 2024-09-14T09:08:34.087 INFO:teuthology.orchestra.run.smithi176.stdout: "num_blocked_ops": 0 2024-09-14T09:08:34.087 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:34.101 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T09:08:34.124 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:34.126+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:34.224 INFO:tasks.ceph.osd.2.smithi176.stderr:2024-09-14T09:08:34.222+0000 7fce5fc6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26415) UID: 0 2024-09-14T09:08:34.274 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:08:34.274 INFO:teuthology.orchestra.run.smithi176.stdout: "size": 20, 2024-09-14T09:08:34.274 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 600, 2024-09-14T09:08:34.274 INFO:teuthology.orchestra.run.smithi176.stdout: "ops": [ 2024-09-14T09:08:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:634 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:08:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 181.19002623099999, 2024-09-14T09:08:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76416783200000005, 2024-09-14T09:08:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 634 2024-09-14T09:08:34.275 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:08:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083631+0000", 2024-09-14T09:08:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.276 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083632+0000", 2024-09-14T09:08:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7900000000000007e-07 2024-09-14T09:08:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083637+0000", 2024-09-14T09:08:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.3240000000000002e-06 2024-09-14T09:08:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083648+0000", 2024-09-14T09:08:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:08:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.277 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083656+0000", 2024-09-14T09:08:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6709999999999999e-06 2024-09-14T09:08:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083728+0000", 2024-09-14T09:08:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1923999999999998e-05 2024-09-14T09:08:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083740+0000", 2024-09-14T09:08:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1922999999999999e-05 2024-09-14T09:08:34.278 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847412+0000", 2024-09-14T09:08:34.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76367168799999996 2024-09-14T09:08:34.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847626+0000", 2024-09-14T09:08:34.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021431099999999999 2024-09-14T09:08:34.279 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.279 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.279 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.279 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.847799+0000", 2024-09-14T09:08:34.279 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017274200000000001 2024-09-14T09:08:34.279 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:637 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 2883584~50170] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:08:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 181.18997157300001, 2024-09-14T09:08:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76591892800000005, 2024-09-14T09:08:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:34.280 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 637 2024-09-14T09:08:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:08:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:08:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.281 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083686+0000", 2024-09-14T09:08:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.4900000000000005e-07 2024-09-14T09:08:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083690+0000", 2024-09-14T09:08:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.675e-06 2024-09-14T09:08:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.282 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083699+0000", 2024-09-14T09:08:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5830000000000007e-06 2024-09-14T09:08:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083706+0000", 2024-09-14T09:08:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4490000000000002e-06 2024-09-14T09:08:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083762+0000", 2024-09-14T09:08:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5856999999999997e-05 2024-09-14T09:08:34.283 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083766+0000", 2024-09-14T09:08:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:08:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848513+0000", 2024-09-14T09:08:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.764747171 2024-09-14T09:08:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.848845+0000", 2024-09-14T09:08:34.284 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00033196799999999998 2024-09-14T09:08:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849604+0000", 2024-09-14T09:08:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00075922800000000003 2024-09-14T09:08:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:638 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:08:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 181.18986619899999, 2024-09-14T09:08:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76635832299999995, 2024-09-14T09:08:34.285 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 638 2024-09-14T09:08:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:08:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.286 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083791+0000", 2024-09-14T09:08:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083792+0000", 2024-09-14T09:08:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.238e-06 2024-09-14T09:08:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083799+0000", 2024-09-14T09:08:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.7159999999999997e-06 2024-09-14T09:08:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.287 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083812+0000", 2024-09-14T09:08:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3614e-05 2024-09-14T09:08:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083821+0000", 2024-09-14T09:08:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8929999999999994e-06 2024-09-14T09:08:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083877+0000", 2024-09-14T09:08:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.5395000000000001e-05 2024-09-14T09:08:34.288 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.083894+0000", 2024-09-14T09:08:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.6824e-05 2024-09-14T09:08:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849749+0000", 2024-09-14T09:08:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76585529699999999 2024-09-14T09:08:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.849938+0000", 2024-09-14T09:08:34.289 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00018888499999999999 2024-09-14T09:08:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850149+0000", 2024-09-14T09:08:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000211461 2024-09-14T09:08:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:647 3.18 3:1d8f60b0:::rbd_data.115e724e1d99.0000000000000008:3 [sparse-read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:34.290 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:08:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 181.18950943499999, 2024-09-14T09:08:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76637317199999999, 2024-09-14T09:08:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 647 2024-09-14T09:08:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.291 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:08:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084148+0000", 2024-09-14T09:08:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084149+0000", 2024-09-14T09:08:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.203e-06 2024-09-14T09:08:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.292 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084155+0000", 2024-09-14T09:08:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6339999999999998e-06 2024-09-14T09:08:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084164+0000", 2024-09-14T09:08:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5189999999999998e-06 2024-09-14T09:08:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084168+0000", 2024-09-14T09:08:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4170000000000002e-06 2024-09-14T09:08:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.293 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084214+0000", 2024-09-14T09:08:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5191999999999997e-05 2024-09-14T09:08:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084222+0000", 2024-09-14T09:08:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.9510000000000006e-06 2024-09-14T09:08:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850189+0000", 2024-09-14T09:08:34.294 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76596774300000003 2024-09-14T09:08:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850318+0000", 2024-09-14T09:08:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00012850900000000001 2024-09-14T09:08:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850521+0000", 2024-09-14T09:08:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00020300400000000001 2024-09-14T09:08:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.295 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:650 3.18 3:1af09a33:::rbd_data.115e724e1d99.0000000000000014:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:08:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 181.18942469699999, 2024-09-14T09:08:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76666613800000005, 2024-09-14T09:08:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 650 2024-09-14T09:08:34.296 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:08:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084232+0000", 2024-09-14T09:08:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.297 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084233+0000", 2024-09-14T09:08:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.78e-07 2024-09-14T09:08:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084238+0000", 2024-09-14T09:08:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8389999999999997e-06 2024-09-14T09:08:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084249+0000", 2024-09-14T09:08:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0556e-05 2024-09-14T09:08:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.298 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084256+0000", 2024-09-14T09:08:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.9199999999999998e-06 2024-09-14T09:08:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084342+0000", 2024-09-14T09:08:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6181999999999996e-05 2024-09-14T09:08:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084351+0000", 2024-09-14T09:08:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.6120000000000005e-06 2024-09-14T09:08:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850565+0000", 2024-09-14T09:08:34.299 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76621391299999997 2024-09-14T09:08:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850682+0000", 2024-09-14T09:08:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000116641 2024-09-14T09:08:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.850899+0000", 2024-09-14T09:08:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00021659700000000001 2024-09-14T09:08:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.300 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:651 3.18 3:188a8141:::rbd_data.115e724e1d99.000000000000001e:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-14T09:08:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:08:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 181.18933701099999, 2024-09-14T09:08:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76711797299999995, 2024-09-14T09:08:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 651 2024-09-14T09:08:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.301 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:08:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084320+0000", 2024-09-14T09:08:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084321+0000", 2024-09-14T09:08:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9299999999999996e-07 2024-09-14T09:08:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084325+0000", 2024-09-14T09:08:34.304 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.1760000000000003e-06 2024-09-14T09:08:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084337+0000", 2024-09-14T09:08:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1928e-05 2024-09-14T09:08:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084345+0000", 2024-09-14T09:08:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0150000000000003e-06 2024-09-14T09:08:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.305 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084377+0000", 2024-09-14T09:08:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.1897999999999999e-05 2024-09-14T09:08:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.084381+0000", 2024-09-14T09:08:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.3800000000000004e-06 2024-09-14T09:08:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851112+0000", 2024-09-14T09:08:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.76673030600000003 2024-09-14T09:08:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.306 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851261+0000", 2024-09-14T09:08:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000149307 2024-09-14T09:08:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:05:33.851438+0000", 2024-09-14T09:08:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00017707000000000001 2024-09-14T09:08:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.307 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:39401 3.b 3:d4914ce3:::rbd_data.115ea0ee3dc0.0000000000000000:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T09:08:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:08:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 148.113045024, 2024-09-14T09:08:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.13041323899999999, 2024-09-14T09:08:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.308 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 39401 2024-09-14T09:08:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:08:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160612+0000", 2024-09-14T09:08:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.309 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160613+0000", 2024-09-14T09:08:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3399999999999998e-07 2024-09-14T09:08:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160617+0000", 2024-09-14T09:08:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.0799999999999999e-06 2024-09-14T09:08:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160627+0000", 2024-09-14T09:08:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0026e-05 2024-09-14T09:08:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.310 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160633+0000", 2024-09-14T09:08:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6679999999999998e-06 2024-09-14T09:08:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160709+0000", 2024-09-14T09:08:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6465999999999996e-05 2024-09-14T09:08:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.160790+0000", 2024-09-14T09:08:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1045999999999996e-05 2024-09-14T09:08:34.311 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 0", 2024-09-14T09:08:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.167913+0000", 2024-09-14T09:08:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.007122438 2024-09-14T09:08:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290937+0000", 2024-09-14T09:08:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123024381 2024-09-14T09:08:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290952+0000", 2024-09-14T09:08:34.312 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.4997000000000001e-05 2024-09-14T09:08:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.290987+0000", 2024-09-14T09:08:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.4601999999999998e-05 2024-09-14T09:08:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:06:06.291025+0000", 2024-09-14T09:08:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.8800999999999997e-05 2024-09-14T09:08:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:97796 3.15 3:aaaa5f7f:::rbd_data.115e113dc8ca.0000000000000029:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T09:08:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:08:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 82.179176154000004, 2024-09-14T09:08:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.14537731300000001, 2024-09-14T09:08:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 97796 2024-09-14T09:08:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.314 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:08:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094481+0000", 2024-09-14T09:08:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.315 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094482+0000", 2024-09-14T09:08:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.6400000000000001e-07 2024-09-14T09:08:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094485+0000", 2024-09-14T09:08:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.693e-06 2024-09-14T09:08:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094494+0000", 2024-09-14T09:08:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.9590000000000001e-06 2024-09-14T09:08:34.316 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094502+0000", 2024-09-14T09:08:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3180000000000001e-06 2024-09-14T09:08:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094567+0000", 2024-09-14T09:08:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.5526999999999998e-05 2024-09-14T09:08:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.317 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.094726+0000", 2024-09-14T09:08:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015842499999999999 2024-09-14T09:08:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "waiting for subops from 4", 2024-09-14T09:08:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.101940+0000", 2024-09-14T09:08:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.0072146559999999998 2024-09-14T09:08:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "op_commit", 2024-09-14T09:08:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239665+0000", 2024-09-14T09:08:34.318 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.137724501 2024-09-14T09:08:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239726+0000", 2024-09-14T09:08:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1550999999999994e-05 2024-09-14T09:08:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "commit_sent", 2024-09-14T09:08:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239800+0000", 2024-09-14T09:08:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3318000000000005e-05 2024-09-14T09:08:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.319 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239858+0000", 2024-09-14T09:08:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8600999999999997e-05 2024-09-14T09:08:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_repop_reply(client.4446.0:97796 3.15 e106/103 ondisk, result = 0)", 2024-09-14T09:08:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:08:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 82.156830284999998, 2024-09-14T09:08:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.123043465, 2024-09-14T09:08:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.320 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:08:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116827+0000", 2024-09-14T09:08:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116828+0000", 2024-09-14T09:08:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8100000000000002e-07 2024-09-14T09:08:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.321 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116831+0000", 2024-09-14T09:08:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.3009999999999998e-06 2024-09-14T09:08:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116841+0000", 2024-09-14T09:08:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.7829999999999993e-06 2024-09-14T09:08:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.116851+0000", 2024-09-14T09:08:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0135999999999999e-05 2024-09-14T09:08:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.322 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239707+0000", 2024-09-14T09:08:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.12285581499999999 2024-09-14T09:08:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239718+0000", 2024-09-14T09:08:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1804000000000001e-05 2024-09-14T09:08:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:12.239870+0000", 2024-09-14T09:08:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00015194500000000001 2024-09-14T09:08:34.323 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117315 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 309248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:08:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.550194548, 2024-09-14T09:08:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11247383800000001, 2024-09-14T09:08:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.324 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117315 2024-09-14T09:08:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:08:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723463+0000", 2024-09-14T09:08:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.325 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723899+0000", 2024-09-14T09:08:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00043641699999999998 2024-09-14T09:08:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723903+0000", 2024-09-14T09:08:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.5389999999999999e-06 2024-09-14T09:08:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723910+0000", 2024-09-14T09:08:34.326 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4229999999999999e-06 2024-09-14T09:08:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723917+0000", 2024-09-14T09:08:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.4560000000000003e-06 2024-09-14T09:08:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835514+0000", 2024-09-14T09:08:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111596258 2024-09-14T09:08:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835571+0000", 2024-09-14T09:08:34.327 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.7317e-05 2024-09-14T09:08:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835936+0000", 2024-09-14T09:08:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000365428 2024-09-14T09:08:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117317 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 313344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:08:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.549727531999999, 2024-09-14T09:08:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112450201, 2024-09-14T09:08:34.328 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117317 2024-09-14T09:08:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:08:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.329 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.723930+0000", 2024-09-14T09:08:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724396+0000", 2024-09-14T09:08:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000466028 2024-09-14T09:08:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.330 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724401+0000", 2024-09-14T09:08:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.2299999999999999e-06 2024-09-14T09:08:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.331 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724412+0000", 2024-09-14T09:08:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.0762e-05 2024-09-14T09:08:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724419+0000", 2024-09-14T09:08:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1010000000000003e-06 2024-09-14T09:08:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.835981+0000", 2024-09-14T09:08:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111562174 2024-09-14T09:08:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.332 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836070+0000", 2024-09-14T09:08:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8839999999999996e-05 2024-09-14T09:08:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836380+0000", 2024-09-14T09:08:34.333 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00031006600000000001 2024-09-14T09:08:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117319 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 317440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:34.334 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:08:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.549222896000003, 2024-09-14T09:08:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112467763, 2024-09-14T09:08:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117319 2024-09-14T09:08:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:08:34.335 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724434+0000", 2024-09-14T09:08:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724846+0000", 2024-09-14T09:08:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00041155400000000001 2024-09-14T09:08:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.336 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724854+0000", 2024-09-14T09:08:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5469999999999997e-06 2024-09-14T09:08:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724862+0000", 2024-09-14T09:08:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.1289999999999996e-06 2024-09-14T09:08:34.337 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724867+0000", 2024-09-14T09:08:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.8720000000000001e-06 2024-09-14T09:08:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836486+0000", 2024-09-14T09:08:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111618357 2024-09-14T09:08:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.338 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836554+0000", 2024-09-14T09:08:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.8378000000000002e-05 2024-09-14T09:08:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836902+0000", 2024-09-14T09:08:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00034792599999999998 2024-09-14T09:08:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.339 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117321 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 321536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:08:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.548778572000003, 2024-09-14T09:08:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11249685600000001, 2024-09-14T09:08:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.340 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117321 2024-09-14T09:08:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:08:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.724879+0000", 2024-09-14T09:08:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.341 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725241+0000", 2024-09-14T09:08:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00036211099999999998 2024-09-14T09:08:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725245+0000", 2024-09-14T09:08:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 3.9179999999999996e-06 2024-09-14T09:08:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.342 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725254+0000", 2024-09-14T09:08:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.0699999999999996e-06 2024-09-14T09:08:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725260+0000", 2024-09-14T09:08:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.1099999999999999e-06 2024-09-14T09:08:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.836935+0000", 2024-09-14T09:08:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111674997 2024-09-14T09:08:34.343 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837021+0000", 2024-09-14T09:08:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5931000000000001e-05 2024-09-14T09:08:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837375+0000", 2024-09-14T09:08:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035471899999999998 2024-09-14T09:08:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.344 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117323 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 325632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:08:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.548384347000002, 2024-09-14T09:08:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11250631, 2024-09-14T09:08:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:34.345 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117323 2024-09-14T09:08:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:08:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.346 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725273+0000", 2024-09-14T09:08:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725941+0000", 2024-09-14T09:08:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00066866500000000004 2024-09-14T09:08:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725946+0000", 2024-09-14T09:08:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9060000000000001e-06 2024-09-14T09:08:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.347 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725959+0000", 2024-09-14T09:08:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2622e-05 2024-09-14T09:08:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.725968+0000", 2024-09-14T09:08:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5660000000000007e-06 2024-09-14T09:08:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837408+0000", 2024-09-14T09:08:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111440626 2024-09-14T09:08:34.348 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837495+0000", 2024-09-14T09:08:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.6654000000000001e-05 2024-09-14T09:08:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.837779+0000", 2024-09-14T09:08:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000284271 2024-09-14T09:08:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.349 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117327 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 333824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:08:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.547648154999997, 2024-09-14T09:08:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112409908, 2024-09-14T09:08:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117327 2024-09-14T09:08:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.350 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:08:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726009+0000", 2024-09-14T09:08:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726303+0000", 2024-09-14T09:08:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029388199999999998 2024-09-14T09:08:34.351 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726307+0000", 2024-09-14T09:08:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4320000000000001e-06 2024-09-14T09:08:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726317+0000", 2024-09-14T09:08:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.4609999999999997e-06 2024-09-14T09:08:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726324+0000", 2024-09-14T09:08:34.352 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.0480000000000001e-06 2024-09-14T09:08:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838051+0000", 2024-09-14T09:08:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111727534 2024-09-14T09:08:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838143+0000", 2024-09-14T09:08:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.1359000000000004e-05 2024-09-14T09:08:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.353 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838419+0000", 2024-09-14T09:08:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00027619199999999997 2024-09-14T09:08:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117329 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 337920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:08:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.547321396000001, 2024-09-14T09:08:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112411122, 2024-09-14T09:08:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:34.354 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117329 2024-09-14T09:08:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:08:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726336+0000", 2024-09-14T09:08:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.355 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726742+0000", 2024-09-14T09:08:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00040626099999999999 2024-09-14T09:08:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726748+0000", 2024-09-14T09:08:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.6860000000000003e-06 2024-09-14T09:08:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726761+0000", 2024-09-14T09:08:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.3599e-05 2024-09-14T09:08:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.356 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726770+0000", 2024-09-14T09:08:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.5990000000000002e-06 2024-09-14T09:08:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838443+0000", 2024-09-14T09:08:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111672664 2024-09-14T09:08:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838487+0000", 2024-09-14T09:08:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.4249000000000003e-05 2024-09-14T09:08:34.357 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838747+0000", 2024-09-14T09:08:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00026006400000000001 2024-09-14T09:08:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.358 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117331 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 342016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:08:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.546853800999997, 2024-09-14T09:08:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112291139, 2024-09-14T09:08:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117331 2024-09-14T09:08:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.359 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:08:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.726803+0000", 2024-09-14T09:08:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.360 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727158+0000", 2024-09-14T09:08:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00035434000000000003 2024-09-14T09:08:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727162+0000", 2024-09-14T09:08:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.5449999999999997e-06 2024-09-14T09:08:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727174+0000", 2024-09-14T09:08:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2278e-05 2024-09-14T09:08:34.361 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727183+0000", 2024-09-14T09:08:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.3820000000000004e-06 2024-09-14T09:08:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838788+0000", 2024-09-14T09:08:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111604748 2024-09-14T09:08:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.838847+0000", 2024-09-14T09:08:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 5.8983000000000001e-05 2024-09-14T09:08:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.362 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.363 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.363 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839094+0000", 2024-09-14T09:08:34.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00024786300000000002 2024-09-14T09:08:34.363 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.363 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.363 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.363 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.363 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.363 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117333 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 346112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:34.363 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:08:34.363 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.546459990000002, 2024-09-14T09:08:34.363 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112319711, 2024-09-14T09:08:34.363 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.363 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.363 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:34.363 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:34.364 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:34.364 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.364 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117333 2024-09-14T09:08:34.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.364 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.364 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:08:34.364 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.364 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.364 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.364 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727197+0000", 2024-09-14T09:08:34.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.365 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727785+0000", 2024-09-14T09:08:34.365 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.000587512 2024-09-14T09:08:34.365 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.365 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.365 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727789+0000", 2024-09-14T09:08:34.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.352e-06 2024-09-14T09:08:34.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727800+0000", 2024-09-14T09:08:34.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.1369000000000001e-05 2024-09-14T09:08:34.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.727807+0000", 2024-09-14T09:08:34.366 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.1300000000000003e-06 2024-09-14T09:08:34.366 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.366 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.366 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.366 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839137+0000", 2024-09-14T09:08:34.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111329898 2024-09-14T09:08:34.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839226+0000", 2024-09-14T09:08:34.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.8337000000000005e-05 2024-09-14T09:08:34.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.367 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.367 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.839517+0000", 2024-09-14T09:08:34.367 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00029111300000000002 2024-09-14T09:08:34.367 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.367 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.367 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.367 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.367 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.368 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117363 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 407552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:34.368 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:08:34.368 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.540363890999998, 2024-09-14T09:08:34.368 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112191306, 2024-09-14T09:08:34.368 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.368 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.368 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:34.368 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:34.368 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:34.368 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.368 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117363 2024-09-14T09:08:34.368 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.368 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.368 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.368 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.368 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:08:34.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733293+0000", 2024-09-14T09:08:34.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.369 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733666+0000", 2024-09-14T09:08:34.369 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00037284500000000001 2024-09-14T09:08:34.369 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.369 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.369 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733671+0000", 2024-09-14T09:08:34.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9269999999999996e-06 2024-09-14T09:08:34.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733681+0000", 2024-09-14T09:08:34.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 9.5599999999999999e-06 2024-09-14T09:08:34.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.370 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.370 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733689+0000", 2024-09-14T09:08:34.370 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.2649999999999996e-06 2024-09-14T09:08:34.370 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.370 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845028+0000", 2024-09-14T09:08:34.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.111338933 2024-09-14T09:08:34.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845094+0000", 2024-09-14T09:08:34.371 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 6.6749999999999996e-05 2024-09-14T09:08:34.371 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.371 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.371 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.371 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845485+0000", 2024-09-14T09:08:34.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00039002600000000002 2024-09-14T09:08:34.372 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.372 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.372 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.372 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.372 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.372 INFO:teuthology.orchestra.run.smithi176.stdout: "description": "osd_op(client.4446.0:117365 3.8 3:157dbf3c:::rbd_data.115e500fbcd9.0000000000000006:27 [read 411648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-14T09:08:34.372 INFO:teuthology.orchestra.run.smithi176.stdout: "initiated_at": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:08:34.372 INFO:teuthology.orchestra.run.smithi176.stdout: "age": 62.539954125999998, 2024-09-14T09:08:34.372 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.112271685, 2024-09-14T09:08:34.372 INFO:teuthology.orchestra.run.smithi176.stdout: "continuous": false, 2024-09-14T09:08:34.373 INFO:teuthology.orchestra.run.smithi176.stdout: "type_data": { 2024-09-14T09:08:34.373 INFO:teuthology.orchestra.run.smithi176.stdout: "flag_point": "started", 2024-09-14T09:08:34.373 INFO:teuthology.orchestra.run.smithi176.stdout: "client_info": { 2024-09-14T09:08:34.373 INFO:teuthology.orchestra.run.smithi176.stdout: "client": "client.4446", 2024-09-14T09:08:34.373 INFO:teuthology.orchestra.run.smithi176.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.373 INFO:teuthology.orchestra.run.smithi176.stdout: "tid": 117365 2024-09-14T09:08:34.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.373 INFO:teuthology.orchestra.run.smithi176.stdout: "events": [ 2024-09-14T09:08:34.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.373 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "initiated", 2024-09-14T09:08:34.373 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:08:34.373 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.373 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "header_read", 2024-09-14T09:08:34.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.733703+0000", 2024-09-14T09:08:34.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0 2024-09-14T09:08:34.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "throttled", 2024-09-14T09:08:34.374 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734147+0000", 2024-09-14T09:08:34.374 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00044374399999999999 2024-09-14T09:08:34.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.374 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.374 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "all_read", 2024-09-14T09:08:34.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734152+0000", 2024-09-14T09:08:34.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 4.9139999999999999e-06 2024-09-14T09:08:34.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "dispatched", 2024-09-14T09:08:34.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734164+0000", 2024-09-14T09:08:34.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 1.2299e-05 2024-09-14T09:08:34.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.375 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.375 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.734172+0000", 2024-09-14T09:08:34.375 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 8.0390000000000005e-06 2024-09-14T09:08:34.375 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.375 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "reached_pg", 2024-09-14T09:08:34.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845528+0000", 2024-09-14T09:08:34.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.11135639899999999 2024-09-14T09:08:34.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "started", 2024-09-14T09:08:34.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845602+0000", 2024-09-14T09:08:34.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 7.3957999999999999e-05 2024-09-14T09:08:34.376 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-09-14T09:08:34.376 INFO:teuthology.orchestra.run.smithi176.stdout: { 2024-09-14T09:08:34.376 INFO:teuthology.orchestra.run.smithi176.stdout: "event": "done", 2024-09-14T09:08:34.376 INFO:teuthology.orchestra.run.smithi176.stdout: "time": "2024-09-14T09:07:31.845975+0000", 2024-09-14T09:08:34.376 INFO:teuthology.orchestra.run.smithi176.stdout: "duration": 0.00037233200000000001 2024-09-14T09:08:34.376 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.377 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.377 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.377 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-09-14T09:08:34.377 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-09-14T09:08:34.377 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:08:34.378 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T09:08:34.379 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-09-14T09:08:34.382+0000 7fdc36f72640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26516) UID: 0 2024-09-14T09:08:34.480 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-09-14T09:08:34.482+0000 7feceb07b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26518) UID: 0 2024-09-14T09:08:34.490 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is unset 2024-09-14T09:08:34.517 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:34.517 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:34.517 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:08:34.517 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:34.535 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T09:08:34.581 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:34.578+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:34.647 INFO:teuthology.orchestra.run.smithi176.stdout:All operations completed A-OK! 2024-09-14T09:08:34.681 INFO:tasks.ceph.osd.0.smithi176.stderr:2024-09-14T09:08:34.682+0000 7f6094847640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26411) UID: 0 2024-09-14T09:08:34.705 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:34.705 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:34.705 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:08:34.705 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:08:34.705 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:34.718 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T09:08:34.734 DEBUG:teuthology.parallel:result is None 2024-09-14T09:08:34.735 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-14T09:08:34.744 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-14T09:08:34.782 INFO:tasks.thrashosds:joining thrashosds 2024-09-14T09:08:34.843 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:34.843 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:08:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:08:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:08:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105615 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 382976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:08:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.982479049000005, 2024-09-14T09:08:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117270900000001, 2024-09-14T09:08:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.844 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.845 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105615 2024-09-14T09:08:34.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.845 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:08:34.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864367+0000", 2024-09-14T09:08:34.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864368+0000", 2024-09-14T09:08:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.043e-06 2024-09-14T09:08:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864373+0000", 2024-09-14T09:08:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4669999999999998e-06 2024-09-14T09:08:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864387+0000", 2024-09-14T09:08:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3876000000000001e-05 2024-09-14T09:08:34.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.864397+0000", 2024-09-14T09:08:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0592e-05 2024-09-14T09:08:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866069+0000", 2024-09-14T09:08:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016714410000000001 2024-09-14T09:08:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866076+0000", 2024-09-14T09:08:34.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0679999999999999e-06 2024-09-14T09:08:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.075854+0000", 2024-09-14T09:08:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.209778501 2024-09-14T09:08:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.075940+0000", 2024-09-14T09:08:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5357000000000003e-05 2024-09-14T09:08:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.076094+0000", 2024-09-14T09:08:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000154745 2024-09-14T09:08:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105629 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 411648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:08:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.980325327000003, 2024-09-14T09:08:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2115591910000001, 2024-09-14T09:08:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.849 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105629 2024-09-14T09:08:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:08:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866521+0000", 2024-09-14T09:08:34.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866522+0000", 2024-09-14T09:08:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6899999999999996e-07 2024-09-14T09:08:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866526+0000", 2024-09-14T09:08:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0110000000000002e-06 2024-09-14T09:08:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866539+0000", 2024-09-14T09:08:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3013000000000001e-05 2024-09-14T09:08:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866549+0000", 2024-09-14T09:08:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8490000000000001e-06 2024-09-14T09:08:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866650+0000", 2024-09-14T09:08:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101054 2024-09-14T09:08:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866664+0000", 2024-09-14T09:08:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3805000000000001e-05 2024-09-14T09:08:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.077849+0000", 2024-09-14T09:08:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2111855419999999 2024-09-14T09:08:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.077945+0000", 2024-09-14T09:08:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6086000000000001e-05 2024-09-14T09:08:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078080+0000", 2024-09-14T09:08:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000134862 2024-09-14T09:08:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105631 3.d 3:b31268cb:::rbd_data.115e500fbcd9.000000000000000e:27 [read 415744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:08:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.980266107999995, 2024-09-14T09:08:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119463509999999, 2024-09-14T09:08:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.854 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105631 2024-09-14T09:08:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:08:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866580+0000", 2024-09-14T09:08:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866581+0000", 2024-09-14T09:08:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.879999999999999e-07 2024-09-14T09:08:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866585+0000", 2024-09-14T09:08:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0239999999999999e-06 2024-09-14T09:08:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866598+0000", 2024-09-14T09:08:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2862e-05 2024-09-14T09:08:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866608+0000", 2024-09-14T09:08:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9890000000000004e-06 2024-09-14T09:08:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866708+0000", 2024-09-14T09:08:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9605999999999995e-05 2024-09-14T09:08:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866720+0000", 2024-09-14T09:08:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.261e-05 2024-09-14T09:08:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078137+0000", 2024-09-14T09:08:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211416571 2024-09-14T09:08:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078293+0000", 2024-09-14T09:08:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000156 2024-09-14T09:08:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078527+0000", 2024-09-14T09:08:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000233701 2024-09-14T09:08:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.858 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105633 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:08:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.980207061000002, 2024-09-14T09:08:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211941734, 2024-09-14T09:08:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105633 2024-09-14T09:08:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.859 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:08:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866639+0000", 2024-09-14T09:08:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866640+0000", 2024-09-14T09:08:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9300000000000006e-07 2024-09-14T09:08:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866656+0000", 2024-09-14T09:08:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5291e-05 2024-09-14T09:08:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866669+0000", 2024-09-14T09:08:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2918000000000001e-05 2024-09-14T09:08:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866678+0000", 2024-09-14T09:08:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8369999999999992e-06 2024-09-14T09:08:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866771+0000", 2024-09-14T09:08:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2250999999999995e-05 2024-09-14T09:08:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866784+0000", 2024-09-14T09:08:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2991e-05 2024-09-14T09:08:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078566+0000", 2024-09-14T09:08:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117822570000001 2024-09-14T09:08:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078581+0000", 2024-09-14T09:08:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5196e-05 2024-09-14T09:08:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105635 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:08:34.863 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.980153233999999, 2024-09-14T09:08:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119313250000001, 2024-09-14T09:08:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105635 2024-09-14T09:08:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:08:34.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866693+0000", 2024-09-14T09:08:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866694+0000", 2024-09-14T09:08:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0210000000000001e-06 2024-09-14T09:08:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866698+0000", 2024-09-14T09:08:34.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.765e-06 2024-09-14T09:08:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866710+0000", 2024-09-14T09:08:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2320000000000001e-05 2024-09-14T09:08:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866722+0000", 2024-09-14T09:08:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2133000000000001e-05 2024-09-14T09:08:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866830+0000", 2024-09-14T09:08:34.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000107668 2024-09-14T09:08:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866843+0000", 2024-09-14T09:08:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2991e-05 2024-09-14T09:08:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078616+0000", 2024-09-14T09:08:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117732459999999 2024-09-14T09:08:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078624+0000", 2024-09-14T09:08:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1810000000000001e-06 2024-09-14T09:08:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105636 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:08:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.980085337999995, 2024-09-14T09:08:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2119085119999999, 2024-09-14T09:08:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.868 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105636 2024-09-14T09:08:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:08:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866761+0000", 2024-09-14T09:08:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866762+0000", 2024-09-14T09:08:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4600000000000003e-07 2024-09-14T09:08:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866766+0000", 2024-09-14T09:08:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9829999999999998e-06 2024-09-14T09:08:34.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866779+0000", 2024-09-14T09:08:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3101999999999999e-05 2024-09-14T09:08:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866789+0000", 2024-09-14T09:08:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8039999999999996e-06 2024-09-14T09:08:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866889+0000", 2024-09-14T09:08:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100147 2024-09-14T09:08:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866902+0000", 2024-09-14T09:08:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3212e-05 2024-09-14T09:08:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078659+0000", 2024-09-14T09:08:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211756362 2024-09-14T09:08:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078670+0000", 2024-09-14T09:08:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0956e-05 2024-09-14T09:08:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105639 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:08:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.980026463000002, 2024-09-14T09:08:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118829259999999, 2024-09-14T09:08:34.873 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105639 2024-09-14T09:08:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:08:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866820+0000", 2024-09-14T09:08:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866821+0000", 2024-09-14T09:08:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9800000000000002e-07 2024-09-14T09:08:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866825+0000", 2024-09-14T09:08:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9269999999999998e-06 2024-09-14T09:08:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866838+0000", 2024-09-14T09:08:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3040000000000001e-05 2024-09-14T09:08:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866848+0000", 2024-09-14T09:08:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7929999999999992e-06 2024-09-14T09:08:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866948+0000", 2024-09-14T09:08:34.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100337 2024-09-14T09:08:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866961+0000", 2024-09-14T09:08:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3106e-05 2024-09-14T09:08:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078693+0000", 2024-09-14T09:08:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117320380000001 2024-09-14T09:08:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078703+0000", 2024-09-14T09:08:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6870000000000006e-06 2024-09-14T09:08:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105641 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:08:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.979967598000002, 2024-09-14T09:08:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118727169999999, 2024-09-14T09:08:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.878 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105641 2024-09-14T09:08:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:08:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866879+0000", 2024-09-14T09:08:34.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866880+0000", 2024-09-14T09:08:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7699999999999992e-07 2024-09-14T09:08:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866884+0000", 2024-09-14T09:08:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9480000000000001e-06 2024-09-14T09:08:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866897+0000", 2024-09-14T09:08:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2832e-05 2024-09-14T09:08:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866906+0000", 2024-09-14T09:08:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7869999999999996e-06 2024-09-14T09:08:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867007+0000", 2024-09-14T09:08:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010103900000000001 2024-09-14T09:08:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867020+0000", 2024-09-14T09:08:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2896e-05 2024-09-14T09:08:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078739+0000", 2024-09-14T09:08:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211718305 2024-09-14T09:08:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078752+0000", 2024-09-14T09:08:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2933e-05 2024-09-14T09:08:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.882 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105643 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:08:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.979909039000006, 2024-09-14T09:08:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211858353, 2024-09-14T09:08:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105643 2024-09-14T09:08:34.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:08:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866937+0000", 2024-09-14T09:08:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866938+0000", 2024-09-14T09:08:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5600000000000004e-07 2024-09-14T09:08:34.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866942+0000", 2024-09-14T09:08:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0350000000000003e-06 2024-09-14T09:08:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866955+0000", 2024-09-14T09:08:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3097e-05 2024-09-14T09:08:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866965+0000", 2024-09-14T09:08:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8179999999999999e-06 2024-09-14T09:08:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867067+0000", 2024-09-14T09:08:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010128699999999999 2024-09-14T09:08:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867080+0000", 2024-09-14T09:08:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3089e-05 2024-09-14T09:08:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078786+0000", 2024-09-14T09:08:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117065250000001 2024-09-14T09:08:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078796+0000", 2024-09-14T09:08:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5459999999999997e-06 2024-09-14T09:08:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105645 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:08:34.887 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.979848644, 2024-09-14T09:08:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211848641, 2024-09-14T09:08:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105645 2024-09-14T09:08:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:08:34.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866998+0000", 2024-09-14T09:08:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.866999+0000", 2024-09-14T09:08:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7999999999999993e-07 2024-09-14T09:08:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867003+0000", 2024-09-14T09:08:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0090000000000001e-06 2024-09-14T09:08:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867016+0000", 2024-09-14T09:08:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3304e-05 2024-09-14T09:08:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867026+0000", 2024-09-14T09:08:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5889999999999992e-06 2024-09-14T09:08:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867128+0000", 2024-09-14T09:08:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000102101 2024-09-14T09:08:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867141+0000", 2024-09-14T09:08:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2903e-05 2024-09-14T09:08:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078833+0000", 2024-09-14T09:08:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116920959999999 2024-09-14T09:08:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078846+0000", 2024-09-14T09:08:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3658999999999999e-05 2024-09-14T09:08:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105647 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:08:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.979790125999997, 2024-09-14T09:08:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211830097, 2024-09-14T09:08:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.892 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105647 2024-09-14T09:08:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:08:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867056+0000", 2024-09-14T09:08:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867057+0000", 2024-09-14T09:08:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.95e-07 2024-09-14T09:08:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867061+0000", 2024-09-14T09:08:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9530000000000001e-06 2024-09-14T09:08:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867074+0000", 2024-09-14T09:08:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3061000000000001e-05 2024-09-14T09:08:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867086+0000", 2024-09-14T09:08:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1460999999999999e-05 2024-09-14T09:08:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867186+0000", 2024-09-14T09:08:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100181 2024-09-14T09:08:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867199+0000", 2024-09-14T09:08:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-05 2024-09-14T09:08:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078874+0000", 2024-09-14T09:08:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211674683 2024-09-14T09:08:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078886+0000", 2024-09-14T09:08:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2853e-05 2024-09-14T09:08:34.897 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105648 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:08:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.979728640000005, 2024-09-14T09:08:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2118073570000001, 2024-09-14T09:08:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.898 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105648 2024-09-14T09:08:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:08:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867118+0000", 2024-09-14T09:08:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867119+0000", 2024-09-14T09:08:34.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8400000000000002e-07 2024-09-14T09:08:34.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867123+0000", 2024-09-14T09:08:34.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9639999999999997e-06 2024-09-14T09:08:34.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867136+0000", 2024-09-14T09:08:34.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.289e-05 2024-09-14T09:08:34.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867145+0000", 2024-09-14T09:08:34.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8919999999999995e-06 2024-09-14T09:08:34.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867245+0000", 2024-09-14T09:08:34.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9282000000000003e-05 2024-09-14T09:08:34.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867258+0000", 2024-09-14T09:08:34.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2935000000000001e-05 2024-09-14T09:08:34.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078910+0000", 2024-09-14T09:08:34.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211652352 2024-09-14T09:08:34.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078925+0000", 2024-09-14T09:08:34.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5058000000000001e-05 2024-09-14T09:08:34.902 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.902 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.902 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.903 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105652 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.903 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:08:34.903 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.979670620999997, 2024-09-14T09:08:34.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211775737, 2024-09-14T09:08:34.903 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.903 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.903 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:34.903 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.903 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.903 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.903 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105652 2024-09-14T09:08:34.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.904 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:08:34.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867176+0000", 2024-09-14T09:08:34.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867177+0000", 2024-09-14T09:08:34.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9100000000000002e-07 2024-09-14T09:08:34.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867181+0000", 2024-09-14T09:08:34.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0029999999999996e-06 2024-09-14T09:08:34.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867193+0000", 2024-09-14T09:08:34.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2675e-05 2024-09-14T09:08:34.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867203+0000", 2024-09-14T09:08:34.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5599999999999999e-06 2024-09-14T09:08:34.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867306+0000", 2024-09-14T09:08:34.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010330100000000001 2024-09-14T09:08:34.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867319+0000", 2024-09-14T09:08:34.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3129000000000001e-05 2024-09-14T09:08:34.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078944+0000", 2024-09-14T09:08:34.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116247520000001 2024-09-14T09:08:34.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078952+0000", 2024-09-14T09:08:34.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4259999999999997e-06 2024-09-14T09:08:34.907 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.907 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.907 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.908 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105653 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.908 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:08:34.908 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.979611950999995, 2024-09-14T09:08:34.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211774895, 2024-09-14T09:08:34.908 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.908 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.908 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:34.908 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.908 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.908 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.908 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105653 2024-09-14T09:08:34.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.908 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:08:34.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867234+0000", 2024-09-14T09:08:34.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867235+0000", 2024-09-14T09:08:34.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9400000000000004e-07 2024-09-14T09:08:34.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867239+0000", 2024-09-14T09:08:34.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9380000000000002e-06 2024-09-14T09:08:34.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867252+0000", 2024-09-14T09:08:34.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.31e-05 2024-09-14T09:08:34.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867264+0000", 2024-09-14T09:08:34.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1275e-05 2024-09-14T09:08:34.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867367+0000", 2024-09-14T09:08:34.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103755 2024-09-14T09:08:34.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867382+0000", 2024-09-14T09:08:34.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4724e-05 2024-09-14T09:08:34.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.078995+0000", 2024-09-14T09:08:34.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116132669999999 2024-09-14T09:08:34.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079009+0000", 2024-09-14T09:08:34.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3942e-05 2024-09-14T09:08:34.912 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.912 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.912 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.913 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105654 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.913 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:08:34.913 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.979550895000003, 2024-09-14T09:08:34.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2117364070000001, 2024-09-14T09:08:34.913 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.913 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.913 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:34.913 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.913 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.913 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.913 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105654 2024-09-14T09:08:34.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.913 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:08:34.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867295+0000", 2024-09-14T09:08:34.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867296+0000", 2024-09-14T09:08:34.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2399999999999996e-07 2024-09-14T09:08:34.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867300+0000", 2024-09-14T09:08:34.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8689999999999997e-06 2024-09-14T09:08:34.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867313+0000", 2024-09-14T09:08:34.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2660000000000001e-05 2024-09-14T09:08:34.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867322+0000", 2024-09-14T09:08:34.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.533e-06 2024-09-14T09:08:34.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867429+0000", 2024-09-14T09:08:34.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106566 2024-09-14T09:08:34.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867442+0000", 2024-09-14T09:08:34.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3244000000000001e-05 2024-09-14T09:08:34.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079024+0000", 2024-09-14T09:08:34.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211581813 2024-09-14T09:08:34.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079032+0000", 2024-09-14T09:08:34.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7980000000000006e-06 2024-09-14T09:08:34.917 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.917 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.917 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.917 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105657 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.917 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:08:34.917 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.979489721999997, 2024-09-14T09:08:34.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211700657, 2024-09-14T09:08:34.918 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.918 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.918 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:34.918 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.918 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.918 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.918 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105657 2024-09-14T09:08:34.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.918 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:08:34.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867357+0000", 2024-09-14T09:08:34.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867358+0000", 2024-09-14T09:08:34.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9700000000000005e-07 2024-09-14T09:08:34.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867367+0000", 2024-09-14T09:08:34.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8919999999999995e-06 2024-09-14T09:08:34.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867383+0000", 2024-09-14T09:08:34.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.579e-05 2024-09-14T09:08:34.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867393+0000", 2024-09-14T09:08:34.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3070000000000008e-06 2024-09-14T09:08:34.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867490+0000", 2024-09-14T09:08:34.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7703999999999998e-05 2024-09-14T09:08:34.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867503+0000", 2024-09-14T09:08:34.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2849e-05 2024-09-14T09:08:34.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079050+0000", 2024-09-14T09:08:34.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211546899 2024-09-14T09:08:34.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079057+0000", 2024-09-14T09:08:34.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3189999999999997e-06 2024-09-14T09:08:34.922 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.922 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.922 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.922 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105658 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.922 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:08:34.922 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.979427337999994, 2024-09-14T09:08:34.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211691217, 2024-09-14T09:08:34.923 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.923 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.923 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:34.923 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.923 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.923 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.923 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105658 2024-09-14T09:08:34.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.923 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:08:34.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867419+0000", 2024-09-14T09:08:34.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867420+0000", 2024-09-14T09:08:34.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5000000000000001e-07 2024-09-14T09:08:34.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867424+0000", 2024-09-14T09:08:34.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8469999999999998e-06 2024-09-14T09:08:34.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867437+0000", 2024-09-14T09:08:34.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3156000000000001e-05 2024-09-14T09:08:34.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867448+0000", 2024-09-14T09:08:34.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1185e-05 2024-09-14T09:08:34.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867552+0000", 2024-09-14T09:08:34.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103362 2024-09-14T09:08:34.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867565+0000", 2024-09-14T09:08:34.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3018e-05 2024-09-14T09:08:34.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079092+0000", 2024-09-14T09:08:34.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211527013 2024-09-14T09:08:34.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079110+0000", 2024-09-14T09:08:34.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8686e-05 2024-09-14T09:08:34.927 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.927 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.927 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.927 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105662 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.927 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:08:34.927 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.979366272999997, 2024-09-14T09:08:34.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211652226, 2024-09-14T09:08:34.927 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.927 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.927 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:34.927 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.927 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.927 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.928 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105662 2024-09-14T09:08:34.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.928 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:08:34.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867480+0000", 2024-09-14T09:08:34.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867481+0000", 2024-09-14T09:08:34.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1800000000000004e-07 2024-09-14T09:08:34.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867485+0000", 2024-09-14T09:08:34.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9199999999999997e-06 2024-09-14T09:08:34.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867498+0000", 2024-09-14T09:08:34.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2896e-05 2024-09-14T09:08:34.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867509+0000", 2024-09-14T09:08:34.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1202e-05 2024-09-14T09:08:34.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867611+0000", 2024-09-14T09:08:34.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010166999999999999 2024-09-14T09:08:34.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867624+0000", 2024-09-14T09:08:34.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2996000000000001e-05 2024-09-14T09:08:34.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079125+0000", 2024-09-14T09:08:34.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2115009299999999 2024-09-14T09:08:34.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079132+0000", 2024-09-14T09:08:34.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6939999999999995e-06 2024-09-14T09:08:34.931 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.931 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.931 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.931 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105663 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.931 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:08:34.932 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.979305221999994, 2024-09-14T09:08:34.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2116218169999999, 2024-09-14T09:08:34.932 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.932 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.932 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:34.932 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.932 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.932 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.932 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105663 2024-09-14T09:08:34.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.932 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:08:34.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867541+0000", 2024-09-14T09:08:34.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867542+0000", 2024-09-14T09:08:34.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1399999999999995e-07 2024-09-14T09:08:34.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867546+0000", 2024-09-14T09:08:34.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7900000000000001e-06 2024-09-14T09:08:34.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867559+0000", 2024-09-14T09:08:34.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2805e-05 2024-09-14T09:08:34.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867568+0000", 2024-09-14T09:08:34.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-09-14T09:08:34.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867669+0000", 2024-09-14T09:08:34.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100655 2024-09-14T09:08:34.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867682+0000", 2024-09-14T09:08:34.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3013000000000001e-05 2024-09-14T09:08:34.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079151+0000", 2024-09-14T09:08:34.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2114689089999999 2024-09-14T09:08:34.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079163+0000", 2024-09-14T09:08:34.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1962e-05 2024-09-14T09:08:34.936 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.936 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.936 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.936 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105665 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-14T09:08:34.936 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:08:34.936 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.979246591000006, 2024-09-14T09:08:34.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211584848, 2024-09-14T09:08:34.937 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:34.937 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:34.937 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:34.937 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:34.937 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:34.937 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:34.937 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105665 2024-09-14T09:08:34.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.937 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:34.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:34.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:08:34.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:34.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867600+0000", 2024-09-14T09:08:34.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:34.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:34.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867601+0000", 2024-09-14T09:08:34.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.569999999999999e-07 2024-09-14T09:08:34.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:34.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867605+0000", 2024-09-14T09:08:34.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8589999999999998e-06 2024-09-14T09:08:34.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:34.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867617+0000", 2024-09-14T09:08:34.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2833000000000001e-05 2024-09-14T09:08:34.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:34.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867627+0000", 2024-09-14T09:08:34.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5219999999999996e-06 2024-09-14T09:08:34.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867728+0000", 2024-09-14T09:08:34.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010057500000000001 2024-09-14T09:08:34.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:34.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.867740+0000", 2024-09-14T09:08:34.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2744e-05 2024-09-14T09:08:34.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:34.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079178+0000", 2024-09-14T09:08:34.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.211437552 2024-09-14T09:08:34.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:34.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:34.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:34.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.079185+0000", 2024-09-14T09:08:34.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8059999999999997e-06 2024-09-14T09:08:34.941 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.941 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.941 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.941 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:34.941 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:34.941 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:34.942 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T09:08:35.069 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:35.069 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:35.069 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:08:35.069 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:35.083 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T09:08:35.206 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:35.206 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:35.206 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:08:35.206 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:08:35.206 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:35.220 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T09:08:35.345 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:35.345 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:08:35.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:08:35.345 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:08:35.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.345 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:65139 3.a 3:54c90b7d:::rbd_data.115e21c31fcf.0000000000000008:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T09:08:35.345 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:08:35.346 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 133.96929372, 2024-09-14T09:08:35.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14429895700000001, 2024-09-14T09:08:35.346 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.346 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.346 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T09:08:35.346 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.346 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.346 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.346 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 65139 2024-09-14T09:08:35.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.346 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:08:35.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379713+0000", 2024-09-14T09:08:35.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379715+0000", 2024-09-14T09:08:35.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.576e-06 2024-09-14T09:08:35.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379724+0000", 2024-09-14T09:08:35.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1779999999999992e-06 2024-09-14T09:08:35.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379746+0000", 2024-09-14T09:08:35.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1661000000000001e-05 2024-09-14T09:08:35.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379759+0000", 2024-09-14T09:08:35.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.324e-05 2024-09-14T09:08:35.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.379882+0000", 2024-09-14T09:08:35.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012239400000000001 2024-09-14T09:08:35.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.380023+0000", 2024-09-14T09:08:35.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000141402 2024-09-14T09:08:35.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-09-14T09:08:35.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.391553+0000", 2024-09-14T09:08:35.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011530197000000001 2024-09-14T09:08:35.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-09-14T09:08:35.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523927+0000", 2024-09-14T09:08:35.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.132373935 2024-09-14T09:08:35.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-09-14T09:08:35.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523941+0000", 2024-09-14T09:08:35.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3326000000000001e-05 2024-09-14T09:08:35.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-09-14T09:08:35.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.523977+0000", 2024-09-14T09:08:35.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6185e-05 2024-09-14T09:08:35.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:21.524012+0000", 2024-09-14T09:08:35.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4863000000000001e-05 2024-09-14T09:08:35.351 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.351 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.351 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.351 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107606 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 14336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:35.351 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:35.352 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.942836819000007, 2024-09-14T09:08:35.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14293513799999999, 2024-09-14T09:08:35.352 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.352 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.352 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.352 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.352 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.352 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.352 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107606 2024-09-14T09:08:35.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.352 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:35.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:35.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406170+0000", 2024-09-14T09:08:35.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1600000000000001e-07 2024-09-14T09:08:35.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406174+0000", 2024-09-14T09:08:35.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4199999999999999e-06 2024-09-14T09:08:35.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406188+0000", 2024-09-14T09:08:35.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4114000000000001e-05 2024-09-14T09:08:35.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406198+0000", 2024-09-14T09:08:35.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.041e-05 2024-09-14T09:08:35.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.548700+0000", 2024-09-14T09:08:35.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14250141299999999 2024-09-14T09:08:35.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.548783+0000", 2024-09-14T09:08:35.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3578999999999994e-05 2024-09-14T09:08:35.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549105+0000", 2024-09-14T09:08:35.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032158600000000001 2024-09-14T09:08:35.356 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.356 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.356 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.356 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107608 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 18432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:35.356 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:35.356 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.942770783, 2024-09-14T09:08:35.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14603042799999999, 2024-09-14T09:08:35.356 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.356 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.356 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.357 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.357 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.357 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.357 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107608 2024-09-14T09:08:35.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.357 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:35.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:35.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406236+0000", 2024-09-14T09:08:35.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1200000000000002e-07 2024-09-14T09:08:35.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406241+0000", 2024-09-14T09:08:35.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7539999999999997e-06 2024-09-14T09:08:35.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406275+0000", 2024-09-14T09:08:35.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3512e-05 2024-09-14T09:08:35.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406322+0000", 2024-09-14T09:08:35.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6764999999999997e-05 2024-09-14T09:08:35.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549149+0000", 2024-09-14T09:08:35.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14282725499999999 2024-09-14T09:08:35.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.549198+0000", 2024-09-14T09:08:35.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9599999999999999e-05 2024-09-14T09:08:35.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552266+0000", 2024-09-14T09:08:35.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030677299999999998 2024-09-14T09:08:35.360 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.360 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.360 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.361 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107610 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 22528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:35.361 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:35.361 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.942648907999995, 2024-09-14T09:08:35.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146437919, 2024-09-14T09:08:35.361 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.361 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.361 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.361 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.361 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.361 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.361 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107610 2024-09-14T09:08:35.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.362 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:35.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:35.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406358+0000", 2024-09-14T09:08:35.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.02e-07 2024-09-14T09:08:35.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406363+0000", 2024-09-14T09:08:35.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8099999999999997e-06 2024-09-14T09:08:35.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406390+0000", 2024-09-14T09:08:35.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7004000000000001e-05 2024-09-14T09:08:35.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406404+0000", 2024-09-14T09:08:35.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3682e-05 2024-09-14T09:08:35.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552335+0000", 2024-09-14T09:08:35.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14593077800000001 2024-09-14T09:08:35.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552431+0000", 2024-09-14T09:08:35.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6086000000000001e-05 2024-09-14T09:08:35.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552795+0000", 2024-09-14T09:08:35.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000364657 2024-09-14T09:08:35.365 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.365 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.365 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.365 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107612 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 26624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:35.365 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:08:35.365 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.942583644999999, 2024-09-14T09:08:35.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146762061, 2024-09-14T09:08:35.365 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.366 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.366 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.366 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.366 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.366 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.366 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107612 2024-09-14T09:08:35.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.366 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:08:35.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406423+0000", 2024-09-14T09:08:35.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406424+0000", 2024-09-14T09:08:35.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2600000000000002e-07 2024-09-14T09:08:35.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406451+0000", 2024-09-14T09:08:35.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7250000000000002e-05 2024-09-14T09:08:35.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406487+0000", 2024-09-14T09:08:35.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6086e-05 2024-09-14T09:08:35.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406498+0000", 2024-09-14T09:08:35.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0858e-05 2024-09-14T09:08:35.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552821+0000", 2024-09-14T09:08:35.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14632319099999999 2024-09-14T09:08:35.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.552875+0000", 2024-09-14T09:08:35.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4262999999999999e-05 2024-09-14T09:08:35.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553185+0000", 2024-09-14T09:08:35.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000309687 2024-09-14T09:08:35.369 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.369 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.370 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.370 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107614 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 30720~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:35.370 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:08:35.370 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.942496110999997, 2024-09-14T09:08:35.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14737121, 2024-09-14T09:08:35.370 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.370 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.370 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.370 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.370 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.370 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.370 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107614 2024-09-14T09:08:35.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.371 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:08:35.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406510+0000", 2024-09-14T09:08:35.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406511+0000", 2024-09-14T09:08:35.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1799999999999995e-07 2024-09-14T09:08:35.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406514+0000", 2024-09-14T09:08:35.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8660000000000002e-06 2024-09-14T09:08:35.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406523+0000", 2024-09-14T09:08:35.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1940000000000004e-06 2024-09-14T09:08:35.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406531+0000", 2024-09-14T09:08:35.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3159999999999997e-06 2024-09-14T09:08:35.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553233+0000", 2024-09-14T09:08:35.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14670117899999999 2024-09-14T09:08:35.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553334+0000", 2024-09-14T09:08:35.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100982 2024-09-14T09:08:35.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553882+0000", 2024-09-14T09:08:35.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00054805499999999998 2024-09-14T09:08:35.374 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.374 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.374 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.374 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107616 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 34816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:35.374 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:08:35.374 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.942464475999998, 2024-09-14T09:08:35.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14780390500000001, 2024-09-14T09:08:35.375 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.375 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.375 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.375 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.375 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.375 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.375 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107616 2024-09-14T09:08:35.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.375 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:08:35.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406542+0000", 2024-09-14T09:08:35.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406543+0000", 2024-09-14T09:08:35.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0200000000000001e-07 2024-09-14T09:08:35.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406545+0000", 2024-09-14T09:08:35.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3360000000000002e-06 2024-09-14T09:08:35.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406552+0000", 2024-09-14T09:08:35.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9779999999999999e-06 2024-09-14T09:08:35.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406559+0000", 2024-09-14T09:08:35.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6919999999999996e-06 2024-09-14T09:08:35.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.553917+0000", 2024-09-14T09:08:35.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14735791600000001 2024-09-14T09:08:35.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554033+0000", 2024-09-14T09:08:35.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116047 2024-09-14T09:08:35.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554346+0000", 2024-09-14T09:08:35.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031323400000000002 2024-09-14T09:08:35.378 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.378 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.379 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.379 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107618 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 38912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:35.379 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:35.379 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.942435676000002, 2024-09-14T09:08:35.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14816009599999999, 2024-09-14T09:08:35.379 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.379 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.379 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.379 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.379 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.379 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.379 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107618 2024-09-14T09:08:35.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.379 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:35.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:35.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406571+0000", 2024-09-14T09:08:35.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9999999999999998e-07 2024-09-14T09:08:35.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406574+0000", 2024-09-14T09:08:35.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2560000000000001e-06 2024-09-14T09:08:35.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406581+0000", 2024-09-14T09:08:35.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1539999999999996e-06 2024-09-14T09:08:35.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406586+0000", 2024-09-14T09:08:35.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.904e-06 2024-09-14T09:08:35.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554385+0000", 2024-09-14T09:08:35.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.147799443 2024-09-14T09:08:35.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554441+0000", 2024-09-14T09:08:35.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5921000000000002e-05 2024-09-14T09:08:35.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554731+0000", 2024-09-14T09:08:35.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028991800000000001 2024-09-14T09:08:35.383 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.383 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.383 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.383 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107620 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 43008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:35.383 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:35.383 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.942413893999998, 2024-09-14T09:08:35.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.148512643, 2024-09-14T09:08:35.383 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.384 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.384 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.384 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.384 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.384 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.384 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107620 2024-09-14T09:08:35.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.384 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:35.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:35.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406593+0000", 2024-09-14T09:08:35.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9999999999999998e-07 2024-09-14T09:08:35.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406595+0000", 2024-09-14T09:08:35.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9360000000000002e-06 2024-09-14T09:08:35.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406605+0000", 2024-09-14T09:08:35.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0268000000000001e-05 2024-09-14T09:08:35.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406614+0000", 2024-09-14T09:08:35.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7180000000000002e-06 2024-09-14T09:08:35.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554750+0000", 2024-09-14T09:08:35.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14813599699999999 2024-09-14T09:08:35.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.554794+0000", 2024-09-14T09:08:35.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.401e-05 2024-09-14T09:08:35.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555105+0000", 2024-09-14T09:08:35.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031131399999999998 2024-09-14T09:08:35.387 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.387 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.388 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.388 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107622 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:35.388 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:35.388 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.942382637999998, 2024-09-14T09:08:35.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14902820899999999, 2024-09-14T09:08:35.388 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.388 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.388 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.388 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.388 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.388 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.389 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107622 2024-09-14T09:08:35.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.389 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:35.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:35.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406624+0000", 2024-09-14T09:08:35.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9200000000000001e-07 2024-09-14T09:08:35.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406636+0000", 2024-09-14T09:08:35.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.189e-05 2024-09-14T09:08:35.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406649+0000", 2024-09-14T09:08:35.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2442e-05 2024-09-14T09:08:35.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406658+0000", 2024-09-14T09:08:35.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3239999999999992e-06 2024-09-14T09:08:35.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555287+0000", 2024-09-14T09:08:35.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14862868400000001 2024-09-14T09:08:35.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555348+0000", 2024-09-14T09:08:35.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1686999999999994e-05 2024-09-14T09:08:35.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555652+0000", 2024-09-14T09:08:35.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00030369000000000001 2024-09-14T09:08:35.392 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.392 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.392 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.392 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107624 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:35.392 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:35.393 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.942338945000003, 2024-09-14T09:08:35.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14934019900000001, 2024-09-14T09:08:35.393 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.393 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.393 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.393 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.393 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.393 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.393 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107624 2024-09-14T09:08:35.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.393 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:35.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:35.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406668+0000", 2024-09-14T09:08:35.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1399999999999997e-07 2024-09-14T09:08:35.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406670+0000", 2024-09-14T09:08:35.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4099999999999998e-06 2024-09-14T09:08:35.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406680+0000", 2024-09-14T09:08:35.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.6460000000000005e-06 2024-09-14T09:08:35.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406688+0000", 2024-09-14T09:08:35.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5859999999999999e-06 2024-09-14T09:08:35.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555693+0000", 2024-09-14T09:08:35.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14900544499999999 2024-09-14T09:08:35.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.555743+0000", 2024-09-14T09:08:35.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9966999999999998e-05 2024-09-14T09:08:35.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556008+0000", 2024-09-14T09:08:35.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026463099999999998 2024-09-14T09:08:35.397 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.397 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.397 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.397 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107626 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:35.397 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:08:35.397 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.942306970999994, 2024-09-14T09:08:35.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14983149500000001, 2024-09-14T09:08:35.397 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.397 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.397 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.397 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.398 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.398 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.398 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107626 2024-09-14T09:08:35.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.398 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:08:35.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.406699+0000", 2024-09-14T09:08:35.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408087+0000", 2024-09-14T09:08:35.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001387735 2024-09-14T09:08:35.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408094+0000", 2024-09-14T09:08:35.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9399999999999996e-06 2024-09-14T09:08:35.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408122+0000", 2024-09-14T09:08:35.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8076e-05 2024-09-14T09:08:35.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.400 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.400 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408147+0000", 2024-09-14T09:08:35.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4726e-05 2024-09-14T09:08:35.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.400 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.400 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556048+0000", 2024-09-14T09:08:35.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14790144299999999 2024-09-14T09:08:35.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.400 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556175+0000", 2024-09-14T09:08:35.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012663 2024-09-14T09:08:35.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556531+0000", 2024-09-14T09:08:35.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035594499999999998 2024-09-14T09:08:35.401 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.401 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.401 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.401 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107628 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:35.402 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:08:35.402 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.940837360000003, 2024-09-14T09:08:35.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14882648900000001, 2024-09-14T09:08:35.402 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.402 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.402 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.402 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.402 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.402 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.402 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107628 2024-09-14T09:08:35.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.402 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:08:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408169+0000", 2024-09-14T09:08:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408878+0000", 2024-09-14T09:08:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070858800000000001 2024-09-14T09:08:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408885+0000", 2024-09-14T09:08:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4839999999999998e-06 2024-09-14T09:08:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408900+0000", 2024-09-14T09:08:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5174000000000001e-05 2024-09-14T09:08:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408918+0000", 2024-09-14T09:08:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.808e-05 2024-09-14T09:08:35.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556572+0000", 2024-09-14T09:08:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14765399500000001 2024-09-14T09:08:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556651+0000", 2024-09-14T09:08:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8919999999999997e-05 2024-09-14T09:08:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.556996+0000", 2024-09-14T09:08:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034424800000000001 2024-09-14T09:08:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107630 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 63488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:08:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.940068526000005, 2024-09-14T09:08:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14855159200000001, 2024-09-14T09:08:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.406 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107630 2024-09-14T09:08:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:08:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.408938+0000", 2024-09-14T09:08:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409510+0000", 2024-09-14T09:08:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00057189700000000003 2024-09-14T09:08:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409518+0000", 2024-09-14T09:08:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7619999999999995e-06 2024-09-14T09:08:35.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409534+0000", 2024-09-14T09:08:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.594e-05 2024-09-14T09:08:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409546+0000", 2024-09-14T09:08:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2604e-05 2024-09-14T09:08:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557048+0000", 2024-09-14T09:08:35.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14750137899999999 2024-09-14T09:08:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557117+0000", 2024-09-14T09:08:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8998999999999994e-05 2024-09-14T09:08:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557490+0000", 2024-09-14T09:08:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037301100000000002 2024-09-14T09:08:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107632 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 67584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:08:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.939437580999993, 2024-09-14T09:08:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14833944199999999, 2024-09-14T09:08:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107632 2024-09-14T09:08:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:08:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.409569+0000", 2024-09-14T09:08:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412609+0000", 2024-09-14T09:08:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003040105 2024-09-14T09:08:35.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412618+0000", 2024-09-14T09:08:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2960000000000004e-06 2024-09-14T09:08:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412640+0000", 2024-09-14T09:08:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1773999999999999e-05 2024-09-14T09:08:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412655+0000", 2024-09-14T09:08:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4538e-05 2024-09-14T09:08:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557538+0000", 2024-09-14T09:08:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14488311800000001 2024-09-14T09:08:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557594+0000", 2024-09-14T09:08:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6122999999999998e-05 2024-09-14T09:08:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557908+0000", 2024-09-14T09:08:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031448799999999997 2024-09-14T09:08:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107634 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:08:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.936319933999997, 2024-09-14T09:08:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.145666186, 2024-09-14T09:08:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.415 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107634 2024-09-14T09:08:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:08:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412687+0000", 2024-09-14T09:08:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412688+0000", 2024-09-14T09:08:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9000000000000005e-07 2024-09-14T09:08:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412707+0000", 2024-09-14T09:08:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9232000000000002e-05 2024-09-14T09:08:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412722+0000", 2024-09-14T09:08:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5722e-05 2024-09-14T09:08:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412733+0000", 2024-09-14T09:08:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0909e-05 2024-09-14T09:08:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.557946+0000", 2024-09-14T09:08:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14521263100000001 2024-09-14T09:08:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558020+0000", 2024-09-14T09:08:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4294000000000004e-05 2024-09-14T09:08:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558353+0000", 2024-09-14T09:08:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00033240800000000002 2024-09-14T09:08:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107636 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:08:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.936255630999995, 2024-09-14T09:08:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14599377399999999, 2024-09-14T09:08:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107636 2024-09-14T09:08:35.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:08:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412751+0000", 2024-09-14T09:08:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412752+0000", 2024-09-14T09:08:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.156e-06 2024-09-14T09:08:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412770+0000", 2024-09-14T09:08:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7521999999999999e-05 2024-09-14T09:08:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412790+0000", 2024-09-14T09:08:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0942e-05 2024-09-14T09:08:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412809+0000", 2024-09-14T09:08:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8887999999999999e-05 2024-09-14T09:08:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558387+0000", 2024-09-14T09:08:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14557798799999999 2024-09-14T09:08:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558448+0000", 2024-09-14T09:08:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0862000000000001e-05 2024-09-14T09:08:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558745+0000", 2024-09-14T09:08:35.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029641599999999998 2024-09-14T09:08:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107638 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:08:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.936168374999994, 2024-09-14T09:08:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146344799, 2024-09-14T09:08:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.424 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107638 2024-09-14T09:08:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:08:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412838+0000", 2024-09-14T09:08:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412839+0000", 2024-09-14T09:08:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0720000000000001e-06 2024-09-14T09:08:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412844+0000", 2024-09-14T09:08:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7759999999999997e-06 2024-09-14T09:08:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412857+0000", 2024-09-14T09:08:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3138e-05 2024-09-14T09:08:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412867+0000", 2024-09-14T09:08:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7229999999999999e-06 2024-09-14T09:08:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558785+0000", 2024-09-14T09:08:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.14591828700000001 2024-09-14T09:08:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.558833+0000", 2024-09-14T09:08:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7790000000000002e-05 2024-09-14T09:08:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559183+0000", 2024-09-14T09:08:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035001299999999998 2024-09-14T09:08:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107640 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:08:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.936123223999999, 2024-09-14T09:08:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.146686804, 2024-09-14T09:08:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.428 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107640 2024-09-14T09:08:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:08:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.412883+0000", 2024-09-14T09:08:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420446+0000", 2024-09-14T09:08:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075632529999999998 2024-09-14T09:08:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420456+0000", 2024-09-14T09:08:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0850000000000003e-06 2024-09-14T09:08:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420475+0000", 2024-09-14T09:08:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9406e-05 2024-09-14T09:08:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420488+0000", 2024-09-14T09:08:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3278000000000001e-05 2024-09-14T09:08:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559205+0000", 2024-09-14T09:08:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13871644999999999 2024-09-14T09:08:35.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559252+0000", 2024-09-14T09:08:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7666000000000001e-05 2024-09-14T09:08:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.559570+0000", 2024-09-14T09:08:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00031766599999999998 2024-09-14T09:08:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:107660 3.10 3:0fc072fb:::rbd_data.115e500fbcd9.0000000000000008:27 [read 124928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-14T09:08:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:08:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.928052981999997, 2024-09-14T09:08:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.142954628, 2024-09-14T09:08:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 107660 2024-09-14T09:08:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.433 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:08:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.420953+0000", 2024-09-14T09:08:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428190+0000", 2024-09-14T09:08:35.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072360339999999997 2024-09-14T09:08:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428398+0000", 2024-09-14T09:08:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020898499999999999 2024-09-14T09:08:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428458+0000", 2024-09-14T09:08:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9985999999999998e-05 2024-09-14T09:08:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.428480+0000", 2024-09-14T09:08:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1284000000000002e-05 2024-09-14T09:08:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563583+0000", 2024-09-14T09:08:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.13510361800000001 2024-09-14T09:08:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563644+0000", 2024-09-14T09:08:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1092000000000004e-05 2024-09-14T09:08:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:23.563908+0000", 2024-09-14T09:08:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026362899999999998 2024-09-14T09:08:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.437 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.437 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:35.438 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T09:08:35.473 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-14T09:08:35.474 INFO:tasks.thrashosds.thrasher:Reviving osd 4 2024-09-14T09:08:35.474 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-14T09:08:35.474 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-14T09:08:35.486 INFO:tasks.ceph.osd.4:Started 2024-09-14T09:08:35.486 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:08:35.569 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:35.570 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:35.570 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:08:35.570 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:35.584 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T09:08:35.610 INFO:teuthology.orchestra.run.smithi183.stderr:no valid command found; 10 closest matches: 2024-09-14T09:08:35.610 INFO:teuthology.orchestra.run.smithi183.stderr:0 2024-09-14T09:08:35.610 INFO:teuthology.orchestra.run.smithi183.stderr:1 2024-09-14T09:08:35.610 INFO:teuthology.orchestra.run.smithi183.stderr:2 2024-09-14T09:08:35.610 INFO:teuthology.orchestra.run.smithi183.stderr:abort 2024-09-14T09:08:35.610 INFO:teuthology.orchestra.run.smithi183.stderr:assert 2024-09-14T09:08:35.610 INFO:teuthology.orchestra.run.smithi183.stderr:config diff 2024-09-14T09:08:35.610 INFO:teuthology.orchestra.run.smithi183.stderr:config diff get 2024-09-14T09:08:35.610 INFO:teuthology.orchestra.run.smithi183.stderr:config get 2024-09-14T09:08:35.610 INFO:teuthology.orchestra.run.smithi183.stderr:config help [] 2024-09-14T09:08:35.610 INFO:teuthology.orchestra.run.smithi183.stderr:config set ... 2024-09-14T09:08:35.610 INFO:teuthology.orchestra.run.smithi183.stderr:admin_socket: invalid command 2024-09-14T09:08:35.612 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T09:08:35.612 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-4, ['dump_ops_in_flight'] 2024-09-14T09:08:35.699 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:35.700 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:35.700 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-09-14T09:08:35.700 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-09-14T09:08:35.700 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:35.713 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T09:08:35.831 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:35.832 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-09-14T09:08:35.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-09-14T09:08:35.832 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-09-14T09:08:35.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.832 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38593 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [watch reconnect cookie 139718844451680 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:08:35.832 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:08:35.832 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 154.018579157, 2024-09-14T09:08:35.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0218296630000001, 2024-09-14T09:08:35.832 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.832 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.832 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.832 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.833 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.833 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.833 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38593 2024-09-14T09:08:35.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.833 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:08:35.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818417+0000", 2024-09-14T09:08:35.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818420+0000", 2024-09-14T09:08:35.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8399999999999999e-06 2024-09-14T09:08:35.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818431+0000", 2024-09-14T09:08:35.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0859999999999999e-05 2024-09-14T09:08:35.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818466+0000", 2024-09-14T09:08:35.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4486999999999997e-05 2024-09-14T09:08:35.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818481+0000", 2024-09-14T09:08:35.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5362999999999998e-05 2024-09-14T09:08:35.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818659+0000", 2024-09-14T09:08:35.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00017807500000000001 2024-09-14T09:08:35.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op must wait for map", 2024-09-14T09:08:35.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.818681+0000", 2024-09-14T09:08:35.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2096999999999999e-05 2024-09-14T09:08:35.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825472+0000", 2024-09-14T09:08:35.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067905170000000003 2024-09-14T09:08:35.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:35.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.825491+0000", 2024-09-14T09:08:35.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9372999999999999e-05 2024-09-14T09:08:35.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835402+0000", 2024-09-14T09:08:35.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0099111620000001 2024-09-14T09:08:35.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.835548+0000", 2024-09-14T09:08:35.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014607300000000001 2024-09-14T09:08:35.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.840247+0000", 2024-09-14T09:08:35.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046988159999999998 2024-09-14T09:08:35.837 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.838 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.838 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.838 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:38615 3.f 3:f1d26bb4:::rbd_header.115e9e133b92:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-14T09:08:35.838 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:08:35.838 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 153.970213949, 2024-09-14T09:08:35.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.97014128899999996, 2024-09-14T09:08:35.838 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.838 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.838 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-09-14T09:08:35.838 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.838 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.838 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.839 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 38615 2024-09-14T09:08:35.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.839 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:08:35.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866783+0000", 2024-09-14T09:08:35.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866787+0000", 2024-09-14T09:08:35.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.476e-06 2024-09-14T09:08:35.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866806+0000", 2024-09-14T09:08:35.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8606000000000001e-05 2024-09-14T09:08:35.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866839+0000", 2024-09-14T09:08:35.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3510000000000003e-05 2024-09-14T09:08:35.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.866867+0000", 2024-09-14T09:08:35.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7610000000000002e-05 2024-09-14T09:08:35.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867031+0000", 2024-09-14T09:08:35.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016414199999999999 2024-09-14T09:08:35.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:35.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:01.867054+0000", 2024-09-14T09:08:35.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2693000000000001e-05 2024-09-14T09:08:35.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836462+0000", 2024-09-14T09:08:35.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.96940797700000003 2024-09-14T09:08:35.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-09-14T09:08:35.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836547+0000", 2024-09-14T09:08:35.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5294e-05 2024-09-14T09:08:35.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:06:02.836924+0000", 2024-09-14T09:08:35.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00037698099999999997 2024-09-14T09:08:35.843 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.843 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.843 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.843 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105723 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:35.843 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:35.843 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.741779905000001, 2024-09-14T09:08:35.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77512799700000001, 2024-09-14T09:08:35.843 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.843 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.844 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:35.844 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.844 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.844 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.844 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105723 2024-09-14T09:08:35.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.844 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:35.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:35.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095217+0000", 2024-09-14T09:08:35.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1399999999999997e-07 2024-09-14T09:08:35.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095220+0000", 2024-09-14T09:08:35.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0510000000000001e-06 2024-09-14T09:08:35.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095228+0000", 2024-09-14T09:08:35.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4159999999999998e-06 2024-09-14T09:08:35.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095234+0000", 2024-09-14T09:08:35.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3010000000000003e-06 2024-09-14T09:08:35.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103902+0000", 2024-09-14T09:08:35.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086678299999999996 2024-09-14T09:08:35.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:35.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103905+0000", 2024-09-14T09:08:35.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5020000000000001e-06 2024-09-14T09:08:35.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870332+0000", 2024-09-14T09:08:35.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76642700900000005 2024-09-14T09:08:35.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870345+0000", 2024-09-14T09:08:35.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2274e-05 2024-09-14T09:08:35.848 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.848 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.848 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.848 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105724 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:35.848 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:08:35.848 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.741755392000002, 2024-09-14T09:08:35.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77515698099999997, 2024-09-14T09:08:35.848 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.848 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.848 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:35.848 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.848 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.849 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.849 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105724 2024-09-14T09:08:35.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.849 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:08:35.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095241+0000", 2024-09-14T09:08:35.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095242+0000", 2024-09-14T09:08:35.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9400000000000001e-07 2024-09-14T09:08:35.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095243+0000", 2024-09-14T09:08:35.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7540000000000001e-06 2024-09-14T09:08:35.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095248+0000", 2024-09-14T09:08:35.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1320000000000004e-06 2024-09-14T09:08:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095251+0000", 2024-09-14T09:08:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2770000000000001e-06 2024-09-14T09:08:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103947+0000", 2024-09-14T09:08:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086960089999999993 2024-09-14T09:08:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103955+0000", 2024-09-14T09:08:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9519999999999994e-06 2024-09-14T09:08:35.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870386+0000", 2024-09-14T09:08:35.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76643141100000001 2024-09-14T09:08:35.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870398+0000", 2024-09-14T09:08:35.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2052000000000001e-05 2024-09-14T09:08:35.852 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.852 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.852 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.853 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105727 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:35.853 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:35.853 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.741715686000006, 2024-09-14T09:08:35.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517292900000001, 2024-09-14T09:08:35.853 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.853 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.853 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:35.853 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.853 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.853 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.853 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105727 2024-09-14T09:08:35.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.853 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:35.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:35.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095281+0000", 2024-09-14T09:08:35.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3300000000000002e-07 2024-09-14T09:08:35.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095284+0000", 2024-09-14T09:08:35.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9790000000000002e-06 2024-09-14T09:08:35.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095292+0000", 2024-09-14T09:08:35.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3810000000000001e-06 2024-09-14T09:08:35.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095298+0000", 2024-09-14T09:08:35.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9710000000000001e-06 2024-09-14T09:08:35.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103987+0000", 2024-09-14T09:08:35.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086893820000000007 2024-09-14T09:08:35.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:35.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.103991+0000", 2024-09-14T09:08:35.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5480000000000002e-06 2024-09-14T09:08:35.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870442+0000", 2024-09-14T09:08:35.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.766451145 2024-09-14T09:08:35.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870454+0000", 2024-09-14T09:08:35.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.199e-05 2024-09-14T09:08:35.857 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.857 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.857 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.857 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105729 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:35.857 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:08:35.858 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.741667328999995, 2024-09-14T09:08:35.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517686500000005, 2024-09-14T09:08:35.858 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.858 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.858 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:35.858 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.858 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.858 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.858 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105729 2024-09-14T09:08:35.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.858 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:08:35.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095329+0000", 2024-09-14T09:08:35.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095330+0000", 2024-09-14T09:08:35.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-09-14T09:08:35.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095332+0000", 2024-09-14T09:08:35.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1560000000000002e-06 2024-09-14T09:08:35.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095338+0000", 2024-09-14T09:08:35.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.592e-06 2024-09-14T09:08:35.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095341+0000", 2024-09-14T09:08:35.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8139999999999998e-06 2024-09-14T09:08:35.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104031+0000", 2024-09-14T09:08:35.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086896330000000004 2024-09-14T09:08:35.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:35.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104039+0000", 2024-09-14T09:08:35.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0900000000000005e-06 2024-09-14T09:08:35.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870495+0000", 2024-09-14T09:08:35.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76645611199999997 2024-09-14T09:08:35.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870506+0000", 2024-09-14T09:08:35.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0986e-05 2024-09-14T09:08:35.862 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.862 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.862 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.862 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105731 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:35.862 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:35.862 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.741649061999993, 2024-09-14T09:08:35.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77520598600000001, 2024-09-14T09:08:35.862 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.862 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.862 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:35.863 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.863 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.863 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.863 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105731 2024-09-14T09:08:35.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.863 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:35.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:35.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095348+0000", 2024-09-14T09:08:35.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8599999999999999e-07 2024-09-14T09:08:35.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095350+0000", 2024-09-14T09:08:35.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1110000000000002e-06 2024-09-14T09:08:35.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095357+0000", 2024-09-14T09:08:35.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9020000000000002e-06 2024-09-14T09:08:35.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095360+0000", 2024-09-14T09:08:35.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1939999999999998e-06 2024-09-14T09:08:35.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104073+0000", 2024-09-14T09:08:35.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087131210000000008 2024-09-14T09:08:35.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:35.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104077+0000", 2024-09-14T09:08:35.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.501e-06 2024-09-14T09:08:35.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870541+0000", 2024-09-14T09:08:35.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76646461700000001 2024-09-14T09:08:35.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870554+0000", 2024-09-14T09:08:35.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2153999999999999e-05 2024-09-14T09:08:35.866 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.866 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.866 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.867 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105733 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:35.867 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:35.867 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.741629031000002, 2024-09-14T09:08:35.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.775237277, 2024-09-14T09:08:35.867 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.867 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.867 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:35.867 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.867 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.867 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.867 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105733 2024-09-14T09:08:35.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.868 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:35.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:35.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095368+0000", 2024-09-14T09:08:35.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8699999999999995e-07 2024-09-14T09:08:35.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095371+0000", 2024-09-14T09:08:35.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5450000000000002e-06 2024-09-14T09:08:35.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095377+0000", 2024-09-14T09:08:35.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.776e-06 2024-09-14T09:08:35.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095383+0000", 2024-09-14T09:08:35.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3129999999999998e-06 2024-09-14T09:08:35.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104124+0000", 2024-09-14T09:08:35.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087409840000000003 2024-09-14T09:08:35.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:35.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104132+0000", 2024-09-14T09:08:35.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3159999999999997e-06 2024-09-14T09:08:35.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870593+0000", 2024-09-14T09:08:35.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76646064800000002 2024-09-14T09:08:35.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870605+0000", 2024-09-14T09:08:35.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2208000000000001e-05 2024-09-14T09:08:35.871 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.871 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.871 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.871 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105734 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:35.872 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:35.872 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.741605949000004, 2024-09-14T09:08:35.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77526569599999995, 2024-09-14T09:08:35.872 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.872 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.872 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:35.872 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.872 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.872 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.872 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105734 2024-09-14T09:08:35.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.872 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:35.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:35.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095391+0000", 2024-09-14T09:08:35.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7399999999999998e-07 2024-09-14T09:08:35.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095394+0000", 2024-09-14T09:08:35.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8030000000000001e-06 2024-09-14T09:08:35.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095401+0000", 2024-09-14T09:08:35.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.046e-06 2024-09-14T09:08:35.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095407+0000", 2024-09-14T09:08:35.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6060000000000002e-06 2024-09-14T09:08:35.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104167+0000", 2024-09-14T09:08:35.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087602530000000008 2024-09-14T09:08:35.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:35.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104171+0000", 2024-09-14T09:08:35.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7620000000000001e-06 2024-09-14T09:08:35.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870645+0000", 2024-09-14T09:08:35.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76647400600000004 2024-09-14T09:08:35.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870656+0000", 2024-09-14T09:08:35.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1746000000000001e-05 2024-09-14T09:08:35.876 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.876 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.876 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.876 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105738 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:35.876 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:08:35.876 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.741566222000003, 2024-09-14T09:08:35.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77527530700000002, 2024-09-14T09:08:35.876 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.876 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.876 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:35.877 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.877 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.877 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.877 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105738 2024-09-14T09:08:35.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.877 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:08:35.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095430+0000", 2024-09-14T09:08:35.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095431+0000", 2024-09-14T09:08:35.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0999999999999999e-07 2024-09-14T09:08:35.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095434+0000", 2024-09-14T09:08:35.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.757e-06 2024-09-14T09:08:35.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095442+0000", 2024-09-14T09:08:35.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9929999999999995e-06 2024-09-14T09:08:35.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095448+0000", 2024-09-14T09:08:35.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1340000000000001e-06 2024-09-14T09:08:35.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104213+0000", 2024-09-14T09:08:35.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087650000000000002 2024-09-14T09:08:35.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:35.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104221+0000", 2024-09-14T09:08:35.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4589999999999998e-06 2024-09-14T09:08:35.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870694+0000", 2024-09-14T09:08:35.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76647303200000005 2024-09-14T09:08:35.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870706+0000", 2024-09-14T09:08:35.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1522000000000001e-05 2024-09-14T09:08:35.881 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.881 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.881 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.881 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105739 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:35.881 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:35.881 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.741541693000002, 2024-09-14T09:08:35.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77530221700000002, 2024-09-14T09:08:35.881 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.881 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.881 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:35.881 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.881 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.881 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.882 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105739 2024-09-14T09:08:35.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.882 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:35.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:35.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095455+0000", 2024-09-14T09:08:35.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.15e-07 2024-09-14T09:08:35.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095457+0000", 2024-09-14T09:08:35.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6470000000000001e-06 2024-09-14T09:08:35.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095461+0000", 2024-09-14T09:08:35.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0910000000000003e-06 2024-09-14T09:08:35.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095464+0000", 2024-09-14T09:08:35.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2059999999999999e-06 2024-09-14T09:08:35.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104253+0000", 2024-09-14T09:08:35.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087884120000000007 2024-09-14T09:08:35.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:35.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104256+0000", 2024-09-14T09:08:35.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5439999999999999e-06 2024-09-14T09:08:35.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870745+0000", 2024-09-14T09:08:35.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76648908000000004 2024-09-14T09:08:35.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870757+0000", 2024-09-14T09:08:35.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1822e-05 2024-09-14T09:08:35.885 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.885 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.886 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.886 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105741 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:35.886 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:08:35.886 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.741472567000002, 2024-09-14T09:08:35.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77528469899999997, 2024-09-14T09:08:35.886 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.886 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.886 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:35.886 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.886 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.886 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.886 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105741 2024-09-14T09:08:35.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.887 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:08:35.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095524+0000", 2024-09-14T09:08:35.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095525+0000", 2024-09-14T09:08:35.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3740000000000001e-06 2024-09-14T09:08:35.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095533+0000", 2024-09-14T09:08:35.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6380000000000004e-06 2024-09-14T09:08:35.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095549+0000", 2024-09-14T09:08:35.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5554999999999999e-05 2024-09-14T09:08:35.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095566+0000", 2024-09-14T09:08:35.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7146000000000002e-05 2024-09-14T09:08:35.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104297+0000", 2024-09-14T09:08:35.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087315750000000001 2024-09-14T09:08:35.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:35.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104305+0000", 2024-09-14T09:08:35.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1389999999999995e-06 2024-09-14T09:08:35.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870797+0000", 2024-09-14T09:08:35.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76649129599999999 2024-09-14T09:08:35.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870809+0000", 2024-09-14T09:08:35.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1976e-05 2024-09-14T09:08:35.890 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.890 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.890 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.891 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105742 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:35.891 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:08:35.891 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.741407783, 2024-09-14T09:08:35.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.775271873, 2024-09-14T09:08:35.891 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.891 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.891 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:35.891 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.891 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.891 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.891 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105742 2024-09-14T09:08:35.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.891 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:08:35.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095589+0000", 2024-09-14T09:08:35.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095590+0000", 2024-09-14T09:08:35.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0400000000000005e-07 2024-09-14T09:08:35.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095594+0000", 2024-09-14T09:08:35.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5639999999999998e-06 2024-09-14T09:08:35.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095620+0000", 2024-09-14T09:08:35.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5494999999999999e-05 2024-09-14T09:08:35.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095639+0000", 2024-09-14T09:08:35.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9639999999999998e-05 2024-09-14T09:08:35.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104340+0000", 2024-09-14T09:08:35.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087004950000000008 2024-09-14T09:08:35.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:35.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104343+0000", 2024-09-14T09:08:35.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4020000000000002e-06 2024-09-14T09:08:35.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870850+0000", 2024-09-14T09:08:35.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76650648499999996 2024-09-14T09:08:35.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870861+0000", 2024-09-14T09:08:35.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0888e-05 2024-09-14T09:08:35.895 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.895 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.895 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.896 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105745 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:35.896 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:08:35.896 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.741309168000001, 2024-09-14T09:08:35.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77521998700000005, 2024-09-14T09:08:35.896 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.896 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.896 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:35.896 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.896 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.896 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.896 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105745 2024-09-14T09:08:35.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.896 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:08:35.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095687+0000", 2024-09-14T09:08:35.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095688+0000", 2024-09-14T09:08:35.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2699999999999998e-07 2024-09-14T09:08:35.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095693+0000", 2024-09-14T09:08:35.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0479999999999998e-06 2024-09-14T09:08:35.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095707+0000", 2024-09-14T09:08:35.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.342e-05 2024-09-14T09:08:35.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095716+0000", 2024-09-14T09:08:35.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9800000000000004e-06 2024-09-14T09:08:35.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104388+0000", 2024-09-14T09:08:35.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086717349999999999 2024-09-14T09:08:35.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:35.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104396+0000", 2024-09-14T09:08:35.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1769999999999998e-06 2024-09-14T09:08:35.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870897+0000", 2024-09-14T09:08:35.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76650102600000003 2024-09-14T09:08:35.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870907+0000", 2024-09-14T09:08:35.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0674e-05 2024-09-14T09:08:35.900 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.900 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.900 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.900 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105747 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:35.900 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:35.900 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.741262947999999, 2024-09-14T09:08:35.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77522190300000005, 2024-09-14T09:08:35.901 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.901 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.901 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:35.901 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.901 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.901 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.901 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105747 2024-09-14T09:08:35.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.901 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:35.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:35.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095734+0000", 2024-09-14T09:08:35.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8999999999999995e-07 2024-09-14T09:08:35.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095738+0000", 2024-09-14T09:08:35.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8430000000000003e-06 2024-09-14T09:08:35.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095750+0000", 2024-09-14T09:08:35.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1229999999999999e-05 2024-09-14T09:08:35.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095758+0000", 2024-09-14T09:08:35.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.337e-06 2024-09-14T09:08:35.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104431+0000", 2024-09-14T09:08:35.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086728780000000002 2024-09-14T09:08:35.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:35.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104434+0000", 2024-09-14T09:08:35.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5049999999999999e-06 2024-09-14T09:08:35.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870945+0000", 2024-09-14T09:08:35.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651031800000002 2024-09-14T09:08:35.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870956+0000", 2024-09-14T09:08:35.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1002000000000001e-05 2024-09-14T09:08:35.905 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.905 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.905 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.905 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105750 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:35.905 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:35.905 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.741193002000003, 2024-09-14T09:08:35.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77519961800000003, 2024-09-14T09:08:35.906 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.906 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.906 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:35.906 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.906 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.906 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.906 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105750 2024-09-14T09:08:35.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.906 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:35.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:35.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095804+0000", 2024-09-14T09:08:35.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3600000000000002e-07 2024-09-14T09:08:35.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095809+0000", 2024-09-14T09:08:35.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4000000000000002e-06 2024-09-14T09:08:35.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095822+0000", 2024-09-14T09:08:35.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2704000000000001e-05 2024-09-14T09:08:35.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095831+0000", 2024-09-14T09:08:35.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9530000000000005e-06 2024-09-14T09:08:35.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104477+0000", 2024-09-14T09:08:35.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086459989999999997 2024-09-14T09:08:35.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:35.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104488+0000", 2024-09-14T09:08:35.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1270000000000001e-05 2024-09-14T09:08:35.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.870992+0000", 2024-09-14T09:08:35.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.766504097 2024-09-14T09:08:35.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871003+0000", 2024-09-14T09:08:35.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1358999999999999e-05 2024-09-14T09:08:35.910 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.910 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.910 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.910 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105751 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:35.910 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:35.911 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.741135116999999, 2024-09-14T09:08:35.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77518966700000003, 2024-09-14T09:08:35.911 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.911 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.911 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:35.911 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.911 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.911 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.911 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105751 2024-09-14T09:08:35.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.911 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:35.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:35.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095862+0000", 2024-09-14T09:08:35.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7300000000000005e-07 2024-09-14T09:08:35.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095867+0000", 2024-09-14T09:08:35.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2729999999999996e-06 2024-09-14T09:08:35.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095879+0000", 2024-09-14T09:08:35.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2917e-05 2024-09-14T09:08:35.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095888+0000", 2024-09-14T09:08:35.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2609999999999993e-06 2024-09-14T09:08:35.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104520+0000", 2024-09-14T09:08:35.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086322970000000006 2024-09-14T09:08:35.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:35.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104524+0000", 2024-09-14T09:08:35.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.54e-06 2024-09-14T09:08:35.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871040+0000", 2024-09-14T09:08:35.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651615399999995 2024-09-14T09:08:35.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871051+0000", 2024-09-14T09:08:35.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1452e-05 2024-09-14T09:08:35.915 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.915 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.915 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.915 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105752 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:35.915 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:08:35.915 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.741074353000002, 2024-09-14T09:08:35.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77517684899999995, 2024-09-14T09:08:35.915 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.915 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.916 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:35.916 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.916 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.916 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.916 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105752 2024-09-14T09:08:35.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.916 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:08:35.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095922+0000", 2024-09-14T09:08:35.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095923+0000", 2024-09-14T09:08:35.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3300000000000001e-07 2024-09-14T09:08:35.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095927+0000", 2024-09-14T09:08:35.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1479999999999999e-06 2024-09-14T09:08:35.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095940+0000", 2024-09-14T09:08:35.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2309e-05 2024-09-14T09:08:35.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095948+0000", 2024-09-14T09:08:35.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9199999999999993e-06 2024-09-14T09:08:35.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104566+0000", 2024-09-14T09:08:35.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086171449999999997 2024-09-14T09:08:35.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:35.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104574+0000", 2024-09-14T09:08:35.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0690000000000002e-06 2024-09-14T09:08:35.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871087+0000", 2024-09-14T09:08:35.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76651370699999999 2024-09-14T09:08:35.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871099+0000", 2024-09-14T09:08:35.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1718e-05 2024-09-14T09:08:35.920 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.920 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.920 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.920 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105755 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:35.920 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:08:35.920 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.74101641, 2024-09-14T09:08:35.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77516787899999995, 2024-09-14T09:08:35.920 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.920 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.920 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:35.920 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.921 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.921 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.921 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105755 2024-09-14T09:08:35.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.921 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:08:35.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095980+0000", 2024-09-14T09:08:35.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095981+0000", 2024-09-14T09:08:35.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2699999999999998e-07 2024-09-14T09:08:35.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095985+0000", 2024-09-14T09:08:35.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4349999999999999e-06 2024-09-14T09:08:35.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.095999+0000", 2024-09-14T09:08:35.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3139000000000001e-05 2024-09-14T09:08:35.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096007+0000", 2024-09-14T09:08:35.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6610000000000006e-06 2024-09-14T09:08:35.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104591+0000", 2024-09-14T09:08:35.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0085841709999999998 2024-09-14T09:08:35.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-09-14T09:08:35.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.104595+0000", 2024-09-14T09:08:35.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5590000000000001e-06 2024-09-14T09:08:35.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871137+0000", 2024-09-14T09:08:35.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76654202100000002 2024-09-14T09:08:35.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871148+0000", 2024-09-14T09:08:35.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1066e-05 2024-09-14T09:08:35.924 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.924 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.925 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.925 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4446.0:105757 3.d 3.d31648cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T09:08:35.925 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:08:35.925 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.740957829999999, 2024-09-14T09:08:35.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.77516401300000004, 2024-09-14T09:08:35.925 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-09-14T09:08:35.925 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-09-14T09:08:35.925 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-09-14T09:08:35.925 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-09-14T09:08:35.925 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4446", 2024-09-14T09:08:35.925 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.176:0/3650604933", 2024-09-14T09:08:35.926 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 105757 2024-09-14T09:08:35.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.926 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-09-14T09:08:35.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-09-14T09:08:35.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:08:35.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-09-14T09:08:35.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.096039+0000", 2024-09-14T09:08:35.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-09-14T09:08:35.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-09-14T09:08:35.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856228+0000", 2024-09-14T09:08:35.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.76018875100000005 2024-09-14T09:08:35.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-09-14T09:08:35.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856240+0000", 2024-09-14T09:08:35.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.291e-05 2024-09-14T09:08:35.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-09-14T09:08:35.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856268+0000", 2024-09-14T09:08:35.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7628999999999999e-05 2024-09-14T09:08:35.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-09-14T09:08:35.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.856307+0000", 2024-09-14T09:08:35.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8967000000000001e-05 2024-09-14T09:08:35.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-09-14T09:08:35.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871192+0000", 2024-09-14T09:08:35.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.01488449 2024-09-14T09:08:35.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-14T09:08:35.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-09-14T09:08:35.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-09-14T09:08:35.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-09-14T09:07:21.871203+0000", 2024-09-14T09:08:35.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1266e-05 2024-09-14T09:08:35.928 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.928 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.929 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.929 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-14T09:08:35.929 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-14T09:08:35.929 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:36.515 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T09:08:36.571 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:08:36.574+0000 7f8fb82f97c0 -1 Falling back to public interface 2024-09-14T09:08:37.499 INFO:teuthology.orchestra.run.smithi176.stderr:noscrub is unset 2024-09-14T09:08:37.526 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T09:08:38.517 INFO:teuthology.orchestra.run.smithi176.stderr:nodeep-scrub is unset 2024-09-14T09:08:40.616 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:08:40.744 INFO:teuthology.orchestra.run.smithi183.stderr:no valid command found; 10 closest matches: 2024-09-14T09:08:40.745 INFO:teuthology.orchestra.run.smithi183.stderr:0 2024-09-14T09:08:40.745 INFO:teuthology.orchestra.run.smithi183.stderr:1 2024-09-14T09:08:40.745 INFO:teuthology.orchestra.run.smithi183.stderr:2 2024-09-14T09:08:40.745 INFO:teuthology.orchestra.run.smithi183.stderr:abort 2024-09-14T09:08:40.745 INFO:teuthology.orchestra.run.smithi183.stderr:assert 2024-09-14T09:08:40.745 INFO:teuthology.orchestra.run.smithi183.stderr:bluefs debug_inject_read_zeros 2024-09-14T09:08:40.745 INFO:teuthology.orchestra.run.smithi183.stderr:bluefs files list 2024-09-14T09:08:40.745 INFO:teuthology.orchestra.run.smithi183.stderr:bluefs stats 2024-09-14T09:08:40.745 INFO:teuthology.orchestra.run.smithi183.stderr:bluestore allocator dump block 2024-09-14T09:08:40.745 INFO:teuthology.orchestra.run.smithi183.stderr:bluestore allocator fragmentation block 2024-09-14T09:08:40.745 INFO:teuthology.orchestra.run.smithi183.stderr:admin_socket: invalid command 2024-09-14T09:08:40.747 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T09:08:40.747 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-4, ['dump_ops_in_flight'] 2024-09-14T09:08:40.840 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:08:40.842+0000 7f8fb82f97c0 -1 osd.4 134 log_to_monitors true 2024-09-14T09:08:41.572 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-09-14T09:08:41.574+0000 7f8fac0d4640 -1 osd.4 174 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T09:08:45.748 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T09:08:45.883 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-14T09:08:45.883 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-09-14T09:08:45.883 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-09-14T09:08:45.883 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-14T09:08:45.897 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-14T09:08:46.042 INFO:teuthology.orchestra.run.smithi176.stdout:mon.a: {} 2024-09-14T09:08:46.042 INFO:teuthology.orchestra.run.smithi176.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T09:08:46.055 INFO:teuthology.orchestra.run.smithi176.stdout:mon.b: {} 2024-09-14T09:08:46.055 INFO:teuthology.orchestra.run.smithi176.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T09:08:46.065 INFO:teuthology.orchestra.run.smithi176.stdout:mon.c: {} 2024-09-14T09:08:46.065 INFO:teuthology.orchestra.run.smithi176.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T09:08:46.083 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T09:08:46.579 INFO:teuthology.orchestra.run.smithi176.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T09:08:46.607 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-14T09:08:47.587 INFO:teuthology.orchestra.run.smithi176.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-14T09:08:47.611 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T09:08:48.598 INFO:teuthology.orchestra.run.smithi176.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T09:08:48.626 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-14T09:08:49.600 INFO:teuthology.orchestra.run.smithi176.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-14T09:08:49.629 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T09:08:50.620 INFO:teuthology.orchestra.run.smithi176.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T09:08:50.648 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-14T09:08:51.636 INFO:teuthology.orchestra.run.smithi176.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-14T09:08:51.663 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T09:08:52.647 INFO:teuthology.orchestra.run.smithi176.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T09:08:52.672 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-14T09:08:53.658 INFO:teuthology.orchestra.run.smithi176.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-14T09:08:53.681 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T09:08:54.669 INFO:teuthology.orchestra.run.smithi176.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T09:08:54.694 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-14T09:08:55.680 INFO:teuthology.orchestra.run.smithi176.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-14T09:08:55.704 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T09:08:56.683 INFO:teuthology.orchestra.run.smithi176.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T09:08:56.709 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-14T09:08:57.861 INFO:teuthology.orchestra.run.smithi176.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-14T09:08:57.883 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T09:08:58.868 INFO:teuthology.orchestra.run.smithi176.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T09:08:58.891 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-14T09:08:59.864 INFO:teuthology.orchestra.run.smithi176.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-14T09:08:59.894 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T09:09:00.888 INFO:teuthology.orchestra.run.smithi176.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T09:09:00.913 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-14T09:09:01.890 INFO:teuthology.orchestra.run.smithi176.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-14T09:09:01.915 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-14T09:09:01.915 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-14T09:09:01.915 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-14T09:09:01.915 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-14T09:09:01.915 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-14T09:09:01.915 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T09:09:02.224 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:09:02.224 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":206,"fsid":"c635667d-994e-454f-bcea-51b45637b2a2","created":"2024-09-14T09:04:57.077293+0000","modified":"2024-09-14T09:09:01.883709+0000","last_up_change":"2024-09-14T09:08:42.531335+0000","last_in_change":"2024-09-14T09:07:21.060249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T09:05:11.997673+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T09:05:15.352551+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T09:05:25.840963+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":46,"snap_epoch":182,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ceedb4f-6398-467f-9562-ccf27200860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6825","nonce":3384710819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6827","nonce":3384710819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6831","nonce":3384710819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6829","nonce":3384710819}]},"public_addr":"172.21.15.176:6825/3384710819","cluster_addr":"172.21.15.176:6827/3384710819","heartbeat_back_addr":"172.21.15.176:6831/3384710819","heartbeat_front_addr":"172.21.15.176:6829/3384710819","state":["exists","up"]},{"osd":1,"uuid":"604e2f62-7c13-46be-a97f-029b28c44a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6801","nonce":612012679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6803","nonce":612012679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6807","nonce":612012679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6805","nonce":612012679}]},"public_addr":"172.21.15.176:6801/612012679","cluster_addr":"172.21.15.176:6803/612012679","heartbeat_back_addr":"172.21.15.176:6807/612012679","heartbeat_front_addr":"172.21.15.176:6805/612012679","state":["exists","up"]},{"osd":2,"uuid":"1f65589e-8b0a-4033-94c0-4ab66531db2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6809","nonce":4167379279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6811","nonce":4167379279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6815","nonce":4167379279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6813","nonce":4167379279}]},"public_addr":"172.21.15.176:6809/4167379279","cluster_addr":"172.21.15.176:6811/4167379279","heartbeat_back_addr":"172.21.15.176:6815/4167379279","heartbeat_front_addr":"172.21.15.176:6813/4167379279","state":["exists","up"]},{"osd":3,"uuid":"209d4f65-79f9-4a7d-9d23-832a4209507f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6817","nonce":1658209015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6819","nonce":1658209015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6823","nonce":1658209015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6821","nonce":1658209015}]},"public_addr":"172.21.15.176:6817/1658209015","cluster_addr":"172.21.15.176:6819/1658209015","heartbeat_back_addr":"172.21.15.176:6823/1658209015","heartbeat_front_addr":"172.21.15.176:6821/1658209015","state":["exists","up"]},{"osd":4,"uuid":"8f9a8c9b-282e-4523-9e62-5824f4aba401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":134,"up_from":186,"up_thru":189,"down_at":135,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":700723532},{"type":"v1","addr":"172.21.15.183:6809","nonce":700723532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":700723532},{"type":"v1","addr":"172.21.15.183:6811","nonce":700723532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":700723532},{"type":"v1","addr":"172.21.15.183:6815","nonce":700723532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":700723532},{"type":"v1","addr":"172.21.15.183:6813","nonce":700723532}]},"public_addr":"172.21.15.183:6809/700723532","cluster_addr":"172.21.15.183:6811/700723532","heartbeat_back_addr":"172.21.15.183:6815/700723532","heartbeat_front_addr":"172.21.15.183:6813/700723532","state":["exists","up"]},{"osd":5,"uuid":"1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6825","nonce":1711292386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6827","nonce":1711292386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6831","nonce":1711292386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6829","nonce":1711292386}]},"public_addr":"172.21.15.183:6825/1711292386","cluster_addr":"172.21.15.183:6827/1711292386","heartbeat_back_addr":"172.21.15.183:6831/1711292386","heartbeat_front_addr":"172.21.15.183:6829/1711292386","state":["exists","up"]},{"osd":6,"uuid":"59d9259d-b9dd-4326-8ab9-9d599a2bf57b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6817","nonce":2721245607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6819","nonce":2721245607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6823","nonce":2721245607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6821","nonce":2721245607}]},"public_addr":"172.21.15.183:6817/2721245607","cluster_addr":"172.21.15.183:6819/2721245607","heartbeat_back_addr":"172.21.15.183:6823/2721245607","heartbeat_front_addr":"172.21.15.183:6821/2721245607","state":["exists","up"]},{"osd":7,"uuid":"e2f27d42-786b-449b-a2fb-9dd28f4e9788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6801","nonce":2122823465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6803","nonce":2122823465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6807","nonce":2122823465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6805","nonce":2122823465}]},"public_addr":"172.21.15.183:6801/2122823465","cluster_addr":"172.21.15.183:6803/2122823465","heartbeat_back_addr":"172.21.15.183:6807/2122823465","heartbeat_front_addr":"172.21.15.183:6805/2122823465","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.411789+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:05.801873+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.206725+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:05.882180+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.013864+0000","dead_epoch":135},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.652975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.741359+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:05.873675+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.0","osds":[4,2]},{"pgid":"3.4","osds":[1,7]}],"pg_upmap_items":[{"pgid":"3.9","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":0}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T09:09:02.243 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-14T09:09:02.244 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T09:09:02.552 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:09:02.552 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":206,"fsid":"c635667d-994e-454f-bcea-51b45637b2a2","created":"2024-09-14T09:04:57.077293+0000","modified":"2024-09-14T09:09:01.883709+0000","last_up_change":"2024-09-14T09:08:42.531335+0000","last_in_change":"2024-09-14T09:07:21.060249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T09:05:11.997673+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T09:05:15.352551+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T09:05:25.840963+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":46,"snap_epoch":182,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ceedb4f-6398-467f-9562-ccf27200860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6825","nonce":3384710819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6827","nonce":3384710819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6831","nonce":3384710819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6829","nonce":3384710819}]},"public_addr":"172.21.15.176:6825/3384710819","cluster_addr":"172.21.15.176:6827/3384710819","heartbeat_back_addr":"172.21.15.176:6831/3384710819","heartbeat_front_addr":"172.21.15.176:6829/3384710819","state":["exists","up"]},{"osd":1,"uuid":"604e2f62-7c13-46be-a97f-029b28c44a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6801","nonce":612012679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6803","nonce":612012679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6807","nonce":612012679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6805","nonce":612012679}]},"public_addr":"172.21.15.176:6801/612012679","cluster_addr":"172.21.15.176:6803/612012679","heartbeat_back_addr":"172.21.15.176:6807/612012679","heartbeat_front_addr":"172.21.15.176:6805/612012679","state":["exists","up"]},{"osd":2,"uuid":"1f65589e-8b0a-4033-94c0-4ab66531db2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6809","nonce":4167379279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6811","nonce":4167379279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6815","nonce":4167379279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6813","nonce":4167379279}]},"public_addr":"172.21.15.176:6809/4167379279","cluster_addr":"172.21.15.176:6811/4167379279","heartbeat_back_addr":"172.21.15.176:6815/4167379279","heartbeat_front_addr":"172.21.15.176:6813/4167379279","state":["exists","up"]},{"osd":3,"uuid":"209d4f65-79f9-4a7d-9d23-832a4209507f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6817","nonce":1658209015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6819","nonce":1658209015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6823","nonce":1658209015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6821","nonce":1658209015}]},"public_addr":"172.21.15.176:6817/1658209015","cluster_addr":"172.21.15.176:6819/1658209015","heartbeat_back_addr":"172.21.15.176:6823/1658209015","heartbeat_front_addr":"172.21.15.176:6821/1658209015","state":["exists","up"]},{"osd":4,"uuid":"8f9a8c9b-282e-4523-9e62-5824f4aba401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":134,"up_from":186,"up_thru":189,"down_at":135,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":700723532},{"type":"v1","addr":"172.21.15.183:6809","nonce":700723532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":700723532},{"type":"v1","addr":"172.21.15.183:6811","nonce":700723532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":700723532},{"type":"v1","addr":"172.21.15.183:6815","nonce":700723532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":700723532},{"type":"v1","addr":"172.21.15.183:6813","nonce":700723532}]},"public_addr":"172.21.15.183:6809/700723532","cluster_addr":"172.21.15.183:6811/700723532","heartbeat_back_addr":"172.21.15.183:6815/700723532","heartbeat_front_addr":"172.21.15.183:6813/700723532","state":["exists","up"]},{"osd":5,"uuid":"1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6825","nonce":1711292386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6827","nonce":1711292386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6831","nonce":1711292386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6829","nonce":1711292386}]},"public_addr":"172.21.15.183:6825/1711292386","cluster_addr":"172.21.15.183:6827/1711292386","heartbeat_back_addr":"172.21.15.183:6831/1711292386","heartbeat_front_addr":"172.21.15.183:6829/1711292386","state":["exists","up"]},{"osd":6,"uuid":"59d9259d-b9dd-4326-8ab9-9d599a2bf57b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6817","nonce":2721245607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6819","nonce":2721245607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6823","nonce":2721245607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6821","nonce":2721245607}]},"public_addr":"172.21.15.183:6817/2721245607","cluster_addr":"172.21.15.183:6819/2721245607","heartbeat_back_addr":"172.21.15.183:6823/2721245607","heartbeat_front_addr":"172.21.15.183:6821/2721245607","state":["exists","up"]},{"osd":7,"uuid":"e2f27d42-786b-449b-a2fb-9dd28f4e9788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6801","nonce":2122823465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6803","nonce":2122823465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6807","nonce":2122823465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6805","nonce":2122823465}]},"public_addr":"172.21.15.183:6801/2122823465","cluster_addr":"172.21.15.183:6803/2122823465","heartbeat_back_addr":"172.21.15.183:6807/2122823465","heartbeat_front_addr":"172.21.15.183:6805/2122823465","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.411789+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:05.801873+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.206725+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:05.882180+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.013864+0000","dead_epoch":135},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.652975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.741359+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:05.873675+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.0","osds":[4,2]},{"pgid":"3.4","osds":[1,7]}],"pg_upmap_items":[{"pgid":"3.9","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":0}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T09:09:02.573 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-14T09:09:02.573 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-14T09:09:02.573 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-14T09:09:02.574 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-14T09:09:02.574 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-14T09:09:02.575 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-14T09:09:02.575 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-14T09:09:02.575 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-14T09:09:02.822 INFO:teuthology.orchestra.run.smithi176.stdout:64424509491 2024-09-14T09:09:02.822 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T09:09:02.832 INFO:teuthology.orchestra.run.smithi176.stdout:64424509491 2024-09-14T09:09:02.832 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T09:09:02.838 INFO:teuthology.orchestra.run.smithi176.stdout:64424509492 2024-09-14T09:09:02.839 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T09:09:02.854 INFO:teuthology.orchestra.run.smithi176.stdout:64424509494 2024-09-14T09:09:02.855 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T09:09:02.855 INFO:teuthology.orchestra.run.smithi176.stdout:60129542195 2024-09-14T09:09:02.856 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T09:09:02.859 INFO:teuthology.orchestra.run.smithi176.stdout:798863917062 2024-09-14T09:09:02.859 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T09:09:02.861 INFO:teuthology.orchestra.run.smithi176.stdout:60129542196 2024-09-14T09:09:02.861 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T09:09:02.865 INFO:teuthology.orchestra.run.smithi176.stdout:60129542194 2024-09-14T09:09:02.865 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T09:09:03.337 INFO:teuthology.orchestra.run.smithi176.stdout:64424509489 2024-09-14T09:09:03.362 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509491 got 64424509489 for osd.6 2024-09-14T09:09:03.373 INFO:teuthology.orchestra.run.smithi176.stdout:64424509489 2024-09-14T09:09:03.389 INFO:teuthology.orchestra.run.smithi176.stdout:64424509492 2024-09-14T09:09:03.391 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509491 got 64424509489 for osd.5 2024-09-14T09:09:03.397 INFO:teuthology.orchestra.run.smithi176.stdout:64424509490 2024-09-14T09:09:03.405 INFO:teuthology.orchestra.run.smithi176.stdout:60129542194 2024-09-14T09:09:03.417 INFO:teuthology.orchestra.run.smithi176.stdout:798863917061 2024-09-14T09:09:03.421 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509494 got 64424509492 for osd.0 2024-09-14T09:09:03.427 INFO:teuthology.orchestra.run.smithi176.stdout:60129542195 2024-09-14T09:09:03.429 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509492 got 64424509490 for osd.2 2024-09-14T09:09:03.436 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542195 got 60129542194 for osd.3 2024-09-14T09:09:03.441 INFO:tasks.ceph.ceph_manager.ceph:need seq 798863917062 got 798863917061 for osd.4 2024-09-14T09:09:03.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542196 got 60129542195 for osd.1 2024-09-14T09:09:03.454 INFO:teuthology.orchestra.run.smithi176.stdout:60129542193 2024-09-14T09:09:03.471 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542194 got 60129542193 for osd.7 2024-09-14T09:09:04.363 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T09:09:04.392 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T09:09:04.421 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T09:09:04.430 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T09:09:04.436 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T09:09:04.442 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T09:09:04.449 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T09:09:04.471 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T09:09:04.901 INFO:teuthology.orchestra.run.smithi176.stdout:64424509489 2024-09-14T09:09:04.935 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509491 got 64424509489 for osd.6 2024-09-14T09:09:04.946 INFO:teuthology.orchestra.run.smithi176.stdout:64424509491 2024-09-14T09:09:04.972 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509491 got 64424509491 for osd.5 2024-09-14T09:09:04.972 DEBUG:teuthology.parallel:result is None 2024-09-14T09:09:04.978 INFO:teuthology.orchestra.run.smithi176.stdout:798863917062 2024-09-14T09:09:04.983 INFO:teuthology.orchestra.run.smithi176.stdout:64424509494 2024-09-14T09:09:04.987 INFO:teuthology.orchestra.run.smithi176.stdout:64424509492 2024-09-14T09:09:05.002 INFO:tasks.ceph.ceph_manager.ceph:need seq 798863917062 got 798863917062 for osd.4 2024-09-14T09:09:05.002 DEBUG:teuthology.parallel:result is None 2024-09-14T09:09:05.002 INFO:teuthology.orchestra.run.smithi176.stdout:60129542196 2024-09-14T09:09:05.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509494 got 64424509494 for osd.0 2024-09-14T09:09:05.007 DEBUG:teuthology.parallel:result is None 2024-09-14T09:09:05.015 INFO:teuthology.orchestra.run.smithi176.stdout:60129542195 2024-09-14T09:09:05.021 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509492 got 64424509492 for osd.2 2024-09-14T09:09:05.021 DEBUG:teuthology.parallel:result is None 2024-09-14T09:09:05.023 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542196 got 60129542196 for osd.1 2024-09-14T09:09:05.023 DEBUG:teuthology.parallel:result is None 2024-09-14T09:09:05.032 INFO:teuthology.orchestra.run.smithi176.stdout:60129542194 2024-09-14T09:09:05.035 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542195 got 60129542195 for osd.3 2024-09-14T09:09:05.035 DEBUG:teuthology.parallel:result is None 2024-09-14T09:09:05.049 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542194 got 60129542194 for osd.7 2024-09-14T09:09:05.049 DEBUG:teuthology.parallel:result is None 2024-09-14T09:09:05.936 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T09:09:06.267 INFO:teuthology.orchestra.run.smithi176.stdout:64424509491 2024-09-14T09:09:06.286 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509491 got 64424509491 for osd.6 2024-09-14T09:09:06.287 DEBUG:teuthology.parallel:result is None 2024-09-14T09:09:06.287 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T09:09:06.287 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:09:06.566 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:09:06.567 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:09:06.582 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-09-14T09:09:06.025817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":285,"num_object_clones":0,"num_object_copies":570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":285,"num_whiteouts":277,"num_read":198278,"num_read_kb":860275,"num_write":25191,"num_write_kb":387556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4050,"num_bytes_recovered":203007312,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22413,"ondisk_log_size":22413,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":422624,"kb_used_data":19480,"kb_used_omap":73,"kb_used_meta":402870,"kb_avail":754552096,"statfs":{"total":773094113280,"available":772661346304,"internally_reserved":0,"allocated":19947520,"data_stored":10780064,"data_compressed":13252,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":74928,"internal_metadata":412539728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.132982"},"pg_stats":[{"pgid":"3.1f","version":"182'1763","reported_seq":7850,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.892276+0000","last_change":"2024-09-14T09:08:34.495405+0000","last_active":"2024-09-14T09:09:01.892276+0000","last_peered":"2024-09-14T09:09:01.892276+0000","last_clean":"2024-09-14T09:09:01.892276+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:09:01.892276+0000","last_undegraded":"2024-09-14T09:09:01.892276+0000","last_fullsized":"2024-09-14T09:09:01.892276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1763,"log_dups_size":0,"ondisk_log_size":1763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:22:02.551926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022047759999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13833,"num_read_kb":24717,"num_write":2684,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'466","reported_seq":5979,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.128471+0000","last_change":"2024-09-14T09:08:34.625492+0000","last_active":"2024-09-14T09:09:02.128471+0000","last_peered":"2024-09-14T09:09:02.128471+0000","last_clean":"2024-09-14T09:09:02.128471+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:09:02.128471+0000","last_undegraded":"2024-09-14T09:09:02.128471+0000","last_fullsized":"2024-09-14T09:09:02.128471+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:01.005568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010401127,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5032,"num_read_kb":31996,"num_write":418,"num_write_kb":13378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"187'676","reported_seq":6606,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.947075+0000","last_change":"2024-09-14T09:08:43.847915+0000","last_active":"2024-09-14T09:09:01.947075+0000","last_peered":"2024-09-14T09:09:01.947075+0000","last_clean":"2024-09-14T09:09:01.947075+0000","last_became_active":"2024-09-14T09:08:43.573212+0000","last_became_peered":"2024-09-14T09:08:43.573212+0000","last_unstale":"2024-09-14T09:09:01.947075+0000","last_undegraded":"2024-09-14T09:09:01.947075+0000","last_fullsized":"2024-09-14T09:09:01.947075+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:26:03.155997+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.147750145,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6669,"num_read_kb":58049,"num_write":727,"num_write_kb":12436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"190'866","reported_seq":5635,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.946852+0000","last_change":"2024-09-14T09:08:46.770708+0000","last_active":"2024-09-14T09:09:01.946852+0000","last_peered":"2024-09-14T09:09:01.946852+0000","last_clean":"2024-09-14T09:09:01.946852+0000","last_became_active":"2024-09-14T09:08:46.599475+0000","last_became_peered":"2024-09-14T09:08:46.599475+0000","last_unstale":"2024-09-14T09:09:01.946852+0000","last_undegraded":"2024-09-14T09:09:01.946852+0000","last_fullsized":"2024-09-14T09:09:01.946852+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:57.157599+0000","scrub_duration":0,"objects_trimmed":60,"snaptrim_duration":0.168855903,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7679,"num_read_kb":27474,"num_write":1043,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'742","reported_seq":5707,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.891412+0000","last_change":"2024-09-14T09:08:34.491862+0000","last_active":"2024-09-14T09:09:01.891412+0000","last_peered":"2024-09-14T09:09:01.891412+0000","last_clean":"2024-09-14T09:09:01.891412+0000","last_became_active":"2024-09-14T09:07:06.842337+0000","last_became_peered":"2024-09-14T09:07:06.842337+0000","last_unstale":"2024-09-14T09:09:01.891412+0000","last_undegraded":"2024-09-14T09:09:01.891412+0000","last_fullsized":"2024-09-14T09:09:01.891412+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:23:25.823078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023564419999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6703,"num_read_kb":24544,"num_write":874,"num_write_kb":16463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":14145319,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"187'724","reported_seq":5502,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.895892+0000","last_change":"2024-09-14T09:08:43.771622+0000","last_active":"2024-09-14T09:09:01.895892+0000","last_peered":"2024-09-14T09:09:01.895892+0000","last_clean":"2024-09-14T09:09:01.895892+0000","last_became_active":"2024-09-14T09:08:43.556338+0000","last_became_peered":"2024-09-14T09:08:43.556338+0000","last_unstale":"2024-09-14T09:09:01.895892+0000","last_undegraded":"2024-09-14T09:09:01.895892+0000","last_fullsized":"2024-09-14T09:09:01.895892+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:41.389225+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.12752904200000001,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6399,"num_read_kb":30687,"num_write":963,"num_write_kb":14233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"174'375","reported_seq":4390,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.127563+0000","last_change":"2024-09-14T09:08:50.629874+0000","last_active":"2024-09-14T09:09:02.127563+0000","last_peered":"2024-09-14T09:09:02.127563+0000","last_clean":"2024-09-14T09:09:02.127563+0000","last_became_active":"2024-09-14T09:08:50.627814+0000","last_became_peered":"2024-09-14T09:08:50.627814+0000","last_unstale":"2024-09-14T09:09:02.127563+0000","last_undegraded":"2024-09-14T09:09:02.127563+0000","last_fullsized":"2024-09-14T09:09:02.127563+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:14:43.966792+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.025218744000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3327,"num_read_kb":10838,"num_write":357,"num_write_kb":6673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":3727781,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"182'624","reported_seq":6370,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.127682+0000","last_change":"2024-09-14T09:08:34.622411+0000","last_active":"2024-09-14T09:09:02.127682+0000","last_peered":"2024-09-14T09:09:02.127682+0000","last_clean":"2024-09-14T09:09:02.127682+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:09:02.127682+0000","last_undegraded":"2024-09-14T09:09:02.127682+0000","last_fullsized":"2024-09-14T09:09:02.127682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:55:38.962684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080911500000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7486,"num_read_kb":17479,"num_write":535,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"182'910","reported_seq":7819,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.892469+0000","last_change":"2024-09-14T09:08:34.496061+0000","last_active":"2024-09-14T09:09:01.892469+0000","last_peered":"2024-09-14T09:09:01.892469+0000","last_clean":"2024-09-14T09:09:01.892469+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:09:01.892469+0000","last_undegraded":"2024-09-14T09:09:01.892469+0000","last_fullsized":"2024-09-14T09:09:01.892469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:02:38.220394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026116440000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10070,"num_read_kb":35479,"num_write":1129,"num_write_kb":10795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"179'629","reported_seq":3942,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.946998+0000","last_change":"2024-09-14T09:08:34.493205+0000","last_active":"2024-09-14T09:09:01.946998+0000","last_peered":"2024-09-14T09:09:01.946998+0000","last_clean":"2024-09-14T09:09:01.946998+0000","last_became_active":"2024-09-14T09:07:25.105674+0000","last_became_peered":"2024-09-14T09:07:25.105674+0000","last_unstale":"2024-09-14T09:09:01.946998+0000","last_undegraded":"2024-09-14T09:09:01.946998+0000","last_fullsized":"2024-09-14T09:09:01.946998+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:20:01.627411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001781447,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4726,"num_read_kb":15599,"num_write":700,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":8467448,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"181'1136","reported_seq":5383,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.134390+0000","last_change":"2024-09-14T09:08:34.507525+0000","last_active":"2024-09-14T09:09:02.134390+0000","last_peered":"2024-09-14T09:09:02.134390+0000","last_clean":"2024-09-14T09:09:02.134390+0000","last_became_active":"2024-09-14T09:07:27.854998+0000","last_became_peered":"2024-09-14T09:07:27.854998+0000","last_unstale":"2024-09-14T09:09:02.134390+0000","last_undegraded":"2024-09-14T09:09:02.134390+0000","last_fullsized":"2024-09-14T09:09:02.134390+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:11:04.062027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001704445,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8595,"num_read_kb":26590,"num_write":1602,"num_write_kb":14717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":4170600,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"187'670","reported_seq":3449,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.896029+0000","last_change":"2024-09-14T09:08:43.840463+0000","last_active":"2024-09-14T09:09:01.896029+0000","last_peered":"2024-09-14T09:09:01.896029+0000","last_clean":"2024-09-14T09:09:01.896029+0000","last_became_active":"2024-09-14T09:08:43.569123+0000","last_became_peered":"2024-09-14T09:08:43.569123+0000","last_unstale":"2024-09-14T09:09:01.896029+0000","last_undegraded":"2024-09-14T09:09:01.896029+0000","last_fullsized":"2024-09-14T09:09:01.896029+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:18.861184+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.15505904000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3583,"num_read_kb":9199,"num_write":691,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":2686146,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":397,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.133359+0000","last_change":"2024-09-14T09:07:21.857426+0000","last_active":"2024-09-14T09:09:02.133359+0000","last_peered":"2024-09-14T09:09:02.133359+0000","last_clean":"2024-09-14T09:09:02.133359+0000","last_became_active":"2024-09-14T09:07:21.856579+0000","last_became_peered":"2024-09-14T09:07:21.856579+0000","last_unstale":"2024-09-14T09:09:02.133359+0000","last_undegraded":"2024-09-14T09:09:02.133359+0000","last_fullsized":"2024-09-14T09:09:02.133359+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:56.694930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"182'791","reported_seq":6687,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.891358+0000","last_change":"2024-09-14T09:08:34.494518+0000","last_active":"2024-09-14T09:09:01.891358+0000","last_peered":"2024-09-14T09:09:01.891358+0000","last_clean":"2024-09-14T09:09:01.891358+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:09:01.891358+0000","last_undegraded":"2024-09-14T09:09:01.891358+0000","last_fullsized":"2024-09-14T09:09:01.891358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:58.731530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016873319999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8676,"num_read_kb":45389,"num_write":707,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":350,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.128364+0000","last_change":"2024-09-14T09:08:51.290129+0000","last_active":"2024-09-14T09:09:02.128364+0000","last_peered":"2024-09-14T09:09:02.128364+0000","last_clean":"2024-09-14T09:09:02.128364+0000","last_became_active":"2024-09-14T09:08:51.289551+0000","last_became_peered":"2024-09-14T09:08:51.289551+0000","last_unstale":"2024-09-14T09:09:02.128364+0000","last_undegraded":"2024-09-14T09:09:02.128364+0000","last_fullsized":"2024-09-14T09:09:02.128364+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:52:01.486956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"182'1187","reported_seq":7252,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.128602+0000","last_change":"2024-09-14T09:08:34.626223+0000","last_active":"2024-09-14T09:09:02.128602+0000","last_peered":"2024-09-14T09:09:02.128602+0000","last_clean":"2024-09-14T09:09:02.128602+0000","last_became_active":"2024-09-14T09:07:26.102666+0000","last_became_peered":"2024-09-14T09:07:26.102666+0000","last_unstale":"2024-09-14T09:09:02.128602+0000","last_undegraded":"2024-09-14T09:09:02.128602+0000","last_fullsized":"2024-09-14T09:09:02.128602+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:03:41.035208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011116867000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10102,"num_read_kb":27247,"num_write":1623,"num_write_kb":13456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":8334712,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":194,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.134185+0000","last_change":"2024-09-14T09:07:21.863290+0000","last_active":"2024-09-14T09:09:02.134185+0000","last_peered":"2024-09-14T09:09:02.134185+0000","last_clean":"2024-09-14T09:09:02.134185+0000","last_became_active":"2024-09-14T09:07:21.862352+0000","last_became_peered":"2024-09-14T09:07:21.862352+0000","last_unstale":"2024-09-14T09:09:02.134185+0000","last_undegraded":"2024-09-14T09:09:02.134185+0000","last_fullsized":"2024-09-14T09:09:02.134185+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:48:30.332891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"179'922","reported_seq":5879,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.128094+0000","last_change":"2024-09-14T09:08:50.627160+0000","last_active":"2024-09-14T09:09:02.128094+0000","last_peered":"2024-09-14T09:09:02.128094+0000","last_clean":"2024-09-14T09:09:02.128094+0000","last_became_active":"2024-09-14T09:08:50.625357+0000","last_became_peered":"2024-09-14T09:08:50.625357+0000","last_unstale":"2024-09-14T09:09:02.128094+0000","last_undegraded":"2024-09-14T09:09:02.128094+0000","last_fullsized":"2024-09-14T09:09:02.128094+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:18:31.289028+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0021152969999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7783,"num_read_kb":22826,"num_write":1134,"num_write_kb":9259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":8151041,"num_keys_recovered":91,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":350,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.128357+0000","last_change":"2024-09-14T09:08:50.628307+0000","last_active":"2024-09-14T09:09:02.128357+0000","last_peered":"2024-09-14T09:09:02.128357+0000","last_clean":"2024-09-14T09:09:02.128357+0000","last_became_active":"2024-09-14T09:08:50.627971+0000","last_became_peered":"2024-09-14T09:08:50.627971+0000","last_unstale":"2024-09-14T09:09:02.128357+0000","last_undegraded":"2024-09-14T09:09:02.128357+0000","last_fullsized":"2024-09-14T09:09:02.128357+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:57.655435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"182'688","reported_seq":2914,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.947150+0000","last_change":"2024-09-14T09:08:34.495006+0000","last_active":"2024-09-14T09:09:01.947150+0000","last_peered":"2024-09-14T09:09:01.947150+0000","last_clean":"2024-09-14T09:09:01.947150+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:09:01.947150+0000","last_undegraded":"2024-09-14T09:09:01.947150+0000","last_fullsized":"2024-09-14T09:09:01.947150+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:25:08.203494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030472329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3594,"num_read_kb":16894,"num_write":791,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"190'884","reported_seq":6708,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.896262+0000","last_change":"2024-09-14T09:08:46.793435+0000","last_active":"2024-09-14T09:09:01.896262+0000","last_peered":"2024-09-14T09:09:01.896262+0000","last_clean":"2024-09-14T09:09:01.896262+0000","last_became_active":"2024-09-14T09:08:46.597293+0000","last_became_peered":"2024-09-14T09:08:46.597293+0000","last_unstale":"2024-09-14T09:09:01.896262+0000","last_undegraded":"2024-09-14T09:09:01.896262+0000","last_fullsized":"2024-09-14T09:09:01.896262+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:48:54.046569+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.19293607300000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8248,"num_read_kb":26883,"num_write":890,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":390,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.946760+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:09:01.946760+0000","last_peered":"2024-09-14T09:09:01.946760+0000","last_clean":"2024-09-14T09:09:01.946760+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:09:01.946760+0000","last_undegraded":"2024-09-14T09:09:01.946760+0000","last_fullsized":"2024-09-14T09:09:01.946760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"180'399","reported_seq":2529,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.127502+0000","last_change":"2024-09-14T09:08:50.625390+0000","last_active":"2024-09-14T09:09:02.127502+0000","last_peered":"2024-09-14T09:09:02.127502+0000","last_clean":"2024-09-14T09:09:02.127502+0000","last_became_active":"2024-09-14T09:08:50.622578+0000","last_became_peered":"2024-09-14T09:08:50.622578+0000","last_unstale":"2024-09-14T09:09:02.127502+0000","last_undegraded":"2024-09-14T09:09:02.127502+0000","last_fullsized":"2024-09-14T09:09:02.127502+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:50:11.634631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00245612,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1873,"num_read_kb":11982,"num_write":352,"num_write_kb":5539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":387,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.890033+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:09:01.890033+0000","last_peered":"2024-09-14T09:09:01.890033+0000","last_clean":"2024-09-14T09:09:01.890033+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:09:01.890033+0000","last_undegraded":"2024-09-14T09:09:01.890033+0000","last_fullsized":"2024-09-14T09:09:01.890033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"187'460","reported_seq":5073,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.891197+0000","last_change":"2024-09-14T09:08:43.789977+0000","last_active":"2024-09-14T09:09:01.891197+0000","last_peered":"2024-09-14T09:09:01.891197+0000","last_clean":"2024-09-14T09:09:01.891197+0000","last_became_active":"2024-09-14T09:08:43.568923+0000","last_became_peered":"2024-09-14T09:08:43.568923+0000","last_unstale":"2024-09-14T09:09:01.891197+0000","last_undegraded":"2024-09-14T09:09:01.891197+0000","last_fullsized":"2024-09-14T09:09:01.891197+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:45:29.077592+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.165000182,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3905,"num_read_kb":25149,"num_write":375,"num_write_kb":14483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":46,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.895756+0000","last_change":"2024-09-14T09:08:44.554373+0000","last_active":"2024-09-14T09:09:01.895756+0000","last_peered":"2024-09-14T09:09:01.895756+0000","last_clean":"2024-09-14T09:09:01.895756+0000","last_became_active":"2024-09-14T09:08:44.553643+0000","last_became_peered":"2024-09-14T09:08:44.553643+0000","last_unstale":"2024-09-14T09:09:01.895756+0000","last_undegraded":"2024-09-14T09:09:01.895756+0000","last_fullsized":"2024-09-14T09:09:01.895756+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:32:52.900856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"181'433","reported_seq":2974,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.127901+0000","last_change":"2024-09-14T09:08:34.620964+0000","last_active":"2024-09-14T09:09:02.127901+0000","last_peered":"2024-09-14T09:09:02.127901+0000","last_clean":"2024-09-14T09:09:02.127901+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:09:02.127901+0000","last_undegraded":"2024-09-14T09:09:02.127901+0000","last_fullsized":"2024-09-14T09:09:02.127901+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:34:59.882295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065530470000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2291,"num_read_kb":11209,"num_write":432,"num_write_kb":6866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":118,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.946781+0000","last_change":"2024-09-14T09:08:06.210516+0000","last_active":"2024-09-14T09:09:01.946781+0000","last_peered":"2024-09-14T09:09:01.946781+0000","last_clean":"2024-09-14T09:09:01.946781+0000","last_became_active":"2024-09-14T09:08:06.205108+0000","last_became_peered":"2024-09-14T09:08:06.205108+0000","last_unstale":"2024-09-14T09:09:01.946781+0000","last_undegraded":"2024-09-14T09:09:01.946781+0000","last_fullsized":"2024-09-14T09:09:01.946781+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:22:05.543865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":200,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.133971+0000","last_change":"2024-09-14T09:07:21.876180+0000","last_active":"2024-09-14T09:09:02.133971+0000","last_peered":"2024-09-14T09:09:02.133971+0000","last_clean":"2024-09-14T09:09:02.133971+0000","last_became_active":"2024-09-14T09:07:21.860117+0000","last_became_peered":"2024-09-14T09:07:21.860117+0000","last_unstale":"2024-09-14T09:09:02.133971+0000","last_undegraded":"2024-09-14T09:09:02.133971+0000","last_fullsized":"2024-09-14T09:09:02.133971+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:58:36.142971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"178'682","reported_seq":7033,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.127799+0000","last_change":"2024-09-14T09:08:34.623351+0000","last_active":"2024-09-14T09:09:02.127799+0000","last_peered":"2024-09-14T09:09:02.127799+0000","last_clean":"2024-09-14T09:09:02.127799+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:09:02.127799+0000","last_undegraded":"2024-09-14T09:09:02.127799+0000","last_fullsized":"2024-09-14T09:09:02.127799+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:48.184932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089062680000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7688,"num_read_kb":41730,"num_write":748,"num_write_kb":19769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"181'469","reported_seq":3204,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.133102+0000","last_change":"2024-09-14T09:08:35.139769+0000","last_active":"2024-09-14T09:09:02.133102+0000","last_peered":"2024-09-14T09:09:02.133102+0000","last_clean":"2024-09-14T09:09:02.133102+0000","last_became_active":"2024-09-14T09:07:06.851020+0000","last_became_peered":"2024-09-14T09:07:06.851020+0000","last_unstale":"2024-09-14T09:09:02.133102+0000","last_undegraded":"2024-09-14T09:09:02.133102+0000","last_fullsized":"2024-09-14T09:09:02.133102+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:19:55.337692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154644,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2080,"num_read_kb":17783,"num_write":389,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":12315549,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"187'749","reported_seq":5078,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.133108+0000","last_change":"2024-09-14T09:08:43.990954+0000","last_active":"2024-09-14T09:09:02.133108+0000","last_peered":"2024-09-14T09:09:02.133108+0000","last_clean":"2024-09-14T09:09:02.133108+0000","last_became_active":"2024-09-14T09:08:43.580245+0000","last_became_peered":"2024-09-14T09:08:43.580245+0000","last_unstale":"2024-09-14T09:09:02.133108+0000","last_undegraded":"2024-09-14T09:09:02.133108+0000","last_fullsized":"2024-09-14T09:09:02.133108+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:04.798520+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.116960367,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5499,"num_read_kb":39102,"num_write":883,"num_write_kb":19962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"181'544","reported_seq":9821,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.127932+0000","last_change":"2024-09-14T09:08:34.624404+0000","last_active":"2024-09-14T09:09:02.127932+0000","last_peered":"2024-09-14T09:09:02.127932+0000","last_clean":"2024-09-14T09:09:02.127932+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:09:02.127932+0000","last_undegraded":"2024-09-14T09:09:02.127932+0000","last_fullsized":"2024-09-14T09:09:02.127932+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:54:24.811934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097700640000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10396,"num_read_kb":61123,"num_write":490,"num_write_kb":11301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"180'493","reported_seq":3386,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.946864+0000","last_change":"2024-09-14T09:08:34.496008+0000","last_active":"2024-09-14T09:09:01.946864+0000","last_peered":"2024-09-14T09:09:01.946864+0000","last_clean":"2024-09-14T09:09:01.946864+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:09:01.946864+0000","last_undegraded":"2024-09-14T09:09:01.946864+0000","last_fullsized":"2024-09-14T09:09:01.946864+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:11.774708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00388507,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3266,"num_read_kb":13215,"num_write":426,"num_write_kb":10574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"179'501","reported_seq":5831,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.134126+0000","last_change":"2024-09-14T09:08:34.506908+0000","last_active":"2024-09-14T09:09:02.134126+0000","last_peered":"2024-09-14T09:09:02.134126+0000","last_clean":"2024-09-14T09:09:02.134126+0000","last_became_active":"2024-09-14T09:07:26.853961+0000","last_became_peered":"2024-09-14T09:07:26.853961+0000","last_unstale":"2024-09-14T09:09:02.134126+0000","last_undegraded":"2024-09-14T09:09:02.134126+0000","last_fullsized":"2024-09-14T09:09:02.134126+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:54.100061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001753449,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5399,"num_read_kb":32061,"num_write":547,"num_write_kb":21536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":25396220,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"187'488","reported_seq":4028,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.134452+0000","last_change":"2024-09-14T09:08:44.221618+0000","last_active":"2024-09-14T09:09:02.134452+0000","last_peered":"2024-09-14T09:09:02.134452+0000","last_clean":"2024-09-14T09:09:02.134452+0000","last_became_active":"2024-09-14T09:08:43.588233+0000","last_became_peered":"2024-09-14T09:08:43.588233+0000","last_unstale":"2024-09-14T09:09:02.134452+0000","last_undegraded":"2024-09-14T09:09:02.134452+0000","last_fullsized":"2024-09-14T09:09:02.134452+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:10.863128+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.14577486200000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4282,"num_read_kb":17652,"num_write":409,"num_write_kb":7874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":14123394,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"187'586","reported_seq":4281,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.134593+0000","last_change":"2024-09-14T09:08:44.145761+0000","last_active":"2024-09-14T09:09:02.134593+0000","last_peered":"2024-09-14T09:09:02.134593+0000","last_clean":"2024-09-14T09:09:02.134593+0000","last_became_active":"2024-09-14T09:08:43.574566+0000","last_became_peered":"2024-09-14T09:08:43.574566+0000","last_unstale":"2024-09-14T09:09:02.134593+0000","last_undegraded":"2024-09-14T09:09:02.134593+0000","last_fullsized":"2024-09-14T09:09:02.134593+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:30.502086+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.141382223,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3641,"num_read_kb":14849,"num_write":554,"num_write_kb":12244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":11864981,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'506","reported_seq":6236,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.133073+0000","last_change":"2024-09-14T09:08:35.140559+0000","last_active":"2024-09-14T09:09:02.133073+0000","last_peered":"2024-09-14T09:09:02.133073+0000","last_clean":"2024-09-14T09:09:02.133073+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:09:02.133073+0000","last_undegraded":"2024-09-14T09:09:02.133073+0000","last_fullsized":"2024-09-14T09:09:02.133073+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:31:57.906202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018540480000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5422,"num_read_kb":60065,"num_write":521,"num_write_kb":20523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"187'649","reported_seq":6138,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.134018+0000","last_change":"2024-09-14T09:08:43.902203+0000","last_active":"2024-09-14T09:09:02.134018+0000","last_peered":"2024-09-14T09:09:02.134018+0000","last_clean":"2024-09-14T09:09:02.134018+0000","last_became_active":"2024-09-14T09:08:43.577551+0000","last_became_peered":"2024-09-14T09:08:43.577551+0000","last_unstale":"2024-09-14T09:09:02.134018+0000","last_undegraded":"2024-09-14T09:09:02.134018+0000","last_fullsized":"2024-09-14T09:09:02.134018+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:43:37.090068+0000","scrub_duration":0,"objects_trimmed":64,"snaptrim_duration":0.121356326,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6082,"num_read_kb":13595,"num_write":653,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":7405746,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"182'790","reported_seq":6120,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.891244+0000","last_change":"2024-09-14T09:08:34.493569+0000","last_active":"2024-09-14T09:09:01.891244+0000","last_peered":"2024-09-14T09:09:01.891244+0000","last_clean":"2024-09-14T09:09:01.891244+0000","last_became_active":"2024-09-14T09:07:26.101586+0000","last_became_peered":"2024-09-14T09:07:26.101586+0000","last_unstale":"2024-09-14T09:09:01.891244+0000","last_undegraded":"2024-09-14T09:09:01.891244+0000","last_fullsized":"2024-09-14T09:09:01.891244+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:11:20.433650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040594799999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7044,"num_read_kb":19519,"num_write":992,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":10260744,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"187'536","reported_seq":6566,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.134058+0000","last_change":"2024-09-14T09:08:44.016919+0000","last_active":"2024-09-14T09:09:02.134058+0000","last_peered":"2024-09-14T09:09:02.134058+0000","last_clean":"2024-09-14T09:09:02.134058+0000","last_became_active":"2024-09-14T09:08:43.579997+0000","last_became_peered":"2024-09-14T09:08:43.579997+0000","last_unstale":"2024-09-14T09:09:02.134058+0000","last_undegraded":"2024-09-14T09:09:02.134058+0000","last_fullsized":"2024-09-14T09:09:02.134058+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:24:16.807890+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.10056556899999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6839,"num_read_kb":29296,"num_write":449,"num_write_kb":10682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":24080444,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":281,"num_object_clones":0,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":277,"num_read":198212,"num_read_kb":860220,"num_write":25088,"num_write_kb":385926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4045,"num_bytes_recovered":201826576,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},"log_size":22338,"ondisk_log_size":22338,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20292,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509491,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29464,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342376,"statfs":{"total":96636764160,"available":96606593024,"internally_reserved":0,"allocated":2482176,"data_stored":1201396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509491,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78036,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293804,"statfs":{"total":96636764160,"available":96556855296,"internally_reserved":0,"allocated":2486272,"data_stored":1201415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509492,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47504,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324336,"statfs":{"total":96636764160,"available":96588120064,"internally_reserved":0,"allocated":2482176,"data_stored":1201396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509494,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64204,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307636,"statfs":{"total":96636764160,"available":96571019264,"internally_reserved":0,"allocated":2539520,"data_stored":1791764,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542194,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51696,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320144,"statfs":{"total":96636764160,"available":96583827456,"internally_reserved":0,"allocated":2510848,"data_stored":1785806,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":186,"seq":798863917062,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35920,"kb_used_data":2428,"kb_used_omap":62,"kb_used_meta":33409,"kb_avail":94335920,"statfs":{"total":96636764160,"available":96599982080,"internally_reserved":0,"allocated":2486272,"data_stored":1201415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":63771,"internal_metadata":34211557},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542195,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59972,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311868,"statfs":{"total":96636764160,"available":96575352832,"internally_reserved":0,"allocated":2469888,"data_stored":1195438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542196,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55828,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316012,"statfs":{"total":96636764160,"available":96579596288,"internally_reserved":0,"allocated":2490368,"data_stored":1201434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:09:06.584 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:09:06.879 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:09:06.880 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:09:06.897 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-14T09:09:06.861379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":285,"num_object_clones":0,"num_object_copies":570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":285,"num_whiteouts":277,"num_read":198278,"num_read_kb":860275,"num_write":25191,"num_write_kb":387556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4050,"num_bytes_recovered":203007312,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22413,"ondisk_log_size":22413,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":422748,"kb_used_data":19612,"kb_used_omap":73,"kb_used_meta":402870,"kb_avail":754551972,"statfs":{"total":773094113280,"available":772661219328,"internally_reserved":0,"allocated":20082688,"data_stored":10846172,"data_compressed":13252,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":74928,"internal_metadata":412539728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.836619"},"pg_stats":[{"pgid":"3.1f","version":"182'1763","reported_seq":7850,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.892276+0000","last_change":"2024-09-14T09:08:34.495405+0000","last_active":"2024-09-14T09:09:01.892276+0000","last_peered":"2024-09-14T09:09:01.892276+0000","last_clean":"2024-09-14T09:09:01.892276+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:09:01.892276+0000","last_undegraded":"2024-09-14T09:09:01.892276+0000","last_fullsized":"2024-09-14T09:09:01.892276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1763,"log_dups_size":0,"ondisk_log_size":1763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:22:02.551926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022047759999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13833,"num_read_kb":24717,"num_write":2684,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'466","reported_seq":5985,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034518+0000","last_change":"2024-09-14T09:08:34.625492+0000","last_active":"2024-09-14T09:09:06.034518+0000","last_peered":"2024-09-14T09:09:06.034518+0000","last_clean":"2024-09-14T09:09:06.034518+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:09:06.034518+0000","last_undegraded":"2024-09-14T09:09:06.034518+0000","last_fullsized":"2024-09-14T09:09:06.034518+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:01.005568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010401127,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5032,"num_read_kb":31996,"num_write":418,"num_write_kb":13378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"187'676","reported_seq":6606,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.947075+0000","last_change":"2024-09-14T09:08:43.847915+0000","last_active":"2024-09-14T09:09:01.947075+0000","last_peered":"2024-09-14T09:09:01.947075+0000","last_clean":"2024-09-14T09:09:01.947075+0000","last_became_active":"2024-09-14T09:08:43.573212+0000","last_became_peered":"2024-09-14T09:08:43.573212+0000","last_unstale":"2024-09-14T09:09:01.947075+0000","last_undegraded":"2024-09-14T09:09:01.947075+0000","last_fullsized":"2024-09-14T09:09:01.947075+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:26:03.155997+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.147750145,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6669,"num_read_kb":58049,"num_write":727,"num_write_kb":12436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"190'866","reported_seq":5635,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.946852+0000","last_change":"2024-09-14T09:08:46.770708+0000","last_active":"2024-09-14T09:09:01.946852+0000","last_peered":"2024-09-14T09:09:01.946852+0000","last_clean":"2024-09-14T09:09:01.946852+0000","last_became_active":"2024-09-14T09:08:46.599475+0000","last_became_peered":"2024-09-14T09:08:46.599475+0000","last_unstale":"2024-09-14T09:09:01.946852+0000","last_undegraded":"2024-09-14T09:09:01.946852+0000","last_fullsized":"2024-09-14T09:09:01.946852+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:57.157599+0000","scrub_duration":0,"objects_trimmed":60,"snaptrim_duration":0.168855903,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7679,"num_read_kb":27474,"num_write":1043,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'742","reported_seq":5707,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.891412+0000","last_change":"2024-09-14T09:08:34.491862+0000","last_active":"2024-09-14T09:09:01.891412+0000","last_peered":"2024-09-14T09:09:01.891412+0000","last_clean":"2024-09-14T09:09:01.891412+0000","last_became_active":"2024-09-14T09:07:06.842337+0000","last_became_peered":"2024-09-14T09:07:06.842337+0000","last_unstale":"2024-09-14T09:09:01.891412+0000","last_undegraded":"2024-09-14T09:09:01.891412+0000","last_fullsized":"2024-09-14T09:09:01.891412+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:23:25.823078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023564419999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6703,"num_read_kb":24544,"num_write":874,"num_write_kb":16463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":14145319,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"187'724","reported_seq":5508,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937332+0000","last_change":"2024-09-14T09:08:43.771622+0000","last_active":"2024-09-14T09:09:05.937332+0000","last_peered":"2024-09-14T09:09:05.937332+0000","last_clean":"2024-09-14T09:09:05.937332+0000","last_became_active":"2024-09-14T09:08:43.556338+0000","last_became_peered":"2024-09-14T09:08:43.556338+0000","last_unstale":"2024-09-14T09:09:05.937332+0000","last_undegraded":"2024-09-14T09:09:05.937332+0000","last_fullsized":"2024-09-14T09:09:05.937332+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:41.389225+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.12752904200000001,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6399,"num_read_kb":30687,"num_write":963,"num_write_kb":14233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"174'375","reported_seq":4396,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937548+0000","last_change":"2024-09-14T09:08:50.629874+0000","last_active":"2024-09-14T09:09:05.937548+0000","last_peered":"2024-09-14T09:09:05.937548+0000","last_clean":"2024-09-14T09:09:05.937548+0000","last_became_active":"2024-09-14T09:08:50.627814+0000","last_became_peered":"2024-09-14T09:08:50.627814+0000","last_unstale":"2024-09-14T09:09:05.937548+0000","last_undegraded":"2024-09-14T09:09:05.937548+0000","last_fullsized":"2024-09-14T09:09:05.937548+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:14:43.966792+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.025218744000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3327,"num_read_kb":10838,"num_write":357,"num_write_kb":6673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":3727781,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"182'624","reported_seq":6376,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033940+0000","last_change":"2024-09-14T09:08:34.622411+0000","last_active":"2024-09-14T09:09:06.033940+0000","last_peered":"2024-09-14T09:09:06.033940+0000","last_clean":"2024-09-14T09:09:06.033940+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:09:06.033940+0000","last_undegraded":"2024-09-14T09:09:06.033940+0000","last_fullsized":"2024-09-14T09:09:06.033940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:55:38.962684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080911500000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7486,"num_read_kb":17479,"num_write":535,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"182'910","reported_seq":7819,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.892469+0000","last_change":"2024-09-14T09:08:34.496061+0000","last_active":"2024-09-14T09:09:01.892469+0000","last_peered":"2024-09-14T09:09:01.892469+0000","last_clean":"2024-09-14T09:09:01.892469+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:09:01.892469+0000","last_undegraded":"2024-09-14T09:09:01.892469+0000","last_fullsized":"2024-09-14T09:09:01.892469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:02:38.220394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026116440000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10070,"num_read_kb":35479,"num_write":1129,"num_write_kb":10795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"179'629","reported_seq":3942,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.946998+0000","last_change":"2024-09-14T09:08:34.493205+0000","last_active":"2024-09-14T09:09:01.946998+0000","last_peered":"2024-09-14T09:09:01.946998+0000","last_clean":"2024-09-14T09:09:01.946998+0000","last_became_active":"2024-09-14T09:07:25.105674+0000","last_became_peered":"2024-09-14T09:07:25.105674+0000","last_unstale":"2024-09-14T09:09:01.946998+0000","last_undegraded":"2024-09-14T09:09:01.946998+0000","last_fullsized":"2024-09-14T09:09:01.946998+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:20:01.627411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001781447,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4726,"num_read_kb":15599,"num_write":700,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":8467448,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"181'1136","reported_seq":5389,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.951079+0000","last_change":"2024-09-14T09:08:34.507525+0000","last_active":"2024-09-14T09:09:05.951079+0000","last_peered":"2024-09-14T09:09:05.951079+0000","last_clean":"2024-09-14T09:09:05.951079+0000","last_became_active":"2024-09-14T09:07:27.854998+0000","last_became_peered":"2024-09-14T09:07:27.854998+0000","last_unstale":"2024-09-14T09:09:05.951079+0000","last_undegraded":"2024-09-14T09:09:05.951079+0000","last_fullsized":"2024-09-14T09:09:05.951079+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:11:04.062027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001704445,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8595,"num_read_kb":26590,"num_write":1602,"num_write_kb":14717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":4170600,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"187'670","reported_seq":3455,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937564+0000","last_change":"2024-09-14T09:08:43.840463+0000","last_active":"2024-09-14T09:09:05.937564+0000","last_peered":"2024-09-14T09:09:05.937564+0000","last_clean":"2024-09-14T09:09:05.937564+0000","last_became_active":"2024-09-14T09:08:43.569123+0000","last_became_peered":"2024-09-14T09:08:43.569123+0000","last_unstale":"2024-09-14T09:09:05.937564+0000","last_undegraded":"2024-09-14T09:09:05.937564+0000","last_fullsized":"2024-09-14T09:09:05.937564+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:18.861184+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.15505904000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3583,"num_read_kb":9199,"num_write":691,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":2686146,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":397,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.133359+0000","last_change":"2024-09-14T09:07:21.857426+0000","last_active":"2024-09-14T09:09:02.133359+0000","last_peered":"2024-09-14T09:09:02.133359+0000","last_clean":"2024-09-14T09:09:02.133359+0000","last_became_active":"2024-09-14T09:07:21.856579+0000","last_became_peered":"2024-09-14T09:07:21.856579+0000","last_unstale":"2024-09-14T09:09:02.133359+0000","last_undegraded":"2024-09-14T09:09:02.133359+0000","last_fullsized":"2024-09-14T09:09:02.133359+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:56.694930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"182'791","reported_seq":6687,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.891358+0000","last_change":"2024-09-14T09:08:34.494518+0000","last_active":"2024-09-14T09:09:01.891358+0000","last_peered":"2024-09-14T09:09:01.891358+0000","last_clean":"2024-09-14T09:09:01.891358+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:09:01.891358+0000","last_undegraded":"2024-09-14T09:09:01.891358+0000","last_fullsized":"2024-09-14T09:09:01.891358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:58.731530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016873319999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8676,"num_read_kb":45389,"num_write":707,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":356,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937898+0000","last_change":"2024-09-14T09:08:51.290129+0000","last_active":"2024-09-14T09:09:05.937898+0000","last_peered":"2024-09-14T09:09:05.937898+0000","last_clean":"2024-09-14T09:09:05.937898+0000","last_became_active":"2024-09-14T09:08:51.289551+0000","last_became_peered":"2024-09-14T09:08:51.289551+0000","last_unstale":"2024-09-14T09:09:05.937898+0000","last_undegraded":"2024-09-14T09:09:05.937898+0000","last_fullsized":"2024-09-14T09:09:05.937898+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:52:01.486956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"182'1187","reported_seq":7258,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034110+0000","last_change":"2024-09-14T09:08:34.626223+0000","last_active":"2024-09-14T09:09:06.034110+0000","last_peered":"2024-09-14T09:09:06.034110+0000","last_clean":"2024-09-14T09:09:06.034110+0000","last_became_active":"2024-09-14T09:07:26.102666+0000","last_became_peered":"2024-09-14T09:07:26.102666+0000","last_unstale":"2024-09-14T09:09:06.034110+0000","last_undegraded":"2024-09-14T09:09:06.034110+0000","last_fullsized":"2024-09-14T09:09:06.034110+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:03:41.035208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011116867000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10102,"num_read_kb":27247,"num_write":1623,"num_write_kb":13456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":8334712,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":200,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.950988+0000","last_change":"2024-09-14T09:07:21.863290+0000","last_active":"2024-09-14T09:09:05.950988+0000","last_peered":"2024-09-14T09:09:05.950988+0000","last_clean":"2024-09-14T09:09:05.950988+0000","last_became_active":"2024-09-14T09:07:21.862352+0000","last_became_peered":"2024-09-14T09:07:21.862352+0000","last_unstale":"2024-09-14T09:09:05.950988+0000","last_undegraded":"2024-09-14T09:09:05.950988+0000","last_fullsized":"2024-09-14T09:09:05.950988+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:48:30.332891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"179'922","reported_seq":5885,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937691+0000","last_change":"2024-09-14T09:08:50.627160+0000","last_active":"2024-09-14T09:09:05.937691+0000","last_peered":"2024-09-14T09:09:05.937691+0000","last_clean":"2024-09-14T09:09:05.937691+0000","last_became_active":"2024-09-14T09:08:50.625357+0000","last_became_peered":"2024-09-14T09:08:50.625357+0000","last_unstale":"2024-09-14T09:09:05.937691+0000","last_undegraded":"2024-09-14T09:09:05.937691+0000","last_fullsized":"2024-09-14T09:09:05.937691+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:18:31.289028+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0021152969999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7783,"num_read_kb":22826,"num_write":1134,"num_write_kb":9259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":8151041,"num_keys_recovered":91,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":356,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937696+0000","last_change":"2024-09-14T09:08:50.628307+0000","last_active":"2024-09-14T09:09:05.937696+0000","last_peered":"2024-09-14T09:09:05.937696+0000","last_clean":"2024-09-14T09:09:05.937696+0000","last_became_active":"2024-09-14T09:08:50.627971+0000","last_became_peered":"2024-09-14T09:08:50.627971+0000","last_unstale":"2024-09-14T09:09:05.937696+0000","last_undegraded":"2024-09-14T09:09:05.937696+0000","last_fullsized":"2024-09-14T09:09:05.937696+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:57.655435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"182'688","reported_seq":2914,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.947150+0000","last_change":"2024-09-14T09:08:34.495006+0000","last_active":"2024-09-14T09:09:01.947150+0000","last_peered":"2024-09-14T09:09:01.947150+0000","last_clean":"2024-09-14T09:09:01.947150+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:09:01.947150+0000","last_undegraded":"2024-09-14T09:09:01.947150+0000","last_fullsized":"2024-09-14T09:09:01.947150+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:25:08.203494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030472329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3594,"num_read_kb":16894,"num_write":791,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"190'884","reported_seq":6714,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937761+0000","last_change":"2024-09-14T09:08:46.793435+0000","last_active":"2024-09-14T09:09:05.937761+0000","last_peered":"2024-09-14T09:09:05.937761+0000","last_clean":"2024-09-14T09:09:05.937761+0000","last_became_active":"2024-09-14T09:08:46.597293+0000","last_became_peered":"2024-09-14T09:08:46.597293+0000","last_unstale":"2024-09-14T09:09:05.937761+0000","last_undegraded":"2024-09-14T09:09:05.937761+0000","last_fullsized":"2024-09-14T09:09:05.937761+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:48:54.046569+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.19293607300000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8248,"num_read_kb":26883,"num_write":890,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":390,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.946760+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:09:01.946760+0000","last_peered":"2024-09-14T09:09:01.946760+0000","last_clean":"2024-09-14T09:09:01.946760+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:09:01.946760+0000","last_undegraded":"2024-09-14T09:09:01.946760+0000","last_fullsized":"2024-09-14T09:09:01.946760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"180'399","reported_seq":2535,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937522+0000","last_change":"2024-09-14T09:08:50.625390+0000","last_active":"2024-09-14T09:09:05.937522+0000","last_peered":"2024-09-14T09:09:05.937522+0000","last_clean":"2024-09-14T09:09:05.937522+0000","last_became_active":"2024-09-14T09:08:50.622578+0000","last_became_peered":"2024-09-14T09:08:50.622578+0000","last_unstale":"2024-09-14T09:09:05.937522+0000","last_undegraded":"2024-09-14T09:09:05.937522+0000","last_fullsized":"2024-09-14T09:09:05.937522+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:50:11.634631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00245612,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1873,"num_read_kb":11982,"num_write":352,"num_write_kb":5539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":387,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.890033+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:09:01.890033+0000","last_peered":"2024-09-14T09:09:01.890033+0000","last_clean":"2024-09-14T09:09:01.890033+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:09:01.890033+0000","last_undegraded":"2024-09-14T09:09:01.890033+0000","last_fullsized":"2024-09-14T09:09:01.890033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"187'460","reported_seq":5073,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.891197+0000","last_change":"2024-09-14T09:08:43.789977+0000","last_active":"2024-09-14T09:09:01.891197+0000","last_peered":"2024-09-14T09:09:01.891197+0000","last_clean":"2024-09-14T09:09:01.891197+0000","last_became_active":"2024-09-14T09:08:43.568923+0000","last_became_peered":"2024-09-14T09:08:43.568923+0000","last_unstale":"2024-09-14T09:09:01.891197+0000","last_undegraded":"2024-09-14T09:09:01.891197+0000","last_fullsized":"2024-09-14T09:09:01.891197+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:45:29.077592+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.165000182,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3905,"num_read_kb":25149,"num_write":375,"num_write_kb":14483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937319+0000","last_change":"2024-09-14T09:08:44.554373+0000","last_active":"2024-09-14T09:09:05.937319+0000","last_peered":"2024-09-14T09:09:05.937319+0000","last_clean":"2024-09-14T09:09:05.937319+0000","last_became_active":"2024-09-14T09:08:44.553643+0000","last_became_peered":"2024-09-14T09:08:44.553643+0000","last_unstale":"2024-09-14T09:09:05.937319+0000","last_undegraded":"2024-09-14T09:09:05.937319+0000","last_fullsized":"2024-09-14T09:09:05.937319+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:32:52.900856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"181'433","reported_seq":2980,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034173+0000","last_change":"2024-09-14T09:08:34.620964+0000","last_active":"2024-09-14T09:09:06.034173+0000","last_peered":"2024-09-14T09:09:06.034173+0000","last_clean":"2024-09-14T09:09:06.034173+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:09:06.034173+0000","last_undegraded":"2024-09-14T09:09:06.034173+0000","last_fullsized":"2024-09-14T09:09:06.034173+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:34:59.882295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065530470000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2291,"num_read_kb":11209,"num_write":432,"num_write_kb":6866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":118,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.946781+0000","last_change":"2024-09-14T09:08:06.210516+0000","last_active":"2024-09-14T09:09:01.946781+0000","last_peered":"2024-09-14T09:09:01.946781+0000","last_clean":"2024-09-14T09:09:01.946781+0000","last_became_active":"2024-09-14T09:08:06.205108+0000","last_became_peered":"2024-09-14T09:08:06.205108+0000","last_unstale":"2024-09-14T09:09:01.946781+0000","last_undegraded":"2024-09-14T09:09:01.946781+0000","last_fullsized":"2024-09-14T09:09:01.946781+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:22:05.543865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":206,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949404+0000","last_change":"2024-09-14T09:07:21.876180+0000","last_active":"2024-09-14T09:09:05.949404+0000","last_peered":"2024-09-14T09:09:05.949404+0000","last_clean":"2024-09-14T09:09:05.949404+0000","last_became_active":"2024-09-14T09:07:21.860117+0000","last_became_peered":"2024-09-14T09:07:21.860117+0000","last_unstale":"2024-09-14T09:09:05.949404+0000","last_undegraded":"2024-09-14T09:09:05.949404+0000","last_fullsized":"2024-09-14T09:09:05.949404+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:58:36.142971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"178'682","reported_seq":7039,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033947+0000","last_change":"2024-09-14T09:08:34.623351+0000","last_active":"2024-09-14T09:09:06.033947+0000","last_peered":"2024-09-14T09:09:06.033947+0000","last_clean":"2024-09-14T09:09:06.033947+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:09:06.033947+0000","last_undegraded":"2024-09-14T09:09:06.033947+0000","last_fullsized":"2024-09-14T09:09:06.033947+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:48.184932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089062680000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7688,"num_read_kb":41730,"num_write":748,"num_write_kb":19769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"181'469","reported_seq":3204,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.133102+0000","last_change":"2024-09-14T09:08:35.139769+0000","last_active":"2024-09-14T09:09:02.133102+0000","last_peered":"2024-09-14T09:09:02.133102+0000","last_clean":"2024-09-14T09:09:02.133102+0000","last_became_active":"2024-09-14T09:07:06.851020+0000","last_became_peered":"2024-09-14T09:07:06.851020+0000","last_unstale":"2024-09-14T09:09:02.133102+0000","last_undegraded":"2024-09-14T09:09:02.133102+0000","last_fullsized":"2024-09-14T09:09:02.133102+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:19:55.337692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154644,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2080,"num_read_kb":17783,"num_write":389,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":12315549,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"187'749","reported_seq":5078,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.133108+0000","last_change":"2024-09-14T09:08:43.990954+0000","last_active":"2024-09-14T09:09:02.133108+0000","last_peered":"2024-09-14T09:09:02.133108+0000","last_clean":"2024-09-14T09:09:02.133108+0000","last_became_active":"2024-09-14T09:08:43.580245+0000","last_became_peered":"2024-09-14T09:08:43.580245+0000","last_unstale":"2024-09-14T09:09:02.133108+0000","last_undegraded":"2024-09-14T09:09:02.133108+0000","last_fullsized":"2024-09-14T09:09:02.133108+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:04.798520+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.116960367,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5499,"num_read_kb":39102,"num_write":883,"num_write_kb":19962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"181'544","reported_seq":9827,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034065+0000","last_change":"2024-09-14T09:08:34.624404+0000","last_active":"2024-09-14T09:09:06.034065+0000","last_peered":"2024-09-14T09:09:06.034065+0000","last_clean":"2024-09-14T09:09:06.034065+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:09:06.034065+0000","last_undegraded":"2024-09-14T09:09:06.034065+0000","last_fullsized":"2024-09-14T09:09:06.034065+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:54:24.811934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097700640000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10396,"num_read_kb":61123,"num_write":490,"num_write_kb":11301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"180'493","reported_seq":3386,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.946864+0000","last_change":"2024-09-14T09:08:34.496008+0000","last_active":"2024-09-14T09:09:01.946864+0000","last_peered":"2024-09-14T09:09:01.946864+0000","last_clean":"2024-09-14T09:09:01.946864+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:09:01.946864+0000","last_undegraded":"2024-09-14T09:09:01.946864+0000","last_fullsized":"2024-09-14T09:09:01.946864+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:11.774708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00388507,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3266,"num_read_kb":13215,"num_write":426,"num_write_kb":10574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"179'501","reported_seq":5831,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.134126+0000","last_change":"2024-09-14T09:08:34.506908+0000","last_active":"2024-09-14T09:09:02.134126+0000","last_peered":"2024-09-14T09:09:02.134126+0000","last_clean":"2024-09-14T09:09:02.134126+0000","last_became_active":"2024-09-14T09:07:26.853961+0000","last_became_peered":"2024-09-14T09:07:26.853961+0000","last_unstale":"2024-09-14T09:09:02.134126+0000","last_undegraded":"2024-09-14T09:09:02.134126+0000","last_fullsized":"2024-09-14T09:09:02.134126+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:54.100061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001753449,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5399,"num_read_kb":32061,"num_write":547,"num_write_kb":21536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":25396220,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"187'488","reported_seq":4034,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949818+0000","last_change":"2024-09-14T09:08:44.221618+0000","last_active":"2024-09-14T09:09:05.949818+0000","last_peered":"2024-09-14T09:09:05.949818+0000","last_clean":"2024-09-14T09:09:05.949818+0000","last_became_active":"2024-09-14T09:08:43.588233+0000","last_became_peered":"2024-09-14T09:08:43.588233+0000","last_unstale":"2024-09-14T09:09:05.949818+0000","last_undegraded":"2024-09-14T09:09:05.949818+0000","last_fullsized":"2024-09-14T09:09:05.949818+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:10.863128+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.14577486200000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4282,"num_read_kb":17652,"num_write":409,"num_write_kb":7874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":14123394,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"187'586","reported_seq":4287,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.950045+0000","last_change":"2024-09-14T09:08:44.145761+0000","last_active":"2024-09-14T09:09:05.950045+0000","last_peered":"2024-09-14T09:09:05.950045+0000","last_clean":"2024-09-14T09:09:05.950045+0000","last_became_active":"2024-09-14T09:08:43.574566+0000","last_became_peered":"2024-09-14T09:08:43.574566+0000","last_unstale":"2024-09-14T09:09:05.950045+0000","last_undegraded":"2024-09-14T09:09:05.950045+0000","last_fullsized":"2024-09-14T09:09:05.950045+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:30.502086+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.141382223,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3641,"num_read_kb":14849,"num_write":554,"num_write_kb":12244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":11864981,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'506","reported_seq":6236,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.133073+0000","last_change":"2024-09-14T09:08:35.140559+0000","last_active":"2024-09-14T09:09:02.133073+0000","last_peered":"2024-09-14T09:09:02.133073+0000","last_clean":"2024-09-14T09:09:02.133073+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:09:02.133073+0000","last_undegraded":"2024-09-14T09:09:02.133073+0000","last_fullsized":"2024-09-14T09:09:02.133073+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:31:57.906202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018540480000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5422,"num_read_kb":60065,"num_write":521,"num_write_kb":20523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"187'649","reported_seq":6144,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949422+0000","last_change":"2024-09-14T09:08:43.902203+0000","last_active":"2024-09-14T09:09:05.949422+0000","last_peered":"2024-09-14T09:09:05.949422+0000","last_clean":"2024-09-14T09:09:05.949422+0000","last_became_active":"2024-09-14T09:08:43.577551+0000","last_became_peered":"2024-09-14T09:08:43.577551+0000","last_unstale":"2024-09-14T09:09:05.949422+0000","last_undegraded":"2024-09-14T09:09:05.949422+0000","last_fullsized":"2024-09-14T09:09:05.949422+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:43:37.090068+0000","scrub_duration":0,"objects_trimmed":64,"snaptrim_duration":0.121356326,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6082,"num_read_kb":13595,"num_write":653,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":7405746,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"182'790","reported_seq":6120,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.891244+0000","last_change":"2024-09-14T09:08:34.493569+0000","last_active":"2024-09-14T09:09:01.891244+0000","last_peered":"2024-09-14T09:09:01.891244+0000","last_clean":"2024-09-14T09:09:01.891244+0000","last_became_active":"2024-09-14T09:07:26.101586+0000","last_became_peered":"2024-09-14T09:07:26.101586+0000","last_unstale":"2024-09-14T09:09:01.891244+0000","last_undegraded":"2024-09-14T09:09:01.891244+0000","last_fullsized":"2024-09-14T09:09:01.891244+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:11:20.433650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040594799999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7044,"num_read_kb":19519,"num_write":992,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":10260744,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"187'536","reported_seq":6572,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949474+0000","last_change":"2024-09-14T09:08:44.016919+0000","last_active":"2024-09-14T09:09:05.949474+0000","last_peered":"2024-09-14T09:09:05.949474+0000","last_clean":"2024-09-14T09:09:05.949474+0000","last_became_active":"2024-09-14T09:08:43.579997+0000","last_became_peered":"2024-09-14T09:08:43.579997+0000","last_unstale":"2024-09-14T09:09:05.949474+0000","last_undegraded":"2024-09-14T09:09:05.949474+0000","last_fullsized":"2024-09-14T09:09:05.949474+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:24:16.807890+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.10056556899999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6839,"num_read_kb":29296,"num_write":449,"num_write_kb":10682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":24080444,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":281,"num_object_clones":0,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":277,"num_read":198212,"num_read_kb":860220,"num_write":25088,"num_write_kb":385926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4045,"num_bytes_recovered":201826576,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},"log_size":22338,"ondisk_log_size":22338,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20292,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509491,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29464,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342376,"statfs":{"total":96636764160,"available":96606593024,"internally_reserved":0,"allocated":2482176,"data_stored":1201396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509491,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78036,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293804,"statfs":{"total":96636764160,"available":96556855296,"internally_reserved":0,"allocated":2486272,"data_stored":1201415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509492,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47504,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324336,"statfs":{"total":96636764160,"available":96588120064,"internally_reserved":0,"allocated":2482176,"data_stored":1201396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509494,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64204,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307636,"statfs":{"total":96636764160,"available":96571019264,"internally_reserved":0,"allocated":2539520,"data_stored":1791764,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542195,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51740,"kb_used_data":2500,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320100,"statfs":{"total":96636764160,"available":96583782400,"internally_reserved":0,"allocated":2560000,"data_stored":1809821,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":186,"seq":798863917063,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35944,"kb_used_data":2452,"kb_used_omap":62,"kb_used_meta":33409,"kb_avail":94335896,"statfs":{"total":96636764160,"available":96599957504,"internally_reserved":0,"allocated":2510848,"data_stored":1213460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":63771,"internal_metadata":34211557},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542196,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60004,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311836,"statfs":{"total":96636764160,"available":96575320064,"internally_reserved":0,"allocated":2506752,"data_stored":1213441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542197,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55852,"kb_used_data":2456,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315988,"statfs":{"total":96636764160,"available":96579571712,"internally_reserved":0,"allocated":2514944,"data_stored":1213479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:09:06.900 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T09:09:06.900 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-14T09:09:07.239 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-14T09:09:07.250 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T09:09:07.250 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:09:07.558 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:09:07.559 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:09:07.577 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-14T09:09:06.861379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":285,"num_object_clones":0,"num_object_copies":570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":285,"num_whiteouts":277,"num_read":198278,"num_read_kb":860275,"num_write":25191,"num_write_kb":387556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4050,"num_bytes_recovered":203007312,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22413,"ondisk_log_size":22413,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":422748,"kb_used_data":19612,"kb_used_omap":73,"kb_used_meta":402870,"kb_avail":754551972,"statfs":{"total":773094113280,"available":772661219328,"internally_reserved":0,"allocated":20082688,"data_stored":10846172,"data_compressed":13252,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":74928,"internal_metadata":412539728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.836619"},"pg_stats":[{"pgid":"3.1f","version":"182'1763","reported_seq":7850,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.892276+0000","last_change":"2024-09-14T09:08:34.495405+0000","last_active":"2024-09-14T09:09:01.892276+0000","last_peered":"2024-09-14T09:09:01.892276+0000","last_clean":"2024-09-14T09:09:01.892276+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:09:01.892276+0000","last_undegraded":"2024-09-14T09:09:01.892276+0000","last_fullsized":"2024-09-14T09:09:01.892276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1763,"log_dups_size":0,"ondisk_log_size":1763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:22:02.551926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022047759999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13833,"num_read_kb":24717,"num_write":2684,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'466","reported_seq":5985,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034518+0000","last_change":"2024-09-14T09:08:34.625492+0000","last_active":"2024-09-14T09:09:06.034518+0000","last_peered":"2024-09-14T09:09:06.034518+0000","last_clean":"2024-09-14T09:09:06.034518+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:09:06.034518+0000","last_undegraded":"2024-09-14T09:09:06.034518+0000","last_fullsized":"2024-09-14T09:09:06.034518+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:01.005568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010401127,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5032,"num_read_kb":31996,"num_write":418,"num_write_kb":13378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"187'676","reported_seq":6606,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.947075+0000","last_change":"2024-09-14T09:08:43.847915+0000","last_active":"2024-09-14T09:09:01.947075+0000","last_peered":"2024-09-14T09:09:01.947075+0000","last_clean":"2024-09-14T09:09:01.947075+0000","last_became_active":"2024-09-14T09:08:43.573212+0000","last_became_peered":"2024-09-14T09:08:43.573212+0000","last_unstale":"2024-09-14T09:09:01.947075+0000","last_undegraded":"2024-09-14T09:09:01.947075+0000","last_fullsized":"2024-09-14T09:09:01.947075+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:26:03.155997+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.147750145,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6669,"num_read_kb":58049,"num_write":727,"num_write_kb":12436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"190'866","reported_seq":5635,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.946852+0000","last_change":"2024-09-14T09:08:46.770708+0000","last_active":"2024-09-14T09:09:01.946852+0000","last_peered":"2024-09-14T09:09:01.946852+0000","last_clean":"2024-09-14T09:09:01.946852+0000","last_became_active":"2024-09-14T09:08:46.599475+0000","last_became_peered":"2024-09-14T09:08:46.599475+0000","last_unstale":"2024-09-14T09:09:01.946852+0000","last_undegraded":"2024-09-14T09:09:01.946852+0000","last_fullsized":"2024-09-14T09:09:01.946852+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:57.157599+0000","scrub_duration":0,"objects_trimmed":60,"snaptrim_duration":0.168855903,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7679,"num_read_kb":27474,"num_write":1043,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'742","reported_seq":5707,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.891412+0000","last_change":"2024-09-14T09:08:34.491862+0000","last_active":"2024-09-14T09:09:01.891412+0000","last_peered":"2024-09-14T09:09:01.891412+0000","last_clean":"2024-09-14T09:09:01.891412+0000","last_became_active":"2024-09-14T09:07:06.842337+0000","last_became_peered":"2024-09-14T09:07:06.842337+0000","last_unstale":"2024-09-14T09:09:01.891412+0000","last_undegraded":"2024-09-14T09:09:01.891412+0000","last_fullsized":"2024-09-14T09:09:01.891412+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:23:25.823078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023564419999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6703,"num_read_kb":24544,"num_write":874,"num_write_kb":16463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":14145319,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"187'724","reported_seq":5508,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937332+0000","last_change":"2024-09-14T09:08:43.771622+0000","last_active":"2024-09-14T09:09:05.937332+0000","last_peered":"2024-09-14T09:09:05.937332+0000","last_clean":"2024-09-14T09:09:05.937332+0000","last_became_active":"2024-09-14T09:08:43.556338+0000","last_became_peered":"2024-09-14T09:08:43.556338+0000","last_unstale":"2024-09-14T09:09:05.937332+0000","last_undegraded":"2024-09-14T09:09:05.937332+0000","last_fullsized":"2024-09-14T09:09:05.937332+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:41.389225+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.12752904200000001,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6399,"num_read_kb":30687,"num_write":963,"num_write_kb":14233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"174'375","reported_seq":4396,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937548+0000","last_change":"2024-09-14T09:08:50.629874+0000","last_active":"2024-09-14T09:09:05.937548+0000","last_peered":"2024-09-14T09:09:05.937548+0000","last_clean":"2024-09-14T09:09:05.937548+0000","last_became_active":"2024-09-14T09:08:50.627814+0000","last_became_peered":"2024-09-14T09:08:50.627814+0000","last_unstale":"2024-09-14T09:09:05.937548+0000","last_undegraded":"2024-09-14T09:09:05.937548+0000","last_fullsized":"2024-09-14T09:09:05.937548+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:14:43.966792+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.025218744000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3327,"num_read_kb":10838,"num_write":357,"num_write_kb":6673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":3727781,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"182'624","reported_seq":6376,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033940+0000","last_change":"2024-09-14T09:08:34.622411+0000","last_active":"2024-09-14T09:09:06.033940+0000","last_peered":"2024-09-14T09:09:06.033940+0000","last_clean":"2024-09-14T09:09:06.033940+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:09:06.033940+0000","last_undegraded":"2024-09-14T09:09:06.033940+0000","last_fullsized":"2024-09-14T09:09:06.033940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:55:38.962684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080911500000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7486,"num_read_kb":17479,"num_write":535,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"182'910","reported_seq":7819,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.892469+0000","last_change":"2024-09-14T09:08:34.496061+0000","last_active":"2024-09-14T09:09:01.892469+0000","last_peered":"2024-09-14T09:09:01.892469+0000","last_clean":"2024-09-14T09:09:01.892469+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:09:01.892469+0000","last_undegraded":"2024-09-14T09:09:01.892469+0000","last_fullsized":"2024-09-14T09:09:01.892469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:02:38.220394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026116440000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10070,"num_read_kb":35479,"num_write":1129,"num_write_kb":10795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"179'629","reported_seq":3942,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.946998+0000","last_change":"2024-09-14T09:08:34.493205+0000","last_active":"2024-09-14T09:09:01.946998+0000","last_peered":"2024-09-14T09:09:01.946998+0000","last_clean":"2024-09-14T09:09:01.946998+0000","last_became_active":"2024-09-14T09:07:25.105674+0000","last_became_peered":"2024-09-14T09:07:25.105674+0000","last_unstale":"2024-09-14T09:09:01.946998+0000","last_undegraded":"2024-09-14T09:09:01.946998+0000","last_fullsized":"2024-09-14T09:09:01.946998+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:20:01.627411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001781447,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4726,"num_read_kb":15599,"num_write":700,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":8467448,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"181'1136","reported_seq":5389,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.951079+0000","last_change":"2024-09-14T09:08:34.507525+0000","last_active":"2024-09-14T09:09:05.951079+0000","last_peered":"2024-09-14T09:09:05.951079+0000","last_clean":"2024-09-14T09:09:05.951079+0000","last_became_active":"2024-09-14T09:07:27.854998+0000","last_became_peered":"2024-09-14T09:07:27.854998+0000","last_unstale":"2024-09-14T09:09:05.951079+0000","last_undegraded":"2024-09-14T09:09:05.951079+0000","last_fullsized":"2024-09-14T09:09:05.951079+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:11:04.062027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001704445,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8595,"num_read_kb":26590,"num_write":1602,"num_write_kb":14717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":4170600,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"187'670","reported_seq":3455,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937564+0000","last_change":"2024-09-14T09:08:43.840463+0000","last_active":"2024-09-14T09:09:05.937564+0000","last_peered":"2024-09-14T09:09:05.937564+0000","last_clean":"2024-09-14T09:09:05.937564+0000","last_became_active":"2024-09-14T09:08:43.569123+0000","last_became_peered":"2024-09-14T09:08:43.569123+0000","last_unstale":"2024-09-14T09:09:05.937564+0000","last_undegraded":"2024-09-14T09:09:05.937564+0000","last_fullsized":"2024-09-14T09:09:05.937564+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:18.861184+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.15505904000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3583,"num_read_kb":9199,"num_write":691,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":2686146,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":397,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.133359+0000","last_change":"2024-09-14T09:07:21.857426+0000","last_active":"2024-09-14T09:09:02.133359+0000","last_peered":"2024-09-14T09:09:02.133359+0000","last_clean":"2024-09-14T09:09:02.133359+0000","last_became_active":"2024-09-14T09:07:21.856579+0000","last_became_peered":"2024-09-14T09:07:21.856579+0000","last_unstale":"2024-09-14T09:09:02.133359+0000","last_undegraded":"2024-09-14T09:09:02.133359+0000","last_fullsized":"2024-09-14T09:09:02.133359+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:56.694930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"182'791","reported_seq":6687,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.891358+0000","last_change":"2024-09-14T09:08:34.494518+0000","last_active":"2024-09-14T09:09:01.891358+0000","last_peered":"2024-09-14T09:09:01.891358+0000","last_clean":"2024-09-14T09:09:01.891358+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:09:01.891358+0000","last_undegraded":"2024-09-14T09:09:01.891358+0000","last_fullsized":"2024-09-14T09:09:01.891358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:58.731530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016873319999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8676,"num_read_kb":45389,"num_write":707,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":356,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937898+0000","last_change":"2024-09-14T09:08:51.290129+0000","last_active":"2024-09-14T09:09:05.937898+0000","last_peered":"2024-09-14T09:09:05.937898+0000","last_clean":"2024-09-14T09:09:05.937898+0000","last_became_active":"2024-09-14T09:08:51.289551+0000","last_became_peered":"2024-09-14T09:08:51.289551+0000","last_unstale":"2024-09-14T09:09:05.937898+0000","last_undegraded":"2024-09-14T09:09:05.937898+0000","last_fullsized":"2024-09-14T09:09:05.937898+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:52:01.486956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"182'1187","reported_seq":7258,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034110+0000","last_change":"2024-09-14T09:08:34.626223+0000","last_active":"2024-09-14T09:09:06.034110+0000","last_peered":"2024-09-14T09:09:06.034110+0000","last_clean":"2024-09-14T09:09:06.034110+0000","last_became_active":"2024-09-14T09:07:26.102666+0000","last_became_peered":"2024-09-14T09:07:26.102666+0000","last_unstale":"2024-09-14T09:09:06.034110+0000","last_undegraded":"2024-09-14T09:09:06.034110+0000","last_fullsized":"2024-09-14T09:09:06.034110+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:03:41.035208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011116867000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10102,"num_read_kb":27247,"num_write":1623,"num_write_kb":13456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":8334712,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":200,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.950988+0000","last_change":"2024-09-14T09:07:21.863290+0000","last_active":"2024-09-14T09:09:05.950988+0000","last_peered":"2024-09-14T09:09:05.950988+0000","last_clean":"2024-09-14T09:09:05.950988+0000","last_became_active":"2024-09-14T09:07:21.862352+0000","last_became_peered":"2024-09-14T09:07:21.862352+0000","last_unstale":"2024-09-14T09:09:05.950988+0000","last_undegraded":"2024-09-14T09:09:05.950988+0000","last_fullsized":"2024-09-14T09:09:05.950988+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:48:30.332891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"179'922","reported_seq":5885,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937691+0000","last_change":"2024-09-14T09:08:50.627160+0000","last_active":"2024-09-14T09:09:05.937691+0000","last_peered":"2024-09-14T09:09:05.937691+0000","last_clean":"2024-09-14T09:09:05.937691+0000","last_became_active":"2024-09-14T09:08:50.625357+0000","last_became_peered":"2024-09-14T09:08:50.625357+0000","last_unstale":"2024-09-14T09:09:05.937691+0000","last_undegraded":"2024-09-14T09:09:05.937691+0000","last_fullsized":"2024-09-14T09:09:05.937691+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:18:31.289028+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0021152969999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7783,"num_read_kb":22826,"num_write":1134,"num_write_kb":9259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":8151041,"num_keys_recovered":91,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":356,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937696+0000","last_change":"2024-09-14T09:08:50.628307+0000","last_active":"2024-09-14T09:09:05.937696+0000","last_peered":"2024-09-14T09:09:05.937696+0000","last_clean":"2024-09-14T09:09:05.937696+0000","last_became_active":"2024-09-14T09:08:50.627971+0000","last_became_peered":"2024-09-14T09:08:50.627971+0000","last_unstale":"2024-09-14T09:09:05.937696+0000","last_undegraded":"2024-09-14T09:09:05.937696+0000","last_fullsized":"2024-09-14T09:09:05.937696+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:57.655435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"182'688","reported_seq":2914,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.947150+0000","last_change":"2024-09-14T09:08:34.495006+0000","last_active":"2024-09-14T09:09:01.947150+0000","last_peered":"2024-09-14T09:09:01.947150+0000","last_clean":"2024-09-14T09:09:01.947150+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:09:01.947150+0000","last_undegraded":"2024-09-14T09:09:01.947150+0000","last_fullsized":"2024-09-14T09:09:01.947150+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:25:08.203494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030472329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3594,"num_read_kb":16894,"num_write":791,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"190'884","reported_seq":6714,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937761+0000","last_change":"2024-09-14T09:08:46.793435+0000","last_active":"2024-09-14T09:09:05.937761+0000","last_peered":"2024-09-14T09:09:05.937761+0000","last_clean":"2024-09-14T09:09:05.937761+0000","last_became_active":"2024-09-14T09:08:46.597293+0000","last_became_peered":"2024-09-14T09:08:46.597293+0000","last_unstale":"2024-09-14T09:09:05.937761+0000","last_undegraded":"2024-09-14T09:09:05.937761+0000","last_fullsized":"2024-09-14T09:09:05.937761+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:48:54.046569+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.19293607300000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8248,"num_read_kb":26883,"num_write":890,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":390,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.946760+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:09:01.946760+0000","last_peered":"2024-09-14T09:09:01.946760+0000","last_clean":"2024-09-14T09:09:01.946760+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:09:01.946760+0000","last_undegraded":"2024-09-14T09:09:01.946760+0000","last_fullsized":"2024-09-14T09:09:01.946760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"180'399","reported_seq":2535,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937522+0000","last_change":"2024-09-14T09:08:50.625390+0000","last_active":"2024-09-14T09:09:05.937522+0000","last_peered":"2024-09-14T09:09:05.937522+0000","last_clean":"2024-09-14T09:09:05.937522+0000","last_became_active":"2024-09-14T09:08:50.622578+0000","last_became_peered":"2024-09-14T09:08:50.622578+0000","last_unstale":"2024-09-14T09:09:05.937522+0000","last_undegraded":"2024-09-14T09:09:05.937522+0000","last_fullsized":"2024-09-14T09:09:05.937522+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:50:11.634631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00245612,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1873,"num_read_kb":11982,"num_write":352,"num_write_kb":5539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":387,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.890033+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:09:01.890033+0000","last_peered":"2024-09-14T09:09:01.890033+0000","last_clean":"2024-09-14T09:09:01.890033+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:09:01.890033+0000","last_undegraded":"2024-09-14T09:09:01.890033+0000","last_fullsized":"2024-09-14T09:09:01.890033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"187'460","reported_seq":5073,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.891197+0000","last_change":"2024-09-14T09:08:43.789977+0000","last_active":"2024-09-14T09:09:01.891197+0000","last_peered":"2024-09-14T09:09:01.891197+0000","last_clean":"2024-09-14T09:09:01.891197+0000","last_became_active":"2024-09-14T09:08:43.568923+0000","last_became_peered":"2024-09-14T09:08:43.568923+0000","last_unstale":"2024-09-14T09:09:01.891197+0000","last_undegraded":"2024-09-14T09:09:01.891197+0000","last_fullsized":"2024-09-14T09:09:01.891197+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:45:29.077592+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.165000182,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3905,"num_read_kb":25149,"num_write":375,"num_write_kb":14483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937319+0000","last_change":"2024-09-14T09:08:44.554373+0000","last_active":"2024-09-14T09:09:05.937319+0000","last_peered":"2024-09-14T09:09:05.937319+0000","last_clean":"2024-09-14T09:09:05.937319+0000","last_became_active":"2024-09-14T09:08:44.553643+0000","last_became_peered":"2024-09-14T09:08:44.553643+0000","last_unstale":"2024-09-14T09:09:05.937319+0000","last_undegraded":"2024-09-14T09:09:05.937319+0000","last_fullsized":"2024-09-14T09:09:05.937319+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:32:52.900856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"181'433","reported_seq":2980,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034173+0000","last_change":"2024-09-14T09:08:34.620964+0000","last_active":"2024-09-14T09:09:06.034173+0000","last_peered":"2024-09-14T09:09:06.034173+0000","last_clean":"2024-09-14T09:09:06.034173+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:09:06.034173+0000","last_undegraded":"2024-09-14T09:09:06.034173+0000","last_fullsized":"2024-09-14T09:09:06.034173+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:34:59.882295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065530470000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2291,"num_read_kb":11209,"num_write":432,"num_write_kb":6866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":118,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.946781+0000","last_change":"2024-09-14T09:08:06.210516+0000","last_active":"2024-09-14T09:09:01.946781+0000","last_peered":"2024-09-14T09:09:01.946781+0000","last_clean":"2024-09-14T09:09:01.946781+0000","last_became_active":"2024-09-14T09:08:06.205108+0000","last_became_peered":"2024-09-14T09:08:06.205108+0000","last_unstale":"2024-09-14T09:09:01.946781+0000","last_undegraded":"2024-09-14T09:09:01.946781+0000","last_fullsized":"2024-09-14T09:09:01.946781+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:22:05.543865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":206,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949404+0000","last_change":"2024-09-14T09:07:21.876180+0000","last_active":"2024-09-14T09:09:05.949404+0000","last_peered":"2024-09-14T09:09:05.949404+0000","last_clean":"2024-09-14T09:09:05.949404+0000","last_became_active":"2024-09-14T09:07:21.860117+0000","last_became_peered":"2024-09-14T09:07:21.860117+0000","last_unstale":"2024-09-14T09:09:05.949404+0000","last_undegraded":"2024-09-14T09:09:05.949404+0000","last_fullsized":"2024-09-14T09:09:05.949404+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:58:36.142971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"178'682","reported_seq":7039,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033947+0000","last_change":"2024-09-14T09:08:34.623351+0000","last_active":"2024-09-14T09:09:06.033947+0000","last_peered":"2024-09-14T09:09:06.033947+0000","last_clean":"2024-09-14T09:09:06.033947+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:09:06.033947+0000","last_undegraded":"2024-09-14T09:09:06.033947+0000","last_fullsized":"2024-09-14T09:09:06.033947+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:48.184932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089062680000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7688,"num_read_kb":41730,"num_write":748,"num_write_kb":19769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"181'469","reported_seq":3204,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.133102+0000","last_change":"2024-09-14T09:08:35.139769+0000","last_active":"2024-09-14T09:09:02.133102+0000","last_peered":"2024-09-14T09:09:02.133102+0000","last_clean":"2024-09-14T09:09:02.133102+0000","last_became_active":"2024-09-14T09:07:06.851020+0000","last_became_peered":"2024-09-14T09:07:06.851020+0000","last_unstale":"2024-09-14T09:09:02.133102+0000","last_undegraded":"2024-09-14T09:09:02.133102+0000","last_fullsized":"2024-09-14T09:09:02.133102+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:19:55.337692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154644,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2080,"num_read_kb":17783,"num_write":389,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":12315549,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"187'749","reported_seq":5078,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.133108+0000","last_change":"2024-09-14T09:08:43.990954+0000","last_active":"2024-09-14T09:09:02.133108+0000","last_peered":"2024-09-14T09:09:02.133108+0000","last_clean":"2024-09-14T09:09:02.133108+0000","last_became_active":"2024-09-14T09:08:43.580245+0000","last_became_peered":"2024-09-14T09:08:43.580245+0000","last_unstale":"2024-09-14T09:09:02.133108+0000","last_undegraded":"2024-09-14T09:09:02.133108+0000","last_fullsized":"2024-09-14T09:09:02.133108+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:04.798520+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.116960367,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5499,"num_read_kb":39102,"num_write":883,"num_write_kb":19962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"181'544","reported_seq":9827,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034065+0000","last_change":"2024-09-14T09:08:34.624404+0000","last_active":"2024-09-14T09:09:06.034065+0000","last_peered":"2024-09-14T09:09:06.034065+0000","last_clean":"2024-09-14T09:09:06.034065+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:09:06.034065+0000","last_undegraded":"2024-09-14T09:09:06.034065+0000","last_fullsized":"2024-09-14T09:09:06.034065+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:54:24.811934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097700640000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10396,"num_read_kb":61123,"num_write":490,"num_write_kb":11301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"180'493","reported_seq":3386,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.946864+0000","last_change":"2024-09-14T09:08:34.496008+0000","last_active":"2024-09-14T09:09:01.946864+0000","last_peered":"2024-09-14T09:09:01.946864+0000","last_clean":"2024-09-14T09:09:01.946864+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:09:01.946864+0000","last_undegraded":"2024-09-14T09:09:01.946864+0000","last_fullsized":"2024-09-14T09:09:01.946864+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:11.774708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00388507,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3266,"num_read_kb":13215,"num_write":426,"num_write_kb":10574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"179'501","reported_seq":5831,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.134126+0000","last_change":"2024-09-14T09:08:34.506908+0000","last_active":"2024-09-14T09:09:02.134126+0000","last_peered":"2024-09-14T09:09:02.134126+0000","last_clean":"2024-09-14T09:09:02.134126+0000","last_became_active":"2024-09-14T09:07:26.853961+0000","last_became_peered":"2024-09-14T09:07:26.853961+0000","last_unstale":"2024-09-14T09:09:02.134126+0000","last_undegraded":"2024-09-14T09:09:02.134126+0000","last_fullsized":"2024-09-14T09:09:02.134126+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:54.100061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001753449,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5399,"num_read_kb":32061,"num_write":547,"num_write_kb":21536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":25396220,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"187'488","reported_seq":4034,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949818+0000","last_change":"2024-09-14T09:08:44.221618+0000","last_active":"2024-09-14T09:09:05.949818+0000","last_peered":"2024-09-14T09:09:05.949818+0000","last_clean":"2024-09-14T09:09:05.949818+0000","last_became_active":"2024-09-14T09:08:43.588233+0000","last_became_peered":"2024-09-14T09:08:43.588233+0000","last_unstale":"2024-09-14T09:09:05.949818+0000","last_undegraded":"2024-09-14T09:09:05.949818+0000","last_fullsized":"2024-09-14T09:09:05.949818+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:10.863128+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.14577486200000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4282,"num_read_kb":17652,"num_write":409,"num_write_kb":7874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":14123394,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"187'586","reported_seq":4287,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.950045+0000","last_change":"2024-09-14T09:08:44.145761+0000","last_active":"2024-09-14T09:09:05.950045+0000","last_peered":"2024-09-14T09:09:05.950045+0000","last_clean":"2024-09-14T09:09:05.950045+0000","last_became_active":"2024-09-14T09:08:43.574566+0000","last_became_peered":"2024-09-14T09:08:43.574566+0000","last_unstale":"2024-09-14T09:09:05.950045+0000","last_undegraded":"2024-09-14T09:09:05.950045+0000","last_fullsized":"2024-09-14T09:09:05.950045+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:30.502086+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.141382223,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3641,"num_read_kb":14849,"num_write":554,"num_write_kb":12244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":11864981,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'506","reported_seq":6236,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:02.133073+0000","last_change":"2024-09-14T09:08:35.140559+0000","last_active":"2024-09-14T09:09:02.133073+0000","last_peered":"2024-09-14T09:09:02.133073+0000","last_clean":"2024-09-14T09:09:02.133073+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:09:02.133073+0000","last_undegraded":"2024-09-14T09:09:02.133073+0000","last_fullsized":"2024-09-14T09:09:02.133073+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:31:57.906202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018540480000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5422,"num_read_kb":60065,"num_write":521,"num_write_kb":20523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"187'649","reported_seq":6144,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949422+0000","last_change":"2024-09-14T09:08:43.902203+0000","last_active":"2024-09-14T09:09:05.949422+0000","last_peered":"2024-09-14T09:09:05.949422+0000","last_clean":"2024-09-14T09:09:05.949422+0000","last_became_active":"2024-09-14T09:08:43.577551+0000","last_became_peered":"2024-09-14T09:08:43.577551+0000","last_unstale":"2024-09-14T09:09:05.949422+0000","last_undegraded":"2024-09-14T09:09:05.949422+0000","last_fullsized":"2024-09-14T09:09:05.949422+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:43:37.090068+0000","scrub_duration":0,"objects_trimmed":64,"snaptrim_duration":0.121356326,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6082,"num_read_kb":13595,"num_write":653,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":7405746,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"182'790","reported_seq":6120,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-14T09:09:01.891244+0000","last_change":"2024-09-14T09:08:34.493569+0000","last_active":"2024-09-14T09:09:01.891244+0000","last_peered":"2024-09-14T09:09:01.891244+0000","last_clean":"2024-09-14T09:09:01.891244+0000","last_became_active":"2024-09-14T09:07:26.101586+0000","last_became_peered":"2024-09-14T09:07:26.101586+0000","last_unstale":"2024-09-14T09:09:01.891244+0000","last_undegraded":"2024-09-14T09:09:01.891244+0000","last_fullsized":"2024-09-14T09:09:01.891244+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:11:20.433650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040594799999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7044,"num_read_kb":19519,"num_write":992,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":10260744,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"187'536","reported_seq":6572,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949474+0000","last_change":"2024-09-14T09:08:44.016919+0000","last_active":"2024-09-14T09:09:05.949474+0000","last_peered":"2024-09-14T09:09:05.949474+0000","last_clean":"2024-09-14T09:09:05.949474+0000","last_became_active":"2024-09-14T09:08:43.579997+0000","last_became_peered":"2024-09-14T09:08:43.579997+0000","last_unstale":"2024-09-14T09:09:05.949474+0000","last_undegraded":"2024-09-14T09:09:05.949474+0000","last_fullsized":"2024-09-14T09:09:05.949474+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:24:16.807890+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.10056556899999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6839,"num_read_kb":29296,"num_write":449,"num_write_kb":10682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":24080444,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":281,"num_object_clones":0,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":277,"num_read":198212,"num_read_kb":860220,"num_write":25088,"num_write_kb":385926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4045,"num_bytes_recovered":201826576,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},"log_size":22338,"ondisk_log_size":22338,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20292,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509491,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29464,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342376,"statfs":{"total":96636764160,"available":96606593024,"internally_reserved":0,"allocated":2482176,"data_stored":1201396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509491,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78036,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293804,"statfs":{"total":96636764160,"available":96556855296,"internally_reserved":0,"allocated":2486272,"data_stored":1201415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509492,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47504,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324336,"statfs":{"total":96636764160,"available":96588120064,"internally_reserved":0,"allocated":2482176,"data_stored":1201396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509494,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64204,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307636,"statfs":{"total":96636764160,"available":96571019264,"internally_reserved":0,"allocated":2539520,"data_stored":1791764,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542195,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51740,"kb_used_data":2500,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320100,"statfs":{"total":96636764160,"available":96583782400,"internally_reserved":0,"allocated":2560000,"data_stored":1809821,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":186,"seq":798863917063,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35944,"kb_used_data":2452,"kb_used_omap":62,"kb_used_meta":33409,"kb_avail":94335896,"statfs":{"total":96636764160,"available":96599957504,"internally_reserved":0,"allocated":2510848,"data_stored":1213460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":63771,"internal_metadata":34211557},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542196,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60004,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311836,"statfs":{"total":96636764160,"available":96575320064,"internally_reserved":0,"allocated":2506752,"data_stored":1213441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542197,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55852,"kb_used_data":2456,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315988,"statfs":{"total":96636764160,"available":96579571712,"internally_reserved":0,"allocated":2514944,"data_stored":1213479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:09:07.579 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:09:07.877 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:09:07.878 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:09:07.894 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-14T09:09:07.864812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":285,"num_object_clones":0,"num_object_copies":570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":285,"num_whiteouts":277,"num_read":198278,"num_read_kb":860275,"num_write":25191,"num_write_kb":387556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4182,"num_bytes_recovered":203007312,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22413,"ondisk_log_size":22413,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":422884,"kb_used_data":19768,"kb_used_omap":73,"kb_used_meta":402870,"kb_avail":754551836,"statfs":{"total":773094113280,"available":772661080064,"internally_reserved":0,"allocated":20242432,"data_stored":10924408,"data_compressed":13252,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":74928,"internal_metadata":412539728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.952939"},"pg_stats":[{"pgid":"3.1f","version":"182'1763","reported_seq":7856,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033212+0000","last_change":"2024-09-14T09:08:34.495405+0000","last_active":"2024-09-14T09:09:06.033212+0000","last_peered":"2024-09-14T09:09:06.033212+0000","last_clean":"2024-09-14T09:09:06.033212+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:09:06.033212+0000","last_undegraded":"2024-09-14T09:09:06.033212+0000","last_fullsized":"2024-09-14T09:09:06.033212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1763,"log_dups_size":0,"ondisk_log_size":1763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:22:02.551926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022047759999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13833,"num_read_kb":24717,"num_write":2684,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'466","reported_seq":5985,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034518+0000","last_change":"2024-09-14T09:08:34.625492+0000","last_active":"2024-09-14T09:09:06.034518+0000","last_peered":"2024-09-14T09:09:06.034518+0000","last_clean":"2024-09-14T09:09:06.034518+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:09:06.034518+0000","last_undegraded":"2024-09-14T09:09:06.034518+0000","last_fullsized":"2024-09-14T09:09:06.034518+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:01.005568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010401127,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5032,"num_read_kb":31996,"num_write":418,"num_write_kb":13378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"187'676","reported_seq":6614,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.865111+0000","last_change":"2024-09-14T09:08:43.847915+0000","last_active":"2024-09-14T09:09:06.865111+0000","last_peered":"2024-09-14T09:09:06.865111+0000","last_clean":"2024-09-14T09:09:06.865111+0000","last_became_active":"2024-09-14T09:08:43.573212+0000","last_became_peered":"2024-09-14T09:08:43.573212+0000","last_unstale":"2024-09-14T09:09:06.865111+0000","last_undegraded":"2024-09-14T09:09:06.865111+0000","last_fullsized":"2024-09-14T09:09:06.865111+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:26:03.155997+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.147750145,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6669,"num_read_kb":58049,"num_write":727,"num_write_kb":12436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"190'866","reported_seq":5643,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864770+0000","last_change":"2024-09-14T09:08:46.770708+0000","last_active":"2024-09-14T09:09:06.864770+0000","last_peered":"2024-09-14T09:09:06.864770+0000","last_clean":"2024-09-14T09:09:06.864770+0000","last_became_active":"2024-09-14T09:08:46.599475+0000","last_became_peered":"2024-09-14T09:08:46.599475+0000","last_unstale":"2024-09-14T09:09:06.864770+0000","last_undegraded":"2024-09-14T09:09:06.864770+0000","last_fullsized":"2024-09-14T09:09:06.864770+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:57.157599+0000","scrub_duration":0,"objects_trimmed":60,"snaptrim_duration":0.168855903,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7679,"num_read_kb":27474,"num_write":1043,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'742","reported_seq":5713,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033232+0000","last_change":"2024-09-14T09:08:34.491862+0000","last_active":"2024-09-14T09:09:06.033232+0000","last_peered":"2024-09-14T09:09:06.033232+0000","last_clean":"2024-09-14T09:09:06.033232+0000","last_became_active":"2024-09-14T09:07:06.842337+0000","last_became_peered":"2024-09-14T09:07:06.842337+0000","last_unstale":"2024-09-14T09:09:06.033232+0000","last_undegraded":"2024-09-14T09:09:06.033232+0000","last_fullsized":"2024-09-14T09:09:06.033232+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:23:25.823078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023564419999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6703,"num_read_kb":24544,"num_write":874,"num_write_kb":16463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":14145319,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"187'724","reported_seq":5508,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937332+0000","last_change":"2024-09-14T09:08:43.771622+0000","last_active":"2024-09-14T09:09:05.937332+0000","last_peered":"2024-09-14T09:09:05.937332+0000","last_clean":"2024-09-14T09:09:05.937332+0000","last_became_active":"2024-09-14T09:08:43.556338+0000","last_became_peered":"2024-09-14T09:08:43.556338+0000","last_unstale":"2024-09-14T09:09:05.937332+0000","last_undegraded":"2024-09-14T09:09:05.937332+0000","last_fullsized":"2024-09-14T09:09:05.937332+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:41.389225+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.12752904200000001,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6399,"num_read_kb":30687,"num_write":963,"num_write_kb":14233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"174'375","reported_seq":4396,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937548+0000","last_change":"2024-09-14T09:08:50.629874+0000","last_active":"2024-09-14T09:09:05.937548+0000","last_peered":"2024-09-14T09:09:05.937548+0000","last_clean":"2024-09-14T09:09:05.937548+0000","last_became_active":"2024-09-14T09:08:50.627814+0000","last_became_peered":"2024-09-14T09:08:50.627814+0000","last_unstale":"2024-09-14T09:09:05.937548+0000","last_undegraded":"2024-09-14T09:09:05.937548+0000","last_fullsized":"2024-09-14T09:09:05.937548+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:14:43.966792+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.025218744000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3327,"num_read_kb":10838,"num_write":357,"num_write_kb":6673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":3727781,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"182'624","reported_seq":6376,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033940+0000","last_change":"2024-09-14T09:08:34.622411+0000","last_active":"2024-09-14T09:09:06.033940+0000","last_peered":"2024-09-14T09:09:06.033940+0000","last_clean":"2024-09-14T09:09:06.033940+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:09:06.033940+0000","last_undegraded":"2024-09-14T09:09:06.033940+0000","last_fullsized":"2024-09-14T09:09:06.033940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:55:38.962684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080911500000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7486,"num_read_kb":17479,"num_write":535,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"182'910","reported_seq":7825,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033222+0000","last_change":"2024-09-14T09:08:34.496061+0000","last_active":"2024-09-14T09:09:06.033222+0000","last_peered":"2024-09-14T09:09:06.033222+0000","last_clean":"2024-09-14T09:09:06.033222+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:09:06.033222+0000","last_undegraded":"2024-09-14T09:09:06.033222+0000","last_fullsized":"2024-09-14T09:09:06.033222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:02:38.220394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026116440000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10070,"num_read_kb":35479,"num_write":1129,"num_write_kb":10795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"179'629","reported_seq":3950,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864528+0000","last_change":"2024-09-14T09:08:34.493205+0000","last_active":"2024-09-14T09:09:06.864528+0000","last_peered":"2024-09-14T09:09:06.864528+0000","last_clean":"2024-09-14T09:09:06.864528+0000","last_became_active":"2024-09-14T09:07:25.105674+0000","last_became_peered":"2024-09-14T09:07:25.105674+0000","last_unstale":"2024-09-14T09:09:06.864528+0000","last_undegraded":"2024-09-14T09:09:06.864528+0000","last_fullsized":"2024-09-14T09:09:06.864528+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:20:01.627411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001781447,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4726,"num_read_kb":15599,"num_write":700,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":8467448,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"181'1136","reported_seq":5389,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.951079+0000","last_change":"2024-09-14T09:08:34.507525+0000","last_active":"2024-09-14T09:09:05.951079+0000","last_peered":"2024-09-14T09:09:05.951079+0000","last_clean":"2024-09-14T09:09:05.951079+0000","last_became_active":"2024-09-14T09:07:27.854998+0000","last_became_peered":"2024-09-14T09:07:27.854998+0000","last_unstale":"2024-09-14T09:09:05.951079+0000","last_undegraded":"2024-09-14T09:09:05.951079+0000","last_fullsized":"2024-09-14T09:09:05.951079+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:11:04.062027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001704445,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8595,"num_read_kb":26590,"num_write":1602,"num_write_kb":14717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":4170600,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"187'670","reported_seq":3455,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937564+0000","last_change":"2024-09-14T09:08:43.840463+0000","last_active":"2024-09-14T09:09:05.937564+0000","last_peered":"2024-09-14T09:09:05.937564+0000","last_clean":"2024-09-14T09:09:05.937564+0000","last_became_active":"2024-09-14T09:08:43.569123+0000","last_became_peered":"2024-09-14T09:08:43.569123+0000","last_unstale":"2024-09-14T09:09:05.937564+0000","last_undegraded":"2024-09-14T09:09:05.937564+0000","last_fullsized":"2024-09-14T09:09:05.937564+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:18.861184+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.15505904000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3583,"num_read_kb":9199,"num_write":691,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":2686146,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":403,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038664+0000","last_change":"2024-09-14T09:07:21.857426+0000","last_active":"2024-09-14T09:09:06.038664+0000","last_peered":"2024-09-14T09:09:06.038664+0000","last_clean":"2024-09-14T09:09:06.038664+0000","last_became_active":"2024-09-14T09:07:21.856579+0000","last_became_peered":"2024-09-14T09:07:21.856579+0000","last_unstale":"2024-09-14T09:09:06.038664+0000","last_undegraded":"2024-09-14T09:09:06.038664+0000","last_fullsized":"2024-09-14T09:09:06.038664+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:56.694930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"182'791","reported_seq":6693,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033604+0000","last_change":"2024-09-14T09:08:34.494518+0000","last_active":"2024-09-14T09:09:06.033604+0000","last_peered":"2024-09-14T09:09:06.033604+0000","last_clean":"2024-09-14T09:09:06.033604+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:09:06.033604+0000","last_undegraded":"2024-09-14T09:09:06.033604+0000","last_fullsized":"2024-09-14T09:09:06.033604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:58.731530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016873319999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8676,"num_read_kb":45389,"num_write":707,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":356,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937898+0000","last_change":"2024-09-14T09:08:51.290129+0000","last_active":"2024-09-14T09:09:05.937898+0000","last_peered":"2024-09-14T09:09:05.937898+0000","last_clean":"2024-09-14T09:09:05.937898+0000","last_became_active":"2024-09-14T09:08:51.289551+0000","last_became_peered":"2024-09-14T09:08:51.289551+0000","last_unstale":"2024-09-14T09:09:05.937898+0000","last_undegraded":"2024-09-14T09:09:05.937898+0000","last_fullsized":"2024-09-14T09:09:05.937898+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:52:01.486956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"182'1187","reported_seq":7258,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034110+0000","last_change":"2024-09-14T09:08:34.626223+0000","last_active":"2024-09-14T09:09:06.034110+0000","last_peered":"2024-09-14T09:09:06.034110+0000","last_clean":"2024-09-14T09:09:06.034110+0000","last_became_active":"2024-09-14T09:07:26.102666+0000","last_became_peered":"2024-09-14T09:07:26.102666+0000","last_unstale":"2024-09-14T09:09:06.034110+0000","last_undegraded":"2024-09-14T09:09:06.034110+0000","last_fullsized":"2024-09-14T09:09:06.034110+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:03:41.035208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011116867000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10102,"num_read_kb":27247,"num_write":1623,"num_write_kb":13456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":8334712,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":200,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.950988+0000","last_change":"2024-09-14T09:07:21.863290+0000","last_active":"2024-09-14T09:09:05.950988+0000","last_peered":"2024-09-14T09:09:05.950988+0000","last_clean":"2024-09-14T09:09:05.950988+0000","last_became_active":"2024-09-14T09:07:21.862352+0000","last_became_peered":"2024-09-14T09:07:21.862352+0000","last_unstale":"2024-09-14T09:09:05.950988+0000","last_undegraded":"2024-09-14T09:09:05.950988+0000","last_fullsized":"2024-09-14T09:09:05.950988+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:48:30.332891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"179'922","reported_seq":5885,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937691+0000","last_change":"2024-09-14T09:08:50.627160+0000","last_active":"2024-09-14T09:09:05.937691+0000","last_peered":"2024-09-14T09:09:05.937691+0000","last_clean":"2024-09-14T09:09:05.937691+0000","last_became_active":"2024-09-14T09:08:50.625357+0000","last_became_peered":"2024-09-14T09:08:50.625357+0000","last_unstale":"2024-09-14T09:09:05.937691+0000","last_undegraded":"2024-09-14T09:09:05.937691+0000","last_fullsized":"2024-09-14T09:09:05.937691+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:18:31.289028+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0021152969999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7783,"num_read_kb":22826,"num_write":1134,"num_write_kb":9259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":8151041,"num_keys_recovered":91,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":356,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937696+0000","last_change":"2024-09-14T09:08:50.628307+0000","last_active":"2024-09-14T09:09:05.937696+0000","last_peered":"2024-09-14T09:09:05.937696+0000","last_clean":"2024-09-14T09:09:05.937696+0000","last_became_active":"2024-09-14T09:08:50.627971+0000","last_became_peered":"2024-09-14T09:08:50.627971+0000","last_unstale":"2024-09-14T09:09:05.937696+0000","last_undegraded":"2024-09-14T09:09:05.937696+0000","last_fullsized":"2024-09-14T09:09:05.937696+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:57.655435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"182'688","reported_seq":2922,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.865338+0000","last_change":"2024-09-14T09:08:34.495006+0000","last_active":"2024-09-14T09:09:06.865338+0000","last_peered":"2024-09-14T09:09:06.865338+0000","last_clean":"2024-09-14T09:09:06.865338+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:09:06.865338+0000","last_undegraded":"2024-09-14T09:09:06.865338+0000","last_fullsized":"2024-09-14T09:09:06.865338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:25:08.203494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030472329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3594,"num_read_kb":16894,"num_write":791,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"190'884","reported_seq":6714,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937761+0000","last_change":"2024-09-14T09:08:46.793435+0000","last_active":"2024-09-14T09:09:05.937761+0000","last_peered":"2024-09-14T09:09:05.937761+0000","last_clean":"2024-09-14T09:09:05.937761+0000","last_became_active":"2024-09-14T09:08:46.597293+0000","last_became_peered":"2024-09-14T09:08:46.597293+0000","last_unstale":"2024-09-14T09:09:05.937761+0000","last_undegraded":"2024-09-14T09:09:05.937761+0000","last_fullsized":"2024-09-14T09:09:05.937761+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:48:54.046569+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.19293607300000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8248,"num_read_kb":26883,"num_write":890,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":398,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864400+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:09:06.864400+0000","last_peered":"2024-09-14T09:09:06.864400+0000","last_clean":"2024-09-14T09:09:06.864400+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:09:06.864400+0000","last_undegraded":"2024-09-14T09:09:06.864400+0000","last_fullsized":"2024-09-14T09:09:06.864400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"180'399","reported_seq":2535,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937522+0000","last_change":"2024-09-14T09:08:50.625390+0000","last_active":"2024-09-14T09:09:05.937522+0000","last_peered":"2024-09-14T09:09:05.937522+0000","last_clean":"2024-09-14T09:09:05.937522+0000","last_became_active":"2024-09-14T09:08:50.622578+0000","last_became_peered":"2024-09-14T09:08:50.622578+0000","last_unstale":"2024-09-14T09:09:05.937522+0000","last_undegraded":"2024-09-14T09:09:05.937522+0000","last_fullsized":"2024-09-14T09:09:05.937522+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:50:11.634631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00245612,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1873,"num_read_kb":11982,"num_write":352,"num_write_kb":5539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":395,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.859085+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:09:06.859085+0000","last_peered":"2024-09-14T09:09:06.859085+0000","last_clean":"2024-09-14T09:09:06.859085+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:09:06.859085+0000","last_undegraded":"2024-09-14T09:09:06.859085+0000","last_fullsized":"2024-09-14T09:09:06.859085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"187'460","reported_seq":5079,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033075+0000","last_change":"2024-09-14T09:08:43.789977+0000","last_active":"2024-09-14T09:09:06.033075+0000","last_peered":"2024-09-14T09:09:06.033075+0000","last_clean":"2024-09-14T09:09:06.033075+0000","last_became_active":"2024-09-14T09:08:43.568923+0000","last_became_peered":"2024-09-14T09:08:43.568923+0000","last_unstale":"2024-09-14T09:09:06.033075+0000","last_undegraded":"2024-09-14T09:09:06.033075+0000","last_fullsized":"2024-09-14T09:09:06.033075+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:45:29.077592+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.165000182,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3905,"num_read_kb":25149,"num_write":375,"num_write_kb":14483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937319+0000","last_change":"2024-09-14T09:08:44.554373+0000","last_active":"2024-09-14T09:09:05.937319+0000","last_peered":"2024-09-14T09:09:05.937319+0000","last_clean":"2024-09-14T09:09:05.937319+0000","last_became_active":"2024-09-14T09:08:44.553643+0000","last_became_peered":"2024-09-14T09:08:44.553643+0000","last_unstale":"2024-09-14T09:09:05.937319+0000","last_undegraded":"2024-09-14T09:09:05.937319+0000","last_fullsized":"2024-09-14T09:09:05.937319+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:32:52.900856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"181'433","reported_seq":2980,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034173+0000","last_change":"2024-09-14T09:08:34.620964+0000","last_active":"2024-09-14T09:09:06.034173+0000","last_peered":"2024-09-14T09:09:06.034173+0000","last_clean":"2024-09-14T09:09:06.034173+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:09:06.034173+0000","last_undegraded":"2024-09-14T09:09:06.034173+0000","last_fullsized":"2024-09-14T09:09:06.034173+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:34:59.882295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065530470000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2291,"num_read_kb":11209,"num_write":432,"num_write_kb":6866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":126,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864412+0000","last_change":"2024-09-14T09:08:06.210516+0000","last_active":"2024-09-14T09:09:06.864412+0000","last_peered":"2024-09-14T09:09:06.864412+0000","last_clean":"2024-09-14T09:09:06.864412+0000","last_became_active":"2024-09-14T09:08:06.205108+0000","last_became_peered":"2024-09-14T09:08:06.205108+0000","last_unstale":"2024-09-14T09:09:06.864412+0000","last_undegraded":"2024-09-14T09:09:06.864412+0000","last_fullsized":"2024-09-14T09:09:06.864412+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:22:05.543865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":206,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949404+0000","last_change":"2024-09-14T09:07:21.876180+0000","last_active":"2024-09-14T09:09:05.949404+0000","last_peered":"2024-09-14T09:09:05.949404+0000","last_clean":"2024-09-14T09:09:05.949404+0000","last_became_active":"2024-09-14T09:07:21.860117+0000","last_became_peered":"2024-09-14T09:07:21.860117+0000","last_unstale":"2024-09-14T09:09:05.949404+0000","last_undegraded":"2024-09-14T09:09:05.949404+0000","last_fullsized":"2024-09-14T09:09:05.949404+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:58:36.142971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"178'682","reported_seq":7039,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033947+0000","last_change":"2024-09-14T09:08:34.623351+0000","last_active":"2024-09-14T09:09:06.033947+0000","last_peered":"2024-09-14T09:09:06.033947+0000","last_clean":"2024-09-14T09:09:06.033947+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:09:06.033947+0000","last_undegraded":"2024-09-14T09:09:06.033947+0000","last_fullsized":"2024-09-14T09:09:06.033947+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:48.184932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089062680000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7688,"num_read_kb":41730,"num_write":748,"num_write_kb":19769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"181'469","reported_seq":3210,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038537+0000","last_change":"2024-09-14T09:08:35.139769+0000","last_active":"2024-09-14T09:09:06.038537+0000","last_peered":"2024-09-14T09:09:06.038537+0000","last_clean":"2024-09-14T09:09:06.038537+0000","last_became_active":"2024-09-14T09:07:06.851020+0000","last_became_peered":"2024-09-14T09:07:06.851020+0000","last_unstale":"2024-09-14T09:09:06.038537+0000","last_undegraded":"2024-09-14T09:09:06.038537+0000","last_fullsized":"2024-09-14T09:09:06.038537+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:19:55.337692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154644,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2080,"num_read_kb":17783,"num_write":389,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":12315549,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"187'749","reported_seq":5084,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038554+0000","last_change":"2024-09-14T09:08:43.990954+0000","last_active":"2024-09-14T09:09:06.038554+0000","last_peered":"2024-09-14T09:09:06.038554+0000","last_clean":"2024-09-14T09:09:06.038554+0000","last_became_active":"2024-09-14T09:08:43.580245+0000","last_became_peered":"2024-09-14T09:08:43.580245+0000","last_unstale":"2024-09-14T09:09:06.038554+0000","last_undegraded":"2024-09-14T09:09:06.038554+0000","last_fullsized":"2024-09-14T09:09:06.038554+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:04.798520+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.116960367,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5499,"num_read_kb":39102,"num_write":883,"num_write_kb":19962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"181'544","reported_seq":9827,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034065+0000","last_change":"2024-09-14T09:08:34.624404+0000","last_active":"2024-09-14T09:09:06.034065+0000","last_peered":"2024-09-14T09:09:06.034065+0000","last_clean":"2024-09-14T09:09:06.034065+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:09:06.034065+0000","last_undegraded":"2024-09-14T09:09:06.034065+0000","last_fullsized":"2024-09-14T09:09:06.034065+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:54:24.811934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097700640000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10396,"num_read_kb":61123,"num_write":490,"num_write_kb":11301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"180'493","reported_seq":3394,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864519+0000","last_change":"2024-09-14T09:08:34.496008+0000","last_active":"2024-09-14T09:09:06.864519+0000","last_peered":"2024-09-14T09:09:06.864519+0000","last_clean":"2024-09-14T09:09:06.864519+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:09:06.864519+0000","last_undegraded":"2024-09-14T09:09:06.864519+0000","last_fullsized":"2024-09-14T09:09:06.864519+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:11.774708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00388507,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3266,"num_read_kb":13215,"num_write":426,"num_write_kb":10574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"179'501","reported_seq":5767,"reported_epoch":210,"state":"peering","last_fresh":"2024-09-14T09:09:06.285712+0000","last_change":"2024-09-14T09:09:07.169910+0000","last_active":"2024-09-14T09:09:07.169910+0000","last_peered":"2024-09-14T09:09:06.285712+0000","last_clean":"2024-09-14T09:08:31.463523+0000","last_became_active":"2024-09-14T09:09:05.965250+0000","last_became_peered":"2024-09-14T09:09:05.965250+0000","last_unstale":"2024-09-14T09:09:06.285712+0000","last_undegraded":"2024-09-14T09:09:06.285712+0000","last_fullsized":"2024-09-14T09:09:05.943570+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022704650000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5399,"num_read_kb":32061,"num_write":547,"num_write_kb":21536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":25396220,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"187'488","reported_seq":4034,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949818+0000","last_change":"2024-09-14T09:08:44.221618+0000","last_active":"2024-09-14T09:09:05.949818+0000","last_peered":"2024-09-14T09:09:05.949818+0000","last_clean":"2024-09-14T09:09:05.949818+0000","last_became_active":"2024-09-14T09:08:43.588233+0000","last_became_peered":"2024-09-14T09:08:43.588233+0000","last_unstale":"2024-09-14T09:09:05.949818+0000","last_undegraded":"2024-09-14T09:09:05.949818+0000","last_fullsized":"2024-09-14T09:09:05.949818+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:10.863128+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.14577486200000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4282,"num_read_kb":17652,"num_write":409,"num_write_kb":7874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":14123394,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"187'586","reported_seq":4287,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.950045+0000","last_change":"2024-09-14T09:08:44.145761+0000","last_active":"2024-09-14T09:09:05.950045+0000","last_peered":"2024-09-14T09:09:05.950045+0000","last_clean":"2024-09-14T09:09:05.950045+0000","last_became_active":"2024-09-14T09:08:43.574566+0000","last_became_peered":"2024-09-14T09:08:43.574566+0000","last_unstale":"2024-09-14T09:09:05.950045+0000","last_undegraded":"2024-09-14T09:09:05.950045+0000","last_fullsized":"2024-09-14T09:09:05.950045+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:30.502086+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.141382223,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3641,"num_read_kb":14849,"num_write":554,"num_write_kb":12244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":11864981,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'506","reported_seq":6242,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038690+0000","last_change":"2024-09-14T09:08:35.140559+0000","last_active":"2024-09-14T09:09:06.038690+0000","last_peered":"2024-09-14T09:09:06.038690+0000","last_clean":"2024-09-14T09:09:06.038690+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:09:06.038690+0000","last_undegraded":"2024-09-14T09:09:06.038690+0000","last_fullsized":"2024-09-14T09:09:06.038690+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:31:57.906202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018540480000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5422,"num_read_kb":60065,"num_write":521,"num_write_kb":20523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"187'649","reported_seq":6144,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949422+0000","last_change":"2024-09-14T09:08:43.902203+0000","last_active":"2024-09-14T09:09:05.949422+0000","last_peered":"2024-09-14T09:09:05.949422+0000","last_clean":"2024-09-14T09:09:05.949422+0000","last_became_active":"2024-09-14T09:08:43.577551+0000","last_became_peered":"2024-09-14T09:08:43.577551+0000","last_unstale":"2024-09-14T09:09:05.949422+0000","last_undegraded":"2024-09-14T09:09:05.949422+0000","last_fullsized":"2024-09-14T09:09:05.949422+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:43:37.090068+0000","scrub_duration":0,"objects_trimmed":64,"snaptrim_duration":0.121356326,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6082,"num_read_kb":13595,"num_write":653,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":7405746,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"182'790","reported_seq":6126,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033034+0000","last_change":"2024-09-14T09:08:34.493569+0000","last_active":"2024-09-14T09:09:06.033034+0000","last_peered":"2024-09-14T09:09:06.033034+0000","last_clean":"2024-09-14T09:09:06.033034+0000","last_became_active":"2024-09-14T09:07:26.101586+0000","last_became_peered":"2024-09-14T09:07:26.101586+0000","last_unstale":"2024-09-14T09:09:06.033034+0000","last_undegraded":"2024-09-14T09:09:06.033034+0000","last_fullsized":"2024-09-14T09:09:06.033034+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:11:20.433650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040594799999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7044,"num_read_kb":19519,"num_write":992,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":10260744,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"187'536","reported_seq":6572,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949474+0000","last_change":"2024-09-14T09:08:44.016919+0000","last_active":"2024-09-14T09:09:05.949474+0000","last_peered":"2024-09-14T09:09:05.949474+0000","last_clean":"2024-09-14T09:09:05.949474+0000","last_became_active":"2024-09-14T09:08:43.579997+0000","last_became_peered":"2024-09-14T09:08:43.579997+0000","last_unstale":"2024-09-14T09:09:05.949474+0000","last_undegraded":"2024-09-14T09:09:05.949474+0000","last_fullsized":"2024-09-14T09:09:05.949474+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:24:16.807890+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.10056556899999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6839,"num_read_kb":29296,"num_write":449,"num_write_kb":10682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":24080444,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":281,"num_object_clones":0,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":277,"num_read":198212,"num_read_kb":860220,"num_write":25088,"num_write_kb":385926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4177,"num_bytes_recovered":201826576,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},"log_size":22338,"ondisk_log_size":22338,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20292,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509492,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29488,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342352,"statfs":{"total":96636764160,"available":96606568448,"internally_reserved":0,"allocated":2519040,"data_stored":1219453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509492,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78076,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293764,"statfs":{"total":96636764160,"available":96556814336,"internally_reserved":0,"allocated":2523136,"data_stored":1219472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509493,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47548,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324292,"statfs":{"total":96636764160,"available":96588075008,"internally_reserved":0,"allocated":2531328,"data_stored":1225461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509495,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64232,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307608,"statfs":{"total":96636764160,"available":96570990592,"internally_reserved":0,"allocated":2576384,"data_stored":1809821,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542195,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51740,"kb_used_data":2500,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320100,"statfs":{"total":96636764160,"available":96583782400,"internally_reserved":0,"allocated":2560000,"data_stored":1809821,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":186,"seq":798863917063,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35944,"kb_used_data":2452,"kb_used_omap":62,"kb_used_meta":33409,"kb_avail":94335896,"statfs":{"total":96636764160,"available":96599957504,"internally_reserved":0,"allocated":2510848,"data_stored":1213460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":63771,"internal_metadata":34211557},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542196,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60004,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311836,"statfs":{"total":96636764160,"available":96575320064,"internally_reserved":0,"allocated":2506752,"data_stored":1213441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542197,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55852,"kb_used_data":2456,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315988,"statfs":{"total":96636764160,"available":96579571712,"internally_reserved":0,"allocated":2514944,"data_stored":1213479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:09:07.896 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:09:08.202 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:09:08.202 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:09:08.221 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-09-14T09:09:08.026480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":285,"num_object_clones":0,"num_object_copies":570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":285,"num_whiteouts":277,"num_read":198278,"num_read_kb":860275,"num_write":25191,"num_write_kb":387556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4182,"num_bytes_recovered":203007312,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22413,"ondisk_log_size":22413,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":422884,"kb_used_data":19768,"kb_used_omap":73,"kb_used_meta":402870,"kb_avail":754551836,"statfs":{"total":773094113280,"available":772661080064,"internally_reserved":0,"allocated":20242432,"data_stored":10924408,"data_compressed":13252,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":74928,"internal_metadata":412539728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001188"},"pg_stats":[{"pgid":"3.1f","version":"182'1763","reported_seq":7856,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033212+0000","last_change":"2024-09-14T09:08:34.495405+0000","last_active":"2024-09-14T09:09:06.033212+0000","last_peered":"2024-09-14T09:09:06.033212+0000","last_clean":"2024-09-14T09:09:06.033212+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:09:06.033212+0000","last_undegraded":"2024-09-14T09:09:06.033212+0000","last_fullsized":"2024-09-14T09:09:06.033212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1763,"log_dups_size":0,"ondisk_log_size":1763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:22:02.551926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022047759999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13833,"num_read_kb":24717,"num_write":2684,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'466","reported_seq":5985,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034518+0000","last_change":"2024-09-14T09:08:34.625492+0000","last_active":"2024-09-14T09:09:06.034518+0000","last_peered":"2024-09-14T09:09:06.034518+0000","last_clean":"2024-09-14T09:09:06.034518+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:09:06.034518+0000","last_undegraded":"2024-09-14T09:09:06.034518+0000","last_fullsized":"2024-09-14T09:09:06.034518+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:01.005568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010401127,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5032,"num_read_kb":31996,"num_write":418,"num_write_kb":13378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"187'676","reported_seq":6614,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.865111+0000","last_change":"2024-09-14T09:08:43.847915+0000","last_active":"2024-09-14T09:09:06.865111+0000","last_peered":"2024-09-14T09:09:06.865111+0000","last_clean":"2024-09-14T09:09:06.865111+0000","last_became_active":"2024-09-14T09:08:43.573212+0000","last_became_peered":"2024-09-14T09:08:43.573212+0000","last_unstale":"2024-09-14T09:09:06.865111+0000","last_undegraded":"2024-09-14T09:09:06.865111+0000","last_fullsized":"2024-09-14T09:09:06.865111+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:26:03.155997+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.147750145,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6669,"num_read_kb":58049,"num_write":727,"num_write_kb":12436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"190'866","reported_seq":5643,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864770+0000","last_change":"2024-09-14T09:08:46.770708+0000","last_active":"2024-09-14T09:09:06.864770+0000","last_peered":"2024-09-14T09:09:06.864770+0000","last_clean":"2024-09-14T09:09:06.864770+0000","last_became_active":"2024-09-14T09:08:46.599475+0000","last_became_peered":"2024-09-14T09:08:46.599475+0000","last_unstale":"2024-09-14T09:09:06.864770+0000","last_undegraded":"2024-09-14T09:09:06.864770+0000","last_fullsized":"2024-09-14T09:09:06.864770+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:57.157599+0000","scrub_duration":0,"objects_trimmed":60,"snaptrim_duration":0.168855903,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7679,"num_read_kb":27474,"num_write":1043,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'742","reported_seq":5713,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033232+0000","last_change":"2024-09-14T09:08:34.491862+0000","last_active":"2024-09-14T09:09:06.033232+0000","last_peered":"2024-09-14T09:09:06.033232+0000","last_clean":"2024-09-14T09:09:06.033232+0000","last_became_active":"2024-09-14T09:07:06.842337+0000","last_became_peered":"2024-09-14T09:07:06.842337+0000","last_unstale":"2024-09-14T09:09:06.033232+0000","last_undegraded":"2024-09-14T09:09:06.033232+0000","last_fullsized":"2024-09-14T09:09:06.033232+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:23:25.823078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023564419999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6703,"num_read_kb":24544,"num_write":874,"num_write_kb":16463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":14145319,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"187'724","reported_seq":5508,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937332+0000","last_change":"2024-09-14T09:08:43.771622+0000","last_active":"2024-09-14T09:09:05.937332+0000","last_peered":"2024-09-14T09:09:05.937332+0000","last_clean":"2024-09-14T09:09:05.937332+0000","last_became_active":"2024-09-14T09:08:43.556338+0000","last_became_peered":"2024-09-14T09:08:43.556338+0000","last_unstale":"2024-09-14T09:09:05.937332+0000","last_undegraded":"2024-09-14T09:09:05.937332+0000","last_fullsized":"2024-09-14T09:09:05.937332+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:41.389225+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.12752904200000001,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6399,"num_read_kb":30687,"num_write":963,"num_write_kb":14233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"174'375","reported_seq":4396,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937548+0000","last_change":"2024-09-14T09:08:50.629874+0000","last_active":"2024-09-14T09:09:05.937548+0000","last_peered":"2024-09-14T09:09:05.937548+0000","last_clean":"2024-09-14T09:09:05.937548+0000","last_became_active":"2024-09-14T09:08:50.627814+0000","last_became_peered":"2024-09-14T09:08:50.627814+0000","last_unstale":"2024-09-14T09:09:05.937548+0000","last_undegraded":"2024-09-14T09:09:05.937548+0000","last_fullsized":"2024-09-14T09:09:05.937548+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:14:43.966792+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.025218744000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3327,"num_read_kb":10838,"num_write":357,"num_write_kb":6673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":3727781,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"182'624","reported_seq":6376,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033940+0000","last_change":"2024-09-14T09:08:34.622411+0000","last_active":"2024-09-14T09:09:06.033940+0000","last_peered":"2024-09-14T09:09:06.033940+0000","last_clean":"2024-09-14T09:09:06.033940+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:09:06.033940+0000","last_undegraded":"2024-09-14T09:09:06.033940+0000","last_fullsized":"2024-09-14T09:09:06.033940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:55:38.962684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080911500000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7486,"num_read_kb":17479,"num_write":535,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"182'910","reported_seq":7825,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033222+0000","last_change":"2024-09-14T09:08:34.496061+0000","last_active":"2024-09-14T09:09:06.033222+0000","last_peered":"2024-09-14T09:09:06.033222+0000","last_clean":"2024-09-14T09:09:06.033222+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:09:06.033222+0000","last_undegraded":"2024-09-14T09:09:06.033222+0000","last_fullsized":"2024-09-14T09:09:06.033222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:02:38.220394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026116440000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10070,"num_read_kb":35479,"num_write":1129,"num_write_kb":10795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"179'629","reported_seq":3950,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864528+0000","last_change":"2024-09-14T09:08:34.493205+0000","last_active":"2024-09-14T09:09:06.864528+0000","last_peered":"2024-09-14T09:09:06.864528+0000","last_clean":"2024-09-14T09:09:06.864528+0000","last_became_active":"2024-09-14T09:07:25.105674+0000","last_became_peered":"2024-09-14T09:07:25.105674+0000","last_unstale":"2024-09-14T09:09:06.864528+0000","last_undegraded":"2024-09-14T09:09:06.864528+0000","last_fullsized":"2024-09-14T09:09:06.864528+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:20:01.627411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001781447,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4726,"num_read_kb":15599,"num_write":700,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":8467448,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"181'1136","reported_seq":5389,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.951079+0000","last_change":"2024-09-14T09:08:34.507525+0000","last_active":"2024-09-14T09:09:05.951079+0000","last_peered":"2024-09-14T09:09:05.951079+0000","last_clean":"2024-09-14T09:09:05.951079+0000","last_became_active":"2024-09-14T09:07:27.854998+0000","last_became_peered":"2024-09-14T09:07:27.854998+0000","last_unstale":"2024-09-14T09:09:05.951079+0000","last_undegraded":"2024-09-14T09:09:05.951079+0000","last_fullsized":"2024-09-14T09:09:05.951079+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:11:04.062027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001704445,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8595,"num_read_kb":26590,"num_write":1602,"num_write_kb":14717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":4170600,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"187'670","reported_seq":3455,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937564+0000","last_change":"2024-09-14T09:08:43.840463+0000","last_active":"2024-09-14T09:09:05.937564+0000","last_peered":"2024-09-14T09:09:05.937564+0000","last_clean":"2024-09-14T09:09:05.937564+0000","last_became_active":"2024-09-14T09:08:43.569123+0000","last_became_peered":"2024-09-14T09:08:43.569123+0000","last_unstale":"2024-09-14T09:09:05.937564+0000","last_undegraded":"2024-09-14T09:09:05.937564+0000","last_fullsized":"2024-09-14T09:09:05.937564+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:18.861184+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.15505904000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3583,"num_read_kb":9199,"num_write":691,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":2686146,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":403,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038664+0000","last_change":"2024-09-14T09:07:21.857426+0000","last_active":"2024-09-14T09:09:06.038664+0000","last_peered":"2024-09-14T09:09:06.038664+0000","last_clean":"2024-09-14T09:09:06.038664+0000","last_became_active":"2024-09-14T09:07:21.856579+0000","last_became_peered":"2024-09-14T09:07:21.856579+0000","last_unstale":"2024-09-14T09:09:06.038664+0000","last_undegraded":"2024-09-14T09:09:06.038664+0000","last_fullsized":"2024-09-14T09:09:06.038664+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:56.694930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"182'791","reported_seq":6693,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033604+0000","last_change":"2024-09-14T09:08:34.494518+0000","last_active":"2024-09-14T09:09:06.033604+0000","last_peered":"2024-09-14T09:09:06.033604+0000","last_clean":"2024-09-14T09:09:06.033604+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:09:06.033604+0000","last_undegraded":"2024-09-14T09:09:06.033604+0000","last_fullsized":"2024-09-14T09:09:06.033604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:58.731530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016873319999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8676,"num_read_kb":45389,"num_write":707,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":356,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937898+0000","last_change":"2024-09-14T09:08:51.290129+0000","last_active":"2024-09-14T09:09:05.937898+0000","last_peered":"2024-09-14T09:09:05.937898+0000","last_clean":"2024-09-14T09:09:05.937898+0000","last_became_active":"2024-09-14T09:08:51.289551+0000","last_became_peered":"2024-09-14T09:08:51.289551+0000","last_unstale":"2024-09-14T09:09:05.937898+0000","last_undegraded":"2024-09-14T09:09:05.937898+0000","last_fullsized":"2024-09-14T09:09:05.937898+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:52:01.486956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"182'1187","reported_seq":7258,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034110+0000","last_change":"2024-09-14T09:08:34.626223+0000","last_active":"2024-09-14T09:09:06.034110+0000","last_peered":"2024-09-14T09:09:06.034110+0000","last_clean":"2024-09-14T09:09:06.034110+0000","last_became_active":"2024-09-14T09:07:26.102666+0000","last_became_peered":"2024-09-14T09:07:26.102666+0000","last_unstale":"2024-09-14T09:09:06.034110+0000","last_undegraded":"2024-09-14T09:09:06.034110+0000","last_fullsized":"2024-09-14T09:09:06.034110+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:03:41.035208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011116867000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10102,"num_read_kb":27247,"num_write":1623,"num_write_kb":13456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":8334712,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":200,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.950988+0000","last_change":"2024-09-14T09:07:21.863290+0000","last_active":"2024-09-14T09:09:05.950988+0000","last_peered":"2024-09-14T09:09:05.950988+0000","last_clean":"2024-09-14T09:09:05.950988+0000","last_became_active":"2024-09-14T09:07:21.862352+0000","last_became_peered":"2024-09-14T09:07:21.862352+0000","last_unstale":"2024-09-14T09:09:05.950988+0000","last_undegraded":"2024-09-14T09:09:05.950988+0000","last_fullsized":"2024-09-14T09:09:05.950988+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:48:30.332891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"179'922","reported_seq":5885,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937691+0000","last_change":"2024-09-14T09:08:50.627160+0000","last_active":"2024-09-14T09:09:05.937691+0000","last_peered":"2024-09-14T09:09:05.937691+0000","last_clean":"2024-09-14T09:09:05.937691+0000","last_became_active":"2024-09-14T09:08:50.625357+0000","last_became_peered":"2024-09-14T09:08:50.625357+0000","last_unstale":"2024-09-14T09:09:05.937691+0000","last_undegraded":"2024-09-14T09:09:05.937691+0000","last_fullsized":"2024-09-14T09:09:05.937691+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:18:31.289028+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0021152969999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7783,"num_read_kb":22826,"num_write":1134,"num_write_kb":9259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":8151041,"num_keys_recovered":91,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":356,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937696+0000","last_change":"2024-09-14T09:08:50.628307+0000","last_active":"2024-09-14T09:09:05.937696+0000","last_peered":"2024-09-14T09:09:05.937696+0000","last_clean":"2024-09-14T09:09:05.937696+0000","last_became_active":"2024-09-14T09:08:50.627971+0000","last_became_peered":"2024-09-14T09:08:50.627971+0000","last_unstale":"2024-09-14T09:09:05.937696+0000","last_undegraded":"2024-09-14T09:09:05.937696+0000","last_fullsized":"2024-09-14T09:09:05.937696+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:57.655435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"182'688","reported_seq":2922,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.865338+0000","last_change":"2024-09-14T09:08:34.495006+0000","last_active":"2024-09-14T09:09:06.865338+0000","last_peered":"2024-09-14T09:09:06.865338+0000","last_clean":"2024-09-14T09:09:06.865338+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:09:06.865338+0000","last_undegraded":"2024-09-14T09:09:06.865338+0000","last_fullsized":"2024-09-14T09:09:06.865338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:25:08.203494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030472329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3594,"num_read_kb":16894,"num_write":791,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"190'884","reported_seq":6714,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937761+0000","last_change":"2024-09-14T09:08:46.793435+0000","last_active":"2024-09-14T09:09:05.937761+0000","last_peered":"2024-09-14T09:09:05.937761+0000","last_clean":"2024-09-14T09:09:05.937761+0000","last_became_active":"2024-09-14T09:08:46.597293+0000","last_became_peered":"2024-09-14T09:08:46.597293+0000","last_unstale":"2024-09-14T09:09:05.937761+0000","last_undegraded":"2024-09-14T09:09:05.937761+0000","last_fullsized":"2024-09-14T09:09:05.937761+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:48:54.046569+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.19293607300000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8248,"num_read_kb":26883,"num_write":890,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":398,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864400+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:09:06.864400+0000","last_peered":"2024-09-14T09:09:06.864400+0000","last_clean":"2024-09-14T09:09:06.864400+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:09:06.864400+0000","last_undegraded":"2024-09-14T09:09:06.864400+0000","last_fullsized":"2024-09-14T09:09:06.864400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"180'399","reported_seq":2535,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937522+0000","last_change":"2024-09-14T09:08:50.625390+0000","last_active":"2024-09-14T09:09:05.937522+0000","last_peered":"2024-09-14T09:09:05.937522+0000","last_clean":"2024-09-14T09:09:05.937522+0000","last_became_active":"2024-09-14T09:08:50.622578+0000","last_became_peered":"2024-09-14T09:08:50.622578+0000","last_unstale":"2024-09-14T09:09:05.937522+0000","last_undegraded":"2024-09-14T09:09:05.937522+0000","last_fullsized":"2024-09-14T09:09:05.937522+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:50:11.634631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00245612,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1873,"num_read_kb":11982,"num_write":352,"num_write_kb":5539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":395,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.859085+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:09:06.859085+0000","last_peered":"2024-09-14T09:09:06.859085+0000","last_clean":"2024-09-14T09:09:06.859085+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:09:06.859085+0000","last_undegraded":"2024-09-14T09:09:06.859085+0000","last_fullsized":"2024-09-14T09:09:06.859085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"187'460","reported_seq":5079,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033075+0000","last_change":"2024-09-14T09:08:43.789977+0000","last_active":"2024-09-14T09:09:06.033075+0000","last_peered":"2024-09-14T09:09:06.033075+0000","last_clean":"2024-09-14T09:09:06.033075+0000","last_became_active":"2024-09-14T09:08:43.568923+0000","last_became_peered":"2024-09-14T09:08:43.568923+0000","last_unstale":"2024-09-14T09:09:06.033075+0000","last_undegraded":"2024-09-14T09:09:06.033075+0000","last_fullsized":"2024-09-14T09:09:06.033075+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:45:29.077592+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.165000182,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3905,"num_read_kb":25149,"num_write":375,"num_write_kb":14483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937319+0000","last_change":"2024-09-14T09:08:44.554373+0000","last_active":"2024-09-14T09:09:05.937319+0000","last_peered":"2024-09-14T09:09:05.937319+0000","last_clean":"2024-09-14T09:09:05.937319+0000","last_became_active":"2024-09-14T09:08:44.553643+0000","last_became_peered":"2024-09-14T09:08:44.553643+0000","last_unstale":"2024-09-14T09:09:05.937319+0000","last_undegraded":"2024-09-14T09:09:05.937319+0000","last_fullsized":"2024-09-14T09:09:05.937319+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:32:52.900856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"181'433","reported_seq":2980,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034173+0000","last_change":"2024-09-14T09:08:34.620964+0000","last_active":"2024-09-14T09:09:06.034173+0000","last_peered":"2024-09-14T09:09:06.034173+0000","last_clean":"2024-09-14T09:09:06.034173+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:09:06.034173+0000","last_undegraded":"2024-09-14T09:09:06.034173+0000","last_fullsized":"2024-09-14T09:09:06.034173+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:34:59.882295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065530470000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2291,"num_read_kb":11209,"num_write":432,"num_write_kb":6866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":126,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864412+0000","last_change":"2024-09-14T09:08:06.210516+0000","last_active":"2024-09-14T09:09:06.864412+0000","last_peered":"2024-09-14T09:09:06.864412+0000","last_clean":"2024-09-14T09:09:06.864412+0000","last_became_active":"2024-09-14T09:08:06.205108+0000","last_became_peered":"2024-09-14T09:08:06.205108+0000","last_unstale":"2024-09-14T09:09:06.864412+0000","last_undegraded":"2024-09-14T09:09:06.864412+0000","last_fullsized":"2024-09-14T09:09:06.864412+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:22:05.543865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":206,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949404+0000","last_change":"2024-09-14T09:07:21.876180+0000","last_active":"2024-09-14T09:09:05.949404+0000","last_peered":"2024-09-14T09:09:05.949404+0000","last_clean":"2024-09-14T09:09:05.949404+0000","last_became_active":"2024-09-14T09:07:21.860117+0000","last_became_peered":"2024-09-14T09:07:21.860117+0000","last_unstale":"2024-09-14T09:09:05.949404+0000","last_undegraded":"2024-09-14T09:09:05.949404+0000","last_fullsized":"2024-09-14T09:09:05.949404+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:58:36.142971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"178'682","reported_seq":7039,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033947+0000","last_change":"2024-09-14T09:08:34.623351+0000","last_active":"2024-09-14T09:09:06.033947+0000","last_peered":"2024-09-14T09:09:06.033947+0000","last_clean":"2024-09-14T09:09:06.033947+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:09:06.033947+0000","last_undegraded":"2024-09-14T09:09:06.033947+0000","last_fullsized":"2024-09-14T09:09:06.033947+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:48.184932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089062680000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7688,"num_read_kb":41730,"num_write":748,"num_write_kb":19769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"181'469","reported_seq":3210,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038537+0000","last_change":"2024-09-14T09:08:35.139769+0000","last_active":"2024-09-14T09:09:06.038537+0000","last_peered":"2024-09-14T09:09:06.038537+0000","last_clean":"2024-09-14T09:09:06.038537+0000","last_became_active":"2024-09-14T09:07:06.851020+0000","last_became_peered":"2024-09-14T09:07:06.851020+0000","last_unstale":"2024-09-14T09:09:06.038537+0000","last_undegraded":"2024-09-14T09:09:06.038537+0000","last_fullsized":"2024-09-14T09:09:06.038537+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:19:55.337692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154644,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2080,"num_read_kb":17783,"num_write":389,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":12315549,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"187'749","reported_seq":5084,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038554+0000","last_change":"2024-09-14T09:08:43.990954+0000","last_active":"2024-09-14T09:09:06.038554+0000","last_peered":"2024-09-14T09:09:06.038554+0000","last_clean":"2024-09-14T09:09:06.038554+0000","last_became_active":"2024-09-14T09:08:43.580245+0000","last_became_peered":"2024-09-14T09:08:43.580245+0000","last_unstale":"2024-09-14T09:09:06.038554+0000","last_undegraded":"2024-09-14T09:09:06.038554+0000","last_fullsized":"2024-09-14T09:09:06.038554+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:04.798520+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.116960367,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5499,"num_read_kb":39102,"num_write":883,"num_write_kb":19962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"181'544","reported_seq":9827,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034065+0000","last_change":"2024-09-14T09:08:34.624404+0000","last_active":"2024-09-14T09:09:06.034065+0000","last_peered":"2024-09-14T09:09:06.034065+0000","last_clean":"2024-09-14T09:09:06.034065+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:09:06.034065+0000","last_undegraded":"2024-09-14T09:09:06.034065+0000","last_fullsized":"2024-09-14T09:09:06.034065+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:54:24.811934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097700640000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10396,"num_read_kb":61123,"num_write":490,"num_write_kb":11301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"180'493","reported_seq":3394,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864519+0000","last_change":"2024-09-14T09:08:34.496008+0000","last_active":"2024-09-14T09:09:06.864519+0000","last_peered":"2024-09-14T09:09:06.864519+0000","last_clean":"2024-09-14T09:09:06.864519+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:09:06.864519+0000","last_undegraded":"2024-09-14T09:09:06.864519+0000","last_fullsized":"2024-09-14T09:09:06.864519+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:11.774708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00388507,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3266,"num_read_kb":13215,"num_write":426,"num_write_kb":10574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"179'501","reported_seq":5767,"reported_epoch":210,"state":"peering","last_fresh":"2024-09-14T09:09:06.285712+0000","last_change":"2024-09-14T09:09:07.169910+0000","last_active":"2024-09-14T09:09:07.169910+0000","last_peered":"2024-09-14T09:09:06.285712+0000","last_clean":"2024-09-14T09:08:31.463523+0000","last_became_active":"2024-09-14T09:09:05.965250+0000","last_became_peered":"2024-09-14T09:09:05.965250+0000","last_unstale":"2024-09-14T09:09:06.285712+0000","last_undegraded":"2024-09-14T09:09:06.285712+0000","last_fullsized":"2024-09-14T09:09:05.943570+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022704650000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5399,"num_read_kb":32061,"num_write":547,"num_write_kb":21536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":25396220,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"187'488","reported_seq":4034,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949818+0000","last_change":"2024-09-14T09:08:44.221618+0000","last_active":"2024-09-14T09:09:05.949818+0000","last_peered":"2024-09-14T09:09:05.949818+0000","last_clean":"2024-09-14T09:09:05.949818+0000","last_became_active":"2024-09-14T09:08:43.588233+0000","last_became_peered":"2024-09-14T09:08:43.588233+0000","last_unstale":"2024-09-14T09:09:05.949818+0000","last_undegraded":"2024-09-14T09:09:05.949818+0000","last_fullsized":"2024-09-14T09:09:05.949818+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:10.863128+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.14577486200000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4282,"num_read_kb":17652,"num_write":409,"num_write_kb":7874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":14123394,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"187'586","reported_seq":4287,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.950045+0000","last_change":"2024-09-14T09:08:44.145761+0000","last_active":"2024-09-14T09:09:05.950045+0000","last_peered":"2024-09-14T09:09:05.950045+0000","last_clean":"2024-09-14T09:09:05.950045+0000","last_became_active":"2024-09-14T09:08:43.574566+0000","last_became_peered":"2024-09-14T09:08:43.574566+0000","last_unstale":"2024-09-14T09:09:05.950045+0000","last_undegraded":"2024-09-14T09:09:05.950045+0000","last_fullsized":"2024-09-14T09:09:05.950045+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:30.502086+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.141382223,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3641,"num_read_kb":14849,"num_write":554,"num_write_kb":12244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":11864981,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'506","reported_seq":6242,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038690+0000","last_change":"2024-09-14T09:08:35.140559+0000","last_active":"2024-09-14T09:09:06.038690+0000","last_peered":"2024-09-14T09:09:06.038690+0000","last_clean":"2024-09-14T09:09:06.038690+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:09:06.038690+0000","last_undegraded":"2024-09-14T09:09:06.038690+0000","last_fullsized":"2024-09-14T09:09:06.038690+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:31:57.906202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018540480000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5422,"num_read_kb":60065,"num_write":521,"num_write_kb":20523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"187'649","reported_seq":6144,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949422+0000","last_change":"2024-09-14T09:08:43.902203+0000","last_active":"2024-09-14T09:09:05.949422+0000","last_peered":"2024-09-14T09:09:05.949422+0000","last_clean":"2024-09-14T09:09:05.949422+0000","last_became_active":"2024-09-14T09:08:43.577551+0000","last_became_peered":"2024-09-14T09:08:43.577551+0000","last_unstale":"2024-09-14T09:09:05.949422+0000","last_undegraded":"2024-09-14T09:09:05.949422+0000","last_fullsized":"2024-09-14T09:09:05.949422+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:43:37.090068+0000","scrub_duration":0,"objects_trimmed":64,"snaptrim_duration":0.121356326,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6082,"num_read_kb":13595,"num_write":653,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":7405746,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"182'790","reported_seq":6126,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033034+0000","last_change":"2024-09-14T09:08:34.493569+0000","last_active":"2024-09-14T09:09:06.033034+0000","last_peered":"2024-09-14T09:09:06.033034+0000","last_clean":"2024-09-14T09:09:06.033034+0000","last_became_active":"2024-09-14T09:07:26.101586+0000","last_became_peered":"2024-09-14T09:07:26.101586+0000","last_unstale":"2024-09-14T09:09:06.033034+0000","last_undegraded":"2024-09-14T09:09:06.033034+0000","last_fullsized":"2024-09-14T09:09:06.033034+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:11:20.433650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040594799999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7044,"num_read_kb":19519,"num_write":992,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":10260744,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"187'536","reported_seq":6572,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949474+0000","last_change":"2024-09-14T09:08:44.016919+0000","last_active":"2024-09-14T09:09:05.949474+0000","last_peered":"2024-09-14T09:09:05.949474+0000","last_clean":"2024-09-14T09:09:05.949474+0000","last_became_active":"2024-09-14T09:08:43.579997+0000","last_became_peered":"2024-09-14T09:08:43.579997+0000","last_unstale":"2024-09-14T09:09:05.949474+0000","last_undegraded":"2024-09-14T09:09:05.949474+0000","last_fullsized":"2024-09-14T09:09:05.949474+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:24:16.807890+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.10056556899999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6839,"num_read_kb":29296,"num_write":449,"num_write_kb":10682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":24080444,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":281,"num_object_clones":0,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":277,"num_read":198212,"num_read_kb":860220,"num_write":25088,"num_write_kb":385926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4177,"num_bytes_recovered":201826576,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},"log_size":22338,"ondisk_log_size":22338,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20292,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509492,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29488,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342352,"statfs":{"total":96636764160,"available":96606568448,"internally_reserved":0,"allocated":2519040,"data_stored":1219453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509492,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78076,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293764,"statfs":{"total":96636764160,"available":96556814336,"internally_reserved":0,"allocated":2523136,"data_stored":1219472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509493,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47548,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324292,"statfs":{"total":96636764160,"available":96588075008,"internally_reserved":0,"allocated":2531328,"data_stored":1225461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509495,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64232,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307608,"statfs":{"total":96636764160,"available":96570990592,"internally_reserved":0,"allocated":2576384,"data_stored":1809821,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542195,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51740,"kb_used_data":2500,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320100,"statfs":{"total":96636764160,"available":96583782400,"internally_reserved":0,"allocated":2560000,"data_stored":1809821,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":186,"seq":798863917063,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35944,"kb_used_data":2452,"kb_used_omap":62,"kb_used_meta":33409,"kb_avail":94335896,"statfs":{"total":96636764160,"available":96599957504,"internally_reserved":0,"allocated":2510848,"data_stored":1213460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":63771,"internal_metadata":34211557},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542196,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60004,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311836,"statfs":{"total":96636764160,"available":96575320064,"internally_reserved":0,"allocated":2506752,"data_stored":1213441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542197,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55852,"kb_used_data":2456,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315988,"statfs":{"total":96636764160,"available":96579571712,"internally_reserved":0,"allocated":2514944,"data_stored":1213479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:09:08.222 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-14T09:09:08.222 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "179'501", 'reported_seq': 5767, 'reported_epoch': 210, 'state': 'peering', 'last_fresh': '2024-09-14T09:09:06.285712+0000', 'last_change': '2024-09-14T09:09:07.169910+0000', 'last_active': '2024-09-14T09:09:07.169910+0000', 'last_peered': '2024-09-14T09:09:06.285712+0000', 'last_clean': '2024-09-14T09:08:31.463523+0000', 'last_became_active': '2024-09-14T09:09:05.965250+0000', 'last_became_peered': '2024-09-14T09:09:05.965250+0000', 'last_unstale': '2024-09-14T09:09:06.285712+0000', 'last_undegraded': '2024-09-14T09:09:06.285712+0000', 'last_fullsized': '2024-09-14T09:09:05.943570+0000', 'mapping_epoch': 210, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 118, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-14T09:05:26.325409+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-14T09:05:26.325409+0000', 'last_clean_scrub_stamp': '2024-09-14T09:05:26.325409+0000', 'objects_scrubbed': 0, 'log_size': 501, 'log_dups_size': 0, 'ondisk_log_size': 501, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 2, 'snaptrim_duration': 0.002270465, 'stat_sum': {'num_bytes': 0, 'num_objects': 9, 'num_object_clones': 0, 'num_object_copies': 18, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9, 'num_whiteouts': 9, 'num_read': 5399, 'num_read_kb': 32061, 'num_write': 547, 'num_write_kb': 21536, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 314, 'num_bytes_recovered': 25396220, 'num_keys_recovered': 24, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-14T09:09:08.223 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-14T09:09:08.576 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:09:08.576 INFO:teuthology.orchestra.run.smithi176.stdout:{"fsid":"c635667d-994e-454f-bcea-51b45637b2a2","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":251,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":211,"num_osds":8,"num_up_osds":8,"osd_up_since":1726304922,"num_in_osds":8,"osd_in_since":1726304841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":41}],"num_pgs":41,"num_pools":3,"num_objects":285,"data_bytes":590406,"bytes_used":432766976,"bytes_avail":772661346304,"bytes_total":773094113280},"fsmap":{"epoch":1,"btime":"2024-09-14T09:04:57:032443+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-14T09:06:13.975920+0000","services":{}},"progress_events":{}} 2024-09-14T09:09:08.593 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-14T09:09:08.593 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:09:08.869 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:09:08.869 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:09:08.886 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-09-14T09:09:08.026480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":285,"num_object_clones":0,"num_object_copies":570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":285,"num_whiteouts":277,"num_read":198278,"num_read_kb":860275,"num_write":25191,"num_write_kb":387556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4182,"num_bytes_recovered":203007312,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22413,"ondisk_log_size":22413,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":422884,"kb_used_data":19768,"kb_used_omap":73,"kb_used_meta":402870,"kb_avail":754551836,"statfs":{"total":773094113280,"available":772661080064,"internally_reserved":0,"allocated":20242432,"data_stored":10924408,"data_compressed":13252,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":74928,"internal_metadata":412539728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001188"},"pg_stats":[{"pgid":"3.1f","version":"182'1763","reported_seq":7856,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033212+0000","last_change":"2024-09-14T09:08:34.495405+0000","last_active":"2024-09-14T09:09:06.033212+0000","last_peered":"2024-09-14T09:09:06.033212+0000","last_clean":"2024-09-14T09:09:06.033212+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:09:06.033212+0000","last_undegraded":"2024-09-14T09:09:06.033212+0000","last_fullsized":"2024-09-14T09:09:06.033212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1763,"log_dups_size":0,"ondisk_log_size":1763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:22:02.551926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022047759999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13833,"num_read_kb":24717,"num_write":2684,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'466","reported_seq":5985,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034518+0000","last_change":"2024-09-14T09:08:34.625492+0000","last_active":"2024-09-14T09:09:06.034518+0000","last_peered":"2024-09-14T09:09:06.034518+0000","last_clean":"2024-09-14T09:09:06.034518+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:09:06.034518+0000","last_undegraded":"2024-09-14T09:09:06.034518+0000","last_fullsized":"2024-09-14T09:09:06.034518+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:01.005568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010401127,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5032,"num_read_kb":31996,"num_write":418,"num_write_kb":13378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"187'676","reported_seq":6614,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.865111+0000","last_change":"2024-09-14T09:08:43.847915+0000","last_active":"2024-09-14T09:09:06.865111+0000","last_peered":"2024-09-14T09:09:06.865111+0000","last_clean":"2024-09-14T09:09:06.865111+0000","last_became_active":"2024-09-14T09:08:43.573212+0000","last_became_peered":"2024-09-14T09:08:43.573212+0000","last_unstale":"2024-09-14T09:09:06.865111+0000","last_undegraded":"2024-09-14T09:09:06.865111+0000","last_fullsized":"2024-09-14T09:09:06.865111+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:26:03.155997+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.147750145,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6669,"num_read_kb":58049,"num_write":727,"num_write_kb":12436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"190'866","reported_seq":5643,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864770+0000","last_change":"2024-09-14T09:08:46.770708+0000","last_active":"2024-09-14T09:09:06.864770+0000","last_peered":"2024-09-14T09:09:06.864770+0000","last_clean":"2024-09-14T09:09:06.864770+0000","last_became_active":"2024-09-14T09:08:46.599475+0000","last_became_peered":"2024-09-14T09:08:46.599475+0000","last_unstale":"2024-09-14T09:09:06.864770+0000","last_undegraded":"2024-09-14T09:09:06.864770+0000","last_fullsized":"2024-09-14T09:09:06.864770+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:57.157599+0000","scrub_duration":0,"objects_trimmed":60,"snaptrim_duration":0.168855903,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7679,"num_read_kb":27474,"num_write":1043,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'742","reported_seq":5713,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033232+0000","last_change":"2024-09-14T09:08:34.491862+0000","last_active":"2024-09-14T09:09:06.033232+0000","last_peered":"2024-09-14T09:09:06.033232+0000","last_clean":"2024-09-14T09:09:06.033232+0000","last_became_active":"2024-09-14T09:07:06.842337+0000","last_became_peered":"2024-09-14T09:07:06.842337+0000","last_unstale":"2024-09-14T09:09:06.033232+0000","last_undegraded":"2024-09-14T09:09:06.033232+0000","last_fullsized":"2024-09-14T09:09:06.033232+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:23:25.823078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023564419999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6703,"num_read_kb":24544,"num_write":874,"num_write_kb":16463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":14145319,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"187'724","reported_seq":5508,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937332+0000","last_change":"2024-09-14T09:08:43.771622+0000","last_active":"2024-09-14T09:09:05.937332+0000","last_peered":"2024-09-14T09:09:05.937332+0000","last_clean":"2024-09-14T09:09:05.937332+0000","last_became_active":"2024-09-14T09:08:43.556338+0000","last_became_peered":"2024-09-14T09:08:43.556338+0000","last_unstale":"2024-09-14T09:09:05.937332+0000","last_undegraded":"2024-09-14T09:09:05.937332+0000","last_fullsized":"2024-09-14T09:09:05.937332+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:41.389225+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.12752904200000001,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6399,"num_read_kb":30687,"num_write":963,"num_write_kb":14233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"174'375","reported_seq":4396,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937548+0000","last_change":"2024-09-14T09:08:50.629874+0000","last_active":"2024-09-14T09:09:05.937548+0000","last_peered":"2024-09-14T09:09:05.937548+0000","last_clean":"2024-09-14T09:09:05.937548+0000","last_became_active":"2024-09-14T09:08:50.627814+0000","last_became_peered":"2024-09-14T09:08:50.627814+0000","last_unstale":"2024-09-14T09:09:05.937548+0000","last_undegraded":"2024-09-14T09:09:05.937548+0000","last_fullsized":"2024-09-14T09:09:05.937548+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:14:43.966792+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.025218744000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3327,"num_read_kb":10838,"num_write":357,"num_write_kb":6673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":3727781,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"182'624","reported_seq":6376,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033940+0000","last_change":"2024-09-14T09:08:34.622411+0000","last_active":"2024-09-14T09:09:06.033940+0000","last_peered":"2024-09-14T09:09:06.033940+0000","last_clean":"2024-09-14T09:09:06.033940+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:09:06.033940+0000","last_undegraded":"2024-09-14T09:09:06.033940+0000","last_fullsized":"2024-09-14T09:09:06.033940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:55:38.962684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080911500000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7486,"num_read_kb":17479,"num_write":535,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"182'910","reported_seq":7825,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033222+0000","last_change":"2024-09-14T09:08:34.496061+0000","last_active":"2024-09-14T09:09:06.033222+0000","last_peered":"2024-09-14T09:09:06.033222+0000","last_clean":"2024-09-14T09:09:06.033222+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:09:06.033222+0000","last_undegraded":"2024-09-14T09:09:06.033222+0000","last_fullsized":"2024-09-14T09:09:06.033222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:02:38.220394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026116440000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10070,"num_read_kb":35479,"num_write":1129,"num_write_kb":10795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"179'629","reported_seq":3950,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864528+0000","last_change":"2024-09-14T09:08:34.493205+0000","last_active":"2024-09-14T09:09:06.864528+0000","last_peered":"2024-09-14T09:09:06.864528+0000","last_clean":"2024-09-14T09:09:06.864528+0000","last_became_active":"2024-09-14T09:07:25.105674+0000","last_became_peered":"2024-09-14T09:07:25.105674+0000","last_unstale":"2024-09-14T09:09:06.864528+0000","last_undegraded":"2024-09-14T09:09:06.864528+0000","last_fullsized":"2024-09-14T09:09:06.864528+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:20:01.627411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001781447,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4726,"num_read_kb":15599,"num_write":700,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":8467448,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"181'1136","reported_seq":5389,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.951079+0000","last_change":"2024-09-14T09:08:34.507525+0000","last_active":"2024-09-14T09:09:05.951079+0000","last_peered":"2024-09-14T09:09:05.951079+0000","last_clean":"2024-09-14T09:09:05.951079+0000","last_became_active":"2024-09-14T09:07:27.854998+0000","last_became_peered":"2024-09-14T09:07:27.854998+0000","last_unstale":"2024-09-14T09:09:05.951079+0000","last_undegraded":"2024-09-14T09:09:05.951079+0000","last_fullsized":"2024-09-14T09:09:05.951079+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:11:04.062027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001704445,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8595,"num_read_kb":26590,"num_write":1602,"num_write_kb":14717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":4170600,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"187'670","reported_seq":3455,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937564+0000","last_change":"2024-09-14T09:08:43.840463+0000","last_active":"2024-09-14T09:09:05.937564+0000","last_peered":"2024-09-14T09:09:05.937564+0000","last_clean":"2024-09-14T09:09:05.937564+0000","last_became_active":"2024-09-14T09:08:43.569123+0000","last_became_peered":"2024-09-14T09:08:43.569123+0000","last_unstale":"2024-09-14T09:09:05.937564+0000","last_undegraded":"2024-09-14T09:09:05.937564+0000","last_fullsized":"2024-09-14T09:09:05.937564+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:18.861184+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.15505904000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3583,"num_read_kb":9199,"num_write":691,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":2686146,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":403,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038664+0000","last_change":"2024-09-14T09:07:21.857426+0000","last_active":"2024-09-14T09:09:06.038664+0000","last_peered":"2024-09-14T09:09:06.038664+0000","last_clean":"2024-09-14T09:09:06.038664+0000","last_became_active":"2024-09-14T09:07:21.856579+0000","last_became_peered":"2024-09-14T09:07:21.856579+0000","last_unstale":"2024-09-14T09:09:06.038664+0000","last_undegraded":"2024-09-14T09:09:06.038664+0000","last_fullsized":"2024-09-14T09:09:06.038664+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:56.694930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"182'791","reported_seq":6693,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033604+0000","last_change":"2024-09-14T09:08:34.494518+0000","last_active":"2024-09-14T09:09:06.033604+0000","last_peered":"2024-09-14T09:09:06.033604+0000","last_clean":"2024-09-14T09:09:06.033604+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:09:06.033604+0000","last_undegraded":"2024-09-14T09:09:06.033604+0000","last_fullsized":"2024-09-14T09:09:06.033604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:58.731530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016873319999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8676,"num_read_kb":45389,"num_write":707,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":356,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937898+0000","last_change":"2024-09-14T09:08:51.290129+0000","last_active":"2024-09-14T09:09:05.937898+0000","last_peered":"2024-09-14T09:09:05.937898+0000","last_clean":"2024-09-14T09:09:05.937898+0000","last_became_active":"2024-09-14T09:08:51.289551+0000","last_became_peered":"2024-09-14T09:08:51.289551+0000","last_unstale":"2024-09-14T09:09:05.937898+0000","last_undegraded":"2024-09-14T09:09:05.937898+0000","last_fullsized":"2024-09-14T09:09:05.937898+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:52:01.486956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"182'1187","reported_seq":7258,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034110+0000","last_change":"2024-09-14T09:08:34.626223+0000","last_active":"2024-09-14T09:09:06.034110+0000","last_peered":"2024-09-14T09:09:06.034110+0000","last_clean":"2024-09-14T09:09:06.034110+0000","last_became_active":"2024-09-14T09:07:26.102666+0000","last_became_peered":"2024-09-14T09:07:26.102666+0000","last_unstale":"2024-09-14T09:09:06.034110+0000","last_undegraded":"2024-09-14T09:09:06.034110+0000","last_fullsized":"2024-09-14T09:09:06.034110+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:03:41.035208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011116867000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10102,"num_read_kb":27247,"num_write":1623,"num_write_kb":13456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":8334712,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":200,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.950988+0000","last_change":"2024-09-14T09:07:21.863290+0000","last_active":"2024-09-14T09:09:05.950988+0000","last_peered":"2024-09-14T09:09:05.950988+0000","last_clean":"2024-09-14T09:09:05.950988+0000","last_became_active":"2024-09-14T09:07:21.862352+0000","last_became_peered":"2024-09-14T09:07:21.862352+0000","last_unstale":"2024-09-14T09:09:05.950988+0000","last_undegraded":"2024-09-14T09:09:05.950988+0000","last_fullsized":"2024-09-14T09:09:05.950988+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:48:30.332891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"179'922","reported_seq":5885,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937691+0000","last_change":"2024-09-14T09:08:50.627160+0000","last_active":"2024-09-14T09:09:05.937691+0000","last_peered":"2024-09-14T09:09:05.937691+0000","last_clean":"2024-09-14T09:09:05.937691+0000","last_became_active":"2024-09-14T09:08:50.625357+0000","last_became_peered":"2024-09-14T09:08:50.625357+0000","last_unstale":"2024-09-14T09:09:05.937691+0000","last_undegraded":"2024-09-14T09:09:05.937691+0000","last_fullsized":"2024-09-14T09:09:05.937691+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:18:31.289028+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0021152969999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7783,"num_read_kb":22826,"num_write":1134,"num_write_kb":9259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":8151041,"num_keys_recovered":91,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":356,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937696+0000","last_change":"2024-09-14T09:08:50.628307+0000","last_active":"2024-09-14T09:09:05.937696+0000","last_peered":"2024-09-14T09:09:05.937696+0000","last_clean":"2024-09-14T09:09:05.937696+0000","last_became_active":"2024-09-14T09:08:50.627971+0000","last_became_peered":"2024-09-14T09:08:50.627971+0000","last_unstale":"2024-09-14T09:09:05.937696+0000","last_undegraded":"2024-09-14T09:09:05.937696+0000","last_fullsized":"2024-09-14T09:09:05.937696+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:57.655435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"182'688","reported_seq":2922,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.865338+0000","last_change":"2024-09-14T09:08:34.495006+0000","last_active":"2024-09-14T09:09:06.865338+0000","last_peered":"2024-09-14T09:09:06.865338+0000","last_clean":"2024-09-14T09:09:06.865338+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:09:06.865338+0000","last_undegraded":"2024-09-14T09:09:06.865338+0000","last_fullsized":"2024-09-14T09:09:06.865338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:25:08.203494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030472329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3594,"num_read_kb":16894,"num_write":791,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"190'884","reported_seq":6714,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937761+0000","last_change":"2024-09-14T09:08:46.793435+0000","last_active":"2024-09-14T09:09:05.937761+0000","last_peered":"2024-09-14T09:09:05.937761+0000","last_clean":"2024-09-14T09:09:05.937761+0000","last_became_active":"2024-09-14T09:08:46.597293+0000","last_became_peered":"2024-09-14T09:08:46.597293+0000","last_unstale":"2024-09-14T09:09:05.937761+0000","last_undegraded":"2024-09-14T09:09:05.937761+0000","last_fullsized":"2024-09-14T09:09:05.937761+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:48:54.046569+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.19293607300000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8248,"num_read_kb":26883,"num_write":890,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":398,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864400+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:09:06.864400+0000","last_peered":"2024-09-14T09:09:06.864400+0000","last_clean":"2024-09-14T09:09:06.864400+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:09:06.864400+0000","last_undegraded":"2024-09-14T09:09:06.864400+0000","last_fullsized":"2024-09-14T09:09:06.864400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"180'399","reported_seq":2535,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937522+0000","last_change":"2024-09-14T09:08:50.625390+0000","last_active":"2024-09-14T09:09:05.937522+0000","last_peered":"2024-09-14T09:09:05.937522+0000","last_clean":"2024-09-14T09:09:05.937522+0000","last_became_active":"2024-09-14T09:08:50.622578+0000","last_became_peered":"2024-09-14T09:08:50.622578+0000","last_unstale":"2024-09-14T09:09:05.937522+0000","last_undegraded":"2024-09-14T09:09:05.937522+0000","last_fullsized":"2024-09-14T09:09:05.937522+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:50:11.634631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00245612,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1873,"num_read_kb":11982,"num_write":352,"num_write_kb":5539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":395,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.859085+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:09:06.859085+0000","last_peered":"2024-09-14T09:09:06.859085+0000","last_clean":"2024-09-14T09:09:06.859085+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:09:06.859085+0000","last_undegraded":"2024-09-14T09:09:06.859085+0000","last_fullsized":"2024-09-14T09:09:06.859085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"187'460","reported_seq":5079,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033075+0000","last_change":"2024-09-14T09:08:43.789977+0000","last_active":"2024-09-14T09:09:06.033075+0000","last_peered":"2024-09-14T09:09:06.033075+0000","last_clean":"2024-09-14T09:09:06.033075+0000","last_became_active":"2024-09-14T09:08:43.568923+0000","last_became_peered":"2024-09-14T09:08:43.568923+0000","last_unstale":"2024-09-14T09:09:06.033075+0000","last_undegraded":"2024-09-14T09:09:06.033075+0000","last_fullsized":"2024-09-14T09:09:06.033075+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:45:29.077592+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.165000182,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3905,"num_read_kb":25149,"num_write":375,"num_write_kb":14483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.937319+0000","last_change":"2024-09-14T09:08:44.554373+0000","last_active":"2024-09-14T09:09:05.937319+0000","last_peered":"2024-09-14T09:09:05.937319+0000","last_clean":"2024-09-14T09:09:05.937319+0000","last_became_active":"2024-09-14T09:08:44.553643+0000","last_became_peered":"2024-09-14T09:08:44.553643+0000","last_unstale":"2024-09-14T09:09:05.937319+0000","last_undegraded":"2024-09-14T09:09:05.937319+0000","last_fullsized":"2024-09-14T09:09:05.937319+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:32:52.900856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"181'433","reported_seq":2980,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034173+0000","last_change":"2024-09-14T09:08:34.620964+0000","last_active":"2024-09-14T09:09:06.034173+0000","last_peered":"2024-09-14T09:09:06.034173+0000","last_clean":"2024-09-14T09:09:06.034173+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:09:06.034173+0000","last_undegraded":"2024-09-14T09:09:06.034173+0000","last_fullsized":"2024-09-14T09:09:06.034173+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:34:59.882295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065530470000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2291,"num_read_kb":11209,"num_write":432,"num_write_kb":6866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":126,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864412+0000","last_change":"2024-09-14T09:08:06.210516+0000","last_active":"2024-09-14T09:09:06.864412+0000","last_peered":"2024-09-14T09:09:06.864412+0000","last_clean":"2024-09-14T09:09:06.864412+0000","last_became_active":"2024-09-14T09:08:06.205108+0000","last_became_peered":"2024-09-14T09:08:06.205108+0000","last_unstale":"2024-09-14T09:09:06.864412+0000","last_undegraded":"2024-09-14T09:09:06.864412+0000","last_fullsized":"2024-09-14T09:09:06.864412+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:22:05.543865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":206,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949404+0000","last_change":"2024-09-14T09:07:21.876180+0000","last_active":"2024-09-14T09:09:05.949404+0000","last_peered":"2024-09-14T09:09:05.949404+0000","last_clean":"2024-09-14T09:09:05.949404+0000","last_became_active":"2024-09-14T09:07:21.860117+0000","last_became_peered":"2024-09-14T09:07:21.860117+0000","last_unstale":"2024-09-14T09:09:05.949404+0000","last_undegraded":"2024-09-14T09:09:05.949404+0000","last_fullsized":"2024-09-14T09:09:05.949404+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:58:36.142971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"178'682","reported_seq":7039,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033947+0000","last_change":"2024-09-14T09:08:34.623351+0000","last_active":"2024-09-14T09:09:06.033947+0000","last_peered":"2024-09-14T09:09:06.033947+0000","last_clean":"2024-09-14T09:09:06.033947+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:09:06.033947+0000","last_undegraded":"2024-09-14T09:09:06.033947+0000","last_fullsized":"2024-09-14T09:09:06.033947+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:48.184932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089062680000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7688,"num_read_kb":41730,"num_write":748,"num_write_kb":19769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"181'469","reported_seq":3210,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038537+0000","last_change":"2024-09-14T09:08:35.139769+0000","last_active":"2024-09-14T09:09:06.038537+0000","last_peered":"2024-09-14T09:09:06.038537+0000","last_clean":"2024-09-14T09:09:06.038537+0000","last_became_active":"2024-09-14T09:07:06.851020+0000","last_became_peered":"2024-09-14T09:07:06.851020+0000","last_unstale":"2024-09-14T09:09:06.038537+0000","last_undegraded":"2024-09-14T09:09:06.038537+0000","last_fullsized":"2024-09-14T09:09:06.038537+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:19:55.337692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154644,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2080,"num_read_kb":17783,"num_write":389,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":12315549,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"187'749","reported_seq":5084,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038554+0000","last_change":"2024-09-14T09:08:43.990954+0000","last_active":"2024-09-14T09:09:06.038554+0000","last_peered":"2024-09-14T09:09:06.038554+0000","last_clean":"2024-09-14T09:09:06.038554+0000","last_became_active":"2024-09-14T09:08:43.580245+0000","last_became_peered":"2024-09-14T09:08:43.580245+0000","last_unstale":"2024-09-14T09:09:06.038554+0000","last_undegraded":"2024-09-14T09:09:06.038554+0000","last_fullsized":"2024-09-14T09:09:06.038554+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:04.798520+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.116960367,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5499,"num_read_kb":39102,"num_write":883,"num_write_kb":19962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"181'544","reported_seq":9827,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.034065+0000","last_change":"2024-09-14T09:08:34.624404+0000","last_active":"2024-09-14T09:09:06.034065+0000","last_peered":"2024-09-14T09:09:06.034065+0000","last_clean":"2024-09-14T09:09:06.034065+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:09:06.034065+0000","last_undegraded":"2024-09-14T09:09:06.034065+0000","last_fullsized":"2024-09-14T09:09:06.034065+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:54:24.811934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097700640000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10396,"num_read_kb":61123,"num_write":490,"num_write_kb":11301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"180'493","reported_seq":3394,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864519+0000","last_change":"2024-09-14T09:08:34.496008+0000","last_active":"2024-09-14T09:09:06.864519+0000","last_peered":"2024-09-14T09:09:06.864519+0000","last_clean":"2024-09-14T09:09:06.864519+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:09:06.864519+0000","last_undegraded":"2024-09-14T09:09:06.864519+0000","last_fullsized":"2024-09-14T09:09:06.864519+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:11.774708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00388507,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3266,"num_read_kb":13215,"num_write":426,"num_write_kb":10574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"179'501","reported_seq":5767,"reported_epoch":210,"state":"peering","last_fresh":"2024-09-14T09:09:06.285712+0000","last_change":"2024-09-14T09:09:07.169910+0000","last_active":"2024-09-14T09:09:07.169910+0000","last_peered":"2024-09-14T09:09:06.285712+0000","last_clean":"2024-09-14T09:08:31.463523+0000","last_became_active":"2024-09-14T09:09:05.965250+0000","last_became_peered":"2024-09-14T09:09:05.965250+0000","last_unstale":"2024-09-14T09:09:06.285712+0000","last_undegraded":"2024-09-14T09:09:06.285712+0000","last_fullsized":"2024-09-14T09:09:05.943570+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022704650000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5399,"num_read_kb":32061,"num_write":547,"num_write_kb":21536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":25396220,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"187'488","reported_seq":4034,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949818+0000","last_change":"2024-09-14T09:08:44.221618+0000","last_active":"2024-09-14T09:09:05.949818+0000","last_peered":"2024-09-14T09:09:05.949818+0000","last_clean":"2024-09-14T09:09:05.949818+0000","last_became_active":"2024-09-14T09:08:43.588233+0000","last_became_peered":"2024-09-14T09:08:43.588233+0000","last_unstale":"2024-09-14T09:09:05.949818+0000","last_undegraded":"2024-09-14T09:09:05.949818+0000","last_fullsized":"2024-09-14T09:09:05.949818+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:10.863128+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.14577486200000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4282,"num_read_kb":17652,"num_write":409,"num_write_kb":7874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":14123394,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"187'586","reported_seq":4287,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.950045+0000","last_change":"2024-09-14T09:08:44.145761+0000","last_active":"2024-09-14T09:09:05.950045+0000","last_peered":"2024-09-14T09:09:05.950045+0000","last_clean":"2024-09-14T09:09:05.950045+0000","last_became_active":"2024-09-14T09:08:43.574566+0000","last_became_peered":"2024-09-14T09:08:43.574566+0000","last_unstale":"2024-09-14T09:09:05.950045+0000","last_undegraded":"2024-09-14T09:09:05.950045+0000","last_fullsized":"2024-09-14T09:09:05.950045+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:30.502086+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.141382223,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3641,"num_read_kb":14849,"num_write":554,"num_write_kb":12244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":11864981,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'506","reported_seq":6242,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038690+0000","last_change":"2024-09-14T09:08:35.140559+0000","last_active":"2024-09-14T09:09:06.038690+0000","last_peered":"2024-09-14T09:09:06.038690+0000","last_clean":"2024-09-14T09:09:06.038690+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:09:06.038690+0000","last_undegraded":"2024-09-14T09:09:06.038690+0000","last_fullsized":"2024-09-14T09:09:06.038690+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:31:57.906202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018540480000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5422,"num_read_kb":60065,"num_write":521,"num_write_kb":20523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"187'649","reported_seq":6144,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949422+0000","last_change":"2024-09-14T09:08:43.902203+0000","last_active":"2024-09-14T09:09:05.949422+0000","last_peered":"2024-09-14T09:09:05.949422+0000","last_clean":"2024-09-14T09:09:05.949422+0000","last_became_active":"2024-09-14T09:08:43.577551+0000","last_became_peered":"2024-09-14T09:08:43.577551+0000","last_unstale":"2024-09-14T09:09:05.949422+0000","last_undegraded":"2024-09-14T09:09:05.949422+0000","last_fullsized":"2024-09-14T09:09:05.949422+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:43:37.090068+0000","scrub_duration":0,"objects_trimmed":64,"snaptrim_duration":0.121356326,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6082,"num_read_kb":13595,"num_write":653,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":7405746,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"182'790","reported_seq":6126,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033034+0000","last_change":"2024-09-14T09:08:34.493569+0000","last_active":"2024-09-14T09:09:06.033034+0000","last_peered":"2024-09-14T09:09:06.033034+0000","last_clean":"2024-09-14T09:09:06.033034+0000","last_became_active":"2024-09-14T09:07:26.101586+0000","last_became_peered":"2024-09-14T09:07:26.101586+0000","last_unstale":"2024-09-14T09:09:06.033034+0000","last_undegraded":"2024-09-14T09:09:06.033034+0000","last_fullsized":"2024-09-14T09:09:06.033034+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:11:20.433650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040594799999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7044,"num_read_kb":19519,"num_write":992,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":10260744,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"187'536","reported_seq":6572,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:05.949474+0000","last_change":"2024-09-14T09:08:44.016919+0000","last_active":"2024-09-14T09:09:05.949474+0000","last_peered":"2024-09-14T09:09:05.949474+0000","last_clean":"2024-09-14T09:09:05.949474+0000","last_became_active":"2024-09-14T09:08:43.579997+0000","last_became_peered":"2024-09-14T09:08:43.579997+0000","last_unstale":"2024-09-14T09:09:05.949474+0000","last_undegraded":"2024-09-14T09:09:05.949474+0000","last_fullsized":"2024-09-14T09:09:05.949474+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:24:16.807890+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.10056556899999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6839,"num_read_kb":29296,"num_write":449,"num_write_kb":10682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":24080444,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":281,"num_object_clones":0,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":277,"num_read":198212,"num_read_kb":860220,"num_write":25088,"num_write_kb":385926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4177,"num_bytes_recovered":201826576,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},"log_size":22338,"ondisk_log_size":22338,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20292,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509492,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29488,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342352,"statfs":{"total":96636764160,"available":96606568448,"internally_reserved":0,"allocated":2519040,"data_stored":1219453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509492,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78076,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293764,"statfs":{"total":96636764160,"available":96556814336,"internally_reserved":0,"allocated":2523136,"data_stored":1219472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509493,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47548,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324292,"statfs":{"total":96636764160,"available":96588075008,"internally_reserved":0,"allocated":2531328,"data_stored":1225461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509495,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64232,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307608,"statfs":{"total":96636764160,"available":96570990592,"internally_reserved":0,"allocated":2576384,"data_stored":1809821,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542195,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51740,"kb_used_data":2500,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320100,"statfs":{"total":96636764160,"available":96583782400,"internally_reserved":0,"allocated":2560000,"data_stored":1809821,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":186,"seq":798863917063,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35944,"kb_used_data":2452,"kb_used_omap":62,"kb_used_meta":33409,"kb_avail":94335896,"statfs":{"total":96636764160,"available":96599957504,"internally_reserved":0,"allocated":2510848,"data_stored":1213460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":63771,"internal_metadata":34211557},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542196,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60004,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311836,"statfs":{"total":96636764160,"available":96575320064,"internally_reserved":0,"allocated":2506752,"data_stored":1213441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542197,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55852,"kb_used_data":2456,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315988,"statfs":{"total":96636764160,"available":96579571712,"internally_reserved":0,"allocated":2514944,"data_stored":1213479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:09:11.888 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:09:12.205 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:09:12.206 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:09:12.222 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-09-14T09:09:12.027559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":285,"num_object_clones":0,"num_object_copies":570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":285,"num_whiteouts":277,"num_read":198278,"num_read_kb":860275,"num_write":25191,"num_write_kb":387556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4182,"num_bytes_recovered":203007312,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22413,"ondisk_log_size":22413,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":423012,"kb_used_data":19900,"kb_used_omap":73,"kb_used_meta":402870,"kb_avail":754551708,"statfs":{"total":773094113280,"available":772660948992,"internally_reserved":0,"allocated":20377600,"data_stored":10990424,"data_compressed":13252,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":74928,"internal_metadata":412539728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.092394"},"pg_stats":[{"pgid":"3.1f","version":"182'1763","reported_seq":7856,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033212+0000","last_change":"2024-09-14T09:08:34.495405+0000","last_active":"2024-09-14T09:09:06.033212+0000","last_peered":"2024-09-14T09:09:06.033212+0000","last_clean":"2024-09-14T09:09:06.033212+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:09:06.033212+0000","last_undegraded":"2024-09-14T09:09:06.033212+0000","last_fullsized":"2024-09-14T09:09:06.033212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1763,"log_dups_size":0,"ondisk_log_size":1763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:22:02.551926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022047759999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13833,"num_read_kb":24717,"num_write":2684,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'466","reported_seq":5989,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.040029+0000","last_change":"2024-09-14T09:08:34.625492+0000","last_active":"2024-09-14T09:09:09.040029+0000","last_peered":"2024-09-14T09:09:09.040029+0000","last_clean":"2024-09-14T09:09:09.040029+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:09:09.040029+0000","last_undegraded":"2024-09-14T09:09:09.040029+0000","last_fullsized":"2024-09-14T09:09:09.040029+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:01.005568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010401127,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5032,"num_read_kb":31996,"num_write":418,"num_write_kb":13378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"187'676","reported_seq":6614,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.865111+0000","last_change":"2024-09-14T09:08:43.847915+0000","last_active":"2024-09-14T09:09:06.865111+0000","last_peered":"2024-09-14T09:09:06.865111+0000","last_clean":"2024-09-14T09:09:06.865111+0000","last_became_active":"2024-09-14T09:08:43.573212+0000","last_became_peered":"2024-09-14T09:08:43.573212+0000","last_unstale":"2024-09-14T09:09:06.865111+0000","last_undegraded":"2024-09-14T09:09:06.865111+0000","last_fullsized":"2024-09-14T09:09:06.865111+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:26:03.155997+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.147750145,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6669,"num_read_kb":58049,"num_write":727,"num_write_kb":12436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"190'866","reported_seq":5643,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864770+0000","last_change":"2024-09-14T09:08:46.770708+0000","last_active":"2024-09-14T09:09:06.864770+0000","last_peered":"2024-09-14T09:09:06.864770+0000","last_clean":"2024-09-14T09:09:06.864770+0000","last_became_active":"2024-09-14T09:08:46.599475+0000","last_became_peered":"2024-09-14T09:08:46.599475+0000","last_unstale":"2024-09-14T09:09:06.864770+0000","last_undegraded":"2024-09-14T09:09:06.864770+0000","last_fullsized":"2024-09-14T09:09:06.864770+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:57.157599+0000","scrub_duration":0,"objects_trimmed":60,"snaptrim_duration":0.168855903,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7679,"num_read_kb":27474,"num_write":1043,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'742","reported_seq":5713,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033232+0000","last_change":"2024-09-14T09:08:34.491862+0000","last_active":"2024-09-14T09:09:06.033232+0000","last_peered":"2024-09-14T09:09:06.033232+0000","last_clean":"2024-09-14T09:09:06.033232+0000","last_became_active":"2024-09-14T09:07:06.842337+0000","last_became_peered":"2024-09-14T09:07:06.842337+0000","last_unstale":"2024-09-14T09:09:06.033232+0000","last_undegraded":"2024-09-14T09:09:06.033232+0000","last_fullsized":"2024-09-14T09:09:06.033232+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:23:25.823078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023564419999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6703,"num_read_kb":24544,"num_write":874,"num_write_kb":16463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":14145319,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"187'724","reported_seq":5512,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.867745+0000","last_change":"2024-09-14T09:08:43.771622+0000","last_active":"2024-09-14T09:09:07.867745+0000","last_peered":"2024-09-14T09:09:07.867745+0000","last_clean":"2024-09-14T09:09:07.867745+0000","last_became_active":"2024-09-14T09:08:43.556338+0000","last_became_peered":"2024-09-14T09:08:43.556338+0000","last_unstale":"2024-09-14T09:09:07.867745+0000","last_undegraded":"2024-09-14T09:09:07.867745+0000","last_fullsized":"2024-09-14T09:09:07.867745+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:41.389225+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.12752904200000001,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6399,"num_read_kb":30687,"num_write":963,"num_write_kb":14233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"174'375","reported_seq":4400,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.862853+0000","last_change":"2024-09-14T09:08:50.629874+0000","last_active":"2024-09-14T09:09:07.862853+0000","last_peered":"2024-09-14T09:09:07.862853+0000","last_clean":"2024-09-14T09:09:07.862853+0000","last_became_active":"2024-09-14T09:08:50.627814+0000","last_became_peered":"2024-09-14T09:08:50.627814+0000","last_unstale":"2024-09-14T09:09:07.862853+0000","last_undegraded":"2024-09-14T09:09:07.862853+0000","last_fullsized":"2024-09-14T09:09:07.862853+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:14:43.966792+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.025218744000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3327,"num_read_kb":10838,"num_write":357,"num_write_kb":6673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":3727781,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"182'624","reported_seq":6380,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039812+0000","last_change":"2024-09-14T09:08:34.622411+0000","last_active":"2024-09-14T09:09:09.039812+0000","last_peered":"2024-09-14T09:09:09.039812+0000","last_clean":"2024-09-14T09:09:09.039812+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:09:09.039812+0000","last_undegraded":"2024-09-14T09:09:09.039812+0000","last_fullsized":"2024-09-14T09:09:09.039812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:55:38.962684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080911500000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7486,"num_read_kb":17479,"num_write":535,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"182'910","reported_seq":7825,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033222+0000","last_change":"2024-09-14T09:08:34.496061+0000","last_active":"2024-09-14T09:09:06.033222+0000","last_peered":"2024-09-14T09:09:06.033222+0000","last_clean":"2024-09-14T09:09:06.033222+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:09:06.033222+0000","last_undegraded":"2024-09-14T09:09:06.033222+0000","last_fullsized":"2024-09-14T09:09:06.033222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:02:38.220394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026116440000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10070,"num_read_kb":35479,"num_write":1129,"num_write_kb":10795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"179'629","reported_seq":3950,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864528+0000","last_change":"2024-09-14T09:08:34.493205+0000","last_active":"2024-09-14T09:09:06.864528+0000","last_peered":"2024-09-14T09:09:06.864528+0000","last_clean":"2024-09-14T09:09:06.864528+0000","last_became_active":"2024-09-14T09:07:25.105674+0000","last_became_peered":"2024-09-14T09:07:25.105674+0000","last_unstale":"2024-09-14T09:09:06.864528+0000","last_undegraded":"2024-09-14T09:09:06.864528+0000","last_fullsized":"2024-09-14T09:09:06.864528+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:20:01.627411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001781447,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4726,"num_read_kb":15599,"num_write":700,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":8467448,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"181'1136","reported_seq":5393,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039887+0000","last_change":"2024-09-14T09:08:34.507525+0000","last_active":"2024-09-14T09:09:08.039887+0000","last_peered":"2024-09-14T09:09:08.039887+0000","last_clean":"2024-09-14T09:09:08.039887+0000","last_became_active":"2024-09-14T09:07:27.854998+0000","last_became_peered":"2024-09-14T09:07:27.854998+0000","last_unstale":"2024-09-14T09:09:08.039887+0000","last_undegraded":"2024-09-14T09:09:08.039887+0000","last_fullsized":"2024-09-14T09:09:08.039887+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:11:04.062027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001704445,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8595,"num_read_kb":26590,"num_write":1602,"num_write_kb":14717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":4170600,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"187'670","reported_seq":3459,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.868133+0000","last_change":"2024-09-14T09:08:43.840463+0000","last_active":"2024-09-14T09:09:07.868133+0000","last_peered":"2024-09-14T09:09:07.868133+0000","last_clean":"2024-09-14T09:09:07.868133+0000","last_became_active":"2024-09-14T09:08:43.569123+0000","last_became_peered":"2024-09-14T09:08:43.569123+0000","last_unstale":"2024-09-14T09:09:07.868133+0000","last_undegraded":"2024-09-14T09:09:07.868133+0000","last_fullsized":"2024-09-14T09:09:07.868133+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:18.861184+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.15505904000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3583,"num_read_kb":9199,"num_write":691,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":2686146,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":403,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038664+0000","last_change":"2024-09-14T09:07:21.857426+0000","last_active":"2024-09-14T09:09:06.038664+0000","last_peered":"2024-09-14T09:09:06.038664+0000","last_clean":"2024-09-14T09:09:06.038664+0000","last_became_active":"2024-09-14T09:07:21.856579+0000","last_became_peered":"2024-09-14T09:07:21.856579+0000","last_unstale":"2024-09-14T09:09:06.038664+0000","last_undegraded":"2024-09-14T09:09:06.038664+0000","last_fullsized":"2024-09-14T09:09:06.038664+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:56.694930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"182'791","reported_seq":6693,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033604+0000","last_change":"2024-09-14T09:08:34.494518+0000","last_active":"2024-09-14T09:09:06.033604+0000","last_peered":"2024-09-14T09:09:06.033604+0000","last_clean":"2024-09-14T09:09:06.033604+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:09:06.033604+0000","last_undegraded":"2024-09-14T09:09:06.033604+0000","last_fullsized":"2024-09-14T09:09:06.033604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:58.731530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016873319999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8676,"num_read_kb":45389,"num_write":707,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":360,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.863161+0000","last_change":"2024-09-14T09:08:51.290129+0000","last_active":"2024-09-14T09:09:07.863161+0000","last_peered":"2024-09-14T09:09:07.863161+0000","last_clean":"2024-09-14T09:09:07.863161+0000","last_became_active":"2024-09-14T09:08:51.289551+0000","last_became_peered":"2024-09-14T09:08:51.289551+0000","last_unstale":"2024-09-14T09:09:07.863161+0000","last_undegraded":"2024-09-14T09:09:07.863161+0000","last_fullsized":"2024-09-14T09:09:07.863161+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:52:01.486956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"182'1187","reported_seq":7262,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.040014+0000","last_change":"2024-09-14T09:08:34.626223+0000","last_active":"2024-09-14T09:09:09.040014+0000","last_peered":"2024-09-14T09:09:09.040014+0000","last_clean":"2024-09-14T09:09:09.040014+0000","last_became_active":"2024-09-14T09:07:26.102666+0000","last_became_peered":"2024-09-14T09:07:26.102666+0000","last_unstale":"2024-09-14T09:09:09.040014+0000","last_undegraded":"2024-09-14T09:09:09.040014+0000","last_fullsized":"2024-09-14T09:09:09.040014+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:03:41.035208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011116867000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10102,"num_read_kb":27247,"num_write":1623,"num_write_kb":13456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":8334712,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":204,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039803+0000","last_change":"2024-09-14T09:07:21.863290+0000","last_active":"2024-09-14T09:09:08.039803+0000","last_peered":"2024-09-14T09:09:08.039803+0000","last_clean":"2024-09-14T09:09:08.039803+0000","last_became_active":"2024-09-14T09:07:21.862352+0000","last_became_peered":"2024-09-14T09:07:21.862352+0000","last_unstale":"2024-09-14T09:09:08.039803+0000","last_undegraded":"2024-09-14T09:09:08.039803+0000","last_fullsized":"2024-09-14T09:09:08.039803+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:48:30.332891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"179'922","reported_seq":5889,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.862993+0000","last_change":"2024-09-14T09:08:50.627160+0000","last_active":"2024-09-14T09:09:07.862993+0000","last_peered":"2024-09-14T09:09:07.862993+0000","last_clean":"2024-09-14T09:09:07.862993+0000","last_became_active":"2024-09-14T09:08:50.625357+0000","last_became_peered":"2024-09-14T09:08:50.625357+0000","last_unstale":"2024-09-14T09:09:07.862993+0000","last_undegraded":"2024-09-14T09:09:07.862993+0000","last_fullsized":"2024-09-14T09:09:07.862993+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:18:31.289028+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0021152969999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7783,"num_read_kb":22826,"num_write":1134,"num_write_kb":9259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":8151041,"num_keys_recovered":91,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":360,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.863248+0000","last_change":"2024-09-14T09:08:50.628307+0000","last_active":"2024-09-14T09:09:07.863248+0000","last_peered":"2024-09-14T09:09:07.863248+0000","last_clean":"2024-09-14T09:09:07.863248+0000","last_became_active":"2024-09-14T09:08:50.627971+0000","last_became_peered":"2024-09-14T09:08:50.627971+0000","last_unstale":"2024-09-14T09:09:07.863248+0000","last_undegraded":"2024-09-14T09:09:07.863248+0000","last_fullsized":"2024-09-14T09:09:07.863248+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:57.655435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"182'688","reported_seq":2922,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.865338+0000","last_change":"2024-09-14T09:08:34.495006+0000","last_active":"2024-09-14T09:09:06.865338+0000","last_peered":"2024-09-14T09:09:06.865338+0000","last_clean":"2024-09-14T09:09:06.865338+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:09:06.865338+0000","last_undegraded":"2024-09-14T09:09:06.865338+0000","last_fullsized":"2024-09-14T09:09:06.865338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:25:08.203494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030472329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3594,"num_read_kb":16894,"num_write":791,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"190'884","reported_seq":6718,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.868090+0000","last_change":"2024-09-14T09:08:46.793435+0000","last_active":"2024-09-14T09:09:07.868090+0000","last_peered":"2024-09-14T09:09:07.868090+0000","last_clean":"2024-09-14T09:09:07.868090+0000","last_became_active":"2024-09-14T09:08:46.597293+0000","last_became_peered":"2024-09-14T09:08:46.597293+0000","last_unstale":"2024-09-14T09:09:07.868090+0000","last_undegraded":"2024-09-14T09:09:07.868090+0000","last_fullsized":"2024-09-14T09:09:07.868090+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:48:54.046569+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.19293607300000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8248,"num_read_kb":26883,"num_write":890,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":398,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864400+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:09:06.864400+0000","last_peered":"2024-09-14T09:09:06.864400+0000","last_clean":"2024-09-14T09:09:06.864400+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:09:06.864400+0000","last_undegraded":"2024-09-14T09:09:06.864400+0000","last_fullsized":"2024-09-14T09:09:06.864400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"180'399","reported_seq":2539,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.863059+0000","last_change":"2024-09-14T09:08:50.625390+0000","last_active":"2024-09-14T09:09:07.863059+0000","last_peered":"2024-09-14T09:09:07.863059+0000","last_clean":"2024-09-14T09:09:07.863059+0000","last_became_active":"2024-09-14T09:08:50.622578+0000","last_became_peered":"2024-09-14T09:08:50.622578+0000","last_unstale":"2024-09-14T09:09:07.863059+0000","last_undegraded":"2024-09-14T09:09:07.863059+0000","last_fullsized":"2024-09-14T09:09:07.863059+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:50:11.634631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00245612,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1873,"num_read_kb":11982,"num_write":352,"num_write_kb":5539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":395,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.859085+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:09:06.859085+0000","last_peered":"2024-09-14T09:09:06.859085+0000","last_clean":"2024-09-14T09:09:06.859085+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:09:06.859085+0000","last_undegraded":"2024-09-14T09:09:06.859085+0000","last_fullsized":"2024-09-14T09:09:06.859085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"187'460","reported_seq":5079,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033075+0000","last_change":"2024-09-14T09:08:43.789977+0000","last_active":"2024-09-14T09:09:06.033075+0000","last_peered":"2024-09-14T09:09:06.033075+0000","last_clean":"2024-09-14T09:09:06.033075+0000","last_became_active":"2024-09-14T09:08:43.568923+0000","last_became_peered":"2024-09-14T09:08:43.568923+0000","last_unstale":"2024-09-14T09:09:06.033075+0000","last_undegraded":"2024-09-14T09:09:06.033075+0000","last_fullsized":"2024-09-14T09:09:06.033075+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:45:29.077592+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.165000182,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3905,"num_read_kb":25149,"num_write":375,"num_write_kb":14483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.867673+0000","last_change":"2024-09-14T09:08:44.554373+0000","last_active":"2024-09-14T09:09:07.867673+0000","last_peered":"2024-09-14T09:09:07.867673+0000","last_clean":"2024-09-14T09:09:07.867673+0000","last_became_active":"2024-09-14T09:08:44.553643+0000","last_became_peered":"2024-09-14T09:08:44.553643+0000","last_unstale":"2024-09-14T09:09:07.867673+0000","last_undegraded":"2024-09-14T09:09:07.867673+0000","last_fullsized":"2024-09-14T09:09:07.867673+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:32:52.900856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"181'433","reported_seq":2984,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039700+0000","last_change":"2024-09-14T09:08:34.620964+0000","last_active":"2024-09-14T09:09:09.039700+0000","last_peered":"2024-09-14T09:09:09.039700+0000","last_clean":"2024-09-14T09:09:09.039700+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:09:09.039700+0000","last_undegraded":"2024-09-14T09:09:09.039700+0000","last_fullsized":"2024-09-14T09:09:09.039700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:34:59.882295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065530470000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2291,"num_read_kb":11209,"num_write":432,"num_write_kb":6866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":126,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864412+0000","last_change":"2024-09-14T09:08:06.210516+0000","last_active":"2024-09-14T09:09:06.864412+0000","last_peered":"2024-09-14T09:09:06.864412+0000","last_clean":"2024-09-14T09:09:06.864412+0000","last_became_active":"2024-09-14T09:08:06.205108+0000","last_became_peered":"2024-09-14T09:08:06.205108+0000","last_unstale":"2024-09-14T09:09:06.864412+0000","last_undegraded":"2024-09-14T09:09:06.864412+0000","last_fullsized":"2024-09-14T09:09:06.864412+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:22:05.543865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":210,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039335+0000","last_change":"2024-09-14T09:07:21.876180+0000","last_active":"2024-09-14T09:09:08.039335+0000","last_peered":"2024-09-14T09:09:08.039335+0000","last_clean":"2024-09-14T09:09:08.039335+0000","last_became_active":"2024-09-14T09:07:21.860117+0000","last_became_peered":"2024-09-14T09:07:21.860117+0000","last_unstale":"2024-09-14T09:09:08.039335+0000","last_undegraded":"2024-09-14T09:09:08.039335+0000","last_fullsized":"2024-09-14T09:09:08.039335+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:58:36.142971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"178'682","reported_seq":7043,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039644+0000","last_change":"2024-09-14T09:08:34.623351+0000","last_active":"2024-09-14T09:09:09.039644+0000","last_peered":"2024-09-14T09:09:09.039644+0000","last_clean":"2024-09-14T09:09:09.039644+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:09:09.039644+0000","last_undegraded":"2024-09-14T09:09:09.039644+0000","last_fullsized":"2024-09-14T09:09:09.039644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:48.184932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089062680000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7688,"num_read_kb":41730,"num_write":748,"num_write_kb":19769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"181'469","reported_seq":3210,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038537+0000","last_change":"2024-09-14T09:08:35.139769+0000","last_active":"2024-09-14T09:09:06.038537+0000","last_peered":"2024-09-14T09:09:06.038537+0000","last_clean":"2024-09-14T09:09:06.038537+0000","last_became_active":"2024-09-14T09:07:06.851020+0000","last_became_peered":"2024-09-14T09:07:06.851020+0000","last_unstale":"2024-09-14T09:09:06.038537+0000","last_undegraded":"2024-09-14T09:09:06.038537+0000","last_fullsized":"2024-09-14T09:09:06.038537+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:19:55.337692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154644,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2080,"num_read_kb":17783,"num_write":389,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":12315549,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"187'749","reported_seq":5084,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038554+0000","last_change":"2024-09-14T09:08:43.990954+0000","last_active":"2024-09-14T09:09:06.038554+0000","last_peered":"2024-09-14T09:09:06.038554+0000","last_clean":"2024-09-14T09:09:06.038554+0000","last_became_active":"2024-09-14T09:08:43.580245+0000","last_became_peered":"2024-09-14T09:08:43.580245+0000","last_unstale":"2024-09-14T09:09:06.038554+0000","last_undegraded":"2024-09-14T09:09:06.038554+0000","last_fullsized":"2024-09-14T09:09:06.038554+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:04.798520+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.116960367,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5499,"num_read_kb":39102,"num_write":883,"num_write_kb":19962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"181'544","reported_seq":9831,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039772+0000","last_change":"2024-09-14T09:08:34.624404+0000","last_active":"2024-09-14T09:09:09.039772+0000","last_peered":"2024-09-14T09:09:09.039772+0000","last_clean":"2024-09-14T09:09:09.039772+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:09:09.039772+0000","last_undegraded":"2024-09-14T09:09:09.039772+0000","last_fullsized":"2024-09-14T09:09:09.039772+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:54:24.811934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097700640000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10396,"num_read_kb":61123,"num_write":490,"num_write_kb":11301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"180'493","reported_seq":3394,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864519+0000","last_change":"2024-09-14T09:08:34.496008+0000","last_active":"2024-09-14T09:09:06.864519+0000","last_peered":"2024-09-14T09:09:06.864519+0000","last_clean":"2024-09-14T09:09:06.864519+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:09:06.864519+0000","last_undegraded":"2024-09-14T09:09:06.864519+0000","last_fullsized":"2024-09-14T09:09:06.864519+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:11.774708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00388507,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3266,"num_read_kb":13215,"num_write":426,"num_write_kb":10574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"179'501","reported_seq":5767,"reported_epoch":210,"state":"peering","last_fresh":"2024-09-14T09:09:06.285712+0000","last_change":"2024-09-14T09:09:07.169910+0000","last_active":"2024-09-14T09:09:07.169910+0000","last_peered":"2024-09-14T09:09:06.285712+0000","last_clean":"2024-09-14T09:08:31.463523+0000","last_became_active":"2024-09-14T09:09:05.965250+0000","last_became_peered":"2024-09-14T09:09:05.965250+0000","last_unstale":"2024-09-14T09:09:06.285712+0000","last_undegraded":"2024-09-14T09:09:06.285712+0000","last_fullsized":"2024-09-14T09:09:05.943570+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022704650000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5399,"num_read_kb":32061,"num_write":547,"num_write_kb":21536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":25396220,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"187'488","reported_seq":4038,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039610+0000","last_change":"2024-09-14T09:08:44.221618+0000","last_active":"2024-09-14T09:09:08.039610+0000","last_peered":"2024-09-14T09:09:08.039610+0000","last_clean":"2024-09-14T09:09:08.039610+0000","last_became_active":"2024-09-14T09:08:43.588233+0000","last_became_peered":"2024-09-14T09:08:43.588233+0000","last_unstale":"2024-09-14T09:09:08.039610+0000","last_undegraded":"2024-09-14T09:09:08.039610+0000","last_fullsized":"2024-09-14T09:09:08.039610+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:10.863128+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.14577486200000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4282,"num_read_kb":17652,"num_write":409,"num_write_kb":7874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":14123394,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"187'586","reported_seq":4291,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039835+0000","last_change":"2024-09-14T09:08:44.145761+0000","last_active":"2024-09-14T09:09:08.039835+0000","last_peered":"2024-09-14T09:09:08.039835+0000","last_clean":"2024-09-14T09:09:08.039835+0000","last_became_active":"2024-09-14T09:08:43.574566+0000","last_became_peered":"2024-09-14T09:08:43.574566+0000","last_unstale":"2024-09-14T09:09:08.039835+0000","last_undegraded":"2024-09-14T09:09:08.039835+0000","last_fullsized":"2024-09-14T09:09:08.039835+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:30.502086+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.141382223,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3641,"num_read_kb":14849,"num_write":554,"num_write_kb":12244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":11864981,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'506","reported_seq":6242,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038690+0000","last_change":"2024-09-14T09:08:35.140559+0000","last_active":"2024-09-14T09:09:06.038690+0000","last_peered":"2024-09-14T09:09:06.038690+0000","last_clean":"2024-09-14T09:09:06.038690+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:09:06.038690+0000","last_undegraded":"2024-09-14T09:09:06.038690+0000","last_fullsized":"2024-09-14T09:09:06.038690+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:31:57.906202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018540480000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5422,"num_read_kb":60065,"num_write":521,"num_write_kb":20523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"187'649","reported_seq":6148,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039335+0000","last_change":"2024-09-14T09:08:43.902203+0000","last_active":"2024-09-14T09:09:08.039335+0000","last_peered":"2024-09-14T09:09:08.039335+0000","last_clean":"2024-09-14T09:09:08.039335+0000","last_became_active":"2024-09-14T09:08:43.577551+0000","last_became_peered":"2024-09-14T09:08:43.577551+0000","last_unstale":"2024-09-14T09:09:08.039335+0000","last_undegraded":"2024-09-14T09:09:08.039335+0000","last_fullsized":"2024-09-14T09:09:08.039335+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:43:37.090068+0000","scrub_duration":0,"objects_trimmed":64,"snaptrim_duration":0.121356326,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6082,"num_read_kb":13595,"num_write":653,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":7405746,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"182'790","reported_seq":6126,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033034+0000","last_change":"2024-09-14T09:08:34.493569+0000","last_active":"2024-09-14T09:09:06.033034+0000","last_peered":"2024-09-14T09:09:06.033034+0000","last_clean":"2024-09-14T09:09:06.033034+0000","last_became_active":"2024-09-14T09:07:26.101586+0000","last_became_peered":"2024-09-14T09:07:26.101586+0000","last_unstale":"2024-09-14T09:09:06.033034+0000","last_undegraded":"2024-09-14T09:09:06.033034+0000","last_fullsized":"2024-09-14T09:09:06.033034+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:11:20.433650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040594799999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7044,"num_read_kb":19519,"num_write":992,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":10260744,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"187'536","reported_seq":6576,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039395+0000","last_change":"2024-09-14T09:08:44.016919+0000","last_active":"2024-09-14T09:09:08.039395+0000","last_peered":"2024-09-14T09:09:08.039395+0000","last_clean":"2024-09-14T09:09:08.039395+0000","last_became_active":"2024-09-14T09:08:43.579997+0000","last_became_peered":"2024-09-14T09:08:43.579997+0000","last_unstale":"2024-09-14T09:09:08.039395+0000","last_undegraded":"2024-09-14T09:09:08.039395+0000","last_fullsized":"2024-09-14T09:09:08.039395+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:24:16.807890+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.10056556899999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6839,"num_read_kb":29296,"num_write":449,"num_write_kb":10682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":24080444,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":281,"num_object_clones":0,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":277,"num_read":198212,"num_read_kb":860220,"num_write":25088,"num_write_kb":385926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4177,"num_bytes_recovered":201826576,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},"log_size":22338,"ondisk_log_size":22338,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20292,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509492,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29488,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342352,"statfs":{"total":96636764160,"available":96606568448,"internally_reserved":0,"allocated":2519040,"data_stored":1219453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509492,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78076,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293764,"statfs":{"total":96636764160,"available":96556814336,"internally_reserved":0,"allocated":2523136,"data_stored":1219472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509493,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47548,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324292,"statfs":{"total":96636764160,"available":96588075008,"internally_reserved":0,"allocated":2531328,"data_stored":1225461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509495,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64232,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307608,"statfs":{"total":96636764160,"available":96570990592,"internally_reserved":0,"allocated":2576384,"data_stored":1809821,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542196,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51760,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320080,"statfs":{"total":96636764160,"available":96583761920,"internally_reserved":0,"allocated":2584576,"data_stored":1821816,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":186,"seq":798863917064,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35980,"kb_used_data":2488,"kb_used_omap":62,"kb_used_meta":33409,"kb_avail":94335860,"statfs":{"total":96636764160,"available":96599920640,"internally_reserved":0,"allocated":2547712,"data_stored":1231467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":63771,"internal_metadata":34211557},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542197,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60040,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311800,"statfs":{"total":96636764160,"available":96575283200,"internally_reserved":0,"allocated":2543616,"data_stored":1231448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542198,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55888,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315952,"statfs":{"total":96636764160,"available":96579534848,"internally_reserved":0,"allocated":2551808,"data_stored":1231486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:09:12.224 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:09:12.523 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:09:12.524 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:09:12.541 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-09-14T09:09:12.027559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":285,"num_object_clones":0,"num_object_copies":570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":285,"num_whiteouts":277,"num_read":198278,"num_read_kb":860275,"num_write":25191,"num_write_kb":387556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4182,"num_bytes_recovered":203007312,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22413,"ondisk_log_size":22413,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":423012,"kb_used_data":19900,"kb_used_omap":73,"kb_used_meta":402870,"kb_avail":754551708,"statfs":{"total":773094113280,"available":772660948992,"internally_reserved":0,"allocated":20377600,"data_stored":10990424,"data_compressed":13252,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":74928,"internal_metadata":412539728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.092394"},"pg_stats":[{"pgid":"3.1f","version":"182'1763","reported_seq":7856,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033212+0000","last_change":"2024-09-14T09:08:34.495405+0000","last_active":"2024-09-14T09:09:06.033212+0000","last_peered":"2024-09-14T09:09:06.033212+0000","last_clean":"2024-09-14T09:09:06.033212+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:09:06.033212+0000","last_undegraded":"2024-09-14T09:09:06.033212+0000","last_fullsized":"2024-09-14T09:09:06.033212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1763,"log_dups_size":0,"ondisk_log_size":1763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:22:02.551926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022047759999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13833,"num_read_kb":24717,"num_write":2684,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'466","reported_seq":5989,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.040029+0000","last_change":"2024-09-14T09:08:34.625492+0000","last_active":"2024-09-14T09:09:09.040029+0000","last_peered":"2024-09-14T09:09:09.040029+0000","last_clean":"2024-09-14T09:09:09.040029+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:09:09.040029+0000","last_undegraded":"2024-09-14T09:09:09.040029+0000","last_fullsized":"2024-09-14T09:09:09.040029+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:01.005568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010401127,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5032,"num_read_kb":31996,"num_write":418,"num_write_kb":13378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"187'676","reported_seq":6614,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.865111+0000","last_change":"2024-09-14T09:08:43.847915+0000","last_active":"2024-09-14T09:09:06.865111+0000","last_peered":"2024-09-14T09:09:06.865111+0000","last_clean":"2024-09-14T09:09:06.865111+0000","last_became_active":"2024-09-14T09:08:43.573212+0000","last_became_peered":"2024-09-14T09:08:43.573212+0000","last_unstale":"2024-09-14T09:09:06.865111+0000","last_undegraded":"2024-09-14T09:09:06.865111+0000","last_fullsized":"2024-09-14T09:09:06.865111+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:26:03.155997+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.147750145,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6669,"num_read_kb":58049,"num_write":727,"num_write_kb":12436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"190'866","reported_seq":5643,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864770+0000","last_change":"2024-09-14T09:08:46.770708+0000","last_active":"2024-09-14T09:09:06.864770+0000","last_peered":"2024-09-14T09:09:06.864770+0000","last_clean":"2024-09-14T09:09:06.864770+0000","last_became_active":"2024-09-14T09:08:46.599475+0000","last_became_peered":"2024-09-14T09:08:46.599475+0000","last_unstale":"2024-09-14T09:09:06.864770+0000","last_undegraded":"2024-09-14T09:09:06.864770+0000","last_fullsized":"2024-09-14T09:09:06.864770+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:57.157599+0000","scrub_duration":0,"objects_trimmed":60,"snaptrim_duration":0.168855903,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7679,"num_read_kb":27474,"num_write":1043,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'742","reported_seq":5713,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033232+0000","last_change":"2024-09-14T09:08:34.491862+0000","last_active":"2024-09-14T09:09:06.033232+0000","last_peered":"2024-09-14T09:09:06.033232+0000","last_clean":"2024-09-14T09:09:06.033232+0000","last_became_active":"2024-09-14T09:07:06.842337+0000","last_became_peered":"2024-09-14T09:07:06.842337+0000","last_unstale":"2024-09-14T09:09:06.033232+0000","last_undegraded":"2024-09-14T09:09:06.033232+0000","last_fullsized":"2024-09-14T09:09:06.033232+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:23:25.823078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023564419999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6703,"num_read_kb":24544,"num_write":874,"num_write_kb":16463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":14145319,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"187'724","reported_seq":5512,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.867745+0000","last_change":"2024-09-14T09:08:43.771622+0000","last_active":"2024-09-14T09:09:07.867745+0000","last_peered":"2024-09-14T09:09:07.867745+0000","last_clean":"2024-09-14T09:09:07.867745+0000","last_became_active":"2024-09-14T09:08:43.556338+0000","last_became_peered":"2024-09-14T09:08:43.556338+0000","last_unstale":"2024-09-14T09:09:07.867745+0000","last_undegraded":"2024-09-14T09:09:07.867745+0000","last_fullsized":"2024-09-14T09:09:07.867745+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:41.389225+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.12752904200000001,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6399,"num_read_kb":30687,"num_write":963,"num_write_kb":14233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"174'375","reported_seq":4400,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.862853+0000","last_change":"2024-09-14T09:08:50.629874+0000","last_active":"2024-09-14T09:09:07.862853+0000","last_peered":"2024-09-14T09:09:07.862853+0000","last_clean":"2024-09-14T09:09:07.862853+0000","last_became_active":"2024-09-14T09:08:50.627814+0000","last_became_peered":"2024-09-14T09:08:50.627814+0000","last_unstale":"2024-09-14T09:09:07.862853+0000","last_undegraded":"2024-09-14T09:09:07.862853+0000","last_fullsized":"2024-09-14T09:09:07.862853+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:14:43.966792+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.025218744000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3327,"num_read_kb":10838,"num_write":357,"num_write_kb":6673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":3727781,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"182'624","reported_seq":6380,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039812+0000","last_change":"2024-09-14T09:08:34.622411+0000","last_active":"2024-09-14T09:09:09.039812+0000","last_peered":"2024-09-14T09:09:09.039812+0000","last_clean":"2024-09-14T09:09:09.039812+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:09:09.039812+0000","last_undegraded":"2024-09-14T09:09:09.039812+0000","last_fullsized":"2024-09-14T09:09:09.039812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:55:38.962684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080911500000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7486,"num_read_kb":17479,"num_write":535,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"182'910","reported_seq":7825,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033222+0000","last_change":"2024-09-14T09:08:34.496061+0000","last_active":"2024-09-14T09:09:06.033222+0000","last_peered":"2024-09-14T09:09:06.033222+0000","last_clean":"2024-09-14T09:09:06.033222+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:09:06.033222+0000","last_undegraded":"2024-09-14T09:09:06.033222+0000","last_fullsized":"2024-09-14T09:09:06.033222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:02:38.220394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026116440000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10070,"num_read_kb":35479,"num_write":1129,"num_write_kb":10795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"179'629","reported_seq":3950,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864528+0000","last_change":"2024-09-14T09:08:34.493205+0000","last_active":"2024-09-14T09:09:06.864528+0000","last_peered":"2024-09-14T09:09:06.864528+0000","last_clean":"2024-09-14T09:09:06.864528+0000","last_became_active":"2024-09-14T09:07:25.105674+0000","last_became_peered":"2024-09-14T09:07:25.105674+0000","last_unstale":"2024-09-14T09:09:06.864528+0000","last_undegraded":"2024-09-14T09:09:06.864528+0000","last_fullsized":"2024-09-14T09:09:06.864528+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:20:01.627411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001781447,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4726,"num_read_kb":15599,"num_write":700,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":8467448,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"181'1136","reported_seq":5393,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039887+0000","last_change":"2024-09-14T09:08:34.507525+0000","last_active":"2024-09-14T09:09:08.039887+0000","last_peered":"2024-09-14T09:09:08.039887+0000","last_clean":"2024-09-14T09:09:08.039887+0000","last_became_active":"2024-09-14T09:07:27.854998+0000","last_became_peered":"2024-09-14T09:07:27.854998+0000","last_unstale":"2024-09-14T09:09:08.039887+0000","last_undegraded":"2024-09-14T09:09:08.039887+0000","last_fullsized":"2024-09-14T09:09:08.039887+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:11:04.062027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001704445,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8595,"num_read_kb":26590,"num_write":1602,"num_write_kb":14717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":4170600,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"187'670","reported_seq":3459,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.868133+0000","last_change":"2024-09-14T09:08:43.840463+0000","last_active":"2024-09-14T09:09:07.868133+0000","last_peered":"2024-09-14T09:09:07.868133+0000","last_clean":"2024-09-14T09:09:07.868133+0000","last_became_active":"2024-09-14T09:08:43.569123+0000","last_became_peered":"2024-09-14T09:08:43.569123+0000","last_unstale":"2024-09-14T09:09:07.868133+0000","last_undegraded":"2024-09-14T09:09:07.868133+0000","last_fullsized":"2024-09-14T09:09:07.868133+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:18.861184+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.15505904000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3583,"num_read_kb":9199,"num_write":691,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":2686146,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":403,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038664+0000","last_change":"2024-09-14T09:07:21.857426+0000","last_active":"2024-09-14T09:09:06.038664+0000","last_peered":"2024-09-14T09:09:06.038664+0000","last_clean":"2024-09-14T09:09:06.038664+0000","last_became_active":"2024-09-14T09:07:21.856579+0000","last_became_peered":"2024-09-14T09:07:21.856579+0000","last_unstale":"2024-09-14T09:09:06.038664+0000","last_undegraded":"2024-09-14T09:09:06.038664+0000","last_fullsized":"2024-09-14T09:09:06.038664+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:56.694930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"182'791","reported_seq":6693,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033604+0000","last_change":"2024-09-14T09:08:34.494518+0000","last_active":"2024-09-14T09:09:06.033604+0000","last_peered":"2024-09-14T09:09:06.033604+0000","last_clean":"2024-09-14T09:09:06.033604+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:09:06.033604+0000","last_undegraded":"2024-09-14T09:09:06.033604+0000","last_fullsized":"2024-09-14T09:09:06.033604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:58.731530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016873319999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8676,"num_read_kb":45389,"num_write":707,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":360,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.863161+0000","last_change":"2024-09-14T09:08:51.290129+0000","last_active":"2024-09-14T09:09:07.863161+0000","last_peered":"2024-09-14T09:09:07.863161+0000","last_clean":"2024-09-14T09:09:07.863161+0000","last_became_active":"2024-09-14T09:08:51.289551+0000","last_became_peered":"2024-09-14T09:08:51.289551+0000","last_unstale":"2024-09-14T09:09:07.863161+0000","last_undegraded":"2024-09-14T09:09:07.863161+0000","last_fullsized":"2024-09-14T09:09:07.863161+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:52:01.486956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"182'1187","reported_seq":7262,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.040014+0000","last_change":"2024-09-14T09:08:34.626223+0000","last_active":"2024-09-14T09:09:09.040014+0000","last_peered":"2024-09-14T09:09:09.040014+0000","last_clean":"2024-09-14T09:09:09.040014+0000","last_became_active":"2024-09-14T09:07:26.102666+0000","last_became_peered":"2024-09-14T09:07:26.102666+0000","last_unstale":"2024-09-14T09:09:09.040014+0000","last_undegraded":"2024-09-14T09:09:09.040014+0000","last_fullsized":"2024-09-14T09:09:09.040014+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:03:41.035208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011116867000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10102,"num_read_kb":27247,"num_write":1623,"num_write_kb":13456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":8334712,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":204,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039803+0000","last_change":"2024-09-14T09:07:21.863290+0000","last_active":"2024-09-14T09:09:08.039803+0000","last_peered":"2024-09-14T09:09:08.039803+0000","last_clean":"2024-09-14T09:09:08.039803+0000","last_became_active":"2024-09-14T09:07:21.862352+0000","last_became_peered":"2024-09-14T09:07:21.862352+0000","last_unstale":"2024-09-14T09:09:08.039803+0000","last_undegraded":"2024-09-14T09:09:08.039803+0000","last_fullsized":"2024-09-14T09:09:08.039803+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:48:30.332891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"179'922","reported_seq":5889,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.862993+0000","last_change":"2024-09-14T09:08:50.627160+0000","last_active":"2024-09-14T09:09:07.862993+0000","last_peered":"2024-09-14T09:09:07.862993+0000","last_clean":"2024-09-14T09:09:07.862993+0000","last_became_active":"2024-09-14T09:08:50.625357+0000","last_became_peered":"2024-09-14T09:08:50.625357+0000","last_unstale":"2024-09-14T09:09:07.862993+0000","last_undegraded":"2024-09-14T09:09:07.862993+0000","last_fullsized":"2024-09-14T09:09:07.862993+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:18:31.289028+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0021152969999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7783,"num_read_kb":22826,"num_write":1134,"num_write_kb":9259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":8151041,"num_keys_recovered":91,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":360,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.863248+0000","last_change":"2024-09-14T09:08:50.628307+0000","last_active":"2024-09-14T09:09:07.863248+0000","last_peered":"2024-09-14T09:09:07.863248+0000","last_clean":"2024-09-14T09:09:07.863248+0000","last_became_active":"2024-09-14T09:08:50.627971+0000","last_became_peered":"2024-09-14T09:08:50.627971+0000","last_unstale":"2024-09-14T09:09:07.863248+0000","last_undegraded":"2024-09-14T09:09:07.863248+0000","last_fullsized":"2024-09-14T09:09:07.863248+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:57.655435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"182'688","reported_seq":2922,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.865338+0000","last_change":"2024-09-14T09:08:34.495006+0000","last_active":"2024-09-14T09:09:06.865338+0000","last_peered":"2024-09-14T09:09:06.865338+0000","last_clean":"2024-09-14T09:09:06.865338+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:09:06.865338+0000","last_undegraded":"2024-09-14T09:09:06.865338+0000","last_fullsized":"2024-09-14T09:09:06.865338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:25:08.203494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030472329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3594,"num_read_kb":16894,"num_write":791,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"190'884","reported_seq":6718,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.868090+0000","last_change":"2024-09-14T09:08:46.793435+0000","last_active":"2024-09-14T09:09:07.868090+0000","last_peered":"2024-09-14T09:09:07.868090+0000","last_clean":"2024-09-14T09:09:07.868090+0000","last_became_active":"2024-09-14T09:08:46.597293+0000","last_became_peered":"2024-09-14T09:08:46.597293+0000","last_unstale":"2024-09-14T09:09:07.868090+0000","last_undegraded":"2024-09-14T09:09:07.868090+0000","last_fullsized":"2024-09-14T09:09:07.868090+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:48:54.046569+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.19293607300000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8248,"num_read_kb":26883,"num_write":890,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":398,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864400+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:09:06.864400+0000","last_peered":"2024-09-14T09:09:06.864400+0000","last_clean":"2024-09-14T09:09:06.864400+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:09:06.864400+0000","last_undegraded":"2024-09-14T09:09:06.864400+0000","last_fullsized":"2024-09-14T09:09:06.864400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"180'399","reported_seq":2539,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.863059+0000","last_change":"2024-09-14T09:08:50.625390+0000","last_active":"2024-09-14T09:09:07.863059+0000","last_peered":"2024-09-14T09:09:07.863059+0000","last_clean":"2024-09-14T09:09:07.863059+0000","last_became_active":"2024-09-14T09:08:50.622578+0000","last_became_peered":"2024-09-14T09:08:50.622578+0000","last_unstale":"2024-09-14T09:09:07.863059+0000","last_undegraded":"2024-09-14T09:09:07.863059+0000","last_fullsized":"2024-09-14T09:09:07.863059+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:50:11.634631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00245612,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1873,"num_read_kb":11982,"num_write":352,"num_write_kb":5539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":395,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.859085+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:09:06.859085+0000","last_peered":"2024-09-14T09:09:06.859085+0000","last_clean":"2024-09-14T09:09:06.859085+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:09:06.859085+0000","last_undegraded":"2024-09-14T09:09:06.859085+0000","last_fullsized":"2024-09-14T09:09:06.859085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"187'460","reported_seq":5079,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033075+0000","last_change":"2024-09-14T09:08:43.789977+0000","last_active":"2024-09-14T09:09:06.033075+0000","last_peered":"2024-09-14T09:09:06.033075+0000","last_clean":"2024-09-14T09:09:06.033075+0000","last_became_active":"2024-09-14T09:08:43.568923+0000","last_became_peered":"2024-09-14T09:08:43.568923+0000","last_unstale":"2024-09-14T09:09:06.033075+0000","last_undegraded":"2024-09-14T09:09:06.033075+0000","last_fullsized":"2024-09-14T09:09:06.033075+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:45:29.077592+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.165000182,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3905,"num_read_kb":25149,"num_write":375,"num_write_kb":14483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.867673+0000","last_change":"2024-09-14T09:08:44.554373+0000","last_active":"2024-09-14T09:09:07.867673+0000","last_peered":"2024-09-14T09:09:07.867673+0000","last_clean":"2024-09-14T09:09:07.867673+0000","last_became_active":"2024-09-14T09:08:44.553643+0000","last_became_peered":"2024-09-14T09:08:44.553643+0000","last_unstale":"2024-09-14T09:09:07.867673+0000","last_undegraded":"2024-09-14T09:09:07.867673+0000","last_fullsized":"2024-09-14T09:09:07.867673+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:32:52.900856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"181'433","reported_seq":2984,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039700+0000","last_change":"2024-09-14T09:08:34.620964+0000","last_active":"2024-09-14T09:09:09.039700+0000","last_peered":"2024-09-14T09:09:09.039700+0000","last_clean":"2024-09-14T09:09:09.039700+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:09:09.039700+0000","last_undegraded":"2024-09-14T09:09:09.039700+0000","last_fullsized":"2024-09-14T09:09:09.039700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:34:59.882295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065530470000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2291,"num_read_kb":11209,"num_write":432,"num_write_kb":6866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":126,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864412+0000","last_change":"2024-09-14T09:08:06.210516+0000","last_active":"2024-09-14T09:09:06.864412+0000","last_peered":"2024-09-14T09:09:06.864412+0000","last_clean":"2024-09-14T09:09:06.864412+0000","last_became_active":"2024-09-14T09:08:06.205108+0000","last_became_peered":"2024-09-14T09:08:06.205108+0000","last_unstale":"2024-09-14T09:09:06.864412+0000","last_undegraded":"2024-09-14T09:09:06.864412+0000","last_fullsized":"2024-09-14T09:09:06.864412+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:22:05.543865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":210,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039335+0000","last_change":"2024-09-14T09:07:21.876180+0000","last_active":"2024-09-14T09:09:08.039335+0000","last_peered":"2024-09-14T09:09:08.039335+0000","last_clean":"2024-09-14T09:09:08.039335+0000","last_became_active":"2024-09-14T09:07:21.860117+0000","last_became_peered":"2024-09-14T09:07:21.860117+0000","last_unstale":"2024-09-14T09:09:08.039335+0000","last_undegraded":"2024-09-14T09:09:08.039335+0000","last_fullsized":"2024-09-14T09:09:08.039335+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:58:36.142971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"178'682","reported_seq":7043,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039644+0000","last_change":"2024-09-14T09:08:34.623351+0000","last_active":"2024-09-14T09:09:09.039644+0000","last_peered":"2024-09-14T09:09:09.039644+0000","last_clean":"2024-09-14T09:09:09.039644+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:09:09.039644+0000","last_undegraded":"2024-09-14T09:09:09.039644+0000","last_fullsized":"2024-09-14T09:09:09.039644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:48.184932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089062680000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7688,"num_read_kb":41730,"num_write":748,"num_write_kb":19769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"181'469","reported_seq":3210,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038537+0000","last_change":"2024-09-14T09:08:35.139769+0000","last_active":"2024-09-14T09:09:06.038537+0000","last_peered":"2024-09-14T09:09:06.038537+0000","last_clean":"2024-09-14T09:09:06.038537+0000","last_became_active":"2024-09-14T09:07:06.851020+0000","last_became_peered":"2024-09-14T09:07:06.851020+0000","last_unstale":"2024-09-14T09:09:06.038537+0000","last_undegraded":"2024-09-14T09:09:06.038537+0000","last_fullsized":"2024-09-14T09:09:06.038537+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:19:55.337692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154644,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2080,"num_read_kb":17783,"num_write":389,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":12315549,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"187'749","reported_seq":5084,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038554+0000","last_change":"2024-09-14T09:08:43.990954+0000","last_active":"2024-09-14T09:09:06.038554+0000","last_peered":"2024-09-14T09:09:06.038554+0000","last_clean":"2024-09-14T09:09:06.038554+0000","last_became_active":"2024-09-14T09:08:43.580245+0000","last_became_peered":"2024-09-14T09:08:43.580245+0000","last_unstale":"2024-09-14T09:09:06.038554+0000","last_undegraded":"2024-09-14T09:09:06.038554+0000","last_fullsized":"2024-09-14T09:09:06.038554+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:04.798520+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.116960367,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5499,"num_read_kb":39102,"num_write":883,"num_write_kb":19962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"181'544","reported_seq":9831,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039772+0000","last_change":"2024-09-14T09:08:34.624404+0000","last_active":"2024-09-14T09:09:09.039772+0000","last_peered":"2024-09-14T09:09:09.039772+0000","last_clean":"2024-09-14T09:09:09.039772+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:09:09.039772+0000","last_undegraded":"2024-09-14T09:09:09.039772+0000","last_fullsized":"2024-09-14T09:09:09.039772+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:54:24.811934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097700640000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10396,"num_read_kb":61123,"num_write":490,"num_write_kb":11301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"180'493","reported_seq":3394,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864519+0000","last_change":"2024-09-14T09:08:34.496008+0000","last_active":"2024-09-14T09:09:06.864519+0000","last_peered":"2024-09-14T09:09:06.864519+0000","last_clean":"2024-09-14T09:09:06.864519+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:09:06.864519+0000","last_undegraded":"2024-09-14T09:09:06.864519+0000","last_fullsized":"2024-09-14T09:09:06.864519+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:11.774708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00388507,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3266,"num_read_kb":13215,"num_write":426,"num_write_kb":10574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"179'501","reported_seq":5767,"reported_epoch":210,"state":"peering","last_fresh":"2024-09-14T09:09:06.285712+0000","last_change":"2024-09-14T09:09:07.169910+0000","last_active":"2024-09-14T09:09:07.169910+0000","last_peered":"2024-09-14T09:09:06.285712+0000","last_clean":"2024-09-14T09:08:31.463523+0000","last_became_active":"2024-09-14T09:09:05.965250+0000","last_became_peered":"2024-09-14T09:09:05.965250+0000","last_unstale":"2024-09-14T09:09:06.285712+0000","last_undegraded":"2024-09-14T09:09:06.285712+0000","last_fullsized":"2024-09-14T09:09:05.943570+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022704650000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5399,"num_read_kb":32061,"num_write":547,"num_write_kb":21536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":25396220,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"187'488","reported_seq":4038,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039610+0000","last_change":"2024-09-14T09:08:44.221618+0000","last_active":"2024-09-14T09:09:08.039610+0000","last_peered":"2024-09-14T09:09:08.039610+0000","last_clean":"2024-09-14T09:09:08.039610+0000","last_became_active":"2024-09-14T09:08:43.588233+0000","last_became_peered":"2024-09-14T09:08:43.588233+0000","last_unstale":"2024-09-14T09:09:08.039610+0000","last_undegraded":"2024-09-14T09:09:08.039610+0000","last_fullsized":"2024-09-14T09:09:08.039610+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:10.863128+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.14577486200000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4282,"num_read_kb":17652,"num_write":409,"num_write_kb":7874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":14123394,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"187'586","reported_seq":4291,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039835+0000","last_change":"2024-09-14T09:08:44.145761+0000","last_active":"2024-09-14T09:09:08.039835+0000","last_peered":"2024-09-14T09:09:08.039835+0000","last_clean":"2024-09-14T09:09:08.039835+0000","last_became_active":"2024-09-14T09:08:43.574566+0000","last_became_peered":"2024-09-14T09:08:43.574566+0000","last_unstale":"2024-09-14T09:09:08.039835+0000","last_undegraded":"2024-09-14T09:09:08.039835+0000","last_fullsized":"2024-09-14T09:09:08.039835+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:30.502086+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.141382223,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3641,"num_read_kb":14849,"num_write":554,"num_write_kb":12244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":11864981,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'506","reported_seq":6242,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038690+0000","last_change":"2024-09-14T09:08:35.140559+0000","last_active":"2024-09-14T09:09:06.038690+0000","last_peered":"2024-09-14T09:09:06.038690+0000","last_clean":"2024-09-14T09:09:06.038690+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:09:06.038690+0000","last_undegraded":"2024-09-14T09:09:06.038690+0000","last_fullsized":"2024-09-14T09:09:06.038690+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:31:57.906202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018540480000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5422,"num_read_kb":60065,"num_write":521,"num_write_kb":20523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"187'649","reported_seq":6148,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039335+0000","last_change":"2024-09-14T09:08:43.902203+0000","last_active":"2024-09-14T09:09:08.039335+0000","last_peered":"2024-09-14T09:09:08.039335+0000","last_clean":"2024-09-14T09:09:08.039335+0000","last_became_active":"2024-09-14T09:08:43.577551+0000","last_became_peered":"2024-09-14T09:08:43.577551+0000","last_unstale":"2024-09-14T09:09:08.039335+0000","last_undegraded":"2024-09-14T09:09:08.039335+0000","last_fullsized":"2024-09-14T09:09:08.039335+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:43:37.090068+0000","scrub_duration":0,"objects_trimmed":64,"snaptrim_duration":0.121356326,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6082,"num_read_kb":13595,"num_write":653,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":7405746,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"182'790","reported_seq":6126,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033034+0000","last_change":"2024-09-14T09:08:34.493569+0000","last_active":"2024-09-14T09:09:06.033034+0000","last_peered":"2024-09-14T09:09:06.033034+0000","last_clean":"2024-09-14T09:09:06.033034+0000","last_became_active":"2024-09-14T09:07:26.101586+0000","last_became_peered":"2024-09-14T09:07:26.101586+0000","last_unstale":"2024-09-14T09:09:06.033034+0000","last_undegraded":"2024-09-14T09:09:06.033034+0000","last_fullsized":"2024-09-14T09:09:06.033034+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:11:20.433650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040594799999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7044,"num_read_kb":19519,"num_write":992,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":10260744,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"187'536","reported_seq":6576,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039395+0000","last_change":"2024-09-14T09:08:44.016919+0000","last_active":"2024-09-14T09:09:08.039395+0000","last_peered":"2024-09-14T09:09:08.039395+0000","last_clean":"2024-09-14T09:09:08.039395+0000","last_became_active":"2024-09-14T09:08:43.579997+0000","last_became_peered":"2024-09-14T09:08:43.579997+0000","last_unstale":"2024-09-14T09:09:08.039395+0000","last_undegraded":"2024-09-14T09:09:08.039395+0000","last_fullsized":"2024-09-14T09:09:08.039395+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:24:16.807890+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.10056556899999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6839,"num_read_kb":29296,"num_write":449,"num_write_kb":10682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":24080444,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":281,"num_object_clones":0,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":277,"num_read":198212,"num_read_kb":860220,"num_write":25088,"num_write_kb":385926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4177,"num_bytes_recovered":201826576,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},"log_size":22338,"ondisk_log_size":22338,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20292,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509492,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29488,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342352,"statfs":{"total":96636764160,"available":96606568448,"internally_reserved":0,"allocated":2519040,"data_stored":1219453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509492,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78076,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293764,"statfs":{"total":96636764160,"available":96556814336,"internally_reserved":0,"allocated":2523136,"data_stored":1219472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509493,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47548,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324292,"statfs":{"total":96636764160,"available":96588075008,"internally_reserved":0,"allocated":2531328,"data_stored":1225461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509495,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64232,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307608,"statfs":{"total":96636764160,"available":96570990592,"internally_reserved":0,"allocated":2576384,"data_stored":1809821,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542196,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51760,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320080,"statfs":{"total":96636764160,"available":96583761920,"internally_reserved":0,"allocated":2584576,"data_stored":1821816,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":186,"seq":798863917064,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35980,"kb_used_data":2488,"kb_used_omap":62,"kb_used_meta":33409,"kb_avail":94335860,"statfs":{"total":96636764160,"available":96599920640,"internally_reserved":0,"allocated":2547712,"data_stored":1231467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":63771,"internal_metadata":34211557},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542197,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60040,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311800,"statfs":{"total":96636764160,"available":96575283200,"internally_reserved":0,"allocated":2543616,"data_stored":1231448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542198,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55888,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315952,"statfs":{"total":96636764160,"available":96579534848,"internally_reserved":0,"allocated":2551808,"data_stored":1231486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:09:12.546 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-14T09:09:12.547 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "179'501", 'reported_seq': 5767, 'reported_epoch': 210, 'state': 'peering', 'last_fresh': '2024-09-14T09:09:06.285712+0000', 'last_change': '2024-09-14T09:09:07.169910+0000', 'last_active': '2024-09-14T09:09:07.169910+0000', 'last_peered': '2024-09-14T09:09:06.285712+0000', 'last_clean': '2024-09-14T09:08:31.463523+0000', 'last_became_active': '2024-09-14T09:09:05.965250+0000', 'last_became_peered': '2024-09-14T09:09:05.965250+0000', 'last_unstale': '2024-09-14T09:09:06.285712+0000', 'last_undegraded': '2024-09-14T09:09:06.285712+0000', 'last_fullsized': '2024-09-14T09:09:05.943570+0000', 'mapping_epoch': 210, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 118, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-14T09:05:26.325409+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-14T09:05:26.325409+0000', 'last_clean_scrub_stamp': '2024-09-14T09:05:26.325409+0000', 'objects_scrubbed': 0, 'log_size': 501, 'log_dups_size': 0, 'ondisk_log_size': 501, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 2, 'snaptrim_duration': 0.002270465, 'stat_sum': {'num_bytes': 0, 'num_objects': 9, 'num_object_clones': 0, 'num_object_copies': 18, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9, 'num_whiteouts': 9, 'num_read': 5399, 'num_read_kb': 32061, 'num_write': 547, 'num_write_kb': 21536, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 314, 'num_bytes_recovered': 25396220, 'num_keys_recovered': 24, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-14T09:09:12.547 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-14T09:09:12.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-09-14T09:09:12.878+0000 7f3772572640 0 --2- 172.21.15.176:0/1646537250 >> [v2:172.21.15.176:3301/0,v1:172.21.15.176:6790/0] conn(0x7f376c072760 0x7f376c0aabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-14T09:09:13.129 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:09:13.129 INFO:teuthology.orchestra.run.smithi176.stdout:{"fsid":"c635667d-994e-454f-bcea-51b45637b2a2","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":256,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":211,"num_osds":8,"num_up_osds":8,"osd_up_since":1726304922,"num_in_osds":8,"osd_in_since":1726304841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":40},{"state_name":"peering","count":1}],"num_pgs":41,"num_pools":3,"num_objects":285,"data_bytes":590406,"bytes_used":433164288,"bytes_avail":772660948992,"bytes_total":773094113280,"inactive_pgs_ratio":0.024390242993831635,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-14T09:04:57:032443+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-14T09:06:13.975920+0000","services":{}},"progress_events":{}} 2024-09-14T09:09:13.148 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-14T09:09:13.148 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:09:13.442 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:09:13.443 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:09:13.460 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-09-14T09:09:12.027559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":285,"num_object_clones":0,"num_object_copies":570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":285,"num_whiteouts":277,"num_read":198278,"num_read_kb":860275,"num_write":25191,"num_write_kb":387556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4182,"num_bytes_recovered":203007312,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22413,"ondisk_log_size":22413,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":423012,"kb_used_data":19900,"kb_used_omap":73,"kb_used_meta":402870,"kb_avail":754551708,"statfs":{"total":773094113280,"available":772660948992,"internally_reserved":0,"allocated":20377600,"data_stored":10990424,"data_compressed":13252,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":74928,"internal_metadata":412539728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.092394"},"pg_stats":[{"pgid":"3.1f","version":"182'1763","reported_seq":7856,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033212+0000","last_change":"2024-09-14T09:08:34.495405+0000","last_active":"2024-09-14T09:09:06.033212+0000","last_peered":"2024-09-14T09:09:06.033212+0000","last_clean":"2024-09-14T09:09:06.033212+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:09:06.033212+0000","last_undegraded":"2024-09-14T09:09:06.033212+0000","last_fullsized":"2024-09-14T09:09:06.033212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1763,"log_dups_size":0,"ondisk_log_size":1763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:22:02.551926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022047759999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13833,"num_read_kb":24717,"num_write":2684,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'466","reported_seq":5989,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.040029+0000","last_change":"2024-09-14T09:08:34.625492+0000","last_active":"2024-09-14T09:09:09.040029+0000","last_peered":"2024-09-14T09:09:09.040029+0000","last_clean":"2024-09-14T09:09:09.040029+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:09:09.040029+0000","last_undegraded":"2024-09-14T09:09:09.040029+0000","last_fullsized":"2024-09-14T09:09:09.040029+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:01.005568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010401127,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5032,"num_read_kb":31996,"num_write":418,"num_write_kb":13378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"187'676","reported_seq":6614,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.865111+0000","last_change":"2024-09-14T09:08:43.847915+0000","last_active":"2024-09-14T09:09:06.865111+0000","last_peered":"2024-09-14T09:09:06.865111+0000","last_clean":"2024-09-14T09:09:06.865111+0000","last_became_active":"2024-09-14T09:08:43.573212+0000","last_became_peered":"2024-09-14T09:08:43.573212+0000","last_unstale":"2024-09-14T09:09:06.865111+0000","last_undegraded":"2024-09-14T09:09:06.865111+0000","last_fullsized":"2024-09-14T09:09:06.865111+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:26:03.155997+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.147750145,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6669,"num_read_kb":58049,"num_write":727,"num_write_kb":12436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"190'866","reported_seq":5643,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864770+0000","last_change":"2024-09-14T09:08:46.770708+0000","last_active":"2024-09-14T09:09:06.864770+0000","last_peered":"2024-09-14T09:09:06.864770+0000","last_clean":"2024-09-14T09:09:06.864770+0000","last_became_active":"2024-09-14T09:08:46.599475+0000","last_became_peered":"2024-09-14T09:08:46.599475+0000","last_unstale":"2024-09-14T09:09:06.864770+0000","last_undegraded":"2024-09-14T09:09:06.864770+0000","last_fullsized":"2024-09-14T09:09:06.864770+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:57.157599+0000","scrub_duration":0,"objects_trimmed":60,"snaptrim_duration":0.168855903,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7679,"num_read_kb":27474,"num_write":1043,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'742","reported_seq":5713,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033232+0000","last_change":"2024-09-14T09:08:34.491862+0000","last_active":"2024-09-14T09:09:06.033232+0000","last_peered":"2024-09-14T09:09:06.033232+0000","last_clean":"2024-09-14T09:09:06.033232+0000","last_became_active":"2024-09-14T09:07:06.842337+0000","last_became_peered":"2024-09-14T09:07:06.842337+0000","last_unstale":"2024-09-14T09:09:06.033232+0000","last_undegraded":"2024-09-14T09:09:06.033232+0000","last_fullsized":"2024-09-14T09:09:06.033232+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:23:25.823078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023564419999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6703,"num_read_kb":24544,"num_write":874,"num_write_kb":16463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":14145319,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"187'724","reported_seq":5512,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.867745+0000","last_change":"2024-09-14T09:08:43.771622+0000","last_active":"2024-09-14T09:09:07.867745+0000","last_peered":"2024-09-14T09:09:07.867745+0000","last_clean":"2024-09-14T09:09:07.867745+0000","last_became_active":"2024-09-14T09:08:43.556338+0000","last_became_peered":"2024-09-14T09:08:43.556338+0000","last_unstale":"2024-09-14T09:09:07.867745+0000","last_undegraded":"2024-09-14T09:09:07.867745+0000","last_fullsized":"2024-09-14T09:09:07.867745+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:41.389225+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.12752904200000001,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6399,"num_read_kb":30687,"num_write":963,"num_write_kb":14233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"174'375","reported_seq":4400,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.862853+0000","last_change":"2024-09-14T09:08:50.629874+0000","last_active":"2024-09-14T09:09:07.862853+0000","last_peered":"2024-09-14T09:09:07.862853+0000","last_clean":"2024-09-14T09:09:07.862853+0000","last_became_active":"2024-09-14T09:08:50.627814+0000","last_became_peered":"2024-09-14T09:08:50.627814+0000","last_unstale":"2024-09-14T09:09:07.862853+0000","last_undegraded":"2024-09-14T09:09:07.862853+0000","last_fullsized":"2024-09-14T09:09:07.862853+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:14:43.966792+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.025218744000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3327,"num_read_kb":10838,"num_write":357,"num_write_kb":6673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":3727781,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"182'624","reported_seq":6380,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039812+0000","last_change":"2024-09-14T09:08:34.622411+0000","last_active":"2024-09-14T09:09:09.039812+0000","last_peered":"2024-09-14T09:09:09.039812+0000","last_clean":"2024-09-14T09:09:09.039812+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:09:09.039812+0000","last_undegraded":"2024-09-14T09:09:09.039812+0000","last_fullsized":"2024-09-14T09:09:09.039812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:55:38.962684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080911500000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7486,"num_read_kb":17479,"num_write":535,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"182'910","reported_seq":7825,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033222+0000","last_change":"2024-09-14T09:08:34.496061+0000","last_active":"2024-09-14T09:09:06.033222+0000","last_peered":"2024-09-14T09:09:06.033222+0000","last_clean":"2024-09-14T09:09:06.033222+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:09:06.033222+0000","last_undegraded":"2024-09-14T09:09:06.033222+0000","last_fullsized":"2024-09-14T09:09:06.033222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:02:38.220394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026116440000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10070,"num_read_kb":35479,"num_write":1129,"num_write_kb":10795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"179'629","reported_seq":3950,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864528+0000","last_change":"2024-09-14T09:08:34.493205+0000","last_active":"2024-09-14T09:09:06.864528+0000","last_peered":"2024-09-14T09:09:06.864528+0000","last_clean":"2024-09-14T09:09:06.864528+0000","last_became_active":"2024-09-14T09:07:25.105674+0000","last_became_peered":"2024-09-14T09:07:25.105674+0000","last_unstale":"2024-09-14T09:09:06.864528+0000","last_undegraded":"2024-09-14T09:09:06.864528+0000","last_fullsized":"2024-09-14T09:09:06.864528+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:20:01.627411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001781447,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4726,"num_read_kb":15599,"num_write":700,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":8467448,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"181'1136","reported_seq":5393,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039887+0000","last_change":"2024-09-14T09:08:34.507525+0000","last_active":"2024-09-14T09:09:08.039887+0000","last_peered":"2024-09-14T09:09:08.039887+0000","last_clean":"2024-09-14T09:09:08.039887+0000","last_became_active":"2024-09-14T09:07:27.854998+0000","last_became_peered":"2024-09-14T09:07:27.854998+0000","last_unstale":"2024-09-14T09:09:08.039887+0000","last_undegraded":"2024-09-14T09:09:08.039887+0000","last_fullsized":"2024-09-14T09:09:08.039887+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:11:04.062027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001704445,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8595,"num_read_kb":26590,"num_write":1602,"num_write_kb":14717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":4170600,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"187'670","reported_seq":3459,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.868133+0000","last_change":"2024-09-14T09:08:43.840463+0000","last_active":"2024-09-14T09:09:07.868133+0000","last_peered":"2024-09-14T09:09:07.868133+0000","last_clean":"2024-09-14T09:09:07.868133+0000","last_became_active":"2024-09-14T09:08:43.569123+0000","last_became_peered":"2024-09-14T09:08:43.569123+0000","last_unstale":"2024-09-14T09:09:07.868133+0000","last_undegraded":"2024-09-14T09:09:07.868133+0000","last_fullsized":"2024-09-14T09:09:07.868133+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:18.861184+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.15505904000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3583,"num_read_kb":9199,"num_write":691,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":2686146,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":403,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038664+0000","last_change":"2024-09-14T09:07:21.857426+0000","last_active":"2024-09-14T09:09:06.038664+0000","last_peered":"2024-09-14T09:09:06.038664+0000","last_clean":"2024-09-14T09:09:06.038664+0000","last_became_active":"2024-09-14T09:07:21.856579+0000","last_became_peered":"2024-09-14T09:07:21.856579+0000","last_unstale":"2024-09-14T09:09:06.038664+0000","last_undegraded":"2024-09-14T09:09:06.038664+0000","last_fullsized":"2024-09-14T09:09:06.038664+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:56.694930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"182'791","reported_seq":6693,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033604+0000","last_change":"2024-09-14T09:08:34.494518+0000","last_active":"2024-09-14T09:09:06.033604+0000","last_peered":"2024-09-14T09:09:06.033604+0000","last_clean":"2024-09-14T09:09:06.033604+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:09:06.033604+0000","last_undegraded":"2024-09-14T09:09:06.033604+0000","last_fullsized":"2024-09-14T09:09:06.033604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:58.731530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016873319999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8676,"num_read_kb":45389,"num_write":707,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":360,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.863161+0000","last_change":"2024-09-14T09:08:51.290129+0000","last_active":"2024-09-14T09:09:07.863161+0000","last_peered":"2024-09-14T09:09:07.863161+0000","last_clean":"2024-09-14T09:09:07.863161+0000","last_became_active":"2024-09-14T09:08:51.289551+0000","last_became_peered":"2024-09-14T09:08:51.289551+0000","last_unstale":"2024-09-14T09:09:07.863161+0000","last_undegraded":"2024-09-14T09:09:07.863161+0000","last_fullsized":"2024-09-14T09:09:07.863161+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:52:01.486956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"182'1187","reported_seq":7262,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.040014+0000","last_change":"2024-09-14T09:08:34.626223+0000","last_active":"2024-09-14T09:09:09.040014+0000","last_peered":"2024-09-14T09:09:09.040014+0000","last_clean":"2024-09-14T09:09:09.040014+0000","last_became_active":"2024-09-14T09:07:26.102666+0000","last_became_peered":"2024-09-14T09:07:26.102666+0000","last_unstale":"2024-09-14T09:09:09.040014+0000","last_undegraded":"2024-09-14T09:09:09.040014+0000","last_fullsized":"2024-09-14T09:09:09.040014+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:03:41.035208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011116867000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10102,"num_read_kb":27247,"num_write":1623,"num_write_kb":13456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":8334712,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":204,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039803+0000","last_change":"2024-09-14T09:07:21.863290+0000","last_active":"2024-09-14T09:09:08.039803+0000","last_peered":"2024-09-14T09:09:08.039803+0000","last_clean":"2024-09-14T09:09:08.039803+0000","last_became_active":"2024-09-14T09:07:21.862352+0000","last_became_peered":"2024-09-14T09:07:21.862352+0000","last_unstale":"2024-09-14T09:09:08.039803+0000","last_undegraded":"2024-09-14T09:09:08.039803+0000","last_fullsized":"2024-09-14T09:09:08.039803+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:48:30.332891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"179'922","reported_seq":5889,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.862993+0000","last_change":"2024-09-14T09:08:50.627160+0000","last_active":"2024-09-14T09:09:07.862993+0000","last_peered":"2024-09-14T09:09:07.862993+0000","last_clean":"2024-09-14T09:09:07.862993+0000","last_became_active":"2024-09-14T09:08:50.625357+0000","last_became_peered":"2024-09-14T09:08:50.625357+0000","last_unstale":"2024-09-14T09:09:07.862993+0000","last_undegraded":"2024-09-14T09:09:07.862993+0000","last_fullsized":"2024-09-14T09:09:07.862993+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:18:31.289028+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0021152969999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7783,"num_read_kb":22826,"num_write":1134,"num_write_kb":9259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":8151041,"num_keys_recovered":91,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":360,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.863248+0000","last_change":"2024-09-14T09:08:50.628307+0000","last_active":"2024-09-14T09:09:07.863248+0000","last_peered":"2024-09-14T09:09:07.863248+0000","last_clean":"2024-09-14T09:09:07.863248+0000","last_became_active":"2024-09-14T09:08:50.627971+0000","last_became_peered":"2024-09-14T09:08:50.627971+0000","last_unstale":"2024-09-14T09:09:07.863248+0000","last_undegraded":"2024-09-14T09:09:07.863248+0000","last_fullsized":"2024-09-14T09:09:07.863248+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:57.655435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"182'688","reported_seq":2922,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.865338+0000","last_change":"2024-09-14T09:08:34.495006+0000","last_active":"2024-09-14T09:09:06.865338+0000","last_peered":"2024-09-14T09:09:06.865338+0000","last_clean":"2024-09-14T09:09:06.865338+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:09:06.865338+0000","last_undegraded":"2024-09-14T09:09:06.865338+0000","last_fullsized":"2024-09-14T09:09:06.865338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:25:08.203494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030472329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3594,"num_read_kb":16894,"num_write":791,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"190'884","reported_seq":6718,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.868090+0000","last_change":"2024-09-14T09:08:46.793435+0000","last_active":"2024-09-14T09:09:07.868090+0000","last_peered":"2024-09-14T09:09:07.868090+0000","last_clean":"2024-09-14T09:09:07.868090+0000","last_became_active":"2024-09-14T09:08:46.597293+0000","last_became_peered":"2024-09-14T09:08:46.597293+0000","last_unstale":"2024-09-14T09:09:07.868090+0000","last_undegraded":"2024-09-14T09:09:07.868090+0000","last_fullsized":"2024-09-14T09:09:07.868090+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:48:54.046569+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.19293607300000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8248,"num_read_kb":26883,"num_write":890,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":398,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864400+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:09:06.864400+0000","last_peered":"2024-09-14T09:09:06.864400+0000","last_clean":"2024-09-14T09:09:06.864400+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:09:06.864400+0000","last_undegraded":"2024-09-14T09:09:06.864400+0000","last_fullsized":"2024-09-14T09:09:06.864400+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"180'399","reported_seq":2539,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.863059+0000","last_change":"2024-09-14T09:08:50.625390+0000","last_active":"2024-09-14T09:09:07.863059+0000","last_peered":"2024-09-14T09:09:07.863059+0000","last_clean":"2024-09-14T09:09:07.863059+0000","last_became_active":"2024-09-14T09:08:50.622578+0000","last_became_peered":"2024-09-14T09:08:50.622578+0000","last_unstale":"2024-09-14T09:09:07.863059+0000","last_undegraded":"2024-09-14T09:09:07.863059+0000","last_fullsized":"2024-09-14T09:09:07.863059+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:50:11.634631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00245612,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1873,"num_read_kb":11982,"num_write":352,"num_write_kb":5539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":395,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.859085+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:09:06.859085+0000","last_peered":"2024-09-14T09:09:06.859085+0000","last_clean":"2024-09-14T09:09:06.859085+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:09:06.859085+0000","last_undegraded":"2024-09-14T09:09:06.859085+0000","last_fullsized":"2024-09-14T09:09:06.859085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"187'460","reported_seq":5079,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033075+0000","last_change":"2024-09-14T09:08:43.789977+0000","last_active":"2024-09-14T09:09:06.033075+0000","last_peered":"2024-09-14T09:09:06.033075+0000","last_clean":"2024-09-14T09:09:06.033075+0000","last_became_active":"2024-09-14T09:08:43.568923+0000","last_became_peered":"2024-09-14T09:08:43.568923+0000","last_unstale":"2024-09-14T09:09:06.033075+0000","last_undegraded":"2024-09-14T09:09:06.033075+0000","last_fullsized":"2024-09-14T09:09:06.033075+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:45:29.077592+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.165000182,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3905,"num_read_kb":25149,"num_write":375,"num_write_kb":14483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.867673+0000","last_change":"2024-09-14T09:08:44.554373+0000","last_active":"2024-09-14T09:09:07.867673+0000","last_peered":"2024-09-14T09:09:07.867673+0000","last_clean":"2024-09-14T09:09:07.867673+0000","last_became_active":"2024-09-14T09:08:44.553643+0000","last_became_peered":"2024-09-14T09:08:44.553643+0000","last_unstale":"2024-09-14T09:09:07.867673+0000","last_undegraded":"2024-09-14T09:09:07.867673+0000","last_fullsized":"2024-09-14T09:09:07.867673+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:32:52.900856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"181'433","reported_seq":2984,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039700+0000","last_change":"2024-09-14T09:08:34.620964+0000","last_active":"2024-09-14T09:09:09.039700+0000","last_peered":"2024-09-14T09:09:09.039700+0000","last_clean":"2024-09-14T09:09:09.039700+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:09:09.039700+0000","last_undegraded":"2024-09-14T09:09:09.039700+0000","last_fullsized":"2024-09-14T09:09:09.039700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:34:59.882295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065530470000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2291,"num_read_kb":11209,"num_write":432,"num_write_kb":6866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":126,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864412+0000","last_change":"2024-09-14T09:08:06.210516+0000","last_active":"2024-09-14T09:09:06.864412+0000","last_peered":"2024-09-14T09:09:06.864412+0000","last_clean":"2024-09-14T09:09:06.864412+0000","last_became_active":"2024-09-14T09:08:06.205108+0000","last_became_peered":"2024-09-14T09:08:06.205108+0000","last_unstale":"2024-09-14T09:09:06.864412+0000","last_undegraded":"2024-09-14T09:09:06.864412+0000","last_fullsized":"2024-09-14T09:09:06.864412+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:22:05.543865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":210,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039335+0000","last_change":"2024-09-14T09:07:21.876180+0000","last_active":"2024-09-14T09:09:08.039335+0000","last_peered":"2024-09-14T09:09:08.039335+0000","last_clean":"2024-09-14T09:09:08.039335+0000","last_became_active":"2024-09-14T09:07:21.860117+0000","last_became_peered":"2024-09-14T09:07:21.860117+0000","last_unstale":"2024-09-14T09:09:08.039335+0000","last_undegraded":"2024-09-14T09:09:08.039335+0000","last_fullsized":"2024-09-14T09:09:08.039335+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:58:36.142971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"178'682","reported_seq":7043,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039644+0000","last_change":"2024-09-14T09:08:34.623351+0000","last_active":"2024-09-14T09:09:09.039644+0000","last_peered":"2024-09-14T09:09:09.039644+0000","last_clean":"2024-09-14T09:09:09.039644+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:09:09.039644+0000","last_undegraded":"2024-09-14T09:09:09.039644+0000","last_fullsized":"2024-09-14T09:09:09.039644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:48.184932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089062680000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7688,"num_read_kb":41730,"num_write":748,"num_write_kb":19769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"181'469","reported_seq":3210,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038537+0000","last_change":"2024-09-14T09:08:35.139769+0000","last_active":"2024-09-14T09:09:06.038537+0000","last_peered":"2024-09-14T09:09:06.038537+0000","last_clean":"2024-09-14T09:09:06.038537+0000","last_became_active":"2024-09-14T09:07:06.851020+0000","last_became_peered":"2024-09-14T09:07:06.851020+0000","last_unstale":"2024-09-14T09:09:06.038537+0000","last_undegraded":"2024-09-14T09:09:06.038537+0000","last_fullsized":"2024-09-14T09:09:06.038537+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:19:55.337692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154644,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2080,"num_read_kb":17783,"num_write":389,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":12315549,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"187'749","reported_seq":5084,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038554+0000","last_change":"2024-09-14T09:08:43.990954+0000","last_active":"2024-09-14T09:09:06.038554+0000","last_peered":"2024-09-14T09:09:06.038554+0000","last_clean":"2024-09-14T09:09:06.038554+0000","last_became_active":"2024-09-14T09:08:43.580245+0000","last_became_peered":"2024-09-14T09:08:43.580245+0000","last_unstale":"2024-09-14T09:09:06.038554+0000","last_undegraded":"2024-09-14T09:09:06.038554+0000","last_fullsized":"2024-09-14T09:09:06.038554+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:04.798520+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.116960367,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5499,"num_read_kb":39102,"num_write":883,"num_write_kb":19962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"181'544","reported_seq":9831,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039772+0000","last_change":"2024-09-14T09:08:34.624404+0000","last_active":"2024-09-14T09:09:09.039772+0000","last_peered":"2024-09-14T09:09:09.039772+0000","last_clean":"2024-09-14T09:09:09.039772+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:09:09.039772+0000","last_undegraded":"2024-09-14T09:09:09.039772+0000","last_fullsized":"2024-09-14T09:09:09.039772+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:54:24.811934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097700640000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10396,"num_read_kb":61123,"num_write":490,"num_write_kb":11301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"180'493","reported_seq":3394,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.864519+0000","last_change":"2024-09-14T09:08:34.496008+0000","last_active":"2024-09-14T09:09:06.864519+0000","last_peered":"2024-09-14T09:09:06.864519+0000","last_clean":"2024-09-14T09:09:06.864519+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:09:06.864519+0000","last_undegraded":"2024-09-14T09:09:06.864519+0000","last_fullsized":"2024-09-14T09:09:06.864519+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:11.774708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00388507,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3266,"num_read_kb":13215,"num_write":426,"num_write_kb":10574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"179'501","reported_seq":5767,"reported_epoch":210,"state":"peering","last_fresh":"2024-09-14T09:09:06.285712+0000","last_change":"2024-09-14T09:09:07.169910+0000","last_active":"2024-09-14T09:09:07.169910+0000","last_peered":"2024-09-14T09:09:06.285712+0000","last_clean":"2024-09-14T09:08:31.463523+0000","last_became_active":"2024-09-14T09:09:05.965250+0000","last_became_peered":"2024-09-14T09:09:05.965250+0000","last_unstale":"2024-09-14T09:09:06.285712+0000","last_undegraded":"2024-09-14T09:09:06.285712+0000","last_fullsized":"2024-09-14T09:09:05.943570+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022704650000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5399,"num_read_kb":32061,"num_write":547,"num_write_kb":21536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":25396220,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"187'488","reported_seq":4038,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039610+0000","last_change":"2024-09-14T09:08:44.221618+0000","last_active":"2024-09-14T09:09:08.039610+0000","last_peered":"2024-09-14T09:09:08.039610+0000","last_clean":"2024-09-14T09:09:08.039610+0000","last_became_active":"2024-09-14T09:08:43.588233+0000","last_became_peered":"2024-09-14T09:08:43.588233+0000","last_unstale":"2024-09-14T09:09:08.039610+0000","last_undegraded":"2024-09-14T09:09:08.039610+0000","last_fullsized":"2024-09-14T09:09:08.039610+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:10.863128+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.14577486200000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4282,"num_read_kb":17652,"num_write":409,"num_write_kb":7874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":14123394,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"187'586","reported_seq":4291,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039835+0000","last_change":"2024-09-14T09:08:44.145761+0000","last_active":"2024-09-14T09:09:08.039835+0000","last_peered":"2024-09-14T09:09:08.039835+0000","last_clean":"2024-09-14T09:09:08.039835+0000","last_became_active":"2024-09-14T09:08:43.574566+0000","last_became_peered":"2024-09-14T09:08:43.574566+0000","last_unstale":"2024-09-14T09:09:08.039835+0000","last_undegraded":"2024-09-14T09:09:08.039835+0000","last_fullsized":"2024-09-14T09:09:08.039835+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:30.502086+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.141382223,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3641,"num_read_kb":14849,"num_write":554,"num_write_kb":12244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":11864981,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'506","reported_seq":6242,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.038690+0000","last_change":"2024-09-14T09:08:35.140559+0000","last_active":"2024-09-14T09:09:06.038690+0000","last_peered":"2024-09-14T09:09:06.038690+0000","last_clean":"2024-09-14T09:09:06.038690+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:09:06.038690+0000","last_undegraded":"2024-09-14T09:09:06.038690+0000","last_fullsized":"2024-09-14T09:09:06.038690+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:31:57.906202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018540480000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5422,"num_read_kb":60065,"num_write":521,"num_write_kb":20523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"187'649","reported_seq":6148,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039335+0000","last_change":"2024-09-14T09:08:43.902203+0000","last_active":"2024-09-14T09:09:08.039335+0000","last_peered":"2024-09-14T09:09:08.039335+0000","last_clean":"2024-09-14T09:09:08.039335+0000","last_became_active":"2024-09-14T09:08:43.577551+0000","last_became_peered":"2024-09-14T09:08:43.577551+0000","last_unstale":"2024-09-14T09:09:08.039335+0000","last_undegraded":"2024-09-14T09:09:08.039335+0000","last_fullsized":"2024-09-14T09:09:08.039335+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:43:37.090068+0000","scrub_duration":0,"objects_trimmed":64,"snaptrim_duration":0.121356326,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6082,"num_read_kb":13595,"num_write":653,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":7405746,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"182'790","reported_seq":6126,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-14T09:09:06.033034+0000","last_change":"2024-09-14T09:08:34.493569+0000","last_active":"2024-09-14T09:09:06.033034+0000","last_peered":"2024-09-14T09:09:06.033034+0000","last_clean":"2024-09-14T09:09:06.033034+0000","last_became_active":"2024-09-14T09:07:26.101586+0000","last_became_peered":"2024-09-14T09:07:26.101586+0000","last_unstale":"2024-09-14T09:09:06.033034+0000","last_undegraded":"2024-09-14T09:09:06.033034+0000","last_fullsized":"2024-09-14T09:09:06.033034+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:11:20.433650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040594799999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7044,"num_read_kb":19519,"num_write":992,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":10260744,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"187'536","reported_seq":6576,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039395+0000","last_change":"2024-09-14T09:08:44.016919+0000","last_active":"2024-09-14T09:09:08.039395+0000","last_peered":"2024-09-14T09:09:08.039395+0000","last_clean":"2024-09-14T09:09:08.039395+0000","last_became_active":"2024-09-14T09:08:43.579997+0000","last_became_peered":"2024-09-14T09:08:43.579997+0000","last_unstale":"2024-09-14T09:09:08.039395+0000","last_undegraded":"2024-09-14T09:09:08.039395+0000","last_fullsized":"2024-09-14T09:09:08.039395+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:24:16.807890+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.10056556899999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6839,"num_read_kb":29296,"num_write":449,"num_write_kb":10682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":24080444,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":281,"num_object_clones":0,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":277,"num_read":198212,"num_read_kb":860220,"num_write":25088,"num_write_kb":385926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4177,"num_bytes_recovered":201826576,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},"log_size":22338,"ondisk_log_size":22338,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20292,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509492,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29488,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342352,"statfs":{"total":96636764160,"available":96606568448,"internally_reserved":0,"allocated":2519040,"data_stored":1219453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509492,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78076,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293764,"statfs":{"total":96636764160,"available":96556814336,"internally_reserved":0,"allocated":2523136,"data_stored":1219472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509493,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47548,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324292,"statfs":{"total":96636764160,"available":96588075008,"internally_reserved":0,"allocated":2531328,"data_stored":1225461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509495,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64232,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307608,"statfs":{"total":96636764160,"available":96570990592,"internally_reserved":0,"allocated":2576384,"data_stored":1809821,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542196,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51760,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320080,"statfs":{"total":96636764160,"available":96583761920,"internally_reserved":0,"allocated":2584576,"data_stored":1821816,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":186,"seq":798863917064,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35980,"kb_used_data":2488,"kb_used_omap":62,"kb_used_meta":33409,"kb_avail":94335860,"statfs":{"total":96636764160,"available":96599920640,"internally_reserved":0,"allocated":2547712,"data_stored":1231467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":63771,"internal_metadata":34211557},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542197,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60040,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311800,"statfs":{"total":96636764160,"available":96575283200,"internally_reserved":0,"allocated":2543616,"data_stored":1231448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542198,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55888,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315952,"statfs":{"total":96636764160,"available":96579534848,"internally_reserved":0,"allocated":2551808,"data_stored":1231486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:09:16.462 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:09:16.782 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:09:16.783 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:09:16.801 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-14T09:09:16.028774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":285,"num_object_clones":0,"num_object_copies":570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":285,"num_whiteouts":277,"num_read":198278,"num_read_kb":860275,"num_write":25191,"num_write_kb":387556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4182,"num_bytes_recovered":203007312,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22413,"ondisk_log_size":22413,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":423096,"kb_used_data":19984,"kb_used_omap":73,"kb_used_meta":402870,"kb_avail":754551624,"statfs":{"total":773094113280,"available":772660862976,"internally_reserved":0,"allocated":20463616,"data_stored":11032396,"data_compressed":13252,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":74928,"internal_metadata":412539728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.167394"},"pg_stats":[{"pgid":"3.1f","version":"182'1763","reported_seq":7860,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.476110+0000","last_change":"2024-09-14T09:08:34.495405+0000","last_active":"2024-09-14T09:09:09.476110+0000","last_peered":"2024-09-14T09:09:09.476110+0000","last_clean":"2024-09-14T09:09:09.476110+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:09:09.476110+0000","last_undegraded":"2024-09-14T09:09:09.476110+0000","last_fullsized":"2024-09-14T09:09:09.476110+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1763,"log_dups_size":0,"ondisk_log_size":1763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:22:02.551926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022047759999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13833,"num_read_kb":24717,"num_write":2684,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'466","reported_seq":5989,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.040029+0000","last_change":"2024-09-14T09:08:34.625492+0000","last_active":"2024-09-14T09:09:09.040029+0000","last_peered":"2024-09-14T09:09:09.040029+0000","last_clean":"2024-09-14T09:09:09.040029+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:09:09.040029+0000","last_undegraded":"2024-09-14T09:09:09.040029+0000","last_fullsized":"2024-09-14T09:09:09.040029+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:01.005568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010401127,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5032,"num_read_kb":31996,"num_write":418,"num_write_kb":13378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"187'676","reported_seq":6616,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.871638+0000","last_change":"2024-09-14T09:08:43.847915+0000","last_active":"2024-09-14T09:09:07.871638+0000","last_peered":"2024-09-14T09:09:07.871638+0000","last_clean":"2024-09-14T09:09:07.871638+0000","last_became_active":"2024-09-14T09:08:43.573212+0000","last_became_peered":"2024-09-14T09:08:43.573212+0000","last_unstale":"2024-09-14T09:09:07.871638+0000","last_undegraded":"2024-09-14T09:09:07.871638+0000","last_fullsized":"2024-09-14T09:09:07.871638+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:26:03.155997+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.147750145,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6669,"num_read_kb":58049,"num_write":727,"num_write_kb":12436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"190'866","reported_seq":5645,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.870986+0000","last_change":"2024-09-14T09:08:46.770708+0000","last_active":"2024-09-14T09:09:07.870986+0000","last_peered":"2024-09-14T09:09:07.870986+0000","last_clean":"2024-09-14T09:09:07.870986+0000","last_became_active":"2024-09-14T09:08:46.599475+0000","last_became_peered":"2024-09-14T09:08:46.599475+0000","last_unstale":"2024-09-14T09:09:07.870986+0000","last_undegraded":"2024-09-14T09:09:07.870986+0000","last_fullsized":"2024-09-14T09:09:07.870986+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:57.157599+0000","scrub_duration":0,"objects_trimmed":60,"snaptrim_duration":0.168855903,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7679,"num_read_kb":27474,"num_write":1043,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'742","reported_seq":5718,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.476034+0000","last_change":"2024-09-14T09:08:34.491862+0000","last_active":"2024-09-14T09:09:09.476034+0000","last_peered":"2024-09-14T09:09:09.476034+0000","last_clean":"2024-09-14T09:09:09.476034+0000","last_became_active":"2024-09-14T09:07:06.842337+0000","last_became_peered":"2024-09-14T09:07:06.842337+0000","last_unstale":"2024-09-14T09:09:09.476034+0000","last_undegraded":"2024-09-14T09:09:09.476034+0000","last_fullsized":"2024-09-14T09:09:09.476034+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:23:25.823078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023564419999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6703,"num_read_kb":24544,"num_write":874,"num_write_kb":16463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":14145319,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"187'724","reported_seq":5512,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.867745+0000","last_change":"2024-09-14T09:08:43.771622+0000","last_active":"2024-09-14T09:09:07.867745+0000","last_peered":"2024-09-14T09:09:07.867745+0000","last_clean":"2024-09-14T09:09:07.867745+0000","last_became_active":"2024-09-14T09:08:43.556338+0000","last_became_peered":"2024-09-14T09:08:43.556338+0000","last_unstale":"2024-09-14T09:09:07.867745+0000","last_undegraded":"2024-09-14T09:09:07.867745+0000","last_fullsized":"2024-09-14T09:09:07.867745+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:41.389225+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.12752904200000001,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6399,"num_read_kb":30687,"num_write":963,"num_write_kb":14233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"174'375","reported_seq":4400,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.862853+0000","last_change":"2024-09-14T09:08:50.629874+0000","last_active":"2024-09-14T09:09:07.862853+0000","last_peered":"2024-09-14T09:09:07.862853+0000","last_clean":"2024-09-14T09:09:07.862853+0000","last_became_active":"2024-09-14T09:08:50.627814+0000","last_became_peered":"2024-09-14T09:08:50.627814+0000","last_unstale":"2024-09-14T09:09:07.862853+0000","last_undegraded":"2024-09-14T09:09:07.862853+0000","last_fullsized":"2024-09-14T09:09:07.862853+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:14:43.966792+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.025218744000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3327,"num_read_kb":10838,"num_write":357,"num_write_kb":6673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":3727781,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"182'624","reported_seq":6380,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039812+0000","last_change":"2024-09-14T09:08:34.622411+0000","last_active":"2024-09-14T09:09:09.039812+0000","last_peered":"2024-09-14T09:09:09.039812+0000","last_clean":"2024-09-14T09:09:09.039812+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:09:09.039812+0000","last_undegraded":"2024-09-14T09:09:09.039812+0000","last_fullsized":"2024-09-14T09:09:09.039812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:55:38.962684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080911500000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7486,"num_read_kb":17479,"num_write":535,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"182'910","reported_seq":7829,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.476168+0000","last_change":"2024-09-14T09:08:34.496061+0000","last_active":"2024-09-14T09:09:09.476168+0000","last_peered":"2024-09-14T09:09:09.476168+0000","last_clean":"2024-09-14T09:09:09.476168+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:09:09.476168+0000","last_undegraded":"2024-09-14T09:09:09.476168+0000","last_fullsized":"2024-09-14T09:09:09.476168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:02:38.220394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026116440000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10070,"num_read_kb":35479,"num_write":1129,"num_write_kb":10795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"179'629","reported_seq":3952,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.871137+0000","last_change":"2024-09-14T09:08:34.493205+0000","last_active":"2024-09-14T09:09:07.871137+0000","last_peered":"2024-09-14T09:09:07.871137+0000","last_clean":"2024-09-14T09:09:07.871137+0000","last_became_active":"2024-09-14T09:07:25.105674+0000","last_became_peered":"2024-09-14T09:07:25.105674+0000","last_unstale":"2024-09-14T09:09:07.871137+0000","last_undegraded":"2024-09-14T09:09:07.871137+0000","last_fullsized":"2024-09-14T09:09:07.871137+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:20:01.627411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001781447,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4726,"num_read_kb":15599,"num_write":700,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":8467448,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"181'1136","reported_seq":5393,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039887+0000","last_change":"2024-09-14T09:08:34.507525+0000","last_active":"2024-09-14T09:09:08.039887+0000","last_peered":"2024-09-14T09:09:08.039887+0000","last_clean":"2024-09-14T09:09:08.039887+0000","last_became_active":"2024-09-14T09:07:27.854998+0000","last_became_peered":"2024-09-14T09:07:27.854998+0000","last_unstale":"2024-09-14T09:09:08.039887+0000","last_undegraded":"2024-09-14T09:09:08.039887+0000","last_fullsized":"2024-09-14T09:09:08.039887+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:11:04.062027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001704445,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8595,"num_read_kb":26590,"num_write":1602,"num_write_kb":14717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":4170600,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"187'670","reported_seq":3459,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.868133+0000","last_change":"2024-09-14T09:08:43.840463+0000","last_active":"2024-09-14T09:09:07.868133+0000","last_peered":"2024-09-14T09:09:07.868133+0000","last_clean":"2024-09-14T09:09:07.868133+0000","last_became_active":"2024-09-14T09:08:43.569123+0000","last_became_peered":"2024-09-14T09:08:43.569123+0000","last_unstale":"2024-09-14T09:09:07.868133+0000","last_undegraded":"2024-09-14T09:09:07.868133+0000","last_fullsized":"2024-09-14T09:09:07.868133+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:18.861184+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.15505904000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3583,"num_read_kb":9199,"num_write":691,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":2686146,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":407,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.480738+0000","last_change":"2024-09-14T09:07:21.857426+0000","last_active":"2024-09-14T09:09:09.480738+0000","last_peered":"2024-09-14T09:09:09.480738+0000","last_clean":"2024-09-14T09:09:09.480738+0000","last_became_active":"2024-09-14T09:07:21.856579+0000","last_became_peered":"2024-09-14T09:07:21.856579+0000","last_unstale":"2024-09-14T09:09:09.480738+0000","last_undegraded":"2024-09-14T09:09:09.480738+0000","last_fullsized":"2024-09-14T09:09:09.480738+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:56.694930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"182'791","reported_seq":6697,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.476046+0000","last_change":"2024-09-14T09:08:34.494518+0000","last_active":"2024-09-14T09:09:09.476046+0000","last_peered":"2024-09-14T09:09:09.476046+0000","last_clean":"2024-09-14T09:09:09.476046+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:09:09.476046+0000","last_undegraded":"2024-09-14T09:09:09.476046+0000","last_fullsized":"2024-09-14T09:09:09.476046+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:58.731530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016873319999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8676,"num_read_kb":45389,"num_write":707,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":360,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.863161+0000","last_change":"2024-09-14T09:08:51.290129+0000","last_active":"2024-09-14T09:09:07.863161+0000","last_peered":"2024-09-14T09:09:07.863161+0000","last_clean":"2024-09-14T09:09:07.863161+0000","last_became_active":"2024-09-14T09:08:51.289551+0000","last_became_peered":"2024-09-14T09:08:51.289551+0000","last_unstale":"2024-09-14T09:09:07.863161+0000","last_undegraded":"2024-09-14T09:09:07.863161+0000","last_fullsized":"2024-09-14T09:09:07.863161+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:52:01.486956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"182'1187","reported_seq":7262,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.040014+0000","last_change":"2024-09-14T09:08:34.626223+0000","last_active":"2024-09-14T09:09:09.040014+0000","last_peered":"2024-09-14T09:09:09.040014+0000","last_clean":"2024-09-14T09:09:09.040014+0000","last_became_active":"2024-09-14T09:07:26.102666+0000","last_became_peered":"2024-09-14T09:07:26.102666+0000","last_unstale":"2024-09-14T09:09:09.040014+0000","last_undegraded":"2024-09-14T09:09:09.040014+0000","last_fullsized":"2024-09-14T09:09:09.040014+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:03:41.035208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011116867000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10102,"num_read_kb":27247,"num_write":1623,"num_write_kb":13456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":8334712,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":204,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039803+0000","last_change":"2024-09-14T09:07:21.863290+0000","last_active":"2024-09-14T09:09:08.039803+0000","last_peered":"2024-09-14T09:09:08.039803+0000","last_clean":"2024-09-14T09:09:08.039803+0000","last_became_active":"2024-09-14T09:07:21.862352+0000","last_became_peered":"2024-09-14T09:07:21.862352+0000","last_unstale":"2024-09-14T09:09:08.039803+0000","last_undegraded":"2024-09-14T09:09:08.039803+0000","last_fullsized":"2024-09-14T09:09:08.039803+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:48:30.332891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"179'922","reported_seq":5889,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.862993+0000","last_change":"2024-09-14T09:08:50.627160+0000","last_active":"2024-09-14T09:09:07.862993+0000","last_peered":"2024-09-14T09:09:07.862993+0000","last_clean":"2024-09-14T09:09:07.862993+0000","last_became_active":"2024-09-14T09:08:50.625357+0000","last_became_peered":"2024-09-14T09:08:50.625357+0000","last_unstale":"2024-09-14T09:09:07.862993+0000","last_undegraded":"2024-09-14T09:09:07.862993+0000","last_fullsized":"2024-09-14T09:09:07.862993+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:18:31.289028+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0021152969999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7783,"num_read_kb":22826,"num_write":1134,"num_write_kb":9259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":8151041,"num_keys_recovered":91,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":360,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.863248+0000","last_change":"2024-09-14T09:08:50.628307+0000","last_active":"2024-09-14T09:09:07.863248+0000","last_peered":"2024-09-14T09:09:07.863248+0000","last_clean":"2024-09-14T09:09:07.863248+0000","last_became_active":"2024-09-14T09:08:50.627971+0000","last_became_peered":"2024-09-14T09:08:50.627971+0000","last_unstale":"2024-09-14T09:09:07.863248+0000","last_undegraded":"2024-09-14T09:09:07.863248+0000","last_fullsized":"2024-09-14T09:09:07.863248+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:57.655435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"182'688","reported_seq":2924,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.871775+0000","last_change":"2024-09-14T09:08:34.495006+0000","last_active":"2024-09-14T09:09:07.871775+0000","last_peered":"2024-09-14T09:09:07.871775+0000","last_clean":"2024-09-14T09:09:07.871775+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:09:07.871775+0000","last_undegraded":"2024-09-14T09:09:07.871775+0000","last_fullsized":"2024-09-14T09:09:07.871775+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:25:08.203494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030472329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3594,"num_read_kb":16894,"num_write":791,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"190'884","reported_seq":6718,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.868090+0000","last_change":"2024-09-14T09:08:46.793435+0000","last_active":"2024-09-14T09:09:07.868090+0000","last_peered":"2024-09-14T09:09:07.868090+0000","last_clean":"2024-09-14T09:09:07.868090+0000","last_became_active":"2024-09-14T09:08:46.597293+0000","last_became_peered":"2024-09-14T09:08:46.597293+0000","last_unstale":"2024-09-14T09:09:07.868090+0000","last_undegraded":"2024-09-14T09:09:07.868090+0000","last_fullsized":"2024-09-14T09:09:07.868090+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:48:54.046569+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.19293607300000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8248,"num_read_kb":26883,"num_write":890,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":400,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.870896+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:09:07.870896+0000","last_peered":"2024-09-14T09:09:07.870896+0000","last_clean":"2024-09-14T09:09:07.870896+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:09:07.870896+0000","last_undegraded":"2024-09-14T09:09:07.870896+0000","last_fullsized":"2024-09-14T09:09:07.870896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"180'399","reported_seq":2539,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.863059+0000","last_change":"2024-09-14T09:08:50.625390+0000","last_active":"2024-09-14T09:09:07.863059+0000","last_peered":"2024-09-14T09:09:07.863059+0000","last_clean":"2024-09-14T09:09:07.863059+0000","last_became_active":"2024-09-14T09:08:50.622578+0000","last_became_peered":"2024-09-14T09:08:50.622578+0000","last_unstale":"2024-09-14T09:09:07.863059+0000","last_undegraded":"2024-09-14T09:09:07.863059+0000","last_fullsized":"2024-09-14T09:09:07.863059+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:50:11.634631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00245612,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1873,"num_read_kb":11982,"num_write":352,"num_write_kb":5539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":397,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.861950+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:09:07.861950+0000","last_peered":"2024-09-14T09:09:07.861950+0000","last_clean":"2024-09-14T09:09:07.861950+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:09:07.861950+0000","last_undegraded":"2024-09-14T09:09:07.861950+0000","last_fullsized":"2024-09-14T09:09:07.861950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"187'460","reported_seq":5083,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.475799+0000","last_change":"2024-09-14T09:08:43.789977+0000","last_active":"2024-09-14T09:09:09.475799+0000","last_peered":"2024-09-14T09:09:09.475799+0000","last_clean":"2024-09-14T09:09:09.475799+0000","last_became_active":"2024-09-14T09:08:43.568923+0000","last_became_peered":"2024-09-14T09:08:43.568923+0000","last_unstale":"2024-09-14T09:09:09.475799+0000","last_undegraded":"2024-09-14T09:09:09.475799+0000","last_fullsized":"2024-09-14T09:09:09.475799+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:45:29.077592+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.165000182,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3905,"num_read_kb":25149,"num_write":375,"num_write_kb":14483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.867673+0000","last_change":"2024-09-14T09:08:44.554373+0000","last_active":"2024-09-14T09:09:07.867673+0000","last_peered":"2024-09-14T09:09:07.867673+0000","last_clean":"2024-09-14T09:09:07.867673+0000","last_became_active":"2024-09-14T09:08:44.553643+0000","last_became_peered":"2024-09-14T09:08:44.553643+0000","last_unstale":"2024-09-14T09:09:07.867673+0000","last_undegraded":"2024-09-14T09:09:07.867673+0000","last_fullsized":"2024-09-14T09:09:07.867673+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:32:52.900856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"181'433","reported_seq":2984,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039700+0000","last_change":"2024-09-14T09:08:34.620964+0000","last_active":"2024-09-14T09:09:09.039700+0000","last_peered":"2024-09-14T09:09:09.039700+0000","last_clean":"2024-09-14T09:09:09.039700+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:09:09.039700+0000","last_undegraded":"2024-09-14T09:09:09.039700+0000","last_fullsized":"2024-09-14T09:09:09.039700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:34:59.882295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065530470000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2291,"num_read_kb":11209,"num_write":432,"num_write_kb":6866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":128,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.870941+0000","last_change":"2024-09-14T09:08:06.210516+0000","last_active":"2024-09-14T09:09:07.870941+0000","last_peered":"2024-09-14T09:09:07.870941+0000","last_clean":"2024-09-14T09:09:07.870941+0000","last_became_active":"2024-09-14T09:08:06.205108+0000","last_became_peered":"2024-09-14T09:08:06.205108+0000","last_unstale":"2024-09-14T09:09:07.870941+0000","last_undegraded":"2024-09-14T09:09:07.870941+0000","last_fullsized":"2024-09-14T09:09:07.870941+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:22:05.543865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":210,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039335+0000","last_change":"2024-09-14T09:07:21.876180+0000","last_active":"2024-09-14T09:09:08.039335+0000","last_peered":"2024-09-14T09:09:08.039335+0000","last_clean":"2024-09-14T09:09:08.039335+0000","last_became_active":"2024-09-14T09:07:21.860117+0000","last_became_peered":"2024-09-14T09:07:21.860117+0000","last_unstale":"2024-09-14T09:09:08.039335+0000","last_undegraded":"2024-09-14T09:09:08.039335+0000","last_fullsized":"2024-09-14T09:09:08.039335+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:58:36.142971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"178'682","reported_seq":7043,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039644+0000","last_change":"2024-09-14T09:08:34.623351+0000","last_active":"2024-09-14T09:09:09.039644+0000","last_peered":"2024-09-14T09:09:09.039644+0000","last_clean":"2024-09-14T09:09:09.039644+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:09:09.039644+0000","last_undegraded":"2024-09-14T09:09:09.039644+0000","last_fullsized":"2024-09-14T09:09:09.039644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:48.184932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089062680000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7688,"num_read_kb":41730,"num_write":748,"num_write_kb":19769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"181'469","reported_seq":3214,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.480643+0000","last_change":"2024-09-14T09:08:35.139769+0000","last_active":"2024-09-14T09:09:09.480643+0000","last_peered":"2024-09-14T09:09:09.480643+0000","last_clean":"2024-09-14T09:09:09.480643+0000","last_became_active":"2024-09-14T09:07:06.851020+0000","last_became_peered":"2024-09-14T09:07:06.851020+0000","last_unstale":"2024-09-14T09:09:09.480643+0000","last_undegraded":"2024-09-14T09:09:09.480643+0000","last_fullsized":"2024-09-14T09:09:09.480643+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:19:55.337692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154644,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2080,"num_read_kb":17783,"num_write":389,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":12315549,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"187'749","reported_seq":5088,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.480676+0000","last_change":"2024-09-14T09:08:43.990954+0000","last_active":"2024-09-14T09:09:09.480676+0000","last_peered":"2024-09-14T09:09:09.480676+0000","last_clean":"2024-09-14T09:09:09.480676+0000","last_became_active":"2024-09-14T09:08:43.580245+0000","last_became_peered":"2024-09-14T09:08:43.580245+0000","last_unstale":"2024-09-14T09:09:09.480676+0000","last_undegraded":"2024-09-14T09:09:09.480676+0000","last_fullsized":"2024-09-14T09:09:09.480676+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:04.798520+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.116960367,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5499,"num_read_kb":39102,"num_write":883,"num_write_kb":19962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"181'544","reported_seq":9831,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039772+0000","last_change":"2024-09-14T09:08:34.624404+0000","last_active":"2024-09-14T09:09:09.039772+0000","last_peered":"2024-09-14T09:09:09.039772+0000","last_clean":"2024-09-14T09:09:09.039772+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:09:09.039772+0000","last_undegraded":"2024-09-14T09:09:09.039772+0000","last_fullsized":"2024-09-14T09:09:09.039772+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:54:24.811934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097700640000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10396,"num_read_kb":61123,"num_write":490,"num_write_kb":11301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"180'493","reported_seq":3396,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.871076+0000","last_change":"2024-09-14T09:08:34.496008+0000","last_active":"2024-09-14T09:09:07.871076+0000","last_peered":"2024-09-14T09:09:07.871076+0000","last_clean":"2024-09-14T09:09:07.871076+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:09:07.871076+0000","last_undegraded":"2024-09-14T09:09:07.871076+0000","last_fullsized":"2024-09-14T09:09:07.871076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:11.774708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00388507,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3266,"num_read_kb":13215,"num_write":426,"num_write_kb":10574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"179'501","reported_seq":5776,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.870536+0000","last_change":"2024-09-14T09:09:07.870373+0000","last_active":"2024-09-14T09:09:07.870536+0000","last_peered":"2024-09-14T09:09:07.870536+0000","last_clean":"2024-09-14T09:09:07.870536+0000","last_became_active":"2024-09-14T09:09:07.867771+0000","last_became_peered":"2024-09-14T09:09:07.867771+0000","last_unstale":"2024-09-14T09:09:07.870536+0000","last_undegraded":"2024-09-14T09:09:07.870536+0000","last_fullsized":"2024-09-14T09:09:07.870536+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:55:24.914026+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022704650000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5399,"num_read_kb":32061,"num_write":547,"num_write_kb":21536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":25396220,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"187'488","reported_seq":4038,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039610+0000","last_change":"2024-09-14T09:08:44.221618+0000","last_active":"2024-09-14T09:09:08.039610+0000","last_peered":"2024-09-14T09:09:08.039610+0000","last_clean":"2024-09-14T09:09:08.039610+0000","last_became_active":"2024-09-14T09:08:43.588233+0000","last_became_peered":"2024-09-14T09:08:43.588233+0000","last_unstale":"2024-09-14T09:09:08.039610+0000","last_undegraded":"2024-09-14T09:09:08.039610+0000","last_fullsized":"2024-09-14T09:09:08.039610+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:10.863128+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.14577486200000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4282,"num_read_kb":17652,"num_write":409,"num_write_kb":7874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":14123394,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"187'586","reported_seq":4291,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039835+0000","last_change":"2024-09-14T09:08:44.145761+0000","last_active":"2024-09-14T09:09:08.039835+0000","last_peered":"2024-09-14T09:09:08.039835+0000","last_clean":"2024-09-14T09:09:08.039835+0000","last_became_active":"2024-09-14T09:08:43.574566+0000","last_became_peered":"2024-09-14T09:08:43.574566+0000","last_unstale":"2024-09-14T09:09:08.039835+0000","last_undegraded":"2024-09-14T09:09:08.039835+0000","last_fullsized":"2024-09-14T09:09:08.039835+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:30.502086+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.141382223,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3641,"num_read_kb":14849,"num_write":554,"num_write_kb":12244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":11864981,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'506","reported_seq":6246,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.480599+0000","last_change":"2024-09-14T09:08:35.140559+0000","last_active":"2024-09-14T09:09:09.480599+0000","last_peered":"2024-09-14T09:09:09.480599+0000","last_clean":"2024-09-14T09:09:09.480599+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:09:09.480599+0000","last_undegraded":"2024-09-14T09:09:09.480599+0000","last_fullsized":"2024-09-14T09:09:09.480599+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:31:57.906202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018540480000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5422,"num_read_kb":60065,"num_write":521,"num_write_kb":20523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"187'649","reported_seq":6148,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039335+0000","last_change":"2024-09-14T09:08:43.902203+0000","last_active":"2024-09-14T09:09:08.039335+0000","last_peered":"2024-09-14T09:09:08.039335+0000","last_clean":"2024-09-14T09:09:08.039335+0000","last_became_active":"2024-09-14T09:08:43.577551+0000","last_became_peered":"2024-09-14T09:08:43.577551+0000","last_unstale":"2024-09-14T09:09:08.039335+0000","last_undegraded":"2024-09-14T09:09:08.039335+0000","last_fullsized":"2024-09-14T09:09:08.039335+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:43:37.090068+0000","scrub_duration":0,"objects_trimmed":64,"snaptrim_duration":0.121356326,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6082,"num_read_kb":13595,"num_write":653,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":7405746,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"182'790","reported_seq":6130,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.475812+0000","last_change":"2024-09-14T09:08:34.493569+0000","last_active":"2024-09-14T09:09:09.475812+0000","last_peered":"2024-09-14T09:09:09.475812+0000","last_clean":"2024-09-14T09:09:09.475812+0000","last_became_active":"2024-09-14T09:07:26.101586+0000","last_became_peered":"2024-09-14T09:07:26.101586+0000","last_unstale":"2024-09-14T09:09:09.475812+0000","last_undegraded":"2024-09-14T09:09:09.475812+0000","last_fullsized":"2024-09-14T09:09:09.475812+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:11:20.433650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040594799999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7044,"num_read_kb":19519,"num_write":992,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":10260744,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"187'536","reported_seq":6576,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039395+0000","last_change":"2024-09-14T09:08:44.016919+0000","last_active":"2024-09-14T09:09:08.039395+0000","last_peered":"2024-09-14T09:09:08.039395+0000","last_clean":"2024-09-14T09:09:08.039395+0000","last_became_active":"2024-09-14T09:08:43.579997+0000","last_became_peered":"2024-09-14T09:08:43.579997+0000","last_unstale":"2024-09-14T09:09:08.039395+0000","last_undegraded":"2024-09-14T09:09:08.039395+0000","last_fullsized":"2024-09-14T09:09:08.039395+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:24:16.807890+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.10056556899999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6839,"num_read_kb":29296,"num_write":449,"num_write_kb":10682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":24080444,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":281,"num_object_clones":0,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":277,"num_read":198212,"num_read_kb":860220,"num_write":25088,"num_write_kb":385926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4177,"num_bytes_recovered":201826576,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},"log_size":22338,"ondisk_log_size":22338,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20292,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509493,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29512,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342328,"statfs":{"total":96636764160,"available":96606543872,"internally_reserved":0,"allocated":2543616,"data_stored":1231448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509493,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78096,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293744,"statfs":{"total":96636764160,"available":96556793856,"internally_reserved":0,"allocated":2547712,"data_stored":1231467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509494,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47564,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324276,"statfs":{"total":96636764160,"available":96588058624,"internally_reserved":0,"allocated":2543616,"data_stored":1231448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509496,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64256,"kb_used_data":2540,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307584,"statfs":{"total":96636764160,"available":96570966016,"internally_reserved":0,"allocated":2600960,"data_stored":1821816,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542196,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51760,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320080,"statfs":{"total":96636764160,"available":96583761920,"internally_reserved":0,"allocated":2584576,"data_stored":1821816,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":186,"seq":798863917064,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35980,"kb_used_data":2488,"kb_used_omap":62,"kb_used_meta":33409,"kb_avail":94335860,"statfs":{"total":96636764160,"available":96599920640,"internally_reserved":0,"allocated":2547712,"data_stored":1231467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":63771,"internal_metadata":34211557},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542197,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60040,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311800,"statfs":{"total":96636764160,"available":96575283200,"internally_reserved":0,"allocated":2543616,"data_stored":1231448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542198,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55888,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315952,"statfs":{"total":96636764160,"available":96579534848,"internally_reserved":0,"allocated":2551808,"data_stored":1231486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:09:16.803 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T09:09:16.803 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:09:17.098 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:09:17.099 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:09:17.115 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-14T09:09:16.028774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":285,"num_object_clones":0,"num_object_copies":570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":285,"num_whiteouts":277,"num_read":198278,"num_read_kb":860275,"num_write":25191,"num_write_kb":387556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4182,"num_bytes_recovered":203007312,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22413,"ondisk_log_size":22413,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":423096,"kb_used_data":19984,"kb_used_omap":73,"kb_used_meta":402870,"kb_avail":754551624,"statfs":{"total":773094113280,"available":772660862976,"internally_reserved":0,"allocated":20463616,"data_stored":11032396,"data_compressed":13252,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":74928,"internal_metadata":412539728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.167394"},"pg_stats":[{"pgid":"3.1f","version":"182'1763","reported_seq":7860,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.476110+0000","last_change":"2024-09-14T09:08:34.495405+0000","last_active":"2024-09-14T09:09:09.476110+0000","last_peered":"2024-09-14T09:09:09.476110+0000","last_clean":"2024-09-14T09:09:09.476110+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:09:09.476110+0000","last_undegraded":"2024-09-14T09:09:09.476110+0000","last_fullsized":"2024-09-14T09:09:09.476110+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1763,"log_dups_size":0,"ondisk_log_size":1763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:22:02.551926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022047759999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13833,"num_read_kb":24717,"num_write":2684,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'466","reported_seq":5989,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.040029+0000","last_change":"2024-09-14T09:08:34.625492+0000","last_active":"2024-09-14T09:09:09.040029+0000","last_peered":"2024-09-14T09:09:09.040029+0000","last_clean":"2024-09-14T09:09:09.040029+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:09:09.040029+0000","last_undegraded":"2024-09-14T09:09:09.040029+0000","last_fullsized":"2024-09-14T09:09:09.040029+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:01.005568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010401127,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5032,"num_read_kb":31996,"num_write":418,"num_write_kb":13378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"187'676","reported_seq":6616,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.871638+0000","last_change":"2024-09-14T09:08:43.847915+0000","last_active":"2024-09-14T09:09:07.871638+0000","last_peered":"2024-09-14T09:09:07.871638+0000","last_clean":"2024-09-14T09:09:07.871638+0000","last_became_active":"2024-09-14T09:08:43.573212+0000","last_became_peered":"2024-09-14T09:08:43.573212+0000","last_unstale":"2024-09-14T09:09:07.871638+0000","last_undegraded":"2024-09-14T09:09:07.871638+0000","last_fullsized":"2024-09-14T09:09:07.871638+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:26:03.155997+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.147750145,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6669,"num_read_kb":58049,"num_write":727,"num_write_kb":12436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"190'866","reported_seq":5645,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.870986+0000","last_change":"2024-09-14T09:08:46.770708+0000","last_active":"2024-09-14T09:09:07.870986+0000","last_peered":"2024-09-14T09:09:07.870986+0000","last_clean":"2024-09-14T09:09:07.870986+0000","last_became_active":"2024-09-14T09:08:46.599475+0000","last_became_peered":"2024-09-14T09:08:46.599475+0000","last_unstale":"2024-09-14T09:09:07.870986+0000","last_undegraded":"2024-09-14T09:09:07.870986+0000","last_fullsized":"2024-09-14T09:09:07.870986+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:57.157599+0000","scrub_duration":0,"objects_trimmed":60,"snaptrim_duration":0.168855903,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7679,"num_read_kb":27474,"num_write":1043,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'742","reported_seq":5718,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.476034+0000","last_change":"2024-09-14T09:08:34.491862+0000","last_active":"2024-09-14T09:09:09.476034+0000","last_peered":"2024-09-14T09:09:09.476034+0000","last_clean":"2024-09-14T09:09:09.476034+0000","last_became_active":"2024-09-14T09:07:06.842337+0000","last_became_peered":"2024-09-14T09:07:06.842337+0000","last_unstale":"2024-09-14T09:09:09.476034+0000","last_undegraded":"2024-09-14T09:09:09.476034+0000","last_fullsized":"2024-09-14T09:09:09.476034+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:23:25.823078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023564419999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6703,"num_read_kb":24544,"num_write":874,"num_write_kb":16463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":14145319,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"187'724","reported_seq":5512,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.867745+0000","last_change":"2024-09-14T09:08:43.771622+0000","last_active":"2024-09-14T09:09:07.867745+0000","last_peered":"2024-09-14T09:09:07.867745+0000","last_clean":"2024-09-14T09:09:07.867745+0000","last_became_active":"2024-09-14T09:08:43.556338+0000","last_became_peered":"2024-09-14T09:08:43.556338+0000","last_unstale":"2024-09-14T09:09:07.867745+0000","last_undegraded":"2024-09-14T09:09:07.867745+0000","last_fullsized":"2024-09-14T09:09:07.867745+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:41.389225+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.12752904200000001,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6399,"num_read_kb":30687,"num_write":963,"num_write_kb":14233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"174'375","reported_seq":4400,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.862853+0000","last_change":"2024-09-14T09:08:50.629874+0000","last_active":"2024-09-14T09:09:07.862853+0000","last_peered":"2024-09-14T09:09:07.862853+0000","last_clean":"2024-09-14T09:09:07.862853+0000","last_became_active":"2024-09-14T09:08:50.627814+0000","last_became_peered":"2024-09-14T09:08:50.627814+0000","last_unstale":"2024-09-14T09:09:07.862853+0000","last_undegraded":"2024-09-14T09:09:07.862853+0000","last_fullsized":"2024-09-14T09:09:07.862853+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:14:43.966792+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.025218744000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3327,"num_read_kb":10838,"num_write":357,"num_write_kb":6673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":3727781,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"182'624","reported_seq":6380,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039812+0000","last_change":"2024-09-14T09:08:34.622411+0000","last_active":"2024-09-14T09:09:09.039812+0000","last_peered":"2024-09-14T09:09:09.039812+0000","last_clean":"2024-09-14T09:09:09.039812+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:09:09.039812+0000","last_undegraded":"2024-09-14T09:09:09.039812+0000","last_fullsized":"2024-09-14T09:09:09.039812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:55:38.962684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080911500000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7486,"num_read_kb":17479,"num_write":535,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"182'910","reported_seq":7829,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.476168+0000","last_change":"2024-09-14T09:08:34.496061+0000","last_active":"2024-09-14T09:09:09.476168+0000","last_peered":"2024-09-14T09:09:09.476168+0000","last_clean":"2024-09-14T09:09:09.476168+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:09:09.476168+0000","last_undegraded":"2024-09-14T09:09:09.476168+0000","last_fullsized":"2024-09-14T09:09:09.476168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:02:38.220394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026116440000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10070,"num_read_kb":35479,"num_write":1129,"num_write_kb":10795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"179'629","reported_seq":3952,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.871137+0000","last_change":"2024-09-14T09:08:34.493205+0000","last_active":"2024-09-14T09:09:07.871137+0000","last_peered":"2024-09-14T09:09:07.871137+0000","last_clean":"2024-09-14T09:09:07.871137+0000","last_became_active":"2024-09-14T09:07:25.105674+0000","last_became_peered":"2024-09-14T09:07:25.105674+0000","last_unstale":"2024-09-14T09:09:07.871137+0000","last_undegraded":"2024-09-14T09:09:07.871137+0000","last_fullsized":"2024-09-14T09:09:07.871137+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:20:01.627411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001781447,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4726,"num_read_kb":15599,"num_write":700,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":8467448,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"181'1136","reported_seq":5393,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039887+0000","last_change":"2024-09-14T09:08:34.507525+0000","last_active":"2024-09-14T09:09:08.039887+0000","last_peered":"2024-09-14T09:09:08.039887+0000","last_clean":"2024-09-14T09:09:08.039887+0000","last_became_active":"2024-09-14T09:07:27.854998+0000","last_became_peered":"2024-09-14T09:07:27.854998+0000","last_unstale":"2024-09-14T09:09:08.039887+0000","last_undegraded":"2024-09-14T09:09:08.039887+0000","last_fullsized":"2024-09-14T09:09:08.039887+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:11:04.062027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001704445,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8595,"num_read_kb":26590,"num_write":1602,"num_write_kb":14717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":4170600,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"187'670","reported_seq":3459,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.868133+0000","last_change":"2024-09-14T09:08:43.840463+0000","last_active":"2024-09-14T09:09:07.868133+0000","last_peered":"2024-09-14T09:09:07.868133+0000","last_clean":"2024-09-14T09:09:07.868133+0000","last_became_active":"2024-09-14T09:08:43.569123+0000","last_became_peered":"2024-09-14T09:08:43.569123+0000","last_unstale":"2024-09-14T09:09:07.868133+0000","last_undegraded":"2024-09-14T09:09:07.868133+0000","last_fullsized":"2024-09-14T09:09:07.868133+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:18.861184+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.15505904000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3583,"num_read_kb":9199,"num_write":691,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":2686146,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":407,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.480738+0000","last_change":"2024-09-14T09:07:21.857426+0000","last_active":"2024-09-14T09:09:09.480738+0000","last_peered":"2024-09-14T09:09:09.480738+0000","last_clean":"2024-09-14T09:09:09.480738+0000","last_became_active":"2024-09-14T09:07:21.856579+0000","last_became_peered":"2024-09-14T09:07:21.856579+0000","last_unstale":"2024-09-14T09:09:09.480738+0000","last_undegraded":"2024-09-14T09:09:09.480738+0000","last_fullsized":"2024-09-14T09:09:09.480738+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:56.694930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"182'791","reported_seq":6697,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.476046+0000","last_change":"2024-09-14T09:08:34.494518+0000","last_active":"2024-09-14T09:09:09.476046+0000","last_peered":"2024-09-14T09:09:09.476046+0000","last_clean":"2024-09-14T09:09:09.476046+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:09:09.476046+0000","last_undegraded":"2024-09-14T09:09:09.476046+0000","last_fullsized":"2024-09-14T09:09:09.476046+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:58.731530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016873319999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8676,"num_read_kb":45389,"num_write":707,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":360,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.863161+0000","last_change":"2024-09-14T09:08:51.290129+0000","last_active":"2024-09-14T09:09:07.863161+0000","last_peered":"2024-09-14T09:09:07.863161+0000","last_clean":"2024-09-14T09:09:07.863161+0000","last_became_active":"2024-09-14T09:08:51.289551+0000","last_became_peered":"2024-09-14T09:08:51.289551+0000","last_unstale":"2024-09-14T09:09:07.863161+0000","last_undegraded":"2024-09-14T09:09:07.863161+0000","last_fullsized":"2024-09-14T09:09:07.863161+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:52:01.486956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"182'1187","reported_seq":7262,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.040014+0000","last_change":"2024-09-14T09:08:34.626223+0000","last_active":"2024-09-14T09:09:09.040014+0000","last_peered":"2024-09-14T09:09:09.040014+0000","last_clean":"2024-09-14T09:09:09.040014+0000","last_became_active":"2024-09-14T09:07:26.102666+0000","last_became_peered":"2024-09-14T09:07:26.102666+0000","last_unstale":"2024-09-14T09:09:09.040014+0000","last_undegraded":"2024-09-14T09:09:09.040014+0000","last_fullsized":"2024-09-14T09:09:09.040014+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:03:41.035208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011116867000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10102,"num_read_kb":27247,"num_write":1623,"num_write_kb":13456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":8334712,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":204,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039803+0000","last_change":"2024-09-14T09:07:21.863290+0000","last_active":"2024-09-14T09:09:08.039803+0000","last_peered":"2024-09-14T09:09:08.039803+0000","last_clean":"2024-09-14T09:09:08.039803+0000","last_became_active":"2024-09-14T09:07:21.862352+0000","last_became_peered":"2024-09-14T09:07:21.862352+0000","last_unstale":"2024-09-14T09:09:08.039803+0000","last_undegraded":"2024-09-14T09:09:08.039803+0000","last_fullsized":"2024-09-14T09:09:08.039803+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:48:30.332891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"179'922","reported_seq":5889,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.862993+0000","last_change":"2024-09-14T09:08:50.627160+0000","last_active":"2024-09-14T09:09:07.862993+0000","last_peered":"2024-09-14T09:09:07.862993+0000","last_clean":"2024-09-14T09:09:07.862993+0000","last_became_active":"2024-09-14T09:08:50.625357+0000","last_became_peered":"2024-09-14T09:08:50.625357+0000","last_unstale":"2024-09-14T09:09:07.862993+0000","last_undegraded":"2024-09-14T09:09:07.862993+0000","last_fullsized":"2024-09-14T09:09:07.862993+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:18:31.289028+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0021152969999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7783,"num_read_kb":22826,"num_write":1134,"num_write_kb":9259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":8151041,"num_keys_recovered":91,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":360,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.863248+0000","last_change":"2024-09-14T09:08:50.628307+0000","last_active":"2024-09-14T09:09:07.863248+0000","last_peered":"2024-09-14T09:09:07.863248+0000","last_clean":"2024-09-14T09:09:07.863248+0000","last_became_active":"2024-09-14T09:08:50.627971+0000","last_became_peered":"2024-09-14T09:08:50.627971+0000","last_unstale":"2024-09-14T09:09:07.863248+0000","last_undegraded":"2024-09-14T09:09:07.863248+0000","last_fullsized":"2024-09-14T09:09:07.863248+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:57.655435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"182'688","reported_seq":2924,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.871775+0000","last_change":"2024-09-14T09:08:34.495006+0000","last_active":"2024-09-14T09:09:07.871775+0000","last_peered":"2024-09-14T09:09:07.871775+0000","last_clean":"2024-09-14T09:09:07.871775+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:09:07.871775+0000","last_undegraded":"2024-09-14T09:09:07.871775+0000","last_fullsized":"2024-09-14T09:09:07.871775+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:25:08.203494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030472329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3594,"num_read_kb":16894,"num_write":791,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"190'884","reported_seq":6718,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.868090+0000","last_change":"2024-09-14T09:08:46.793435+0000","last_active":"2024-09-14T09:09:07.868090+0000","last_peered":"2024-09-14T09:09:07.868090+0000","last_clean":"2024-09-14T09:09:07.868090+0000","last_became_active":"2024-09-14T09:08:46.597293+0000","last_became_peered":"2024-09-14T09:08:46.597293+0000","last_unstale":"2024-09-14T09:09:07.868090+0000","last_undegraded":"2024-09-14T09:09:07.868090+0000","last_fullsized":"2024-09-14T09:09:07.868090+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:48:54.046569+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.19293607300000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8248,"num_read_kb":26883,"num_write":890,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":400,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.870896+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:09:07.870896+0000","last_peered":"2024-09-14T09:09:07.870896+0000","last_clean":"2024-09-14T09:09:07.870896+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:09:07.870896+0000","last_undegraded":"2024-09-14T09:09:07.870896+0000","last_fullsized":"2024-09-14T09:09:07.870896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"180'399","reported_seq":2539,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.863059+0000","last_change":"2024-09-14T09:08:50.625390+0000","last_active":"2024-09-14T09:09:07.863059+0000","last_peered":"2024-09-14T09:09:07.863059+0000","last_clean":"2024-09-14T09:09:07.863059+0000","last_became_active":"2024-09-14T09:08:50.622578+0000","last_became_peered":"2024-09-14T09:08:50.622578+0000","last_unstale":"2024-09-14T09:09:07.863059+0000","last_undegraded":"2024-09-14T09:09:07.863059+0000","last_fullsized":"2024-09-14T09:09:07.863059+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:50:11.634631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00245612,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1873,"num_read_kb":11982,"num_write":352,"num_write_kb":5539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":397,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.861950+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:09:07.861950+0000","last_peered":"2024-09-14T09:09:07.861950+0000","last_clean":"2024-09-14T09:09:07.861950+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:09:07.861950+0000","last_undegraded":"2024-09-14T09:09:07.861950+0000","last_fullsized":"2024-09-14T09:09:07.861950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"187'460","reported_seq":5083,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.475799+0000","last_change":"2024-09-14T09:08:43.789977+0000","last_active":"2024-09-14T09:09:09.475799+0000","last_peered":"2024-09-14T09:09:09.475799+0000","last_clean":"2024-09-14T09:09:09.475799+0000","last_became_active":"2024-09-14T09:08:43.568923+0000","last_became_peered":"2024-09-14T09:08:43.568923+0000","last_unstale":"2024-09-14T09:09:09.475799+0000","last_undegraded":"2024-09-14T09:09:09.475799+0000","last_fullsized":"2024-09-14T09:09:09.475799+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:45:29.077592+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.165000182,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3905,"num_read_kb":25149,"num_write":375,"num_write_kb":14483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.867673+0000","last_change":"2024-09-14T09:08:44.554373+0000","last_active":"2024-09-14T09:09:07.867673+0000","last_peered":"2024-09-14T09:09:07.867673+0000","last_clean":"2024-09-14T09:09:07.867673+0000","last_became_active":"2024-09-14T09:08:44.553643+0000","last_became_peered":"2024-09-14T09:08:44.553643+0000","last_unstale":"2024-09-14T09:09:07.867673+0000","last_undegraded":"2024-09-14T09:09:07.867673+0000","last_fullsized":"2024-09-14T09:09:07.867673+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:32:52.900856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"181'433","reported_seq":2984,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039700+0000","last_change":"2024-09-14T09:08:34.620964+0000","last_active":"2024-09-14T09:09:09.039700+0000","last_peered":"2024-09-14T09:09:09.039700+0000","last_clean":"2024-09-14T09:09:09.039700+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:09:09.039700+0000","last_undegraded":"2024-09-14T09:09:09.039700+0000","last_fullsized":"2024-09-14T09:09:09.039700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:34:59.882295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065530470000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2291,"num_read_kb":11209,"num_write":432,"num_write_kb":6866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":128,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.870941+0000","last_change":"2024-09-14T09:08:06.210516+0000","last_active":"2024-09-14T09:09:07.870941+0000","last_peered":"2024-09-14T09:09:07.870941+0000","last_clean":"2024-09-14T09:09:07.870941+0000","last_became_active":"2024-09-14T09:08:06.205108+0000","last_became_peered":"2024-09-14T09:08:06.205108+0000","last_unstale":"2024-09-14T09:09:07.870941+0000","last_undegraded":"2024-09-14T09:09:07.870941+0000","last_fullsized":"2024-09-14T09:09:07.870941+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:22:05.543865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":210,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039335+0000","last_change":"2024-09-14T09:07:21.876180+0000","last_active":"2024-09-14T09:09:08.039335+0000","last_peered":"2024-09-14T09:09:08.039335+0000","last_clean":"2024-09-14T09:09:08.039335+0000","last_became_active":"2024-09-14T09:07:21.860117+0000","last_became_peered":"2024-09-14T09:07:21.860117+0000","last_unstale":"2024-09-14T09:09:08.039335+0000","last_undegraded":"2024-09-14T09:09:08.039335+0000","last_fullsized":"2024-09-14T09:09:08.039335+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:58:36.142971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"178'682","reported_seq":7043,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039644+0000","last_change":"2024-09-14T09:08:34.623351+0000","last_active":"2024-09-14T09:09:09.039644+0000","last_peered":"2024-09-14T09:09:09.039644+0000","last_clean":"2024-09-14T09:09:09.039644+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:09:09.039644+0000","last_undegraded":"2024-09-14T09:09:09.039644+0000","last_fullsized":"2024-09-14T09:09:09.039644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:48.184932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089062680000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7688,"num_read_kb":41730,"num_write":748,"num_write_kb":19769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"181'469","reported_seq":3214,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.480643+0000","last_change":"2024-09-14T09:08:35.139769+0000","last_active":"2024-09-14T09:09:09.480643+0000","last_peered":"2024-09-14T09:09:09.480643+0000","last_clean":"2024-09-14T09:09:09.480643+0000","last_became_active":"2024-09-14T09:07:06.851020+0000","last_became_peered":"2024-09-14T09:07:06.851020+0000","last_unstale":"2024-09-14T09:09:09.480643+0000","last_undegraded":"2024-09-14T09:09:09.480643+0000","last_fullsized":"2024-09-14T09:09:09.480643+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:19:55.337692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154644,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2080,"num_read_kb":17783,"num_write":389,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":12315549,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"187'749","reported_seq":5088,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.480676+0000","last_change":"2024-09-14T09:08:43.990954+0000","last_active":"2024-09-14T09:09:09.480676+0000","last_peered":"2024-09-14T09:09:09.480676+0000","last_clean":"2024-09-14T09:09:09.480676+0000","last_became_active":"2024-09-14T09:08:43.580245+0000","last_became_peered":"2024-09-14T09:08:43.580245+0000","last_unstale":"2024-09-14T09:09:09.480676+0000","last_undegraded":"2024-09-14T09:09:09.480676+0000","last_fullsized":"2024-09-14T09:09:09.480676+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:04.798520+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.116960367,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5499,"num_read_kb":39102,"num_write":883,"num_write_kb":19962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"181'544","reported_seq":9831,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039772+0000","last_change":"2024-09-14T09:08:34.624404+0000","last_active":"2024-09-14T09:09:09.039772+0000","last_peered":"2024-09-14T09:09:09.039772+0000","last_clean":"2024-09-14T09:09:09.039772+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:09:09.039772+0000","last_undegraded":"2024-09-14T09:09:09.039772+0000","last_fullsized":"2024-09-14T09:09:09.039772+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:54:24.811934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097700640000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10396,"num_read_kb":61123,"num_write":490,"num_write_kb":11301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"180'493","reported_seq":3396,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.871076+0000","last_change":"2024-09-14T09:08:34.496008+0000","last_active":"2024-09-14T09:09:07.871076+0000","last_peered":"2024-09-14T09:09:07.871076+0000","last_clean":"2024-09-14T09:09:07.871076+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:09:07.871076+0000","last_undegraded":"2024-09-14T09:09:07.871076+0000","last_fullsized":"2024-09-14T09:09:07.871076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:11.774708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00388507,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3266,"num_read_kb":13215,"num_write":426,"num_write_kb":10574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"179'501","reported_seq":5776,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.870536+0000","last_change":"2024-09-14T09:09:07.870373+0000","last_active":"2024-09-14T09:09:07.870536+0000","last_peered":"2024-09-14T09:09:07.870536+0000","last_clean":"2024-09-14T09:09:07.870536+0000","last_became_active":"2024-09-14T09:09:07.867771+0000","last_became_peered":"2024-09-14T09:09:07.867771+0000","last_unstale":"2024-09-14T09:09:07.870536+0000","last_undegraded":"2024-09-14T09:09:07.870536+0000","last_fullsized":"2024-09-14T09:09:07.870536+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:55:24.914026+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022704650000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5399,"num_read_kb":32061,"num_write":547,"num_write_kb":21536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":25396220,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"187'488","reported_seq":4038,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039610+0000","last_change":"2024-09-14T09:08:44.221618+0000","last_active":"2024-09-14T09:09:08.039610+0000","last_peered":"2024-09-14T09:09:08.039610+0000","last_clean":"2024-09-14T09:09:08.039610+0000","last_became_active":"2024-09-14T09:08:43.588233+0000","last_became_peered":"2024-09-14T09:08:43.588233+0000","last_unstale":"2024-09-14T09:09:08.039610+0000","last_undegraded":"2024-09-14T09:09:08.039610+0000","last_fullsized":"2024-09-14T09:09:08.039610+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:10.863128+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.14577486200000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4282,"num_read_kb":17652,"num_write":409,"num_write_kb":7874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":14123394,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"187'586","reported_seq":4291,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039835+0000","last_change":"2024-09-14T09:08:44.145761+0000","last_active":"2024-09-14T09:09:08.039835+0000","last_peered":"2024-09-14T09:09:08.039835+0000","last_clean":"2024-09-14T09:09:08.039835+0000","last_became_active":"2024-09-14T09:08:43.574566+0000","last_became_peered":"2024-09-14T09:08:43.574566+0000","last_unstale":"2024-09-14T09:09:08.039835+0000","last_undegraded":"2024-09-14T09:09:08.039835+0000","last_fullsized":"2024-09-14T09:09:08.039835+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:30.502086+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.141382223,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3641,"num_read_kb":14849,"num_write":554,"num_write_kb":12244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":11864981,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'506","reported_seq":6246,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.480599+0000","last_change":"2024-09-14T09:08:35.140559+0000","last_active":"2024-09-14T09:09:09.480599+0000","last_peered":"2024-09-14T09:09:09.480599+0000","last_clean":"2024-09-14T09:09:09.480599+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:09:09.480599+0000","last_undegraded":"2024-09-14T09:09:09.480599+0000","last_fullsized":"2024-09-14T09:09:09.480599+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:31:57.906202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018540480000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5422,"num_read_kb":60065,"num_write":521,"num_write_kb":20523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"187'649","reported_seq":6148,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039335+0000","last_change":"2024-09-14T09:08:43.902203+0000","last_active":"2024-09-14T09:09:08.039335+0000","last_peered":"2024-09-14T09:09:08.039335+0000","last_clean":"2024-09-14T09:09:08.039335+0000","last_became_active":"2024-09-14T09:08:43.577551+0000","last_became_peered":"2024-09-14T09:08:43.577551+0000","last_unstale":"2024-09-14T09:09:08.039335+0000","last_undegraded":"2024-09-14T09:09:08.039335+0000","last_fullsized":"2024-09-14T09:09:08.039335+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:43:37.090068+0000","scrub_duration":0,"objects_trimmed":64,"snaptrim_duration":0.121356326,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6082,"num_read_kb":13595,"num_write":653,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":7405746,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"182'790","reported_seq":6130,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.475812+0000","last_change":"2024-09-14T09:08:34.493569+0000","last_active":"2024-09-14T09:09:09.475812+0000","last_peered":"2024-09-14T09:09:09.475812+0000","last_clean":"2024-09-14T09:09:09.475812+0000","last_became_active":"2024-09-14T09:07:26.101586+0000","last_became_peered":"2024-09-14T09:07:26.101586+0000","last_unstale":"2024-09-14T09:09:09.475812+0000","last_undegraded":"2024-09-14T09:09:09.475812+0000","last_fullsized":"2024-09-14T09:09:09.475812+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:11:20.433650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040594799999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7044,"num_read_kb":19519,"num_write":992,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":10260744,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"187'536","reported_seq":6576,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039395+0000","last_change":"2024-09-14T09:08:44.016919+0000","last_active":"2024-09-14T09:09:08.039395+0000","last_peered":"2024-09-14T09:09:08.039395+0000","last_clean":"2024-09-14T09:09:08.039395+0000","last_became_active":"2024-09-14T09:08:43.579997+0000","last_became_peered":"2024-09-14T09:08:43.579997+0000","last_unstale":"2024-09-14T09:09:08.039395+0000","last_undegraded":"2024-09-14T09:09:08.039395+0000","last_fullsized":"2024-09-14T09:09:08.039395+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:24:16.807890+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.10056556899999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6839,"num_read_kb":29296,"num_write":449,"num_write_kb":10682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":24080444,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":281,"num_object_clones":0,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":277,"num_read":198212,"num_read_kb":860220,"num_write":25088,"num_write_kb":385926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4177,"num_bytes_recovered":201826576,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},"log_size":22338,"ondisk_log_size":22338,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20292,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509493,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29512,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342328,"statfs":{"total":96636764160,"available":96606543872,"internally_reserved":0,"allocated":2543616,"data_stored":1231448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509493,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78096,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293744,"statfs":{"total":96636764160,"available":96556793856,"internally_reserved":0,"allocated":2547712,"data_stored":1231467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509494,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47564,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324276,"statfs":{"total":96636764160,"available":96588058624,"internally_reserved":0,"allocated":2543616,"data_stored":1231448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509496,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64256,"kb_used_data":2540,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307584,"statfs":{"total":96636764160,"available":96570966016,"internally_reserved":0,"allocated":2600960,"data_stored":1821816,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542196,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51760,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320080,"statfs":{"total":96636764160,"available":96583761920,"internally_reserved":0,"allocated":2584576,"data_stored":1821816,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":186,"seq":798863917064,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35980,"kb_used_data":2488,"kb_used_omap":62,"kb_used_meta":33409,"kb_avail":94335860,"statfs":{"total":96636764160,"available":96599920640,"internally_reserved":0,"allocated":2547712,"data_stored":1231467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":63771,"internal_metadata":34211557},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542197,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60040,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311800,"statfs":{"total":96636764160,"available":96575283200,"internally_reserved":0,"allocated":2543616,"data_stored":1231448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542198,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55888,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315952,"statfs":{"total":96636764160,"available":96579534848,"internally_reserved":0,"allocated":2551808,"data_stored":1231486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:09:17.117 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T09:09:17.402 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:09:17.402 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":211,"fsid":"c635667d-994e-454f-bcea-51b45637b2a2","created":"2024-09-14T09:04:57.077293+0000","modified":"2024-09-14T09:09:07.854650+0000","last_up_change":"2024-09-14T09:08:42.531335+0000","last_in_change":"2024-09-14T09:07:21.060249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T09:05:11.997673+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T09:05:15.352551+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T09:05:25.840963+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":46,"snap_epoch":182,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4ceedb4f-6398-467f-9562-ccf27200860b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6825","nonce":3384710819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6827","nonce":3384710819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6831","nonce":3384710819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3384710819},{"type":"v1","addr":"172.21.15.176:6829","nonce":3384710819}]},"public_addr":"172.21.15.176:6825/3384710819","cluster_addr":"172.21.15.176:6827/3384710819","heartbeat_back_addr":"172.21.15.176:6831/3384710819","heartbeat_front_addr":"172.21.15.176:6829/3384710819","state":["exists","up"]},{"osd":1,"uuid":"604e2f62-7c13-46be-a97f-029b28c44a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6801","nonce":612012679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6803","nonce":612012679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6807","nonce":612012679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":612012679},{"type":"v1","addr":"172.21.15.176:6805","nonce":612012679}]},"public_addr":"172.21.15.176:6801/612012679","cluster_addr":"172.21.15.176:6803/612012679","heartbeat_back_addr":"172.21.15.176:6807/612012679","heartbeat_front_addr":"172.21.15.176:6805/612012679","state":["exists","up"]},{"osd":2,"uuid":"1f65589e-8b0a-4033-94c0-4ab66531db2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6809","nonce":4167379279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6811","nonce":4167379279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6815","nonce":4167379279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":4167379279},{"type":"v1","addr":"172.21.15.176:6813","nonce":4167379279}]},"public_addr":"172.21.15.176:6809/4167379279","cluster_addr":"172.21.15.176:6811/4167379279","heartbeat_back_addr":"172.21.15.176:6815/4167379279","heartbeat_front_addr":"172.21.15.176:6813/4167379279","state":["exists","up"]},{"osd":3,"uuid":"209d4f65-79f9-4a7d-9d23-832a4209507f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6817","nonce":1658209015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6819","nonce":1658209015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6823","nonce":1658209015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":1658209015},{"type":"v1","addr":"172.21.15.176:6821","nonce":1658209015}]},"public_addr":"172.21.15.176:6817/1658209015","cluster_addr":"172.21.15.176:6819/1658209015","heartbeat_back_addr":"172.21.15.176:6823/1658209015","heartbeat_front_addr":"172.21.15.176:6821/1658209015","state":["exists","up"]},{"osd":4,"uuid":"8f9a8c9b-282e-4523-9e62-5824f4aba401","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":134,"up_from":186,"up_thru":189,"down_at":135,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":700723532},{"type":"v1","addr":"172.21.15.183:6809","nonce":700723532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":700723532},{"type":"v1","addr":"172.21.15.183:6811","nonce":700723532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":700723532},{"type":"v1","addr":"172.21.15.183:6815","nonce":700723532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":700723532},{"type":"v1","addr":"172.21.15.183:6813","nonce":700723532}]},"public_addr":"172.21.15.183:6809/700723532","cluster_addr":"172.21.15.183:6811/700723532","heartbeat_back_addr":"172.21.15.183:6815/700723532","heartbeat_front_addr":"172.21.15.183:6813/700723532","state":["exists","up"]},{"osd":5,"uuid":"1a3ed4e1-4ec9-4a34-ad39-ee9ba2b9aa80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6825","nonce":1711292386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6827","nonce":1711292386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6831","nonce":1711292386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1711292386},{"type":"v1","addr":"172.21.15.183:6829","nonce":1711292386}]},"public_addr":"172.21.15.183:6825/1711292386","cluster_addr":"172.21.15.183:6827/1711292386","heartbeat_back_addr":"172.21.15.183:6831/1711292386","heartbeat_front_addr":"172.21.15.183:6829/1711292386","state":["exists","up"]},{"osd":6,"uuid":"59d9259d-b9dd-4326-8ab9-9d599a2bf57b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6817","nonce":2721245607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6819","nonce":2721245607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6823","nonce":2721245607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2721245607},{"type":"v1","addr":"172.21.15.183:6821","nonce":2721245607}]},"public_addr":"172.21.15.183:6817/2721245607","cluster_addr":"172.21.15.183:6819/2721245607","heartbeat_back_addr":"172.21.15.183:6823/2721245607","heartbeat_front_addr":"172.21.15.183:6821/2721245607","state":["exists","up"]},{"osd":7,"uuid":"e2f27d42-786b-449b-a2fb-9dd28f4e9788","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6801","nonce":2122823465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6803","nonce":2122823465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6807","nonce":2122823465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2122823465},{"type":"v1","addr":"172.21.15.183:6805","nonce":2122823465}]},"public_addr":"172.21.15.183:6801/2122823465","cluster_addr":"172.21.15.183:6803/2122823465","heartbeat_back_addr":"172.21.15.183:6807/2122823465","heartbeat_front_addr":"172.21.15.183:6805/2122823465","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.411789+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:05.801873+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.206725+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:05.882180+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.013864+0000","dead_epoch":135},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.652975+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:06.741359+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T09:05:05.873675+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.0","osds":[4,2]},{"pgid":"3.4","osds":[1,7]}],"pg_upmap_items":[{"pgid":"3.9","mappings":[{"from":4,"to":6}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":0}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T09:09:18.422 INFO:tasks.ceph:Scrubbing osd.0 2024-09-14T09:09:18.422 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-14T09:09:18.583 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:09:18.584 INFO:teuthology.orchestra.run.smithi176.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T09:09:18.584 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:09:18.599 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-14T09:09:18.886 INFO:teuthology.orchestra.run.smithi176.stderr:instructed osd(s) 0 to deep-scrub 2024-09-14T09:09:18.905 INFO:tasks.ceph:Scrubbing osd.1 2024-09-14T09:09:18.905 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-14T09:09:19.042 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:09:19.042 INFO:teuthology.orchestra.run.smithi176.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T09:09:19.042 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:09:19.057 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-14T09:09:19.341 INFO:teuthology.orchestra.run.smithi176.stderr:instructed osd(s) 1 to deep-scrub 2024-09-14T09:09:19.359 INFO:tasks.ceph:Scrubbing osd.2 2024-09-14T09:09:19.359 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-14T09:09:19.507 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:09:19.507 INFO:teuthology.orchestra.run.smithi176.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T09:09:19.507 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:09:19.522 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-14T09:09:19.807 INFO:teuthology.orchestra.run.smithi176.stderr:instructed osd(s) 2 to deep-scrub 2024-09-14T09:09:19.893 INFO:tasks.ceph:Scrubbing osd.3 2024-09-14T09:09:19.893 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-14T09:09:20.045 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:09:20.045 INFO:teuthology.orchestra.run.smithi176.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T09:09:20.045 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:09:20.060 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-14T09:09:20.346 INFO:teuthology.orchestra.run.smithi176.stderr:instructed osd(s) 3 to deep-scrub 2024-09-14T09:09:20.364 INFO:tasks.ceph:Scrubbing osd.4 2024-09-14T09:09:20.364 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-14T09:09:20.510 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:09:20.510 INFO:teuthology.orchestra.run.smithi176.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-14T09:09:20.510 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:09:20.526 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-14T09:09:20.818 INFO:teuthology.orchestra.run.smithi176.stderr:instructed osd(s) 4 to deep-scrub 2024-09-14T09:09:20.836 INFO:tasks.ceph:Scrubbing osd.5 2024-09-14T09:09:20.836 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-14T09:09:20.972 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:09:20.972 INFO:teuthology.orchestra.run.smithi176.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T09:09:20.972 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:09:20.988 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-14T09:09:21.289 INFO:teuthology.orchestra.run.smithi176.stderr:instructed osd(s) 5 to deep-scrub 2024-09-14T09:09:21.307 INFO:tasks.ceph:Scrubbing osd.6 2024-09-14T09:09:21.307 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-14T09:09:21.452 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:09:21.452 INFO:teuthology.orchestra.run.smithi176.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T09:09:21.452 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:09:21.466 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-14T09:09:21.761 INFO:teuthology.orchestra.run.smithi176.stderr:instructed osd(s) 6 to deep-scrub 2024-09-14T09:09:21.779 INFO:tasks.ceph:Scrubbing osd.7 2024-09-14T09:09:21.780 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-14T09:09:21.926 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-09-14T09:09:21.926 INFO:teuthology.orchestra.run.smithi176.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T09:09:21.926 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-09-14T09:09:21.940 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-14T09:09:22.229 INFO:teuthology.orchestra.run.smithi176.stderr:instructed osd(s) 7 to deep-scrub 2024-09-14T09:09:22.247 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:09:22.546 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:09:22.547 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:09:22.564 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-14T09:09:22.030521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":285,"num_object_clones":0,"num_object_copies":570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":285,"num_whiteouts":277,"num_read":198278,"num_read_kb":860275,"num_write":25191,"num_write_kb":387556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4182,"num_bytes_recovered":203007312,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22433,"ondisk_log_size":22433,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":423088,"kb_used_data":19984,"kb_used_omap":73,"kb_used_meta":402870,"kb_avail":754551632,"statfs":{"total":773094113280,"available":772660871168,"internally_reserved":0,"allocated":20463616,"data_stored":11032396,"data_compressed":13252,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":74928,"internal_metadata":412539728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003513"},"pg_stats":[{"pgid":"3.1f","version":"182'1763","reported_seq":7860,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.476110+0000","last_change":"2024-09-14T09:08:34.495405+0000","last_active":"2024-09-14T09:09:09.476110+0000","last_peered":"2024-09-14T09:09:09.476110+0000","last_clean":"2024-09-14T09:09:09.476110+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:09:09.476110+0000","last_undegraded":"2024-09-14T09:09:09.476110+0000","last_fullsized":"2024-09-14T09:09:09.476110+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1763,"log_dups_size":0,"ondisk_log_size":1763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:22:02.551926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022047759999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13833,"num_read_kb":24717,"num_write":2684,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"181'466","reported_seq":5989,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.040029+0000","last_change":"2024-09-14T09:08:34.625492+0000","last_active":"2024-09-14T09:09:09.040029+0000","last_peered":"2024-09-14T09:09:09.040029+0000","last_clean":"2024-09-14T09:09:09.040029+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:09:09.040029+0000","last_undegraded":"2024-09-14T09:09:09.040029+0000","last_fullsized":"2024-09-14T09:09:09.040029+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:01.005568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010401127,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5032,"num_read_kb":31996,"num_write":418,"num_write_kb":13378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"187'676","reported_seq":6616,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.871638+0000","last_change":"2024-09-14T09:08:43.847915+0000","last_active":"2024-09-14T09:09:07.871638+0000","last_peered":"2024-09-14T09:09:07.871638+0000","last_clean":"2024-09-14T09:09:07.871638+0000","last_became_active":"2024-09-14T09:08:43.573212+0000","last_became_peered":"2024-09-14T09:08:43.573212+0000","last_unstale":"2024-09-14T09:09:07.871638+0000","last_undegraded":"2024-09-14T09:09:07.871638+0000","last_fullsized":"2024-09-14T09:09:07.871638+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:26:03.155997+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.147750145,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6669,"num_read_kb":58049,"num_write":727,"num_write_kb":12436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"190'866","reported_seq":5645,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.870986+0000","last_change":"2024-09-14T09:08:46.770708+0000","last_active":"2024-09-14T09:09:07.870986+0000","last_peered":"2024-09-14T09:09:07.870986+0000","last_clean":"2024-09-14T09:09:07.870986+0000","last_became_active":"2024-09-14T09:08:46.599475+0000","last_became_peered":"2024-09-14T09:08:46.599475+0000","last_unstale":"2024-09-14T09:09:07.870986+0000","last_undegraded":"2024-09-14T09:09:07.870986+0000","last_fullsized":"2024-09-14T09:09:07.870986+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:59:57.157599+0000","scrub_duration":0,"objects_trimmed":60,"snaptrim_duration":0.168855903,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7679,"num_read_kb":27474,"num_write":1043,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"179'742","reported_seq":5718,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.476034+0000","last_change":"2024-09-14T09:08:34.491862+0000","last_active":"2024-09-14T09:09:09.476034+0000","last_peered":"2024-09-14T09:09:09.476034+0000","last_clean":"2024-09-14T09:09:09.476034+0000","last_became_active":"2024-09-14T09:07:06.842337+0000","last_became_peered":"2024-09-14T09:07:06.842337+0000","last_unstale":"2024-09-14T09:09:09.476034+0000","last_undegraded":"2024-09-14T09:09:09.476034+0000","last_fullsized":"2024-09-14T09:09:09.476034+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:23:25.823078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023564419999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6703,"num_read_kb":24544,"num_write":874,"num_write_kb":16463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":14145319,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"187'724","reported_seq":5512,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.867745+0000","last_change":"2024-09-14T09:08:43.771622+0000","last_active":"2024-09-14T09:09:07.867745+0000","last_peered":"2024-09-14T09:09:07.867745+0000","last_clean":"2024-09-14T09:09:07.867745+0000","last_became_active":"2024-09-14T09:08:43.556338+0000","last_became_peered":"2024-09-14T09:08:43.556338+0000","last_unstale":"2024-09-14T09:09:07.867745+0000","last_undegraded":"2024-09-14T09:09:07.867745+0000","last_fullsized":"2024-09-14T09:09:07.867745+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:41.389225+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.12752904200000001,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6399,"num_read_kb":30687,"num_write":963,"num_write_kb":14233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"174'375","reported_seq":4400,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.862853+0000","last_change":"2024-09-14T09:08:50.629874+0000","last_active":"2024-09-14T09:09:07.862853+0000","last_peered":"2024-09-14T09:09:07.862853+0000","last_clean":"2024-09-14T09:09:07.862853+0000","last_became_active":"2024-09-14T09:08:50.627814+0000","last_became_peered":"2024-09-14T09:08:50.627814+0000","last_unstale":"2024-09-14T09:09:07.862853+0000","last_undegraded":"2024-09-14T09:09:07.862853+0000","last_fullsized":"2024-09-14T09:09:07.862853+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:14:43.966792+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.025218744000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3327,"num_read_kb":10838,"num_write":357,"num_write_kb":6673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":3727781,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"182'624","reported_seq":6380,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039812+0000","last_change":"2024-09-14T09:08:34.622411+0000","last_active":"2024-09-14T09:09:09.039812+0000","last_peered":"2024-09-14T09:09:09.039812+0000","last_clean":"2024-09-14T09:09:09.039812+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:09:09.039812+0000","last_undegraded":"2024-09-14T09:09:09.039812+0000","last_fullsized":"2024-09-14T09:09:09.039812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:55:38.962684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080911500000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7486,"num_read_kb":17479,"num_write":535,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"182'910","reported_seq":7829,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.476168+0000","last_change":"2024-09-14T09:08:34.496061+0000","last_active":"2024-09-14T09:09:09.476168+0000","last_peered":"2024-09-14T09:09:09.476168+0000","last_clean":"2024-09-14T09:09:09.476168+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:09:09.476168+0000","last_undegraded":"2024-09-14T09:09:09.476168+0000","last_fullsized":"2024-09-14T09:09:09.476168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:02:38.220394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026116440000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10070,"num_read_kb":35479,"num_write":1129,"num_write_kb":10795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"179'629","reported_seq":3952,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.871137+0000","last_change":"2024-09-14T09:08:34.493205+0000","last_active":"2024-09-14T09:09:07.871137+0000","last_peered":"2024-09-14T09:09:07.871137+0000","last_clean":"2024-09-14T09:09:07.871137+0000","last_became_active":"2024-09-14T09:07:25.105674+0000","last_became_peered":"2024-09-14T09:07:25.105674+0000","last_unstale":"2024-09-14T09:09:07.871137+0000","last_undegraded":"2024-09-14T09:09:07.871137+0000","last_fullsized":"2024-09-14T09:09:07.871137+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:20:01.627411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001781447,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4726,"num_read_kb":15599,"num_write":700,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":8467448,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"181'1136","reported_seq":5393,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039887+0000","last_change":"2024-09-14T09:08:34.507525+0000","last_active":"2024-09-14T09:09:08.039887+0000","last_peered":"2024-09-14T09:09:08.039887+0000","last_clean":"2024-09-14T09:09:08.039887+0000","last_became_active":"2024-09-14T09:07:27.854998+0000","last_became_peered":"2024-09-14T09:07:27.854998+0000","last_unstale":"2024-09-14T09:09:08.039887+0000","last_undegraded":"2024-09-14T09:09:08.039887+0000","last_fullsized":"2024-09-14T09:09:08.039887+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:11:04.062027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001704445,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8595,"num_read_kb":26590,"num_write":1602,"num_write_kb":14717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":4170600,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"187'670","reported_seq":3459,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.868133+0000","last_change":"2024-09-14T09:08:43.840463+0000","last_active":"2024-09-14T09:09:07.868133+0000","last_peered":"2024-09-14T09:09:07.868133+0000","last_clean":"2024-09-14T09:09:07.868133+0000","last_became_active":"2024-09-14T09:08:43.569123+0000","last_became_peered":"2024-09-14T09:08:43.569123+0000","last_unstale":"2024-09-14T09:09:07.868133+0000","last_undegraded":"2024-09-14T09:09:07.868133+0000","last_fullsized":"2024-09-14T09:09:07.868133+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:34:18.861184+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.15505904000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3583,"num_read_kb":9199,"num_write":691,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":2686146,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":407,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.480738+0000","last_change":"2024-09-14T09:07:21.857426+0000","last_active":"2024-09-14T09:09:09.480738+0000","last_peered":"2024-09-14T09:09:09.480738+0000","last_clean":"2024-09-14T09:09:09.480738+0000","last_became_active":"2024-09-14T09:07:21.856579+0000","last_became_peered":"2024-09-14T09:07:21.856579+0000","last_unstale":"2024-09-14T09:09:09.480738+0000","last_undegraded":"2024-09-14T09:09:09.480738+0000","last_fullsized":"2024-09-14T09:09:09.480738+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:40:56.694930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"182'791","reported_seq":6697,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.476046+0000","last_change":"2024-09-14T09:08:34.494518+0000","last_active":"2024-09-14T09:09:09.476046+0000","last_peered":"2024-09-14T09:09:09.476046+0000","last_clean":"2024-09-14T09:09:09.476046+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:09:09.476046+0000","last_undegraded":"2024-09-14T09:09:09.476046+0000","last_fullsized":"2024-09-14T09:09:09.476046+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:58.731530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016873319999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8676,"num_read_kb":45389,"num_write":707,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":360,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.863161+0000","last_change":"2024-09-14T09:08:51.290129+0000","last_active":"2024-09-14T09:09:07.863161+0000","last_peered":"2024-09-14T09:09:07.863161+0000","last_clean":"2024-09-14T09:09:07.863161+0000","last_became_active":"2024-09-14T09:08:51.289551+0000","last_became_peered":"2024-09-14T09:08:51.289551+0000","last_unstale":"2024-09-14T09:09:07.863161+0000","last_undegraded":"2024-09-14T09:09:07.863161+0000","last_fullsized":"2024-09-14T09:09:07.863161+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:52:01.486956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"182'1187","reported_seq":7262,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.040014+0000","last_change":"2024-09-14T09:08:34.626223+0000","last_active":"2024-09-14T09:09:09.040014+0000","last_peered":"2024-09-14T09:09:09.040014+0000","last_clean":"2024-09-14T09:09:09.040014+0000","last_became_active":"2024-09-14T09:07:26.102666+0000","last_became_peered":"2024-09-14T09:07:26.102666+0000","last_unstale":"2024-09-14T09:09:09.040014+0000","last_undegraded":"2024-09-14T09:09:09.040014+0000","last_fullsized":"2024-09-14T09:09:09.040014+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:03:41.035208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011116867000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10102,"num_read_kb":27247,"num_write":1623,"num_write_kb":13456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":8334712,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":204,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039803+0000","last_change":"2024-09-14T09:07:21.863290+0000","last_active":"2024-09-14T09:09:08.039803+0000","last_peered":"2024-09-14T09:09:08.039803+0000","last_clean":"2024-09-14T09:09:08.039803+0000","last_became_active":"2024-09-14T09:07:21.862352+0000","last_became_peered":"2024-09-14T09:07:21.862352+0000","last_unstale":"2024-09-14T09:09:08.039803+0000","last_undegraded":"2024-09-14T09:09:08.039803+0000","last_fullsized":"2024-09-14T09:09:08.039803+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:48:30.332891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"179'922","reported_seq":5889,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.862993+0000","last_change":"2024-09-14T09:08:50.627160+0000","last_active":"2024-09-14T09:09:07.862993+0000","last_peered":"2024-09-14T09:09:07.862993+0000","last_clean":"2024-09-14T09:09:07.862993+0000","last_became_active":"2024-09-14T09:08:50.625357+0000","last_became_peered":"2024-09-14T09:08:50.625357+0000","last_unstale":"2024-09-14T09:09:07.862993+0000","last_undegraded":"2024-09-14T09:09:07.862993+0000","last_fullsized":"2024-09-14T09:09:07.862993+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:18:31.289028+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0021152969999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7783,"num_read_kb":22826,"num_write":1134,"num_write_kb":9259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":8151041,"num_keys_recovered":91,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":368,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:19.689398+0000","last_change":"2024-09-14T09:09:19.689398+0000","last_active":"2024-09-14T09:09:19.689398+0000","last_peered":"2024-09-14T09:09:19.689398+0000","last_clean":"2024-09-14T09:09:19.689398+0000","last_became_active":"2024-09-14T09:08:50.627971+0000","last_became_peered":"2024-09-14T09:08:50.627971+0000","last_unstale":"2024-09-14T09:09:19.689398+0000","last_undegraded":"2024-09-14T09:09:19.689398+0000","last_fullsized":"2024-09-14T09:09:19.689398+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:09:19.689329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:09:19.689329+0000","last_clean_scrub_stamp":"2024-09-14T09:09:19.689329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:59:32.239744+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"182'688","reported_seq":2924,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.871775+0000","last_change":"2024-09-14T09:08:34.495006+0000","last_active":"2024-09-14T09:09:07.871775+0000","last_peered":"2024-09-14T09:09:07.871775+0000","last_clean":"2024-09-14T09:09:07.871775+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:09:07.871775+0000","last_undegraded":"2024-09-14T09:09:07.871775+0000","last_fullsized":"2024-09-14T09:09:07.871775+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:25:08.203494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030472329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3594,"num_read_kb":16894,"num_write":791,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"190'884","reported_seq":6718,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.868090+0000","last_change":"2024-09-14T09:08:46.793435+0000","last_active":"2024-09-14T09:09:07.868090+0000","last_peered":"2024-09-14T09:09:07.868090+0000","last_clean":"2024-09-14T09:09:07.868090+0000","last_became_active":"2024-09-14T09:08:46.597293+0000","last_became_peered":"2024-09-14T09:08:46.597293+0000","last_unstale":"2024-09-14T09:09:07.868090+0000","last_undegraded":"2024-09-14T09:09:07.868090+0000","last_fullsized":"2024-09-14T09:09:07.868090+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:48:54.046569+0000","scrub_duration":0,"objects_trimmed":50,"snaptrim_duration":0.19293607300000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8248,"num_read_kb":26883,"num_write":890,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":400,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.870896+0000","last_change":"2024-09-14T09:05:19.312262+0000","last_active":"2024-09-14T09:09:07.870896+0000","last_peered":"2024-09-14T09:09:07.870896+0000","last_clean":"2024-09-14T09:09:07.870896+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:09:07.870896+0000","last_undegraded":"2024-09-14T09:09:07.870896+0000","last_fullsized":"2024-09-14T09:09:07.870896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:58.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"211'408","reported_seq":2556,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:20.673099+0000","last_change":"2024-09-14T09:09:20.673099+0000","last_active":"2024-09-14T09:09:20.673099+0000","last_peered":"2024-09-14T09:09:20.673099+0000","last_clean":"2024-09-14T09:09:20.673099+0000","last_became_active":"2024-09-14T09:08:50.622578+0000","last_became_peered":"2024-09-14T09:08:50.622578+0000","last_unstale":"2024-09-14T09:09:20.673099+0000","last_undegraded":"2024-09-14T09:09:20.673099+0000","last_fullsized":"2024-09-14T09:09:20.673099+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'408","last_scrub_stamp":"2024-09-14T09:09:20.673019+0000","last_deep_scrub":"211'408","last_deep_scrub_stamp":"2024-09-14T09:09:20.673019+0000","last_clean_scrub_stamp":"2024-09-14T09:09:20.673019+0000","objects_scrubbed":9,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:51:10.697063+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00245612,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1873,"num_read_kb":11982,"num_write":352,"num_write_kb":5539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":397,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.861950+0000","last_change":"2024-09-14T09:05:19.303250+0000","last_active":"2024-09-14T09:09:07.861950+0000","last_peered":"2024-09-14T09:09:07.861950+0000","last_clean":"2024-09-14T09:09:07.861950+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:09:07.861950+0000","last_undegraded":"2024-09-14T09:09:07.861950+0000","last_fullsized":"2024-09-14T09:09:07.861950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:24.151837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"187'460","reported_seq":5083,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.475799+0000","last_change":"2024-09-14T09:08:43.789977+0000","last_active":"2024-09-14T09:09:09.475799+0000","last_peered":"2024-09-14T09:09:09.475799+0000","last_clean":"2024-09-14T09:09:09.475799+0000","last_became_active":"2024-09-14T09:08:43.568923+0000","last_became_peered":"2024-09-14T09:08:43.568923+0000","last_unstale":"2024-09-14T09:09:09.475799+0000","last_undegraded":"2024-09-14T09:09:09.475799+0000","last_fullsized":"2024-09-14T09:09:09.475799+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:45:29.077592+0000","scrub_duration":0,"objects_trimmed":62,"snaptrim_duration":0.165000182,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3905,"num_read_kb":25149,"num_write":375,"num_write_kb":14483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.867673+0000","last_change":"2024-09-14T09:08:44.554373+0000","last_active":"2024-09-14T09:09:07.867673+0000","last_peered":"2024-09-14T09:09:07.867673+0000","last_clean":"2024-09-14T09:09:07.867673+0000","last_became_active":"2024-09-14T09:08:44.553643+0000","last_became_peered":"2024-09-14T09:08:44.553643+0000","last_unstale":"2024-09-14T09:09:07.867673+0000","last_undegraded":"2024-09-14T09:09:07.867673+0000","last_fullsized":"2024-09-14T09:09:07.867673+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:32:52.900856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"211'444","reported_seq":3003,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:20.746802+0000","last_change":"2024-09-14T09:09:20.746802+0000","last_active":"2024-09-14T09:09:20.746802+0000","last_peered":"2024-09-14T09:09:20.746802+0000","last_clean":"2024-09-14T09:09:20.746802+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:09:20.746802+0000","last_undegraded":"2024-09-14T09:09:20.746802+0000","last_fullsized":"2024-09-14T09:09:20.746802+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'444","last_scrub_stamp":"2024-09-14T09:09:20.746566+0000","last_deep_scrub":"211'444","last_deep_scrub_stamp":"2024-09-14T09:09:20.746566+0000","last_clean_scrub_stamp":"2024-09-14T09:09:20.746566+0000","objects_scrubbed":11,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:06.080305+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0065530470000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2291,"num_read_kb":11209,"num_write":432,"num_write_kb":6866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":128,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.870941+0000","last_change":"2024-09-14T09:08:06.210516+0000","last_active":"2024-09-14T09:09:07.870941+0000","last_peered":"2024-09-14T09:09:07.870941+0000","last_clean":"2024-09-14T09:09:07.870941+0000","last_became_active":"2024-09-14T09:08:06.205108+0000","last_became_peered":"2024-09-14T09:08:06.205108+0000","last_unstale":"2024-09-14T09:09:07.870941+0000","last_undegraded":"2024-09-14T09:09:07.870941+0000","last_fullsized":"2024-09-14T09:09:07.870941+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:16.247798+0000","last_clean_scrub_stamp":"2024-09-14T09:05:16.247798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:22:05.543865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":211,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039776+0000","last_change":"2024-09-14T09:07:21.876180+0000","last_active":"2024-09-14T09:09:08.039776+0000","last_peered":"2024-09-14T09:09:08.039776+0000","last_clean":"2024-09-14T09:09:08.039776+0000","last_became_active":"2024-09-14T09:07:21.860117+0000","last_became_peered":"2024-09-14T09:07:21.860117+0000","last_unstale":"2024-09-14T09:09:08.039776+0000","last_undegraded":"2024-09-14T09:09:08.039776+0000","last_fullsized":"2024-09-14T09:09:08.039776+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:12.213934+0000","last_clean_scrub_stamp":"2024-09-14T09:05:12.213934+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:58:36.142971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"178'682","reported_seq":7043,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039644+0000","last_change":"2024-09-14T09:08:34.623351+0000","last_active":"2024-09-14T09:09:09.039644+0000","last_peered":"2024-09-14T09:09:09.039644+0000","last_clean":"2024-09-14T09:09:09.039644+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:09:09.039644+0000","last_undegraded":"2024-09-14T09:09:09.039644+0000","last_fullsized":"2024-09-14T09:09:09.039644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:48.184932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089062680000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7688,"num_read_kb":41730,"num_write":748,"num_write_kb":19769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"181'469","reported_seq":3215,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.480788+0000","last_change":"2024-09-14T09:08:35.139769+0000","last_active":"2024-09-14T09:09:09.480788+0000","last_peered":"2024-09-14T09:09:09.480788+0000","last_clean":"2024-09-14T09:09:09.480788+0000","last_became_active":"2024-09-14T09:07:06.851020+0000","last_became_peered":"2024-09-14T09:07:06.851020+0000","last_unstale":"2024-09-14T09:09:09.480788+0000","last_undegraded":"2024-09-14T09:09:09.480788+0000","last_fullsized":"2024-09-14T09:09:09.480788+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:19:55.337692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154644,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2080,"num_read_kb":17783,"num_write":389,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":12315549,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"187'749","reported_seq":5088,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.480676+0000","last_change":"2024-09-14T09:08:43.990954+0000","last_active":"2024-09-14T09:09:09.480676+0000","last_peered":"2024-09-14T09:09:09.480676+0000","last_clean":"2024-09-14T09:09:09.480676+0000","last_became_active":"2024-09-14T09:08:43.580245+0000","last_became_peered":"2024-09-14T09:08:43.580245+0000","last_unstale":"2024-09-14T09:09:09.480676+0000","last_undegraded":"2024-09-14T09:09:09.480676+0000","last_fullsized":"2024-09-14T09:09:09.480676+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:04.798520+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.116960367,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5499,"num_read_kb":39102,"num_write":883,"num_write_kb":19962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"181'544","reported_seq":9831,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.039772+0000","last_change":"2024-09-14T09:08:34.624404+0000","last_active":"2024-09-14T09:09:09.039772+0000","last_peered":"2024-09-14T09:09:09.039772+0000","last_clean":"2024-09-14T09:09:09.039772+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:09:09.039772+0000","last_undegraded":"2024-09-14T09:09:09.039772+0000","last_fullsized":"2024-09-14T09:09:09.039772+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:54:24.811934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097700640000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10396,"num_read_kb":61123,"num_write":490,"num_write_kb":11301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"180'493","reported_seq":3396,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.871076+0000","last_change":"2024-09-14T09:08:34.496008+0000","last_active":"2024-09-14T09:09:07.871076+0000","last_peered":"2024-09-14T09:09:07.871076+0000","last_clean":"2024-09-14T09:09:07.871076+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:09:07.871076+0000","last_undegraded":"2024-09-14T09:09:07.871076+0000","last_fullsized":"2024-09-14T09:09:07.871076+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:11.774708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00388507,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3266,"num_read_kb":13215,"num_write":426,"num_write_kb":10574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"179'501","reported_seq":5776,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:07.870536+0000","last_change":"2024-09-14T09:09:07.870373+0000","last_active":"2024-09-14T09:09:07.870536+0000","last_peered":"2024-09-14T09:09:07.870536+0000","last_clean":"2024-09-14T09:09:07.870536+0000","last_became_active":"2024-09-14T09:09:07.867771+0000","last_became_peered":"2024-09-14T09:09:07.867771+0000","last_unstale":"2024-09-14T09:09:07.870536+0000","last_undegraded":"2024-09-14T09:09:07.870536+0000","last_fullsized":"2024-09-14T09:09:07.870536+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:55:24.914026+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022704650000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5399,"num_read_kb":32061,"num_write":547,"num_write_kb":21536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":25396220,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"187'488","reported_seq":4038,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039610+0000","last_change":"2024-09-14T09:08:44.221618+0000","last_active":"2024-09-14T09:09:08.039610+0000","last_peered":"2024-09-14T09:09:08.039610+0000","last_clean":"2024-09-14T09:09:08.039610+0000","last_became_active":"2024-09-14T09:08:43.588233+0000","last_became_peered":"2024-09-14T09:08:43.588233+0000","last_unstale":"2024-09-14T09:09:08.039610+0000","last_undegraded":"2024-09-14T09:09:08.039610+0000","last_fullsized":"2024-09-14T09:09:08.039610+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:10.863128+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.14577486200000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4282,"num_read_kb":17652,"num_write":409,"num_write_kb":7874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":14123394,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"187'586","reported_seq":4291,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039835+0000","last_change":"2024-09-14T09:08:44.145761+0000","last_active":"2024-09-14T09:09:08.039835+0000","last_peered":"2024-09-14T09:09:08.039835+0000","last_clean":"2024-09-14T09:09:08.039835+0000","last_became_active":"2024-09-14T09:08:43.574566+0000","last_became_peered":"2024-09-14T09:08:43.574566+0000","last_unstale":"2024-09-14T09:09:08.039835+0000","last_undegraded":"2024-09-14T09:09:08.039835+0000","last_fullsized":"2024-09-14T09:09:08.039835+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:30.502086+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.141382223,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3641,"num_read_kb":14849,"num_write":554,"num_write_kb":12244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":11864981,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"176'506","reported_seq":6246,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.480599+0000","last_change":"2024-09-14T09:08:35.140559+0000","last_active":"2024-09-14T09:09:09.480599+0000","last_peered":"2024-09-14T09:09:09.480599+0000","last_clean":"2024-09-14T09:09:09.480599+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:09:09.480599+0000","last_undegraded":"2024-09-14T09:09:09.480599+0000","last_fullsized":"2024-09-14T09:09:09.480599+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:31:57.906202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018540480000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5422,"num_read_kb":60065,"num_write":521,"num_write_kb":20523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"187'649","reported_seq":6148,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039335+0000","last_change":"2024-09-14T09:08:43.902203+0000","last_active":"2024-09-14T09:09:08.039335+0000","last_peered":"2024-09-14T09:09:08.039335+0000","last_clean":"2024-09-14T09:09:08.039335+0000","last_became_active":"2024-09-14T09:08:43.577551+0000","last_became_peered":"2024-09-14T09:08:43.577551+0000","last_unstale":"2024-09-14T09:09:08.039335+0000","last_undegraded":"2024-09-14T09:09:08.039335+0000","last_fullsized":"2024-09-14T09:09:08.039335+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:43:37.090068+0000","scrub_duration":0,"objects_trimmed":64,"snaptrim_duration":0.121356326,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6082,"num_read_kb":13595,"num_write":653,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":7405746,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"182'790","reported_seq":6130,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:09.475812+0000","last_change":"2024-09-14T09:08:34.493569+0000","last_active":"2024-09-14T09:09:09.475812+0000","last_peered":"2024-09-14T09:09:09.475812+0000","last_clean":"2024-09-14T09:09:09.475812+0000","last_became_active":"2024-09-14T09:07:26.101586+0000","last_became_peered":"2024-09-14T09:07:26.101586+0000","last_unstale":"2024-09-14T09:09:09.475812+0000","last_undegraded":"2024-09-14T09:09:09.475812+0000","last_fullsized":"2024-09-14T09:09:09.475812+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:11:20.433650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040594799999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7044,"num_read_kb":19519,"num_write":992,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":10260744,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"187'536","reported_seq":6576,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:08.039395+0000","last_change":"2024-09-14T09:08:44.016919+0000","last_active":"2024-09-14T09:09:08.039395+0000","last_peered":"2024-09-14T09:09:08.039395+0000","last_clean":"2024-09-14T09:09:08.039395+0000","last_became_active":"2024-09-14T09:08:43.579997+0000","last_became_peered":"2024-09-14T09:08:43.579997+0000","last_unstale":"2024-09-14T09:09:08.039395+0000","last_undegraded":"2024-09-14T09:09:08.039395+0000","last_fullsized":"2024-09-14T09:09:08.039395+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:05:26.325409+0000","last_clean_scrub_stamp":"2024-09-14T09:05:26.325409+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:24:16.807890+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.10056556899999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6839,"num_read_kb":29296,"num_write":449,"num_write_kb":10682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":24080444,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":281,"num_object_clones":0,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":277,"num_read":198212,"num_read_kb":860220,"num_write":25088,"num_write_kb":385926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4177,"num_bytes_recovered":201826576,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},"log_size":22358,"ondisk_log_size":22358,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20292,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509494,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29512,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342328,"statfs":{"total":96636764160,"available":96606543872,"internally_reserved":0,"allocated":2543616,"data_stored":1231448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509494,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78096,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293744,"statfs":{"total":96636764160,"available":96556793856,"internally_reserved":0,"allocated":2547712,"data_stored":1231467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509495,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47564,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324276,"statfs":{"total":96636764160,"available":96588058624,"internally_reserved":0,"allocated":2543616,"data_stored":1231448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509497,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64256,"kb_used_data":2540,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307584,"statfs":{"total":96636764160,"available":96570966016,"internally_reserved":0,"allocated":2600960,"data_stored":1821816,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542198,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51760,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320080,"statfs":{"total":96636764160,"available":96583761920,"internally_reserved":0,"allocated":2584576,"data_stored":1821816,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":186,"seq":798863917066,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35976,"kb_used_data":2488,"kb_used_omap":62,"kb_used_meta":33409,"kb_avail":94335864,"statfs":{"total":96636764160,"available":96599924736,"internally_reserved":0,"allocated":2547712,"data_stored":1231467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":63771,"internal_metadata":34211557},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542199,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60040,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311800,"statfs":{"total":96636764160,"available":96575283200,"internally_reserved":0,"allocated":2543616,"data_stored":1231448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542200,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55884,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315956,"statfs":{"total":96636764160,"available":96579538944,"internally_reserved":0,"allocated":2551808,"data_stored":1231486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:09:22.569 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.569 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.569 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.570 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.570 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.570 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.570 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.570 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.570 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.570 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.570 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.571 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.571 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-14T09:05:16.247798+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=16, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.571 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.571 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-14T09:05:16.247798+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=16, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.571 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.571 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-14T09:05:16.247798+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=16, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.571 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.571 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.571 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.572 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-14T09:05:16.247798+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=16, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.572 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-14T09:05:16.247798+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=16, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.572 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.572 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-14T09:05:16.247798+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=16, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.572 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-14T09:05:16.247798+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=16, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.572 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-14T09:05:12.213934+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=12, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.572 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.572 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.572 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.573 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.573 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.573 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.573 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.573 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.573 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.573 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.573 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.573 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-14T09:05:26.325409+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=9, tm_min=9, tm_sec=17, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T09:09:22.574 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-14T09:09:42.575 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T09:09:42.888 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-09-14T09:09:42.889 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-09-14T09:09:42.906 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-09-14T09:09:42.037559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":285,"num_object_clones":0,"num_object_copies":570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":285,"num_whiteouts":277,"num_read":198278,"num_read_kb":860275,"num_write":25191,"num_write_kb":387556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4182,"num_bytes_recovered":203007312,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22698,"ondisk_log_size":22698,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":410576,"kb_used_data":19984,"kb_used_omap":73,"kb_used_meta":390390,"kb_avail":754564144,"statfs":{"total":773094113280,"available":772673683456,"internally_reserved":0,"allocated":20463616,"data_stored":11032396,"data_compressed":13252,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":74928,"internal_metadata":399760208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004344"},"pg_stats":[{"pgid":"3.1f","version":"211'1767","reported_seq":7872,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:23.104513+0000","last_change":"2024-09-14T09:09:23.104513+0000","last_active":"2024-09-14T09:09:23.104513+0000","last_peered":"2024-09-14T09:09:23.104513+0000","last_clean":"2024-09-14T09:09:23.104513+0000","last_became_active":"2024-09-14T09:06:41.345900+0000","last_became_peered":"2024-09-14T09:06:41.345900+0000","last_unstale":"2024-09-14T09:09:23.104513+0000","last_undegraded":"2024-09-14T09:09:23.104513+0000","last_fullsized":"2024-09-14T09:09:23.104513+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'1767","last_scrub_stamp":"2024-09-14T09:09:23.104315+0000","last_deep_scrub":"211'1767","last_deep_scrub_stamp":"2024-09-14T09:09:23.104315+0000","last_clean_scrub_stamp":"2024-09-14T09:09:23.104315+0000","objects_scrubbed":4,"log_size":1767,"log_dups_size":0,"ondisk_log_size":1767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:21:36.245298+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0022047759999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":13833,"num_read_kb":24717,"num_write":2684,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":2210112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"211'474","reported_seq":6006,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:25.874223+0000","last_change":"2024-09-14T09:09:25.874076+0000","last_active":"2024-09-14T09:09:25.874223+0000","last_peered":"2024-09-14T09:09:25.874223+0000","last_clean":"2024-09-14T09:09:25.874223+0000","last_became_active":"2024-09-14T09:05:26.824828+0000","last_became_peered":"2024-09-14T09:05:26.824828+0000","last_unstale":"2024-09-14T09:09:25.874223+0000","last_undegraded":"2024-09-14T09:09:25.874223+0000","last_fullsized":"2024-09-14T09:09:25.874223+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'474","last_scrub_stamp":"2024-09-14T09:09:25.873894+0000","last_deep_scrub":"211'474","last_deep_scrub_stamp":"2024-09-14T09:09:25.873894+0000","last_clean_scrub_stamp":"2024-09-14T09:09:25.873894+0000","objects_scrubbed":8,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:23.538210+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.010401127,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5032,"num_read_kb":31996,"num_write":418,"num_write_kb":13378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"211'683","reported_seq":6631,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:27.447648+0000","last_change":"2024-09-14T09:09:27.447648+0000","last_active":"2024-09-14T09:09:27.447648+0000","last_peered":"2024-09-14T09:09:27.447648+0000","last_clean":"2024-09-14T09:09:27.447648+0000","last_became_active":"2024-09-14T09:08:43.573212+0000","last_became_peered":"2024-09-14T09:08:43.573212+0000","last_unstale":"2024-09-14T09:09:27.447648+0000","last_undegraded":"2024-09-14T09:09:27.447648+0000","last_fullsized":"2024-09-14T09:09:27.447648+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'683","last_scrub_stamp":"2024-09-14T09:09:27.447440+0000","last_deep_scrub":"211'683","last_deep_scrub_stamp":"2024-09-14T09:09:27.447440+0000","last_clean_scrub_stamp":"2024-09-14T09:09:27.447440+0000","objects_scrubbed":7,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:15:25.845040+0000","scrub_duration":15,"objects_trimmed":62,"snaptrim_duration":0.147750145,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6669,"num_read_kb":58049,"num_write":727,"num_write_kb":12436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"211'873","reported_seq":5660,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:26.467078+0000","last_change":"2024-09-14T09:09:26.467078+0000","last_active":"2024-09-14T09:09:26.467078+0000","last_peered":"2024-09-14T09:09:26.467078+0000","last_clean":"2024-09-14T09:09:26.467078+0000","last_became_active":"2024-09-14T09:08:46.599475+0000","last_became_peered":"2024-09-14T09:08:46.599475+0000","last_unstale":"2024-09-14T09:09:26.467078+0000","last_undegraded":"2024-09-14T09:09:26.467078+0000","last_fullsized":"2024-09-14T09:09:26.467078+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'873","last_scrub_stamp":"2024-09-14T09:09:26.466876+0000","last_deep_scrub":"211'873","last_deep_scrub_stamp":"2024-09-14T09:09:26.466876+0000","last_clean_scrub_stamp":"2024-09-14T09:09:26.466876+0000","objects_scrubbed":7,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:38.841331+0000","scrub_duration":19,"objects_trimmed":60,"snaptrim_duration":0.168855903,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7679,"num_read_kb":27474,"num_write":1043,"num_write_kb":9252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":4377092,"num_keys_recovered":91,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"211'748","reported_seq":5732,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:22.120621+0000","last_change":"2024-09-14T09:09:22.120621+0000","last_active":"2024-09-14T09:09:22.120621+0000","last_peered":"2024-09-14T09:09:22.120621+0000","last_clean":"2024-09-14T09:09:22.120621+0000","last_became_active":"2024-09-14T09:07:06.842337+0000","last_became_peered":"2024-09-14T09:07:06.842337+0000","last_unstale":"2024-09-14T09:09:22.120621+0000","last_undegraded":"2024-09-14T09:09:22.120621+0000","last_fullsized":"2024-09-14T09:09:22.120621+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'748","last_scrub_stamp":"2024-09-14T09:09:22.120366+0000","last_deep_scrub":"211'748","last_deep_scrub_stamp":"2024-09-14T09:09:22.120366+0000","last_clean_scrub_stamp":"2024-09-14T09:09:22.120366+0000","objects_scrubbed":6,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:58:19.351679+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0023564419999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6703,"num_read_kb":24544,"num_write":874,"num_write_kb":16463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":14145319,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"211'732","reported_seq":5528,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:24.829417+0000","last_change":"2024-09-14T09:09:24.829417+0000","last_active":"2024-09-14T09:09:24.829417+0000","last_peered":"2024-09-14T09:09:24.829417+0000","last_clean":"2024-09-14T09:09:24.829417+0000","last_became_active":"2024-09-14T09:08:43.556338+0000","last_became_peered":"2024-09-14T09:08:43.556338+0000","last_unstale":"2024-09-14T09:09:24.829417+0000","last_undegraded":"2024-09-14T09:09:24.829417+0000","last_fullsized":"2024-09-14T09:09:24.829417+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'732","last_scrub_stamp":"2024-09-14T09:09:24.829216+0000","last_deep_scrub":"211'732","last_deep_scrub_stamp":"2024-09-14T09:09:24.829216+0000","last_clean_scrub_stamp":"2024-09-14T09:09:24.829216+0000","objects_scrubbed":8,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:25:10.146499+0000","scrub_duration":19,"objects_trimmed":46,"snaptrim_duration":0.12752904200000001,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6399,"num_read_kb":30687,"num_write":963,"num_write_kb":14233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":5986120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"211'383","reported_seq":4416,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:23.659918+0000","last_change":"2024-09-14T09:09:23.659918+0000","last_active":"2024-09-14T09:09:23.659918+0000","last_peered":"2024-09-14T09:09:23.659918+0000","last_clean":"2024-09-14T09:09:23.659918+0000","last_became_active":"2024-09-14T09:08:50.627814+0000","last_became_peered":"2024-09-14T09:08:50.627814+0000","last_unstale":"2024-09-14T09:09:23.659918+0000","last_undegraded":"2024-09-14T09:09:23.659918+0000","last_fullsized":"2024-09-14T09:09:23.659918+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'383","last_scrub_stamp":"2024-09-14T09:09:23.659624+0000","last_deep_scrub":"211'383","last_deep_scrub_stamp":"2024-09-14T09:09:23.659624+0000","last_clean_scrub_stamp":"2024-09-14T09:09:23.659624+0000","objects_scrubbed":8,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:28:57.889429+0000","scrub_duration":15,"objects_trimmed":3,"snaptrim_duration":0.025218744000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3327,"num_read_kb":10838,"num_write":357,"num_write_kb":6673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":3727781,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"211'630","reported_seq":6394,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:24.838656+0000","last_change":"2024-09-14T09:09:24.838656+0000","last_active":"2024-09-14T09:09:24.838656+0000","last_peered":"2024-09-14T09:09:24.838656+0000","last_clean":"2024-09-14T09:09:24.838656+0000","last_became_active":"2024-09-14T09:06:41.341844+0000","last_became_peered":"2024-09-14T09:06:41.341844+0000","last_unstale":"2024-09-14T09:09:24.838656+0000","last_undegraded":"2024-09-14T09:09:24.838656+0000","last_fullsized":"2024-09-14T09:09:24.838656+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'630","last_scrub_stamp":"2024-09-14T09:09:24.838480+0000","last_deep_scrub":"211'630","last_deep_scrub_stamp":"2024-09-14T09:09:24.838480+0000","last_clean_scrub_stamp":"2024-09-14T09:09:24.838480+0000","objects_scrubbed":6,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:20:15.500216+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0080911500000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7486,"num_read_kb":17479,"num_write":535,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":3070689,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"211'913","reported_seq":7840,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:24.083289+0000","last_change":"2024-09-14T09:09:24.083289+0000","last_active":"2024-09-14T09:09:24.083289+0000","last_peered":"2024-09-14T09:09:24.083289+0000","last_clean":"2024-09-14T09:09:24.083289+0000","last_became_active":"2024-09-14T09:05:26.826856+0000","last_became_peered":"2024-09-14T09:05:26.826856+0000","last_unstale":"2024-09-14T09:09:24.083289+0000","last_undegraded":"2024-09-14T09:09:24.083289+0000","last_fullsized":"2024-09-14T09:09:24.083289+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'913","last_scrub_stamp":"2024-09-14T09:09:24.083095+0000","last_deep_scrub":"211'913","last_deep_scrub_stamp":"2024-09-14T09:09:24.083095+0000","last_clean_scrub_stamp":"2024-09-14T09:09:24.083095+0000","objects_scrubbed":3,"log_size":913,"log_dups_size":0,"ondisk_log_size":913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:51:01.747653+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0026116440000000002,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10070,"num_read_kb":35479,"num_write":1129,"num_write_kb":10795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"211'634","reported_seq":3965,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:25.428896+0000","last_change":"2024-09-14T09:09:25.428896+0000","last_active":"2024-09-14T09:09:25.428896+0000","last_peered":"2024-09-14T09:09:25.428896+0000","last_clean":"2024-09-14T09:09:25.428896+0000","last_became_active":"2024-09-14T09:07:25.105674+0000","last_became_peered":"2024-09-14T09:07:25.105674+0000","last_unstale":"2024-09-14T09:09:25.428896+0000","last_undegraded":"2024-09-14T09:09:25.428896+0000","last_fullsized":"2024-09-14T09:09:25.428896+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'634","last_scrub_stamp":"2024-09-14T09:09:25.428651+0000","last_deep_scrub":"211'634","last_deep_scrub_stamp":"2024-09-14T09:09:25.428651+0000","last_clean_scrub_stamp":"2024-09-14T09:09:25.428651+0000","objects_scrubbed":5,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:02:27.155866+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.001781447,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4726,"num_read_kb":15599,"num_write":700,"num_write_kb":7558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":8467448,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"211'1147","reported_seq":5412,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:28.104465+0000","last_change":"2024-09-14T09:09:28.104465+0000","last_active":"2024-09-14T09:09:28.104465+0000","last_peered":"2024-09-14T09:09:28.104465+0000","last_clean":"2024-09-14T09:09:28.104465+0000","last_became_active":"2024-09-14T09:07:27.854998+0000","last_became_peered":"2024-09-14T09:07:27.854998+0000","last_unstale":"2024-09-14T09:09:28.104465+0000","last_undegraded":"2024-09-14T09:09:28.104465+0000","last_fullsized":"2024-09-14T09:09:28.104465+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'1147","last_scrub_stamp":"2024-09-14T09:09:28.104261+0000","last_deep_scrub":"211'1147","last_deep_scrub_stamp":"2024-09-14T09:09:28.104261+0000","last_clean_scrub_stamp":"2024-09-14T09:09:28.104261+0000","objects_scrubbed":11,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:13:36.246238+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.001704445,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8595,"num_read_kb":26590,"num_write":1602,"num_write_kb":14717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":4170600,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"211'680","reported_seq":3477,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:23.791530+0000","last_change":"2024-09-14T09:09:23.791530+0000","last_active":"2024-09-14T09:09:23.791530+0000","last_peered":"2024-09-14T09:09:23.791530+0000","last_clean":"2024-09-14T09:09:23.791530+0000","last_became_active":"2024-09-14T09:08:43.569123+0000","last_became_peered":"2024-09-14T09:08:43.569123+0000","last_unstale":"2024-09-14T09:09:23.791530+0000","last_undegraded":"2024-09-14T09:09:23.791530+0000","last_fullsized":"2024-09-14T09:09:23.791530+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'680","last_scrub_stamp":"2024-09-14T09:09:23.791307+0000","last_deep_scrub":"211'680","last_deep_scrub_stamp":"2024-09-14T09:09:23.791307+0000","last_clean_scrub_stamp":"2024-09-14T09:09:23.791307+0000","objects_scrubbed":10,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:21:14.095607+0000","scrub_duration":19,"objects_trimmed":50,"snaptrim_duration":0.15505904000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3583,"num_read_kb":9199,"num_write":691,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":2686146,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":416,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:22.171670+0000","last_change":"2024-09-14T09:09:22.171670+0000","last_active":"2024-09-14T09:09:22.171670+0000","last_peered":"2024-09-14T09:09:22.171670+0000","last_clean":"2024-09-14T09:09:22.171670+0000","last_became_active":"2024-09-14T09:07:21.856579+0000","last_became_peered":"2024-09-14T09:07:21.856579+0000","last_unstale":"2024-09-14T09:09:22.171670+0000","last_undegraded":"2024-09-14T09:09:22.171670+0000","last_fullsized":"2024-09-14T09:09:22.171670+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:09:22.171495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:09:22.171495+0000","last_clean_scrub_stamp":"2024-09-14T09:09:22.171495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:12:32.919725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197219999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"211'805","reported_seq":6719,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:19.142158+0000","last_change":"2024-09-14T09:09:19.142158+0000","last_active":"2024-09-14T09:09:19.142158+0000","last_peered":"2024-09-14T09:09:19.142158+0000","last_clean":"2024-09-14T09:09:19.142158+0000","last_became_active":"2024-09-14T09:05:26.815378+0000","last_became_peered":"2024-09-14T09:05:26.815378+0000","last_unstale":"2024-09-14T09:09:19.142158+0000","last_undegraded":"2024-09-14T09:09:19.142158+0000","last_fullsized":"2024-09-14T09:09:19.142158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'805","last_scrub_stamp":"2024-09-14T09:09:19.142059+0000","last_deep_scrub":"211'805","last_deep_scrub_stamp":"2024-09-14T09:09:19.142059+0000","last_clean_scrub_stamp":"2024-09-14T09:09:19.142059+0000","objects_scrubbed":14,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:35:14.463267+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0016873319999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8676,"num_read_kb":45389,"num_write":707,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":369,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:22.649731+0000","last_change":"2024-09-14T09:09:22.649587+0000","last_active":"2024-09-14T09:09:22.649731+0000","last_peered":"2024-09-14T09:09:22.649731+0000","last_clean":"2024-09-14T09:09:22.649731+0000","last_became_active":"2024-09-14T09:08:51.289551+0000","last_became_peered":"2024-09-14T09:08:51.289551+0000","last_unstale":"2024-09-14T09:09:22.649731+0000","last_undegraded":"2024-09-14T09:09:22.649731+0000","last_fullsized":"2024-09-14T09:09:22.649731+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:09:22.649410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:09:22.649410+0000","last_clean_scrub_stamp":"2024-09-14T09:09:22.649410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:59:58.535098+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"211'1194","reported_seq":7277,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:23.816742+0000","last_change":"2024-09-14T09:09:23.816742+0000","last_active":"2024-09-14T09:09:23.816742+0000","last_peered":"2024-09-14T09:09:23.816742+0000","last_clean":"2024-09-14T09:09:23.816742+0000","last_became_active":"2024-09-14T09:07:26.102666+0000","last_became_peered":"2024-09-14T09:07:26.102666+0000","last_unstale":"2024-09-14T09:09:23.816742+0000","last_undegraded":"2024-09-14T09:09:23.816742+0000","last_fullsized":"2024-09-14T09:09:23.816742+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'1194","last_scrub_stamp":"2024-09-14T09:09:23.816533+0000","last_deep_scrub":"211'1194","last_deep_scrub_stamp":"2024-09-14T09:09:23.816533+0000","last_clean_scrub_stamp":"2024-09-14T09:09:23.816533+0000","objects_scrubbed":7,"log_size":1194,"log_dups_size":0,"ondisk_log_size":1194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:46:04.722696+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.011116867000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10102,"num_read_kb":27247,"num_write":1623,"num_write_kb":13456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":8334712,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":213,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:24.058795+0000","last_change":"2024-09-14T09:09:24.058637+0000","last_active":"2024-09-14T09:09:24.058795+0000","last_peered":"2024-09-14T09:09:24.058795+0000","last_clean":"2024-09-14T09:09:24.058795+0000","last_became_active":"2024-09-14T09:07:21.862352+0000","last_became_peered":"2024-09-14T09:07:21.862352+0000","last_unstale":"2024-09-14T09:09:24.058795+0000","last_undegraded":"2024-09-14T09:09:24.058795+0000","last_fullsized":"2024-09-14T09:09:24.058795+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:09:24.058460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:09:24.058460+0000","last_clean_scrub_stamp":"2024-09-14T09:09:24.058460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:48:34.190515+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"211'932","reported_seq":5908,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:21.640748+0000","last_change":"2024-09-14T09:09:21.640618+0000","last_active":"2024-09-14T09:09:21.640748+0000","last_peered":"2024-09-14T09:09:21.640748+0000","last_clean":"2024-09-14T09:09:21.640748+0000","last_became_active":"2024-09-14T09:08:50.625357+0000","last_became_peered":"2024-09-14T09:08:50.625357+0000","last_unstale":"2024-09-14T09:09:21.640748+0000","last_undegraded":"2024-09-14T09:09:21.640748+0000","last_fullsized":"2024-09-14T09:09:21.640748+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'932","last_scrub_stamp":"2024-09-14T09:09:21.640492+0000","last_deep_scrub":"211'932","last_deep_scrub_stamp":"2024-09-14T09:09:21.640492+0000","last_clean_scrub_stamp":"2024-09-14T09:09:21.640492+0000","objects_scrubbed":10,"log_size":932,"log_dups_size":0,"ondisk_log_size":932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:27:01.207284+0000","scrub_duration":19,"objects_trimmed":2,"snaptrim_duration":0.0021152969999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7783,"num_read_kb":22826,"num_write":1134,"num_write_kb":9259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":8151041,"num_keys_recovered":91,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":369,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:19.689445+0000","last_change":"2024-09-14T09:09:19.689398+0000","last_active":"2024-09-14T09:09:19.689445+0000","last_peered":"2024-09-14T09:09:19.689445+0000","last_clean":"2024-09-14T09:09:19.689445+0000","last_became_active":"2024-09-14T09:08:50.627971+0000","last_became_peered":"2024-09-14T09:08:50.627971+0000","last_unstale":"2024-09-14T09:09:19.689445+0000","last_undegraded":"2024-09-14T09:09:19.689445+0000","last_fullsized":"2024-09-14T09:09:19.689445+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:09:19.689329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:09:19.689329+0000","last_clean_scrub_stamp":"2024-09-14T09:09:19.689329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:59:32.239744+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"211'697","reported_seq":2942,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:23.435917+0000","last_change":"2024-09-14T09:09:23.435778+0000","last_active":"2024-09-14T09:09:23.435917+0000","last_peered":"2024-09-14T09:09:23.435917+0000","last_clean":"2024-09-14T09:09:23.435917+0000","last_became_active":"2024-09-14T09:05:26.827786+0000","last_became_peered":"2024-09-14T09:05:26.827786+0000","last_unstale":"2024-09-14T09:09:23.435917+0000","last_undegraded":"2024-09-14T09:09:23.435917+0000","last_fullsized":"2024-09-14T09:09:23.435917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'697","last_scrub_stamp":"2024-09-14T09:09:23.435579+0000","last_deep_scrub":"211'697","last_deep_scrub_stamp":"2024-09-14T09:09:23.435579+0000","last_clean_scrub_stamp":"2024-09-14T09:09:23.435579+0000","objects_scrubbed":9,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:37:42.518049+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0030472329999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3594,"num_read_kb":16894,"num_write":791,"num_write_kb":13710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"211'891","reported_seq":6733,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:22.772690+0000","last_change":"2024-09-14T09:09:22.772690+0000","last_active":"2024-09-14T09:09:22.772690+0000","last_peered":"2024-09-14T09:09:22.772690+0000","last_clean":"2024-09-14T09:09:22.772690+0000","last_became_active":"2024-09-14T09:08:46.597293+0000","last_became_peered":"2024-09-14T09:08:46.597293+0000","last_unstale":"2024-09-14T09:09:22.772690+0000","last_undegraded":"2024-09-14T09:09:22.772690+0000","last_fullsized":"2024-09-14T09:09:22.772690+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'891","last_scrub_stamp":"2024-09-14T09:09:22.772469+0000","last_deep_scrub":"211'891","last_deep_scrub_stamp":"2024-09-14T09:09:22.772469+0000","last_clean_scrub_stamp":"2024-09-14T09:09:22.772469+0000","objects_scrubbed":7,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:02.686428+0000","scrub_duration":19,"objects_trimmed":50,"snaptrim_duration":0.19293607300000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8248,"num_read_kb":26883,"num_write":890,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4978984,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"211'3","reported_seq":410,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:21.422834+0000","last_change":"2024-09-14T09:09:21.422706+0000","last_active":"2024-09-14T09:09:21.422834+0000","last_peered":"2024-09-14T09:09:21.422834+0000","last_clean":"2024-09-14T09:09:21.422834+0000","last_became_active":"2024-09-14T09:05:17.277941+0000","last_became_peered":"2024-09-14T09:05:17.277941+0000","last_unstale":"2024-09-14T09:09:21.422834+0000","last_undegraded":"2024-09-14T09:09:21.422834+0000","last_fullsized":"2024-09-14T09:09:21.422834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'3","last_scrub_stamp":"2024-09-14T09:09:21.422501+0000","last_deep_scrub":"211'3","last_deep_scrub_stamp":"2024-09-14T09:09:21.422501+0000","last_clean_scrub_stamp":"2024-09-14T09:09:21.422501+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:56:55.704249+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0022689540000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"211'408","reported_seq":2557,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:20.673158+0000","last_change":"2024-09-14T09:09:20.673099+0000","last_active":"2024-09-14T09:09:20.673158+0000","last_peered":"2024-09-14T09:09:20.673158+0000","last_clean":"2024-09-14T09:09:20.673158+0000","last_became_active":"2024-09-14T09:08:50.622578+0000","last_became_peered":"2024-09-14T09:08:50.622578+0000","last_unstale":"2024-09-14T09:09:20.673158+0000","last_undegraded":"2024-09-14T09:09:20.673158+0000","last_fullsized":"2024-09-14T09:09:20.673158+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'408","last_scrub_stamp":"2024-09-14T09:09:20.673019+0000","last_deep_scrub":"211'408","last_deep_scrub_stamp":"2024-09-14T09:09:20.673019+0000","last_clean_scrub_stamp":"2024-09-14T09:09:20.673019+0000","objects_scrubbed":9,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:51:10.697063+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00245612,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":1873,"num_read_kb":11982,"num_write":352,"num_write_kb":5539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":406,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:20.153085+0000","last_change":"2024-09-14T09:09:20.152979+0000","last_active":"2024-09-14T09:09:20.153085+0000","last_peered":"2024-09-14T09:09:20.153085+0000","last_clean":"2024-09-14T09:09:20.153085+0000","last_became_active":"2024-09-14T09:05:17.389110+0000","last_became_peered":"2024-09-14T09:05:17.389110+0000","last_unstale":"2024-09-14T09:09:20.153085+0000","last_undegraded":"2024-09-14T09:09:20.153085+0000","last_fullsized":"2024-09-14T09:09:20.153085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:09:20.152851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:09:20.152851+0000","last_clean_scrub_stamp":"2024-09-14T09:09:20.152851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:36:23.985176+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0012401980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"211'473","reported_seq":5105,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:20.191310+0000","last_change":"2024-09-14T09:09:20.191254+0000","last_active":"2024-09-14T09:09:20.191310+0000","last_peered":"2024-09-14T09:09:20.191310+0000","last_clean":"2024-09-14T09:09:20.191310+0000","last_became_active":"2024-09-14T09:08:43.568923+0000","last_became_peered":"2024-09-14T09:08:43.568923+0000","last_unstale":"2024-09-14T09:09:20.191310+0000","last_undegraded":"2024-09-14T09:09:20.191310+0000","last_fullsized":"2024-09-14T09:09:20.191310+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'473","last_scrub_stamp":"2024-09-14T09:09:20.191150+0000","last_deep_scrub":"211'473","last_deep_scrub_stamp":"2024-09-14T09:09:20.191150+0000","last_clean_scrub_stamp":"2024-09-14T09:09:20.191150+0000","objects_scrubbed":13,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:18:46.642135+0000","scrub_duration":15,"objects_trimmed":62,"snaptrim_duration":0.165000182,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3905,"num_read_kb":25149,"num_write":375,"num_write_kb":14483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":7437531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":64,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:21.802305+0000","last_change":"2024-09-14T09:09:21.802305+0000","last_active":"2024-09-14T09:09:21.802305+0000","last_peered":"2024-09-14T09:09:21.802305+0000","last_clean":"2024-09-14T09:09:21.802305+0000","last_became_active":"2024-09-14T09:08:44.553643+0000","last_became_peered":"2024-09-14T09:08:44.553643+0000","last_unstale":"2024-09-14T09:09:21.802305+0000","last_undegraded":"2024-09-14T09:09:21.802305+0000","last_fullsized":"2024-09-14T09:09:21.802305+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T09:09:21.802090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T09:09:21.802090+0000","last_clean_scrub_stamp":"2024-09-14T09:09:21.802090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:47:52.910310+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"211'444","reported_seq":3004,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:20.747043+0000","last_change":"2024-09-14T09:09:20.746802+0000","last_active":"2024-09-14T09:09:20.747043+0000","last_peered":"2024-09-14T09:09:20.747043+0000","last_clean":"2024-09-14T09:09:20.747043+0000","last_became_active":"2024-09-14T09:05:26.827613+0000","last_became_peered":"2024-09-14T09:05:26.827613+0000","last_unstale":"2024-09-14T09:09:20.747043+0000","last_undegraded":"2024-09-14T09:09:20.747043+0000","last_fullsized":"2024-09-14T09:09:20.747043+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'444","last_scrub_stamp":"2024-09-14T09:09:20.746566+0000","last_deep_scrub":"211'444","last_deep_scrub_stamp":"2024-09-14T09:09:20.746566+0000","last_clean_scrub_stamp":"2024-09-14T09:09:20.746566+0000","objects_scrubbed":11,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:06:06.080305+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0065530470000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2291,"num_read_kb":11209,"num_write":432,"num_write_kb":6866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"211'2","reported_seq":137,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:22.425815+0000","last_change":"2024-09-14T09:09:22.425815+0000","last_active":"2024-09-14T09:09:22.425815+0000","last_peered":"2024-09-14T09:09:22.425815+0000","last_clean":"2024-09-14T09:09:22.425815+0000","last_became_active":"2024-09-14T09:08:06.205108+0000","last_became_peered":"2024-09-14T09:08:06.205108+0000","last_unstale":"2024-09-14T09:09:22.425815+0000","last_undegraded":"2024-09-14T09:09:22.425815+0000","last_fullsized":"2024-09-14T09:09:22.425815+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'2","last_scrub_stamp":"2024-09-14T09:09:22.425636+0000","last_deep_scrub":"211'2","last_deep_scrub_stamp":"2024-09-14T09:09:22.425636+0000","last_clean_scrub_stamp":"2024-09-14T09:09:22.425636+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:53:25.757237+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"211'74","reported_seq":221,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:23.104501+0000","last_change":"2024-09-14T09:09:23.104501+0000","last_active":"2024-09-14T09:09:23.104501+0000","last_peered":"2024-09-14T09:09:23.104501+0000","last_clean":"2024-09-14T09:09:23.104501+0000","last_became_active":"2024-09-14T09:07:21.860117+0000","last_became_peered":"2024-09-14T09:07:21.860117+0000","last_unstale":"2024-09-14T09:09:23.104501+0000","last_undegraded":"2024-09-14T09:09:23.104501+0000","last_fullsized":"2024-09-14T09:09:23.104501+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'74","last_scrub_stamp":"2024-09-14T09:09:23.104228+0000","last_deep_scrub":"211'74","last_deep_scrub_stamp":"2024-09-14T09:09:23.104228+0000","last_clean_scrub_stamp":"2024-09-14T09:09:23.104228+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:12:25.600834+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"211'689","reported_seq":7058,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:21.763561+0000","last_change":"2024-09-14T09:09:21.763561+0000","last_active":"2024-09-14T09:09:21.763561+0000","last_peered":"2024-09-14T09:09:21.763561+0000","last_clean":"2024-09-14T09:09:21.763561+0000","last_became_active":"2024-09-14T09:05:26.817938+0000","last_became_peered":"2024-09-14T09:05:26.817938+0000","last_unstale":"2024-09-14T09:09:21.763561+0000","last_undegraded":"2024-09-14T09:09:21.763561+0000","last_fullsized":"2024-09-14T09:09:21.763561+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'689","last_scrub_stamp":"2024-09-14T09:09:21.763399+0000","last_deep_scrub":"211'689","last_deep_scrub_stamp":"2024-09-14T09:09:21.763399+0000","last_clean_scrub_stamp":"2024-09-14T09:09:21.763399+0000","objects_scrubbed":7,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:32:10.678841+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0089062680000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7688,"num_read_kb":41730,"num_write":748,"num_write_kb":19769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"211'478","reported_seq":3232,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:24.169987+0000","last_change":"2024-09-14T09:09:24.169987+0000","last_active":"2024-09-14T09:09:24.169987+0000","last_peered":"2024-09-14T09:09:24.169987+0000","last_clean":"2024-09-14T09:09:24.169987+0000","last_became_active":"2024-09-14T09:07:06.851020+0000","last_became_peered":"2024-09-14T09:07:06.851020+0000","last_unstale":"2024-09-14T09:09:24.169987+0000","last_undegraded":"2024-09-14T09:09:24.169987+0000","last_fullsized":"2024-09-14T09:09:24.169987+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'478","last_scrub_stamp":"2024-09-14T09:09:24.169788+0000","last_deep_scrub":"211'478","last_deep_scrub_stamp":"2024-09-14T09:09:24.169788+0000","last_clean_scrub_stamp":"2024-09-14T09:09:24.169788+0000","objects_scrubbed":9,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:39:00.659459+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.001154644,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2080,"num_read_kb":17783,"num_write":389,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":12315549,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.a","version":"211'759","reported_seq":5106,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:23.153853+0000","last_change":"2024-09-14T09:09:23.153853+0000","last_active":"2024-09-14T09:09:23.153853+0000","last_peered":"2024-09-14T09:09:23.153853+0000","last_clean":"2024-09-14T09:09:23.153853+0000","last_became_active":"2024-09-14T09:08:43.580245+0000","last_became_peered":"2024-09-14T09:08:43.580245+0000","last_unstale":"2024-09-14T09:09:23.153853+0000","last_undegraded":"2024-09-14T09:09:23.153853+0000","last_fullsized":"2024-09-14T09:09:23.153853+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'759","last_scrub_stamp":"2024-09-14T09:09:23.153629+0000","last_deep_scrub":"211'759","last_deep_scrub_stamp":"2024-09-14T09:09:23.153629+0000","last_clean_scrub_stamp":"2024-09-14T09:09:23.153629+0000","objects_scrubbed":10,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:12:18.425865+0000","scrub_duration":23,"objects_trimmed":52,"snaptrim_duration":0.116960367,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5499,"num_read_kb":39102,"num_write":883,"num_write_kb":19962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":13008987,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"211'554","reported_seq":9849,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:22.794711+0000","last_change":"2024-09-14T09:09:22.794711+0000","last_active":"2024-09-14T09:09:22.794711+0000","last_peered":"2024-09-14T09:09:22.794711+0000","last_clean":"2024-09-14T09:09:22.794711+0000","last_became_active":"2024-09-14T09:06:02.836188+0000","last_became_peered":"2024-09-14T09:06:02.836188+0000","last_unstale":"2024-09-14T09:09:22.794711+0000","last_undegraded":"2024-09-14T09:09:22.794711+0000","last_fullsized":"2024-09-14T09:09:22.794711+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'554","last_scrub_stamp":"2024-09-14T09:09:22.794477+0000","last_deep_scrub":"211'554","last_deep_scrub_stamp":"2024-09-14T09:09:22.794477+0000","last_clean_scrub_stamp":"2024-09-14T09:09:22.794477+0000","objects_scrubbed":10,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:20:50.603051+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0097700640000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10396,"num_read_kb":61123,"num_write":490,"num_write_kb":11301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":400379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"211'502","reported_seq":3413,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:24.406758+0000","last_change":"2024-09-14T09:09:24.406758+0000","last_active":"2024-09-14T09:09:24.406758+0000","last_peered":"2024-09-14T09:09:24.406758+0000","last_clean":"2024-09-14T09:09:24.406758+0000","last_became_active":"2024-09-14T09:05:26.828135+0000","last_became_peered":"2024-09-14T09:05:26.828135+0000","last_unstale":"2024-09-14T09:09:24.406758+0000","last_undegraded":"2024-09-14T09:09:24.406758+0000","last_fullsized":"2024-09-14T09:09:24.406758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'502","last_scrub_stamp":"2024-09-14T09:09:24.406563+0000","last_deep_scrub":"211'502","last_deep_scrub_stamp":"2024-09-14T09:09:24.406563+0000","last_clean_scrub_stamp":"2024-09-14T09:09:24.406563+0000","objects_scrubbed":9,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:51:32.901345+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.00388507,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3266,"num_read_kb":13215,"num_write":426,"num_write_kb":10574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"211'510","reported_seq":5793,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:21.193264+0000","last_change":"2024-09-14T09:09:21.193264+0000","last_active":"2024-09-14T09:09:21.193264+0000","last_peered":"2024-09-14T09:09:21.193264+0000","last_clean":"2024-09-14T09:09:21.193264+0000","last_became_active":"2024-09-14T09:09:07.867771+0000","last_became_peered":"2024-09-14T09:09:07.867771+0000","last_unstale":"2024-09-14T09:09:21.193264+0000","last_undegraded":"2024-09-14T09:09:21.193264+0000","last_fullsized":"2024-09-14T09:09:21.193264+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'510","last_scrub_stamp":"2024-09-14T09:09:21.193068+0000","last_deep_scrub":"211'510","last_deep_scrub_stamp":"2024-09-14T09:09:21.193068+0000","last_clean_scrub_stamp":"2024-09-14T09:09:21.193068+0000","objects_scrubbed":9,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:04:39.940619+0000","scrub_duration":15,"objects_trimmed":2,"snaptrim_duration":0.0022704650000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5399,"num_read_kb":32061,"num_write":547,"num_write_kb":21536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":25396220,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"211'500","reported_seq":4059,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:26.101925+0000","last_change":"2024-09-14T09:09:26.101786+0000","last_active":"2024-09-14T09:09:26.101925+0000","last_peered":"2024-09-14T09:09:26.101925+0000","last_clean":"2024-09-14T09:09:26.101925+0000","last_became_active":"2024-09-14T09:08:43.588233+0000","last_became_peered":"2024-09-14T09:08:43.588233+0000","last_unstale":"2024-09-14T09:09:26.101925+0000","last_undegraded":"2024-09-14T09:09:26.101925+0000","last_fullsized":"2024-09-14T09:09:26.101925+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'500","last_scrub_stamp":"2024-09-14T09:09:26.101575+0000","last_deep_scrub":"211'500","last_deep_scrub_stamp":"2024-09-14T09:09:26.101575+0000","last_clean_scrub_stamp":"2024-09-14T09:09:26.101575+0000","objects_scrubbed":12,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:52:32.208459+0000","scrub_duration":23,"objects_trimmed":38,"snaptrim_duration":0.14577486200000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4282,"num_read_kb":17652,"num_write":409,"num_write_kb":7874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":14123394,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"211'595","reported_seq":4309,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:25.078891+0000","last_change":"2024-09-14T09:09:25.078687+0000","last_active":"2024-09-14T09:09:25.078891+0000","last_peered":"2024-09-14T09:09:25.078891+0000","last_clean":"2024-09-14T09:09:25.078891+0000","last_became_active":"2024-09-14T09:08:43.574566+0000","last_became_peered":"2024-09-14T09:08:43.574566+0000","last_unstale":"2024-09-14T09:09:25.078891+0000","last_undegraded":"2024-09-14T09:09:25.078891+0000","last_fullsized":"2024-09-14T09:09:25.078891+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'595","last_scrub_stamp":"2024-09-14T09:09:25.078423+0000","last_deep_scrub":"211'595","last_deep_scrub_stamp":"2024-09-14T09:09:25.078423+0000","last_clean_scrub_stamp":"2024-09-14T09:09:25.078423+0000","objects_scrubbed":9,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:40:44.754844+0000","scrub_duration":19,"objects_trimmed":52,"snaptrim_duration":0.141382223,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3641,"num_read_kb":14849,"num_write":554,"num_write_kb":12244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":11864981,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"211'514","reported_seq":6262,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:25.184442+0000","last_change":"2024-09-14T09:09:25.184442+0000","last_active":"2024-09-14T09:09:25.184442+0000","last_peered":"2024-09-14T09:09:25.184442+0000","last_clean":"2024-09-14T09:09:25.184442+0000","last_became_active":"2024-09-14T09:06:00.059315+0000","last_became_peered":"2024-09-14T09:06:00.059315+0000","last_unstale":"2024-09-14T09:09:25.184442+0000","last_undegraded":"2024-09-14T09:09:25.184442+0000","last_fullsized":"2024-09-14T09:09:25.184442+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'514","last_scrub_stamp":"2024-09-14T09:09:25.184257+0000","last_deep_scrub":"211'514","last_deep_scrub_stamp":"2024-09-14T09:09:25.184257+0000","last_clean_scrub_stamp":"2024-09-14T09:09:25.184257+0000","objects_scrubbed":8,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:56:37.224795+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0018540480000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5422,"num_read_kb":60065,"num_write":521,"num_write_kb":20523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":5226557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"211'667","reported_seq":6175,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:27.131055+0000","last_change":"2024-09-14T09:09:27.130913+0000","last_active":"2024-09-14T09:09:27.131055+0000","last_peered":"2024-09-14T09:09:27.131055+0000","last_clean":"2024-09-14T09:09:27.131055+0000","last_became_active":"2024-09-14T09:08:43.577551+0000","last_became_peered":"2024-09-14T09:08:43.577551+0000","last_unstale":"2024-09-14T09:09:27.131055+0000","last_undegraded":"2024-09-14T09:09:27.131055+0000","last_fullsized":"2024-09-14T09:09:27.131055+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'667","last_scrub_stamp":"2024-09-14T09:09:27.130685+0000","last_deep_scrub":"211'667","last_deep_scrub_stamp":"2024-09-14T09:09:27.130685+0000","last_clean_scrub_stamp":"2024-09-14T09:09:27.130685+0000","objects_scrubbed":18,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T20:44:36.108194+0000","scrub_duration":35,"objects_trimmed":64,"snaptrim_duration":0.121356326,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6082,"num_read_kb":13595,"num_write":653,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":7405746,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"211'798","reported_seq":6146,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:21.142544+0000","last_change":"2024-09-14T09:09:21.142544+0000","last_active":"2024-09-14T09:09:21.142544+0000","last_peered":"2024-09-14T09:09:21.142544+0000","last_clean":"2024-09-14T09:09:21.142544+0000","last_became_active":"2024-09-14T09:07:26.101586+0000","last_became_peered":"2024-09-14T09:07:26.101586+0000","last_unstale":"2024-09-14T09:09:21.142544+0000","last_undegraded":"2024-09-14T09:09:21.142544+0000","last_fullsized":"2024-09-14T09:09:21.142544+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'798","last_scrub_stamp":"2024-09-14T09:09:21.142452+0000","last_deep_scrub":"211'798","last_deep_scrub_stamp":"2024-09-14T09:09:21.142452+0000","last_clean_scrub_stamp":"2024-09-14T09:09:21.142452+0000","objects_scrubbed":8,"log_size":798,"log_dups_size":0,"ondisk_log_size":798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:27:56.758488+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0040594799999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7044,"num_read_kb":19519,"num_write":992,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":10260744,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"211'545","reported_seq":6594,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-14T09:09:29.125385+0000","last_change":"2024-09-14T09:09:29.125237+0000","last_active":"2024-09-14T09:09:29.125385+0000","last_peered":"2024-09-14T09:09:29.125385+0000","last_clean":"2024-09-14T09:09:29.125385+0000","last_became_active":"2024-09-14T09:08:43.579997+0000","last_became_peered":"2024-09-14T09:08:43.579997+0000","last_unstale":"2024-09-14T09:09:29.125385+0000","last_undegraded":"2024-09-14T09:09:29.125385+0000","last_fullsized":"2024-09-14T09:09:29.125385+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'545","last_scrub_stamp":"2024-09-14T09:09:29.125034+0000","last_deep_scrub":"211'545","last_deep_scrub_stamp":"2024-09-14T09:09:29.125034+0000","last_clean_scrub_stamp":"2024-09-14T09:09:29.125034+0000","objects_scrubbed":9,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:35:59.071327+0000","scrub_duration":19,"objects_trimmed":52,"snaptrim_duration":0.10056556899999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6839,"num_read_kb":29296,"num_write":449,"num_write_kb":10682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":24080444,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":281,"num_object_clones":0,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":277,"num_read":198212,"num_read_kb":860220,"num_write":25088,"num_write_kb":385926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4177,"num_bytes_recovered":201826576,"num_keys_recovered":652,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},"log_size":22619,"ondisk_log_size":22619,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":20292,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509498,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29508,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342332,"statfs":{"total":96636764160,"available":96606547968,"internally_reserved":0,"allocated":2543616,"data_stored":1231448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509498,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78088,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293752,"statfs":{"total":96636764160,"available":96556802048,"internally_reserved":0,"allocated":2547712,"data_stored":1231467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509499,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47556,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324284,"statfs":{"total":96636764160,"available":96588066816,"internally_reserved":0,"allocated":2543616,"data_stored":1231448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509501,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47612,"kb_used_data":2540,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324228,"statfs":{"total":96636764160,"available":96588009472,"internally_reserved":0,"allocated":2600960,"data_stored":1821816,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542202,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55916,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315924,"statfs":{"total":96636764160,"available":96579506176,"internally_reserved":0,"allocated":2584576,"data_stored":1821816,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":186,"seq":798863917070,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35976,"kb_used_data":2488,"kb_used_omap":62,"kb_used_meta":33409,"kb_avail":94335864,"statfs":{"total":96636764160,"available":96599924736,"internally_reserved":0,"allocated":2547712,"data_stored":1231467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":63771,"internal_metadata":34211557},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60036,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311804,"statfs":{"total":96636764160,"available":96575287296,"internally_reserved":0,"allocated":2543616,"data_stored":1231448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542204,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55884,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315956,"statfs":{"total":96636764160,"available":96579538944,"internally_reserved":0,"allocated":2551808,"data_stored":1231486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6212,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":7040,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19453,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T09:09:42.909 DEBUG:teuthology.orchestra.run.smithi176:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-14T09:09:43.236 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-14T09:09:43.236 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-14T09:09:43.237 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-14T09:09:43.237 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T09:09:43.323 INFO:tasks.ceph.osd.0:Stopped 2024-09-14T09:09:43.323 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-14T09:09:43.323 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T09:09:43.416 INFO:tasks.ceph.osd.1:Stopped 2024-09-14T09:09:43.417 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-14T09:09:43.417 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T09:09:46.879 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-14T09:09:49.518 INFO:tasks.ceph.osd.2:Stopped 2024-09-14T09:09:49.519 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-14T09:09:49.519 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T09:09:49.603 INFO:tasks.ceph.osd.3:Stopped 2024-09-14T09:09:49.604 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-14T09:09:49.604 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T09:09:49.697 INFO:tasks.ceph.osd.4:Stopped 2024-09-14T09:09:49.698 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-14T09:09:49.698 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T09:09:49.792 INFO:tasks.ceph.osd.5:Stopped 2024-09-14T09:09:49.792 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-14T09:09:49.792 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T09:09:49.884 INFO:tasks.ceph.osd.6:Stopped 2024-09-14T09:09:49.884 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-14T09:09:49.884 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T09:09:52.384 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-14T09:09:52.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-14T09:09:55.985 INFO:tasks.ceph.osd.7:Stopped 2024-09-14T09:09:55.986 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-14T09:09:55.986 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-14T09:09:55.986 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T09:09:56.015 INFO:tasks.ceph.mgr.y:Stopped 2024-09-14T09:09:56.015 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-14T09:09:56.015 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T09:09:56.046 INFO:tasks.ceph.mgr.x:Stopped 2024-09-14T09:09:56.046 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-14T09:09:56.046 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-14T09:09:56.046 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T09:09:56.071 INFO:tasks.ceph.mon.a:Stopped 2024-09-14T09:09:56.072 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-14T09:09:56.072 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T09:09:56.127 INFO:tasks.ceph.mon.c:Stopped 2024-09-14T09:09:56.128 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-14T09:09:56.128 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T09:09:56.149 INFO:tasks.ceph.mon.b:Stopped 2024-09-14T09:09:56.149 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-14T09:09:56.149 DEBUG:teuthology.orchestra.run.smithi176:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-14T09:09:56.187 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T09:09:56.187 DEBUG:teuthology.orchestra.run.smithi176:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-14T09:09:56.535 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T09:09:56.536 DEBUG:teuthology.orchestra.run.smithi176:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-14T09:09:56.705 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T09:09:56.706 DEBUG:teuthology.orchestra.run.smithi176:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-14T09:09:56.905 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T09:09:56.906 DEBUG:teuthology.orchestra.run.smithi176:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-14T09:09:57.098 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi183.front.sepia.ceph.com 2024-09-14T09:09:57.099 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-14T09:09:57.310 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi183.front.sepia.ceph.com 2024-09-14T09:09:57.311 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-14T09:09:57.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-14T09:09:57.530 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi183.front.sepia.ceph.com 2024-09-14T09:09:57.530 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-14T09:09:57.708 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi183.front.sepia.ceph.com 2024-09-14T09:09:57.709 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-14T09:09:57.879 INFO:tasks.ceph:Archiving mon data... 2024-09-14T09:09:57.880 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900516/data/mon.a.tgz 2024-09-14T09:09:57.880 DEBUG:teuthology.orchestra.run.smithi176:> mktemp 2024-09-14T09:09:57.886 INFO:teuthology.orchestra.run.smithi176.stdout:/tmp/tmp.si5QVXPMhJ 2024-09-14T09:09:57.887 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.si5QVXPMhJ 2024-09-14T09:09:58.043 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 0666 /tmp/tmp.si5QVXPMhJ 2024-09-14T09:09:58.109 DEBUG:teuthology.orchestra.remote:smithi176:/tmp/tmp.si5QVXPMhJ is 469KB 2024-09-14T09:09:58.172 DEBUG:teuthology.orchestra.run.smithi176:> rm -fr /tmp/tmp.si5QVXPMhJ 2024-09-14T09:09:58.178 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900516/data/mon.c.tgz 2024-09-14T09:09:58.178 DEBUG:teuthology.orchestra.run.smithi176:> mktemp 2024-09-14T09:09:58.226 INFO:teuthology.orchestra.run.smithi176.stdout:/tmp/tmp.JycdrGNX8b 2024-09-14T09:09:58.227 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.JycdrGNX8b 2024-09-14T09:09:58.420 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 0666 /tmp/tmp.JycdrGNX8b 2024-09-14T09:09:58.481 DEBUG:teuthology.orchestra.remote:smithi176:/tmp/tmp.JycdrGNX8b is 637KB 2024-09-14T09:09:58.544 DEBUG:teuthology.orchestra.run.smithi176:> rm -fr /tmp/tmp.JycdrGNX8b 2024-09-14T09:09:58.549 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900516/data/mon.b.tgz 2024-09-14T09:09:58.550 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2024-09-14T09:09:58.555 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.aEGqk3GEBf 2024-09-14T09:09:58.556 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.aEGqk3GEBf 2024-09-14T09:09:58.757 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.aEGqk3GEBf 2024-09-14T09:09:58.820 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.aEGqk3GEBf is 637KB 2024-09-14T09:09:58.882 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.aEGqk3GEBf 2024-09-14T09:09:58.887 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-14T09:09:58.887 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-14T09:09:58.890 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-14T09:09:58.950 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-14T09:09:58.992 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-14T09:09:58.993 INFO:tasks.ceph:Archiving crash dumps... 2024-09-14T09:09:58.994 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900516/remote/smithi176/crash 2024-09-14T09:09:58.995 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-14T09:09:59.007 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900516/remote/smithi183/crash 2024-09-14T09:09:59.008 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-14T09:09:59.048 INFO:tasks.ceph:Compressing logs... 2024-09-14T09:09:59.048 DEBUG:teuthology.orchestra.run.smithi176:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-14T09:09:59.055 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-14T09:09:59.075 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39645.log 2024-09-14T09:09:59.075 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39768.log 2024-09-14T09:09:59.076 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37379.log 2024-09-14T09:09:59.076 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39645.log.gz 2024-09-14T09:09:59.076 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39768.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39768.log.gz 2024-09-14T09:09:59.076 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33631.log 2024-09-14T09:09:59.077 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39299.log 2024-09-14T09:09:59.079 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37379.log: /var/log/ceph/ceph-client.admin.33631.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33631.log.gz 2024-09-14T09:09:59.079 INFO:teuthology.orchestra.run.smithi176.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34810.log 2024-09-14T09:09:59.079 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37379.log.gz 2024-09-14T09:09:59.079 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38343.log 2024-09-14T09:09:59.079 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39299.log.gz 2024-09-14T09:09:59.079 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34810.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.36661.log -- replaced with /var/log/ceph/ceph-client.admin.34810.log.gz 2024-09-14T09:09:59.079 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-14T09:09:59.079 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.38343.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38343.log.gz 2024-09-14T09:09:59.080 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31329.log 2024-09-14T09:09:59.080 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37195.log 2024-09-14T09:09:59.080 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36661.log.gz 2024-09-14T09:09:59.080 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36826.log 2024-09-14T09:09:59.080 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31329.log.gz 2024-09-14T09:09:59.083 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41737.log 2024-09-14T09:09:59.083 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37195.log.gz 2024-09-14T09:09:59.083 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31056.log 2024-09-14T09:09:59.083 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36826.log.gz 2024-09-14T09:09:59.083 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-14T09:09:59.083 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41737.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41737.log.gz 2024-09-14T09:09:59.083 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42207.log 2024-09-14T09:09:59.083 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31056.log.gz 2024-09-14T09:09:59.084 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40208.log 2024-09-14T09:09:59.084 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.42207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35100.log 2024-09-14T09:09:59.084 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42207.log.gz 2024-09-14T09:09:59.084 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40208.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40208.log.gz 2024-09-14T09:09:59.084 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41239.log 2024-09-14T09:09:59.084 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39132.log 2024-09-14T09:09:59.084 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35100.log.gz 2024-09-14T09:09:59.084 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31767.log 2024-09-14T09:09:59.084 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41239.log.gz 2024-09-14T09:09:59.084 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39240.log 2024-09-14T09:09:59.084 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39132.log.gz 2024-09-14T09:09:59.090 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34182.log 2024-09-14T09:09:59.090 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31767.log.gz 2024-09-14T09:09:59.090 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36096.log 2024-09-14T09:09:59.090 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39240.log.gz 2024-09-14T09:09:59.090 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39953.log 2024-09-14T09:09:59.090 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34182.log.gz 2024-09-14T09:09:59.090 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32920.log 2024-09-14T09:09:59.090 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36096.log.gz 2024-09-14T09:09:59.090 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34495.log 2024-09-14T09:09:59.090 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39953.log.gz 2024-09-14T09:09:59.090 INFO:teuthology.orchestra.run.smithi176.stderr: 93.5%/var/log/ceph/ceph-client.admin.32920.log: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-14T09:09:59.090 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32920.log.gz 2024-09-14T09:09:59.091 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41036.log 2024-09-14T09:09:59.091 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30717.log 2024-09-14T09:09:59.091 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34495.log.gz 2024-09-14T09:09:59.091 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41036.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-14T09:09:59.091 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41036.log.gz 2024-09-14T09:09:59.091 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30500.log 2024-09-14T09:09:59.091 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30717.log.gz 2024-09-14T09:09:59.091 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34330.log 2024-09-14T09:09:59.091 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37725.log 2024-09-14T09:09:59.091 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30500.log.gz 2024-09-14T09:09:59.091 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34330.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34330.log.gz 2024-09-14T09:09:59.091 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42311.log 2024-09-14T09:09:59.091 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42233.log 2024-09-14T09:09:59.092 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37725.log.gz 2024-09-14T09:09:59.092 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.42311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31600.log 2024-09-14T09:09:59.092 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42311.log.gz 2024-09-14T09:09:59.092 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.42233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41633.log 2024-09-14T09:09:59.092 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42233.log.gz 2024-09-14T09:09:59.092 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35975.log 0.0% 2024-09-14T09:09:59.092 INFO:teuthology.orchestra.run.smithi176.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31600.log.gz 2024-09-14T09:09:59.092 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42051.log 2024-09-14T09:09:59.092 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41633.log.gz 2024-09-14T09:09:59.104 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38741.log 2024-09-14T09:09:59.105 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35975.log.gz 2024-09-14T09:09:59.105 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.42051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35152.log 2024-09-14T09:09:59.105 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42051.log.gz 2024-09-14T09:09:59.105 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.38741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37108.log 2024-09-14T09:09:59.105 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38741.log.gz 2024-09-14T09:09:59.105 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37073.log 2024-09-14T09:09:59.105 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35152.log.gz 2024-09-14T09:09:59.105 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33740.log 2024-09-14T09:09:59.105 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37108.log.gz 2024-09-14T09:09:59.105 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30491.log 2024-09-14T09:09:59.105 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37073.log.gz 2024-09-14T09:09:59.105 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37256.log 2024-09-14T09:09:59.105 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33740.log.gz 2024-09-14T09:09:59.106 INFO:teuthology.orchestra.run.smithi176.stderr:gzip/var/log/ceph/ceph-client.admin.30491.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33492.log 2024-09-14T09:09:59.106 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30491.log.gz 2024-09-14T09:09:59.106 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34911.log 2024-09-14T09:09:59.106 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37256.log.gz 2024-09-14T09:09:59.106 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37571.log 2024-09-14T09:09:59.106 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33492.log.gz 2024-09-14T09:09:59.106 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41026.log 2024-09-14T09:09:59.106 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34911.log.gz 2024-09-14T09:09:59.106 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36307.log 2024-09-14T09:09:59.106 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37571.log.gz 2024-09-14T09:09:59.106 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42259.log 2024-09-14T09:09:59.106 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41026.log.gz 2024-09-14T09:09:59.106 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35349.log 2024-09-14T09:09:59.107 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36307.log.gz 2024-09-14T09:09:59.107 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.42259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40302.log 2024-09-14T09:09:59.107 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42259.log.gz 2024-09-14T09:09:59.107 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39728.log 2024-09-14T09:09:59.107 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35349.log.gz 2024-09-14T09:09:59.107 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35842.log 2024-09-14T09:09:59.107 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40302.log.gz 2024-09-14T09:09:59.107 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40022.log 2024-09-14T09:09:59.107 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39728.log.gz 2024-09-14T09:09:59.107 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36571.log 2024-09-14T09:09:59.107 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35842.log.gz 2024-09-14T09:09:59.107 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35747.log 2024-09-14T09:09:59.107 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40022.log.gz 2024-09-14T09:09:59.107 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33856.log 2024-09-14T09:09:59.108 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36571.log.gz 2024-09-14T09:09:59.108 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41020.log 2024-09-14T09:09:59.108 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35747.log.gz 2024-09-14T09:09:59.108 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34685.log 2024-09-14T09:09:59.108 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33856.log.gz 2024-09-14T09:09:59.108 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35672.log 2024-09-14T09:09:59.108 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41020.log.gz 2024-09-14T09:09:59.108 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36822.log 2024-09-14T09:09:59.108 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34685.log.gz 2024-09-14T09:09:59.108 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38116.log 2024-09-14T09:09:59.108 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35672.log.gz 2024-09-14T09:09:59.108 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35219.log 2024-09-14T09:09:59.108 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36822.log.gz 2024-09-14T09:09:59.108 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.38116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30303.log 2024-09-14T09:09:59.109 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38116.log.gz 2024-09-14T09:09:59.109 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35623.log 2024-09-14T09:09:59.109 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35219.log.gz 2024-09-14T09:09:59.109 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35253.log 2024-09-14T09:09:59.109 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30303.log.gz 2024-09-14T09:09:59.109 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37934.log 2024-09-14T09:09:59.109 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35623.log.gz 2024-09-14T09:09:59.109 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38907.log 2024-09-14T09:09:59.109 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35253.log.gz 2024-09-14T09:09:59.109 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40276.log 2024-09-14T09:09:59.109 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37934.log.gz 2024-09-14T09:09:59.109 INFO:teuthology.orchestra.run.smithi176.stderr:gzip/var/log/ceph/ceph-client.admin.38907.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36161.log 2024-09-14T09:09:59.109 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38907.log.gz 2024-09-14T09:09:59.109 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36066.log 2024-09-14T09:09:59.110 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40276.log.gz 2024-09-14T09:09:59.110 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38304.log 2024-09-14T09:09:59.110 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36161.log.gz 2024-09-14T09:09:59.110 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38776.log 2024-09-14T09:09:59.110 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36066.log.gz 2024-09-14T09:09:59.110 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.38304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32217.log 2024-09-14T09:09:59.110 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38304.log.gz 2024-09-14T09:09:59.110 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.38776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40817.log 2024-09-14T09:09:59.110 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38776.log.gz 2024-09-14T09:09:59.110 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41947.log 2024-09-14T09:09:59.110 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32217.log.gz 2024-09-14T09:09:59.122 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34034.log 2024-09-14T09:09:59.122 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40817.log.gz 2024-09-14T09:09:59.122 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41947.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-14T09:09:59.122 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41947.log.gz 2024-09-14T09:09:59.122 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31359.log 2024-09-14T09:09:59.122 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34034.log.gz 2024-09-14T09:09:59.122 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40149.log 2024-09-14T09:09:59.122 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35156.log 2024-09-14T09:09:59.123 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31359.log.gz 2024-09-14T09:09:59.123 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38248.log 2024-09-14T09:09:59.123 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40149.log.gz 2024-09-14T09:09:59.123 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33033.log 2024-09-14T09:09:59.123 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35156.log.gz 2024-09-14T09:09:59.123 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.38248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33607.log 2024-09-14T09:09:59.123 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38248.log.gz 2024-09-14T09:09:59.123 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31547.log 2024-09-14T09:09:59.123 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33033.log.gz 2024-09-14T09:09:59.123 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40541.log 2024-09-14T09:09:59.123 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33607.log.gz 2024-09-14T09:09:59.123 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36905.log 2024-09-14T09:09:59.123 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31547.log.gz 2024-09-14T09:09:59.123 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32303.log 2024-09-14T09:09:59.124 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40541.log.gz 2024-09-14T09:09:59.124 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31717.log 2024-09-14T09:09:59.124 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36905.log.gz 2024-09-14T09:09:59.124 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39047.log 2024-09-14T09:09:59.124 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32303.log.gz 2024-09-14T09:09:59.124 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30296.log 2024-09-14T09:09:59.124 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31717.log.gz 2024-09-14T09:09:59.124 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39047.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-14T09:09:59.124 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39047.log.gz 2024-09-14T09:09:59.124 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32560.log 2024-09-14T09:09:59.124 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30296.log.gz 2024-09-14T09:09:59.124 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41555.log 2024-09-14T09:09:59.124 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32660.log 2024-09-14T09:09:59.124 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32560.log.gz 2024-09-14T09:09:59.124 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41013.log 2024-09-14T09:09:59.125 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41555.log.gz 2024-09-14T09:09:59.125 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32660.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32660.log.gz 2024-09-14T09:09:59.125 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31671.log 2024-09-14T09:09:59.125 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32619.log 2024-09-14T09:09:59.125 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41013.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41013.log.gz 2024-09-14T09:09:59.125 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30290.log 2024-09-14T09:09:59.125 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31671.log.gz 2024-09-14T09:09:59.125 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34730.log 2024-09-14T09:09:59.125 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32619.log.gz 2024-09-14T09:09:59.125 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26422.log 2024-09-14T09:09:59.125 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30290.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30290.log.gz 2024-09-14T09:09:59.125 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39160.log 2024-09-14T09:09:59.125 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34730.log.gz 2024-09-14T09:09:59.126 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.26422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34738.log 2024-09-14T09:09:59.126 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26422.log.gz 2024-09-14T09:09:59.126 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38090.log 2024-09-14T09:09:59.126 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39160.log.gz 2024-09-14T09:09:59.126 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34465.log 2024-09-14T09:09:59.126 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34738.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34738.log.gz 2024-09-14T09:09:59.126 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.38090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34974.log 2024-09-14T09:09:59.126 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38090.log.gz 2024-09-14T09:09:59.126 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38991.log 2024-09-14T09:09:59.126 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34465.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34465.log.gz 2024-09-14T09:09:59.126 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41029.log 2024-09-14T09:09:59.126 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34974.log.gz 2024-09-14T09:09:59.126 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.38991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37167.log 2024-09-14T09:09:59.126 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38991.log.gz 2024-09-14T09:09:59.127 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39686.log 2024-09-14T09:09:59.127 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41029.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41029.log.gz 2024-09-14T09:09:59.127 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33368.log 2024-09-14T09:09:59.127 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37167.log.gz 2024-09-14T09:09:59.127 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40593.log 2024-09-14T09:09:59.127 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39686.log.gz 2024-09-14T09:09:59.127 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33657.log 2024-09-14T09:09:59.127 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33368.log.gz 2024-09-14T09:09:59.127 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40593.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.40593.log.gz --verbose 2024-09-14T09:09:59.127 INFO:teuthology.orchestra.run.smithi176.stderr: -- /var/log/ceph/ceph-client.admin.26275.log 2024-09-14T09:09:59.127 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38040.log 2024-09-14T09:09:59.127 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33657.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33657.log.gz 2024-09-14T09:09:59.127 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.26275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39188.log 2024-09-14T09:09:59.128 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26275.log.gz 2024-09-14T09:09:59.128 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.38040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35477.log 2024-09-14T09:09:59.128 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38040.log.gz 2024-09-14T09:09:59.128 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37830.log 2024-09-14T09:09:59.128 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39188.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39188.log.gz 2024-09-14T09:09:59.140 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.31752.log 2024-09-14T09:09:59.140 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-14T09:09:59.140 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/objectstore_tool.31752.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-14T09:09:59.140 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.32144.log 2024-09-14T09:09:59.140 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26439.log 2024-09-14T09:09:59.140 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/objectstore_tool.32144.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-14T09:09:59.140 INFO:teuthology.orchestra.run.smithi183.stderr: 94.8% -- replaced with /var/log/ceph/objectstore_tool.31752.log.gz 2024-09-14T09:09:59.141 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.26439.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-14T09:09:59.141 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26439.log.gz 2024-09-14T09:09:59.141 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26343.log 2024-09-14T09:09:59.141 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-14T09:09:59.141 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.26343.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.26487.log 2024-09-14T09:09:59.141 INFO:teuthology.orchestra.run.smithi183.stderr: -- replaced with /var/log/ceph/ceph-client.admin.26343.log.gz 2024-09-14T09:09:59.141 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.32924.log 2024-09-14T09:09:59.141 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26367.log 2024-09-14T09:09:59.141 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.26487.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26487.log.gz 2024-09-14T09:09:59.141 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/objectstore_tool.32924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26319.log 2024-09-14T09:09:59.141 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.26367.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26367.log.gz 2024-09-14T09:09:59.141 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-14T09:09:59.141 INFO:teuthology.orchestra.run.smithi183.stderr: 85.8% -- replaced with /var/log/ceph/ceph.log.gz/var/log/ceph/ceph-client.admin.26319.log: 2024-09-14T09:09:59.142 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26319.log.gz 2024-09-14T09:09:59.142 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.31948.log 2024-09-14T09:09:59.142 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26415.log 2024-09-14T09:09:59.142 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/objectstore_tool.31948.log: 95.0% -- replaced with /var/log/ceph/objectstore_tool.32144.log.gz 2024-09-14T09:09:59.142 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26391.log 2024-09-14T09:09:59.142 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.26415.log: 89.3% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26415.log.gz 2024-09-14T09:09:59.142 INFO:teuthology.orchestra.run.smithi183.stderr: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-14T09:09:59.142 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-14T09:09:59.142 INFO:teuthology.orchestra.run.smithi183.stderr: 94.8% -- replaced with /var/log/ceph/objectstore_tool.31948.log.gz 2024-09-14T09:09:59.142 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.26391.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-14T09:09:59.142 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26391.log.gz 2024-09-14T09:09:59.142 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26463.log 2024-09-14T09:09:59.142 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.26463.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26463.log.gz 2024-09-14T09:09:59.142 INFO:teuthology.orchestra.run.smithi183.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.32924.log.gz 2024-09-14T09:09:59.174 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35318.log 2024-09-14T09:09:59.174 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35477.log.gz 2024-09-14T09:09:59.174 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38959.log 2024-09-14T09:09:59.174 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37830.log.gz 2024-09-14T09:09:59.174 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41763.log 2024-09-14T09:09:59.174 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35318.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35318.log.gz 2024-09-14T09:09:59.175 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.38959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37908.log 2024-09-14T09:09:59.175 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38959.log.gz 2024-09-14T09:09:59.175 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32000.log 2024-09-14T09:09:59.175 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41763.log.gz 2024-09-14T09:09:59.175 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39104.log 2024-09-14T09:09:59.175 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37908.log.gz 2024-09-14T09:09:59.175 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30696.log 2024-09-14T09:09:59.175 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32000.log.gz 2024-09-14T09:09:59.175 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42025.log 2024-09-14T09:09:59.175 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39104.log.gz 2024-09-14T09:09:59.175 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40825.log 2024-09-14T09:09:59.175 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30696.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30696.log.gz 2024-09-14T09:09:59.175 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.42025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35698.log 2024-09-14T09:09:59.175 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42025.log.gz 2024-09-14T09:09:59.176 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42287.log 2024-09-14T09:09:59.176 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40825.log.gz 2024-09-14T09:09:59.176 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30501.log 2024-09-14T09:09:59.176 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35698.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35698.log.gz 2024-09-14T09:09:59.176 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.42287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40749.log 2024-09-14T09:09:59.176 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42287.log.gz 2024-09-14T09:09:59.176 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31030.log 2024-09-14T09:09:59.176 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30501.log.gz 2024-09-14T09:09:59.176 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34371.log 2024-09-14T09:09:59.176 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40749.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40749.log.gz 2024-09-14T09:09:59.176 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31502.log 2024-09-14T09:09:59.176 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31030.log.gz 2024-09-14T09:09:59.176 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31004.log 2024-09-14T09:09:59.176 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34371.log.gz 2024-09-14T09:09:59.177 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30718.log 2024-09-14T09:09:59.177 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31502.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31502.log.gz 2024-09-14T09:09:59.177 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35003.log 2024-09-14T09:09:59.177 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31004.log.gz 2024-09-14T09:09:59.177 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31132.log 2024-09-14T09:09:59.177 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30718.log.gz 2024-09-14T09:09:59.177 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35003.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.35003.log.gz /var/log/ceph/ceph-client.admin.41217.log 2024-09-14T09:09:59.177 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-14T09:09:59.177 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31132.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31132.log.gz 2024-09-14T09:09:59.177 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26522.log 2024-09-14T09:09:59.177 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35125.log 2024-09-14T09:09:59.177 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41217.log.gz 2024-09-14T09:09:59.177 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.26522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30152.log 2024-09-14T09:09:59.177 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26522.log.gz 2024-09-14T09:09:59.178 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30250.log 2024-09-14T09:09:59.178 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35125.log.gz 2024-09-14T09:09:59.178 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41025.log 2024-09-14T09:09:59.178 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30152.log.gz 2024-09-14T09:09:59.178 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32743.log 2024-09-14T09:09:59.178 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30250.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30250.log.gz 2024-09-14T09:09:59.178 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41025.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-14T09:09:59.178 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41025.log.gz 2024-09-14T09:09:59.178 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32946.log 2024-09-14T09:09:59.178 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32743.log.gz 2024-09-14T09:09:59.178 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33794.log 2024-09-14T09:09:59.178 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.32946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36232.log 2024-09-14T09:09:59.179 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32946.log.gz 2024-09-14T09:09:59.179 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34907.log 2024-09-14T09:09:59.179 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33794.log.gz 2024-09-14T09:09:59.179 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38662.log 2024-09-14T09:09:59.179 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36232.log.gz 2024-09-14T09:09:59.179 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39236.log 2024-09-14T09:09:59.179 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34907.log.gz 2024-09-14T09:09:59.179 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.38662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40567.log 2024-09-14T09:09:59.179 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38662.log.gz 2024-09-14T09:09:59.179 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30926.log 2024-09-14T09:09:59.179 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39236.log.gz 2024-09-14T09:09:59.179 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36274.log 2024-09-14T09:09:59.179 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40567.log.gz 2024-09-14T09:09:59.179 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40723.log 2024-09-14T09:09:59.180 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30926.log.gz 2024-09-14T09:09:59.180 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36274.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36274.log.gz 2024-09-14T09:09:59.180 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40333.log 2024-09-14T09:09:59.180 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41973.log 2024-09-14T09:09:59.180 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40723.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40723.log.gz 2024-09-14T09:09:59.180 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41235.log 2024-09-14T09:09:59.180 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40333.log.gz 2024-09-14T09:09:59.180 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31461.log 2024-09-14T09:09:59.180 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41973.log.gz 2024-09-14T09:09:59.180 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30286.log 2024-09-14T09:09:59.180 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41235.log.gz 2024-09-14T09:09:59.180 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33372.log 2024-09-14T09:09:59.180 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31461.log.gz 2024-09-14T09:09:59.181 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38597.log 2024-09-14T09:09:59.181 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30286.log.gz 2024-09-14T09:09:59.181 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40827.log 2024-09-14T09:09:59.181 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33372.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33372.log.gz 2024-09-14T09:09:59.181 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.38597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33451.log 2024-09-14T09:09:59.181 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38597.log.gz 2024-09-14T09:09:59.181 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35585.log 2024-09-14T09:09:59.181 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40827.log.gz 2024-09-14T09:09:59.181 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42155.log 2024-09-14T09:09:59.181 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33451.log.gz 2024-09-14T09:09:59.181 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29830.log 2024-09-14T09:09:59.181 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35585.log.gz 2024-09-14T09:09:59.181 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.42155.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-14T09:09:59.181 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42155.log.gz 2024-09-14T09:09:59.182 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.29830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41843.log 2024-09-14T09:09:59.182 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29830.log.gz 2024-09-14T09:09:59.182 INFO:teuthology.orchestra.run.smithi176.stderr:gzip/var/log/ceph/ceph-osd.1.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31268.log 2024-09-14T09:09:59.182 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41221.log 2024-09-14T09:09:59.182 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41843.log.gz 2024-09-14T09:09:59.182 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31268.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39906.log 2024-09-14T09:09:59.182 INFO:teuthology.orchestra.run.smithi176.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31268.log.gz 2024-09-14T09:09:59.182 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40821.log 2024-09-14T09:09:59.182 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41221.log.gz 2024-09-14T09:09:59.182 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39906.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39906.log.gz 2024-09-14T09:09:59.182 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35888.log 2024-09-14T09:09:59.182 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37638.log 2024-09-14T09:09:59.182 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0%/var/log/ceph/ceph-client.admin.35888.log: -- replaced with /var/log/ceph/ceph-client.admin.40821.log.gz 2024-09-14T09:09:59.183 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40385.log 2024-09-14T09:09:59.183 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35888.log.gz 2024-09-14T09:09:59.183 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41010.log 2024-09-14T09:09:59.183 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37638.log: /var/log/ceph/ceph-client.admin.40385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30719.log 2024-09-14T09:09:59.183 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37638.log.gz 2024-09-14T09:09:59.183 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40385.log.gz 2024-09-14T09:09:59.183 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41010.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39881.log 2024-09-14T09:09:59.183 INFO:teuthology.orchestra.run.smithi176.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41010.log.gz 2024-09-14T09:09:59.183 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38185.log 2024-09-14T09:09:59.183 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30719.log.gz 2024-09-14T09:09:59.183 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31953.log 2024-09-14T09:09:59.183 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39881.log.gz 2024-09-14T09:09:59.183 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.38185.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.38185.log.gz /var/log/ceph/ceph-client.admin.32397.log 2024-09-14T09:09:59.183 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-14T09:09:59.183 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31953.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31953.log.gz 2024-09-14T09:09:59.184 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37975.log 2024-09-14T09:09:59.184 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31419.log 2024-09-14T09:09:59.184 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32397.log.gz 2024-09-14T09:09:59.184 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41607.log 2024-09-14T09:09:59.184 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37975.log.gz 2024-09-14T09:09:59.184 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41529.log 2024-09-14T09:09:59.184 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31419.log: /var/log/ceph/ceph-client.admin.41607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33741.log 2024-09-14T09:09:59.184 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31419.log.gz 2024-09-14T09:09:59.184 INFO:teuthology.orchestra.run.smithi176.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41607.log.gz 2024-09-14T09:09:59.184 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41529.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41529.log.gz -5 2024-09-14T09:09:59.184 INFO:teuthology.orchestra.run.smithi176.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40645.log 2024-09-14T09:09:59.184 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34490.log 2024-09-14T09:09:59.184 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33741.log.gz 2024-09-14T09:09:59.185 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40645.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34839.log 2024-09-14T09:09:59.185 INFO:teuthology.orchestra.run.smithi176.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40645.log.gz 2024-09-14T09:09:59.185 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41477.log 2024-09-14T09:09:59.185 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34490.log.gz 2024-09-14T09:09:59.185 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36976.log 2024-09-14T09:09:59.185 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34839.log.gz 2024-09-14T09:09:59.185 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33087.log 2024-09-14T09:09:59.185 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41477.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41477.log.gz 2024-09-14T09:09:59.185 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36976.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36976.log.gz 2024-09-14T09:09:59.185 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32876.log 2024-09-14T09:09:59.185 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33128.log 2024-09-14T09:09:59.185 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33087.log.gz 2024-09-14T09:09:59.185 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35512.log 2024-09-14T09:09:59.185 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32876.log.gz 2024-09-14T09:09:59.186 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41238.log 2024-09-14T09:09:59.186 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33128.log.gz 2024-09-14T09:09:59.186 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39438.log 2024-09-14T09:09:59.186 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35512.log: /var/log/ceph/ceph-client.admin.41238.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35512.log.gz 2024-09-14T09:09:59.186 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41238.log.gz 2024-09-14T09:09:59.186 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32452.log 2024-09-14T09:09:59.186 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31853.log 2024-09-14T09:09:59.186 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39438.log.gz 2024-09-14T09:09:59.186 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41790.log 2024-09-14T09:09:59.186 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32452.log.gz 2024-09-14T09:09:59.186 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38865.log 2024-09-14T09:09:59.186 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31853.log.gz 2024-09-14T09:09:59.186 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40826.log 2024-09-14T09:09:59.186 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41790.log: /var/log/ceph/ceph-client.admin.38865.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41790.log.gz 2024-09-14T09:09:59.186 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31828.log 2024-09-14T09:09:59.187 INFO:teuthology.orchestra.run.smithi176.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38865.log.gz 2024-09-14T09:09:59.187 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40489.log 2024-09-14T09:09:59.187 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40826.log.gz 2024-09-14T09:09:59.187 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31828.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31828.log.gz 2024-09-14T09:09:59.187 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30102.log 2024-09-14T09:09:59.187 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30720.log 2024-09-14T09:09:59.187 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40489.log.gz 2024-09-14T09:09:59.187 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39818.log 2024-09-14T09:09:59.187 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30102.log.gz 2024-09-14T09:09:59.187 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35356.log 2024-09-14T09:09:59.187 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30720.log.gz 2024-09-14T09:09:59.187 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37882.log 2024-09-14T09:09:59.187 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39818.log: /var/log/ceph/ceph-client.admin.35356.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39818.log.gz 2024-09-14T09:09:59.187 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35356.log.gz 2024-09-14T09:09:59.188 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34109.log 2024-09-14T09:09:59.188 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37882.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30952.log 2024-09-14T09:09:59.188 INFO:teuthology.orchestra.run.smithi176.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37882.log.gz 2024-09-14T09:09:59.188 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34109.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34109.log.gz 2024-09-14T09:09:59.188 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-14T09:09:59.188 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37536.log 2024-09-14T09:09:59.188 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30952.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30952.log.gz 2024-09-14T09:09:59.188 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37752.log 2024-09-14T09:09:59.188 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37536.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37536.log.gz 2024-09-14T09:09:59.188 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31130.log 2024-09-14T09:09:59.188 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42103.log 2024-09-14T09:09:59.188 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37752.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37752.log.gz 2024-09-14T09:09:59.188 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31130.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39212.log 2024-09-14T09:09:59.188 INFO:teuthology.orchestra.run.smithi176.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31130.log.gz 2024-09-14T09:09:59.189 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.42103.log: 92.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-14T09:09:59.189 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42103.log.gz 2024-09-14T09:09:59.189 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40619.log 2024-09-14T09:09:59.189 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41451.log 2024-09-14T09:09:59.189 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39212.log.gz 2024-09-14T09:09:59.189 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32336.log 2024-09-14T09:09:59.189 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40619.log.gz 2024-09-14T09:09:59.189 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30304.log 2024-09-14T09:09:59.189 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41451.log.gz 2024-09-14T09:09:59.189 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33906.log 2024-09-14T09:09:59.189 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32336.log.gz 2024-09-14T09:09:59.189 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30304.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30512.log 2024-09-14T09:09:59.189 INFO:teuthology.orchestra.run.smithi176.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30304.log.gz 2024-09-14T09:09:59.189 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41255.log 2024-09-14T09:09:59.190 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33906.log.gz 2024-09-14T09:09:59.190 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40463.log 2024-09-14T09:09:59.190 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30512.log.gz 2024-09-14T09:09:59.190 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41240.log 2024-09-14T09:09:59.190 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41255.log: /var/log/ceph/ceph-client.admin.40463.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41255.log.gz 2024-09-14T09:09:59.190 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40463.log.gz 2024-09-14T09:09:59.190 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40437.log 2024-09-14T09:09:59.190 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33182.log 2024-09-14T09:09:59.190 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41240.log.gz 2024-09-14T09:09:59.190 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34071.log 2024-09-14T09:09:59.190 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40437.log.gz 2024-09-14T09:09:59.190 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35066.log 2024-09-14T09:09:59.190 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33182.log.gz 2024-09-14T09:09:59.190 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34439.log 2024-09-14T09:09:59.191 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34071.log: /var/log/ceph/ceph-client.admin.35066.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34071.log.gz 2024-09-14T09:09:59.191 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35066.log.gz 2024-09-14T09:09:59.191 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41503.log 2024-09-14T09:09:59.191 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37143.log 2024-09-14T09:09:59.191 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34439.log.gz 2024-09-14T09:09:59.191 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36021.log 2024-09-14T09:09:59.191 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41503.log.gz 2024-09-14T09:09:59.191 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41685.log 2024-09-14T09:09:59.191 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37143.log.gz 2024-09-14T09:09:59.191 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37778.log 2024-09-14T09:09:59.191 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36021.log: /var/log/ceph/ceph-client.admin.41685.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36021.log.gz 2024-09-14T09:09:59.191 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41685.log.gz 2024-09-14T09:09:59.191 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39412.log 2024-09-14T09:09:59.191 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30715.log 2024-09-14T09:09:59.192 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37778.log.gz 2024-09-14T09:09:59.192 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33294.log 2024-09-14T09:09:59.192 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39412.log.gz 2024-09-14T09:09:59.192 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30127.log 2024-09-14T09:09:59.192 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30715.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30715.log.gz 2024-09-14T09:09:59.192 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33294.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33294.log.gz 2024-09-14T09:09:59.192 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39545.log 2024-09-14T09:09:59.212 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30127.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30127.log.gz -5 2024-09-14T09:09:59.212 INFO:teuthology.orchestra.run.smithi176.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36731.log 2024-09-14T09:09:59.212 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37380.log 2024-09-14T09:09:59.212 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39545.log.gz 2024-09-14T09:09:59.212 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32478.log 2024-09-14T09:09:59.212 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36731.log.gz 2024-09-14T09:09:59.213 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35438.log 2024-09-14T09:09:59.213 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37380.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37380.log.gz 2024-09-14T09:09:59.213 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32478.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32478.log.gz 2024-09-14T09:09:59.213 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32426.log 2024-09-14T09:09:59.213 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30495.log 2024-09-14T09:09:59.213 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35438.log.gz 2024-09-14T09:09:59.213 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32426.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-14T09:09:59.213 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32426.log.gz 2024-09-14T09:09:59.213 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38713.log 2024-09-14T09:09:59.213 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30495.log.gz 2024-09-14T09:09:59.213 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34845.log 2024-09-14T09:09:59.213 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.38713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.31119.log 2024-09-14T09:09:59.213 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38713.log.gz 2024-09-14T09:09:59.214 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34845.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34845.log.gz 2024-09-14T09:09:59.214 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39076.log 2024-09-14T09:09:59.214 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37666.log 2024-09-14T09:09:59.214 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.0.31119.log: /var/log/ceph/ceph-client.admin.39076.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39076.log.gz 2024-09-14T09:09:59.214 INFO:teuthology.orchestra.run.smithi176.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.0.31119.log.gz 2024-09-14T09:09:59.214 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40829.log 2024-09-14T09:09:59.214 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38802.log 2024-09-14T09:09:59.214 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37666.log: /var/log/ceph/ceph-client.admin.40829.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40829.log.gz -- replaced with /var/log/ceph/ceph-client.admin.37666.log.gz 2024-09-14T09:09:59.214 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-14T09:09:59.214 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37351.log 2024-09-14T09:09:59.214 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34606.log 2024-09-14T09:09:59.214 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.38802.log: /var/log/ceph/ceph-client.admin.37351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38802.log.gz 2024-09-14T09:09:59.214 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37351.log.gz 2024-09-14T09:09:59.214 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39598.log 2024-09-14T09:09:59.215 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33935.log 2024-09-14T09:09:59.215 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34606.log: /var/log/ceph/ceph-client.admin.39598.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34606.log.gz 2024-09-14T09:09:59.215 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39598.log.gz 2024-09-14T09:09:59.215 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37856.log 2024-09-14T09:09:59.215 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31299.log 2024-09-14T09:09:59.215 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33935.log: /var/log/ceph/ceph-client.admin.37856.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33935.log.gz 2024-09-14T09:09:59.215 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37856.log.gz 2024-09-14T09:09:59.215 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30714.log 2024-09-14T09:09:59.215 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33684.log 2024-09-14T09:09:59.215 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31299.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31299.log.gz 2024-09-14T09:09:59.215 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30714.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30714.log.gzgzip 2024-09-14T09:09:59.215 INFO:teuthology.orchestra.run.smithi176.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40671.log 2024-09-14T09:09:59.215 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33684.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33684.log.gz 2024-09-14T09:09:59.215 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32781.log 2024-09-14T09:09:59.215 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31894.log 2024-09-14T09:09:59.216 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40671.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40671.log.gz 2024-09-14T09:09:59.216 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32781.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32781.log.gz 2024-09-14T09:09:59.216 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31086.log 2024-09-14T09:09:59.216 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31894.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31894.log.gz 2024-09-14T09:09:59.216 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30298.log 2024-09-14T09:09:59.216 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36616.log 2024-09-14T09:09:59.216 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31086.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31086.log.gz 2024-09-14T09:09:59.216 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30298.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30298.log.gz 2024-09-14T09:09:59.216 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39513.log 2024-09-14T09:09:59.216 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36616.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36616.log.gz 2024-09-14T09:09:59.216 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32984.log 2024-09-14T09:09:59.216 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41999.log 2024-09-14T09:09:59.216 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39513.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39513.log.gz 2024-09-14T09:09:59.216 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32984.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32984.log.gz 2024-09-14T09:09:59.217 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-14T09:09:59.217 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41999.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41999.log.gz 2024-09-14T09:09:59.217 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32826.log 2024-09-14T09:09:59.217 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41921.log 2024-09-14T09:09:59.217 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.32826.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32826.log.gz 2024-09-14T09:09:59.217 INFO:teuthology.orchestra.run.smithi176.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-14T09:09:59.217 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37477.log 2024-09-14T09:09:59.217 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30692.log 2024-09-14T09:09:59.217 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41921.log.gz 2024-09-14T09:09:59.217 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37477.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37477.log.gz 2024-09-14T09:09:59.217 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35547.log 2024-09-14T09:09:59.217 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32036.log 2024-09-14T09:09:59.217 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30692.log: /var/log/ceph/ceph-client.admin.35547.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30692.log.gz 2024-09-14T09:09:59.217 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35547.log.gz 2024-09-14T09:09:59.218 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31646.log 2024-09-14T09:09:59.218 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32307.log 2024-09-14T09:09:59.218 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32036.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32036.log.gz 2024-09-14T09:09:59.218 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31646.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34412.log 2024-09-14T09:09:59.218 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-14T09:09:59.218 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32307.log.gz 2024-09-14T09:09:59.218 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36202.log 2024-09-14T09:09:59.218 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33830.log 2024-09-14T09:09:59.218 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34412.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34412.log.gz 2024-09-14T09:09:59.218 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36202.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36202.log.gz 2024-09-14T09:09:59.218 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30504.log 2024-09-14T09:09:59.218 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33830.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33830.log.gz 2024-09-14T09:09:59.218 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33550.log 2024-09-14T09:09:59.218 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36529.log 2024-09-14T09:09:59.219 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30504.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30504.log.gz 2024-09-14T09:09:59.219 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33550.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33550.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30300.log 2024-09-14T09:09:59.219 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-14T09:09:59.219 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36529.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36529.log.gz 2024-09-14T09:09:59.219 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33178.log 2024-09-14T09:09:59.219 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41895.log 2024-09-14T09:09:59.219 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30300.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30300.log.gz 2024-09-14T09:09:59.219 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33178.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41037.log 2024-09-14T09:09:59.219 INFO:teuthology.orchestra.run.smithi176.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33178.log.gz 2024-09-14T09:09:59.219 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41895.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41895.log.gz 2024-09-14T09:09:59.219 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37293.log 2024-09-14T09:09:59.219 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39988.log 2024-09-14T09:09:59.219 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41037.log: /var/log/ceph/ceph-client.admin.37293.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41037.log.gz 2024-09-14T09:09:59.220 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37293.log.gz 2024-09-14T09:09:59.220 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39353.log 2024-09-14T09:09:59.220 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32192.log 2024-09-14T09:09:59.220 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39988.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39988.log.gz 2024-09-14T09:09:59.220 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.39353.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39353.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37026.log 2024-09-14T09:09:59.220 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-14T09:09:59.220 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32192.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32192.log.gz 2024-09-14T09:09:59.220 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36468.log 2024-09-14T09:09:59.220 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31389.log 2024-09-14T09:09:59.220 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37026.log: /var/log/ceph/ceph-client.admin.36468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37026.log.gz 2024-09-14T09:09:59.221 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36468.log.gz 2024-09-14T09:09:59.221 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34231.log 2024-09-14T09:09:59.221 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30224.log 2024-09-14T09:09:59.221 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31389.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31389.log.gz 2024-09-14T09:09:59.221 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34231.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34231.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38484.log 2024-09-14T09:09:59.221 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-14T09:09:59.221 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30224.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30224.log.gz 2024-09-14T09:09:59.225 INFO:teuthology.orchestra.run.smithi183.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-14T09:09:59.239 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38573.log 2024-09-14T09:09:59.239 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40775.log 2024-09-14T09:09:59.239 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.38484.log: /var/log/ceph/ceph-client.admin.38573.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38484.log.gz 2024-09-14T09:09:59.239 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38573.log.gz 2024-09-14T09:09:59.239 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40820.log 2024-09-14T09:09:59.240 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33710.log 2024-09-14T09:09:59.240 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40775.log: /var/log/ceph/ceph-client.admin.40820.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40775.log.gz 2024-09-14T09:09:59.240 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40820.log.gz 2024-09-14T09:09:59.240 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34647.log 2024-09-14T09:09:59.240 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32486.log 2024-09-14T09:09:59.240 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33710.log: /var/log/ceph/ceph-client.admin.34647.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33710.log.gz 2024-09-14T09:09:59.240 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34647.log.gz 2024-09-14T09:09:59.240 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35801.log 2024-09-14T09:09:59.240 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41711.log 2024-09-14T09:09:59.240 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32486.log: /var/log/ceph/ceph-client.admin.35801.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32486.log.gz 2024-09-14T09:09:59.240 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35801.log.gz 2024-09-14T09:09:59.240 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32827.log 2024-09-14T09:09:59.240 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32690.log 2024-09-14T09:09:59.240 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41711.log: 29.2% -- replaced with /var/log/ceph/ceph-client.admin.41711.log.gz 2024-09-14T09:09:59.241 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32827.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32827.log.gz 2024-09-14T09:09:59.241 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41241.log 2024-09-14T09:09:59.241 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32690.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32690.log.gz 2024-09-14T09:09:59.241 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36727.log 2024-09-14T09:09:59.241 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40063.log 2024-09-14T09:09:59.241 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41241.log.gz 2024-09-14T09:09:59.241 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36727.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36727.log.gz 2024-09-14T09:09:59.241 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38456.log 2024-09-14T09:09:59.241 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40063.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40063.log.gz 2024-09-14T09:09:59.241 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32142.log 2024-09-14T09:09:59.241 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.38456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38409.log 2024-09-14T09:09:59.241 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38456.log.gz/var/log/ceph/ceph-client.admin.32142.log: 2024-09-14T09:09:59.241 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32142.log.gz 2024-09-14T09:09:59.241 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36935.log 2024-09-14T09:09:59.242 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34257.log 2024-09-14T09:09:59.242 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.38409.log: /var/log/ceph/ceph-client.admin.36935.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38409.log.gz 2024-09-14T09:09:59.242 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36935.log.gz 2024-09-14T09:09:59.242 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36389.log 2024-09-14T09:09:59.242 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30198.log 2024-09-14T09:09:59.242 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34257.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34257.log.gz 2024-09-14T09:09:59.242 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36389.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36389.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30302.log 2024-09-14T09:09:59.242 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-14T09:09:59.242 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30198.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30198.log.gz 2024-09-14T09:09:59.242 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33997.log 2024-09-14T09:09:59.242 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35884.log 2024-09-14T09:09:59.242 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30302.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30302.log.gz 2024-09-14T09:09:59.242 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33997.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33997.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30499.log 2024-09-14T09:09:59.243 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-14T09:09:59.243 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35884.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35884.log.gz 2024-09-14T09:09:59.243 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41581.log 2024-09-14T09:09:59.243 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40697.log 2024-09-14T09:09:59.243 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30499.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30499.log.gz 2024-09-14T09:09:59.243 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41581.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41581.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40411.log 2024-09-14T09:09:59.243 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-14T09:09:59.243 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40697.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40697.log.gz 2024-09-14T09:09:59.243 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41817.log 2024-09-14T09:09:59.243 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30978.log 2024-09-14T09:09:59.243 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40411.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40411.log.gz 2024-09-14T09:09:59.243 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41817.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40250.log 2024-09-14T09:09:59.243 INFO:teuthology.orchestra.run.smithi176.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41817.log.gz 2024-09-14T09:09:59.243 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30978.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30978.log.gz 2024-09-14T09:09:59.244 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42129.log 2024-09-14T09:09:59.244 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40515.log 2024-09-14T09:09:59.244 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40250.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40250.log.gz 2024-09-14T09:09:59.244 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.42129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42129.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42077.log 2024-09-14T09:09:59.244 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-14T09:09:59.244 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40515.log.gz 2024-09-14T09:09:59.244 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30172.log 2024-09-14T09:09:59.244 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37443.log 2024-09-14T09:09:59.244 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.42077.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42077.log.gz 2024-09-14T09:09:59.244 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30172.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30172.log.gzgzip 2024-09-14T09:09:59.244 INFO:teuthology.orchestra.run.smithi176.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36426.log 2024-09-14T09:09:59.244 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37443.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37443.log.gz 2024-09-14T09:09:59.244 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30511.log 2024-09-14T09:09:59.245 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41869.log 2024-09-14T09:09:59.245 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.36426.log: /var/log/ceph/ceph-client.admin.30511.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36426.log.gz 2024-09-14T09:09:59.245 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30511.log.gz 2024-09-14T09:09:59.245 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42181.log 2024-09-14T09:09:59.245 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31227.log 2024-09-14T09:09:59.245 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41869.log: /var/log/ceph/ceph-client.admin.42181.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41869.log.gz 2024-09-14T09:09:59.245 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42181.log.gz 2024-09-14T09:09:59.245 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37804.log 2024-09-14T09:09:59.245 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41425.log 2024-09-14T09:09:59.245 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31227.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31227.log.gz 2024-09-14T09:09:59.245 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.37804.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37804.log.gz 2024-09-14T09:09:59.245 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35287.log 2024-09-14T09:09:59.245 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.41425.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41425.log.gz 2024-09-14T09:09:59.246 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32088.log 2024-09-14T09:09:59.246 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31127.log 2024-09-14T09:09:59.246 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.35287.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35287.log.gz 2024-09-14T09:09:59.249 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.32088.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40359.log 2024-09-14T09:09:59.249 INFO:teuthology.orchestra.run.smithi176.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32088.log.gz 2024-09-14T09:09:59.249 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.31127.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31127.log.gz 2024-09-14T09:09:59.249 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21481.log 2024-09-14T09:09:59.249 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40111.log 2024-09-14T09:09:59.249 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40359.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40359.log.gz 2024-09-14T09:09:59.250 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph.tmp-client.admin.21481.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34144.log 2024-09-14T09:09:59.250 INFO:teuthology.orchestra.run.smithi176.stderr: -- replaced with /var/log/ceph/ceph.tmp-client.admin.21481.log.gz 2024-09-14T09:09:59.250 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.40111.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40111.log.gz 2024-09-14T09:09:59.250 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40824.log 2024-09-14T09:09:59.250 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33269.log 2024-09-14T09:09:59.250 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.34144.log: /var/log/ceph/ceph-client.admin.40824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34144.log.gz 2024-09-14T09:09:59.250 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40824.log.gz 2024-09-14T09:09:59.250 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41659.log 2024-09-14T09:09:59.250 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30900.log 2024-09-14T09:09:59.250 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.33269.log: /var/log/ceph/ceph-client.admin.41659.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33269.log.gz 2024-09-14T09:09:59.250 INFO:teuthology.orchestra.run.smithi176.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41659.log.gz 2024-09-14T09:09:59.250 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.admin.30900.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30900.log.gz 2024-09-14T09:09:59.285 INFO:teuthology.orchestra.run.smithi183.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-14T09:09:59.351 INFO:teuthology.orchestra.run.smithi176.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-14T09:09:59.576 INFO:teuthology.orchestra.run.smithi176.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-14T09:09:59.785 INFO:teuthology.orchestra.run.smithi176.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-14T09:10:00.579 INFO:teuthology.orchestra.run.smithi183.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-14T09:10:01.014 INFO:teuthology.orchestra.run.smithi183.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-14T09:10:01.358 INFO:teuthology.orchestra.run.smithi176.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-14T09:10:02.229 INFO:teuthology.orchestra.run.smithi183.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-14T09:10:02.480 INFO:teuthology.orchestra.run.smithi183.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-14T09:10:02.481 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-09-14T09:10:02.481 INFO:teuthology.orchestra.run.smithi183.stderr:real 0m3.385s 2024-09-14T09:10:02.481 INFO:teuthology.orchestra.run.smithi183.stderr:user 0m9.674s 2024-09-14T09:10:02.481 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m0.441s 2024-09-14T09:10:02.980 INFO:teuthology.orchestra.run.smithi176.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-14T09:10:03.149 INFO:teuthology.orchestra.run.smithi176.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-14T09:10:03.150 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-09-14T09:10:03.150 INFO:teuthology.orchestra.run.smithi176.stderr:real 0m4.093s 2024-09-14T09:10:03.150 INFO:teuthology.orchestra.run.smithi176.stderr:user 0m11.233s 2024-09-14T09:10:03.150 INFO:teuthology.orchestra.run.smithi176.stderr:sys 0m0.576s 2024-09-14T09:10:03.151 INFO:tasks.ceph:Archiving logs... 2024-09-14T09:10:03.151 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900516/remote/smithi176/log 2024-09-14T09:10:03.152 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-14T09:10:04.617 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900516/remote/smithi183/log 2024-09-14T09:10:04.618 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-14T09:10:05.654 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-14T09:10:05.664 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-14T09:10:05.664 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-14T09:10:05.667 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-14T09:10:05.736 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-14T09:10:05.747 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-14T09:10:05.747 DEBUG:teuthology.orchestra.run.smithi176:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T09:10:05.750 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T09:10:05.758 INFO:teuthology.orchestra.run.smithi176.stdout: remote refid st t when poll reach delay offset jitter 2024-09-14T09:10:05.758 INFO:teuthology.orchestra.run.smithi176.stdout:============================================================================== 2024-09-14T09:10:05.759 INFO:teuthology.orchestra.run.smithi176.stdout:+hv01.front.sepi 67.205.162.81 3 u 2 64 377 0.085 -1.904 0.400 2024-09-14T09:10:05.759 INFO:teuthology.orchestra.run.smithi176.stdout:*hv02.front.sepi 63.231.80.2 3 u 4 64 377 0.091 +0.235 0.418 2024-09-14T09:10:05.759 INFO:teuthology.orchestra.run.smithi176.stdout:+hv03.front.sepi 74.6.168.72 3 u 6 64 377 0.078 +6.119 0.414 2024-09-14T09:10:05.759 INFO:teuthology.orchestra.run.smithi176.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T09:10:05.760 INFO:teuthology.orchestra.run.smithi183.stdout: remote refid st t when poll reach delay offset jitter 2024-09-14T09:10:05.760 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================== 2024-09-14T09:10:05.760 INFO:teuthology.orchestra.run.smithi183.stdout:+hv01.front.sepi 67.205.162.81 3 u 4 64 377 0.107 -2.287 3.711 2024-09-14T09:10:05.761 INFO:teuthology.orchestra.run.smithi183.stdout:*hv02.front.sepi 63.231.80.2 3 u 2 64 377 0.082 -6.446 4.712 2024-09-14T09:10:05.761 INFO:teuthology.orchestra.run.smithi183.stdout:+hv03.front.sepi 74.6.168.72 3 u 3 64 377 0.081 +5.111 2.821 2024-09-14T09:10:05.761 INFO:teuthology.orchestra.run.smithi183.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T09:10:05.761 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-14T09:10:05.780 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-14T09:10:05.780 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-14T09:10:05.814 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-14T09:10:05.846 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-14T09:10:05.880 INFO:teuthology.task.internal:Duration was 920.869239 seconds 2024-09-14T09:10:05.880 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-14T09:10:05.911 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-14T09:10:05.911 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-14T09:10:05.914 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-14T09:10:05.951 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-14T09:10:05.951 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi176.front.sepia.ceph.com 2024-09-14T09:10:05.951 DEBUG:teuthology.orchestra.run.smithi176:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-14T09:10:06.003 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2024-09-14T09:10:06.003 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-14T09:10:06.013 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-14T09:10:06.013 DEBUG:teuthology.orchestra.run.smithi176:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-14T09:10:06.047 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-14T09:10:06.082 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-14T09:10:06.082 DEBUG:teuthology.orchestra.run.smithi176:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-14T09:10:06.119 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-14T09:10:06.197 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-14T09:10:06.207 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-14T09:10:06.207 DEBUG:teuthology.orchestra.run.smithi176:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-14T09:10:06.218 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-14T09:10:06.230 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-14T09:10:06.247 DEBUG:teuthology.orchestra.run.smithi176:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-14T09:10:06.262 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-14T09:10:06.269 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern = core 2024-09-14T09:10:06.284 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2024-09-14T09:10:06.313 DEBUG:teuthology.orchestra.run.smithi176:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-14T09:10:06.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T09:10:06.333 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-14T09:10:06.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T09:10:06.359 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-14T09:10:06.370 INFO:teuthology.task.internal:Transferring archived files... 2024-09-14T09:10:06.371 DEBUG:teuthology.misc:Transferring archived files from smithi176:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900516/remote/smithi176 2024-09-14T09:10:06.371 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-14T09:10:06.401 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900516/remote/smithi183 2024-09-14T09:10:06.402 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-14T09:10:06.423 INFO:teuthology.task.internal:Removing archive directory... 2024-09-14T09:10:06.424 DEBUG:teuthology.orchestra.run.smithi176:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-14T09:10:06.439 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-14T09:10:06.467 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-14T09:10:06.478 INFO:teuthology.task.internal:Not uploading archives. 2024-09-14T09:10:06.478 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-14T09:10:06.506 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-14T09:10:06.506 DEBUG:teuthology.orchestra.run.smithi176:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-14T09:10:06.508 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-14T09:10:06.512 INFO:teuthology.orchestra.run.smithi176.stdout: 658633 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 14 09:10 /home/ubuntu/cephtest 2024-09-14T09:10:06.516 INFO:teuthology.orchestra.run.smithi183.stdout: 658465 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 14 09:10 /home/ubuntu/cephtest 2024-09-14T09:10:06.517 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-14T09:10:06.539 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-14T09:10:06.608 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} duration: 920.8692388534546 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-14T09:10:06.608 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-14T09:10:06.683 INFO:teuthology.run:pass